Shusaku Sasaki, a Specially Appointed Professor (Full time) at Center for Infectious Disease Education and Research (CiDER) and the EIPM Center at the University of Osaka, conducted joint research with Takeru Kawanishi and Aiko Horiuchi of INTAGE Inc., a specialized company in market/marketing research. The research revealed that messages (nudges) based on the findings of behavioral economics are effective in reducing IERs in online surveys.

In this study, more than 10,000 participants were asked to write their opinions after watching videos. They were presented with different messages, such as (1) commitment, (2) a loss-framed message, and (3) a gain-framed message. Each message significantly reduced IERs.

Furthermore, it was clarified that a loss-framed nudge was more effective than devising the questions (such as reducing the number of open-ended questions).

Not only will this contribute to improving the efficiency of social surveys and marketing research, but it will also be a new insight that demonstrates the usability of nudges that influence human behavior.

Figure: Influences on the incidence rate of IERs (Error bars represent one standard error)

Research Background

Online surveys are efficient in terms of time and cost, and are now used in a variety of fields, including corporate marketing and awareness surveys for government policymaking. As a contactless research method, its use has increased worldwide since the COVID-19 pandemic.

However, there was an issue that it easily gets a lot of IERs such as sentences unrelated to the question or meaningless character strings. There are concerns that such answers will be factors to reduce the efficiency and quality of the survey and undermine the reliability of the results. For the surveys which participants were asked to write freely their impressions of videos, this tendency was particularly strong.

Research Contents

In this study, the researchers examined whether it would be possible to reduce IERs using nudges from behavioral economics. They conducted a survey targeting over 10,000 people nationwide, asking them to watch videos and share their thoughts, and three behavioral economics interventions (nudges), (1) commitment, (2) a loss-framed message, and (3) a gain-framed message, were presented randomly. The results showed that those three nudges significantly reduced IERs.

Furthermore, a loss-framed message was shown to be particularly effective when compared to methods such as replacing some of the questions with multiple-choice questions or informing participants of the number of videos to be shown in advance.

Social impact of the research

This study offers a new way for improving the reliability of online surveys. By displaying messages without cost, it can reduce IERs, so it is expected to be applicable to a wide range of situations, including not only corporate marketing surveys but also resident awareness surveys conducted by local governments. It has potential to be used as a simple and effective measure in such situations where it is needed to gather as many accurate opinions as possible. This is a highly practical result for frontlines where surveys are conducted, as it can be implemented without compromising the quality of the survey design.

Note

The article, “Behavioral Economics Interventions to Reduce Insufficient Effort Responses in Open-Ended Questions of Online Surveys: A Randomized Controlled Trial,” was published in Journal of Behavioral Economics and Finance at DOI: https://doi.org/10.11167/jbef.18.1.

Links