With several colleagues, I have done a lot of research in the last few years on what we call Motivated Misreporting – where respondents give false answers in surveys to try to make the surveys shorter. We have found evidence of this behavior in several countries (US, Germany, the Netherlands) and several modes (web, phone, in-person).
We’ve looked at several different types of questions where motivated misreporting occurs.
Filter Questions: when answering Yes to 1 question triggers one or more follow ups, we call that first question a filter question. Asking such questions in a way that makes the structure obvious leads to underreporting. However, the effect does not always occur.
Looping Questions: similar to filter questions, but ask about all related events. Again, making the follow up questions obvious leads to underreporting by respondents.
Screener Questions: respondents tend to make themselves ineligible for surveys, if given the chance. We recommend asking indirectly about eligibility to avoid the problem.
Summary of my research on Motivated Misreporting:
Eckman, S, Kreuter, F, Kirchner, A, Jäckle, A, Tourangeau, R and Presser, S. (2014) “Assessing the Mechanisms of Misreporting to Filter Questions in Surveys” Public Opinion Quarterly.
Filter questions are prone to motivated misreporting, but there is also a topic or exposure effect. We use administrative data to show that motivated misreporting is driven by underreporting in some formats.
Tourangeau, R, Kreuter, F and Eckman, S. (2012) “Motivated Underreporting in Screening Interviews" Public Opinion Quarterly.
Given the opportunity, respondents hide their eligibility in screener questions, to avoid having to do the full interview.
Tourangeau, R, Kreuter, F and Eckman, S. (2015) “Motivated Misreporting: Shaping Answers to Reduce Survey Burden" in Engel, U. Survey Measurement: Techniques and Findings from Recent Research. Campus: 24-41.
Summary of research on filter and screener questions.
Kreuter, F, Eckman, S and Tourangeau, R. “Salience of Survey Burden and Its Effects on Response Behavior to Skip Questions. Experimental Results from Telephone and Web-Surveys" forthcoming in Beatty, P, Collins, D, Kaye, L, Padilla, J, Willis, G and Wilmot, A. Advances in Questionnaire Design, Development, Evaluation and Testing. Wiley.
Discusses ways to make the motivated misreporting phenomenon weaker. Intended to offer advice to survey designers who have good reasons to use questions that are vulnerable to motivated misreporting.
Bach, R and Eckman, S. (2017) “Motivated Misreporting in Web Panels" Journal of Survey Statistics and Methodology.
A web panel survey in the Netherlands shows strong motivated misreporting in 2 consecutive waves. However, there is no evidence that respondents remember from one wave to the other. That is, motivated misreporting does not get worse over waves.
Eckman, S and Kreuter, F. “Misreporting to Looping Questions in Surveys: Recall, Motivation and Burden" forthcoming in Survey Research Methods.
Shows motivated misreporting also occurs in looping questions and offers practical advice for researchers designing questionnaires with loops.