Understanding the Effects of Questionnaire Changes when Experiments are Not Possible: Application to the U.S. Consumer Expenditure Survey

Abstract

Many large-scale government surveys do not have the budget or time to conduct experiments to assess the effects of questionnaire changes. Rather than conducting an experiment, this paper proposes the use of a parallel web survey and multiple imputation to estimate the impact of changes to a questionnaire. The approach is applied to the U. S. Consumer Expenditure Interview Survey, which informs the calculation of the official measure of inflation, among other uses. Results suggest that revising the questionnaire could increase the report of household purchases by approximately five percentage points in the sections tested. The approach used here, involving a web survey built to mimic the expenditure survey, could be applied in other large surveys where budget or logistical constraints prevent experimentation.