data collection

People consistently overreport how much news they consume. We compared survey responses to passively collected behavioral data and found large gaps, highlighting the need to combine data sources for accurate measurement.

Data scientists are increasingly turning their attention to the collection of high quality training data. Those of us with expertise in data collection can apply lessons in web survey design to them.

Small changes in how you ask annotators to label data can dramatically change your model’s behavior. We tested 5 versions of a hate speech labeling task and found significant differences in model performance.

Motivated misreporting occurs when respondents give incorrect responses to survey questions to shorten the interview; studies have …

Do survey respondents learn to game the questionnaire over time? We tested this in the U.S. Consumer Expenditure Survey and found the opposite. Responses actually improve in later waves.

Many large-scale government surveys do not have the budget or time to conduct experiments to assess the effects of questionnaire …

Motivated misreporting occurs when respondents give incorrect responses to questions to shorten a survey interview. Previous research …

Although counts of the novel Coronavirus (SARS-CoV-2) infections and deaths are reported by several sources online, precise estimation …

As survey costs increase and response rates decrease, researchers are looking to alternative methods to collect data from study …

How you design a labeling interface affects the labels you get. We show that task structure, ordering, and annotator backgrounds all shape training data quality.