Data Quality, Data Centric AI

Feb 13, 2023

Data scientists receive little training in how to collect data, leaving them unprepared when they need to quickly gather annotations.

My background in survey data collection has taught me how to collect high quality data; my expertise can save data scientists money and time in model development.

Stephanie Eckman
Stephanie Eckman
Researcher & Data Scientist

I collect high quality data for social science and model training

Publications

Position: Insights from Survey Methodology can Improve Training Data
Whether future AI models are fair, trustworthy, and aligned with the public’s interests rests in part on our ability to collect …
Stephanie Eckman, Barbara Plank, Frauke Kreuter
PDF Project DOI
Position: Insights from Survey Methodology can Improve Training Data
Annotation Sensitivity: Training Data Collection Methods Affect Model Performance
When training data are collected from human annotators, the design of the annotation instrument, the instructions given to annotators, …
Christoph Kern, Stephanie Eckman, Jacob Beck, Rob Chew, Bolei Ma, Frauke Kreuter
PDF Project DOI
Annotation Sensitivity: Training Data Collection Methods Affect Model Performance
Improving Labeling Through Social Science Insights: Preliminary Results and Research Agenda
Although often seen as a gold-standard, human labeled training data is not error free. Decisions in the design of labeling tasks can …
Jacob Beck, Stephanie Eckman, Rob Chew, Frauke Kreuter
PDF Project DOI
Improving Labeling Through Social Science Insights: Preliminary Results and Research Agenda

Events

The Science of Data Collection: Insights from Surveys can Improve Machine Learning Models
Whether future AI models make the world safer or less safe for humans rests in part on our ability to efficiently collect accurate data …
May 30, 2024 12:00 PM
Stephanie Eckman
Project Slides
The Science of Data Collection: Insights from Surveys can Improve Machine Learning Models
Annotation Sensitivity: Training Data Collection Methods Affect Model Performance
5 minute lightening talk on methods to collect annotations for NLP models
Aug 22, 2023 6:30 PM Arlington, VA
Stephanie Eckman
Project Slides Video
Annotation Sensitivity: Training Data Collection Methods Affect Model Performance
Improving Label Collection Through Social Science Insights
As many in the AI field transition from a model-centric to a data-centric approach, the quality of training data is receiving increased …
May 11, 2022 8:30 AM
Stephanie Eckman
Project Slides Video stephnie
Improving Label Collection Through Social Science Insights
Data Quality in Data Science
Data scientists are increasingly turning their attention to the collection of high quality training data. Those of us with expertise in data collection can apply lessons in web survey design to them.
Aug 5, 2021 10:00 AM
Stephanie Eckman
Project Slides Video stephnie
Data Quality in Data Science