Not Unbiased: The Implications of Human-Algorithm Interaction on Training Data and Algorithm Performance

This project is supported by National Science Foundation grant NSF IIS-1549981.


This INSPIRE award is partially funded by the Information Integration and Informatics program in the Division of Information and Intelligent Systems in the Directorate for Computer & Information Science & Engineering, the Perception, Action & Cognition program in the Division of Behavioral and Cognitive Sciences in the Directorate for Social, Behavioral & Economic Sciences, and the Office of Integrative Activities in the Office of the Director.

One of the most common uses of machine learning is to learn to replicate human decisions, a common example is recommender systems. In these systems, computers are trained to replicate the recommendation a collaboration of hundreds or thousands of humans would give, if that were possible. Most of the data used to train these systems are not from a controlled random sample, but are obtained from users based on outputs of algorithms (e.g., which search engine results do users click on?), which introduces bias into the process and ultimately impacts the quality of the results. This project addresses this problem by examining how the human decision process that creates these data in the first place is affected by the data coming from machine algorithms, how this in turn impacts the algorithms themselves, and how to ultimately adjust for human bias in the machine learning process. Specific areas tackled are filtering (e.g., web search) and recommender systems. The deep research into how the human decision process affects machine learning, and how machine learning impacts the human decision process, can provide significant advances in the accuracy and utility of systems using machine learning.

The project builds on analysis of machine learning algorithms based on Hidden Markov Models (HMMs). The formal analysis initially looks at "blind spots" - the impact of bias from users not getting complete (or a random sample) of data. Further analysis will be based on the outcome of two human experiments: Two category recommendation (labeling items, with items to be labelled chosen by random, active learning, and filter-based algorithms), and movie recommendation. The results will be used to develop improved machine learning approaches based on antidotes (altering learned models to reduce bias) and reactive learning (active learning that takes into account the human and machine biases). The PIs also have plans to capitalize on the lessons learned by providing examples of the use of cognitive science in a Web Mining course, and of the impact of machine learning in Data Science for Psychologists courses


Faculty Investigators

Olfa Nasraoui - Principal Investigator

Patrick Shafto - Co-Principal Investigator

Graduate Students

Mahsa Badami

Wenlong Sun

Undergraduate Students

Jared Jones

Kiana Cramer

Sam Naser


IMDb POLAR Dataset

This dataset is released for research that involves polarized items, substantially better data than previous benchmark datasets. The scope of research is typically where ratings are used for data analysis or to make predictions, such as in recommendation systems, in particular when studying the effect of algorithmic bias and iterated bias in recommender systems. This research is supported by the National Science Foundation INSPIRE grant IIS-1549981

When using this dataset, please cite the following paper :

Badami, M. and Nasraoui, O. and Sun, W. and Shafto, P., “Detecting Polarization in Ratings: An Automated Pipeline and a Preliminary Quantification on Several Benchmark Data Sets”, IEEE International Conference on Big Data, 2017 .

Download IMDb Dataset (README)