Machine Learning Models Can Fail When They Try to Make Predictions for Individuals Who Were Underrepresented in the Datasets They Were Trained On
Removing Bad Examples
Machine learning models can fail when they try to make predictions for individuals who were underrepresented in the datasets they were trained on. For instance, a model that predicts the best treatment option for someone with a chronic disease may be trained using a dataset that contains mostly male patients. That model might make incorrect predictions for female patients when deployed in a hospital.
A New Technique
To improve outcomes, engineers can try balancing the training dataset by removing data points until all subgroups are represented equally. While dataset balancing is promising, it often requires removing a large amount of data, hurting the model’s overall performance.
MIT researchers have developed a new technique that identifies and removes specific points in a training dataset that contribute most to a model’s failures on minority subgroups. By removing far fewer data points than other approaches, this technique maintains the overall accuracy of the model while improving its performance regarding underrepresented groups.
How it Works
The researchers combined two ideas: the idea that machine learning models are trained using huge datasets gathered from many sources across the internet, and the idea that some data points impact a model’s performance on certain downstream tasks more than others. They developed an approach that identifies and removes these problematic datapoints.
The researchers take incorrect predictions the model made about minority subgroups and use a method called TRAK to identify which training examples contributed the most to that incorrect prediction. By aggregating this information across bad test predictions in the right way, they are able to find the specific parts of the training that are driving worst-group accuracy down overall.
Results
The researchers tested their technique on three machine-learning datasets and found that it outperformed multiple techniques. In one instance, it boosted worst-group accuracy while removing about 20,000 fewer training samples than a conventional data balancing method. Their technique also achieved higher accuracy than methods that require making changes to the inner workings of a model.
A More Accessible Approach
Because the MIT method involves changing a dataset instead of a model, it would be easier for a practitioner to use and can be applied to many types of models. It can also be used when bias is unknown because subgroups in a training dataset are not labeled. By identifying datapoints that contribute most to a feature the model is learning, they can understand the variables it is using to make a prediction.
Conclusion
The researchers’ new technique is a tool that can be used to detect and remove sources of bias in machine learning models. By identifying and removing specific points in a training dataset that contribute to a model’s failures on minority subgroups, this technique can improve the performance of machine learning models on underrepresented groups.
Frequently Asked Questions
Q: How does the new technique work?
A: The technique identifies and removes specific points in a training dataset that contribute most to a model’s failures on minority subgroups.
Q: What are the benefits of this technique?
A: The technique maintains the overall accuracy of the model while improving its performance on minority subgroups, and can be used when bias is unknown because subgroups in a training dataset are not labeled.
Q: How does it compare to other techniques?
A: The technique outperforms multiple techniques, including dataset balancing and making changes to the inner workings of a model.
