Blog 8: Harms in Machine Learning

The following case study examines different ways in which bias/harm can appear in machine learning training along with examples. It also provides methods for combatting these different problems.

Read The Case Study Here:
Understanding Potential Sources of Harm throughout the Machine Learning Life Cycle

Regarding the study

This case study gives many ways to identify bias in machine learning algorithms that are all very useful for developers to self-diagnose, but I feel it misses one of the most important methods, which is general user testing. This was actually the crux of the case study in my previous post, which had people of different backgrounds try and apply image generators to their own culture to find its shortcomings. It does talk a little bit about hiring experts to interpret data when pointing out aggregation bias, but that still strays somewhat from what the previous study did. User testing is especially crucial for systems in which the developers are not a part of the target audience, but they’re still beneficial in any case to avoid evaluation bias. It would be great then if developers could hold focus groups to adequately solve these problems not just on their own, as it would make these models far more diverse and equitable. I don’t believe all issues could be solved this way though since some statistics (like anticipating a re-arrest) will always be unpredictable to a certain degree.

Regarding the specific types of bias outlined though, I do imagine that not all of it happens inadvertently. As is mentioned in a piece I recently watched, Coded Bias, the people who own the code ultimately control whether or not bias is perpetuated, so bias could exist intentionally in certain situations. That would seem counter-intuitive to me because you’d essentially be limiting the applicability of your model and thus limiting potential profit, though it’s a very real possibility for people who want to shape the world via their own personal bias.

Now, I haven’t used machine learning on any project I’ve worked on yet, though I’ve used some of the more popular generative AI models like everyone else. If I were to have a hand in designing one of those, I would make sure to utilize focus groups and broader user-testing as a key aspect of ridding my model from bias. One big issue with user-testing is that if there is something very very wrong with the model that makes it act in a hugely discriminatory or offensive way, that it could be met with immense backlash. I think a lot about Microsoft’s Tay AI which spiraled downward real quick (you can learn more about that here. The user base would need to be specifically selected and coached beforehand that these types of things could happen to avoid any PR nightmares. I would of course consider all of the other forms of bias outlined in the study, though I feel much of it would be done for you when straight up consulting your user base.

Consider the following

What are some other ways in which developers could analyze bias in their data/training without checking it themselves?

This question is basically written for my blog post to be an answer to it. Technically the case study asks for additional ways to detect/mitigate harm, but I wanted to specifically frame it in a way that has readers considering options that don’t directly involve developers, since they’re not always the ones most-equipt for analyzing bias.

Final Reflection

This case study was pretty informative, though a little bit confusing at times. I’m not sure if I was just unfocused when I read it, but I had to go over parts multiple times because the wording was sometimes strange. The examples provided made everything make much more sense though, and it was cool seeing a reference to Buolamwini’s work (especially since it allowed me to reference it in my post as well). The discussion questions left a lot to be desired though, but that could be because the audience for this case study was more narrow than previous ones meaning I couldn’t answer as thoughtfully to everything as I wanted to. It was neat learning about all the different ways bias can be present in models though.