There is a common belief that AI makes unbiased decisions. In reality, the kind of narrow artificial intelligence that exists today is far from unbiased, see for instance Richards Fall’s articles https://labs.sogeti.com/when-ai-goes-bad/ and https://labs.sogeti.com/algorithms-and-bias-in-the-criminal-justice-system/ which talks about this very issue. You also have a great article by Rahul Bhargava https://medium.com/mit-media-lab/the-algorithms-arent-biased-we-are-a691f5f6f6f2 that talks about the need to shift focus from the learning to the teaching aspect of machine learning. Because that’s the thing, no machine is learning in a vacuum. They are learning in our world, that’s full of bias. It’s also humans that choose which training data to use and what criteria to use for decisions. There is a lot of examples of bias in systems with learning capabilities that are evident, for example, racist twitter bots, recruitment systems only choosing male applicants and systems that predict black defendants will have higher risks of recidivism than they actually do. I wonder how many biased systems there are that we haven’t discovered…
This means that when we test systems with learning capabilities, it’s important that we test for these aspects as well. Is the system behaving with unaccepting prejudice? And because it continues to learn, we also need to monitor to catch biased behavior. How do you suggest we do this?
Eva Holmquist is a senior test specialist at Sogeti. She has worked with activities from test planning to execution of tests. She has also worked with Test Process Improvement and Test Education. She is the author of the book "Praktisk mjukvarutestning" which is a book on Software Testing in Practice.