New York City businesses that use artificial intelligence to help find hires now have to show the process was free from sexism and racism.
One easy way to prove that the AI is racist is whether race is one of its parameters. However, it is possible for an AI to be racist even without an explicit parameter.
Not necessarily, something that overt would be obvious and avoided. I’m pretty sure that what they’re looking for is more for subtle biases caused by bad datasets to train the AI.
For instance, you train the AI and tell it which candidates are good or bad. But maybe, by pure happenstance, the best candidates in your dataset are all male. If so, the AI might be accidentally trained to believe that all good candidates are male.
Or you train the AI based on data from humans who are racist. If an AI learns how to hire from a racist set of human managers, it may take those biases into account when it is choosing who to hire.
I remember hearing about a high-profile case where the AI would dock points if someone’s resume listed them as participating in women’s sports as an extracurricular, while giving extra points if it listed them as participating in men’s sports.
Also, bias doesn’t necessarily have to come from happenstance. Unfortunately, humans tend to have unconcious (or, sometimes, not-so-unconcious) biases against women and people of color. There was a study where researchers sent identical resumes to a random group of recruiters-- but half of the resumes had a male name and half had a female name.
They found that both male and female recruiters were more likely to rate the resumes with the male name higher and be more likely to recommend they be advanced to the next round of interviews. IIRC, similar studies have found similar results if you give the resumes a “Black sounding” name versus a “white sounding” name.
So if you train an AI on your own company’s hiring data-- which is likely to be tainted by the unconcious bias of your own recruiters and hiring managers-- then the AI might pick up on that and replicate it in its results.
Can’t prove the human ones were not… do we send a witch hunt for them too?
