AI could be the key to ending discrimination in hiring, but experts warn it can be just as biased as humans

  • Employers are increasingly turning to artificial intelligence-driven tools to carry out recruitment and hiring. Companies like Amazon, FedEx, Target, and Capital One have tested or used AI hiring software.
  • These tools sift through heaps of resumes quickly, evaluate candidates' answers to written questions and interactive games, and conduct video interviews with candidates.
  • Vendors tout AI hiring tools as a way to root out discrimination, but experts and activists warn that AI-driven hiring tools are just as biased as the humans who train them.
  • Visit Business Insider's homepage for more stories.

The next time you apply for a job, the fate of your application may not be in human hands - instead, an algorithm could determine whether or not you make the cut.

Employers are increasingly turning to artificial intelligence-driven tools to carry out recruitment and hiring. Target, Hilton, Pepsi, Amazon, and Ikea are among the companies who have tested or used algorithms to determine who to hire, and the list is growing across the employment spectrum from low-wage jobs to white-collar positions. Advertisement

Predictive hiring tools are mostly used to weed out candidates deemed unfit for hire, rather than affirmatively choose who gets a job. These tools sift through huge numbers of resumes in short time periods, evaluate candidates' answers to written questions and interactive games, and conduct video interviews with candidates.

Read more: How AI is changing everything

Companies who sell the AI tools say they expect that the job of human recruiters will soon be entirely replaced by robots in some sectors, and investor interest in these vendors is growing. Researchers at Cornell found that there are at least 19 companies providing AI-driven hiring tools worldwide, with funding ranging from $1 million to $93 million per vendor.

"Technology is rapidly changing how people find jobs, learn about jobs, apply to those jobs, and are evaluated for those jobs," Aaron Rieke, managing director at tech equity nonprofit Upturn, told Business Insider. "The fear is if that is not done very carefully and thoughtfully, there could be problems down the road."Advertisement

Meanwhile, AI-driven hiring tools haven't drawn much scrutiny from regulators and lawmakers. However, the House Education and Labor Committee is holding a series of hearings on the impact of AI on the workforce this fall, which could be a precursor to legislation.

rise of deepfakes 4x3

Samantha Lee/Business Insider

AI hiring tools could automatically discriminate against certain job applicants

Just like all algorithms, AI hiring tools are trained by humans, typically using data sets from the real world meant to help AI recognize the features of a "good" or "bad" candidate. Advertisement

Accordingly, tech experts and activists have warned that AI hiring tools might pick up on preexisting human biases, especially in fields like tech, where progress towards diversity has been slow.

In a report published in December, Upturn researchers highlighted how these assumptions could lead to unintentional discrimination by AI hiring tools.

"What can naturally happen is you build a model that identifies common characteristics of your current workforce, which isn't diverse," Rieke said. "Or might reflect the fact that hiring managers have traditionally given preference to male candidates over women."Advertisement

"Even if a company has good intentions, products that are trained on data collected from current employees deemed successful could learn discriminating characteristics that are due to identity traits instead of job competency," Buolamwini wrote in an email to Business Insider. "Data is not necessarily neutral."

korea resume job application fair

Lee Jae Won/Reuters

Can AI be trained to root out human bias?

Vendors of AI-driven hiring tools argue the opposite, claiming that AI is the most surefire way to ultimately eliminate human bias in the hiring process.Advertisement

Eric Sydell, the executive vice president of innovation at Modern Hire, told Business Insider he believes algorithms can be programmed with the specific goal of ruling out the unfair preferences that humans act on. Modern Hire sells a software that allows companies to put candidates through interviews and "virtual job tryouts," and is used by companies including Capital One, Allstate, SunTrust, and FedEx.

Rieke said it's too soon to tell whether AI-driven hiring tools are already working against bias, but he believes it's possible that such tools could one day prove less biased than human recruiters.Advertisement

"The old-fashioned, human way of doing this is a process that has consistently yielded staggering bias against people," Rieke said. "I think there's the possibility for hiring tools to fairly and equitably hire people … but the devil's in the details."

Exclusive FREE Slide Deck: 40 Big Tech Predictions for 2019 by Business Insider Intelligence