United StatesRetail and e-commerce2014–2017Amazon
Machine-learning résumé screening tool
Amazon dropped an experimental résumé-rating AI after it learned to penalize women
Problem
Amazon was hiring at a very fast pace and wanted to find strong candidates among large numbers of résumés with less manual screening. Engineers described the goal as a system that would take 100 résumés and return the top five to hire.
Approach
From 2014 a team of about a dozen people at Amazon's Edinburgh engineering hub built models that scored applicants from one to five stars. The models learned from patterns in résumés submitted to Amazon over a 10-year period, most of them from men. By 2015 the company found that the system downgraded résumés containing the word "women's" and graduates of two all-women's colleges.
Results
Takeaway
The model reproduced the gender imbalance in past hiring data; Amazon made it neutral to the terms but could not rule out other discriminatory patterns and disbanded the team by early 2017. Reuters' sources said recruiters consulted its rankings without relying on them alone; Amazon declined to comment on the tool.
Categories
- Size
- Large enterprise
- Industry
- Retail and e-commerce
- Business function
- Human resources
- Innovation type
- Process innovation
- AI technology
- Predictive analytics and ML
- Adoption
- Built in-house
- Evidence
- Press reporting
Sources
Source summaries
Reuters reports, citing five people familiar with the project, that an Edinburgh team built a tool from 2014 that rated applicants one to five stars using ten years of mostly male résumés. By 2015 the system was penalizing the word "women's" and graduates of two all-women's colleges; edits could not guarantee it would not find other ways to discriminate. The team was disbanded by early 2017, and Amazon declined to comment on the tool.
BBC News reports that the tool used ten years of mostly male training data to rate candidates on a one-to-five-star scale. It notes that recruiters reportedly looked at the tool's recommendations but never relied on them alone, and that Amazon had not responded to the claims. It adds that Amazon's global workforce was split 60:40 in favor of men.
Full details are in the original sources.
