https://www.cam.ac.uk/research/news/claims-ai-can-boost-workplace-diversity-are-spurious-and-dangerous-researchers-argue Home * Study at Cambridge * About the University * Research at Cambridge * For staff * For Cambridge students * For alumni * For our researchers * Business and enterprise * Colleges and Departments * Email and phone search * Give to Cambridge * Libraries * Museums and collections Search [Search ] Search Search [ ] Search * Study at Cambridge * Undergraduate + Courses + Applying + Events and open days + Fees and finance * Postgraduate + Postgraduate courses + How to apply + Postgraduate events + Fees and funding * International students * Continuing education * Executive and professional education * Courses in education * About the University * How the University and Colleges work * Term dates * History * Map * Visiting the University * Annual reports * Equality and diversity * News * A global university * Events * Public engagement * Jobs * Give to Cambridge * Research at Cambridge * For staff * For Cambridge students * For alumni * For our researchers * Business and enterprise * Colleges and Departments * Email and phone search * Give to Cambridge * Libraries * Museums and collections * Home * Research * News * Claims AI can boost workplace diversity are 'spurious and dangerous', researchers argue Research * Research home * News * Our people * Spotlights * About research + About research overview + Animal research o Animal research overview o Overseeing animal research # Overseeing animal research overview # The Animal Welfare and Ethical Review Body o Animal welfare and ethics o Report on the allegations and matters raised in the BUAV report o What types of animal do we use? # What types of animal do we use? overview # Chickens # Guinea pigs # Hamsters # Mice # Naked mole-rats # Non-human primates (marmosets and rhesus macaques) # Other birds # Pigs # Rats # Sheep # Xenopus # Zebrafish o Non-technical summaries o Animal Welfare Policy o Alternatives to animal use o FAQs o Further information + Funding Agency Committee Members + Research integrity + Horizons magazine + Strategic Initiatives & Networks + Nobel Prize + Interdisciplinary Research Centres + Open access + Energy sector partnerships + Podcasts o Podcasts overview o S2 ep1: What is the future? o S2 ep2: What did the future look like in the past? o S2 ep 3: What is the future of wellbeing? o S2 ep4: What would a more just future look like? o S2 ep 5: What is the future of artificial intelligence? o S2 ep 6: The future of reproduction * Business and enterprise * Research impact * Animal research Claims AI can boost workplace diversity are 'spurious and dangerous', researchers argue Co-author Dr Eleanor Drage testing the 'personality machine' built by Cambridge undergraduates. Research highlights growing market in AI-powered recruitment tools that claim to bypass human bias to remove discrimination from hiring. While companies may not be acting in bad faith, there is little accountability for how these products are built or tested Eleanor Drage Recent years have seen the emergence of AI tools marketed as an answer to lack of diversity in the workforce, from use of chatbots and CV scrapers to line up prospective candidates, through to analysis software for video interviews. Those behind the technology claim it cancels out human biases against gender and ethnicity during recruitment, instead using algorithms that read vocabulary, speech patterns and even facial micro-expressions to assess huge pools of job applicants for the right personality type and 'culture fit'. However, in a new report published in Philosophy and Technology, researchers from Cambridge's Centre for Gender Studies argue these claims make some uses of AI in hiring little better than an 'automated pseudoscience' reminiscent of physiognomy or phrenology: the discredited beliefs that personality can be deduced from facial features or skull shape. They say it is a dangerous example of 'technosolutionism': turning to technology to provide quick fixes for deep-rooted discrimination issues that require investment and changes to company culture. In fact, the researchers have worked with a team of Cambridge computer science undergraduates to debunk these new hiring techniques by building an AI tool modelled on the technology, available online. The 'Personality Machine' demonstrates how arbitrary changes in facial expression, clothing, lighting and background can give radically different personality readings - and so could make the difference between rejection and progression for a generation of job seekers vying for graduate positions. The Cambridge team say that use of AI to narrow candidate pools may ultimately increase uniformity rather than diversity in the workforce, as the technology is calibrated to search for the employer's fantasy 'ideal candidate'. This could see those with the right training and background "win over the algorithms" by replicating behaviours the AI is programmed to identify, and taking those attitudes into the workplace, say the researchers. Additionally, as algorithms are honed using past data, they argue that candidates considered the best fit are likely to end up those that most closely resembling the current workforce. "We are concerned that some vendors are wrapping 'snake oil' products in a shiny package and selling them to unsuspecting customers," said co-author Dr Eleanor Drage. "By claiming that racism, sexism and other forms of discrimination can be stripped away from the hiring process using artificial intelligence, these companies reduce race and gender down to insignificant data points, rather than systems of power that shape how we move through the world." The researchers point out that these AI recruitment tools are often proprietary - or 'black box' - so how they work is a mystery. "While companies may not be acting in bad faith, there is little accountability for how these products are built or tested," said Drage. "As such, this technology, and the way it is marketed, could end up as dangerous sources of misinformation about how recruitment can be 'de-biased' and made fairer." Despite some pushback - the EU's proposed AI Act classifies AI-powered hiring software as 'high risk', for example - researchers say that tools made by companies such as Retorio and myInterview are deployed with little regulation, and point to surveys suggesting use of AI in hiring is snowballing. A 2020 study of 500 organisations across various industries in five countries found 24% of businesses have implemented AI for recruitment purposes and 56% of hiring managers planned to adopt it in the next year. Another poll of 334 leaders in human resources, conducted in April 2020, as the pandemic took hold, found that 86% of organisations were incorporating new virtual technology into hiring practices. "This trend was in already in place as the pandemic began, and the accelerated shift to online working caused by COVID-19 is likely to see greater deployment of AI tools by HR departments in future," said co-author Dr Kerry Mackereth, who presents the Good Robot podcast with Drage, in which the duo explore the ethics of technology. COVID-19 is not the only factor, according to HR operatives the researchers have interviewed. "Volume recruitment is increasingly untenable for human resources teams that are desperate for software to cut costs as well as numbers of applicants needing personal attention," said Mackereth. Drage and Mackereth say many companies now use AI to analyse videos of candidates, interpreting personality by assessing regions of a face - similar to lie-detection AI - and scoring for the 'big five' personality tropes: extroversion, agreeableness, openness, conscientiousness, and neuroticism. The undergraduates behind the 'Personality Machine', which uses a similar technique to expose its flaws, say that while their tool may not help users beat the algorithm, it will give job seekers a flavour of the kinds of AI scrutiny they might be under - perhaps even without their knowledge. "All too often, the hiring process is oblique and confusing," said Euan Ong, one of the student developers. "We want to give people a visceral demonstration of the sorts of judgements that are now being made about them automatically". "These tools are trained to predict personality based on common patterns in images of people they've previously seen, and often end up finding spurious correlations between personality and apparently unrelated properties of the image, like brightness. We made a toy version of the sorts of models we believe are used in practice, in order to experiment with it ourselves," Ong said. --------------------------------------------------------------------- Creative Commons License The text in this work is licensed under a Creative Commons Attribution 4.0 International License. Images, including our videos, are Copyright (c)University of Cambridge and licensors/contributors as identified. All rights reserved. We make our image and video content available in a number of ways - as here, on our main website under its Terms and conditions, and on a range of channels including social media that permit your use and sharing of our content under their respective Terms. --------------------------------------------------------------------- Read this next People charging their electric cars at charging station Machine learning algorithm predicts how to get the most out of electric vehicle batteries Example image of the 3D printer nozzle used by the machine learning algorithm to detect and correct errors in real time. Algorithm learns to correct 3D printing errors for different parts, materials and systems [artificial-intelligence-digital-concept-credit-o] Cambridge research centre puts people at the heart of AI [gettyimages-1305428682_web] AI needs to serve people, science, and society --------------------------------------------------------------------- Media enquiries Fred Lewsey External Affairs and Communications team Published 10 Oct 2022 Image Co-author Dr Eleanor Drage testing the 'personality machine' built by Cambridge undergraduates. Credit: Eleanor Drage Share Search research Keyword search [ ] Go Sign up to receive our weekly research email Our selection of the week's biggest Cambridge research news and features sent directly to your inbox. Enter your email address, confirm you're happy to receive our emails and then select 'Subscribe'. Email [ ] [ ]I wish to receive a weekly Cambridge research news summary by email. [Subscribe] The University of Cambridge will use your email address to send you our weekly research news email. We are committed to protecting your personal information and being transparent about what information we hold. Please read our email privacy notice for details. Subjects * employment * Artificial intelligence People * Eleanor Drage * Kerry Mackareth Places * University of Cambridge Centre for Gender Studies (UCCGS) * Leverhulme Centre for the Future of Intelligence Horizons magazine Cover of Horizons edition 40 Download issue 40 (PDF) Connect with us * Facebook * Instagram * LinkedIn * Twitter * YouTube * Sina Weibo Cambridge University (c) 2022 University of Cambridge * Contact the University * Accessibility * Freedom of information * Privacy policy and cookies * Statement on Modern Slavery * Terms and conditions * University A-Z Study at Cambridge * Undergraduate * Postgraduate * Continuing education * Executive and professional education * Courses in education About the University * How the University and Colleges work * Give to Cambridge * Jobs * Maps * Visiting the University Research at Cambridge * Research news * About research at Cambridge * Public engagement * Spotlight on... *