https://www.nature.com/articles/d41586-021-00868-5 Skip to main content Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript. Advertisement Advertisement Nature * View all journals * Search * My Account Login * Explore content * Journal information * Publish with us Subscribe * Sign up for alerts * RSS feed 1. nature 2. world view 3. article WORLD VIEW 06 April 2021 Time to regulate AI that interprets human emotions [d41586-021] The pandemic is being used as a pretext to push unproven artificial-intelligence tools into workplaces and schools. * Kate Crawford ^0 1. Kate Crawford 1. Kate Crawford is a research professor at University of Southern California Annenberg in Los Angeles and a senior principal researcher at Microsoft Research in New York. Her latest book is Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence. Twitter: @katecrawford View author publications You can also search for this author in PubMed Google Scholar * Share on Twitter * Share on Facebook * Share via E-Mail PDF version During the pandemic, technology companies have been pitching their emotion-recognition software for monitoring workers and even children remotely. Take, for example, a system named 4 Little Trees. Developed in Hong Kong, the program claims to assess children's emotions while they do classwork. It maps facial features to assign each pupil's emotional state into a category such as happiness, sadness, anger, disgust, surprise and fear. It also gauges 'motivation' and forecasts grades. Similar tools have been marketed to provide surveillance for remote workers. By one estimate, the emotion-recognition industry will grow to US$37 billion by 2026. There is deep scientific disagreement about whether AI can detect emotions. A 2019 review found no reliable evidence for it. "Tech companies may well be asking a question that is fundamentally wrong," the study concluded (L. F. Barrett et al. Psychol. Sci. Public Interest 20, 1-68; 2019). [d41586-021] Am I arguing with a machine? AI debaters highlight need for transparency And there is growing scientific concern about the use and misuse of these technologies. Last year, Rosalind Picard, who co-founded an artificial intelligence (AI) start-up called Affectiva in Boston and heads the Affective Computing Research Group at the Massachusetts Institute of Technology in Cambridge, said she supports regulation. Scholars have called for mandatory, rigorous auditing of all AI technologies used in hiring, along with public disclosure of the findings. In March, a citizen's panel convened by the Ada Lovelace Institute in London said that an independent, legal body should oversee development and implementation of biometric technologies (see go.nature.com/3cejmtk). Such oversight is essential to defend against systems driven by what I call the phrenological impulse: drawing faulty assumptions about internal states and capabilities from external appearances, with the aim of extracting more about a person than they choose to reveal. Countries around the world have regulations to enforce scientific rigour in developing medicines that treat the body. Tools that make claims about our minds should be afforded at least the same protection. For years, scholars have called for federal entities to regulate robotics and facial recognition; that should extend to emotion recognition, too. It is time for national regulatory agencies to guard against unproven applications, especially those targeting children and other vulnerable populations. [d41586-021] Why faces don't always tell the truth about feelings Lessons from clinical trials show why regulation is important. Federal requirements and subsequent advocacy have made many more clinical-trial data available to the public and subject to rigorous verification. This becomes the bedrock for better policymaking and public trust. Regulatory oversight of affective technologies would bring similar benefits and accountability. It could also help in establishing norms to counter over-reach by corporations and governments. The polygraph is a useful parallel. This 'lie detector' test was invented in the 1920s and used by the FBI and US military for decades, with inconsistent results that harmed thousands of people until its use was largely prohibited by federal law. It wasn't until 1998 that the US Supreme Court concluded that "there was simply no consensus that polygraph evidence is reliable". A formative figure behind the claim that there are universal facial expressions of emotion is the psychologist Paul Ekman. In the 1960s, he travelled the highlands of Papua New Guinea to test his controversial hypothesis that all humans exhibit a small number of 'universal' emotions that are innate, cross-cultural and consistent. Early on, anthropologist Margaret Mead disputed this idea, saying that it discounted context, culture and social factors. [d41586-021] Resisting the rise of facial recognition But the six emotions Ekman described fit perfectly into the model of the emerging field of computer vision. As I write in my 2021 book Atlas of AI, his theory was adopted because it fit what the tools could do. Six consistent emotions could be standardized and automated at scale -- as long as the more complex issues were ignored. Ekman sold his system to the US Transportation Security Administration after the 11 September 2001 terrorist attacks, to assess which airline passengers were showing fear or stress, and so might be terrorists. It was strongly criticized for lacking credibility and for being racially biased. However, many of today's tools, such as 4 Little Trees, are based on Ekman's six-emotion categorization. (Ekman maintains that faces do convey universal emotions, but says he's seen no evidence that automated technologies work.) Yet companies continue to sell software that will affect people's opportunities without clearly documented, independently audited evidence of effectiveness. Job applicants are being judged unfairly because their facial expressions or vocal tones don't match those of employees; students are being flagged at school because their faces seem angry. Researchers have also shown that facial-recognition software interprets Black faces as having more negative emotions than white faces do. We can no longer allow emotion-recognition technologies to go unregulated. It is time for legislative protection from unproven uses of these tools in all domains -- education, health care, employment and criminal justice. These safeguards will recentre rigorous science and reject the mythology that internal states are just another data set that can be scraped from our faces. Nature 592, 167 (2021) doi: https://doi.org/10.1038/d41586-021-00868-5 Latest on: Society Am I arguing with a machine? AI debaters highlight need for transparency Am I arguing with a machine? AI debaters highlight need for transparency Editorial 07 APR 21 India: draft science policy calls for public engagement India: draft science policy calls for public engagement Correspondence 30 MAR 21 People of faith are allies to stall climate change People of faith are allies to stall climate change World View 30 MAR 21 Policy National COVID debts: climate change imperils countries' ability to repay National COVID debts: climate change imperils countries' ability to repay Comment 06 APR 21 The world must learn from COVID before diving into a pandemic treaty The world must learn from COVID before diving into a pandemic treaty Editorial 06 APR 21 US universities call for clearer rules on science espionage amid China crackdown US universities call for clearer rules on science espionage amid China crackdown News 06 APR 21 Computer science Am I arguing with a machine? AI debaters highlight need for transparency Am I arguing with a machine? AI debaters highlight need for transparency Editorial 07 APR 21 Quantum computer based on shuttling trapped ions Quantum computer based on shuttling trapped ions News & Views 07 APR 21 How athletes hit a fastball How athletes hit a fastball Outlook 31 MAR 21 Jobs from Nature Careers * + o All jobs + o Business Development Manager, Strategic Partnerships Springer Nature New York, NY, United States JOB POST o Chief Technician Columbia University Medical Center (CUMC), CU New York, NY, United States JOB POST o Postdoctoral Research Scientist Columbia University Medical Center (CUMC), CU New York, NY, United States JOB POST o Postdoctoral Fellow in Dr. Logothetis and Dr. Panaretakis Lab, GU Medical Oncology The University of Texas MD Anderson Cancer Center Houston, TX, United States JOB POST Nature Briefing An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday. Email address [ ] [ ] Yes! Sign me up to receive the daily Nature Briefing email. I agree my information will be processed in accordance with the Nature and Springer Nature Limited Privacy Policy. Sign up Related Articles * [d41586-021] Am I arguing with a machine? AI debaters highlight need for transparency * [d41586-021] Why faces don't always tell the truth about feelings * [d41586-021] Resisting the rise of facial recognition * Halt the use of facial-recognition technology until it is regulated Subjects * Society * Policy * Computer science Advertisement Sign up to Nature Briefing An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday. Email address [ ] [ ] Yes! Sign me up to receive the daily Nature Briefing email. I agree my information will be processed in accordance with the Nature and Springer Nature Limited Privacy Policy. Sign up * Close Nature Briefing Sign up for the Nature Briefing newsletter -- what matters in science, free to your inbox daily. Email address [ ] Sign up [ ] I agree my information will be processed in accordance with the Nature and Springer Nature Limited Privacy Policy. Close Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing * Explore content * Research articles * News * Opinion * Research Analysis * Careers * Books & Culture * Podcasts * Videos * Current issue * Browse issues * Collections * Subjects * Follow us on Facebook * Follow us on Twitter * Subscribe * Sign up for alerts * RSS feed Journal information * About the Journal * Awards Publish with us * For Authors * For Referees * Submit manuscript Search Search articles by subject, keyword or author [ ] Show results from [All journals] Search Advanced search Quick links * Explore articles by subject * Find a job * Guide to authors * Editorial policies Nature ISSN 1476-4687 (online) nature.com sitemap Nature portfolio * About us * Press releases * Press office * Contact us * * * Discover content * Journals A-Z * Articles by subject * Nano * Protocol Exchange * Nature Index Publishing policies * Nature portfolio policies * Open access Author & Researcher services * Reprints & permissions * Research data * Language editing * Scientific editing * Nature Masterclasses * Nature Research Academies Libraries & institutions * Librarian service & tools * Librarian portal * Open research * Recommend to library Advertising & partnerships * Advertising * Partnerships & Services * Media kits * Branded content Career development * Nature Careers * Nature Conferences * Nature events Regional websites * Nature Africa * Nature China * Nature India * Nature Italy * Nature Japan * Nature Korea * Nature Middle East Legal & Privacy * Privacy Policy * Use of cookies * Manage cookies/Do not sell my data * Legal notice * Accessibility statement * Terms & Conditions * California Privacy Statement Springer Nature (c) 2021 Springer Nature Limited \