https://www.reuters.com/technology/artificial-intelligence/if-your-ai-seems-smarter-its-thanks-smarter-human-trainers-2024-09-28/ Skip to main content Exclusive news, data and analytics for financial market professionals Learn more aboutRefinitiv * World Browse World + Africa + Americas + Asia Pacific + China + Europe + India + Israel and Hamas at War + Japan + Middle East + Ukraine and Russia at War + United Kingdom + United States + Reuters NEXT * US Election * Business Browse Business + Aerospace & Defense + Autos & Transportation + Davos + Energy + Environment + Finance + Healthcare & Pharmaceuticals + Media & Telecom + Retail & Consumer + Future of Health + Future of Money + Take Five + World at Work * Markets Browse Markets + Asian Markets + Carbon Markets + Commodities + Currencies + Deals + Emerging Markets + ETFs + European Markets + Funds + Global Market Data + Rates & Bonds + Stocks + U.S. Markets + Wealth + Macro Matters * Sustainability Browse Sustainability + Boards, Policy & Regulation + Climate & Energy + Land Use & Biodiversity + Society & Equity + Sustainable Finance & Reporting + The Switch + Reuters Impact * Legal Browse Legal + Government + Legal Industry + Litigation + Transactional + US Supreme Court * Breakingviews Browse Breakingviews + Breakingviews Predictions * Technology Browse Technology + Artificial Intelligence + Cybersecurity + Space + Disrupted * More InvestigationsSports + Athletics + Baseball + Basketball + Cricket + Cycling + Formula 1 + Golf + NFL + NHL + Soccer + Tennis ScienceLifestyleGraphics PicturesWider ImagePodcastsFact CheckVideoSponsored Content + Reuters Plus + Press Releases My News Register * Artificial Intelligence If your AI seems smarter , it's thanks to smarter human trainers By Supantha Mukherjee and Anna Tong September 28, 20242:08 PM UTCUpdated ago * * * * * * * * Illustration shows AI (Artificial Intelligence) letters and robot hand REUTERS/Dado Ruvic Purchase Licensing Rights, opens new tab * Summary * Companies * AI models now require trainers with advanced degrees * Invisible Tech employs 5,000 specialized trainers globally * It takes smart humans to avoid hallucinations in AI STOCKHOLM/SAN FRANCISCO, Sept 28 (Reuters) - In the early years, getting AI models like ChatGPT or its rival Cohere to spit out human-like responses required vast teams of low-cost workers helping models distinguish basic facts such as if an image was of a car or a carrot. But more sophisticated updates to AI models in the fiercely competitive arena are now demanding a rapidly expanding network of human trainers who have specialized knowledge -- from historians to scientists, some with doctorate degrees. "A year ago, we could get away with hiring undergraduates, to just generally teach AI on how to improve," said Cohere co-founder Ivan Zhang, talking about its internal human trainers. "Now we have licensed physicians teaching the models how to behave in medical environments, or financial analysts or accountants." For more training, Cohere, which was last valued at over $5 billion, works with a startup called Invisible Tech. Cohere is one of the main rivals of OpenAI and specializes in AI for businesses. The startup Invisible Tech employs thousands of trainers, working remotely, and has become one of the main partners of AI companies ranging from AI21 to Microsoft to train their AI models to reduce errors, known in the AI world as hallucinations. "We have 5,000 people in over 100 countries around the world that are PhDs, Master's degree holders and knowledge work specialists," said Invisible founder Francis Pedraza. Invisible pays as much as $40 per hour, depending on the location of the worker and the complexity of work. Some companies such as Outlier pay up to $50 per hour, while another company called Labelbox said it pays up to $200 per hour for "high expertise" subjects like quantum physics, but starts with $15 for basic topics. Invisible was founded in 2015 as a workflow automation company catering to the likes of food delivery company DoorDash to digitize their delivery menu. But things changed when a relatively unknown research firm called OpenAI contacted them in the spring of 2022, ahead of the public launch of ChatGPT. "OpenAI came to us with a problem, which is that when you were asking an early version of ChatGPT a question, it was going to hallucinate. You couldn't trust the answer," Pedraza told Reuters. "They needed an advanced AI training partner to provide reinforcement learning with human feedback." OpenAI did not respond to request for comment. Generative AI produces new content based on past data used to train it. However, sometimes it can't distinguish between true and false information and generates false outputs known as hallucinations. In one notable example, in 2023 a Google chatbot shared inaccurate information about which satellite first took pictures of a planet outside the Earth's solar system in a promotional video. AI companies are aware that hallucinations can derail GenAI's attractiveness to businesses and are trying various ways to reduce it, including using human trainers to teach the concept of fact and fiction. Since getting onboard with OpenAI, Invisible says it has become AI training partners to most of the GenAI companies, including Cohere, AI21 and Microsoft. Cohere and AI21 confirmed they are clients. Microsoft did not confirm it is a client of Invisible. "These are all companies that had training challenges, where their number one cost was compute power, and then the number two cost is quality training," Pedraza said. HOW DOES IT WORK? OpenAI, which started off the frenzy around GenAI, has a team of researchers aptly named "Human Data Team" that works with AI trainers to gather specialized data for training its models like ChatGPT. OpenAI researchers come up with various experiments like reducing hallucinations or to improve writing style and work with AI trainers from Invisible and other vendors, a source familiar with the company's processes said. At any point, dozens of experiments are being run, some with tools developed by OpenAI and others by tools of vendors, the person said. Based on what the AI companies want - from getting better at Swedish history or doing financial modeling - Invisible hires workers with relevant degrees for those projects, reducing the burden of managing hundreds of trainers by the AI companies. "OpenAI has some of the most incredible computer scientists in the world but they're not necessarily an expert in Swedish history or chemistry questions or biology questions or anything you can ask it," Pedraza said, adding that over 1,000 contract workers cater to OpenAI alone. Cohere's Zhang said he has personally used Invisible's trainers to find a way to teach its GenAI model to find relevant information from a big data set. COMPETITION Among the competitors in this space is Scale AI, a private start-up last valued at $14 billion which provides AI companies with sets of training data. It has also ventured into the area of providing AI trainers, and counts OpenAI as a customer. Scale AI did not respond to requests for an interview for this story. Invisible, which has been profitable since 2021, has raised only $8 million of primary capital, "We are 70% owned by the team, and only 30% owned by investors," Pedraza said. "We do facilitate secondary rounds, and the most recent traded price was at a half a billion dollar valuation." Reuters could not confirm that valuation. Human trainers first got into AI training through data-labelling work that required less qualification and was also paid less, sometimes as low as $2, opens new tab, mostly done by people in African and Asian countries. As AI companies launch more advanced models, the demand for specialized trainers and across dozens of languages is on the rise, creating a well-paid niche where workers from a variety of subjects could become AI trainers without even knowing how to code. Demand from AI companies is leading to the creation of more companies that are offering similar services. "My inbox is basically inundated with new firms that pop up here and there. I do see this as a new space where companies hire humans just to create data for AI labs like us," Zhang said. Sign up here. Reporting by Supantha Mukherjee in Stockholm and Anna Tong in San Francisco, editing by Kenneth Li and Claudia Parsons Our Standards: The Thomson Reuters Trust Principles., opens new tab * * * * * Purchase Licensing Rights [caf76daf-1] Supantha Mukherjee Thomson Reuters Supantha leads the European Technology and Telecoms coverage, with a special focus on emerging technologies such as AI and 5G. He has been a journalist for about 18 years. He joined Reuters in 2006 and has covered a variety of beats ranging from financial sector to technology. He is based in Stockholm, Sweden. * [75b029fc-7] Anna Tong Thomson Reuters Anna Tong is a correspondent for Reuters based in San Francisco, where she reports on the technology industry. She joined Reuters in 2023 after working at the San Francisco Standard as a data editor. Tong previously worked at technology startups as a product manager and at Google where she worked in user insights and helped run a call center. Tong graduated from Harvard University. * * * Read Next * Illustration shows AI (Artificial Intelligence) letters and robot hand Society & EquitycategorySouth Korea summit to target 'blueprint' for using AI in the militarySeptember 10, 2024 * Apple logo at an Apple store in Paris categoryApple drops out of talks to join OpenAI investment round, WSJ reports5:39 AM UTC * Illustration shows OpenAI logo Artificial IntelligencecategoryOpenAI offers one investor a sweetener that no others are getting12:13 AM UTC * Reuters logo WorldcategoryUAE foreign minister says his country is doubling down on U.S. tiesSeptember 27, 2024 LSEG Workspace Technology * Tata iPhone component plant disrupted by fire Tata iPhone component plant disrupted by fire, 10 given medical aid category * September 28, 2024 * 3:09 PM UTC No decision has been made on when manufacturing can restart. * Illustration shows AI (Artificial Intelligence) letters and robot hand Artificial IntelligencecategoryIf your AI seems smarter , it's thanks to smarter human trainers2:08 PM UTC * EV production line at a Volkswagen Anhui factory in Hefei categoryEU to vote on Oct 4 to finalize tariffs for China-made EVs, Bloomberg News reports7:43 AM UTC * Apple logo at an Apple store in Paris categoryApple drops out of talks to join OpenAI investment round, WSJ reports5:39 AM UTC * Binance founder and former chief Changpeng Zhao arrives for his sentencing in federal district court in Seattle TechnologycategoryBinance founder Zhao released from US custody, Bloomberg News reports12:20 AM UTC Site Index Browse * World * Business * Markets * Sustainability * Legal * Breakingviews * Technology * Investigations * Sports * Science * Lifestyle About Reuters * About Reuters, opens new tab * Careers, opens new tab * Reuters News Agency, opens new tab * Brand Attribution Guidelines, opens new tab * Reuters Leadership, opens new tab * Reuters Fact Check * Reuters Diversity Report, opens new tab Stay Informed * Download the App (iOS), opens new tab * Download the App (Android), opens new tab * Newsletters Information you can trust Reuters, the news and media division of Thomson Reuters, is the world's largest multimedia news provider, reaching billions of people worldwide every day. Reuters provides business, financial, national and international news to professionals via desktop terminals, the world's media organizations, industry events and directly to consumers. Follow Us * * * * * Thomson Reuters Products * Westlaw, opens new tab Build the strongest argument relying on authoritative content, attorney-editor expertise, and industry defining technology. * Onesource, opens new tab The most comprehensive solution to manage all your complex and ever-expanding tax and compliance needs. * Checkpoint, opens new tab The industry leader for online information for tax, accounting and finance professionals. LSEG Products * Workspace, opens new tab Access unmatched financial data, news and content in a highly-customised workflow experience on desktop, web and mobile. * Data Catalogue, opens new tab Browse an unrivalled portfolio of real-time and historical market data and insights from worldwide sources and experts. * World-Check, opens new tab Screen for heightened risk individual and entities globally to help uncover hidden risks in business relationships and human networks. * Advertise With Us, opens new tab * Advertising Guidelines * Purchase Licensing Rights, opens new tab * Cookies, opens new tab * Terms of Use * Privacy, opens new tab * Digital Accessibility, opens new tab * Corrections * Site Feedback, opens new tab All quotes delayed a minimum of 15 minutes. See here for a complete list of exchanges and delays. (c) 2024 Reuters. All rights reserved