https://www.theregister.com/2023/02/14/microsoft_ai_bing_error/ [user] [user] Sign in / up The Register(r) -- Biting the hand that feeds IT [magn] [burg] [burg] Topics Security Security All SecurityCyber-crimePatchesResearchCSO (X) Off-Prem Off-Prem All Off-PremEdge + IoTChannelPaaS + IaaSSaaS (X) On-Prem On-Prem All On-PremSystemsStorageNetworksHPCPersonal Tech (X) Software Software All SoftwareAI + MLApplicationsDatabasesDevOpsOSesVirtualization (X) Offbeat Offbeat All OffbeatDebatesColumnistsScienceGeek's GuideBOFHLegalBootnotesSite NewsAbout Us (X) Vendor Voice Vendor Voice Vendor Voice All Vendor VoiceAmazon Web Services (AWS) Business Transformation Google Cloud for Startups (X) Resources Resources Whitepapers Webinars Newsletters [aiml] AI + ML 40 comment bubble on white Microsoft's AI Bing also factually wrong, fabricated text during launch demo 40 comment bubble on white Redmond's hype box and Google's Bard just as bad as each other icon Katyanna Quach Tue 14 Feb 2023 // 11:33 UTC # Microsoft's new AI-powered Bing search engine generated false information on products, places, and could not accurately summarize financial documents, according to the company's promo video used to launch the product last week. After months of speculation, CEO Satya Nadella finally confirmed rumors that Microsoft was going to revamp Bing with an OpenAI chatbot, reportedly more powerful than ChatGPT. New capabilities showcasing Bing's potential to make web search more flexible and efficient were demoed at a private, invite-only press event held at the company's headquarters in Washington. The following day, Google launched its own rival AI-search chatbot Bard and was heavily criticized when it made a factual error about the James Webb Space Telescope. Google's parent biz Alphabet's market value temporarily dropped by 9 percent shortly afterwards, a decline worth over $120 billion, prompting investors and analysts to debate whether it was losing to Microsoft. [aiml] In reality, both Microsoft's Bing and Google's Bard are just as bad as each other. Both companies launched shoddy AI chatbots that generated text containing false information, but Microsoft's mistakes were not immediately caught. Now, some of its errors have been spotted by Dmitri Brereton, a search engine researcher. [aiml] [aiml] Brereton pointed out that Bing claimed a specific pet hair vacuum cleaner had a "short cord length of 16 feet" despite it being a handheld machine, and that it may be too noisy. A link providing the website Bing summarized information from, however, said the vacuum is actually quiet and cordless. When Yusuf Mehdi, Microsoft's Corporate Vice President, Modern Life, Search, and Devices, asked Bing about the nightlife in Mexico, it fabricated some details about existing bars and clubs. The opening hours for one listing were wrong, for example, whilst it claimed another had a website for users to browse when it did not. Bing also missed vital information too, and didn't mention El Almacen was, in fact, one of Mexico's oldest gay bars. [aiml] Microsoft also touted a feature where Bing could summarise information from financial documents, but the software made glaring errors here too. A demonstration of Bing generating key takeaways from department stores Gap and Lulelemon's financials shows it quoting wrong numbers and figures that don't appear in the original documents at all. "Bing AI got some answers completely wrong during their demo. But no one noticed. Instead, everyone jumped on the Bing hype train," Brereton wrote in a blog post on Substack. "Google's Bard got an answer wrong during an ad, which everyone noticed. Now the narrative is 'Google is rushing to catch up to Bing and making mistakes!'. That would be a fine narrative if Bing didn't make even worse mistakes during its own demo." None of this is surprising. Language models powering the new Bing and Bard are prone to fabricating text that is often false. They learn to generate text by predicting what words should go next given the sentences in an input query with little understanding of the tons of data scraped from the internet ingested during their training. Experts even have a word for it: hallucination. [aiml] If Microsoft and Google can't fix their models' hallucinations, AI-powered search is not to be trusted no matter how alluring the technology appears to be. Chatbots may be easy and fun to use, but what's the point if they can't give users useful, factual information? Automation always promises to reduce human workloads, but current AI is just going to make us work harder to avoid making mistakes. * Google's AI search bot Bard makes $120b error on day one * Google unleashes fightback against ChatGPT, a Bard by any other name * This product is terrible. Can you deliver it in 20 years' time when it becomes popular? * Life's a beach - then you're the comms nexus of the British Empire and Marconi-baiting hax0rs "I think everyone can see the amazing potential for LLM powered search engines," Brereton told The Register. "It feels like we're so close to having it, and it's a huge shift from the past, and a much smoother user experience. Everyone wants it to happen right now. Some people are already using ChatGPT as their main search engine, even though the answers may not be accurate. It's just such a superior experience that people can't help but hop on the hype train." The Register asked Microsoft for comment and the company told us it is aware of this report "and have analyzed its findings in our efforts to improve this experience." "It's important to note that we ran our demo using a preview version," a Microsoft spokesperson added. "Over the past week alone, thousands of users have interacted with our product and found significant user value while sharing their feedback with us, allowing the model to learn and make many improvements already. We recognize that there is still work to be done and are expecting that the system may make mistakes during this preview period, which is why the feedback is critical so we can learn and help the models get better." (r) Get our Tech Resources # Share Similar topics * Bing * Google * Microsoft More like these x Similar topics * Bing * Google * Microsoft Narrower topics * Active Directory * Android * App stores * Azure * BSoD * Chrome * Chromium * Excel * Exchange Server * Google AI * Google Cloud Platform * Google Nest * G Suite * HoloLens * Internet Explorer * Kubernetes * LinkedIn * Microsoft 365 * Microsoft Build * Microsoft Edge * Microsoft Ignite * Microsoft Office * Microsoft Surface * Microsoft Teams * .NET * Office 365 * OS/2 * Outlook * Patch Tuesday * Pixel * Pluton * Privacy Sandbox * SharePoint * Skype * SQL Server * Tavis Ormandy * Visual Studio * Visual Studio Code * Windows * Windows 10 * Windows 11 * Windows 7 * Windows 8 * Windows Server * Windows Server 2003 * Windows Server 2008 * Windows Server 2012 * Windows Server 2013 * Windows Server 2016 * Windows Subsystem for Linux * Windows XP * Xbox * Xbox 360 Broader topics * Alphabet * Bill Gates * Search * Search Engine Similar topics # Share 40 comment bubble on white COMMENTS Similar topics * Bing * Google * Microsoft More like these x Similar topics * Bing * Google * Microsoft Narrower topics * Active Directory * Android * App stores * Azure * BSoD * Chrome * Chromium * Excel * Exchange Server * Google AI * Google Cloud Platform * Google Nest * G Suite * HoloLens * Internet Explorer * Kubernetes * LinkedIn * Microsoft 365 * Microsoft Build * Microsoft Edge * Microsoft Ignite * Microsoft Office * Microsoft Surface * Microsoft Teams * .NET * Office 365 * OS/2 * Outlook * Patch Tuesday * Pixel * Pluton * Privacy Sandbox * SharePoint * Skype * SQL Server * Tavis Ormandy * Visual Studio * Visual Studio Code * Windows * Windows 10 * Windows 11 * Windows 7 * Windows 8 * Windows Server * Windows Server 2003 * Windows Server 2008 * Windows Server 2012 * Windows Server 2013 * Windows Server 2016 * Windows Subsystem for Linux * Windows XP * Xbox * Xbox 360 Broader topics * Alphabet * Bill Gates * Search * Search Engine TIP US OFF Send us news --------------------------------------------------------------------- Other stories you might like Conversational AI tells us what we want to hear - a fib that the Web is reliable and friendly Opinion As Google and Microsoft make us all AI consumers, remember that to err is human, but to really mess things up at scale takes a computer AI + ML8 Feb 2023 | 48 Google unleashes fightback against ChatGPT, a Bard by any other name Competition for AI-powered search between Microsoft and Google heats up AI + ML6 Feb 2023 | 43 Google now won't black-hole all AI-made pages as spam Plus: Microsoft's nu-Bing can't answer question on exoplanets correctly either AI + ML9 Feb 2023 | 33 It's time to fill those cloud security gaps Here's how Wiz can help Sponsored Feature [aiml] Microsoft tells people to prepare for AI search engine that goes Bing! Redmond thinks OpenAI can be secret sauce to beat Google AI + ML7 Feb 2023 | 47 Google shows off upcoming AI search features, leaves Bard waiting in the wings If you were hoping for more info about the ChatGPT competitor, keep hoping AI + ML8 Feb 2023 | 10 AWS, Microsoft, Google among businesses owed money after FTX collapse Join the queue, eh? Personal Tech27 Jan 2023 | 5 Google opens arms to VMware in the cloud and Microsoft 365 on ChromeOS From the Department of Improbable Alliances Off-Prem2 Feb 2023 | 9 Google miscalculates severance payments for some Googlers Double counting share units led to mistake, 'we have corrected' the error, says ad search titan Off-Prem13 Feb 2023 | 6 Google pushes fake abortion clinic ads to lower-income women, report says At least those who live in Phoenix and Atlanta Personal Tech8 Feb 2023 | 27 Google's AI search bot Bard makes $120b error on day one 'This highlights the importance of a rigorous testing process' says Choc Fact. No sh%t... AI + ML8 Feb 2023 | 48 Microsoft-Activision deal will hurt UK gamers, says watchdog Competition and Markets Authority isn't buying promises of equal access to Call of Duty Personal Tech8 Feb 2023 | 3 The Register icon Biting the hand that feeds IT About Us* * Contact us * Advertise with us * Who we are Our Websites* * The Next Platform * DevClass * Blocks and Files Your Privacy* * Cookies Policy * Privacy Policy * T's & C's * Do not sell my personal information Situation Publishing Copyright. All rights reserved (c) 1998-2023 no-js