https://www.businessinsider.com/company-using-chatgpt-mental-health-support-ethical-issues-2023-1 A vertical stack of three evenly spaced horizontal lines. A magnifying glass. It indicates, "Click to perform a search". The word "Insider". 0 Newsletters An icon in the shape of a person's head and shoulders. It often indicates a user profile. Log in An icon in the shape of a person's head and shoulders. It often indicates a user profile. Subscribe * The word Business * The word Life * The word News * The word Reviews * Business + Tech + Finance + Markets + Strategy + Retail + Advertising + Healthcare + Premium * Life + Entertainment + Culture + Travel + Food + Health + Parenting + Beauty + Style * News + Politics + Military & Defense + Sports + Opinion * Reviews + Tech + Streaming + Home + Kitchen + Style + Beauty + Gifts + Deals + Pets + Parenting + Coupons + Health + Learning + Hobbies & Crafts + Travel * All + A-Z + Advertising + Business + Careers + Coupons o DoorDash o Walmart o Dell o Staples o Under Armour + Culture + Design + Entertainment + Executive Lifestyle + Finance + Food + Health + Healthcare + Intelligence + Latest + Life + Markets Insider + Media + Military & Defense + News + Opinion + People + Personal Finance o Banking o Credit Cards o Insurance o Investing o Loans o Mortgages + Politics + Premium + Retail + Reviews + Science + Sports + Strategy + Tech + Transportation + Travel + TV o Beauty Explorers o Best in Town o Bonkers Closets o Festival Foodies o Movies Insider o Pop Culture Decoded o The Making Of o The Ultimate List + Featured + Talent Insider + About + About + Advertise + Careers + Code of Ethics + Contact Us + Corporate + Corrections Policy + Follow + RSS + Sitemap + Facebook + Twitter + Instagram + YouTube + LinkedIn + Subscriptions + Intelligence + Premium US Markets Loading... H M S In the news Two crossed lines that form an 'X'. It indicates a way to close an interaction, or dismiss a notification. It indicates an expandable section or menu, or sometimes previous / next navigation options. HOMEPAGE 0 Newsletters An icon in the shape of a person's head and shoulders. It often indicates a user profile. Log in Subscribe Home It indicates an expandable section or menu, or sometimes previous / next navigation options. Tech Online mental health company uses ChatGPT to help respond to users in experiment -- raising ethical concerns around healthcare and AI technology Bethany Biron 2023-01-07T19:34:19Z A bookmark The letter F. An envelope. It indicates the ability to send an email. An curved arrow pointing right. A stylized bird with an open mouth, tweeting. Twitter The word "in". LinkedIn A stylized letter F. Flipboard The letter F. Facebook An envelope. It indicates the ability to send an email. Email An image of a chain link. It symobilizes a website link url. Copy Link Read in app An image of a phone with ChatGPT and OpenAI's logo visible. ChatGPT, a AI chat bot, has gone viral in the past two weeks. NurPhoto/Getty Images * A digital mental health company is drawing ire for using GPT-3 technology without informing users. * Koko co-founder Robert Morris told Insider the experiment is "exempt" from informed consent law due to the nature of the test. * Some medical and tech professionals said they feel the experiment was unethical. Sign up for our newsletter for the latest tech news and scoops -- delivered daily to your inbox. Something is loading. Thanks for signing up! Access your favorite topics in a personalized feed while you're on the go. download the app Email address [ ] [Sign up] By clicking 'Sign up', you agree to receive marketing emails from Insider as well as other partner offers and accept our Terms of Service and Privacy Policy. As ChatGPT's use cases expand, one company is using the artificial intelligence to experiment with digital mental health care, shedding light on ethical gray areas around the use of the technology. Rob Morris -- co-founder of Koko, a free mental health service and nonprofit that partners with online communities to find and treat at-risk individuals -- wrote in a Twitter thread on Friday that his company used GPT-3 chatbots to help develop responses to 4,000 users. Morris said in the thread that the company tested a "co-pilot approach with humans supervising the AI as needed" in messages sent via Koko peer support, a platform he described in an accompanying video as "a place where you can get help from our network or help someone else." "We make it very easy to help other people and with GPT-3 we're making it even easier to be more efficient and effective as a help provider," Morris said in the video. ChatGPT is a variant of GPT-3, which creates human-like text based on prompts, both created by OpenAI. Koko users were not initially informed the responses were developed by a bot, and "once people learned the messages were co-created by a machine, it didn't work," Morris wrote on Friday. "Simulated empathy feels weird, empty. Machines don't have lived, human experience so when they say 'that sounds hard' or 'I understand', it sounds inauthentic," Morris wrote in the thread. "A chatbot response that's generated in 3 seconds, no matter how elegant, feels cheap somehow." However, on Saturday, Morris tweeted "some important clarification." "We were not pairing people up to chat with GPT-3, without their knowledge. (in retrospect, I could have worded my first tweet to better reflect this)," the tweet said. "This feature was opt-in. Everyone knew about the feature when it was live for a few days." Morris said Friday that Koko "pulled this from our platform pretty quickly." He noted that AI-based messages were "rated significantly higher than those written by humans on their own," and that response times decreased by 50% thanks to the technology. Ethical and legal concerns The experiment led to outcry on Twitter, with some public health and tech professionals calling out the company on claims it violated informed consent law, a federal policy which mandates that human subjects provide consent before involvement in research purposes. "This is profoundly unethical," media strategist and author Eric Seufert tweeted on Saturday. "Wow I would not admit this publicly," Christian Hesketh, who describes himself on Twitter as a clinical scientist, tweeted Friday. "The participants should have given informed consent and this should have passed through an IRB [institutional review board]." In a statement to Insider on Saturday, Morris said the company was "not pairing people up to chat with GPT-3" and said the option to use the technology was removed after realizing it "felt like an inauthentic experience." "Rather, we were offering our peer supporters the opportunity to use GPT-3 to help them compose better responses," he said. "They were getting suggestions to help them write more supportive responses more quickly." Morris told Insider that Koko's study is "exempt" from informed consent law, and cited previous published research by the company that was also exempt. "Every individual has to provide consent to use the service," Morris said. "If this were a university study (which it's not, it was just a product feature explored), this would fall under an 'exempt' category of research." He continued: "This imposed no further risk to users, no deception, and we don't collect any personally identifiable information or personal health information (no email, phone number, ip, username, etc)." A woman sits on a couch with her phone A women seeks mental health support on her phone. Beatriz Vera/EyeEm/ Getty Images ChatGPT and the mental health gray area Still, the experiment is raising questions about ethics and the gray areas surrounding the use of AI chatbots in healthcare overall, after already prompting unrest in academia. Arthur Caplan, professor of bioethics at New York University's Grossman School of Medicine, wrote in an email to Insider that using AI technology without informing users is "grossly unethical." "The ChatGPT intervention is not standard of care," Caplan told Insider. "No psychiatric or psychological group has verified its efficacy or laid out potential risks." He added that people with mental illness "require special sensitivity in any experiment," including "close review by a research ethics committee or institutional review board prior to, during, and after the intervention" Caplan said use of GPT-3 technology in such ways could impact its future in the healthcare industry more broadly. "ChatGPT may have a future as do many AI programs such as robotic surgery," he said. "But what happened here can only delay and complicate that future." Morris told Insider his intention was to "emphasize the importance of the human in the human-AI discussion." "I hope that doesn't get lost here," he said. A picture of a switch and lightbulb [light-swit] Sign up for notifications from Insider! Stay up to date with what you want to know. Subscribe to push notifications Read next NOW WATCH: ChatGPT GPT-3 Tech Insider More... Two crossed lines that form an 'X'. It indicates a way to close an interaction, or dismiss a notification. Recommended Video Insider [Insider-lo] Business Insider [BI-logo-da] Follow us on: [svg] [facebook] [svg] [twitter] [svg] [linkedin] [svg] [youtube] [svg] [instagram] Download on the App Store [app-store-] Get it on Google Play [google-pla] * * Copyright (c) 2023 Insider Inc. All rights reserved. Registration on or use of this site constitutes acceptance of our * Terms of Service , Privacy Policy and Cookies Policy . * Contact Us * Sitemap * Disclaimer * Accessibility * Commerce Policy * Advertising Policies * CA Privacy Rights * Coupons * Made in NYC * Jobs @ Insider * Stock quotes by finanzen.net * Reprints & Permissions * International Editions: * United States US * International INTL * Asia AS * Deutschland & Osterreich AT * Deutschland DE * Espana ES * India IN * Japan JP * Mexico MX * Netherlands NL * Polska PL * South Africa ZA Insider.com TM Logo [insider-co] Insider-Inc Logo [insider-in] Business Insider DE Logo [BI-DE-Blac] Tech Insider Logo [Tech-Insid]