https://www.schneier.com/blog/archives/2024/02/microsoft-is-spying-on-users-of-its-ai-tools.html Schneier on Security Menu * Blog * Newsletter * Books * Essays * News * Talks * Academic * About Me Search Powered by DuckDuckGo [ ] [Go] ( ) Blog ( ) Essays (*) Whole site Subscribe Atom FeedFacebookTwitterKindleE-Mail Newsletter (Crypto-Gram) HomeBlog Microsoft Is Spying on Users of Its AI Tools Microsoft announced that it caught Chinese, Russian, and Iranian hackers using its AI tools--presumably coding tools--to improve their hacking abilities. From their report: In collaboration with OpenAI, we are sharing threat intelligence showing detected state affiliated adversaries--tracked as Forest Blizzard, Emerald Sleet, Crimson Sandstorm, Charcoal Typhoon, and Salmon Typhoon--using LLMs to augment cyberoperations. The only way Microsoft or OpenAI would know this would be to spy on chatbot sessions. I'm sure the terms of service--if I bothered to read them--gives them that permission. And of course it's no surprise that Microsoft and OpenAI (and, presumably, everyone else) are spying on our usage of AI, but this confirms it. Tags: artificial intelligence, cyberespionage, espionage, Microsoft Posted on February 20, 2024 at 7:02 AM * 13 Comments Comments Gideon * February 20, 2024 7:53 AM OpenAI are "spying" on ChatGPT sessions the same way that email providers "spy" on your email to filter spam. This sounds like a variation on "Google is reading your email". Snarki, child of Loki * February 20, 2024 8:55 AM Years ago, when governments were considering "banning" crypto, I came up with a program to turn random bits (i.e., encrypted data) into "Vogon Poetry". Now that AI-training is scraping websites, if I can get a way of detecting when that's happening, I'll redirect the web-scraper to a cgi producing Vogon Poetry from /dev/random. They can drink from the firehose, and nice to know that old software still has utility. echo * February 20, 2024 8:59 AM Microsoft and third parties spying on end users isn't a surprise. It would happen anyway. We know that. The difference is who does it and whether it's siloed or not. I can't comment on Microsoft AI tools being used for coding as I've never tried it but for the stuff I have tried with Microsoft Image AI their database is polluted and their filters are junk. ChatGPT is so generic it's not worth the bother. Actually, thinking of the list of usual suspects, I read a good essay the other week which explained why the US can be monopolar. The basic gist was that the expertise and knowledge was there but the second it hit the upper tier reporting in to cabinet level jobs the bandwidth throttled massively before hitting a huge opportunity cost for high level political decisions hence not being able to politically track multiple targets. It sounds plausible. There's bigger questions about AI. Who frames it? Who uses it? How it's being used. It's not just a straightforward linear minded thing in its own echo chamber full of chiseled men and sweaty bearded hackers under the heel of [dogma of choice] tyrants feeding back on itself. That... would... be... silly... I can't easily find a single article which is properly up to date in all the areas to discuss the topic. No wonder AI data is polluted... One reason why military AI simulations which rapidly escalate to global thermonuclear war may do this for a reason. They're kinda, like, missing all the other stuff? Huh? As for all those clowns in Russia, Iran, and China stuck using the 21st Century equivalent of computer magazine cover-discs you haven't got anything we want. Stop objectifying women and beating your wife. Find another job. You'll be happier. https://womenlovetech.com/women-artificial-intelligence/ From transportation and education to media, customer service, and healthcare and wellness--industries are increasingly integrating artificial intelligence into their systems. Without more representation of women, the data these industries work off of and use to improve their operations will be deeply inaccurate. We don't just need more women researching AI; we must have them. Continuing to leave them out is not an option. It is vital to the growth and success of AI itself and our growth as a society, and our ability to advance. And: https://www.bbc.co.uk/news/business-67217915 Mr Chambers also says that women may fear having their ability questioned, if they use AI tools. "Women are more likely to be accused of not being competent, so they have to emphasise their credentials more to demonstrate their subject matter expertise in a particular field," he says. "There could be this feeling that if people know that you, as a woman, use AI, it's suggesting that you might not be as qualified as you are. "Women are already discredited, and have their ideas taken by men and passed off as their own, so having people knowing that you use an AI might also play into that narrative that you're not qualified enough. It's just another thing that's debasing your skills, your competence, your value." Or as Harriet Kelsall puts it: "I value authenticity and human creativity." Just a wannabe techguy * February 20, 2024 9:22 AM "AI"- what could possibly go wrong? Clive Robinson * February 20, 2024 9:42 AM @ Bruce, ALL, Re : Just saying yer know... "And of course it's no surprise that Microsoft and OpenAI (and, presumably, everyone else) are spying on our usage of AI, but this confirms it." As I've noted a few times already AI is rather more than just "spying" in the current sense of the word. As I've said "AI is the ultimate surveillance tool" we currently have, and it's use will be forced onto us one way or another Microsoft and OpenAI are just the start of it and if certain "Venture Capitalist"(VC) investment houses have their way they will pump-n-dump LLM's way above anything tech has ever seen before and not constructively but destructively from the get go. People should remember the secret behind AI's as surveillance tools is the "Be" path of, "Bedazzle, Beguile, Bewitch, befriend, and Betrayal." And by far the majority of Internet users will walk some or all of those stages and will end up harmed in ways that few can currently imagine. I'd like to think the warning would be heeded but history says otherwise. wiredog * February 20, 2024 9:51 AM I suspect they're doing lookups to see where the connections are coming from and, if the connections are from $BadPlaces, doing a deeper dive. As far as listening in general, how else are they going to improve the user experience? They record everything and send bits that the AI had a harder time dealing with to Real People(TM) to review. Just the way Apple, Amazon, and Microsoftd do with voice controlled evices. bcs * February 20, 2024 11:08 AM Based on the above quoted bit, It's possible the threat actors were detected using other means and only then one of the things they were observed using was the LLMs. Even assuming they were first detected while using the LLMs, it's also possible the detection was first via traffic analysis and only then was the activity inspected. That would imply the technical capability to do the generalized spying that the title suggests but doesn't require it actually be used. tl;dr; what is proven to exist by this report is related to what people will think given the post title, but is still short enough of it that I'd still call it misleading, and it runs the risk of prompting accusations of being alarmist. Or maybe it just looked like there might be state actors using the LLMs and so they are bluffing on the assumption that the mentioned states would never distribute enough information to convince even their own people it isn't true. That would actually work better at discouraging the practice the better a job the actors did at hiding their work. Probably Jesus * February 20, 2024 11:43 AM Microsoft built it's AI lab in China. Sounds like they sold out the US to China and is now trying to engage in damage control and playing victim Anonymous * February 20, 2024 11:47 AM Yeah, the all allow humans to review chat sessions. In case of OpenAI they might even train future models based on your conversations (might also be true for others, but OAI for sure) Nuno Sempere * February 20, 2024 12:38 PM I'm sure the terms of service--if I bothered to read them--gives them that permission I did bother to read them a few months ago. Terms of service allowed them to keep records for 30 days I'mMoreStupidThanYouCanImagineWhichMakesMeSmarterThanYou * February 20, 2024 2:08 PM YOU ARE THE PRODUCT! Humanoids: from the day you are conceived, to the day you are pooped out by a cow in a pasture (regardless of whether our dead bodies are buried and decomposed naturally over time, in the ground, or if they are turned into ashes via cremation, there is no escaping the fact that you will end up as grass fertilizer, and the grass is consumed and digested by cows, or anything else that feeds on it). Including everything that happens in-between (during the - (Great Dash) period, a.k.a. Life. Unless, of course, one pays for a "Moon Funeral", or maybe Mars, some day soon??? or, or, or... Speakin' of which, I gotta ask Roger W. if he'll be buried, (or his ashes spread), on The Dark Side Of Moon? In due time, of course. Disclaimer: The above paragraph is the output of my own AI Assistant, trained by yours truly, and fed an insane amount of phrases that reek of nothing but cold hard truths and PURE LOGIC. ClueBatIncoming * February 20, 2024 4:46 PM Good grief people. We are talking about a company that moves your browser tabs, including connected sessions, passwords, and tokens, from Chrome to Edge behind your back. We are talking about a company that scans your harddrive to see if you have any older versions of their software installed so they can target you for upselling. We are talking about a company that wants you to fill out a survey every time you try to use a competitor's product. That requires you to sign up for an internet account with them, and sign away all your legal rights, before you can log into the computer you bought. Sheesh. And you think they might be spying on you when you use their AI software? That ship sailed back in my great-grandfather's day. Stephane * February 20, 2024 5:01 PM It's documented here: https://learn.microsoft.com/en-us/azure/ ai-services/openai/concepts/abuse-monitoring You can ask for an opt out: https://learn.microsoft.com/en-us/legal/ cognitive-services/openai/data-privacy?context= %2Fazure%2Fai-services%2Fopenai%2Fcontext%2Fcontext# how-can-customers-get-an-exemption-from-abuse-monitoring-and-human-review Pretty much transparent to me. Atom Feed Subscribe to comments on this entry Leave a comment Cancel reply Login Name [ ] Email [ ] URL: [ ] [ ] Remember personal info? Fill in the blank: the name of this blog is Schneier on ___________ (required): [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] Comments: [ ] [loader] Allowed HTML * * * *
    1. * 
       Markdown Extra syntax via
https://michelf.ca/projects/php-markdown/extra/

[Preview] [Edit]

[Submit] 

 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
 [                                             ] 
D[                                             ] 

- EU Court of Human Rights Rejects Encryption Backdoors

Sidebar photo of Bruce Schneier by Joe MacInnis.

Powered by WordPress Hosted by Pressable

About Bruce Schneier

[Bruce-Schn]

I am a public-interest technologist, working at the intersection of
security, technology, and people. I've been writing about security
issues on my blog since 2004, and in my monthly newsletter since
1998. I'm a fellow and lecturer at Harvard's Kennedy School, a board
member of EFF, and the Chief of Security Architecture at Inrupt, Inc.
This personal website expresses the opinions of none of those
organizations.

Related Entries

  * Microsoft Executives Hacked
  * Poisoning AI Models
  * AI Bots on X (Twitter)
  * Code Written with AI Assistants Is Less Secure
  * AI Is Scarily Good at Guessing the Location of Random Photos

Featured Essays

  * The Value of Encryption
  * Data Is a Toxic Asset, So Why Not Throw It Out?
  * How the NSA Threatens National Security
  * Terrorists May Use Google Earth, But Fear Is No Reason to Ban It
  * In Praise of Security Theater
  * Refuse to be Terrorized
  * The Eternal Value of Privacy
  * Terrorists Don't Do Movie Plots

More Essays

Blog Archives

  * Archive by Month
  * 100 Latest Comments

Blog Tags

  * 3d printers
  * 9/11
  * A Hacker's Mind
  * Aaron Swartz
  * academic
  * academic papers
  * accountability
  * ACLU
  * activism
  * Adobe
  * advanced persistent threats
  * adware
  * AES
  * Afghanistan
  * air marshals
  * air travel
  * airgaps
  * al Qaeda
  * alarms
  * algorithms
  * alibis
  * Amazon
  * Android
  * anonymity
  * Anonymous
  * antivirus
  * Apache
  * Apple
  * Applied Cryptography
  * artificial intelligence

More Tags

Latest Book

A Hacker's Mind

More Books

Support Bloggers' Rights! Defend Privacy--Support Epic

  * Blog
  * Newsletter
  * Books
  * Essays
  * News
  * Talks
  * Academic
  * About Me