Post B6uZnzp6q4aFkPwN3g by rysiek@mstdn.social
 (DIR) More posts by rysiek@mstdn.social
 (DIR) Post #B6uZnz17pubLFOYS9I by rysiek@mstdn.social
       0 likes, 1 repeats
       
       RE: https://cyberplace.social/@GossiTheDog/116676826944489315I need people to understand that stuff like this will keep happening, for two reasons:1. To be useful these chatbots need to have full access to everything they are supposed to "manage"; otherwise they are pointless.2. Trying to stop prompt injection is basically trying to semantically filter natural language.These tools have no model of the world, no ontology to anchor any "safety instructions" in. There will always be a way to talk one's way around them.#InfoSecRT: https://cyberplace.social/users/GossiTheDog/statuses/116676826944489315
       
 (DIR) Post #B6uZnzNoTYkWNlAZVo by rysiek@mstdn.social
       0 likes, 0 repeats
       
       One way out of this is compartmentalization, hard-limiting chatbot's access to certain resources. But that defeats the purpose of the chatbot – you can't have a chatbot that manages your mail without giving that chatbot access to your mail...Another is to move towards more formalized instructions, which can then be properly constrained by permissions etc. But then you're re-inventing programming languages and access control, again defeating the purpose of a natural-language-processing chatbot.
       
 (DIR) Post #B6uZnzp6q4aFkPwN3g by rysiek@mstdn.social
       0 likes, 0 repeats
       
       We are several years into this and the biggest companies peddling these tools still cannot figure out how to make their products not fall for advanced cyberattack techniques like *checks notes* asking nicely again.Microslop Slopilot had (has?) a similar issue – Reprompt attack simply repeated the malicious prompt in a query parameter: https://www.techrepublic.com/article/news-reprompt-attack-microsoft-copilot/These are not going away.
       
 (DIR) Post #B6wJ9vnTBDNOj4y1BI by rysiek@mstdn.social
       0 likes, 0 repeats
       
       @paco Satya Nadella made sure Microsoft focused on security over 2 years ago, after all!https://www.geekwire.com/2024/haunted-by-repeated-breaches-microsoft-is-putting-security-above-all-else-vows-ceo-satya-nadella/
       
 (DIR) Post #B6wJ9whpneSlXtL22K by dgodon@mastodon.online
       0 likes, 0 repeats
       
       @rysiek @paco so you’re telling me they treat security as seriously as Meta treats privacy?
       
 (DIR) Post #B6wJ9x9U8qa4veH78S by Hex@kolektiva.social
       0 likes, 0 repeats
       
       @dgodon @rysiek @paco I'm glad old tech is back. I really missed being able to bypass auth by clicking "cancel" and knocking people offline with a single packet. I'm excited to see how Meta and Microsoft fulfil my nostalgia for easy bugs, but somehow with new technology. It's really a great mashup of the old and the new.
       
 (DIR) Post #B6wJ9xlProvEpI1PG4 by crowbriarhexe@tech.lgbt
       0 likes, 0 repeats
       
       @Hex @dgodon @rysiek @paco I’m old enough to remember “Writing Secure Code” and “Next Generation Secure Computing Base”
       
 (DIR) Post #B6wJ9yT1Fhnh0WQEDo by Rairii@labyrinth.zone
       0 likes, 0 repeats
       
       @crowbriarhexe @Hex @dgodon @rysiek @paco fun fact: both bitlocker and pluton related stuff is still referred to as "ngscb" internally today