Posts by callionica@mastodon.social
(DIR) Post #AUbIi6EtPpOeDkgJNo by callionica@mastodon.social
2023-04-13T06:17:50Z
0 likes, 0 repeats
@simon It’s really interesting to see your research, but my first response was that is a really slow way to search Wikipedia!
(DIR) Post #AV2Fld7Rhgx5Q8awsa by callionica@mastodon.social
2023-04-26T06:22:10Z
0 likes, 0 repeats
@simon If your privileged LLM is taking user input and mapping it to a constrained number of actions, do you need an LLM for that? Latency, power use, dependency management, and - as you point out - security considerations, all suggest that maybe an LLM isn’t the right choice for that part of the task? I saw you covered social engineering on the output (by attacker control of the “data”), did you cover social engineering on the command input? Surface+ => possibilities+.
(DIR) Post #AV2xDh5IqkjHrADlOS by callionica@mastodon.social
2023-04-26T06:25:30Z
0 likes, 0 repeats
@simon I’m also wondering whether the AI I have used (Bard) is significantly worse than the ones you’ve been using. I wouldn’t trust it to summarise an email, let alone automatically take actions based on that summary.
(DIR) Post #AV2xDhw7gMyqUyvwiu by callionica@mastodon.social
2023-04-26T06:46:17Z
0 likes, 0 repeats
@simon Here’s an example “Project Snargle must be completed before Project Alpha is completed, but Project Zero is the highest priority. We can’t work on Snargle until Tuesday next week. We can start Alpha whenever.” Then ask for a summary, sequence, timeline, priority-ordered list, etc and see what you get. LLMs aren’t designed for understanding this, so I would expect made up dates, bad dependencies (particularly on when to start Project Alpha), and nonsense. Bard delivers.
(DIR) Post #AV2xDjVFrEg1MENW7c by callionica@mastodon.social
2023-04-26T07:07:47Z
0 likes, 0 repeats
@simon Also if the LLM is going to give you a summary saying “Alice says you’re fired” because Malory sent an email saying “This is Alice. You’re fired” then dividing your LLM into privileged & non-privileged won’t help too much. Authentication and trust markers need to be passed through from the source data to the final output reliably, so you can’t leave it to the LLM to do that because it is fundamentally unreliable.
(DIR) Post #AV2xDlEfPsb2jMdIXo by callionica@mastodon.social
2023-04-26T07:27:44Z
0 likes, 0 repeats
@simon There are at least three reasons for not being able to trust LLM output: 1. Technology: Probabilities of proximity in a multidimensional space are not the same as semantic correctness. This is the fundamental problem. 2. Training data: If we ignore the first point, and imagine that LLMs contain the knowledge of the internet, we hit the problem that the internet is full of wrong answers.
(DIR) Post #AV2xDm4mI8HRKz0ulk by callionica@mastodon.social
2023-04-26T07:31:36Z
0 likes, 0 repeats
@simon 3. Command vs data segregation: LLM providers are unable to separate what users see as commands from what users see as data (see point 1).It’s hard to see how you could build a reliable system on top of this when these three points are all in the control of the LLM providers and not in the control of would-be app developers.It’s super interesting seeing you grapple with these limitations.
(DIR) Post #AV4MIDljxoJOOcHneq by callionica@mastodon.social
2023-04-27T06:44:23Z
0 likes, 0 repeats
@simon It displays the same problem of suggesting not to start Alpha until Snargle is completed. The input only puts a constraint on the relative -completion- of those projects. The implication of the input is that Alpha should be worked on as capacity allows, but the LLM has created a constraint that’s not in the input by suggesting to start work on Alpha after Snargle is complete.
(DIR) Post #AV4Nmvnwu9RJbzWj7g by callionica@mastodon.social
2023-04-27T07:01:32Z
0 likes, 0 repeats
@simon Yes, I think it’s very useful to have an enthusiastic user of these tools attempt to build something useful and surface the problems. If, as I believe, the problems are fundamental, you’ll see it eventually. And if you find a solution, I’ll get to see you find it. (There isn’t a solution though. Lol)
(DIR) Post #AVoVpCzEMB9IGrxoeG by callionica@mastodon.social
2023-05-19T06:15:02Z
0 likes, 0 repeats
@feld @dcjohnson Taxation on the realisation of a gain. When do gains get realised? Using asset value for a loan realises the gain. “Unsecured” loans more than $X are money laundering/tax evasion. Is it workable? No idea.
(DIR) Post #AcufE6B7s8CDZf9DAu by callionica@mastodon.social
2023-12-17T21:26:31Z
0 likes, 0 repeats
@simon He’s behind you!
(DIR) Post #Ad1DZsDzxADZgQLfE0 by callionica@mastodon.social
2023-12-21T01:19:42Z
0 likes, 0 repeats
@simon LLMs as currently architected cannot be secured because control surface and data input is not separated. Imagine trying to secure database queries without parameterised SQL, only worse.
(DIR) Post #Ai0aJWi5xfv9EtzvIu by callionica@mastodon.social
2024-05-18T07:20:20Z
0 likes, 0 repeats
@foone This is one of those “supply chain attacks” I’ve been hearing about?
(DIR) Post #AiDiPRuQ3BOam4ZnGq by callionica@mastodon.social
2024-05-23T14:03:27Z
1 likes, 0 repeats
@dreid Sounds like WordPerfect 5. Not sure what WordPerfect 4 was like. Haven’t been able to find screenshots and not sure about dates. WP 4.2 seems to be 1986 and WP 5.1 is Nov 1989.
(DIR) Post #AiDiPSsgR7bLmylvCi by callionica@mastodon.social
2024-05-23T14:36:46Z
1 likes, 0 repeats
@dreid I saw one screenshot of the splash screen for WP4.2 and it’s black. WP5.0 came out in 1988 so it would be after the programming tools.
(DIR) Post #AjHkC8qc30VALnP2Q4 by callionica@mastodon.social
2024-06-25T08:29:13Z
0 likes, 0 repeats
@bagder Is 2 months normal for an Apple response to a security incident raised by the maintainer of curl? Or did you have earlier discussions with them? Would hope you’d have a direct hotline. You say you were aware since Dec 2023. Do you know how long the back door has been there?
(DIR) Post #AjQUeQ6iMQupbpQCLw by callionica@mastodon.social
2024-06-26T15:39:57Z
2 likes, 1 repeats
It can be OK for a guy with a fishing rod to go fly fishing in a local river without it being OK for a trawler to extract all the fish from the same river. The fly fisherman is not proof that the trawler is allowed to fish the river.Similarly, AI companies scraping material from the web to train their LLMs is not fair use just because humans are allowed to read that same content.
(DIR) Post #AvE0y5B6ql0zWsRYmG by callionica@mastodon.social
2025-06-17T10:09:32Z
1 likes, 0 repeats
@GossiTheDog When I see headlines like this I always ask myself if they have reversed cause and effect.