[HN Gopher] Superhuman AI Exfiltrates Emails
___________________________________________________________________
Superhuman AI Exfiltrates Emails
A bit more at https://simonwillison.net/2026/Jan/12/superhuman-ai-
exfiltra...
Author : takira
Score : 79 points
Date : 2026-01-12 18:38 UTC (1 days ago)
(HTM) web link (www.promptarmor.com)
(TXT) w3m dump (www.promptarmor.com)
| sarelta wrote:
| I'm impressed Superhuman seems to have handled this so well -
| lots of big names are fumbling with AI vuln disclosures.
| Grammarly is not necessarily who I would have bet on to get it
| right
| empiko wrote:
| I wonder how they handled it. Everybody's connecfing their AI
| to the Web, but it automatically means that any data AI has
| access to can be extracted by the attacker. The only safe way
| forward is to 1. disconnect the Web or 2. perhaps to filter the
| generated URLs aggressively.
| ttoinou wrote:
| We should have a clearer view of permissions of the AI,
| operations it does, and have one button per day to
| accept/deny operations from given data. Instead of auto
| approval.
| wat10000 wrote:
| Private data, untrusted data, communication: an LLM can
| safely have two of these, but never all three.
|
| Browsing the web is both communication and untrusted data, so
| it must never have access to any trusted data if it has the
| ability to browse the web.
|
| The problem is, so much of what people want from these things
| involves having all three.
| TeMPOraL wrote:
| > _The problem is, so much of what people want from these
| things involves having all three._
|
| Pretty much. Also there's no way of "securing" LLMs without
| destroying the quality that makes them interesting and
| useful in the first place.
|
| I'm putting "securing" in scare quotes because IMO it's
| fool's errand to even try - LLMs are _fundamentally_ not
| securable like regular, narrow-purpose software, and should
| not be treated as such.
| 0xferruccio wrote:
| The primary exfiltration vector for LLMs is making network
| requests via images with sensitive data as parameters.
|
| As Claude Code increasingly uses browser tools, we may need to
| move away from .env files to something encrypted, kind of like
| rails credentials, but without the secret key in the .env
| SahAssar wrote:
| So you are going to take the untrusted tool that kept leaking
| your secrets, keep the secrets away from it but still use it to
| code the thing that uses the secrets? Are you actually
| reviewing the code it produces? In 99% of cases that's a "no"
| or a soft "sometimes".
| TeMPOraL wrote:
| That's exactly what one does with their employees when one
| deploys "credential vaults", so?
| SahAssar wrote:
| Employees are under contract and are screened for basic
| competence. LLMs aren't and can't be.
| TeMPOraL wrote:
| > _Employees are under contract and are screened for
| basic competence. LLMs aren 't_
|
| So perhaps they should be.
|
| > _and can 't be._
|
| Ah but they must, because there's not much else you can
| do.
|
| You can't secure LLMs like they were just regular,
| narrow-purpose software, because they aren't. They're by
| nature more like little people on a chip (this is an
| explicit design goal) - and need to be treated
| accordingly.
| SahAssar wrote:
| > So perhaps they should be.
|
| Unless both the legalities and technology radically
| change they will not be. And the companies building them
| will not take on the burden since the technology has
| proved to be so unpredictable (partially by design) and
| unsafe.
|
| > designed to be more like little people on a chip - and
| need to be treated accordingly
|
| Deeply unpredictable and unsafe people on a chip, so not
| the sort that I generally want to trust secrets with.
|
| I don't think it's that complex, you can have secure
| systems or you can have current gen LLMs. You can't have
| both in the same place.
| TeMPOraL wrote:
| > _Deeply unpredictable and unsafe people on a chip, so
| not the sort that I generally want to trust secrets
| with._
|
| Very true when comparing to acquaintances, but at a scale
| of any company or system except the tiniest ones, you
| can't blindly trust people in general either. Building
| systems involving people and LLMs is pretty similar.
|
| > _I don 't think it's that complex, you can have secure
| systems or you can have current gen LLMs. You can't have
| both in the same place._
|
| That is, indeed, the key. My point is that, unlike the
| popular opinion in threads like this, it does _not_
| follow that we need to give up on LLMs, or that we need
| to fix the security issues. The former is undesirable,
| the latter is fundamentally impossible.
|
| What we need is what we've been doing ever since
| civilization took shape, ever since we've started
| building machines: recognize that automatons and people
| are different kinds of components, with different
| reliability and security characteristics. You can't
| blindly substitute one for the other, but there are ways
| to make them work together. Most systems we've created
| are of that nature.
|
| What people still get wrong is treating LLMs as
| "automatons" components. They're not, they're "people"
| components.
| xyzzy123 wrote:
| One tactic I've seen used in various situations is proxies
| _outside_ the sandbox that augment requests with credentials /
| secrets etc.
|
| Doesn't help in the case where the LLM is processing actually
| sensitive data, ofc.
| djaouen wrote:
| Programming used to prevent this by separating code from data. AI
| (currently) has no such safeguards.
| TeMPOraL wrote:
| _Reality_ doesn 't have a distinction between "code" and
| "data"; those are categories of convenience, and don't even
| have a proper definition (what is code and what is data depends
| on who's asking and why). Any such distinction requires
| mechanically enforcing it; AI won't have it, because it's not
| natural, and adding it destroys generality of the model.
| observationist wrote:
| As limited as they are, LLMs are demonstrably smarter than a
| whole lot of people, and the number of people more clever than
| the best AI is going to dwindle, rapidly, especially in the
| domain of doing sneaky shit really fast on a computer.
|
| There are countless examples of schemes in stories where codes
| and cryptography are used to exfiltrate information and evade
| detection, and these models are trained on every last piece of
| technical, practical text humanity has produced on the subject.
| All they have to do is contextualize what's likely being done to
| check and mash together two or three systems it thinks is likely
| to go under the radar.
| ineedasername wrote:
| Why does an agent tasked with email summarizing have access to
| anything else? There's plenty of difference between an agent and
| a background service or daemon but it's at minimum got to be
| given the same restrictions in scope they would be, or an intern
| using your system for the same purpose. Developers need to bring
| the same ZTA mindset to agent permissions they would to building
| the other services and infrastructure they rely on.
___________________________________________________________________
(page generated 2026-01-13 23:00 UTC)