[HN Gopher] Launch HN: Tinfoil (YC X25): Verifiable Privacy for ...
___________________________________________________________________
Launch HN: Tinfoil (YC X25): Verifiable Privacy for Cloud AI
Hello HN! We're Tanya, Sacha, Jules and Nate from Tinfoil:
https://tinfoil.sh. We host models and AI workloads on the cloud
while guaranteeing zero data access and retention. This lets us run
open-source LLMs like Llama, or Deepseek R1 on cloud GPUs without
you having to trust us--or any cloud provider--with private data.
Since AI performs better the more context you give it, we think
solving AI privacy will unlock more valuable AI applications, just
how TLS on the Internet enabled e-commerce to flourish knowing that
your credit card info wouldn't be stolen by someone sniffing
internet packets. We come from backgrounds in cryptography,
security, and infrastructure. Jules did his PhD in trusted hardware
and confidential computing at MIT, and worked with NVIDIA and
Microsoft Research on the same, Sacha did his PhD in privacy-
preserving cryptography at MIT, Nate worked on privacy tech like
Tor, and I (Tanya) was on Cloudflare's cryptography team. We were
unsatisfied with band-aid techniques like PII redaction (which is
actually undesirable in some cases like AI personal assistants) or
"pinky promise" security through legal contracts like DPAs. We
wanted a real solution that replaced trust with provable security.
Running models locally or on-prem is an option, but can be
expensive and inconvenient. Fully Homomorphic Encryption (FHE) is
not practical for LLM inference for the foreseeable future. The
next best option is using secure enclaves: a secure environment on
the chip that no other software running on the host machine can
access. This lets us perform LLM inference in the cloud while being
able to prove that no one, not even Tinfoil or the cloud provider,
can access the data. And because these security mechanisms are
implemented in hardware, there is minimal performance overhead.
Even though we (Tinfoil) control the host machine, we do not have
any visibility into the data processed inside of the enclave. At a
high level, a secure enclave is a set of cores that are reserved,
isolated, and locked down to create a sectioned off area.
Everything that comes out of the enclave is encrypted: memory and
network traffic, but also peripheral (PCIe) traffic to other
devices such as the GPU. These encryptions are performed using
secret keys that are generated inside the enclave during setup,
which never leave its boundaries. Additionally, a "hardware root of
trust" baked into the chip lets clients check security claims and
verify that all security mechanisms are in place. Up until
recently, secure enclaves were only available on CPUs. But NVIDIA
confidential computing recently added these hardware-based
capabilities to their latest GPUs, making it possible to run GPU-
based workloads in a secure enclave. Here's how it works in a
nutshell: 1. We publish the code that should run inside the secure
enclave to Github, as well as a hash of the compiled binary to a
transparency log called Sigstore 2. Before sending data to the
enclave, the client fetches a signed document from the enclave
which includes a hash of the running code signed by the CPU
manufacturer. It then verifies the signature with the hardware
manufacturer to prove the hardware is genuine. Then the client
fetches a hash of the source code from a transparency log
(Sigstore) and checks that the hash equals the one we got from the
enclave. This lets the client get verifiable proof that the enclave
is running the exact code we claim. 3. With the assurance that the
enclave environment is what we expect, the client sends its data to
the enclave, which travels encrypted (TLS) and is only decrypted
inside the enclave. 4. Processing happens entirely within this
protected environment. Even an attacker that controls the host
machine can't access this data. We believe making end-to-end
verifiability a "first class citizen" is key. Secure enclaves have
traditionally been used to remove trust from the cloud provider,
not necessarily from the application provider. This is evidenced by
confidential VM technologies such as Azure Confidential VM allowing
ssh access by the host into the confidential VM. Our goal is to
provably remove trust both from ourselves, aka the application
provider, as well as the cloud provider. We encourage you to be
skeptical of our privacy claims. Verifiability is our answer. It's
not just us saying it's private; the hardware and cryptography let
you check. Here's a guide that walks you through the verification
process: https://docs.tinfoil.sh/verification/attestation-
architectur.... People are using us for analyzing sensitive docs,
building copilots for proprietary code, and processing user data in
agentic AI applications without the privacy risks that previously
blocked cloud AI adoption. We're excited to share Tinfoil with HN!
* Try the chat (https://tinfoil.sh/chat): It verifies attestation
with an in-browser check. Free, limited messages, $20/month for
unlimited messages and additional models * Use the API
(https://tinfoil.sh/inference): OpenAI API compatible interface. $2
/ 1M tokens * Take your existing Docker image and make it end to
end confidential by deploying on Tinfoil. Here's a demo of how you
could use Tinfoil to run a deepfake detection service that could
run securely on people's private videos:
https://www.youtube.com/watch?v=_8hLmqoutyk. Note: This feature is
not currently self-serve. * Reach out to us at contact@tinfoil.sh
if you want to run a different model or want to deploy a custom
application, or if you just want to learn more! Let us know what
you think, we'd love to hear about your experiences and ideas in
this space!
Author : FrasiertheLion
Score : 89 points
Date : 2025-05-15 16:19 UTC (6 hours ago)
| Onavo wrote:
| Are you HIPAA compliant?
| FrasiertheLion wrote:
| Not yet, we're about one week away from SOC2, will pursue HIPAA
| which is arguably easier next.
| blintz wrote:
| Excited to see someone finally doing this! I can imagine folks
| with sensitive model weights being especially interested.
|
| Do you run into rate limits or other issues with TLS cert
| issuance? One problem we had when doing this before is that each
| spinup of the enclave must generate a fresh public key, so it
| needs a fresh, publicly trusted TLS cert. Do you have a
| workaround for that, or do you just have the enclaves run for
| long enough that it doesn't matter?
| FrasiertheLion wrote:
| We actually run into the rate limit issue often particularly
| while spinning up new enclaves while debugging. We plan on
| moving to HPKE: https://www.rfc-editor.org/rfc/rfc9180.html
| over the next couple months. This will let us generate keys
| inside the enclave and encrypt the payload with the enclave
| specific keys, while letting us terminate TLS in a proxy
| outside the enclave. All the data is still encrypted to the
| enclave using HPKE (and still verifiable).
|
| This will let us fix the rate limit issue.
| etaioinshrdlu wrote:
| Does the secure enclave also perform the TLS encryption on data
| leaving the enclave?
|
| Also, if you're decoding TLS on the enclave, wouldn't that imply
| that you're parsing HTTP and JSON on the GPU itself? Very
| interesting if true.
| natesales wrote:
| The verified trust boundary extends from the CPU to GPU [1],
| and TLS encrypts all data to/from the enclave and client so we
| can't see anything in the clear.
|
| HTTP parsing and application logic happens on the CPU like
| normal. The GPU runs CUDA just like any other app, after it's
| integrity is verified by the CPU. Data on the PCIe bus is
| encrypted between the CPU and GPU too.
|
| [1]
| https://github.com/NVIDIA/nvtrust/blob/main/guest_tools/atte...
| etaioinshrdlu wrote:
| Could you talk more about how how this works? I don't think
| linked article doesn't given enough detail on how the trust
| boundary extends from CPU to GPU.
|
| Does the CPU have the ability to see unencrypted data?
| natesales wrote:
| The keys are generated on the CPU and never leave the
| enclave, but the data is decrypted on the CPU so it hits
| the registers in plaintext.
|
| When the enclave starts, the CPU does a few things:
|
| 1. The CPU does a key exchange with the GPU (in
| confidential compute mode [1]) to derive a key to encrypt
| data over PCIe
|
| 2. The CPU verifies the integrity of the GPU against
| NVIDIA's root of trust [2]
|
| [1] https://developer.nvidia.com/blog/confidential-
| computing-on-...
|
| [2] https://github.com/tinfoilsh/cvmimage/blob/b65ced8796e8
| a8687...
|
| edit: formatting
| candiddevmike wrote:
| You're not terminating the TLS connection from the client
| anywhere besides the enclave? How do you load balance or
| front end all of this effectively?
| FrasiertheLion wrote:
| >You're not terminating the TLS connection from the client
| anywhere besides the enclave?
|
| Yes.
|
| >How do you load balance or front end all of this
| effectively?
|
| We don't, atleast not yet. That's why all our model
| endpoints have different subdomains. In the next couple
| months, we're planning to generate a keypair inside the
| enclave using HPKE that will be used to encrypt the data,
| as I described in this comment:
| https://news.ycombinator.com/item?id=43996849
| danr4 wrote:
| great name. good idea if it works.
| leboshkibowl wrote:
| hasn't iexec (french co) been doing this for years? what's your
| competitive advantage or moat, considering they are the first-
| movers?
| ramoz wrote:
| Their GTM doesn't include a $ in front of their company
| acronym.
|
| I think there is similarity to https://www.anjuna.io/ and
| https://www.opaque.co/ here. I've heard of these, never iExec.
| FrasiertheLion wrote:
| CPU-based TEEs (AWS Nitro Enclaves, AMD SEV, Intel TDX) have
| been around for a few years, but aren't widely used because
| they are more akin to primitives than fully baked security
| solutions. We are trying to make this as user friendly and self
| serve as possible, with full verifiability by open sourcing the
| entire server that runs inside the enclave. So far we have not
| found any end to end verifiably private solution on the market
| that we could just sign up for to try, which was a big reason
| we started Tinfoil in the first place. We also strongly believe
| that verifiably private AI should be the norm, so the more
| players in the space, the better because a missing piece is
| market awareness and convincing folks this is actually possible
| and real.
| EGreg wrote:
| Been building something along these lines for a while. At
| Qbix, we call it our QBOX. Full stack, using Nix for the
| base, and Nitro attestation. No SSH. We have the exact same
| approach -- cron running and only downloading signed scripts
| and binaries from endpoints. But there is a lot more... Would
| be great to connect and maybe join forces.
|
| Want to connect some time next Tuesday or Wednesday?
| https://calendly.com/qbix/meeting
| FrasiertheLion wrote:
| Yes, excited to connect, scheduled a call! We used Nitro
| back in December when we were prototyping but moved to
| NVIDIA CC because we wanted to support LLMs.
| meelvidushi wrote:
| So impressive - cloud AI that is verifiable with zero trust
| assumptions is going to be game-changing regardless of the
| industry application. Looks like it could be used by anyone for
| making anything trustworthy.
| jMyles wrote:
| > with zero trust assumptions
|
| It's not that though. Not close. You are trusting the chip
| maker, whose process is secret (actually worse, it's almost
| certainly shared with the state).
| 3s wrote:
| We do have to trust the chip maker until open hardware
| catches up [1].
|
| [1] https://news.ycombinator.com/item?id=43997856
| davidczech wrote:
| Even if you had open hardware, how would you even know a
| chip you have sitting in front of you was fabricated
| correctly?
| julesdrean wrote:
| Check out incredible work by Bunnie to make this possible
| at home https://www.bunniestudios.com/blog/2024/iris-
| infra-red-in-si...
| Etheryte wrote:
| How large do you wager your moat to be? Confidential computing is
| something all major cloud providers either have or are about to
| have and from there it's a very small step to offer LLM-s under
| the same umbrella. First mover advantage is of course
| considerable, but I can't help but feel that this market will
| very quickly be swallowed by the hyperscalers.
| itsafarqueue wrote:
| Being gobbled by the hyperscalers may well be the plan.
| Reasonable bet.
| 3s wrote:
| Confidential computing as a technology will become (and should
| be) commoditized, so the value add comes down to security and
| UX. We don't want to be a confidential computing company, we
| want to use the right tool for the job of building private &
| verifiable AI. If that becomes FHE in a few years, then we will
| use that. We are starting with easy-to-use inference, but our
| goal of having any AI application be provably private
| ATechGuy wrote:
| This. Big tech providers already offer confidential inference
| today.
| julesdrean wrote:
| Yes Azure has! They have very different trust assumptions
| though. We wrote about this here
| https://tinfoil.sh/blog/2025-01-30-how-do-we-compare
| mnahkies wrote:
| Last I checked it was only Azure offering the Nvidia specific
| confidential compute extensions, I'm likely out of date - a
| quick Google was inconclusive.
|
| Have GCP and AWS started offering this for GPUs?
| candiddevmike wrote:
| GCP, yes: https://cloud.google.com/confidential-
| computing/confidential...
| julesdrean wrote:
| Azure and GCP offer Confidential VMs which removes trust
| from the cloud providers. We're trying to also remove
| trust in the service provider (aka ourselves). One
| example is that when you use Azure or GCP, by default,
| the service operator can SSH into the VM. We cannot SSH
| into our inference server and you can check that's true.
| threeseed wrote:
| Cloud providers aren't going to care too much about this.
|
| I have worked for many enterprise companies e.g. banks who are
| trialling AI and none of them have any use for something like
| this. Because the entire foundation of the IT industry is based
| on trusting the privacy and security policies of Azure, AWS and
| GCP. And in the decades since they've been around not heard of
| a single example of them breaking this.
|
| The proposition here is to tell a company that they can trust
| Azure with their banking websites, identity services and data
| engineering workloads but not for their model services. It just
| doesn't make any sense.
|
| Also you have the issue of smaller sized open source models
| e.g. DeepSeek R1 lagging far behind the bigger ones and so
| you're giving me some unnecessary privacy attestation at the
| expense of a model that will give me far better accuracy and
| performance.
| amanda99 wrote:
| Does this not require one to trust the hardware? I'm not an
| expert in hardware root of trust, etc, but if Intel (or whatever
| chip maker) decides to just sign code that doesn't do what they
| say it does (coerced or otherwise) or someone finds a vuln; would
| that not defeat the whole purpose?
|
| I'm not entirely sure this is different than "security by
| contract", except the contracts get bigger and have more
| technology around them?
| natesales wrote:
| We have to trust the hardware manufacturer (Intel/AMD/NVIDIA)
| designed their chips to execute the instructions we inspect, so
| we're assuming trust in vendor silicon either way.
|
| The real benefit of confidential computing is to extend that
| trust to the source code too (the inference server, OS,
| firmware).
|
| Maybe one day we'll have truly open hardware ;)
| ignoramous wrote:
| Hi Nate. Routinely your various networking-related FOSS
| tools. Surprising to see you now work in the AI
| infrastructure space let alone co-founding a startup funded
| by YC! Tinfoil looks uber neat. All the best (:
|
| > _Maybe one day we 'll have truly open hardware_
|
| At least the RoT/SE if nothing else: https://opentitan.org/
| julesdrean wrote:
| Love Open Titan! RISC-V all the way babe! The team is
| bunker: several of my labmates now work there
| rkagerer wrote:
| I agree, it's lifting trust to the manufacturer (which could
| still be an improvement over the cloud status quo).
|
| Another (IMO more likely) scenario is someone finds a hardware
| vulnerability (or leaked signing keys) that let's them achieve
| a similar outcome.
| max_ wrote:
| The only way to guarantee privacy in cloud computing is via
| homorphic encryption.
|
| This approach relies too much on trust.
|
| If you have data you are seriously sensitive about, its better
| for you to run models locally on air gapped instances.
|
| If you think this is an overkill, just see what happened to
| coinbase of recent. [0]
|
| [0]: https://www.cnbc.com/2025/05/15/coinbase-says-hackers-
| bribed...
| FrasiertheLion wrote:
| Yeah, totally agree with you. We would love to use FHE as soon
| as it's practical. And if you have the money and infra
| expertise to deploy air gapped LLMs locally, you should
| absolutely do that. We're trying to do the best we can with
| today's technology, in a way that is cheap and accessible to
| most people.
| sigmaisaletter wrote:
| Looks great. Not sure how big the market is between "need max
| privacy, need on-prem" and "don't care, just use what is
| cheap/popular" tho.
|
| Can you talk about how this relates to / is different / is
| differentiated from what Apple _claimed_ to do during their last
| WWDC? They called it "private cloud compute". (To be clear,
| after 11 months, this is still "announced", with no
| implementation anywhere, as far as I can see.)
|
| Here is their blog post on Apple Security, dated June 10:
| https://security.apple.com/blog/private-cloud-compute/
|
| EDIT: JUST found the tinfoil blog post on exactly this topic.
| https://tinfoil.sh/blog/2025-01-30-how-do-we-compare
| davidczech wrote:
| Private Cloud Compute has been in use since iOS 18 released.
| sigmaisaletter wrote:
| It seems that PCC indeed went live with 18.1 - tho not in
| Europe (which is where I am located). Thanks for the heads
| up, I will look into this further.
| DrBenCarson wrote:
| Private Cloud Compute has been live in production for 8 months
| sigmaisaletter wrote:
| It seems that PCC indeed went live with 18.1 - tho not in
| Europe (which is where I am located). Thanks for the heads
| up, I will look into this further.
| ts6000 wrote:
| Companies like Edgeless Systems have been building open-source
| confidential computing for cloud and AI for years, they are
| open-source, and have published in 2024 how they compare to
| Apple Private Cloud Compute.
| https://www.edgeless.systems/blog/apple-private-cloud-comput...
| gojomo wrote:
| Is there a frozen client that someone could audit for assurance,
| then repeatedly use with your TEE-hosted backend?
|
| If instead users must use your web-served client code each time,
| you could subtly alter that over time or per-user, in ways
| unlikely to be detected by casual users - who'd then again be
| required to trust you (Tinfoil), rather than the goal on only
| having to trust the design & chip-manufacturer.
| FrasiertheLion wrote:
| Yes, we have a customer who is indeed interested in having a
| frozen client for their app, which we're making possible. We
| currently have not frozen our client because we're in the early
| days and want to be able to iterate quickly on functionality.
| But happy to do so on a case-by-case basis for customers.
| ignoramous wrote:
| > _rather than the goal on only having to trust the design &
| chip-manufacturer_
|
| If you'd rather self-host, then the HazyResearch Lab at
| Stanford recently announced a FOSS e2ee implementation
| ("Minions") for Inference:
| https://hazyresearch.stanford.edu/blog/2025-05-12-security /
| https://github.com/HazyResearch/Minions
| computerbuster wrote:
| This is an incredibly robust solution to a really pressing
| problem for a lot of individuals/orgs who want to use/deploy
| reasonably powerful LLMs without paying through the nose for
| hardware. Others have mentioned the hyperscalers have solutions
| that make some amount of sense (Azure confidential computing, AWS
| nitro enclaves) but if you read a bit more about Tinfoil, it is
| clear they want to operate with far less explicit user trust (and
| thus much better security). This team is setting the standard for
| provably private LLM inference, and to me, it makes other
| solutions seem half-baked by comparison. Props to this talented
| group of people.
| offmycloud wrote:
| > https://docs.tinfoil.sh/verification/attestation-architectur...
|
| I tried taking a look at your documentation, but the site search
| is very slow and laggy in Firefox.
| 3s wrote:
| Interesting, we haven't noticed that (on Firefox as well).
| We'll look into it!
| offmycloud wrote:
| It looks like it might be the blur effect in a VM with no
| Firefox video acceleration. Also, email to support@tinfoil.sh
| (from "contact" link) just bounced back to me.
| FrasiertheLion wrote:
| Ah we don't have support@tinfoil.sh set up yet. Can you try
| contact@tinfoil.sh?
| SebP wrote:
| Thats impressive, congrats. You've taken the "verifiable
| security" concept to the next level. I'm working on a similar
| concept, without "verifiable" part... trust remains to be built,
| but adding RAG ad fine tuned modelds to the use of open source
| LLMs, deployed in the cloud: https://gptsafe.ai/
| japborst wrote:
| How do you see this compare to things like Amazon Bedrock, where
| it runs OSS in my own infra?
| FrasiertheLion wrote:
| Bedrock has strong contractual guarantees, but it's still only
| a legal contract and runs on AWS infra. This is certainly okay
| for many use cases, we're trying to build for users who want
| verifiable privacy guarantees beyond legal contracts.
|
| We're also doing more than pure inference, and trying to work
| with other companies who want to provide _their_ users
| additional verifiability and confidentiality guarantees by
| running their entire private data processing pipeline on our
| platform.
| binary132 wrote:
| I love the brand and logo
| ts6000 wrote:
| NVIDIA shared open-source solutions for confidential AI already
| in mid-2024 https://developer.nvidia.com/blog/advancing-security-
| for-lar...
| cuuupid wrote:
| This is a great concept but I think "Enterprise-Ready Security"
| and your competitive comparison chart are kind of misleading.
| Yes, zero trust is huge. But, virtually everyone who has a use
| case for max privacy AI, has that use case because of compliance
| and IP concerns. Enterprise-Ready Security doesn't mean sigstore
| or zero trust, it means you have both the security at a technical
| level as well as certification by an auditor that you do.
|
| You aren't enterprise ready because to address those concerns you
| need to get the laundry list of compliance certs: SOC 2:2, ISO
| 27k1/2 and 9k1, HIPPA, GDPR, CMMC, FedRAMP, NIST, etc.
| 3s wrote:
| We're going through the audit process for SOC2 right now and
| we're planning on doing HIPPA soon
| coolcase wrote:
| Tinfoil hat on: say you are compelled to execute a FISA warrant
| and access the LLM data, is it technically possible? What about
| an Australian or UK style "please add a backdoor".
|
| I see you have to trust NVidia etc. so maybe there are such
| backdoors.
| natesales wrote:
| An attacker would need to compromise our build pipeline to
| publish a backdoored VM image [1] and extract key material to
| forge an attestation from the hardware [2]. The build process
| publishes a hash of the code to Sigstore's transparency log
| [3], which would make the attack auditable.
|
| That said, a sufficiently resourced attacker wouldn't need to
| inject a backdoor at all. If the attacker already possesses the
| keys (e.g. the attacker IS the hardware manufacturer, or
| they've coerced the manufacturer to hand the keys over), then
| they would just need to gain access to the host server (which
| we control) to get access to the hypervisor, then use their
| keys to read memory or launch a new enclave with a forged
| attestation. We're planning on writing a much more detailed
| blog post about "how to hack ourselves" in the future.
|
| We actually plan to do an experiment at DEFCON, likely next
| year where we gives ssh access to a test machine running the
| enclave and have people try to exfiltrate data from inside the
| enclave while keeping the machine running.
|
| [1] https://github.com/tinfoilsh/cvmimage
|
| [2] https://arxiv.org/abs/2108.04575
|
| [3] https://github.com/tinfoilsh/cvmimage/attestations
| internetter wrote:
| > the client fetches a signed document from the enclave which
| includes a hash of the running code signed
|
| Why couldn't the enclave claim to be running an older hash?
| 3s wrote:
| This is enforced by the hardware (that's where the root of
| trust goes back to NVDIA+AMD). The hardware will only send back
| signed enclave hashes of the code it's running and cannot be
| coerced by us (or anyone else) into responding with a fake or
| old measurement.
| rkagerer wrote:
| What's your revenue model?
|
| The pricing page implies you're basically reselling access to
| confidential-wrapped AI instances.
|
| Since you rightly open-sourced the code (AGPL) is there anything
| stopping the cloud vendors from running and selling access to
| their own instances of your server-side magic?
|
| Is your secret sauce the tooling to spin up and manage instances
| and ease customer UX? Do you aim to attract an ecosystem of
| turnkey, confidential applications running on your platform?
|
| Do you envision an exit strategy that sells said secret sauce and
| customers to a cloud provider or confidential computing
| middleware provider?
|
| Ps. Congrats on the launch.
| FrasiertheLion wrote:
| >Since you rightly open-sourced the code (AGPL) is there
| anything stopping the cloud vendors from running and selling
| access to their own instances of your server-side magic?
|
| Sure they can do that. Despite being open source, CC-mode on
| GPUs is quite difficult to work with especially when you start
| thinking about secrets management, observability etc, so we'd
| actually like to work with smaller cloud providers who want to
| provide this as a service and become competitive with the big
| clouds.
|
| >Is your secret sauce the tooling to spin up and manage
| instances and ease customer UX?
|
| Pretty much. Confidential computing has been around a while,
| and we still don't see widespread adoption of it, largely
| because of the difficulty. If we're successful, we absolutely
| expect there to be a healthy ecosystem of competitors both
| cloud provider and startup.
|
| >Do you envision an exit strategy that sells that secret sauce
| to a cloud provider or confidential computing middleware
| provider?
|
| We're not really trying to be a confidential computing
| provider, but more so, a verifiably private layer for AI. Which
| means we will try to make integration points as seamless as
| possible. For inference, that meant OpenAI API compatible
| client SDKs, we will eventually do the same for training/post-
| training, or MCP/OpenAI Agents SDK, etc. We want our
| integration points to be closely compatible with existing
| pipelines.
___________________________________________________________________
(page generated 2025-05-15 23:00 UTC)