[HN Gopher] An introduction to zero-knowledge machine learning
___________________________________________________________________
An introduction to zero-knowledge machine learning
Author : sethbannon
Score : 75 points
Date : 2023-04-05 16:47 UTC (6 hours ago)
(HTM) web link (worldcoin.org)
(TXT) w3m dump (worldcoin.org)
| wslh wrote:
| Just posted another thread from an article of a16z and a Zcash
| tweet: https://news.ycombinator.com/item?id=35457720
| lukeschlather wrote:
| I'm very confused by the use case here, and this doesn't make
| sense to me:
|
| > A good example of this would be applying a machine learning
| model on some sensitive data where a user would be able to know
| the result of model inference on their data without revealing
| their input to any third party (e.g., in the medical industry).
|
| I don't get why I would care that the answer was generated
| specifically by GPT4. It sounds like they're billing this as some
| sort of "run a model on input with homomorphic encryption" but
| that doesn't really sound possible, and to the extent that it is
| I don't think you could ever convince me that the people managing
| the model on the GPU couldn't get access to both the plaintext
| input and plaintext output.
|
| The way to get this kind of security is both simple and hard:
| make models that can run on consumer hardware.
| ruuda wrote:
| > but that doesn't really sound possible
|
| It sounds almost too good to be true, but snarks enable a
| prover to convince a verifier in O(log n) time that a statement
| of size n is true. In fact many constructs enable this in O(1)
| verifier time (but the prover is quite slow). With zk-snarks,
| part of the statement can even be private: the proof reveals
| nothing about the input, yet it can convince a verifier.
|
| All of this is probabilistic and making some assumptions about
| the complexity of an adversary, but that is very normal in
| cryptography. We consider eddsa signatures secure, even though
| one could in theory find the private key by brute force. Snarks
| "convince" a verifier in the same manner: generating a proof of
| a false statement is computationally infeasible, but in
| principle not impossible.
| mxwsn wrote:
| An important use-case is federated learning, which Google, and
| many healthcare / pharmaceutical companies are very interested
| in. In federated learning, multiple companies or groups with
| their own private data come together to train a model jointly
| on all the private data, while keeping the data private. You
| need more than zero-knowledge proofs to actually do federated
| learning securely, but to my limited knowledge they are one
| tool in the toolbox that can be useful.
| reaperman wrote:
| > make models that can run on consumer hardware.
|
| This will not be hard at all in 10-20 years given the pace of
| semiconductor FLOPS per watt improvement.
| https://en.wikipedia.org/wiki/Koomey%27s_law
|
| The neural engine in the A16 bionic on the latest iPhones can
| perform 17 TOPS. The A100 is about 1250 TOPS. Both these
| performance metrics are _very_ subject to how you measure them,
| and I 'm absolutely not sure I'm comparing apples to bananas
| properly.
|
| However, we'd expect the iPhone has reached its maximum thermal
| load. So without increasing power use, it should match the A100
| in about 6 to 7 doublings, which would be about 11 years. In 20
| years the iPhone would be expected to reach the performance of
| approximately 1000 A100's.
|
| At which point anyone will be able to train a GPT-4 in their
| pocket in a matter of days.
| MacsHeadroom wrote:
| You're assuming no algorithmic enhancements and missing the
| currently happening shift from 16bit to 4bit operations which
| will soon give ML hardware a 4x improvement on top of
| everything else.
|
| We could be training GPT-4s in our pockets by the end of this
| decade.
| thehumanmeat wrote:
| One large use case of ML and ZKML is Verifiable Computing. You
| can have an IoT device be able to enforce an untrusted super
| computer to process it's data on a certain program in a correct
| manner.
| rasengan wrote:
| If Facebook releases Llama, and updated models thereafter, for
| purchase or as freeware, there will not really be as much need
| for this since everything will happen safely, locally, no?
|
| It would be cool to see Meta release a 7B parameter as shareware,
| and subsequent larger models for a fee.
|
| Edit: To be clear, I'm all for ZK, generally!
| satoshiwasme wrote:
| [dead]
| BiasRegularizer wrote:
| A 17 million parameter model (~Resnet50) takes more than 50s
| proof time. Is this on top of the inference time?
|
| I can see some niche applications for this system, but I am very
| skeptical it's ability to handle larger models (100M+) and the
| ability to and it's scalability when there are increased demand.
| iskander wrote:
| ZK is currently stuck with arithmetic circuit representations
| which are predictably very expensive to use as a representation
| for tensor data.
|
| The matrix based formulations are still limited and don't play
| nicely with the parts of ML models which go beyond simple
| matrix multiplication.
|
| I suspect someone will unify the two threads of research
| eventually, but it doesn't seem like it's there yet.
|
| (FHE ML is even further away)
| [deleted]
___________________________________________________________________
(page generated 2023-04-05 23:00 UTC)