[HN Gopher] End-to-End Encryption through Kafka
___________________________________________________________________
End-to-End Encryption through Kafka
Author : mattgreg
Score : 57 points
Date : 2021-08-04 12:14 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| gopalv wrote:
| Looking at the setup code here, this looks a lot like SSL
| reinvented over Kafka (instead of over TCP), which is not a bad
| thing, just the mechanism feels a lot like you've automated a
| tcpdump into the protocol by making the handshake packets
| persisted in the stream + committed to disks.
|
| This somewhat makes sense if you're going to buy Kafka from a
| cloud provider without the ability to trust them, but the trust
| here is not very useful beyond securing transport trust.
|
| E2e for Kafka usually shows up as a mechanism to encrypt each
| item differently, because that has a bigger use-case in it.
|
| Kafka being an immutable log, there is no way to go back and
| delete data for a customer who asks for a data deletion or
| legally mandated GDPR data removals.
|
| Being able to shred the keys is very useful if you have a
| destination or source attribute based keys - instead of purging
| data out of the kafka log you can throw away the key.
| mwadhwa wrote:
| Great observations!
|
| Data removal from Kafka for GDPR compliance is trivial in this
| setting. You just purge your keys. There are also key rotation
| and revocation protocols that are part of Ockam and we're
| making them simple to use from our libraries.
|
| The underlying cryptography in Ockam SecureChannels comes from
| the Noise Protocol Framework (created by one of the authors of
| Signal Messenger's protocols). The Noise Framework protocols
| have many published security proofs and is gaining adoption in
| multiple secure systems. We are proposing an alternative to
| SSL/TLS but this alternative is well proven/understood in terms
| of it security guarantees.
| bdamm wrote:
| Perfect Forward Secrecy is pretty standard in TLS handshakes
| these days, so the persistence of messages should not be an
| issue.
|
| This pattern of burying TLS inside of application payloads is
| popping up all over the place. People realize that securing the
| transport isn't the same as securing the application, which
| often hops over multiple transports.
| mattgreg wrote:
| "People realize that securing the transport isn't the same as
| securing the application, which often hops over multiple
| transports."
|
| Exactly right. Application layer trust, with the transport
| and security layers abstracted away. This is important, since
| app developers typically can't control the transports from
| end to end, or at scale.
| voz_ wrote:
| What are some use cases / problems this solves?
| mwadhwa wrote:
| Some examples would be: 1. Alice is the controller of a machine
| in a factory, Bob is a remote worker controlling the machine
| from home. (there could be thousands of Alice, one Bob)
|
| 2. Alice is an internal system in Enterprise Data Center 1, Bob
| is a Microsevice in the Cloud
|
| 3. Alice is health monitoring device, Bob is a remote doctor's
| tablet. (there could be 100s of thousands of Alice. On of Bob)
|
| 4. Alice is a connected car, Bob is a service in the cloud,
| that uses a third party vendor's managed Kafka to communicate
| with Alice and wants to guarantee secure software updates /
| machine learning model updates to Alice (there is a fleet of
| Alice).
|
| etc.
|
| Any use case where you're using third party
| services/intermediaries and want to keep the vulnerability
| surface of your application as small as possible.
| vatkhrt wrote:
| Does the threat model include malicious system software on the
| endpoints? Or slightly differently, have you looked at any
| mechanisms to provide encryption when in use.
| mwadhwa wrote:
| We have a little bit and are watching the development of
| techniques like computing on Homomorphically Encrypted data,
| Multi-party Computation and Zero Knowledge Proofs with great
| interest.
|
| For example, the credentials functionality in Ockam is using
| BBS+ signatures that enable selective disclosure of attested
| attributes. This has both privacy and bandwidth benefits when
| proving possession of a credential.
|
| But we feel end-to-end encrypted communication is a
| prerequisite to building systems that compute on encrypted
| data. So we started with a focus on making end-to-end easy.
| mwadhwa wrote:
| Also, about malicious software at the endpoint ... Ockam
| Vaults can be backed by Hardware and Vault implementations
| are pluggable. So Key Storage and Cryptographic Operations
| can be delegated off the primary the processor.
|
| So far we've built vault implementations for the Microchip
| ATEEC hardware which is popular in IoT circles. We plan to
| have Vaults for Cloud HSMs, Cloud Secret Stores, ARM
| TrustZone etc.
|
| That said, if an attacker in the primary processor of an
| endpoint. We could protect agains key theft (with vault) ...
| but its very hard (maybe impossible) to protect against the
| attacker using the key until they are kicked out somehow.
| unixhero wrote:
| What are some use cases for this?
| bdamm wrote:
| It is right at the top of the page. Lack of trust in the
| intermediaries. Which is often the correct level of trust.
| unixhero wrote:
| Yes it is very well described, as a meditation on
| architecture. But I still don't see what real world use cases
| are.
| mwadhwa wrote:
| Some examples would be:
|
| 1. Alice is the controller of a machine in a factory, Bob is a
| remote worker controlling the machine from home. (there could
| be thousands of Alice, one Bob)
|
| 2. Alice is an internal system in Enterprise Data Center 1, Bob
| is a Microsevice in the Cloud
|
| 3. Alice is health monitoring device, Bob is a remote doctor's
| tablet. (there could 100s of thousands of Alice. On of Bob)
|
| 4. Alice is a connected car, Bob is a service in the cloud,
| that uses a third party vendor's managed Kafka to communicate
| with Alice and wants to guarantee secure software updates /
| machine learning model updates to Alice (there is a fleet of
| Alice).
|
| etc.
|
| Any use case where you're using third party
| services/intermediaries and want to keep the vulnerability
| surface of your application as small as possible.
| jack2 wrote:
| Instead of bidirectional messages using two topics, could you use
| messages in a single topic encrypted with something like PGP,
| S/MIME or JWE?
___________________________________________________________________
(page generated 2021-08-05 23:02 UTC)