[HN Gopher] Kafka without ZooKeeper
___________________________________________________________________
Kafka without ZooKeeper
Author : toomanybits
Score : 117 points
Date : 2021-03-30 20:35 UTC (2 hours ago)
(HTM) web link (www.confluent.io)
(TXT) w3m dump (www.confluent.io)
| CatDogIsGod wrote:
| Awesome!
| airhead969 wrote:
| IIRC, ZK would be more modern and cloud-friendly if it could
| self-assemble with a preshared passphrase alone. It's good
| technology otherwise, it's just a PITA to deploy, configure, and
| support.
| cmckn wrote:
| I really love a lot of the software under the Hadoop umbrella,
| but so much of it assumes a static deployment on bare metal
| hosts, it's a struggle to use it in "modern" setups (HBase, for
| example; I miss my old friend).
| autophagian wrote:
| One immediate downside to this: it is now likely that fewer and
| fewer people will be exposed to the absolutely excellent
| Zookeeper logo.
| colineartheta wrote:
| My, what big hands you have.
| jaytaylor wrote:
| See:
|
| https://en.wikipedia.org/wiki/Apache_ZooKeeper
|
| https://upload.wikimedia.org/wikipedia/en/thumb/8/81/Apache_...
| AdmiralAsshat wrote:
| Those are some impressively sized hands.
| ignoramous wrote:
| https://m.huffpost.com/us/entry/us_55c36ebce4b0923c12bbb2cd
| koolba wrote:
| Pretty sure he has gout too.
| temuze wrote:
| This is the ideal open source logo. You may not like it,
| but this is what peak performance looks like.
| jakebasile wrote:
| Every once in a while I learn of something on HN that really
| changes my life. This is one of those times. Thank you.
| timdorr wrote:
| Is there some backstory on this? It looks like it's modelled
| after a real person. Perhaps one of its creators?
| spicybright wrote:
| I did a google image search and wish I didn't.
| lotyrin wrote:
| Still better than the sudo logo.
| 0x008 wrote:
| Wait.. sudo has a logo? Wow, that logo is ... special.
| deveffort wrote:
| Sweet!! Thank you.
| cmckn wrote:
| Oh hell yeah! That's great news, tons of work went into this --
| props to the contributors!
|
| I _will_ take the opportunity to say that Kafka is kind of
| painful, with or without ZK. Check out NATS! [0]. It doesn 't
| solve all the same problems, but is so much easier to use (during
| development especially) and can do a lot of the same things.
|
| [0]: https://docs.nats.io/whats_new_20
| [deleted]
| vorpalhex wrote:
| NATS isn't actually using a log structure though, it's a
| streaming message broker with a different set of
| consistency/delivery promises.
| cmckn wrote:
| Correct; but I've seen many uses of Kafka that NATS could
| totally be used for. For example, load balancing across
| subscribers (use a NATS queue instead of a Kafka consumer
| group).
|
| NATS doesn't ever store messages persistently; but this might
| be fine for your application, and then you don't have to
| worry about setting 5 different config options to make sure
| Kafka actually frees up disk space like you expect it to ;)
|
| NATS also enables some unique patterns like request/reply via
| a "reply to" message header.
|
| Anyway, it's been a joy to use!
| simplecto wrote:
| Using Kafka only for ordered pub-sub is like ordering tap water
| at French Laundry.
| andyxor wrote:
| "The" French Laundry
| yes_but_no wrote:
| I believe NATS Streaming[0] and upcoming Nats Jetstream[1]
| could be more relevant to those who looking for a Kafka
| alternative. They offer persistent messages, at-least-once
| delivery similar to Kafka.
|
| [0] https://docs.nats.io/nats-streaming-concepts/intro [1]
| https://github.com/nats-io/jetstream
| foobiekr wrote:
| NATS is not a replacement for any use of kafka that I think is
| actually a good fit.
| bruth wrote:
| There was a more recent 2.2 release with durable streams [0].
| There is also a comparison page including Kafka [1].
|
| [0]: https://docs.nats.io/whats_new_22 [1]:
| https://docs.nats.io/compare-nats
| haolez wrote:
| Kafka is a pretty cool technology, but for every project that I
| work on, it's never used because it feels like it's overkill
| (costly and operation heavy). Maybe I should start looking for
| bigger projects :D
| keithnz wrote:
| yeah, really needs a use case that justifies it, I have a
| particular IoT backend where I made it pluggable between kafka
| and rabbitmq, ended up just using rabbitmq as it is simpler to
| work with / manage, and still not really pushing it in terms of
| performance with thousands of devices.
| toomanybits wrote:
| I think that's one of the main points. Now you can run it as a
| single process more like a traditional broker (although it's
| obviously still a log).
| cmason wrote:
| What do you use instead?
| NicoJuicy wrote:
| Nats
| unixhero wrote:
| What kind if projects so you work om? I am genuinely
| curious about use cases. Feel free to obfuscate.
| math wrote:
| You can tune Kafka down fairly well if you know what you're
| doing, but it's not optimised for that OOTB. Or just use
| Confluent Cloud, which is fully managed and scales down as low
| as you want (costs cents per Gb). Disclosure: work for
| Confluent.
| unixhero wrote:
| Thank you for disclosing and not disclaiming.
| dmlittle wrote:
| Haven't used it personally myself but I've heard it enough to
| remember it. Redpanda[1] aims to be a Kafka replacement without
| having to worry about Zookeeper or the JVM
|
| [1] https://vectorized.io/
| [deleted]
| timdorr wrote:
| https://vectorized.io/redpanda/ is a more useful link, since
| the main domain appears to have some JS errors right now.
___________________________________________________________________
(page generated 2021-03-30 23:00 UTC)