[HN Gopher] Show HN: UI for Apache Kafka
___________________________________________________________________
Show HN: UI for Apache Kafka
Author : germanosin
Score : 124 points
Date : 2021-10-01 13:19 UTC (9 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| dillon wrote:
| Where I work we use Confluent Cloud who has their own proprietary
| UI. I've always found it to be lacking, hard to use and not very
| good. We substituted it with AKHQ https://akhq.io/ , which is
| miles ahead of anything I've seen. The main issue with it are the
| interesting UX decisions that requires learning. For example, a
| lot of links require a double click, which isn't a common
| behavior in Web Apps. Besides that, it's absolutely wonderful and
| goes beyond just Kafka. We use Kafka Connect very heavily and
| AKHQ seems to implement the full CRUD for Connect.
| Leshe4ka wrote:
| Nice product, very happy to see it like an open source
| marat_galiev wrote:
| Cool, we're using this in a couple projects, awesome tool!
| PhilippGille wrote:
| There are several alternatives posted here already, but after
| comparing several ones maybe a year ago I found Kowl very nice:
| Nice UI, backend written in Go (quick startup, low memory usage),
| viewing and filtering of messages, support for schema registry,
| comes with a Docker image, usable locally and Helm chart for
| Kubernetes.
|
| https://github.com/cloudhut/kowl
| smsm42 wrote:
| kowl is awesome. The only missing feature I've found is that
| when looking at groups, only lag is shown but not the actual
| position.
| dxxvi wrote:
| I'm using kafdrop but modify it a bit. Reasons: - the front page
| is very slow if there are a lot of topics (the kafka cluster I'm
| using has 6k topics). It's because kafdrop tries to get a lot of
| information of every topic even though it doesn't display them on
| the screen. I modified it like this: after display 6k topic
| names, I send ajax requests to get info (# of partitions, start &
| end offsets of each partition, number of messages in each
| partition) of 20 topics each. - kafdrop doesn't have the feature
| to send a message to a topic. I added a text area below each
| topic name so that I can type the mesaage there and send. - If
| there's a new message, kafdrop cannot tell me which partition
| that new message goes to. So I added this feature: suppose that a
| topic has 3 partitions with 3, 4 and 5 messages. If there are
| some new messages coming to that topic, I can refresh that topic
| in the UI and the UI will show something like partition 2: 3 -> 4
| messages. So I know that 1 new message goes to partition 2.
|
| I like the multi-cluster management feature of this project.
| Unfortunately I don't know react (angular + typescript is ok with
| me), so I'm wondering if I should try this one knowing that I
| cannot modify it. The reasons I modified kafdrop are also the
| features I like to see in a new Kafka UI.
| faxmeyourcode wrote:
| We use Kafdrop for viewing topics/messages/info and it's really
| nice. It's read-only though, so if you need to configure a topic
| or push a message you have to use something else.
|
| https://github.com/HomeAdvisor/Kafdrop
| andrenotgiant wrote:
| This is the more up-to-date repo:
| https://github.com/obsidiandynamics/kafdrop
|
| Kafdrop is great, especially if you're already running
| everything else in Docker, just add a few lines to your docker-
| compose and get a free web UI for
| troubleshooting/introspecting.
| db3pt0 wrote:
| A few pieces of feedback.
|
| Positive: - Multi-cluster support is a great
| feature! That's the main thing missing from Kowl, in my opinion
| - The UI feels clean - Message streaming feels smooth, and
| the search feature is a great addition. Also love the ability to
| see a message for a given partition/offset, and that JSON
| messages are nicely rendered as JSON!
|
| Negative/Feature Requests: - The CleanupPolicy
| logic doesn't handle "delete,compact" (I see bug number 829
| addressed a slightly related issue, maybe that logic could be
| expanded [1]) - When viewing Cluster -> Topic -> Consumers,
| it seems like far more is being loaded than just consumer groups
| for that particular topic. It takes much longer to load than,
| e.g., Kowl does for pulling the same information - Similar
| feedback for when clicking on an individual consumer group
| (example URL path: "/ui/clusters/dev/consumer-groups/my-group").
| It takes a very long time to load - If any permissions
| issues are encountered while loading the individual consumer
| group, the entire request fails. That wouldn't necessarily be an
| issue, but it is when you're loading all consumer groups, and not
| just the one requested - I don't see any docs on how to
| access a topic that is secured with certificates (Kowl's relevant
| config [2]) - If a schema registry uses a self-signed
| certificate (or one signed by a corporation's CA), there's no way
| to pass the certificate through a config or skip validation
| (Kowl's relevant config [3]). As it is, you get a 500 error in
| the API call, but the UI doesn't show an error
|
| Overall, it seems like a fair bit of thought went into this! But
| there are a few quirks that make it very tough to use in a
| corporate environment with (perhaps excessively) large Kafka
| clusters with hundreds/thousands of topics, or secured with
| internal certificates
|
| 1. https://github.com/provectus/kafka-ui/blob/master/kafka-
| ui-a...
|
| 2.
| https://github.com/cloudhut/kowl/blob/10776b81e45abfa716548d...
|
| 3.
| https://github.com/cloudhut/kowl/blob/10776b81e45abfa716548d...
| epberry wrote:
| Looks pretty nice. The codebase is half typescript and half Java?
| Guessing that is because Kafka is written in Java.
|
| Another question: how might this compare to the Confluent web
| GUI? Obviously this is open source which is great.
| germanosin wrote:
| Btw I have an article comparing it with confluent web GUI
| https://towardsdatascience.com/overview-of-ui-tools-for-moni...
| spenczar5 wrote:
| You should have mentioned that you work at Provectus in this
| article. Feels pretty sneaky to claim to do an impartial
| review, but provide no negative critique of the one made by
| your company...
| germanosin wrote:
| This is mentioned in my medium profile. If you think that
| my comparison is not objective feel free to mention it on
| comment there)
| tyingq wrote:
| For example, it seems a bit odd that all of your "no"
| answers in a feature matrix are labelled "roadmap", but
| every other tool just gets "no".
| agons wrote:
| > feel free to drive engagement on my post
| zorr wrote:
| At a first glance this seems to be an api written in Java using
| Spring Boot in a reactive style with Webflux. The api wraps the
| official Java Kafka client and some other libraries.
|
| The frontend is a React app in Typescript.
| germanosin wrote:
| Hi, yes exactly. Backend is on Java + spring boot webflux,
| because of Kafka most mature client and frontend react +
| typescript.
| stillbourne wrote:
| I use the vscode plugin. As a matter of fact I seem to be using
| vscode for more and more admin tasks including kube management.
| brastick wrote:
| Love it, just got on Kafka a few days ago
| spushkarev wrote:
| Thanks for posting. Does it support AWS MSK?
| germanosin wrote:
| Sure, This fully support AWS MSK including AWS IAM
| authentication, Glue schema registry and AWS Kafka Connect
| spushkarev wrote:
| Nice. Is there are documentation on how to set it up for MSK?
| germanosin wrote:
| You could start with documentation here
| https://github.com/provectus/kafka-
| ui/blob/master/guides/AWS.... Feel free to create an issue
| if you have any questions.
| Antwnis wrote:
| https://lenses.io/cloud/aws-msk/
| daniel-grigg wrote:
| Neat! Just be wary around licensing, we've been using a similar
| management tool for a couple years https://github.com/yahoo/CMAK
| which was originally Kafka manager but they were forced to rename
| it.
| anVlad11 wrote:
| There was a whole wave of renamings in the past two years, it
| looked like Apache Foundation just carpet bombed everything
| with "Kafka" in the name.
| qrpike wrote:
| What is the difference between this and
| https://github.com/yahoo/CMAK
| mrweasel wrote:
| There's a lot great and developing tools for Kafka right now.
| CMAK doesn't seem that active, and I believe it was one of the
| tools we quickly gave up on, because it was a little difficult
| to get going.
|
| Currently we're using Kowl, and we're looking into Cruise
| Control.
|
| One limiting factor is that many of these tools doesn't support
| authentication and RBAC, Kowl does, and that has been a
| requirement for at least some of our projects, where not
| everyone is allowed to view certain topics.
| germanosin wrote:
| Yahoo CMAK is mostly focused on ops, UI for Apache Kafka is
| mostly for devs, providing Kafka ecosystem integration and
| observability.
| burusha wrote:
| Oh, cool! We are already using this tool and happy to it here
| kkcorps wrote:
| https://lenses.io/ Kafka UI is pretty solid. Nice to see an open
| source alternative here.
| KptMarchewa wrote:
| Can't get over the amount of corporate memphis on that page.
| vosper wrote:
| > corporate memphis
|
| Links for the lazy. The GP page (https://lenses.io) certainly
| has a lot of good examples.
|
| https://en.wikipedia.org/wiki/Corporate_Memphis
|
| https://t-artmagazine.com/what-is-corporate-memphis-and-
| why-...
| barbarbar wrote:
| Haha thank you. Had no idea.
___________________________________________________________________
(page generated 2021-10-01 23:01 UTC)