[HN Gopher] The end of the road for Kafka-delta-ingest
       ___________________________________________________________________
        
       The end of the road for Kafka-delta-ingest
        
       Author : alex_hirner
       Score  : 12 points
       Date   : 2025-11-30 17:25 UTC (7 days ago)
        
 (HTM) web link (brokenco.de)
 (TXT) w3m dump (brokenco.de)
        
       | nevi-me wrote:
       | It's great to see my name mentioned here, cheered me up as I'm
       | not having a good weekend. It feels like a lifetime ago since I
       | got to work on this and parquet-rs.
       | 
       | Yeah, I agree with the conclusion on Kafka here. Kafka is a very
       | resource-heavy application that's worth replacing with leaner
       | options.
       | 
       | Oxbow seems very interesting, I presume it's designed to forward
       | data that trickles into some S3 bucket as parquet data?
        
       | raverbashing wrote:
       | > The big argument against kafka-delta-ingest was Apache Kafka.
       | If an organization has Kafka for other reasons, then kafka-delta-
       | ingest can be a useful "sidecar" process to persist data flowing
       | through Kafka. If however the organization is running Kafka just
       | for ingestion, there are cheaper options available. As the
       | organization evolved, the other consumers of Kafka drifted away,
       | driving the value proposition of kafka-delta-ingest lower and
       | lower.
       | 
       | Sounds like how most of Kafka stories end.
        
       | rmnclmnt wrote:
       | If only for the initiative driving the work behind delta-rs, I'd
       | say it was well worth it!
        
       | redwood wrote:
       | What about WarpStream? Isn't that purpose built for lower Kafka
       | ingest costs?
        
       ___________________________________________________________________
       (page generated 2025-12-07 23:01 UTC)