[HN Gopher] Scaling request logging with ClickHouse, Kafka, and ...
       ___________________________________________________________________
        
       Scaling request logging with ClickHouse, Kafka, and Vector
        
       Author : mjwhansen
       Score  : 91 points
       Date   : 2025-10-08 09:56 UTC (5 days ago)
        
 (HTM) web link (www.geocod.io)
 (TXT) w3m dump (www.geocod.io)
        
       | rozenmd wrote:
       | Great write-up!
       | 
       | I had a similar project back in August when I realised my DB's
       | performance (Postgres) was blocking me from implementing features
       | users commonly ask for (querying out to 30 days of historical
       | uptime data).
       | 
       | I was already blown away at the performance (200ms to query what
       | Postgres was doing in 500-600ms), but then I realized I hadn't
       | put an index on the Clickhouse table. Now the query returns in
       | 50-70ms, and that includes network time.
        
       | nasretdinov wrote:
       | BTW you could've used e.g. kittenhouse
       | (https://github.com/YuriyNasretdinov/kittenhouse, my fork) or
       | just a simpler buffer table, with 2 layers and a larger
       | aggregation period than in the example.
       | 
       | Alternatively, you could've used async insert functionality built
       | into ClickHouse:
       | https://clickhouse.com/docs/optimize/asynchronous-inserts . All
       | of these solutions are operationally simpler than Kafka + Vector,
       | although obviously it's all tradeoffs.
        
         | devmor wrote:
         | There were a lot of simpler options that came to mind while
         | reading through this, frankly.
         | 
         | But I imagine the writeup eschews myriad future concerns and
         | does not entirely illustrate the pressure and stress of trying
         | to solve such a high-scale problem.
         | 
         | Ultimately, going with a somewhat more complex solution that
         | involves additional architecture but has been tried and tested
         | by a 3rd party that you trust can sometimes be the more fitting
         | end result. Assurance often weighs more than simplicity, I
         | think.
        
           | nasretdinov wrote:
           | While kittenhouse is, unfortunately, abandonware (even though
           | you can still use it and it works), you can't say the same
           | about e.g. async inserts in ClickHouse: it's a very simple
           | and robust solution to tackle exactly the problem the PHP
           | (and some other languages') backends often face when trying
           | to use ClickHouse
        
       | frenchmajesty wrote:
       | Thanks for sharing I enjoyed reading this.
        
       | tlaverdure wrote:
       | Thanks for sharing. I really enjoyed the breakdown, and great to
       | see small tech companies helping each other out!
        
       | mperham wrote:
       | Seems weird not to use Redis as the buffering layer + minutely
       | cron job. Seems a lot simpler than installing Kafka + Vector.
        
         | SteveNuts wrote:
         | Vector is very simple to operate and (mostly) stateless, and
         | can handle buffering if you choose.
         | 
         | Kafka and Redis is a "pick your poison" IMO, scaling and
         | operating those have their own headaches.
        
       | albertgoeswoof wrote:
       | Currently at the millions stage with https://mailpace.com relying
       | mostly on Postgres
       | 
       | Tbh this terrifies me! We don't just have to log the requests but
       | also store the full emails for a few days, and they can be up to
       | 50 mib in total size.
       | 
       | But it will be exciting when we get there!
        
       | fnord77 wrote:
       | How does Clickhouse compare to Druid, Pinot or Star Tree?
        
         | jamesblonde wrote:
         | Here's a good performance study by OneHouse comparing
         | Clickhouse, StarRocks, Trino:
         | 
         | https://www.onehouse.ai/blog/apache-spark-vs-clickhouse-vs-p...
         | 
         | Druid is real-time analytics, similar to Clickhouse. StarRocks
         | is best at Joins - Clickhouse is not good for joins.
        
           | manish_gill wrote:
           | > Clickhouse is not good for joins
           | 
           | This is less and less true as time goes on tbh. 25.9
           | introduced Join Reordering as well -
           | https://clickhouse.com/blog/clickhouse-release-25-09
        
       | saisrirampur wrote:
       | Sai from ClickHouse here. Very compelling story! Really love your
       | emphasis on using the right tool for the right job - power of row
       | vs column stores.
       | 
       | We recently added a MySQL/MariaDB CDC connector in ClickPipes on
       | ClickHouse Cloud. This would have simplified your migration from
       | MariaDB.
       | 
       | https://clickhouse.com/docs/integrations/clickpipes/mysql
       | https://clickhouse.com/docs/integrations/clickpipes/mysql/so...
        
       ___________________________________________________________________
       (page generated 2025-10-13 23:00 UTC)