[HN Gopher] Haydex: From Zero to 178.6B rows a second in 30 days
       ___________________________________________________________________
        
       Haydex: From Zero to 178.6B rows a second in 30 days
        
       Author : pdubroy
       Score  : 49 points
       Date   : 2025-09-25 18:07 UTC (3 days ago)
        
 (HTM) web link (axiom.co)
 (TXT) w3m dump (axiom.co)
        
       | dmitrygr wrote:
       | 178.6e9rows/s/30days = 66150rows/s^2
        
       | alexfromapex wrote:
       | I usually just call it 178 billion
        
         | gmueckl wrote:
         | But only if your billion is 10^9, not 10^12.
        
       | ccleve wrote:
       | 178 billion? That's nothing. I did trillions just this morning. I
       | went to the grocery store and picked an item off the shelf,
       | effectively filtering out the trillions of other products that I
       | could have picked but didn't.
       | 
       | They did not process 178 billion rows per second. They did a
       | search that found something in a large data set by eliminating
       | the parts of the data set that could not have contained the item.
       | Same way I did by picking one grocery store and going straight to
       | the shelf.
        
         | sally_glance wrote:
         | Hm, if I understand their product correctly they are building a
         | DB and their filtering actually returns correct results.
         | 
         | So, the analogy doesn't really hold true unless you actually
         | have these trillions of alternate products stored in your brain
         | and manage to cite the matching subset on demand.
        
           | Sesse__ wrote:
           | Seemingly, their way of thinking goes roughly like this:
           | 
           | If I have 10 billion rows in an SQL database, with a UNIQUE
           | index, and do SELECT * FROM t1 WHERE pk=<number>, then I have
           | "processed" 10 billion rows.
           | 
           | If I do 10k of these queries per second, I have processed
           | 100T rows per second.
        
       | timhigins wrote:
       | This kind of reads like an action or war novel
        
         | twoodfin wrote:
         | As edited by ChatGPT...
        
           | rhaps0dy wrote:
           | Yeah, it's very clearly LLM-edited, but it's fun to read. The
           | LLM did a good job.
           | 
           | It's not just a tech blog post - it's a thriller. ;)
        
             | smartbit wrote:
             | Write an adventure where we implemented _Bloofi
             | Multidimensional Bloom Filters_ from this 2015 article
             | https://arxiv.org/abs/1501.01941. At the end mention the
             | second author, don't mention the algorithm nor that it is
             | based on the 1970 Bloom-filter algorithm
             | https://en.wikipedia.org/wiki/Bloom_filter. Make me the
             | main character that did all the hard work and caused our
             | customer to win. The adventure should be some 5000 words
             | long and use each of these words 20-30 times: data, axiom,
             | filter, query, hashcolumn, haydex.
        
               | llllm wrote:
               | Please share your prompt, was it 'tell HN I'm an asshole
               | while flexing my fake tech creds'?
        
       | jacquesm wrote:
       | That looks like a variation on a Bloom filter to me.
       | 
       | https://en.wikipedia.org/wiki/Bloom_filter
       | 
       | In the 80's or so when I thought I was being really clever I came
       | up with another variation on this and I recall being quite
       | annoyed when someone on HN pointed out (many years later) that
       | this was a staple of computing science for longer than that I had
       | been busy with computers. So much for having original thoughts...
        
         | tsenart wrote:
         | Author here, indeed a variation of bloom filters:
         | https://x.com/lemire/status/1971279371131646063
        
           | jacquesm wrote:
           | Ok. I have blocked X at the router level here since Elon went
           | certifiable so I can't read that link but I will happily take
           | your word for it.
        
             | 1000units wrote:
             | It's funny how this comment chain is about how names stick
             | to ideas in somewhat arbitrary ways, and you are using
             | "Elon" to explain a personal policy for information
             | grooming.
        
               | jacquesm wrote:
               | I think 'don't give your data to assholes' is a pretty
               | good policy, regardless of whether it is personal or
               | business.
        
         | teaearlgraycold wrote:
         | It does go to show that a huge number of inventions we consider
         | foundational are really just from a normal person being in the
         | right place at the right time. When a field is emerging there
         | is a lot of low hanging fruit you can get your name stamped
         | upon.
        
       | mrbluecoat wrote:
       | Same EventDB as https://github.com/ahri/eventdb or proprietary?
        
         | tsenart wrote:
         | Proprietary.
        
       | bluelightning2k wrote:
       | This was very well written. Rare to see such spark in a tech
       | write up.
        
       ___________________________________________________________________
       (page generated 2025-09-28 23:01 UTC)