[HN Gopher] Gazette - Build platforms that flexibly mix SQL, bat...
       ___________________________________________________________________
        
       Gazette - Build platforms that flexibly mix SQL, batch, and stream
       processing
        
       Author : twooster
       Score  : 42 points
       Date   : 2022-07-16 10:28 UTC (12 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | akshayshah wrote:
       | The GitHub README and docs mention that Gazette has been running
       | in production for 5 years, but I don't see any mention of
       | _where_. I assume this began as an internal project at some
       | company - does anyone know which?
        
         | jaw0 wrote:
         | it came out of arbor.io (now part of liveramp.com)
        
           | dominotw wrote:
           | > liveramp.com
           | 
           | I am going to give this project a big no if it has anything
           | to do with with liveramp.
        
             | poulpi wrote:
             | Did you have a bad previous experience from their
             | engineering?
        
       | jimsparkman wrote:
       | The slides [1] helped me grok what problems this tool solves
       | pretty quickly.
       | 
       | Everything big data is moving to blob storage these days, but
       | streaming can lead to small files problem or longer latencies.
       | File fragments stored locally with proxied readers seems like a
       | simple solution to that.
       | 
       | [1] https://gazette.readthedocs.io/en/latest/overview-
       | slides.htm...
        
         | sitkack wrote:
         | Small files should be in arrow/avro/parquet if your
         | architecture allows (one should strive for this from the
         | beginning).
        
       ___________________________________________________________________
       (page generated 2022-07-16 23:01 UTC)