[HN Gopher] Sparkey is a simple constant key/value storage library
       ___________________________________________________________________
        
       Sparkey is a simple constant key/value storage library
        
       Author : mooreds
       Score  : 20 points
       Date   : 2024-01-04 14:31 UTC (8 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | vitiral wrote:
       | Hey, this is effectively the same idea I'm going to implement!
       | 
       | https://github.com/civboot/civboot/blob/main/blog/0012-dev-l...
       | 
       | Mine will be slightly more than key/value. My index file will
       | support indexes of arbitrary record fields as well as the index
       | of all items in loam, but it's effectively the same idea.
       | 
       | Note you could keep a live instance running to handle live
       | writes, which uses an in-memory hash for recent writes and
       | updates the hash index as-needed. Because the index can be
       | rebuilt from the data there is no concern about data loss.
        
       | jitl wrote:
       | At Airbnb we implementing translation serving on top of Sparkey;
       | part of our Ruby system open sourced here:
       | https://github.com/airbnb/hammerspace
       | 
       | The way the system worked (iirc it's been a while): we'd
       | periodically dump all the translation strings from our
       | translation editor app (MySQL backed) into a sparkey S3 path, and
       | use a Kubernetes daemonset to fetch the sparkey file to each host
       | in the cluster. Finally application pods written in any language
       | use a host volume to read translation strings from the local
       | sparkey file. Certainly overkill for smaller apps but with a
       | bunch of services and hundreds of thousands of translation
       | strings, the out of band updates are well worth it and keep the
       | app container small.
        
         | igor47 wrote:
         | Ah yes! For the Rails monolith the sparkey integration was
         | accomplished through https://github.com/airbnb/hammerspace
        
         | esprehn wrote:
         | IIRC many of Airbnb's systems were implemented on top of
         | sparkey: translations, feature flags, experiments, dynamic
         | config. It stood the test of time really well. Probably all
         | built by Jon Tai.
        
           | nurgasemetey wrote:
           | Very ingenious indeed. How would you deal with data
           | corruptions? How it was detected? Is repair automatic?
        
         | mooreds wrote:
         | Do you know if that system is still in use? The gem looks
         | abandoned.
        
       | KAdot wrote:
       | A similar idea implemented in Go
       | https://github.com/akrylysov/pogreb (disclaimer: I'm the author
       | of this package).
        
       ___________________________________________________________________
       (page generated 2024-01-04 23:01 UTC)