[HN Gopher] Olric: Distributed, embeddable data structures in Go
___________________________________________________________________
Olric: Distributed, embeddable data structures in Go
Author : mastabadtomm
Score : 105 points
Date : 2022-11-27 11:03 UTC (11 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| kevml wrote:
| > Olric v0.4 and previous versions use Olric Binary Protocol,
| v0.5 uses Redis serialization protocol for communication and the
| API was significantly changed. Olric v0.4.x tree is going to
| receive bug fixes and security updates forever, but I would
| recommend considering an upgrade to the new version.
|
| Seems like a tall order to promise updates "forever",
| particularly for a v0 release.
| Philip-J-Fry wrote:
| But considering there will be no features added to it, it's
| probably not a lot of effort to fix a few bugs or security
| issues here and there when people report them.
| didip wrote:
| Relevant discussion when it comes to Redis compatible API:
| https://news.ycombinator.com/item?id=31796311
| abusaidm wrote:
| I was excited to read its embeddable data structure then found it
| did Redis serialisation. I was hoping for something close to
| plasma store given that one isn't getting much love these days.
| Would be great to get something that can save binary formats like
| arrow.
| pgwhalen wrote:
| Ultimately Redis just stores bytes, there's no reason you
| couldn't put arrow formatted data in it (or in Olric, as I
| understand it).
|
| I do wish plasma wasn't dead/dying though. There is a huge gap
| in the (extremely crowded) key-value store space for zero-copy
| shared memory IPC. I assume it doesn't get built because it's
| not very "cloud native," but such a thing would be undeniably
| useful.
| alexk307 wrote:
| I don't understand why people need another abstraction layer on
| top of Redis? What can this do that Redis can't?
| delijati wrote:
| Two to X programes (any language that has redis client) can
| access the same memory via redis protocol. Without a running
| redis server. Or i missed something !?
| kamikazechaser wrote:
| Can run as both embedded and server mode. Essentially a re-
| imagined redis from a general point of view.
| resoluteteeth wrote:
| I don't think this is an abstraction layer on top of Redis? It
| appears to be a reimplementation of Redis in go.
| ozirus wrote:
| olric. Great name :)
| undefuser wrote:
| While it sounds interesting, I'm not sure in which scenario do I
| pick this over Redis
| konart wrote:
| Well, you can use it as a package in your Go service(s) if you
| don't want to or can't communicate with Redis
| bpicolo wrote:
| I know Java and Erlang ecosystems have been big on this at times
| (Hazelcast, mnesia). When are these the appropriate architectural
| choice, if ever?
|
| Definitely fun/cool tech, I just don't understand how pragmatic
| it is.
| jlouis wrote:
| In Erlang, mnesia has a couple of things going for it:
|
| One, it stores Erlang terms. The consequence is that you need
| no data conversion when you want to store data persistently on
| disk. You just save whatever term you have.
|
| Two, it provides Key/Value lookup at memory read speeds. It
| also does so across Erlang processes, so you don't have to call
| into another process to grab data in most cases. This was
| originally a main design constraint. You need it for low
| latency operation, and the only way to avoid a disk read is to
| keep things in memory. Nowadays, since we have fast disks with
| no seek-time, it matters less. But when Mnesia was built, we
| didn't have that luxury. Note that many other systems run in
| separate processes (memcached, redis, ...) which incurs a
| context switch and data transfer across a OS process boundary.
| When the K/V store is in the same memory space things are much
| faster.
|
| Three, it is an excellent tool for rapid prototyping, and early
| development work. You can easily buy two terabytes of memory
| nowadays. Hence, scaling memory is relatively easy (save the
| time it takes to make that data hot in memory). This means your
| first million customers are easily served from memory, and
| it'll take a while before you reach a million customers. Mnesia
| also allows you to plug in other storage backends, so you can
| eventually move to a more traditional setup later.
|
| Four, mnesia does provide query capability through a query
| planner. So for the occasional larger query, there are tools
| built-in which solves your problem in a very small amount of
| code.
| dmitriid wrote:
| > You can easily buy two terabytes of memory nowadays. Hence,
| scaling memory is relatively easy (save the time it takes to
| make that data hot in memory). This means your first million
| customers are easily served from memory, and it'll take a
| while before you reach a million customers.
|
| However, by this time your entire business depends on
| submillisecond responses from Mnesia in memory and you can't
| transition to a different database, or even to on-disk
| Mnesia.
|
| This has happened :)
| makapuf wrote:
| That and your app depend on 2TB being all in memory NOW so
| after one error you have to reload it all from disk before
| serving any request, because you had full fast backup
| right?
| Kinrany wrote:
| Libraries simplify ops compared to external services, including
| local development. An embedded database can be used in unit
| tests with zero effort.
| lokar wrote:
| I don't understand why anyone would pick the redis wire protocol.
| kgeist wrote:
| >So Olric has client implementations in all major programming
| languages.
| jensneuse wrote:
| Congrats Burak on making it to the main page!
| mastabadtomm wrote:
| Thank you, Jens!
___________________________________________________________________
(page generated 2022-11-27 23:01 UTC)