[HN Gopher] Open Riak - open, modern Riak fork
___________________________________________________________________
Open Riak - open, modern Riak fork
Author : amarsahinovic
Score : 81 points
Date : 2024-11-19 20:29 UTC (2 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| amarsahinovic wrote:
| Lightning Talk: Introducing OpenRiak - Nicholas Adams:
| https://www.youtube.com/watch?v=0GLBsBeM4Kc
| carterschonwald wrote:
| Cool! I never used it but really liked the engineers I met who
| worked at basho on risk. AFAIK, they basically had an engineering
| dream team until their last ceo had them go hard in certain
| directions that didn't pan out.
| sitkack wrote:
| Thanks!
| chadd wrote:
| I used Riak for a project back in 2012, the app that became the
| Whisper App, and as a huge Erlang fanboy, I was so excited about
| it.
|
| But it was incredibly unreliable at scale, and my colleague and I
| spent a week of sleepless nights under incredible personal and
| business pressure - as the servers got busier and busier -
| ripping it out.
|
| Still love vector clocks, though, and have fond memories of the
| Basho team presenting at Erlang Factory
| amanj41 wrote:
| Vector clocks are very cool. Having read through how they were
| initially used in Riak, I was blown away that such an
| implementation could scale. I guess this is why Cassandra took
| a different approach?
| tibbar wrote:
| Vector clocks are certainly cool but fundamentally premised
| on the idea of having multiple 'live' versions of a value at
| once. Amazon's original Dynamo paper required conflict
| resolution at the application level, which is a very strange
| framework to build applications on. (Notably DynamoDB has
| moved away from this, I believe to Last Write Wins.)
| Cassandra takes the latter approach by default as well, I
| believe.
| amanj41 wrote:
| yes there's that idiosyncrasy, as well as client ideally
| needing to read the previous clock from the DB before
| writing an update for that key unless it's ok with the
| write being viewed as concurrent. Plus the extra memory
| overhead to store the clocks in the client.
| vosper wrote:
| Who would this be for in 2024?
|
| I remember evaluating Riak back in 2011 or so for an analytics
| solution, but ended up going with a more traditional OLAP
| database that was a much better option.
|
| It's hard for me to imagine where Riak would be a good option
| given how many choices we have today for various data stores.
| felixgallo wrote:
| Riak isn't remotely like OLAP. What was your use case?
| ramon156 wrote:
| Think they meant OLTP
| sitkack wrote:
| It is a fault tolerant massively scalable key value store
| capable of handling hundreds of terabytes of data.
|
| What are these options you are thinking of?
|
| The only thing that comes to my mind is Aerospike and possibly
| ScyllaDB.
| EwanToo wrote:
| It's realistically for the handful (dozens at most?) of very
| large Riak implementations where it would be enormously
| expensive to rewrite the application running on top of it.
|
| For example, the UK NHS Spine messaging system which has been
| building on Riak for 10 years
|
| https://riak.com/posts/press/nhs-launches-upgraded-it-backbo...
| tibbar wrote:
| I've never met an engineering team that used Riak, but it is used
| heavily as an example technology in Kleppmann's 'Designing Data
| Intensive Applications'. (I would say, informally, it's usually
| the example of the "other way" as opposed to other more well-
| known databases.) This does make me wonder what became of it, why
| it didn't take off.
| encoderer wrote:
| Inscrutable erlang stack traces definitely played a part. They
| were horrible.
| macintux wrote:
| Speaking as a former tech evangelist/engineer at Basho, there
| were a few significant challenges.
|
| Riak is horribly unfriendly as a database: no SQL, it exposes
| eventual consistency directly to the developer, it's relatively
| slow, and Erlang is a fairly unusual language.
|
| While you _can_ run Riak on a single server, you'd have to
| really want to.
|
| Its strength is the ability to scale massively, but not many
| projects need that scale, and by the time you do, you're
| probably already using some friendlier database and you'd
| rather make that one work.
| binary132 wrote:
| I wonder if some of these issues could be addressed sanely in
| an extension to the functionality
| cmrdporcupine wrote:
| Thing is, Cassandra became and remained popular, with similar
| aspects (though in JVM instead of Erlang, so).
|
| Though it had a couple years head start when there really no
| other options for people wanting that kind of kit.
| btilly wrote:
| Back in 2011 I was working on a project that involved Riak.
| The difficulty and slowness for doing stuff corresponding to
| basic SQL operations was certainly a giant strike against it,
| and helped sink that project before it was released.
| anotherjesse wrote:
| https://howfuckedismydatabase.com/nosql/ this infamous comic
| is about riak
| red_hare wrote:
| My old team used Riak in production for time series data in a
| real-time system.
|
| Our code was in Clojure, and we just wrapped the Java client.
| The conflict resolution was a steep learning curve, but
| overall, it was kind of nice (coming from Mongo).
|
| But man, Clojure stack traces wrapping Java stack traces
| wrapping Erlang stack traces in a Kafka consumer... I wish that
| hell on no one.
| veyh wrote:
| We used Riak at $dayjob at around 2014-2017 (iirc). I don't
| exactly remember it fondly. It was slow and unreliable. You
| could make it freeze/crash with the wrong SOLR query. (I was
| pretty good at that...)
| masleeds wrote:
| The SOLR part has now been retired from the last few
| releases.
|
| Current development has been focused on improving the
| flexibility of secondary indexes. There was some funky stuff
| achieved by some users using overloaded 2i terms and
| distributed processing of regular expressions against those
| terms - the aim is now to make this more flexible to the
| modern developer using the language of projected attributes
| and filter expressions (ala DynamoDB). There's also some
| active work to both replicate-to and full-sync (i.e.
| reconcile with) external OpenSearch clusters.
|
| The primary goal for OpenRiak is stability under load/failure
| as a K/V store - so the ultra-flexibility of in-built SOLR
| querying has been sacrificed in the move towards that aim.
| Anything that can do harm is to be offloaded or constrained.
| 0xbadcafebee wrote:
| I worked on a team that built a massive, high-performance
| internal service based on Riak. There are many things I learned
| from that system. Here is the best takeaway I can offer:
|
| It does not matter what your technology is, or how
| theoretically superior it is. Getting it to actually work well
| "in production" is a whole separate _thing_ than simply
| designing it and writing code. When it 's a very small system,
| it will look like it's doing great. As it gets bigger, the
| seams will start to burst, and you will find out that promises
| and theory don't always match reality.
|
| In the end, while its aims are great, it takes a whoooooole lot
| of work to smooth out the bumps in such a system. You need
| experts in that technology to address bugs in a timely manner.
| You need developers versed in the system to properly build apps
| utilizing it. You need competent operators to build,
| orchestrate, operate and maintain the whole thing.
|
| All of that is made easier by using simple technology that
| everybody knows, that there's a huge support community for,
| professional services for, etc. A technology like MySQL or
| Postgres etc, has the corporate, development, support, etc to
| make it easy to work with at any scale. A little janky at
| times, limited, but dependable, predictable, controllable.
|
| A small bespoke system with a small support community and
| virtually no corporate support is, comparatively, a hell of a
| lot more difficult/costly to support and harder to make work
| reliably.
| bojo wrote:
| I'm pretty sure Stripe was a heavy user of this for a while.
| They used it due to their write-heavy system, if I recall.
|
| I fondly remember writing a Go driver for it. Was a good
| experience: https://github.com/riaken/riaken-core
| isoos wrote:
| Basho team was very kind to open source contributions in
| ~2011-12: I've written an open source Riak client in Dart, and
| they had sent me t-shirts (the quality ones that are rare today).
| Nice treats for a fun project :)
| binary132 wrote:
| It's actually kinda silly how exciting this is to me
| masleeds wrote:
| Riak has been maintained through the post-basho years by
| engineers at some of its larger customers (disclaimer - including
| myself).
|
| The focus has been on trying to improve the stability of the
| database when subject to complex failure scenarios under
| stressful load, with minimal need for urgent operator
| intervention. The focus has been on keeping those existing
| operators happy rather than seeking out new users. Evolution of
| the product since basho has been slow but significant.
|
| The project now has support from Erlang Ecosystem Foundation, and
| we're looking to invest some effort over the next few months
| explaining what we've done, and to start to articulate what we
| see as the future for Riak. So if you're interested watch this
| space.
|
| It is expected to remain a niche product though. However, it may
| still find a home for those demanding specific non-functional
| requirements, with an acceptance of some functional constraints.
| freerobby wrote:
| I led a migration from Mongo to Riak at Shareaholic about 12
| years ago: https://www.slideshare.net/slideshow/migrating-to-
| riak-at-sh...
|
| It was successful at first, but ultimately we traded one set of
| problems for another (how novel, I know).
|
| In particular, I underestimated the pain of troubleshooting the
| database itself. Riak was a new product, we were a small team
| that had never run anything on BEAM, and ultimately we lost too
| many days debugging and trying to make sense of Erlang
| stacktraces.
|
| The Basho folks were great, and to this day I appreciate how
| quickly they fixed a number of bugs for us. But ultimately it
| wasn't enough -- we found problems faster than they could be
| patched.
___________________________________________________________________
(page generated 2024-11-19 23:00 UTC)