[HN Gopher] The hunt for a cluster-killer Erlang bug
       ___________________________________________________________________
        
       The hunt for a cluster-killer Erlang bug
        
       Author : eproxus
       Score  : 53 points
       Date   : 2022-06-14 21:05 UTC (1 hours ago)
        
 (HTM) web link (engineering.klarna.com)
 (TXT) w3m dump (engineering.klarna.com)
        
       | tiffanyh wrote:
       | Fantastic detailed write up. Wish there was more of these style
       | of articles on HN.
        
       | banashark wrote:
       | Very interesting writeup. Distributed systems problem solving is
       | always a very interesting process. It very frequently uncovers
       | areas ripe for instrumentation improvement.
       | 
       | The Erlang Ecosystem seemed very mature and iterated. It almost
       | seemed like the "rails of distributed system" with things like
       | Mnesia.
       | 
       | The one downside to that seemed to be that while I was working on
       | grokking the system, the limits and observability of some of
       | these built-in solutions was not so clear. What happens when a
       | mailbox exceeds it's limit? Does the data get dropped? Or, how to
       | recover from a network segmentation? These proved somewhat
       | challenging to reproduce and troubleshoot (as distributed
       | problems can be).
       | 
       | There are answers for all of these interesting scenarios, but in
       | some cases it almost would have been simpler to use an external
       | technology (redis/etc) with established
       | scalability/observability.
       | 
       | I do say this knowing that there was plenty I did not get time to
       | learn about the ecosystem in the depth that I desired, but was
       | curious how more experienced Erlang engineers viewed the problem.
        
       | davidw wrote:
       | > So our initial 1 GB binary data pretty printed as a string will
       | take about 1 GB x 3.57 characters/byte x 2 words/character x 8
       | bytes/word = 57.12 GB memory.
       | 
       | Yeah, I saw that one in an Erlang system too. It was pretty ugly.
        
       ___________________________________________________________________
       (page generated 2022-06-14 23:00 UTC)