[HN Gopher] Erlang's not about lightweight processes and message...
___________________________________________________________________
Erlang's not about lightweight processes and message passing (2023)
Author : todsacerdoti
Score : 168 points
Date : 2025-04-11 15:50 UTC (7 hours ago)
(HTM) web link (stevana.github.io)
(TXT) w3m dump (stevana.github.io)
| whalesalad wrote:
| I disagree. Interfaces are a trivial concept that can get bolted-
| on to any language. Even in languages without an official
| interface construct, you can replicate them in the program space.
|
| The BEAM succeeds because you can run 1M processes on a single
| node, represent complex distributed state machines with ease, and
| restart portions of the system with zero downtime. Among many
| other things.
|
| I really don't think behaviors/interfaces is the most critical
| piece.
| hinkley wrote:
| I haven't used it enough to be able to say yet, but I believe
| the BEAM avoids part of the problem Ian Cooper (Where Did It
| All Go Wrong?) rediscovered, which is that microservices don't
| min-max the inter- versus intra-modular friction in systems.
|
| I would not say that Beam eliminates this problem in any way,
| but I do think it lowers the slope of the line. The self-
| consistent idioms and functionality, especially with
| deployment, auto recovery and load balancing, reduce the inter-
| module friction. It makes a system where 12 engineers can
| easily manage 30 endpoints, and your surface area can still
| follow a power law.
| rdtsc wrote:
| I see your point to a degree.
|
| That's kind of how Erlang is. At first, anything Erlang has,
| some other system has too:
|
| Isolated process heaps? - Just use OS processes
|
| Supervision trees? - Use kubernetes.
|
| Message passing? - Not a big deal, I can write two threads and
| a shared queue in Java.
|
| Hot code loading? - Java can do that too
|
| Low latency processing? - I can tune my LMAX disruptor to kick
| Erlang's butt any day.
|
| Now getting all that into one platform or library that's the
| main idea. OS processes are heavyweight. Running 2M of them on
| a server is not easy. You could use some green threads or
| promises but now you lost the isolated heap bit.
|
| You can use kubernetes to some degree but it does not do nested
| supervision trees well. I guess it would work, but now you have
| your code, and you have pods and controllers, and volumes and
| all the shit.
|
| You can do message passing with an "actors" libraries in many
| language. But you cannot do pattern matching on receive, and it
| doesn't transparently integrate with sending it across nodes to
| another thread.
|
| You can do hot code loading, but how do you deal with runtime
| data structure and state. Erlang is built around that:
| gen_servers since the state is immutable and explicit has
| callbacks to upgrade not just the code but the state itself.
| whalesalad wrote:
| > Isolated process heaps? - Just use OS processes
|
| OS procs are heavyweight. Erlang procs are ~2KB each. You can
| spin up millions on one box while traditional OS procs would
| melt your machine. Not even in the same league.
|
| > Supervision trees? - Use kubernetes.
|
| Comparing Kubernetes to Erlang supervision trees misses the
| mark. K8s is infrastructure that requires significant
| configuration and maintenance outside your codebase. Erlang's
| supervision is just code - it's part of your application
| logic. And those nested supervision trees? Good luck
| implementing that cleanly in K8s without a ton of custom
| work.
|
| > Message passing? - Not a big deal, I can write two threads
| and a shared queue in Java.
|
| Basic threads and queues don't compare to Erlang's
| sophisticated message passing. Pattern matching on receive
| makes for cleaner, more maintainable code, and the
| transparent distribution across nodes comes standard.
| Building equivalent functionality from scratch would require
| significantly more code and infrastructure.
|
| "Any sufficiently complicated concurrent program contains an
| ad-hoc, informally-specified, bug-ridden, slow implementation
| of half of Erlang."
|
| > Hot code loading? - Java can do that too
|
| Java can reload classes, but Erlang was designed for seamless
| evolution of running systems. The gen_server callbacks for
| upgrading state during hot code loading show how deeply this
| concept is integrated into Erlang's core.
| hackernudes wrote:
| It's funny reading your response because I think you
| skipped the second half of the comment! Your rebuttals to
| the rebuttals are pretty similar.
| whalesalad wrote:
| its friday my brain is cooked - you're right.
| rdtsc wrote:
| No worries, at all I had figured you'd notice eventually.
| But I essentially agreed with you was just saying that
| common perceptions and how Erlang managed to do well
| there.
| bmitc wrote:
| The BEAM's real special sauce is in its preemptive scheduler
| in that it is impossible for one process to take down the
| whole system, even if bad processes are eating up the entire
| CPU. This cannot be done in any other language.
| macintux wrote:
| Worth noting as a disclaimer for people reading this: this
| is only true so long as external interfaces are not used.
| As soon as the BEAM calls out to other binaries, you lose
| that guarantee.
| myth_drannon wrote:
| "In February 1998 Erlang was banned for new product development
| within Ericsson--the main reason for the ban was that Ericsson
| wanted to be a consumer of software technologies rather than a
| producer." - The creator of the language banned any use of it
| internally.
| vvpan wrote:
| But from the quote it seems that for reasons unrelated to the
| language itself?
| bluGill wrote:
| Right. Companies need to figure out what they are going to be
| good at and what you buy from someone else. There is no one
| correct answer, and often the best answer changes over time
| (both reasons that different companies will have different
| answers). My company architects have been debating what IPC
| framework we should adopt for longer than it would take me to
| just write one from scratch - but they are correct to debate
| this instead of writing one because adopting means we don't
| have to maintain it and eventually that should cost more than
| all the debate. (note I didn't say will, though I think it is
| likely)
| zdragnar wrote:
| Being a consumer rather than a producer of tech is strictly a
| business decision. There's significant cost to producing and
| maintaining a language, and Ericsson no longer wanted to pay
| the upkeep.
|
| That's not necessarily an indictment on the language itself.
| The alternative would have been to keep using it while also
| open sourcing it, but I'm guessing they just wanted to be able
| to hire cheaper C developers or whatever the flavor of the time
| was.
| adamkittelson wrote:
| It is _wildly_ disingenuous to just copy paste that line from
| wikipedia and not the rest of the paragraph.
|
| > In February 1998, Ericsson Radio Systems banned the in-house
| use of Erlang for new products, citing a preference for non-
| proprietary languages.[15] The ban caused Armstrong and others
| to make plans to leave Ericsson.[16] In March 1998 Ericsson
| announced the AXD301 switch,[8] containing over a million lines
| of Erlang and reported to achieve a high availability of nine
| "9"s.[17] In December 1998, the implementation of Erlang was
| open-sourced and most of the Erlang team resigned to form a new
| company, Bluetail AB.[8] Ericsson eventually relaxed the ban
| and re-hired Armstrong in 2004.
|
| - edit, poster was quoting a quote in the article, not
| wikipedia, the article is the one omitting the context
| debugnik wrote:
| No, the company the creators worked for. And six years later
| they hired Armstrong again and silently lifted the ban.
| aanet wrote:
| Thanks for posting this. What a fantastic post on the history or
| Erlang. Very illuminating (all of this is new to me).
| bcardarella wrote:
| The amazing thing about Erlang and the BEAM is it's depth of
| features. To the OP the Behaviour/Interface of Erlang is their
| biggest take away. For me I believe it is how you require far
| _far_ less development resources to build complex systems than
| you would require in any other language (provided comparable
| experience in both stacks). And for many the lightweight
| processes and programming model.
|
| OTP itself has _so much_ in it. We 've been working on compiling
| Elixir to run on iOS devices. Not only can we do that through the
| release process but through using the ei library provided in
| Erlang we can compile a Node in C that will interface with any
| other Erlang node over a typical distributed network as you would
| for Erlang, Elixir, Gleam, etc... furthermore there is a rpc
| library in Erlang where from C we can make function calls and
| interface with our Elixir application. Yes, the encoding/decoding
| has an overhead and FFI would be faster but we're still way
| within our latency budget and we got this stood up in a few days
| without even have heard of it before.
|
| The larger point here is that Erlang has been solving many of the
| problems that modern tech stacks are struggling with and it has
| solved for scale and implementation cost and it solved these
| problems decades ago. I know HN has a bit of a click-bait love
| relationship with Erlang/Elixir but it hasn't translated over to
| adoption and there are companies that are just burning money
| trying to do what you get out of the box for free with the Erlang
| stack.
| relistan wrote:
| C nodes are under appreciated. We have one (Cgo) for
| communicating between Go and Elixir services running in the
| same Kubernetes pod. The docs are also pretty good for Erlang
| and its C libs.
| agent281 wrote:
| > I know HN has a bit of a click-bait love relationship with
| Erlang/Elixir but it hasn't translated over to adoption and
| there are companies that are just burning money trying to do
| what you get out of the box for free with the Erlang stack.
|
| Do you or the community have a sense why that is?
| bsder wrote:
| Erlang looks weird--Prolog-base, tail recursive loops,
| extensive pattern matching.
|
| Also, a lot of the power of Erlang is the OTP (Open Telecom
| Platform) even more than Erlang, itself. You have to
| internalize those architectural decisions (expect crashes--do
| fast restart) to get the full power of Erlang.
|
| Elixir seems like it has been finding more traction by
| looking more like mainstream languages. In addition,
| languages on the BEAM (like Elixir) made the BEAM much better
| documented, understood and portable.
| hosh wrote:
| Eventually, you use Elixir enough, and Erlang starts
| looking pretty.
| solid_fuel wrote:
| My personal opinion as a fan and adopter of the stack is that
| the benefit is often seen down the line, with the upfront
| adoption cost being roughly the same.
|
| E.g. the built in telemetry system is fantastic, but when you
| are first adopting the stack it still takes a day or two to
| read the docs and get events flowing into - say - DataDog,
| which is roughly the same amount of time as basically every
| other solution.
|
| The benefit of Elixir here is that the telemetry stack is
| very standardized across Elixir projects and libraries, and
| there are fewer moving pieces - no extra microservices or
| docker containers to ship with everything else. But that
| benefit comes 2 years down the line when you need to change
| the telemetry system.
| jacobsenscott wrote:
| These incremental benefits don't translate to an order of
| magnitude more productivity, or stability, or profitability.
| Given the choice, as a business owner, future proofing is
| about being able to draw from the most plentiful and cheapest
| pool of workers. The sausage all looks the same on the
| outside.
| hosh wrote:
| That is not true, especially with Section 174 (for the US).
| Right now, if you want to hire an Elixir engineer, you're
| better off finding a generalist willing to learn and use
| Elixir, and you would probably get someone who is very
| capable.
|
| With Section 174 in play in the US, it tends to drive
| companies hiring specialists and attempting to use AI for
| the rest of it.
|
| My own experience is that ... I don't really want to draw
| from the most plentiful and cheapest pool of workers. I've
| seen the kind of tech that produces. You basically have a
| small handful of software engineers carrying the rest.
|
| Elixir itself is a kind of secret, unfair advantage for
| tech startups that uses it.
| throwawaymaths wrote:
| Elixir is "bad" because it is not a friendly language for
| people who want to be architecture astronauts at the code
| level (you can definitely be an architecture astronaut at the
| process management level but that's a very advanced concept).
| And a lot of CTOs are architecture astronauts.
| hosh wrote:
| That's the opposite of my experience. I tend to get those
| "architect astronauts" in teams using other languages
| platforms, and the folks I work with Erlang or Elixir tend
| to be pragmatic and willing to dig down the stack to
| troubleshoot problems.
| throwawaymaths wrote:
| That's what I wrote! (Read the first three words with a
| heap of sardonicism). Edited to add quotes around bad
| hosh wrote:
| I went from a company that used Elixir in the backend to one
| that uses Nodejs.
|
| I had gone in neutral about Nodejs, having never really used it
| much.
|
| These projects I worked on were backend data pipeline that did
| not even process that much data. And yet somehow, it was
| incredibly difficult to isolate exactly the main bug. Along the
| way, I found out all sorts of things about Nodejs and when I
| compare it with Elixir/Erlang/OTP, I came to the conclusion
| that Node.js is unreliable by design.
|
| Don't get me wrong. I've done a lot of Ruby work before, and
| I've messed with Python. Many current-generation language
| platforms are struggling with building reliable distributed
| systems, things that the BEAM VM and OTP platform had already
| figured out.
| jerf wrote:
| "This begs the question: why aren't language and library
| designers stealing the structure behind Erlang's behaviours,
| rather than copying the ideas of lightweight processes and
| message passing?"
|
| Because the function signatures of Erlang's behaviors are
| critically tied to Erlang's other functionality, specifically its
| unusual use of immutability. You need a separate init call for
| its servers because of that, and a very distinct use of the state
| management to work exactly the same way.
|
| But to _achieve the same goals_ in other languages, you almost
| always _shouldn 't_ directly copy what Erlang is doing. In fact
| when I see "Look! I ported gen_server into $SOME_OTHER_LANGUAGE"
| and I see exactly and precisely the exact interface Erlang has, I
| know that the port doesn't deeply understand what Erlang is
| doing.
|
| When I ported the idea of supervisor trees into Go [1], I did so
| _idiomatically_. It turns out in modern Go the correct interface
| for "a thing that can be supervised" is not precisely the same
| signature that Erlang has, but type Service
| interface { Serve(context.Context) }
|
| That's all you need and all you should use... _in Go_. Your other
| language may vary. Go doesn 't need a "handle_event/2" because it
| has channels, and you should use those, not because they are
| "better" or "worse" but because _that 's what this language
| does_. In another language you may use something else. In another
| _infrastructure_ you may end up sending things over Kafka or some
| cloud event bus rather than "calling a handle_event/2". The key
| is in building an event-based system, not copying the exact
| implementation Erlang has.
|
| A peculiar issue the Erlang community has is getting excessively
| convinced that there's something super-mega-special about the
| _exact_ way Erlang does it, and that if you do it any other way
| it is _ipso facto_ wrong and therefore not reliable. This may
| have been true in 2005; it is not true in 2025. Where once Erlang
| had almost the only sensible answer, in 2025 the problem is
| poking through the ocean of answers deluging us! While I
| recommend learning from Erlang about reliable software, I
| _strongly_ recommend _against_ just blind-porting out the exact
| way Erlang achieves it into any other language. It is in almost
| any other language context the wrong answer. Even other immutable
| languages generally vary enough that they can 't just copy the
| same structure.
|
| [1]: https://jerf.org/iri/post/2930/
| asa400 wrote:
| To follow on from your excellent post, I think a reasonable
| next question is, "why have these kinds of approaches and ideas
| in other languages and systems succeeded in gaining market
| adoption, but Erlang/Elixir has not?"
|
| This to me is the most interesting question about Erlang, and I
| say this as someone who works professionally in Elixir.
|
| It's _clear_ that there is incredible appetite for tools that
| help us design reliable concurrent systems given the wild
| success of things like k8s, Kafka, AWS's distributed systems
| products, etc., but why hasn't Erlang/Elixir been able to
| capture that share?
|
| My friends and I debate this all the time, but I don't know the
| answer.
| bad_haircut72 wrote:
| Decision making in computing is mostly a matter of fashion /
| cargo-culting.
| matthew16550 wrote:
| And Resume padding
| ngrilly wrote:
| I think that's mostly due to Erlang looking too alien
| compared to mainstream languages. Elixir is changing that but
| it arrived a bit late.
| jerf wrote:
| "but why hasn't Erlang/Elixir been able to capture that
| share?"
|
| Because Erlang has a well-integrated collection of what are
| by 2025 standards mediocre tools.
|
| There is value to that integration, and I absolutely won't
| deny that.
|
| However, the state of the art has moved beyond Erlang in a
| number of ways, and you're taking a pretty big penalty to
| stick to BEAM on a number of fronts now. Its performance is
| sub-par, and if you're running a large cluster, that's
| actually going to matter. Erlang _qua_ Erlang I 'd call a
| subpar language, and Elixir _qua_ Elixir is merely
| competitive; there are many places to get similar
| capabilities, with a wide variety of other available cost
| /benefit choices. Erlang's message bus is not terribly
| resilient itself; modern message busses can be resilient
| against individual nodes in the message bus going down, and
| it's a powerful pattern to have multiple consumers against a
| single queue, which Erlang's focus on PIDs tends to inhibit.
| Erlang's message bus is 0-or-1 when as near as I can tell the
| rest of the world has decided, correctly IMHO, that 1-or-n is
| superior. Erlang is fairly insular; once you have to hook up
| _one_ non-BEAM service to the system, well, you 're going to
| do that over some sort of message bus or something, and you
| pretty quickly get to the point that you might as well let
| that be your core architecture rather than the BEAM cluster.
| Once you're heterogeneous, and BEAM is just another node on
| the net, there isn't necessarily a lot of reason to stay
| there. And as a system scales up, the pull to heterogeneity
| approaches infinity; takes a lot of work to go to an entire
| company and force them to work entirely in BEAM.
|
| Plus, some of the problems Erlang solved in one way have
| developed better solutions. Erlang solves the problem of
| multiple code bases possibly simultaneously existing in the
| same cluster by basically making everything untyped. That was
| a nifty solution for the 1990s, but today I think we've
| gotten a lot better at having typed data structures that
| still retain backwards compatibility if necessary. So
| throwing away the entire type system, including all the
| methods and inheritance or composition or whatever, to solve
| that problem is a heck of a blow.
|
| I do want to close out with a repetition of the fact that
| there _is_ value in that solid integration. More people today
| are aware of the various tools like "message busses", but it
| is still clearly not as common knowledge as I'd like and I
| still see entire teams struggling along basically crafting an
| ad-hoc half-specified custom message bus every so often,
| which in 2025 is insane. (I have written a couple of services
| where I have basically had to provide HTTP "REST" endpoints
| that end up just being proxies on to my internal message bus
| that my system is really based on, because they'd rather POST
| HTTP than have to use a message bus library, even though it
| doesn't really buy them anything.) Erlang does help educate
| people about what are now the basics of cloud architecture.
| And that "well-integrated collection of mediocre tools" can
| still solve a _lot_ of problems. Many sins can be forgiven by
| a 32 4GHz cores backed by high powered RAM, disk, and
| networking.
|
| But it would take a lot of backwards-incompatible changes to
| create a BEAM 2.0 that would be competitive on all fronts...
| if indeed such a thing is even possible. The variety of techs
| exist for a reason. It stinks to have to paw through them
| sometimes, but the upside is you'll often find the exact
| right solution for your needs.
| Thaxll wrote:
| "It's _clear_ that there is incredible appetite for tools
| that help us design reliable concurrent systems given the
| wild success of things like k8s, Kafka, AWS's distributed
| systems products, etc., but why hasn't Erlang/Elixir been
| able to capture that share?"
|
| Becasuse Erlang is a runtime + language and Kubernetes is a
| neutral platform. You can build concurrent and reliable
| solution without the need of locking you down to a single
| language.
|
| Someone can start by just porting its Python code on
| Kubernetes to make it more reliable and fault tolerent.
| hackingonempty wrote:
| Talk to some engineering managers. Their concerns are hiring
| people to get the job done. You can't easily hire devs for
| obscure languages like Erlang and Elixir. If you can find any
| that are looking for a gig they want too much money. On the
| contrary, if you are hiring for C++/C#/Java/JS/TS your
| problem is separating good from bad candidates but good ones
| are available.
|
| Likewise, most devs don't want to learn an obscure language
| for one job even if they are more than capable. Either they
| get stuck doing that language or they earn a hole in their
| resume instead of additional experience in what future
| employers care about.
|
| Finally, the vast majority of applications and systems don't
| need ultra high reliability and don't have the budget for it.
| It isn't clear that downtime impedes success for anything but
| the most critical businesses.
| int_19h wrote:
| > Likewise, most devs don't want to learn an obscure
| language for one job even if they are more than capable.
| Either they get stuck doing that language or they earn a
| hole in their resume instead of additional experience in
| what future employers care about.
|
| This is less of an issue with accumulated experience.
| Personally I would actually welcome the kind of job that
| would involve learning a new niche language, since I
| already have >10 years of experience in several mainstream
| languages, and there's diminishing returns wrt resumes and
| interviews past this point.
| klabb3 wrote:
| Go is my favorite language but:
|
| > Go doesn't need a "handle_event/2" because it has channels,
| and you should use those
|
| Of what type? But most importantly, channels are local to the
| process, so you need glue to make it networked. (I assume
| erlang has networked message handling abstracted away). In
| addition I've seen 3-4 different variations of your proposed
| pattern for long-running server like things.
|
| I agree fully that porting should make use of idiomatic
| constructs. But I also think languages can have hidden
| mechanics that loses the valuable essence while porting - a
| form of anti-relativism of PLs if you will.
|
| It's entirely possible to me that this "oh a channel? just wrap
| it in X" is much more detrimental to interop than what it
| sounds like. For instance take http.Handler in Go. Similarly
| simple but what are the real world implications of having it in
| std? An ecosystem of middleware that is largely compatible with
| one another, without pre-coordination (a non-std http server X
| can be used with auth middleware Y and logging middleware Z).
| Similar things can be said about io.Reader and friends. These
| extremely simply interfaces are arguably more valuable than the
| implementations.
|
| If, and I'm speculating here, Erlang got many of the interfaces
| for reliable distributed systems right, that can be what
| enables the _whole_.
| jerf wrote:
| "Of what type?"
|
| Of the type of the messages you're sending. Which can either
| be an interface for multiple messages, or you can use
| multiple channels with one type each. I've done both. This is
| not an important question when actually programming in Go.
|
| "But most importantly, channels are local to the process, so
| you need glue to make it networked."
|
| This _is_ an important consideration if you are using Go.
| Although I would observe that it isn 't so much that
| "channels don't do network" as that "channels are a local
| tool"; e.g., we do not complain that OS mutexes are not
| "network capable", because they're intrinsically local.
| Network locking uses different solutions, and we don't really
| consider etcd a competitor to a local "lock" call.
|
| But there are dozens of message busses in the world now, and
| Erlang's isn't really all that competitive modulo its
| integration.
| senderista wrote:
| For me the most interesting concept in Erlang/BEAM is that
| partial recovery is built in from the ground up. When an
| unexpected state is encountered, instead of either killing the
| entire process or trying to proceed and risking corruption, you
| just roll back to a known good state, at the most granular level
| possible. This idea was researched many years ago under the name
| of "microreboots"(associated with "crash-only software"), but
| only Erlang/BEAM made it a first-class concept in a production
| system.
| benmmurphy wrote:
| You still have to be careful with supervision trees and parts
| of the tree restarting. For example your system might work if
| the whole erlang operating system process is suddenly killed
| and restarted but your system might start corrupting data if
| parts of the erlang process tree is restarted. Erlang gives you
| a good model to work with these problems but it doesn't allow
| you to completely turn off your brain. If you walk in thinking
| that you can just let things restart and everything will be
| fine then you might end up getting burnt.
| juped wrote:
| Yes, you can design your system pathologically to make it
| wrong.
| groestl wrote:
| > When an unexpected state is encountered, instead of either
| killing the entire process or trying to proceed and risking
| corruption, you just roll back to a known good state, at the
| most granular level possible.
|
| > but only Erlang/BEAM made it a first-class concept in a
| production system.
|
| Exceptions?
| juped wrote:
| Can you explain how exceptions (partial stack unwinding while
| carrying a value) do this?
| Towaway69 wrote:
| I've just gotten back into Erlang becuase of the lightweight
| processes and message passing, so far behaviour has been
| secondary (i.e. just learning about them)!
|
| The project is about bring visual Flow Based Programming(FBP)[1]
| to Erlang. FBP seems to be made for Erlang and I was surprised
| there was something already but there does not seem to be.
|
| My goto tool for FBP is Node-RED and hence the basic idea is to
| bolt a Node-RED frontend on to an Erlang backend and to have
| every node being a process. Node-REDs frontend is great for
| modelling message passing between nodes, hence there is a very
| simply one-to-one mapping to Erlangs processes and messages.
|
| I've implemented some basics and started to create some unit
| tests as flows to slowly build up functionality. I would really
| like this to be 100% compatiable to Node-RED the NodeJS backend.
| For more details, the github repo -->
| https://github.com/gorenje/erlang-red
|
| Overall Erlang is amazingly well suited to this and astonished
| that no one else has done anything like this - or have they?
|
| [1] = https://jpaulm.github.io/fbp/index.html
| __jonas wrote:
| Oh that's really cool to see! I always thought a visual
| programming language on the BEAM would be fun
| hinkley wrote:
| I've worked with a few individuals, mostly managers, who intended
| to write books informed by our experiences. It was always
| frustrating for me to see that we disagreed about what aspects of
| our work made us successful. There was always something they
| minimized as being nice that I felt was essential.
|
| And here we see someone claiming that lightweight processes and
| message passing aren't the secret sauce, missing that Erlang as
| Communicating Sequential Processes is indivisible from those
| qualities, and then repeatedly mentioning CSP as part of the
| secret sauce.
|
| Examples:
|
| > The application programmer writes sequential code, all
| concurrency is hidden away in the behaviour;
|
| > Easier for new team members to get started: business logic is
| sequential, similar structure that they might have seen before
| elsewhere;
|
| > Supervisors and the "let it crash" philosophy, appear to
| produce reliable systems. Joe uses the Ericsson AXD301 telephone
| switch example again (p. 191):
|
| Behaviors are interesting and solve a commonly encountered
| problem in the 80's that was still being solved in some cases in
| the 00's, but it's a means as much as an end in Erlang. It's how
| they implemented those other qualities. But I don't know if they
| had to, to make Erlang still mostly be Erlang.
| sitkack wrote:
| Managers make up their own narrative based on vibes.
| silisili wrote:
| Is Erlang considered CSP? I've always thought it wasn't really,
| and had its own thing called 'actors' which are id'd and can
| communicate directly, vs CSP which are anonymous and use
| channel messaging.
|
| I've always thought the actor model made more sense, but highly
| YMMV.
| hinkley wrote:
| The erlang docs only go as far as saying it's functionally
| similar to CSP.
|
| I think the term Actor Model has been so semantically diluted
| at this point that the phrase also understates what Erlang
| has as well.
|
| Neither CSP nor AM require process isolation to work, which
| means they can work when they work but fail much much worse.
| They are necessary but insufficient.
| fidotron wrote:
| Erlang isn't CSP, it's the Actor model.
| https://en.wikipedia.org/wiki/Actor_model
|
| CSP is what inspired the golang channels, via occam and some
| other languages. The whole synchronization on unbuffered
| channels is the most obvious differentiator, though there are
| others like the actor concept of pattern matching over a
| mailbox.
|
| The whole CSP vs actor debate is quite interesting when you get
| down to it because they superficially look kind of similar but
| are radically different in implications.
| coolKid721 wrote:
| Watch the guy who came up with Erlang, the Actor model and
| CSP discuss it: https://youtu.be/37wFVVVZlVU One of my
| favorite videos on youtube.
| fidotron wrote:
| Totally! I love the Alan Kay and Armstrong one too
| https://www.youtube.com/watch?v=fhOHn9TClXY
| hinkley wrote:
| There are a lot of languages that now claim to be 'Actor
| Model' and have only a shade on Erlang's fault tolerance and
| load balancing. That term no longer has the gravitas it once
| had.
| Supersaiyan_IV wrote:
| "In February 1998 Erlang was banned for new product development
| within Ericsson"
|
| False statement. Ericsson still uses Erlang, for example in their
| MME. Source: I used to work at Ericsson.
| bee_rider wrote:
| Is there any additional context here? (Is this a common
| misperception that you've come across?)
| macintux wrote:
| Not the original commenter, but from hearing Joe Armstrong
| speak on multiple occasions my recollection is that Erlang
| was blacklisted for some length of time (which allowed it to
| become an open source language), but eventually the company
| realized it still offered significant value.
| 4ad wrote:
| It is simultaneously possible that Ericsson banned Erlang in
| 1998 (a statement claimed multiple times by the creators of
| Erlang) and that Ericsson rescinded the ban later in 2004, when
| they hired back Joe Armstrong.
| jesperwe wrote:
| And there is a small team of Ericsson full time devs working on
| developing the language itself and the BEAM.
| lysace wrote:
| My impression from Ericssonland:
|
| Around year 2008 being an Erlang coder was often more or less
| seen as being a COBOL coder in Sweden. Bluetail had sort of
| failed, having burned lots of VC, iirc.
|
| So Erlang was something weird and custom that Ericsson used to
| build software for legacy phone exchanges. I remember that a
| colleague's wife working at Ericsson had received on-the-job
| training from essentially zero programming knowledge to become
| an Erlang developer in order to maintain some phone exchange
| software.
|
| It's been fascinating to see it morph into something cool.
| Whatsapp, etc.
| whorleater wrote:
| Yeah, I don't know why this falsehood continues to persist.
| WhatsApp and Ericsson engineers continue to work together to
| evolve Erlang, alongside a bunch of other people across the
| industry.
|
| Source: I work at WhatsApp
| pessimizer wrote:
| It's not a falsehood. The confidence of working at Ericsson
| and WhatsApp seems to have kept you both from just looking it
| up.
| behnamoh wrote:
| Is it just me or does Erlang's syntax look a little bit nicer
| than Elixir's?
| Capricorn2481 wrote:
| I'm an outsider to this ecosystem, but I've seen a few people
| share that same opinion. They prefer the explicitness of
| Erlang.
| sodapopcan wrote:
| Elixir is still very explicit from a syntactic point of view.
| Macros allow hiding a significant amount of boilerplate in
| certain behaviours, though. So it's a matter of preference,
| of course.
| SoftTalker wrote:
| It's inspired/descended from Prolog, and my impression is that
| many people find it a bit odd. It is at first, but I quickly
| adjusted to it and quite like it now.
| whalesalad wrote:
| gleam is probably my favorite middle ground between elixir and
| erlang.
| bmitc wrote:
| Elixir came from Ruby developers and thus has similarly verbose
| syntax and macros. Erlang's syntax came from Prolog, which was
| used to implement the first compiler and is why Erlang's syntax
| is more concise.
| ValtteriL wrote:
| I learned Erlang at school and used to prefer its syntax for
| years. However, after giving Elixir a chance and writing 1000
| loc I was converted. Now I look at snippets of Erlang in docs
| with mild disgust.
| pton_xd wrote:
| Erlangs syntax takes a bit of getting used to but it's very
| pleasant to use once you're familiar with it. I like it a lot.
| nailer wrote:
| "In 1998 Ericsson decided to ban all use of Erlang. The people
| responsible for developing it argued that if they were going to
| ban it, then they might as well open source it. Which Ericsson
| did and shortly after most of the team that created Erlang quit
| and started their own company."
|
| Bwahahaha. Reminds me of the JRuby team, who left Sun as a single
| unit and resumed work as a team at another company (I can't
| remember where) when Oracle acquired Sun.
| hk__2 wrote:
| (2023)
| layer8 wrote:
| From this article and others, it's still unclear to me what the
| state-handling and state-sharing model of Erlang is. Presumably,
| the granularity of the crashing/restarting sequential processes
| is also the granularity of in-memory state sharing. But what
| about external state, like databases, queues, file systems? For
| example, if a process has taken an item off a queue and then
| crashes before having fully processed it, how is that accounted
| for? Or you might not even know from the outside if it has been
| fully, partially, or not at all processed yet. This is an example
| where correct error handling or not crashing is crucial, in my
| experience. Or what about processing pipelines where a component
| in the middle crashes. Is there something like that in Erlang? Is
| there an article explaining Erlang from that perspective?
| fidotron wrote:
| > For example, if a process has taken an item off a queue and
| then crashes before having fully processed it, how is that
| accounted for?
|
| I have worked with people that had deployed huge amounts on the
| BEAM that had a real problem with the answer to that, and
| resort to magical thinking.
|
| When erlang processes "crash", assuming the whole system didn't
| crash, they almost certainly alerted a monitoring process of
| the fact, so that a process can be quickly restarted. This is
| the core of how supervision trees in erlang are built.
|
| There are a lot of subtleties to that. The whole system may or
| may not be a single BEAM instance, and if more than one then
| they can be distributed, i.e. processes on one machine receive
| failure messages from processes on others, and can restart the
| processes elsewhere. These mechanisms on a practical basis are
| sufficient to automatically pick up the majority of transient
| failures. (I should add there are two classic ways to blow up a
| BEAM instance which make this less good than it should be: a
| bad C function call "NIF" for native something function, or
| posting messages to a process faster than it can consume them,
| which will eventually cause an OOM).
|
| But this differs from the underlying philosophy of the runtime,
| which is that things are only done when they're done, and you
| should expect failures at any time. This maps on to their
| messaging paradigm.
|
| What you actually sound like you want is a universe more like
| FoundationDB and QuiCK
| https://www.foundationdb.org/files/QuiCK.pdf where the DB and
| worker queue all live in one single transactional space, which
| certainly makes reasoning about a lot of these things easier,
| but have nothing to do with erlang.
| sshine wrote:
| > _what about [...] if a process has taken an item off a queue
| and then crashes before having fully processed it_
|
| > _you might not even know from the outside if it has been
| fully, partially, or not at all processed yet_
|
| Erlang does not propose a unique solution to distributed
| problems, just good primitives.
|
| So the answer would be the same; you'd keep track in the queue
| if the element was partially popped, but not completed, and you
| report back to the queue that the processing failed and that
| the element should be fully put back.
|
| So in Erlang you might monitor a worker process and requeue
| items handled by processes that failed.
| layer8 wrote:
| Thanks. So Erlang is really only about managing process
| lifetimes and simple RPC? In my experience processes often
| have meaningful internal state, meaningful in the sense that
| it matters if it gets lost due to a crash. If I understand
| correctly, Erlang doesn't provide any particular model or
| mechanisms for dealing with that?
| fidotron wrote:
| > In my experience processes often have meaningful internal
| state, meaningful in the sense that it matters if it gets
| lost due to a crash.
|
| The erlang process state will be simply what it has on the
| stack. (Ignoring things like ETS tables for the moment).
|
| Erlang has the concept of ports, used to interface to the
| world outside, that provide a sort of hook for cleanup in
| the event of a crash. Ports belong to processes, in the
| event of a crash all associated ports are cleaned up. You
| can also set this sort of thing up between purely erlang
| processes as well.
|
| As the other commenter observed, erlang gives you the
| primitives to make distributed systems work; it does not
| prescribe solutions, especially around distributed
| transactions, which imo is one of the reasons some of the
| hype around the BEAM is misguided.
| whorleater wrote:
| One thing I haven't seen being discussed is the BEAM internals
| becoming a little long in the tooth. We still have static
| reductions before the scheduler switches to another task, the
| priority system in scheduling is a bit dodgy, flipping vmargs is
| kinda complex, lock counting and crash dump tooling kinda suck,
| etc.
|
| BEAM is great, although it's definitely missing something like
| pprof for go or java flight recorder.
| geophile wrote:
| In 2003 I joined a startup building a horizontally scalable
| archive. You could add nodes to add capacity for storing data and
| metadata, and the system could tolerate up to a configured number
| of failures and carry on without loss of data or service. (This
| was not a general-purpose file system, it was for write-
| once/read-many objects.)
|
| We built the system in Java and C. The distribution layer was
| done completely in Java. It was only after the system was done
| that I discovered Erlang. I REALLY wish I had known about it
| earlier. Erlang solved so many of the problems we had to solve by
| ourselves.
| pkilgore wrote:
| Interesting. It strikes me that some of this rhymes with the
| platform abstraction of roc[1]
|
| [1] https://www.roc-lang.org/platforms
| jFriedensreich wrote:
| TLDR title for erlang people: erlang is not just erlang but
| erlang + OTP.
| jiggawatts wrote:
| Someone explain to me why I should prefer Erlang/BEAM/Elixir over
| something like Akka.NET?
|
| With the latter I get a huge ecosystem of packages and wide
| compatibility with platforms and tooling _and also_ a robust and
| scalable actor model.
|
| Learning Erlang or any related language meanwhile feels like
| learning Tolkien's Elvish for the purposes of international
| trade.
| travisgriggs wrote:
| To me, Erlang/Elixir's power is not necessarily the Actor model
| implementation, the matching from prolog, immutability,
| behaviors, etc, but Joes desire to demonstrate you could do more
| with less.
|
| It is a well thought out and trued system of computation that has
| a consistency rarely witnessed in other languages, much less the
| "web". It is not perfect. But it is pretty impressive.
|
| Unfortunately, I find the appreciation and uptake for what
| simplicity empowers in the software world pretty under
| appreciated. Complexity allows people to become specialists,
| managers to have big teams and lots of meetings, experts to stay
| experts.
|
| Erlang was being developed in a period where companies were
| trying to implement software solutions with smaller headcounts,
| limited horsepower, etc. A multi decade outpouring of cash into
| the domain has made the value of "less will mean more for all of
| us in good ways" less of an attractor.
| HeavyRain266 wrote:
| Erlang, OTP, and the BEAM offer much more than just behaviours.
| The VM is similar to a virtual kernel with supervisor, isolated
| processes, and distributed mode that treats multiple (physical or
| virtual) machines as a single pool of resources. OTP provides
| numerous useful modes, such as Mnesia (database) and atomic
| counters/ETS tables (for caching), among others. The runtime also
| supports bytecode hot-reloading, a feature used to apply patches
| without any system downtime. While the syntax is not very screen
| reader-friendly, it is digestable.
|
| Apache Mesos[1] is the only thing that comes to my mind as a
| similar platform to BEAM in its ability to treat multi-machine
| resources as a single pool.
|
| Over a year ago, my private consulting company decided to adopt
| Erlang as our backend language. After some time, we started
| exploring BEAM's internals to, for example, replace the TCP-based
| stack with QUIC and integrate some Rust patches. A truly
| fantastic choice for lightweight and high-throughput systems that
| are only failing in case of kernel panic or power loss. We are
| currently working on very "busy", concurrent software like a
| film/game production tracker and pipeline manager, and are now
| also preparing R&D for a private hospital management services.
|
| [1]: https://mesos.apache.org/
| HeavyRain266 wrote:
| Before you ask, we're not going to ever fully adopt Elixir (or
| Gleam) as its ecosystem is built around Phoenix framework and
| external services/databases. We would have to maintain internal
| bindings/implementations of things that are unmaintained on
| Elixir's side. Also worth to mention that it has a large amount
| of syntax sugar and its users have that weird fetish for
| abstracting stuff into DSL interfaces.
| throwawaymaths wrote:
| I don't think behaviours are all that interesting; after all
| other programming languages have them.
|
| Rather, what is interesting about the BEAM is that throwing an
| error is so graceful that it's not such a sin to just throw an
| error. In otherwords, a component that CAN error or get into a
| weird state can be shoved into a behaviour that CANNOT. And by
| default you are safe from certain operational errors becoming
| logic or business errors.
|
| For example. You might have a defined "get" interface that
| doesn't return an error -- let's say it starts as an in-memory
| K/V store and it returns an optional(value), which is NULL in the
| case that the key didn't exist.
|
| But suppose you want to have two datastores that the same
| interface targets, so you might abstract that to a filesystem,
| and you could have a permission error. And returning "NULL" is
| not actually "correct". You should throw, because that bubbles up
| the error to ops teams instead of swallowing it whole. A panic in
| this case is probably fine.
|
| What if now you're going over a filesystem that's over the
| network, and the line to the datacenter was backhoe'd and there
| was a 10 millisecond failover by your SDN -- returning "NULL" is
| _really_ not correct, because consumers of your getter are liable
| to have a bad time managing real consistency business cases that
| could cost $$$. And in this case a panic is not necessarily
| great, because you bring down everything over a minor hiccup.
|
| The other power with throwing errors + behaviors is that it makes
| trapping errors with contextual information reporting (e.g. a
| user-bound 500 error with stack trace information sent somewhere
| where ops can take a gander) really easy and generically
| composable, that's not so for error monads or panics.
|
| Anyways it was always strange to me that erlang-inspired actor
| system programming languages came out that obsessed over "never
| having errors" as a principle (like ponylang) because that's
| throwing out a big part of erlang.
| dang wrote:
| Discussed at the time:
|
| _Erlang 's not about lightweight processes and message passing_
| - https://news.ycombinator.com/item?id=34545061 - Jan 2023 (274
| comments)
___________________________________________________________________
(page generated 2025-04-11 23:00 UTC)