[HN Gopher] Why I wrote the BEAM book
___________________________________________________________________
Why I wrote the BEAM book
Author : lawik
Score : 385 points
Date : 2025-06-04 10:36 UTC (12 hours ago)
(HTM) web link (happihacking.com)
(TXT) w3m dump (happihacking.com)
| asib wrote:
| I guess I've never worked on something of Klarna's scale, but
| 15ms seems like a very small amount of time to cause a post-
| mortem-worthy event!
| __jonas wrote:
| I am also curious about the mentioned incident, does anyone
| have a link to the postmortem the post talks about? Couldn't
| find anything online.
| lawik wrote:
| I would assume it was an internal post-mortem. Far from all
| are public affairs.
| mindjiver wrote:
| Also I think the incident is over 10 years old as well, if
| it's the problem I think it is.
| abrookewood wrote:
| Yes, it does seem small, but the BEAM often has response times
| in microseconds (ms). If you are used to that and something
| 'blows out' to milliseconds, then I can see why alarms might
| get triggered.
| regularfry wrote:
| If it's millions of calls and it's 15 additional milliseconds
| _per call_ , that's going to spectacularly gum up a system
| that's tuned in microseconds. Even if there's no additional
| load, the queues all just got several thousand times longer.
| klank wrote:
| This exchange made me think about how I've always been a
| fan of using the "natural" units of a system when reporting
| metrics.
|
| I do this because it helps those unfamiliar with the system
| more intuitively understand relative sizes, those
| unfamiliar with converting between units in their head to
| understand numbers, and even for those familiar, it can
| avoid confusion or misattributed units.
| benmmurphy wrote:
| this can easily happen in a BEAM system. say you have some
| shared state you want to access. you create a gen_server to
| protect this shared state. a gen_server is basically a huge
| mutex. the gen_server is just a normal beam process that
| handles requests sent to its message queue and then sends a
| reply message back. lets say it can process a request normally
| in 20us. so a 15ms pause would stack up 750 messages in its
| message queue. now maybe this is not enough to generate a huge
| outage on its own but maybe as part of your handling you are
| using the message queue in an unsafe way. so when you check the
| message queue for a message the BEAM will just search the whole
| message queue for a message that matches. there are certain
| patterns the BEAM is able to optimize to prevent the whole
| message queue being searched (i think almost every pattern is
| unsafe and the BEAM only optimizes the gen rpc style message
| patterns) . but if you are using an unsafe pattern when you
| have a message queue backlog it will destroy the throughput in
| the system because the time taken to process a message is a
| function of the message queue length and the message queue
| length becomes a function of how long it takes to process a
| message.
|
| Also, the great thing is you might not even have an explicit
| `receive` statement in your gen_server code. You might just be
| using a library that is using a `receive` somewhere that is
| unsafe with a large message queue and now you are burned. The
| BEAM also added some alternative message queue thing so you are
| able to use this instead of the main message queue of a process
| which should be a lot safer but I think a lot of libraries
| still do not use this. This alternative is 'alias' (https://www
| .erlang.org/doc/system/ref_man_processes.html#pro...) which
| does something slightly different from what I thought which is
| to protect the queue from 'lost' messages. Without aliases
| 'timeouts' can end up causing the process message queue to be
| polluted with messages that are no longer being waited on. This
| can lead to the same problems with large message queues causing
| throughput of a process to drop. However, usually long lived
| processes will have a loop that handles messages in the queue.
| valenterry wrote:
| > lets say it can process a request normally in 20us.
|
| Then what if the OS/thread hangs? Or maybe a hardware issue
| even. Seems a bit weird to have critical path be blocked by a
| single mutex. That's a recipe for problems or am I missing
| something?
| benmmurphy wrote:
| Part of the problem with BEAM is it doesn't have great ways
| of dealing with concurrency beyond gen_server (effectively
| a mutex) and ETS tables
| (https://www.erlang.org/doc/apps/stdlib/ets). So I think
| usually the solution would be to use ETS if its possible
| which is kind of like a ConcurrentHashMap in other
| languages or to shard or replicate the shared state so it
| can be accessed in parallel. For read only data that does
| not change very often the BEAM also has persistent term (ht
| tps://www.erlang.org/doc/apps/erts/persistent_term.html).
| techpression wrote:
| Depending on what you want to do, there are ways to change
| where the blocking occurs, like
| https://blog.sequinstream.com/genserver-reply-dont-call-
| us-w...
| toast0 wrote:
| Hardware issues happen, but if you're lucky it's a simple
| failure and the box stops dead. Not much fun, but recovery
| can be quick and automated.
|
| What's real trouble is when the hardware fault is like one
| of the 16 nic queues stopped, so most connections work, but
| not all (depends on the hash of the 4-tuple) or some bit in
| the ram failed and now you're hitting thousands of ECC
| correctable errors per second and your effective cpu
| capacity is down to 10% ... the system is now too slow to
| work properly, but manages to stay connected to dist and
| still attracts traffic it can't reasonably serve.
|
| But OS/thread hangs are avoidable in my experience. If you
| run your beam system with very few OS processes, there's no
| reason for the OS to cause trouble.
|
| But on the topic of a 15ms pause... it's likely that that
| pause is causally related to cascading pauses, it might be
| the beginning or the end or the middle... But when one
| thing slows down, others do too, and some processes can't
| recover when the backlog gets over a critical threshold
| which is kind of unknowable without experiencing it.
| WhatsApp had a couple of hacks to deal with this. A) Our
| gen_server aggregation framework used our hacky version of
| priority messages to let the worker determine the age of
| requests and drop them if they're too old. B) we had a hack
| to drop all messages in a process's mailbox through the
| introspection facilities and sometimes we automated that
| with cron... Very few processes can work through a mailbox
| with 1 million messages, dropping them all gets to recovery
| faster. C) we tweaked garbage collection to run less often
| when the mailbox was very large --- i think this is
| addressed by off-heap mailboxes now, but when GC looks
| through the mailbox every so many iterations and the
| mailbox is very large, it can drive an unrecoverable cycle
| as eventually GC time limits throughput below accumulation
| and you'll never catch up. D) we added process stats so we
| could see accumulation and drain rates and estimate time to
| drain / or if the process won't drain and built monitoring
| around that.
| Rietty wrote:
| > we had a hack to drop all messages in a process's
| mailbox through the introspection facilities and
| sometimes we automated that with cron...
|
| What happens to the messages? Do they get processed at a
| slower rate or on a subsystem that works in the
| background without having more messages being constantly
| added? Or do you just nuke them out of orbit and not
| care? That doesn't seem like a good idea to me since loss
| of information. Would love to know more about this!
| toast0 wrote:
| Nuked; it's the only way to be sure. It's not that we
| didn't care about the messages in the queue, it's just
| there's too many of them, they can't be processed, and so
| into the bin they go. This strategy is more viable for
| reads and less viable for writes, and you shouldn't nuke
| the mnesia processes's queues, even when they're very
| backlogged ... you've got to find a way to put
| backpressure on those things --- maybe a flag to error
| out on writes before they're sent into the overlarge
| queue.
|
| Mostly this is happening in the context of
| request/response. If you're a client and connect to the
| frontend you send a auth blob, and the frontend sends it
| to the auth daemon to check it out. If the auth daemon
| can't respond to the frontend in a reasonable time, the
| frontend will drop the client; so there's no point in the
| auth daemon looking at old messages. If it's developed a
| backlog so high it can't get it back, we failed and
| clients are having trouble connecting, but the fastest
| path to recovery is dropping all the current requests in
| progress and starting fresh.
|
| In some scenarios even if the process knew it was
| backlogged and wanted to just accept messages one at a
| time and drop them, that's not fast enough to catch up to
| the backlog. The longer you're in unrecoverable backlog,
| the worse the backlog gets, because in addition to the
| regular load from clients waking up, you've also got all
| those clients that tried and failed going to retry. If
| the outage is long enough, you do get a bit of a drop
| off, because clients that can't connect don't send
| messages that require waking up other clients, but that
| effect isn't so big when you've only got a large backlog
| a few shards.
| cess11 wrote:
| If the user client is well implemented either it or the
| user notices that an action didn't take effect and tries
| again, similar to what you would do if a phone call was
| disconnected unexpectedly or what most people would do if
| a clicked button didn't have the desired effect, i.e.
| click it repeatedly.
|
| In many cases it's not a big problem if some traffic is
| wasted, compared to desperately trying to process exactly
| all of it in the correct order, which at times might
| degrade service for every user or bring the system down
| entirely.
| davidw wrote:
| > this can easily happen in a BEAM system
|
| Wait... I thought all you had to do is write it in Erlang and
| it scales magically!
| toast0 wrote:
| Erlang makes a lot of hard things _possible_ , some tricky
| things easy, and some easy things tricky.
|
| There's no magic fairy dust. Just a lot of things that fit
| together in nice ways if you use them well, and blow up in
| predictable ways if you have learned how to predict the
| system.
| davidw wrote:
| I know that, having built several myself. Sometimes
| people get a bit ahead of themselves with the marketing
| though.
| nilirl wrote:
| Books ask for a lot of organization in general. And books on an
| evolving subject never stop asking.
|
| I don't use Erlang, but for 13 years in the making, I'm getting a
| copy.
|
| Thank you.
|
| p.s if the author sees this: Kindle edition too, please.
| ErikStenman wrote:
| OK. It is currently under review by Amazon. Remember, a PDF is
| free on GitHub; the Kindle thingy will cost some money. - Happi
| ErikStenman wrote:
| It is now available. https://www.amazon.com/dp/B0FBX7YWB1
| cmoski wrote:
| Thank you very much!
| graboid wrote:
| Are there any other VM's like the BEAM? I never heard of any
| (admittedly I know little of this subject), and I wondered, is it
| because there is no need for another one because the BEAM is just
| so good, or is it because the amount of work and skill required
| to get another BEAM-like runtime with comparable quality is too
| demanding?
| lawik wrote:
| There is a separate implementation for really constrained
| device use, called AtomVM.
|
| There are many dead efforts to implement something like the
| BEAM or OTP within other ecosystems. Usually not as a VM.
| reddit_clone wrote:
| I have seen attempts at actor systems like erlang/beam's.
| Scala/Akka seems to have been a success?
|
| Share nothing, green-thread/coroutines also seem popular now
| a days.
| lvass wrote:
| C++ Actor framework is also considerably popular, and
| Scylladb has their own framework which I believe does
| something similar.
| lamuswawir wrote:
| Foundationdb also has it's own C++ actor framework.
| jerf wrote:
| At the time that BEAM was invented in the late 1990s, and in
| the early 2000s, it was a fairly unique proposition.
|
| Nowadays there isn't anywhere near as much stuff that it does
| uniquely. That's probably why there isn't another one. All of
| the compiled languages off-the-shelf can solve the same
| problems that BEAM does now, and often with other advantages to
| boot.
|
| There's something about the Erlang community that convinces
| people in it that if it isn't solve the exact same way that
| BEAM does, then it must _ipso facto_ not be as good as BEAM,
| but that 's not true. If you ask the question _can I solve the
| same problems, even if it 's in a different way?_, you've got a
| zoo of options in 2025, whereas your options in 2000 were much
| much weaker.
|
| And yes, being BEAM-compatible is harder than meets the eye.
| There are projects like https://github.com/ergo-services/ergo
| that can do it, and I believe there are some in other
| languages. It's a pretty niche need in my opinion, though. If
| you don't need to hook up to some existing BEAM infrastructure,
| I don't consider it a very good solution for a greenfield
| project. You're better off with more modern tooling and
| solutions that are more native to your chosen dev environment.
| prophesi wrote:
| I'm not sure if it's about the implementation rather than the
| vast reduction of complexity/boilerplate to attain fault-
| tolerant distributed systems compared to other solutions.
| giraffe_lady wrote:
| I agree with what I think is your overall point, that there
| are other solutions that adequately solve the problems BEAM
| does but in different ways.
|
| I really strongly disagree with the idea that there's no
| modern use for BEAM because of these other solutions. It's
| not simply that we've convinced ourselves that "if it isn't
| solve the exact same way that BEAM does, then it must ipso
| facto not be as good as BEAM" though I understand how you
| could see it that way.
|
| Frankly what it is is that BEAM has an exceptionally well
| chosen position among the possible tradeoffs of solving these
| problems, which you are otherwise on your own to figure out
| and which are in themselves some of the most difficult
| practical considerations of systems design.
|
| So again it's not that only BEAM can possibly do it right,
| but it's that BEAM _does_ do it right. Having seen so many
| other attempts to do it better fail, at tremendous expense,
| it 's an easy choice for me for systems that are expected to
| encounter these problems.
| jerf wrote:
| "I understand how you could see it that way."
|
| It is less that "I see it that way" then that "I encounter
| plenty of people who speak that way.", and that the Erlang
| community still definitely indoctrinates people to think
| that way.
|
| See my other post on the topic:
| https://news.ycombinator.com/item?id=44181668 Which echos
| some of your points.
| cianuro_ wrote:
| Can you map the BEAM features against the zoo of options you
| mentioned so readers understand what exactly are these other
| options and how they compare to BEAM?
| jerf wrote:
| BEAM is basically the combination of a clustering system, a
| message bus for communicating among its nodes with a
| defined protocol, and a shared data model that can be
| thought of as basically "JSON, and only JSON, is the data
| model"; not JSON mapped into data structures or classes,
| but just JSON.
|
| The biggest thing you need to have to have BEAM-like
| reliability is a message bus. Build a message-bus based
| system and use it properly and you're already 80% of the
| way there. In 1998, who knew what a "message bus" was?
| Today, it's a field so stuffed with options I won't even
| try to summarize them here. The one thing I will point out
| is that BEAM is 0-or-1 and 1-or-n seems to have won the
| race; this is one of the things I don't like about BEAM's
| message bus.
|
| BEAM is based on a JSON-like data scheme because it wasn't
| clear how to have what you'd consider "classes" and such on
| a cluster with multiple versions of things any other way.
| Since then, there are now multiple technologies to solve
| this problem, like gRPC, Cap'n Proto, and again, what was
| "who's heard of that?" in 1998 is now an entire field of
| options I can barely summarize. It is no longer necessary
| to sacrifice everything we get with "methods" and "user
| data types" to have cross-cluster compatibility.
|
| Bringing up clusters is now just Tuesday for a devops team.
| Kubernetes, Docker Cloud, all the cloud-specific
| orchestrations like CloudFormation, bringing up clusters of
| things now has many technologies you can deploy. Moreover,
| these technologies do not impose the requirement that your
| nodes be essentially homogeneous, all running BEAM. You can
| bring up any tech stack you like doing any specialized tech
| stuff you need, all communicating over shared message
| busses.
|
| Reliability options vary widely, from running internal
| services in OS processes to capture and restart things, to
| things like lambda functions that are their own solution to
| that sort of reliability, to isolated micro-VMs, to
| threading-based solutions... while there is a certain
| appeal to "just crash and take the thread down" it's not
| the only option anymore. Now that every language community
| that can is building huge servers with millions of threads
| in them, many solutions to this problem have developed,
| with different cost/benefit tradeoffs for each. The "crash
| out the whole thread" is not the only option, merely one
| interesting one.
|
| As to how they compare to BEAM, that does slant the
| question a little bit, as if BEAM is the golden standard
| that everyone else is still desperately aspiring to. How
| they compare to BEAM is that there is now a zoo of options
| of every sort, with a huge variety of tradeoffs in
| performance and cost and ease-of-use and ease-of-
| deployment, and BEAM is merely a particular point in the
| huge field now, which I wouldn't even characterize as
| particularly standing out on any front. For every thing it
| does like "have a nice crashing story" there's a tradeoff
| where it did things like "give up user-defined data types
| because they couldn't figure out how do to them in a
| cluster in the late 1990s". BEAM's feature set, on a
| feature-by-feature basis, isn't particularly special any
| more. There's faster options, there's easier options,
| there's "works with my current language" options, there's
| more reliable options, there's cheaper options, there's all
| kinds of things... but not all of these at once
| necessarily.
|
| So, what is BEAM's unique value proposition in 2025? It is
| the _integration_ of the solutions, and picking a set of
| solutions that IMHO may not be "the best" on any
| particular front any more but has proved to be "adequate to
| many tasks" for decades now. You can assemble all those
| technologies I mentioned in an incredible number of
| combinations, but you have to do the assembly yourself, and
| burn time asking yourself, which message bus? Which
| clustering? Which orchestration? It's overwhelming, and
| made even harder by the fact that even with all these
| choices there's still a lot of developers who don't even
| know these things exist, let alone how to evaluate them and
| start integrating them into a system successfully. BEAM
| offers a one-stop shop, with prepared and opinionated
| answers with a proved track record, and a community that
| knows how to use them, libraries that integrate with the
| opinionated answers. I.e. who can write a library that
| "works with Kafka, Amazon SQS, NATS, Redis streams,
| Postgres events, and a dozen other messaging libraries"
| without having to work with a lowest common denominator
| that is almost useless? But someone can definitely provide
| "an Erlang library" that meaningfully integrates with all
| of the BEAM infrastructure without an issue. I don't think
| BEAM is best-of-breed on any particular front but it is
| arguably best-of-breed in providing them _an_ answer to you
| for all these problems under a single roof, and then being
| able to build on the advantage that picking a set of
| solutions brings you.
|
| I wish the BEAM/Erlang/Elixir community would lean more
| into this as their pitch, and stop running around and
| acting like they've got the _only_ solution to the problems
| and as if they are the gold standard. The advantage of BEAM
| is not that they have the only reliability solution, the
| only message bus, the only clustering solution, the only
| etc. etc. anymore... the advantage is in the _integration_.
| The individual components of the solutions are not where
| the awesomeness lies; in 2025 most of them have been
| handily exceeded on many fronts now, but that _integration_
| and the subsequent things like libraries built on that
| integration _is_ a unique proposition, even today. The very
| explosion of choice in the solutions to the problems BEAM
| addresses and the resulting diaspora in all the code bases
| in the world make it difficult to build very much on top of
| these things because there 's so many differences in so
| many places.
| lvass wrote:
| Do note that many things mentioned just happen to be
| incentivized or default in the BEAM, but are not required
| at all. Your processes don't have to crash if you don't
| want to let them crash, exception handling is there.
| Messages don't have to be 0-or-1, Manifold exists. You
| are absolutely not forced to use some language, you have
| NIFs, dirty NIFs, ports, rustler, zigler, pythonx, etc,
| many offering impeccable ergonomics. Some things may not
| be as optimized as alternatives are? Sure, but the BEAM
| works for Whatsapp and Discord and of all things they
| just use raw ETS for their most important data structure.
| You almost surely don't need something "better".
| garbthetill wrote:
| I'm actually kind of a fan, I've read your past comments
| on BEAM and Elixir while digging through HN to get a
| sense of how people view it. I appreciate your takes, but
| I'm honestly confused by this one and wanted to push back
| a bit, I'm not as knowledgeable as you on this topic, but
| bear with me
|
| I get your point that BEAM's individual components might
| not be the best in 2025 and you get the point of
| uniformity. so whats the point of saying there's a better
| BEAM like system, but then fail to point out one?
| Elixir/BEAM community promoting themselves as the only
| solution to said problems isn't a bad thing imo, because
| what other system can give me the guarantee without
| forcing me to learn a bunch of new DSLs or scripting
| languages, and deal with the idiosyncrasies of a bunch of
| different systems? With Elixir or Erlang, I can stick to
| one coherent environment and get all that.
|
| Again you state all this in your post, yet say
| elixir/beam isn't the gold standard, then what is? As i
| am having a blast working with phoenix, liveview and
| nerves and the BEAMS guarantee of a ms soft real time
| fault tolerant system hasn't failed me yet and there
| doesnt seem to be anything like it in the market. The
| only thing I hate about elixir is types and would switch
| to rust/go if there was a similar offering
| jerf wrote:
| "but then fail to point out one?"
|
| I didn't point out one, I pointed out thousands. All the
| combinations of message busses and serializations and
| schedulers and monitors you can imagine. Systemd
| monitoring a selection of processes that read and write
| from kafka queues in a cluster of VMs. Lambda functions
| that read and write Amazon SQS/SNS queues written in Go.
| Azure Functions using JSON to communicate over Azure
| Service Bus with Terraform configuring redundancy. A
| microservices architecture orchestrated with etcd based
| on gRPC services communicating with each other on a NATS
| message bus. Arbitrary mixes and matches of all of these
| things, probably the most common architecture of all at a
| corporate level.
|
| Many of these beat BEAM on some metric or other that may
| be important at some point. For instance it's not hard to
| beat BEAM on performance (though it is also not hard to
| lose to it on performance; it's a very middle-of-the-road
| system on that front, which I mean completely literally).
| Another thing that you get very naturally is
| heterogeneity; if you want everything on a BEAM system
| you really have to stick to BEAM's languages, which is a
| pretty big issue in most cases.
|
| The reason I say BEAM is not the gold standard is that
| there's still people running around who speak as if BEAM
| is _the only_ way to write reliable software, that it is
| still the _only_ way to have lots of independent services
| that communicate by passing messages, that if you don 't
| implement _every single one_ of gen_server 's features
| and work _exactly_ like OTP 's supervision trees and if
| you don't handle errors by crashing processes, then
| you're not operating at Erlang's Golden Standard.
|
| And that's not true anymore. There's plenty of
| alternatives, many better than BEAM on many fronts. BEAM
| is not the natural obvious leader standing above the rest
| of the crowd, secure in the knowledge that nothing is as
| good as it is and never will be. It's now down in the
| scrum, and as long as it is metaphorically running around
| claiming it's somehow unique I'm going to grumble about
| it. It's not even good for BEAM itself, which really
| ought to be pitching integration rather than "Look! We
| solve the reliability problem uniquely!"
|
| To the extent that people are reading my post and
| basically backpedaling and insisting "Oh, no, it's the
| integration all along that we've been pitching"... well,
| I don't particularly enjoy the rewriting of history, but
| other than that... yes! Good! Do that! Pitch the
| integration! It's a legitimate advantage! But it's not
| 2005 anymore. Every major compiled language can handle
| tens or hundreds of thousands of connections on a
| reasonably sized server. Every language has solutions for
| running this robustly and with decoupled architecture.
| Every language has solutions to the problems now. BEAM's
| in a crowd now, whether it likes it or not.
|
| There is no gold standard for these technologies any
| more. One might as well ask "well, what's the best
| computer?" There's no answer to that question. Narrow it
| down to "what's the best gaming computer" and you can
| still ask "what sort of games" and it'll be a crowded
| field. There's more options that anyone can even analyze
| anymore.
| mustermannBB wrote:
| Correct me if I'm wrong but a big part of the value
| proposition these days for Erlang/Elixir would be that it
| comes build in with a lot of the tools that other runtimes
| would require. For example a message bus is kinda integrated
| in the BEAM runtime or Erlang, while most other runtimes
| would have to use something like Kafka or AWS SQS etc. Or
| caching and some other things. Of course I'm not sure how
| good all the inbuilt solutions are compared to dedicated
| tools, overall or usually IMHO I prefer dedicated tools as
| opposed to all in one as the dedicated tools/services usually
| are more flexible. On the other hand of course it is also
| much overhead.
| url00 wrote:
| The reality is modern Kubernetes infrastructure looks a lot
| like BEAM, at least in the capabilities it offers. That's the
| far more common way of deploying highly scalable, self-healing
| systems in the current year. Plus, with k8s you're not
| constricted to a single language (there are a few more than
| Erlang/Elixir, but nothing popular) with limited developer
| resources and interest.
| MarceColl wrote:
| So now you need several servers, an orchestrator, tons of
| YAML, arcane and terrible error messages and a devops team to
| kind of match the BEAM? That's... not a good look
| tough wrote:
| is that a fair comparison? i dont love k8s but you can
| deploy anything to it, not just erlang or elixir
| giraffe_lady wrote:
| It's snarky but I think it is a fair comparison. You do
| have extra capabilities over what BEAM offers, in
| exchange for having to manually handle a lot of things
| BEAM either handles invisibly or at least comes with
| usable defaults for.
| toast0 wrote:
| You can use a BEAM system to orchestrate other code, too.
| As ports, port drivers, nifs, c-nodes, just other OS
| processes spawned and using IPC/sockets. Lots of options.
| Using Erlang to supervise an OS process doing work
| perhaps enhances the isolation principles of BEAM.
| tough wrote:
| oh interesting wasnt really aware
|
| so you could use beam to orchestrate go or rust services
| communicating over IPC? Nice
| rramadass wrote:
| See also _CloudI_ a Cloud System built using Erlang -
| https://cloudi.org/index.html
| tough wrote:
| TY! Very interesting!
| cess11 wrote:
| To add to that, the BEAM has binaries as a data type so
| when you're talking to a foreign program you can quickly
| prototype or design binary protocols once you get tired
| of parsing/encoding JSON or XML or something at the
| edges. Depending on the facilities of the foreign
| language it might be more or less feasible, of course.
| lamuswawir wrote:
| The binary interface makes interop easier with other
| langauges. A note of incompatibility in versions after
| OTP-24 with those before.
| Thaxll wrote:
| Where does your Erlang code runs in the first place? (
| maybe kubernetes already )
|
| Kubernetes does all of that in a standard and easy way but
| also is completely language agnostic. So your Python code
| without modidication can benefit from it.
| MarceColl wrote:
| in my case it runs on a good old server.
|
| > So your Python code without modidication can benefit
| from it.
|
| that's not completely true though, say you have two
| python processes, you need to solve yourself how they
| communicate, HTTP? a message broker? through the DB? You
| need to handle errors, stateful deployments.
|
| You can deploy python code without modification if the
| python code does very simple things. My point is that the
| BEAM gives you a lot of this mostly presolved for you
| without having to add more infrastructure.
| AlchemistCamp wrote:
| Yes. This is a big part of what initially drew me to
| Elixir. It's more than feasible to run a server on a cheap
| VPS, get great, though not quite Golang or low-level
| language performance and have a much easier scaling story
| when you need multiple machines.
|
| More importantly, you generally don't need an external
| queue service, in-memory KV store, task scheduler or many
| of the other things that JS/Ruby/Python stacks need. By
| consolidating just about everything but the DB in a single,
| well designed system, it's possible for a very small team
| to take on relatively large challenges on a smaller budget.
| da02 wrote:
| Which VPS providers do you recommend? Which relational
| databases do you use with Elixir?
| MarceColl wrote:
| I use a dedicated hetzner server (12 cores 64gb ram) for
| 60 euros with postgres
| Cyph0n wrote:
| Not an Elixir expert, but my impression is that Postgres
| is a common choice. It's well supported by both Ecto
| (ORM) and Oban (job queue).
| rkangel wrote:
| I think the closest in wide use is probably Go.
|
| The BEAM is set up for "Erlang like languages" (or maybe it's
| the other way around). Writing Elixir, still feels a lot like
| Erlang because of the underlying semantics of how it operates.
| Even Gleam is closer to Erlang than anything else once you get
| past the types.
|
| Go also has goroutines/green threads/Erlang-like processes as a
| core primitive for its parallelism. it doesn't have the same
| "opinion" about how to structure a concurrent application that
| you get from OTP though.
| johnnyjeans wrote:
| Dis is probably the closest, followed by the smalltalk VMs. It
| depends on what you mean by "like the BEAM". The VM itself
| isn't that special, it's really the whole OTP on top of BEAM
| that makes Erlang good at what it does.
| k__ wrote:
| _" We will try to use the term Erlang Runtime System to refer to
| the general idea of any Erlang Runtime System as opposed to the
| specific implementation by Ericsson which we call the Erlang
| RunTime System or usually just ERTS."_
|
| Love it.
| justin66 wrote:
| That is... _not_ dumb?
| techpression wrote:
| Bought it instantly, even if it's available online for free I
| guess this way supports the author a little bit
| elric wrote:
| > I kept going because I wanted to understand the BEAM properly.
| There's value in following the real logic, not just the surface
| explanations.
|
| This resonates with me. That's the kind of drive that results in
| great output. Buying it just for that.
|
| I've been approached by publishers several times throughout my
| career. Each time the process was similar: they had an idea, I
| had an idea, we tried to come to common ground, and then the deal
| fell through because we couldn't find any. E.g. I didn't want to
| write a Java book aimed at 14 year olds. They didn't want me to
| write about classloaders (or whatever niche subject I was diving
| into at the time).
|
| Would love to learn how people find (non-empty) intersections of
| their passions & what readers want.
| the_arun wrote:
| Do we need to depend on Publishers? Can't we write books
| independently? Or is it because of the "Brand" and other perks
| that come with Publishers?
| elric wrote:
| Self publishing is easier than ever. But even shitty
| publishers like Packt still have a huge networking/marketing
| machine. My skillset does not include marketing, or checking
| whether a target audience exists.
|
| If I want to write for just myself, I can just journal or
| blog. A book is a significant undertaking, writing one which
| no one reads would just be depressing.
| tough wrote:
| someone should build a cooperative/self serve after stripe
| press books
| Kon-Peki wrote:
| Cooperative/self-serve? What does that mean?
| tough wrote:
| yeah idk but something like curation is done by commitee
| (to try and maintain a minimum of quality overall) but as
| an author is super easy to appply/get
| editors/feedback/publish
|
| (that'd be the self-serve part i guess)
|
| tbh hardest is still marketing. good books are not only
| text but also covers and the like
| sobadically wrote:
| So basically a business, where day to day is run by
| people focused on the day to day and everyone else can
| pick and choose to use their stuff with low friction, not
| having to focus on the day to day
|
| The hardest part still being networking into others lives
| to distribute the message
|
| You just re-invented our economy
| christhekeele wrote:
| > yeah idk but something like curation is done by
| commitee (to try and maintain a minimum of quality
| overall) but as an author is super easy to appply/get
| editors/feedback/publish
|
| This is very much what pragprog.com is meant to be. I'm
| only on the volunteer curation committee so have less
| insight into the feedback cycle for authors post-
| acceptance, but every author who's published on the
| platform I've talked to has been pretty positive about
| the experience.
|
| The OP didn't go into nearly as many (indeed, any)
| details as to why their second publishing attempt with
| them in particular did not work out, I'd be curious to
| learn more.
| tough wrote:
| Yeah Pragmatic Programmers seems one of the best
| publishers on our industry
| connicpu wrote:
| The problem with marketing just anything is that if you
| market too many things that people don't actually want,
| your brand starts to lose its power as people stop
| trusting that what you're advertising is something they
| might want.
| tough wrote:
| Indeed curation remains the important function of any
| publication/brand/company
|
| the gist of the idea: >
|
| yeah idk but something like curation is done by commitee
| (to try and maintain a minimum of quality overall)
| spelunker wrote:
| I mean in the blog post the author literally open sourced and
| then self-published his book after being dropped by multiple
| publishers for taking too long.
| drob518 wrote:
| Publishers add the following:
|
| 1. marketing and reach
|
| 2. financial risk on paper copies
|
| 3. Production services (e.g. editing and artwork)
|
| If you don't need those or can get them some other way (e.g.
| hire an independent editor), then you are better off self
| publishing and not giving the publisher a cut.
| munificent wrote:
| _> 1. marketing and reach_
|
| In theory, yes, but they have less expertise than you might
| imagine. For technical writing, keep in mind that editors
| at publishing companies aren't actually tech people. They
| may have been at one point, years ago, but they don't
| _really_ know what matters to programmers today in that way
| that an active working engineer does.
|
| _> 2. financial risk on paper copies_
|
| That was much more of a thing before print-on-demand. You
| don't have to take the risk of a several thousand copy
| offset press run anymore.
|
| There is maybe an argument that offset printing is higher
| quality, but I have textbooks from major academic
| publishers whose print quality is clearly worse than the
| POD stuff I get from Amazon for my book.
|
| _> 3. Production services (e.g. editing and artwork)_
|
| This is absolutely critical, agreed. Though they are often
| contracting out for this and if you're comfortable finding
| and vetting freelancers yourself, then they don't add a ton
| of value.
| dangoor wrote:
| I don't know if publishers manage to get better deals from
| Amazon, but Amazon's Kindle Direct Publishing has an annoying
| aspect that the ebook royalty rate falls from 70% to 30%
| after $9.99, which gets very awkward for a tech book you'd
| like to sell for, say, $19.99.
|
| Also, piracy is rampant and Amazon doesn't do much about it.
| Publishers have more resources to stay on top of it, I
| suspect.
| crystal_revenge wrote:
| Personally my favorite part of working with real publishers
| is _editors_. Having a development editor, technical editor,
| and a team of copy editors really helps making sure the book
| comes out polished. Additionally, I don 't want to deal with
| all the layout work and the details of printing.
|
| And, depending on the book, yes the distributor the publisher
| has can be very helpful for sales. It's nice to be able to
| grab your book off the shelf at Barnes and Noble (and, does
| lend a bit more credibility to your work).
|
| All that said, if you're writing for purely economic reasons
| (which I would caution against regardless), you're probably
| going to make roughly the same if you self-publish for a
| small audience vs go with a traditional publisher for a
| larger audience, and if you can get a larger audience self-
| publishing then there will be no comparison.
| drob518 wrote:
| Having written three books, what I found was that you either
| self publish or you write books that publishers want. Some of
| that is choosing publishers that specialize in certain types of
| books. Some publishers want "Learn AI in 21 Minutes with
| Python," while others want "Deep Dark Secrets of Java Class
| Loaders." O'Reilly is the best for niche technical stuff. Most
| of the rest of the industry wants beginner stuff because that's
| where the volume is. Fortunately, self-publishing is easier
| than ever and various sites make it easy to sell copies online,
| so you don't necessarily have to just give it away for free.
| But yea, there's no magic formula here. If you really want to
| write something niche, don't expect that a publisher will be
| interested in. Expect that you'll have to self publish and
| promote it yourself.
| asplake wrote:
| That's my experience too. My most recent book (my fifth) was
| closer to one publisher's stated mission than 95% of its
| catalog, but they all but admitted that they had given up on
| it. After that rejection I didn't have the energy to approach
| anyone else and did it myself instead. LeanPub first, then
| Amazon (print and kindle), Apple, Kobo, and Google.
| drob518 wrote:
| Yea, good call. My books were all published back in the
| 1990s, back when paper was still king and you really
| couldn't self-publish. Today, I'd probably look much more
| heavily at the self-publishing route, unless I wanted to
| write something uber-commercial. What was your experience
| with the order volume you could drive via self-publishing?
| Did you do any of your own marketing, or did you just list
| it in LeanPub and Amazon and let search engines lead people
| to it organically. If your subject was niche, there isn't
| much volume in any case, but you want to get all you can of
| that small market.
| asplake wrote:
| More niche than mainstream I guess, but I do have what
| could be called a community, a large following on
| LinkedIn, a mailing list in the low thousands, paid
| subscribers, and a partner programme. Not enough to
| generate the kind of sales that anyone could live on, but
| not nothing either. Much of my blogging since publication
| has been in support of the book.
|
| Two of my five did have publishers. I'm grateful for that
| experience. Learned a lot!
| drob518 wrote:
| Nice! Thanks for the follow up.
| turkeygizzard wrote:
| Shameless self promotion but this is exactly how I ended up
| writing a book on strength training for climbing, just pursuing
| every rabbit hole I could.
|
| I was ready to self publish but found a publisher who was
| interested. I had to make some changes to make it more
| readable, but you might have luck approaching publishers
| yourself
| tarunkotia wrote:
| If you have something interesting to say then people will
| figure a way to understand it. Early in my career I remember
| coming across "Essential .Net" by Don Box and I don't think Don
| Box had a particular audience in mind. He just unwrapped what's
| under the hood in Common Language Runtime (CLR). It took me
| multiple times to really understand "essential .net".
| ww520 wrote:
| > I kept going because I wanted to understand the BEAM
| properly. There's value in following the real logic, not just
| the surface explanations.
|
| Teaching is the best way to learn. I found that out when I
| started tutoring classmates for math in high school.
|
| Same thing with writing a book. Something about learning a
| subject and turning around to speak/write out about it that
| really crystallizes the in-depth understanding beneath the
| surface.
| the_arun wrote:
| Here is the link to the git repo -
| https://github.com/happi/theBeamBook
| spelunker wrote:
| Plenty of documentation can be found about the JVM, but BEAM has
| always seemed like a bit of a mystery to me. This is great!
| lamuswawir wrote:
| The official documentation explains things a little, but only a
| little.
| tra3 wrote:
| I've been watching erlang from afar for a few years. Here's to
| hoping to actually get to work with it one day.
| fridder wrote:
| Elixir is a pretty great alternative
| bigbuppo wrote:
| Erlang is one of those languages that will make you a better
| programmer in other languages.
| lamuswawir wrote:
| Fullstops.
| game_the0ry wrote:
| The BEAM might be the most under-rated tech in all of open
| source.
|
| Example -- whatsapp.
|
| Why Elixir + Erlang are not more popular for high concurrency
| projects is a mystery to me.
| honkycat wrote:
| everyone is trying to hire the best, but caters to the lowest
| common denominator.
|
| "But what if we hire an engineer to stupid to learn a new
| language in a few weeks???" and on and on and on
| paradox460 wrote:
| I believe it's purely marketing
|
| Other languages (Java, C#, Go) are supported by massive
| corporate backers, who have vested interest in seeing them
| succeed. Erlang's own corporate owner tried to strangle it in
| the crib, and even since then they've been standoffishness
| towards anything apart from technical resources.
|
| We didn't really see much marketing like material arise until
| things like Elixir came about, and even that is more following
| the ruby model, which is very dev oriented. And the world to
| which Elixir emerged in 2014 is very different than the world
| rails sprung into in 2004.
|
| Devs can usually be convinced about how good BEAMlangs are, but
| without the glossy, you have a harder time convincing the MBA
| set
| sargun wrote:
| Investment gap is what I'd say too. While Rust, Go, Python,
| etc... have had massive backers that have managed to invest a
| ton more into things like static analysis, type checking, and
| developer ergonomics, the Erlang ecosystem hasn't necessarily
| had the same love, and instead the major users have typically
| chosen to pivot, or build something outside of the BEAM.
| bsder wrote:
| > Why Elixir + Erlang are not more popular for high concurrency
| projects is a mystery to me.
|
| 1) Because the amount of "high concurrency" you can handle with
| a single machine with "standard" languages keeps moving up
| every year.
|
| 2) Because you had a single, nigh undocumented, very
| impenetrable to port implementation of BEAM for the longest
| time.
|
| 3) Because "Erlang" isn't the magic. _OTP_ is the magic. And
| that requires that you shift your mindset dramatically. And you
| can do the OTP abstractions in other languages!
|
| 4) I think Scala sucked up a lot of Erlang's oxygen for a long
| time. Erlang is helping with that.
| d3ckard wrote:
| 1. Fair, but at the same time - OTP is the only _sane_
| concurrency framework. To the point I still struggle to
| comprehend why it wasn 't copied. 2. Fair. 3. I am not aware
| of a single language doing OTP abstractions, especially
| implementing preemptive scheduler for green threads. 4.
| Possibly, but I haven't seen a single instance of a company
| choosing between the two.
|
| I think marketing is a way better explanation to be honest.
| Jose lives in Poland, was very active and _a lot_ of Ruby
| shops moved to Elixir.
| lamuswawir wrote:
| There have been multiple attempts in various languages to
| implement Erlang-like processes and claims of parity,
| Erlang processes remain unbeaten at what they do.
| agos wrote:
| OTP is the magic but the ability of the Beam to handle that
| many processes gracefully is part of the magic
| schonfinkel wrote:
| And now with Gleam moving past version 1.0 last year, the BEAM
| also boasts a decent amount of "flavors" of functional
| programming. You can go with Erlang/OTP (the alien tech that
| has been battle tested for almost 30 years), Elixir for it's
| great webdev ecosystem, Gleam if you like an ML-like
| programming language on the BEAM. LFE is also another great
| option if need a BEAM-native LISP.
| agos wrote:
| How I wish that Gleam had OTP!
|
| Elixir is also getting some tooling love in the form of a new
| language server and some very much appreciated type checking
| dimator wrote:
| Forgive my ignorance, I was looking at this the other day,
| isn't this otp for gleam?
|
| https://github.com/gleam-lang/otp
| d3ckard wrote:
| At the same time, the least secret "secret sauce" out there.
| BBC moved to Elixir (quite recent news), so I guess traction is
| there.
| franticgecko3 wrote:
| >Why Elixir + Erlang are not more popular for high concurrency
| projects is a mystery to me.
|
| I work at an Erlang shop.
|
| For Erlang to be useful you need to have massive scale,
| millions of DAU. Yes Elixir might run your website serving
| thousands of DAU but Erlang and the BEAM was not invented for
| you. Few companies have the scale that makes Erlang a
| reasonable choice.
|
| More pressing these days I believe is that the Erlang ecosystem
| is all or nothing, the BEAM is like its own little operating
| system and has extremely (IMHO) complicated setup and
| configuration with many moving parts: ERTS, epmd, Rebar,... You
| don't need or shouldn't use container tech and k8s with Erlang
| because it's already doing that stuff in its own Erlang way -
| but it's the Erlang way it's extremely unique and unfamiliar to
| most.
|
| When you have the right use case for Erlang and you see it in
| action it is truly like black magic, it's been a highlight of
| my career to work with it.
| borromakot wrote:
| Elixir has been quite easy to set up, configure and deploy,
| at least for the last 5-6 years. I can't speak for Erlang
| though.
| throwawaymaths wrote:
| elixir softens a whole ton of the sharp edges around getting
| started and there are only a handful of "gotchas" coming from
| other hlls (like all terms being immutable when you pass to a
| function -- but thats honestly almost a problem with the
| "mainstream" languages).
| ludicity wrote:
| We swapped our Squarespace website out for a Phoenix
| application yesterday and it was such a delightful experience.
| schonfinkel wrote:
| Learning to work on the BEAM with Erlang/OTP has been one of the
| best experiences I had in the past year, I used Joe Armstrong's
| "Programming Erlang: Software for a Concurrent World" book (and
| highly recommend it to beginners), I also heard good things about
| "Designing for Scalability with Erlang/OTP" (but didn't start
| this one yet), but nothing comes close to this in terms of depth.
|
| The BEAM really feels like alien tech left from a highly advanced
| civilization and this book dropped in a such a great timing!
| Bought it right away, kudos to Dr. Erik Stenman for keeping it up
| after two cancellations!
| da02 wrote:
| What kind of software applications do you develop with
| BEAM/Erlang/OTP?
| schonfinkel wrote:
| Right now they've been mostly toy projects, the more complex
| one being the back-end of an (WIP) MMO game
| (https://github.com/Dr-Nekoma/lyceum) and a couple handy
| libraries that I wrote myself (like https://github.com/dont-
| rely-on-nulls/migraterl).
| fennecfoxy wrote:
| Oh I thought this was gonna be about BEAM as in
| https://smfr.org/robots/ ha ha.
| einpoklum wrote:
| I've not used Erlang before (and naturally have not read the BEAM
| book); I mostly live in the world of C++ (and CUDA/OpenCL) these
| days. Would this be of interest to me?
| sgarland wrote:
| TIL that Klarna runs on Erlang.
| bicx wrote:
| Elixir and BEAM are my all-time favorite choice for building
| networked or pipeline-heavy systems. I used it every day in
| production for several years, and while it's a hard sell for
| recent projects (and generally the wrong call), I enjoy keeping
| up with it as much as I can.
|
| Thank you for writing this book! I really wanted this a few years
| ago as I was debugging production Elixir, but existing learning
| sources were pretty dense and dry (or too simple and shallow).
___________________________________________________________________
(page generated 2025-06-04 23:00 UTC)