[HN Gopher] Caramel: An OCaml for the Erlang VM
___________________________________________________________________
Caramel: An OCaml for the Erlang VM
Author : alokrai
Score : 122 points
Date : 2021-03-05 07:48 UTC (15 hours ago)
(HTM) web link (caramel.run)
(TXT) w3m dump (caramel.run)
| jaegerpicker wrote:
| I loved working with the Beam VM and Elixir. I just wish it had a
| stronger presence in the Data/ML/AI fields. As scalable and
| trivially parallizable as Elixir is with a fast
| SciPy/NumPy/Pandas type of eco-system and you'd have a fantastic
| environment for data heavy apps IMO.
| a_bored_husky wrote:
| Have you seen project Nx?
|
| https://github.com/elixir-nx/nx
| jbott wrote:
| Have you seen nx[1]? It looks like the first step towards this!
|
| 1. https://dashbit.co/blog/nx-numerical-elixir-is-now-
| publicly-...
| joisig wrote:
| Take a look at https://github.com/elixir-nx/nx
| macintux wrote:
| Obligatory list of alternative Erlang VM languages:
|
| https://gist.github.com/macintux/6349828#alternative-languag...
| jimbokun wrote:
| Knowing nothing else about this project, it falls in the category
| of things about which I thought "Wouldn't it be cool if
| somebody..."
| toolz wrote:
| being unable to do dynamic dispatch seems like a big show-stopper
| for distributed/meshed apps, which seems to be the only reason
| for targeting the BEAM to begin with, no?
|
| If you don't pass functions around as MFAs in a distributed app
| you run into trouble with upgrading your cluster as they'll have
| different versions of a function during a rolling deploy
| jolux wrote:
| Different organizations use different amounts of the tools the
| BEAM provides in building their distributed applications. We
| containerize everything and don't really use the hot upgrade
| features.
| toolz wrote:
| sure, but I'm not talking about hot upgrades, I'm talking
| about upgrading entire nodes that exist in a cluster. The
| nodes will not be able to pass around functions as an
| upgraded node in a cluster will have a different version of
| that function. So this is an issue without considering hot
| code reloading, as I don't use that feature either.
| jolux wrote:
| I don't quite follow how static types change this
| situation?
| yawaramin wrote:
| So we're not talking about hot code reloading here, we're
| talking exactly about upgrading nodes in a cluster. Many--
| I'd say most--Erlang/Elixir deployments today don't do
| that. They use Kubernetes.
| toolz wrote:
| Kubernetes does not solve the issue, it merely has
| strategies for how to upgrade nodes. The issue is that
| when you compile new code, even if the function didn't
| change at all, it will be a new version of the function.
| If the upgraded node then sends a function to another
| node to be executed (as is a very common practice in
| meshed OTP apps) the older node will not be able to
| execute the function.
|
| Now, I understand that most erlang apps still don't
| utilize clustering and instead are stateless apps and
| that's fine, but ocaml can already do stateless apps just
| fine, right? So my assumption here is that they're
| putting ocaml on the BEAM to utilize OTP and clustered
| apps, correct? If that's the case I don't see how you
| could have a real-world app that you can quickly scale
| up/down (which is something you often want in a clustered
| app) - instead it seems like you'd be forced to spin up
| entirely new clusters and then direct traffic to the new
| cluster to bypass this problem. This of course isn't a
| viable tradeoff for very large clusters.
| derefr wrote:
| I think what the parent is saying isn't that most
| production Erlang apps aren't clustered; it's that most
| production Erlang apps aren't clustered _using the
| distribution protocol_. Instead, they 're clustered at
| the application layer, with explicit wire protocols like
| gRPC. You can upgrade nodes within such clusters just
| fine, because the ABI of the wire protocol is an explicit
| part of the application, something that can be stabilized
| separately from the app itself, rather than being an
| implicit property of what ERTS version you're using, or
| about how the application's processes communicate
| internally.
|
| Note that even in the context of Ericsson, Erlang's
| distribution protocol was never designed for the use-case
| of horizontal N-node clustering. It was designed for
| static-role clustering -- where each node is like an
| organ in your body, with a name and a specific function
| relative to the "system" that is your body. For example,
| "master" and "warm standby" roles. (By analogy to
| Kubernetes, the Erlang nodes of a distributed-OTP
| architecture are closest to being like the sibling
| _containers_ of a single k8s _pod_. Except that Erlang 's
| "containers" can be running on separate machines while
| still being part of the same "pod".)
|
| If you want to scale such a system (a "pod" of nodes),
| you are supposed to spawn N copies of _the entire
| distribution set_ ; and then those N system-copies will
| coordinate with other such clones not via the
| distribution protocol, but via explicit coordination
| protocols. (Sometimes this explicit coordination protocol
| is designed to use the distribution protocol as a
| _carrier_ -- this is one of the reasons that Erlang
| supports making manual distribution-protocol connections
| between nodes, rather than forcing a connected topology
| on you. But such connections carefully avoid passing
| arbitrary RPC data across them, instead usually having an
| explicit "peer server" on both ends, where the servers
| speak a limited and ABI-stable term protocol to one
| another.)
|
| Yes, the Erlang distribution protocol has been repeatedly
| taken beyond its design tolerances to accomplish
| horizontal scaling. CouchDB does this, for example. But
| many Erlang applications that you might think of as doing
| this, are actually much closer to static "organs in a
| body" architecture than you'd think, despite claims of
| scalability. Riak and Ejabberd, for two examples, both
| treat their nodes very much like static organs, rather
| than a fleet.
| yawaramin wrote:
| > Kubernetes does not solve the issue, it merely has
| strategies for how to upgrade nodes. The issue is that...
|
| I understand the issue, Gleam/Caramel/et al. (static
| typing layers on BEAM) are not trying to solve this
| issue.
|
| > So my assumption here is that they're putting ocaml on
| the BEAM to utilize OTP and clustered apps, correct?
|
| That's actually not my assumption. I'm assuming that the
| point of Caramel is to get sound static typing for (some
| of) the code that we are deploying on the BEAM. It
| doesn't necessarily have to be for clustered apps. As I
| said, many deploys of even Erlang and Elixir today are
| not using clustered nodes. They don't care about those
| capabilities. They use Kubernetes to manage nodes and
| state.
|
| > instead it seems like you'd be forced to spin up
| entirely new clusters and then direct traffic to the new
| cluster to bypass this problem.
|
| This is exactly what Kubernetes (or Nomad etc.) will let
| you avoid--they can spin up and spin down nodes within
| the same cluster.
| jolux wrote:
| As I understand it, there are real questions about the
| type theory needed to correctly express what the BEAM
| does, and they have yet to be answered sufficiently.
| Nonetheless, a lot of people clearly want static types in
| their BEAM applications, and there are certainly parts of
| any application that don't cross the network. I'm
| interested in efforts like Caramel and Gleam mostly
| because of my interest in BEAM as a general-purpose
| platform, and because I want to see if they come up with
| solutions to the type theory problems.
| pharmakom wrote:
| The creator of Erlang said that static typing was not appropriate
| for the applications Erlang is designed for. Worth keeping this
| in mind!
| StreamBright wrote:
| I don't think he ever said such a thing. Do you have the source
| of this claim?
|
| Some back story:
|
| "Through the years, there were some attempts to build type
| systems on top of Erlang. One such attempt happened back in
| 1997, conducted by Simon Marlow, one of the lead developers of
| the Glasgow Haskell Compiler, and Philip Wadler, who worked on
| Haskell's design and has contributed to the theory behind
| monads (Read the paper on said type system). Joe Armstrong
| later commented on the paper:
|
| One day Phil phoned me up and announced that a) Erlang needed a
| type system, b) he had written a small prototype of a type
| system and c) he had a one year's sabbatical and was going to
| write a type system for Erlang and "were we interested?" Answer
| --"Yes."
|
| Phil Wadler and Simon Marlow worked on a type system for over a
| year and the results were published in [20]. The results of the
| project were somewhat disappointing. To start with, only a
| subset of the language was type-checkable, the major omission
| being the lack of process types and of type checking inter-
| process messages."
|
| https://learnyousomeerlang.com/types-or-lack-thereof
| Twisol wrote:
| And I think that speaks to the current state of type system
| research on communicating processes, not to any fundamental
| inability thereof. Session types are a recent approach
| (though one that I don't think goes far enough; give me full-
| duplex channels!)
| pharmakom wrote:
| sorry I can't remember where I read it.
|
| one quote I found:
|
| Q: "Erlang is currently a dynamically typed language. Are you
| saying you would like some static type as optional typing or
| both, or would you change the nature of Erlang to static
| typing?"
|
| A: "No, I wouldn't change it, but I would like subdomains
| where it is statically typed. You could actually encapsulate
| parts in statically typed."
|
| So it sounds like he stands by dynamic as the correct choice
| for Erlang, although not all of the time.
| jasone wrote:
| Erlang APIs are designed around the availability of dynamic
| pattern matching on messages, but static message typing is
| definitely feasible. The only feature of Erlang that stands out
| me as fundamentally incompatible with static typing is "live
| upgrade". And although this feature might be critical in the
| context for which it was designed (Ericcson telecom switches),
| it appears that very few Erlang-based deployments use live
| upgrade; the most common practice is to restart services.
| toast0 wrote:
| I'm not that up on static typing, but it would seem to me
| that static typing would be difficult with different
| versioned nodes on dist as well as hot upgrade.
|
| While a lot of users avoid hot load and restart/replace nodes
| instead, I don't think many restart/replace the whole
| cluster.
| dnautics wrote:
| > different versioned nodes
|
| Exactly. Blue-green deploys could get seriously messed up
| if the static typing system doesn't strongly think about
| this.
| may4m wrote:
| I was looking at elixir and F# for my next project. I liked F#
| for the type system and elixir for OTP/Beam. This looks like
| something I wanted
| bmitc wrote:
| You might also be interested in Gleam.
|
| https://gleam.run/
|
| Is it possible to share any details about your project? I'm
| just curious since I also use F# and am learning Elixir.
| devmunchies wrote:
| Does gleam let me destructure a list so that I can bind the
| head and body each to a variable?
|
| e.g. in f# let splitHeadAndTail = function
| | h :: [] -> printfn "got head and EMPTY tail" | h
| :: t -> printfn "got head and tail" | [] -> printfn
| "got empty list"
|
| The example in their docs does a 1:1 map and doesn't show if
| you can bind the whole tail[1]
|
| [1]: https://gleam.run/book/tour/case-
| expressions.html#destructur...
| derefr wrote:
| Yes; see https://hexdocs.pm/gleam_stdlib/gleam/list/. The
| syntax is [head, ..tail]
|
| It'd be kind of silly to create a greenfield BEAM language
| that doesn't expose one of the most fundamental operations
| you can do in BEAM bytecode, and that every list-processing
| function ultimately relies upon :)
| devmunchies wrote:
| > It'd be kind of silly to have a BEAM language that
| doesn't expose one of the most primitive operations you
| can do in the BEAM, that every list-processing function
| ultimately relies upon :)
|
| Agree, which is why i was confused it wasn't in the tour
| I linked to in the Lists or Destructuring sections
| centimeter wrote:
| Why not just use Haskell? You don't get OTP/Beam, but the
| GHC runtime provides almost all the same functionality
| (plus a lot more), to the point where the Cloud Haskell
| library is basically a full clone of Erlang's actor
| functionality.
| cultofmetatron wrote:
| 2 years ago, I went all in with elixir/phoenix with my startup.
| Happy to see a language like this which appears to handle
| integration with the beam ecosystem as a first class
| consideration. I can see this being useful for some parts of the
| app where I need to integrate with external services.
| praveenperera wrote:
| If you're looking for a typed BEAM language with great interop
| with Elixir and other BEAM languages, have you seen Gleam?
|
| https://github.com/gleam-lang/gleam
___________________________________________________________________
(page generated 2021-03-05 23:01 UTC)