[HN Gopher] Typing lists and tuples in Elixir
___________________________________________________________________
Typing lists and tuples in Elixir
Author : idmitrievsky
Score : 175 points
Date : 2024-08-28 11:49 UTC (11 hours ago)
(HTM) web link (elixir-lang.org)
(TXT) w3m dump (elixir-lang.org)
| munchler wrote:
| F# has both a `head` and `tryHead` function to handle lists that
| may or may not be empty. In general, `tryFoo` is a good pattern
| for naming functions that might fail.
|
| Having a separate NonEmptyList type might seem like a good idea
| in theory, but in my experience, it leads to code that is
| significantly more complicated.
| truculent wrote:
| How does it make it more complicated?
|
| In my view, you're moving the potential for failure to a
| different place (the constructor), rather than changing some
| fundamental property or introducing new complexity.
|
| Is it handling the construction of these types you find
| complicated? And is it simply not worth the guarantees?
| greener_grass wrote:
| The intuitive definition of NonEmptyList is:
| type NonEmptyList<'t> = 't * 't list
|
| But this cannot be passed to any function that expects a
| List<'t>. This is odd though, since intuitively, all non-
| empty lists are lists.
|
| See Rich Hickey's "Maybe Not" talk
|
| OOP solution is to use inheritance. Typical ML solution is to
| use type-classes.
|
| F# sits in an awkward middle-ground where neither is a
| perfect fit.
|
| I believe that dependently-typed languages solve this more
| elegantly.
|
| There's also the syntactic inconvenience of wrapping at
| construction, where in theory the compiler could figure it
| out for you.
|
| For example: let xs = 1 :: 2 :: 3 :: []
|
| Here xs is non-empty, but we must tell the compiler:
| let xs = 1, 2 :: 3 :: []
|
| TypeScript does a better job here (although a great cost!)
|
| What you end up with is massive code duplication or lots of
| extra function calls: xs |>
| NonEmptyList.toList |> List.map (fun x -> x + 1)
| |> NonEmptyList.unsafeFromList
|
| _(I say this all as someone who really likes F#)_
| munchler wrote:
| Yes, this is exactly what I was getting at. One ends up
| needing an explicit `toList` helper function that converts
| a non-empty list into a plain list.
|
| Pattern matching on a non-empty list is also inelegant,
| because it is implemented as a tuple, which creates a leaky
| abstraction.
| greener_grass wrote:
| F# actually does have a good solution for pattern
| matching here, which is Active Patterns.
| josevalim wrote:
| We (Elixir) would rather define lists on top of non empty
| lists: list(a) = empty_list() or
| non_empty_list(a)
|
| So you should pass non-empty lists everywhere a list is
| expected. But you can't pass a list where a non-empty one
| is expected.
|
| But overall, you are right: our concern is exactly all of
| the extra function calls that may now suddenly become
| necessary (and the tension mentioned in the article). We
| will review our design decisions as we keep on rolling out
| the type system!
| yawboakye wrote:
| for when one expects a list to be non-empty, i think
| there's strong argument in favor of an enforcement from
| the type checker, given that the prove will very likely
| be necessary. if not in the application code then in the
| tests.
| greener_grass wrote:
| Does this break down if we want some other types?
|
| e.g. list(a) = empty_list() or
| singleton_list(a) or two_or_more_list(a)
| jerf wrote:
| "OOP solution is to use inheritance. Typical ML solution is
| to use type-classes."
|
| Yes, type classes can "work" to help a NonEmptyList
| degenerate to a normal List of some sort, if the function
| accepting the list accepts the type class instead of a
| hard-coded List. Unfortunately, at least for this exact
| task, taking hard-coded types is pretty common. I've
| sometimes wondered about the utility of a language that
| provided all of its most atomic types solely as typeclasses
| within its standard library, so that calling for a "List a"
| or "[a]" automatically was turned into the relevant type
| class.
|
| Inheritance doesn't actually work here. I assume you mean
| inheriting a NonEmptyList from some sort of List, from the
| perspective of a user facing a language that has a standard
| List and they want to create a NonEmptyList that things
| taking List will accept. Unfortunately, that is a flagrant
| violation of the Liskov Substitution Principle and will
| create architecturally-fragile code.
|
| Compilers can't enforce the LSP (with anything short of the
| dependently typed code you mention), so you can bash out a
| subclass that will throw an exception if you try to take
| the last element out of a NonEmptyList or violate the rules
| some other way, and if you pass your new NonEmptyList to
| something that _happens_ to not do anything broken, you may
| get away with it, but by the standards of OO theory you 're
| definitely "getting away" with something, not solving the
| problem.
|
| I haven't studied this extensively beyond just thinking
| here for a moment, but I don't think you can LSP-legally go
| the other way either. A subclassed List can't revoke a
| parent's NonEmptyList property that the list is guaranteed
| to not be empty. Again, you can bash the methods into place
| to make it work, but as this is a very basic standard
| library sort of thing for a language it really needs to be
| right.
|
| Edit: Yes, it's certainly illegal. You can take a List
| inherited from the NonEmptyList, have it be empty, but you
| have to be able to pass it to something accepting a
| NonEmptyList, but it will then be empty. So you can't LSP-
| legally inherit either way.
|
| (This is one of the "deep reasons" why inheritance is
| actually not a terribly useful architectural tool. It
| technically breaks really, really easily... like, probably
| most non-trivial uses of inheritance in most code bases is
| actually wrong _somehow_ , even if never happens to
| outright crash. We tend to just code past the problem and
| not think about it too hard.)
| greener_grass wrote:
| > You can take a List inherited from the NonEmptyList
|
| Shouldn't it go the other way?
|
| All NonEmptyLists are Lists, but not all Lists are
| NonEmptyList.
|
| So NonEmptyList inherits from List
| jerf wrote:
| Neither direction works. More directly (since I was
| working it out as I typed above):
|
| A NonEmptyList promises that its .Head method will always
| produce a value. An inherited List can not maintain that
| property, it must add either an error return or a
| possible exception (which is the same thing from this
| point of view), and so violates the LSP.
|
| A List promises that if it has an element, you can remove
| it and have another List, whether by mutation or
| returning a new List. A NonEmptyList breaks that promise.
| If that sounds like a "so what", bear in mind that
| "removing an element" includes things like a "Filter"
| method, or a "split" method, or any of several other such
| methods beyond just iteration that a List is likely to
| have that a NonEmptyList is going to need a different
| type signature and/or exception profile to implement
| properly.
|
| You could define a bare-bones superclass for both of them
| that allows indexing, iteration, appending, and a length
| method, without much else, and that does work. However,
| if you start trying to get very granular with that,
| across more data structures, you'll start to need
| multiple inheritance and that becomes a mess really
| quickly. There's a reason that, for instance, the C++ STL
| does not go the "inheritance hierarchy" route for this
| stuff.
|
| Like I said, inheritance done _properly_ is really
| restrictive. We often do a lot of sweeping under the rug
| without even realizing it, and that "works" but it still
| eats away at the architecture, all the more so if the
| people involved don't even realize what they are doing.
| ryangs wrote:
| But the nonempty list never has an element, so we don't
| need to worry about the type mutation of removing an
| element from it. Filter just returns a nonempty list.
| jerf wrote:
| "But the nonempty list never has an element, so we don't
| need to worry about the type mutation of removing an
| element from it."
|
| NonEmpty _always_ has an element.
| greener_grass wrote:
| NonEmptyList<'t> has a method Uncons that returns 't *
| List<'t>
|
| List<'t> has a function TryUncons that gives Option<'t *
| List<'t>>
|
| List<'t> does not have a method Uncons.
|
| We can define TryUncons for NonEmptyList<'t> in terms of
| Uncons, specifically: this.TryUncons()
| = this.Uncons() |> Some
| jerf wrote:
| I didn't spell it out adequately, only implied it, but I
| am talking about a fully loaded List, not one that just
| barely works, e.g., it has filter, it has all the other
| things you'd expect from a List. I did discuss the "just
| barely works" case at the end.
|
| It would be an odd "object oriented language" list that
| lacked such things, e.g., https://docs.oracle.com/javase/
| 8/docs/api/java/util/List.htm... . We're in OO land here,
| not FP land.
| sdeframond wrote:
| > A List promises that if it has an element, you can
| remove it and have another [..]. A NonEmptyList breaks
| that promise.
|
| No. Removing an element from a NonEmptyList returns a
| List. LSP is respected when NonEmptyList is a List.
| derriz wrote:
| Why not in the other direction? If NonEmptyList inherits
| from List, then head (and tail) would be methods of
| NonEmptyList but not of List.
| Volundr wrote:
| > A List promises that if it has an element, you can
| remove it and have another List, whether by mutation or
| returning a new List. A NonEmptyList breaks that promise.
|
| I don't follow. Remove the head from a NonEmptyList and
| the tail will be a List. It might not be a NonEmptyList,
| but that's not the contract of List.
| aloisdg wrote:
| As a fellow F# dev currently learning Elixir, this kind of
| thing is a burden
| bmitc wrote:
| Just to clarify for the crowd though. In F#, `List.head` throws
| an exception when it fails whereas `List.tryHead` returns an
| `option`, which returns `None` when it fails instead of an
| exception.
|
| A general confusion of mine in Elixir is generally how
| libraries and functions treat errors. There's the common idiot
| of returning either `{:ok, ____}` or `{:error, ____}`, but what
| can be inside the error tuple is not always clear. The other
| thing is that sometimes a function can both throw an exception
| and also return a success tuple. Such cases are confusing to
| handle, and there's a large gap between handling cases like
| that and the philosophy of "let it crash", which I think is
| preached a little looser than it should actually be practiced.
|
| I do like F#'s way of disambiguating the two situations. The
| only issue I have in F#, which actually exists in every
| language that I know of that has exceptions, is that there is
| no way to know, up front and clearly, what exceptions can be
| thrown by a given function. This is particularly frustrating in
| F#, which has fantastic pattern matching for exceptions. I wish
| there was exhaustive pattern matching in F# for exception
| handling, such that it would warn you that you have an
| unhandled exception in a try/with expression
| (https://learn.microsoft.com/en-us/dotnet/fsharp/language-
| ref...) but of course would allow for wildcard patterns.
| IshKebab wrote:
| Java has checked exceptions and they are used, e.g. in
| Android. But it seems to be the exception (heh) rather than
| the rule.
|
| IMO the problem is _proper_ exception handling with checked
| exceptions and wrapping each function (or at least small
| blocks) in try catch is just so insanely verbose that even
| though it is _possible_ to get error handling as good as
| something like Rust, nobody actually does it in practice.
| ku1ik wrote:
| I really respect Elixir core team's approach to adding gradual
| typing to the language. They don't rush it. They didn't put too
| much focus on syntax so far (I'd argue the syntax in many cases
| is less important than foundations) and instead they focused on
| soundness of the system. With each new Elixir version the
| compiler is getting smarter, catching more bugs. Not hugely
| smarter, but smarter enough that I feel safer. Looking forward to
| Elixir 1.18!
| jonnycat wrote:
| I'm not sure... I'm a huge Elixir fan and I trust Jose to build
| a great solution, but I've found the rollout to be a bit
| confusing. There was the announcement that "Elixir is now a
| gradually typed language" prior to 1.17 - but it seems that
| most of the changes were behind the scenes, and 1.17 largely
| didn't expose user-facing type errors or warnings.
|
| Again, I definitely trust them to get it right in the long
| term, but in the meantime, the progress has been a bit
| confusing to me.
| sodapopcan wrote:
| The wording was a little odd, but there are certainly user-
| facing errors in 1.17, namely:
|
| - Map keys (called with '.') are checked at compile time.
|
| - Using comparison operators with different types causes a
| warning.
|
| I may be forgetting something.
| josevalim wrote:
| Thanks for vote of confidence!
|
| We need to type every data type and every function, so the
| type system will be rolled out over a long period of time.
|
| The 1.17 release meant that we now have a gradual type
| system, which runs in every code being compiled, but it only
| supports a handful of types (including the dynamic one). The
| full list of supported types and examples of typing
| violations it can now detect is on the announcement:
| https://elixir-
| lang.org/blog/2024/06/12/elixir-v1-17-0-relea...
|
| There is no support for type annotations, that comes in a
| later stage. The overall stages have been described in an
| earlier article (and I believe also in the paper):
| https://elixir-lang.org/blog/2023/06/22/type-system-
| updates-...
| klibertp wrote:
| > but it seems that most of the changes were behind the
| scenes, and 1.17 largely didn't expose user-facing type
| errors or warnings.
|
| That's how it normally goes with gradual type systems for
| existing languages, I think. The first step seems to be
| almost always adding a type checker that doesn't do anything
| in particular other than handling untyped code. Since being
| able to handle untyped code makes a type system gradual,
| announcing Elixir as "gradually typed" when this milestone is
| reached seems justified. After that, you're free to improve
| the type system and type checker(s), improve type inference,
| add specialized syntax, improve typed/untyped interactions,
| cover more language patterns, and so on. MyPy for Python also
| started without support for many things that were added later
| (and it's still being actively developed ten years later).
| btbuildem wrote:
| I've always viewed Erlang (and by extension, Elixir) as safe from
| these types of improvements. Maybe I don't have enough experience
| in these languages, but having guards, and the different approach
| to conditionals seemed to do away with most of the pitfalls
| usually "safeguarded from" by the caution tape and excessive road
| signage of type systems.
|
| I'm curious to learn more, but I can't shake a feeling of vague
| trepidation here.
| rkangel wrote:
| Erlang and Elixir have long had "dialyzer" as a type checker.
| The problem is that it had some severe limitations in approach
| (and implementation) and so the cost/benefit ratio wasn't
| great.
|
| This article really speaks to how they're thinking about the
| costs of the type system, so that you mostly get benefit, and
| that's great.
| 29athrowaway wrote:
| Erlang is an influential language, it has its uses.
|
| But Erlang and by extension Elixir are a hard sell unless you are
| writing a system like Whatsapp.
| SoftTalker wrote:
| Erlang/OTP is the nicest environment for building applications
| that I've ever experienced. I agree though that it's not
| popular and I've never actually been paid to use it, with the
| exception of some small projects that were completely under my
| control.
| scop wrote:
| I build web apps. Basic CRUD stuff with a little bit of
| business logic sprinkled in. Elixir/Phoenix has been an
| absolute pleasure to use.
| doakes wrote:
| Shouldn't be a hard sell to developers. According to this
| year's StackOverflow survey, Phoenix is by far the most admired
| web framework (10% above the second most popular). Elixir is
| the second most admired language, behind Rust.
|
| https://survey.stackoverflow.co/2024/technology#2-web-framew...
| insane_dreamer wrote:
| The questions in that survey are poorly worded so as to make
| the results ambiguous, since they confound 1) languages in
| which X person has worked with in the past year, with 2)
| languages in which X person would like to work on next year.
| These express two very different concepts: 1) popularity in
| the workplace, 2) developers' personal preferences. (Neither
| of them have to do with the most "admired".
| doawoo wrote:
| I disagree entirely. Having written everything form little
| tools for my own use to entire linux based firmware for small
| embedded machines, Erlang and Elixir are powerful tools. I
| genuinely find Elixir fun to write in a way no other language
| manages to match.
| lawik wrote:
| Heck, I script with Elixir like it was Python.
| insane_dreamer wrote:
| That's because its inspiration was to create a language with
| the power of Erlang but that was as enjoyable to code in as
| Ruby (many people's "most fun to code in" language if they've
| actually worked with it; unfortunately it can't compete with
| Python's ecosystem).
| klibertp wrote:
| Erlang and Elixir have a pretty good interop with Python
| through Erlport. It's not a deep language-level integration
| (as in, inheriting modules from Python classes or something
| like that), but it implements a binary message format
| Erlang nodes use to communicate and offers an API for IPC
| with Python on top of that. It's not for all use cases, but
| I had great success deploying a fleet of Raspberries for
| home automation using nothing but Ansible, Elixir, and
| Python. Elixir would handle distribution and manage
| messaging between nodes and local Python processes, while
| Python scripts would control peripherals. I stopped
| developing it since I moved to a smaller place, but I
| firmly believe Elixir+Python is a great solution for
| anything distributed that would benefit from the Python
| ecosystem.
| insane_dreamer wrote:
| the big(gest?) advantage that python has are its data
| science and ML/DL packages
| sbuttgereit wrote:
| I agree. However, the reasoning behind that hard sell has less
| to do with actual technical considerations and more to do, I
| believe, with a combination of long held assumptions (not
| always right assumptions) about what Erlang is and the fact
| other other stacks have become entrenched and are good enough
| in the ways that are easiest to assess that Erlang/Elixir would
| be a tough sell. Add to that recruiting talent to work with
| Erlang/Elixir requires some outside the box thinking in HR
| hiring practices (something I rarely accuse HR departments of)
| and you have a tough sell.
|
| This is all unfortunate. I do think the BEAM based stacks are
| underrated for applications outside of communications. Many of
| the same traits of resilience and resource utilization designed
| to facilitate communications systems actually apply to web apps
| and APIs, too. Elixir is very expressive and a good fit for
| writing business applications like accounting related software
| (what I'm currently working on). But... you have to think you
| can arbitrage those operational advantages into an overall
| competitive advantage... and that's a tough sell especially
| because it involves a lot of speculation which doesn't play out
| until you're getting to the end of a project.
| hmmokidk wrote:
| L take
| mikhmha wrote:
| I don't understand the reasoning? I've been working on an
| MMORPG game since quitting my job - and I settled on using
| Elixir for this project despite never using it before. I have a
| good understanding of distributed systems, and the
| features/tooling Erlang & Elixir provided were like a dream for
| me. Initially I thought I'd just try making some proof of
| concept thing. Fast forward several months later, and I
| seriously credit Elixir for how far I've gotten with this game.
| Most of my time is spent writing server-side gameplay code, not
| tracking down obscure networking & memory bugs. Its even caught
| cascading bugs caused by gameplay system interactions - when
| players and a.i were being (wrongly) resurrected from the dead
| the system just crashed! When 500 a.i agents doing pathfinding
| every single tick was starting to make the a.i system lag and
| delay inputs - it was trivial to understand the bottleneck
| causing the system to degrade. I can go on and on.
| pdimitar wrote:
| Please go and on! If you write a blog post I promise you it
| will have an ardent following and a very warm reception,
| starting with yours truly.
| mike1o1 wrote:
| I disagree, and I think this sentiment is due to a failure in
| messaging from the Elixir core team and community which is a
| shame. There is a huge focus on LiveView (which is fantastic)
| in the Elixir community, but Elixir is so much more than
| LiveView! Even just writing a simple rest API (or GraphQL
| through Absinthe) is a great experience, due to libraries like
| Phoenix and Ecto. It's a joy to work with, even if you aren't
| writing a distrubuted/realtime system.
|
| I wish there was more of an embrace in the broader ecosystem
| rather than the focus on LiveView.
___________________________________________________________________
(page generated 2024-08-28 23:00 UTC)