[HN Gopher] Metaprogramming in Elixir
___________________________________________________________________
Metaprogramming in Elixir
Author : vroccolli
Score : 109 points
Date : 2021-05-21 09:15 UTC (1 days ago)
(HTM) web link (serokell.io)
(TXT) w3m dump (serokell.io)
| claudiojulio wrote:
| I don't like the operators' logic in elixir. There should be one
| for assigning variables, one for pattern matching and there is no
| rebinding of variables.
|
| Could be like that: // assigning x = 1
| // pattern matching x ^ 1 // pattern
| matching and assigning x =^ 1
|
| And they should remove the rebinding that is useless. But I find
| it very difficult for that to happen at that time.
|
| _Automatically translated._
| [deleted]
| lessthanseventy wrote:
| https://elixir-lang.org/getting-started/pattern-matching.htm...
| dmitriid wrote:
| Having had experience with Erlang, I still find it weird that
| you can update variables :)
| spapas82 wrote:
| I think that the metaprogramming situation is a little
| problematic in Elixir. Elixir makes heavy use of macros in basic
| libraries like the Phoenix or Ecto frameworks that are used by
| more or less everybody.
|
| I understand that macros are useful to build a DSL for a specific
| project but using macros in basic libraries like this leads to
| "way too much magic": Trying to understand (or somehow extend)
| Ecto is not really possible for all practical purposes and
| following the control flow in Phoenix is like a maze because of
| all the macro substitutions. This also leads to cryptic errors
| where you get an error in non existant lines of code.
|
| This heavy of macros is my main complain about Elixir and I think
| it's one of the main problems that new people face when trying
| the language resulting in either not using it at all (or if they
| start it they will abandon it).
|
| Finally, I understand that using macros in these libs allows for
| much more compact (and maybe beautiful) code, however I will
| always pick a more verbose (but less magic) version of this if I
| had the choice. I think this is the case for most programmers
| that want to use a language as a tool to do their job.
| dqv wrote:
| I see this criticism a lot but I don't think it has anything to
| do with macros specifically and more so to do with lack of
| familiarity with Elixir. I've felt the same way about Django
| being magic because I had trouble following the class
| hierarchy. It makes a lot more sense now because I'm more
| familiar with Python and Django. But even today I'll be looking
| deeper at something and ask WTF it's doing. In that respect,
| Elixir codebases are easier to me. The module depth seems
| "shallower" and I don't have to disambiguate between what
| behavior is caused by class inheritance or an imported
| function.
|
| When I first tried to use Elixir several years ago Ecto.Schema
| [0] seemed complex and magical, but then I came to realize it's
| just converting module attributes to runtime code. There is not
| really that much complex macro logic going on.
|
| >This also leads to cryptic errors where you get an error in
| non existant lines of code.
|
| When was the last time you used Elixir? This isn't a problem I
| can recall having in the last 4 years or so of using Elixir.
|
| >following the control flow in Phoenix is like a maze because
| of all the macro substitutions.
|
| Can you clarify what you mean by this? A specific case as to
| where this happened for you would help. Phoenix's use of macros
| is actually pretty light [1] except for some very low level
| stuff. You can even see how frequently a developer will use
| macros in Phoenix by searching `__using__` in the codebase [2].
| It's not used as much as people think. The majority is for
| views and controllers and only to provide a very thin layer of
| support on top of your regular use of code. As an example, the
| "macro magic" in Phoenix.Controller is just handling some
| basics for giving a layout and view to Plug and handling
| fallback actions for exceptions. You could do the plug calls
| manually and I think it would be safe to not use any macros in
| your controller code.
|
| Another familiarity issue with the language (and any language
| really) is understanding what is meaningful in a stack trace
| and what isn't. And the likely cause of the error in the first
| place. Is it syntax? Is it mistyping a variable? Is a function
| just used improperly? (wtf is init_p_do_apply and why does it
| show up in every stacktrace?) You're juggling all these
| different issues - learning a new paradigm, a new syntax, not
| knowing how to extend things. It's obviously going to be a
| little overwhelming and, if not strictly required, we might
| just pick a different language that we're more familiar with.
|
| [0]: https://github.com/elixir-
| ecto/ecto/blob/master/lib/ecto/sch...
|
| [1]:
| https://github.com/thechangelog/changelog.com/blob/master/li...
| (not mine, I just go here to show the most frequent use of
| macros in Phoenix)
|
| [2]:
| https://github.com/phoenixframework/phoenix/search?q=__using...
| cuddlecake wrote:
| Not sure if I would consider Phoenix basic. As far as I know,
| it's a wrapper around (and configurator for) Cowboy and Plug. I
| would consider Cowboy basic and Phoenix advanced.
| natrys wrote:
| Macros can be de-mystified with proper editor support. In Emacs
| and Lisp you can (recursively) use "macroexpand" to see what
| code it produces in-place. I am not an Elixir user (though
| thinking about hopping on), apparently there is alchemist.el[1]
| that lets you do that (with caveat, and the project seems dead
| tbh).
|
| [1] https://github.com/tonini/alchemist.el
| lukaszsamson wrote:
| Macro expansion has recently been added to elixir-ls[1] and
| is supported via custom command in vscode extension.
|
| [1] https://github.com/elixir-lsp/elixir-ls
| dnautics wrote:
| Completely disagree with ecto[0] and plug. (I have some issues
| with ecto, but it has nothing to do with macros). The use of
| macros in those two libraries is parsimonious and exactly what
| you want in a DSL. For that matter ExUnit, too, has exactly the
| 'right' amount of macro usage. Phoenix, on the other hand,
| drives me up the wall.
|
| I would not call Phoenix a "basic library". It's definitely a
| heavy framework. You can of course use "just Plug" (though very
| few people do).
|
| Finally, I think the bigger problem is when you get devs that
| "take inspiration" from some of the way that Phoenix uses
| macros. It's a serious problem when a dev has built a macro
| that cleverly autogenerates a function call (so you can't
| search the codebase for that function to chase the call path)
| is not fun. This is the strategy that the Phoenix Routes (is
| that the right name? not to be confused with Router) uses. I
| hate it so much.
|
| [0] as an ecto consumer. if you ever try to develop an adapter
| for ecto, you'll run into these problems. I think for example
| this is why it took forever for someone to build an ecto3
| SQLite adapter.
| tokamak-teapot wrote:
| Yes I never got into Rails for perhaps the same reason. There
| were tiny source files with some very succinct and clever
| code that obviously caused a huge amount to go on behind the
| scenes but it was all magic and impossible to trace or
| diagnose anything - for me.
|
| Also discoverability simply wasn't there in Rails and I feel
| the same with Phoenix. I have no idea what pieces might fit
| in anywhere. Having to know that you have to create a file
| with a particular name in a particular place if you want
| something to happen is a bit of a pain if you can't find a
| solid reference on that, but also knowing that this is even a
| possibility - that a feature exists - means reading and re-
| reading huge documentation on every release just to have some
| idea of what might actually exist.
|
| It's very likely that I really just haven't 'clicked' with
| the right way to work with either, but I fear that the only
| real way to do so is to become an expert - and that would be
| a huge time investment.
|
| Still, I do enjoy looking at the code of projects using
| Phoenix, even if I can't fathom how they knew to create
| certain files and what to put in them.
| dnautics wrote:
| I'm about three years into using elixir professionally and
| I still kind of hate phoenix. There's a lot more to love in
| elixir than phoenix, and to be honest, especially if you're
| getting paid to work on a phoenix codebase, you can kind of
| grin and bear it, ask for help on slack/elixirforums, and
| it's mostly fine. There are truly only a handful of things
| where "you kind of have to know the magic incantations",
| and it's very easy to find someone who knows them. Often
| it's a generator. Typically you only need to be shown once
| per project at most (and everything else is a normal
| copy/paste away). And the debug lines making no sense is
| relatively rare. In the best sense of "being a framework",
| once you're past these problems, you're into pure elixir
| and all is good again.
|
| Jose has stated that fixing internalized pain is a real
| priority for the elixir ecosystem moving forward, so it's
| entirely possible that these things will be fixed over time
| (or it's possible it won't!).
| msie wrote:
| I agree. I was trying to learn Phoenix but had a hard time with
| the DSL because it was so non-functional in appearance.
| dqv wrote:
| Are you talking about routes or Phoenix.HTML? Those are the
| only two places I can think of having a DSL. Phoenix.HTML
| isn't strictly required and the routes look like data to me.
| iudqnolq wrote:
| It depends on what you mean by extending Ecto.
|
| I've read bits of the source code to better understand what
| commands are doing, and you're right you can't use an IDE to
| jump from the public api to the internal implementation.
| However, the project has put a lot of work into an extremely
| clear internal naming convention, so you can just go to the
| implementation directory and grep for the public api name.
|
| If you mean writing your own command that composes like the
| builtin ones, that's extremely easy. Often you can do it with
| just a plain function, and rarely you need a macro. Suppose you
| have something like this (haven't written elixir in a while,
| syntax may be slightly wrong) def
| where_admin_over(q, age) do: where(q, [u], u.age >
| ^age and u.is_admin) end def foo do
| from(User) |> where_admin_over(18) |>
| where([u], u.country == "de") |> Repo.all()
| end
|
| It's also quite easy to make that custom command run raw sql.
| astrowilson wrote:
| Not sure I agree on Phoenix. There are macros, but only around
| plugs and routing, not much different to what you'd find in
| Django and definitely less magic than Rails.
|
| Plugs are straightforward function pipelines and a declarative
| DSL really helps structure your routes.
|
| The controllers and channels, which are responsible for
| coordinating with your domain layer, don't really add anything
| in the macro department. Plus there is no inheritance, which in
| my opinion is the biggest source of confusion on most OO
| frameworks, often requiring you to chase a behavior up and down
| the class chain.
| yur3i__ wrote:
| Metaprogramming is so much easier in an sexp type syntax than in
| any other language, I really do wish it was more common
| AlchemistCamp wrote:
| I very rarely write macros but Elixir is much, much nicer to work
| with for having them.
|
| Probably my favorite overall library in Elixir is Ecto. It's the
| go-to relational DB library and makes great use of macros for its
| query syntax DSL. The Phoenix router, controllers, etc also
| benefit quite a bit.
|
| The thing that makes Elixir's macro implementation a _really_ big
| win for me as a user is how explicit it is.
|
| There is never a case of hidden magic. Just as you can see what
| functions are brought into a module by scrolling up and looking
| for the keywords "alias" and "import", you can find macros by
| looking for "use" and "using". This is very different from
| JavaScript, where another module could alter the prototype chain
| or overwrite attributes on any non-const object it has access to.
| arjan_sch wrote:
| A few years back I wrote an Elixir AST explorer, as part of a
| presentation at ElixirConf EU: https://ast.ninja/
| burnswheniphp wrote:
| I really love Elixir but our company had to drop it and rewrite a
| few projects in Go and Ruby, was too hard hiring and the existing
| team preferred not to continue with it. I think the learning
| curve is also quite harder than it initially seems, might be why
| some people give up on Elixir. But nothing but respect for
| Elixir.
| nelsonic wrote:
| Is there a public blog post about this with specific issues? Or
| is this just trolling?
| thibaut_barrere wrote:
| Having trained a couple of people on Elixir, I have had good
| success.
|
| I believe there is more a "critical mass" coupled with "matter
| of taste" issue, which I hope will be solved as more good stuff
| comes out (e.g. graphing support via Vegalite for Livebook as
| announced yesterday
| https://twitter.com/dashbit/status/1395763964215185409).
|
| I believe a number of people would currently like to get paid
| to work with Elixir, and hope the critical mass will be reached
| soon.
| styluss wrote:
| Which concepts did you find that the team struggled with?
| [deleted]
| toast0 wrote:
| > was too hard hiring and the existing team preferred not to
| continue with it.
|
| The second problem is bigger than the first.
|
| For hiring, I worked at a company running Erlang for our main
| server app; very few people were hired with Erlang knowledge,
| almost everyone had to learn it on the job. As an earlyish
| hire, I had the most experience before being hired because I
| sort of remembered seeing a Slashdot post when Ericsson open
| sourced it; but I didn't do anything with it until I was hired.
| The couple of big issues this lead to is we never really did
| anything useful with the OTP application concept, no releases
| and what not, just hotloading with l/1 in the shell; and maybe
| it took us a bit longer to figure out how to scale BEAM because
| we didn't have anyone with that experience... But BEAM is
| written so nicely that people with experience optimizing C
| stuff can dig in and optimize BEAM without a huge learning
| curve, and most of the bottlenecks fit the same pattern of
| something simple that works fine at reasonable volume, but
| blows up consistently and clearly when you run it at large
| volume, so pretty soon you learn the pattern and know what to
| look for... And of course 24 has a lot less of these
| bottlenecks than r14 did.
| logical_proof wrote:
| I don't know why you are getting downvoted. I think this is a
| valid criticism that should be discussed and not ignored. I
| find elixir to be quite elegant to work with but that's just
| like my opinion. I guess my question is what about it gives it
| the steep learning curve?
| conradfr wrote:
| It was discussed four days ago on HN
| https://news.ycombinator.com/item?id=27192873
| logical_proof wrote:
| Thank you for sharing this.
| rubyn00bie wrote:
| I've seen this a lot. The biggest reason for failure is people
| assume they already know everything about the VM and language.
| Erlang + BEAM are the most integrated pair of VM and language I
| know of. It's the only VM that will preempt running processes.
| It's also truly functional.
|
| A lot of really brilliant folks have written truly shit Elixir
| code because they underestimated the challenge. Oh you're
| passing that blob to four different gen servers? Cool expect
| you blew out your memory copying terms between them. It's also
| operationally complex and doesn't like the normal way of doing
| things because quite frankly it'll do it better.
|
| Take metrics or logging, both will slow down your application
| in Elixir/Erlang/BEAM by a not insignificant amount but people
| just shovel as much as they can into their apps assuming truths
| from other languages still hold. When your request times are
| measured in microseconds, that instrumentation is gonna slow
| you down. People literally buried boxes with erlang on them in
| the ground and they've worked for 20 years. Maybe give sampling
| a shot if it worked for a telecom maybe it'll work for your
| blog :p
|
| I'll stop rambling here. While I truly love elixir it's a lot
| more of a rodeo than people expect and it extremely subtle
| ways. It's also extremely expensive to hire developers. Because
| you're not going to be able to hire juniors. They just don't
| exist for a language like Elixir.
| jolux wrote:
| We train people on Elixir where I work and it seems to go
| well. I think I was one of the only people with prior Elixir
| experience they've ever hired, and I am arguably a junior.
| bcrosby95 wrote:
| Yes. I view Elixir kinda like 2 different languages in 1: the
| functional language and the process language. The process
| language is the most unique one though so you will probably
| spend more time learning that than the functional one.
| mattste wrote:
| My team has had success training experienced devs in Elixir
| that were coming from other backgrounds. I would caveat this
| with the fact that we just have a Phoenix app with hardly any
| code that leverages OTP.
| agd wrote:
| As a counter, I built an elixir team from 0-15 people and the
| devs love it.
|
| No particular problems hiring - finding fantastic devs is
| always hard, but lots of people we hired were attracted by the
| language.
___________________________________________________________________
(page generated 2021-05-22 23:02 UTC)