https://serokell.io/blog/elixir-metaprogramming

 

  * 
  * Projects
  * Labs
  * Blog
  * Academy
  * Team
  * Contacts
  * Hire us

Your browser seems to have problems showing our website properly so
it's switched to a simplified version.
We use cookies to personalize content and give you the best web
experience. For more details, please read our Privacy Notice
Accept
ResearchProgrammingArtificial IntelligenceInterviewsOther

Metaprogramming in Elixir

Gints Dreimanis
Article by Gints Dreimanis
May 21st, 2021
9 min read
TwitterFacebookLinkedin TelegramMail WhatsAppCopy Link
Upvote blog postBlog post is upvoted
30

Usually, we think of a program as something that manipulates data to
achieve some result.

But what is data?

Can we use the programs themselves as data? 

In today's article, we'll go down the rabbit hole with the assistance
of Elixir, a programming language that is permeated by
metaprogramming.

I'll introduce you to metaprogramming in Elixir and show you how to
create a macro for defining curried functions.

What is metaprogramming?

Metaprogramming is just writing programs that manipulate programs.
It's quite a wide term that can include compilers, interpreters, and
other kinds of programs.

In this article, we will focus on metaprogramming as it is done in
Elixir, which involves macros and compile-time code generation.

Metaprogramming in Elixir

To understand how metaprogramming works in Elixir, it is important to
understand a few things about how compilers work. During the
compilation process, every computer program is transformed into an
abstract syntax tree (AST) - a tree structure that enables the
computer to understand the contents of the program.

AST
                            Quick maths.

In Elixir, each node of the AST (except basic values) is a tuple of
three parts: function name, metadata, function arguments.

Elixir enables us to access this internal AST representation via
quote.

iex(1)> quote do 2 + 2 - 1 end
{:-, [context: Elixir, import: Kernel],
 [{:+, [context: Elixir, import: Kernel], [2, 2]}, 1]}

We can modify these ASTs with macros, which are functions from AST to
AST that are executed at compile-time.

You can use macros to generate boilerplate code, create new language
features, or even build domain-specific languages (DSLs).

Actually, a lot of the language constructs that we regularly use in
Elixir such as def, defmodule, if, and others are macros.
Furthermore, many popular libraries like Phoenix, Ecto, and Absinthe
use macros liberally to create convenient developer experiences.

Here's an example Ecto query from the documentation:

query = from u in "users",
          where: u.age > 18,
          select: u.name

Metaprogramming in Elixir is quite a powerful tool. It approaches
LISP (the OG metaprogramming steam roller) in expressivity but keeps
things one level above in abstraction, enabling you to delve into AST
only when you need to. In other words, Elixir is basically LISP but
readable. 

Getting started

So how do we channel this immense power? 

While metaprogramming can be rather tricky, it is quite simple to
start metaprogramming in Elixir. All you need to know are three
things.

        quote       unquote                       defmacro
        code -> AST outside quote -> inside quote AST -> AST

quote

quote converts Elixir code to its internal AST representation.

You can think of the difference between regular and quoted
expressions to be the difference in two different requests.

  * Say your name, please. Here, the request is to reply with your
    name.
  * Say "your name", please. Here, the request is to reply with the
    internal representation of the request in the language - "your
    name".

iex(1)> 2 + 2
4
iex(2)> quote do 2 + 2 end
{:+, [context: Elixir, import: Kernel], [2, 2]}

quote makes it a breeze to write macros since we don't have to
generate or write the AST by hand.

unquote

But what if we want to have access to variables inside quote? The
solution is unquote.

unquote functions like string interpolation, enabling you to pull
variables into quoted blocks from the surrounding context.

Here's how it looks in Elixir:

iex(1)> two = 2
2
iex(2)> quote do 2 + 2 end
{:+, [context: Elixir, import: Kernel], [2, 2]}
iex(3)> quote do two + two end
{:+, [context: Elixir, import: Kernel],
 [{:two, [], Elixir}, {:two, [], Elixir}]}
iex(4)> quote do unquote(two) + unquote(two) end
{:+, [context: Elixir, import: Kernel], [2, 2]}

If we don't unquote two, we will get Elixir's internal representation
of some unassigned variable called two. If we unquote it, we get
access to the variable inside the quote block.

defmacro

Macros are functions from ASTs to ASTs.

For example, suppose we want to make a new type of expression that
checks for the oddity of numbers.

We can make a macro for it in just a few lines with defmacro, quote,
and unquote.

defmodule My do
  defmacro odd(number, do: do_clause, else: else_clause) do
    quote do
      if rem(unquote(number), 2) == 1, do: unquote(do_clause), else: unquote(else_clause)
    end
  end
end

iex(1)> require My
My
iex(2)> My.odd 5, do: "is odd", else: "is not odd"
"is odd"
iex(3)> My.odd 6, do: "is odd", else: "is not odd"
"is not odd"

When should you use metaprogramming?

    "Rule 1: Don't Write Macros" - Chris McCord, Metaprogramming in
    Elixir

While metaprogramming can be an awesome tool, it should be used with
caution.

Macros can make debugging much harder and increase overall
complexity. They should be turned to only when it is necessary - when
you run into problems you can't solve with regular functions or when
there is a lot of plumbing behind the scenes that you need to hide.

When used correctly, they can be very rewarding, though. To see how
they can improve developer life, let's look at some real-life
examples from Phoenix, the main Elixir web framework.

How macros are used in Phoenix

In the following section, we'll analyze the router submodule of a
freshly made Phoenix project as an example of how macros are used in
Elixir.

use

If you look at the top of basically any Phoenix file, you will most
likely see a use macro. Our router submodule has one.

defmodule HelloWeb.Router do
  use HelloWeb, :router

What this one expands to is:

require HelloWeb
HelloWeb.__using__(:router)

require asks HelloWeb to compile its macros so that they can be used
for the module. But what's using? It, as you might have guessed, is
another macro!

  defmacro __using__(which) when is_atom(which) do
    apply(__MODULE__, which, [])
  end

In our case, this macro invokes the router function from the HelloWeb
module.

  def router do
    quote do
      use Phoenix.Router

      import Plug.Conn
      import Phoenix.Controller
    end
  end

router imports two modules and launches another __using__ macro.

As you can see, this hides a lot, which can be both a good and a bad
thing. But it also gives us access to a magical use HelloWeb, :router
to have everything ready for quick webdev action whenever we need.

pipeline

Now, look below use.

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
    plug :protect_from_forgery
    plug :put_secure_browser_headers
  end

Yup, more macros.

pipeline and plug define pipelines of plugs, which are functions of
functions that transform the connection data structure.

While the previous macro was used for convenient one-line imports,
this one helps to write pipelines in a very clear and natural
language.

scope

And, of course, the routing table is a macro as well.

  scope "/", HelloWeb do
    pipe_through :browser

    get "/", PageController, :index

  end

scope, pipe_through, get - all macros.

In fact, the whole module is macros and an if statement (which is a
macro) that adds an import statement and executes a macro.

I hope that this helps you see how metaprogramming is at the heart of
Elixir.

Now, let's try to build our own macro.

Build your own macro in Elixir

Elixir and currying don't really vibe together. But with some effort,
you can create a curried function in Elixir.

Here's a regular Elixir sum function:

def sum(a, b), do: a + b

Here's a curried sum function:

def sum() do
  fn x ->
    fn y ->
      x + y
    end
  end
end

Here's how they both behave:

iex(1)> Example.sum(1,2)
3
iex(2)> plustwo = Example.sum.(2)
#Function<10.76762873/1 in Example.sum/0>
iex(3)> plustwo.(2)
4

Let's say that we want to use curried functions in Elixir for some
reason (for example, we want to create a monad library). Writing out
every function in our code like that would be, to say the least,
inconvenient.

But with the power of metaprogramming, we can introduce curried
functions without a lot of boilerplate. Let's define our own defc
macro that will define curried functions for us.

First, we need to take a look at how a regular def looks as an AST:

iex(1)> quote do def sum(a, b), do: a + b end
{:def, [context: Elixir, import: Kernel],
 [
   {:sum, [context: Elixir], [{:a, [], Elixir}, {:b, [], Elixir}]},
   [
     do: {:+, [context: Elixir, import: Kernel],
      [{:a, [], Elixir}, {:b, [], Elixir}]}
   ]
 ]}

It is a macro with two arguments: the function definition (in this
case, sum is being defined) and a do: expression.

Therefore, our defc (which should take the same data) will be a macro
that takes two things:

 1. A function definition, which consists of the function name,
    context, and supplied arguments.
 2. A do: expression, which consists of everything that should be
    done with these arguments.

defmodule Curry do
  defmacro defc({name, ctx, arguments} = clause, do: expression) do
  end
end

We want the macro to define two functions:

 1. The function defined in defc.
 2. A 0-argument function that returns the 1st function, curried.

  defmacro defc({name, ctx, arguments} = clause, do: expression) do
    quote do
      def unquote(clause), do: unquote(expression)
      def unquote({name, ctx, []}), do: unquote(body)
    end
  end

That's more or less the macro. Now, we need to generate the main part
of it, the body.

It's quite simple. We just need to go through the whole argument list
and, for each argument, wrap the expression in a lambda.

Here's a recursive function to do that.

  defp create_fun([h | t], expression) do
    rest = create_fun(t, expression)

    quote do
      fn unquote(h) -> unquote(rest) end
    end
  end

  defp create_fun([], expression) do
    quote do
      unquote(expression)
    end
  end

Then, we assign the variable body in the macro to be the result of
create_fun, applied to the arguments and the expression.

  defmacro defc({name, ctx, arguments} = clause, do: expression) do
    body = create_fun(arguments, expression)

    quote do
      def unquote(clause), do: unquote(expression)
      def unquote({name, ctx, []}), do: unquote(body)
    end
  end

That's it! 

To try it out, let's define another module with a sum function in it.

defmodule Example do
  import Curry

  defc sum(a, b), do: a + b

end

iex(1)> Example.sum(2,2)
4
iex(2)> Example.sum.(2).(2)
4
iex(3)> plustwo = Example.sum.(2)
#Function<5.100981091/1 in Example.sum/0>
iex(4)> plustwo.(2)
4

You can see the full code here.

In our example, the macro provides only the sum() and sum(a,b)
functions. But from here, it's easy to extend our macro to generate
partial functions for all arities.

In the case of sum, we can make it so that the macro generates sum(),
sum(a), and sum(a,b), modifying the function definition to account
for the missing arguments.

Since it is an awesome exercise to try on your own, I will not spoil
the answer. 

Further learning

If you want to learn more about macros in Elixir, here are a few
resources I suggest to check out:

  * Metaprogramming in Elixir. Chris McCord's book gives quite an
    in-depth introduction to macros with awesome and practical code
    examples.

  * Understanding Elixir Macros. This article series by Sasa Juric
    covers the topic in much more detail than I do, and it's a nice
    read if you don't have time to read Chris's book.

  * Don't Write Macros But Do Learn How They Work. If you prefer to
    watch videos, this is a nice talk by Jesse Anderson.

For more articles on Elixir, you can go to our Elixir section or
follow us on Twitter or Medium to receive updates whenever we publish
a new one.

tagged:
elixirelixir tutorial
share:
       
Upvote blog postBlog post is upvoted
30 upvotes
[                    ]Suggest your topic for our next post...
Submit
Metaprogramming in Elixir

  * What is metaprogramming?
  * Metaprogramming in Elixir
      + Getting started
          o quote
          o unquote
          o defmacro
  * When should you use metaprogramming?
  * How macros are used in Phoenix
      + use
      + pipeline
      + scope
  * Build your own macro in Elixir
  * Further learning

More from Serokell
elixir parser combinatorselixir parser combinators
Parser Combinators in Elixir
Learn what parser combinators are, what they are made of, and how to
make your own CSV parser using NimbleParsec, a parser combinator
library written in Elixir.
April 6th, 2021
11 min read
8 Companies That Use Elixir to Win8 Companies That Use Elixir to Win
8 Companies That Use Elixir in Production
What's the one programming language we will surely hear more about in
the next 5 years?
May 6th, 2020
6 min read
Elixir in Production: PapercupsElixir in Production: Papercups
Elixir in Production: Papercups
What are the benefits of Elixir for SaaS products? Find out in our
interview with Alex Reichert, the co-founder of Papercups.
February 23rd, 2021
5 min read
(+372) 699-1531
hi@serokell.io
Pille tn 7/5-13, Kesklinna linnaosa,
Tallinn, Harju maakond, 10135, Estonia
Serokell: Rate5.0based on 9 Google Business reviews
       
Serokell LabsFintechManaged IT servicesHow we workML consulting
Privacy notice
 
(c) 2015-2021 Serokell
Close
[                    ]