TL;DR: Elixir - a language that positively celebrates the use of macros

I’ve been picking up some Elixir so I can try my hand at a new project I want to build using the Erlang/OTP ecosystem.

Elixir uses lots and lots of compile-time macros. The language’s pervasive use of macros is unprecedented in my experience. Most (all?) of the core features of Elixir (its special forms) are (non-overridable) macros (e.g. the case statement)

Everybody likes and wants to use a language that has macros. But you are not supposed to use them right? ☺ ☺ ☺

Possibly the single most comment that has endeared me to Elixir and its community is a passage from Chris McCord’s great book Metaprogramming Elixir.

Some of the greatest insights I’ve had with macros have been with irresponsible code that I would never ship to production. There’s no substitute for learning by experimentation.

Write irresponsible code, experiment, and have fun. Use the insight you gain to drive design decisions of things you would ship to a production system.

In the same vein as Chris’s just-give-it-a-go-and-learn-some-lessons meme, I recently watched Zack Tellman’s talk Some Things that Macros do at last year’s Curry On! where he made the point that macro usage in Clojure has tended to be either at the low-end, standard fare of boilerplate removal, or up at the high-end where the doyen of Clojure macros core.async towers. Zack reckoned the gap between the two extremes offered plenty of space for people to experiment with macros.

So, with that thought, this post is my first irresponsible experiment in designing, writing and using Elixir macros. Caveat Emptor

# Elixir’s built-in pipe |> operator?

Of course anybody who has written even the merest snippet of Elixir knows it already has a statement threading operator: the famous pipe operator |>

The docs say

it simply takes the output from the expression on its left side and passes it as the first argument to the function call on its right side.

The |> is really syntactic sugar that enables this code that creates the string HelloWorld

… to be written like this:

… or even like this where each |> breaks onto a new line.

No difference at all to the final, generated, code (and result) but again a welcome boost to visual clarity.

# Elixir Metaprogramming

To explain how the |> macro works needs a bit of an explanation of how Elixir does code generation. Many people have already covered the structure of an AST notably Chris’s book but also in many fine blogs out there (see for example Sasa Juric’s series on macros). So I’m not going to delve too deep.

To set the scene, when you (run-time) metaprogram in Ruby you build the string representation of the new code and call e.g. class_eval to compile it.

Clojure, being a Lisp, is homoiconic so you’d write a (compile-time) macro to construct s-expressions with the new code.

Elixir’s lingua franca of code generation is a quoted abstract syntax tree (AST). Elixir ASTs are homoiconic and fractal-like.

Elixir macros take an AST, transform it and return the transformed AST which is then compiled by the compiler.

## What’s in an AST?

An Elixir AST is a tuple with three elements:

1. The function-to-call is often an atom of the function to call e.g. :hd (to take the head of a list), or it may be another AST for example representing a call to a function in another module; .

2. The function-metadata is “red tape” which I wont dwell on here.

3. The function-arguments is an list containing the arguments to the functions. The arguments may be literals e.g. 42 or, again, ASTs.

ASTs are regular, recursive structure including tuples, lists, atoms and “literals” such as strings, booleans (i.e. true or false) and numbers (1, 2, etc)

## Show me an AST!

The AST for any code can be created using the quote function.

Try this in Elixir’s interactive repl iex to see the AST for the first step of the example above, splitting the “hello world” string

and you’ll see a simple AST showing the the Kernel function . (dot) being used to call the split function in module String with the argument “hello world”:

If you have a look at the AST generated when using the pipe operator i.e.

and you should see something like this:

This AST is quite hard to parse by eye, but if you stare at it for a while, you should be able to discern its the |> operator being called with two arguments:

1. The first argument is “hello world” string; and

2. The second argument is essentially the same AST as above when Kernel . (dot) was used to call split in String but without the “hello world” string as the first argument i.e.

To labour this point: the AST for the |> has two arguments, one of which is another AST.

# The |> is just an Elixir macro

The |> operator is just a macro, and a very brief one at that with only two lines using Macro pipe and unpipe.

How does so little code do so much?

## The Macro.unpipe call

Lets generate the AST for the complete HelloWorld pipeline

you’ll see

Now run the Macro unpipe call on the pipeline_ast

you’ll see a list - pipeline_tuples - with four items which I’ve laid out and annotated below to try to highlight the structure and contents.

To sum up:

1. the list has for items;

2. all items are tuples of size 2 (2tuples);

3. the first element of each 2tuple is an AST;

4. for the 2nd and subsequent tuples, the AST is missing its first argument

5. the second element of each 2tuple is 0; this is the position of the previous AST in the arguments to the current AST i.e. its missing first argument

## The Macro.pipe call

The Macro unpipe call has done most of the heavy lifting.

If you take another peek at the unpipe statement in the |> macro you’ll see it destructures the unpipe result - pipeline_tuples in my example code above - to pick out the first 2tuple, and drop its second element i.e 0 (the index):

The call to Macro pipe has the relatively simpler job of creating as AST that looks like the code when =|> has not been used i.e, from the above,

The call to Macro pipe in the |> uses a call to Erlang’s foldl which may not be familar to many who only know Elixir. So I’m going the rewrite this step using the more familiar Enum reduce

The pipe inserts the this_ast (e.g “hello world”) as the rest_index argument (e.g. 0th) in the rest_ast (e.g. the ast for the call to String.split).

To see what the code for final_ast looks like, Macro to_string will “convert” the AST back to Elixir

and you should see:

Voilá!

# Too many pipes!

Yeah well that’s all fine and awesome but, like, you know, too many pipes dude, too many pipes!

You may feel having to type the |> each time gets a bit old, after all you’ve already signalled your intention of creating a thread-first pipeline. What I’d really like to write is something like this:

Of course this will give the parser kittens because each |> takes two arguments: one on the left and one on the right! Oh well. Lost cause? Maybe not?

# How about a “Thread First” macro?

Consider the same code but written like this using a familiar Elixir do block passed to a macro called thread-first (like a def with no arguments and just a do block)

What would a thread-first macro need to do to make this work?

You may have wondered why I took so much time explaining how |> works. Well its because its implementation holds the answer to my question - and it is really is quite simple.

We’ve already seen how to thread ASTs together using Macro pipe. How can we produce a list of ASTs from the do block to feed pipe?

As you may or may not know a do block is just the value of the :do key in the Keyword list passed as the last argument to the macro.

Lets have a stab at writing thread_first:

# Proof of the Pudding!

Lets do some quick tests. First the original example creating the HelloWorld string.

It doesn’t make any sense really to have a single line do block but thread_first handles it:

A daft example: a calculator

# Final Words

A bit of fun but it demonstrates how easy it is to roll-your-own code using Elixir’s macros.

My thread_first macro has only about 10 lines of actual code. That’s not only a tribute to the power of Elixir’s macros but also the standard library that includes utility modules such as Macro and Code that do so much of the heavy lifting for you.

Bit of a fess-up. Those of you who know any Clojure will have spotted fairly quickly that thread_first is equivalent to Clojure’s own thread first macro → which I’ve written about here.

# Final Final Words

People are generally “warned off” writing and using macros. Don’t be. Use your own judgement when they are necessary and when not. And when they are, use them effectively just like any other feature of Elixir. And remember: Sometimes only a macro will do.

# Code on Github

There is not much to the code but its on Github if anybody wants it: