When I first started learning Elixir, I didn’t begin to appreciate the power of pattern matching until I’d read Sasa’s book and Dave’s book. Chapter 5 in Dave’s book had some really good prose explaining pattern matching the arguments of anonymous and (Chapter 6) named functions.

But it took me a while to assimilate what the Elixir (and Erlang) books were telling me, so I started writing my Elixir functions rather ignorant of how I could better structure them.

But as ignorance was displaced by experience with Elixir, I started to use pattern matching for functions all over the place, once I’d “seen” that many of my functions were variations (variants) on the same theme, really families of functions with the same purpose.

After a while of using bread’n’butter function matching, I found I’d developed a style of using pattern matching to create a DSL with its own set of verbs.

Then I’d write dsl programs using the verbs. Coffee anyone?

My very own Drink Maker

I’m using the example a (virtual) Drink Maker. Its not the real application that took me on my “unexpected journey”, but it demonstrates what I want to show in a (I hope) simple way.

Have a look at the CoffeeMaker drink dsl module below which has a number of variants of the same function make_drink.

make_drink is passed a state (a map), a dsl verb (an atom e.g. boil_water) and an optional keyword list of arguments (opts). All the make_drink variants have the same signature and return the (usually changed) state. A familiar enough (recursive) pattern and pretty simple stuff, but highly effective.

make_drink pattern matches on the verb (but could also use the contents of the state).

The variant make_instant_coffee runs an Enum.reduce/3 with another set of verbs specific to that recipe. The last thing it does is set the :ready_drink key in the state to :instant_coffee – I’ll use this value in the testing later on.

Lets have coffee!

Yes I know real developers supposedly don’t drink instant coffee but, hey, we’re a broad church :)

Diaster Strikes!

The dsl style worked really well and I created other dsl programs to make different types of coffee (hot, iced, espresso, latte, etc).

Then disaster struck. Already, at the back of my mind, I’d been a bit worried at the growing line count of the CoffeeMaker module; it was becoming unmanageable.

But the clincher came when I realised I wanted to be able to use my drink dsl in other applications (e.g. a TeaMaker) and enable those applications to make new drinks, adding their own recipes (make_drink) for tea, juice, beers, cocktails, whatever. And also maybe selectively modify existing verbs in the base recipes (e.g there lots of different ways to boil_water).

I wanted to allow any drink dsl module to be able to use normal pattern matching in its own code, but also call the verbs in other drink dsl modules, something like this where TeaMaker needs to use some verbs from CoffeeMaker

Tea time!

Fail! This wont work of course because pattern matching on functions is scoped to the module and TeaMaker has no access to CoffeeMaker’s make_drink so boil_water and wash_cup will fail to pattern match.

Thinking how to get around this, I considered making the list of steps in the recipe (partial) functions rather than verbs, but it starts tightly coupling verbs to modules, and visually looks a bit of a mess and difficult to parse by eye. In short: yuck!

Or I could go back to having unique, explicit functions I could call from any where but that felt horrible too.

I also thought about structuring the dsl into a hierarchical form such as a module for the different ways of boiling water, one for cleaning the cups, and so on.

But everthing I thought of introduced constraints, confusion and complexity into what was currently quite a simple, easy to understand and elegant solution.

To maintain the simplicity of an apparently “global” make_drink function, I needed to collect all the make_drink variants with different verbs from CoffeeMaker, TeaMaker, and any other drink dsl module, into a combined drinks_menu, and use the combined drinks_menu when I need to find the make_drink variant for a particular verb (e.g. boil_water).

Somehow I needed to

1. identify the various make_drink variants from each drink dsl module

2. save their details in their own drinks_menu

4. make the combined drinks_menu accessible by all the modules

But I didn’t want to have to have write the make_drink variants in radically different ways to be usable either within the module or from another module. (“Pro cake and pro eating it”)

By “save the details” I mean: the module the make_drink variant was defined in, its verb and any other arguments.

I decided to save the details using the MFA (module, function, arguments list) format that you can use with Kernel.apply/3 to call the function.

For example, the MFA 3tuple for the boil_water variant in CofffeeMaker would be

The state and opts are variable so I’ve abbreviated the stored MFA to hold only the “constant” elements like the verb, and leave the variable arguments to be supplied at the time of use (see later).

All of the make_drink variants for a module can then be held in a module-specific drinks_menu (a map), keyed on the verb e.g. for CofffeeMaker

Its worth stressing the drinks menu is just a plain map you can get, put, etc the verbs, and the values can be whatever you like; interpreting the values is a job for some other code (I’ll come on to that).

Also the value is not necessarily dependent (coupled) in any way on the verb itself, and one could have other non-MFA values. One obvious non-MFA value would be an anonymous function:

Saving the module’s drinks_menu in a persistent module attribute

The module’s drinks_menu is built at compile time but needs to be available at run time.

I bumped into how to do this while reading Chapter 2 of Chris’s metaprogramming book but the :persist option of Module.register_attribute/3 is in the reference documentation.

Simply, if you register a module attribute, e.g. @drinks_menu, with the persist: true option,

… you can get the value of drinks_menu at run time using the module’s __info__ function called with :attributes

Collecting the details of the make_drink variants

I’ve talked about how I’d save the details of the make_drink variants in the drinks_menu but not how I’d find out what they are in the first place.

I needed to be able to run some of my code when a new make_drink variant was defined so I could save its details in the drinks_menu.

I’d already done this sort of thing in Ruby (at run-time of course) using Module’s #method_added.

Well it turns out that Elixir has the same (compile-time) functionality: Module’s compiler callback @on_definition registers the callback function (e.g. make_drink_on_def_callback in the example below) the compiler will call when a variant is defined.

The @on_definition callback function

The callback is passed all the information you could wish for in a comprehensive list of six arguments:

ArgName Contents
env the module environment - think __ENV__
kind one of :def, :defp, :defmacro, or :defmacrop
name the function's name
args quoted arguments
guards quoted guard clauses
body quoted body

The callback will be called for all functions, not just make_drink, but simple pattern matching on the name (i.e. make_drink) allows the selection of wanted variants.

The args value will be a list:

Remember these are quoted so the state and opts arguments are in Macro.var/2 format. The body is similarly quoted.

Here is the callback’s code (note the return value is ignored).

Registering the @on_definition callback

To register the callback for the module is quite simple:

Pulling all the drinks_menu code together

Since all the drink dsl modules will need to register the @on_definition callback, create the drinks_menu, etc, its more useful to put all drinks_menu code, including the callback, in a utility module: DrinkMaker.Menu. (I’ve not shown the complete module here as much has already been shown above.)

Generating the drinks_menu in any drink dsl module is then just a matter of use-ing DrinkMaker.Menu before the make_drink variants are defined e.g.

Creating a combined drinks_menu is just a matter of unifying the individual drinks_menus into one.

The drinks_menu is really a dictionaries of transformations that can be run in an e.g. Enum.reduce/3 pipeline (i.e one or more verbs) against a supplied state

Since the drinks menus are maps, they could be merged using Map.merge/2. Alternatively, one could e.g. take or drop verbs as needed and combine them that way.

How the combination is done depends on the application’s need and there is nothing to prevent the creation of different menus for different purposes e.g. hot_drinks_menu, cold_drinks_menu, free_drinks_menu, paid_drinks_menu, whatever.

Here the drinks_menus are just merged:

To make drinks from a combined drinks_menu needs a helper function — defined for convenience in the DrinkMaker.Menu and also, but not necessarily, called make_drink — to find the verbs and call (usually) the right drink dsl module’s make_drink variant.

Note, the helper expects to find the drinks_menu in the state.

Its worth stressing the helper doesn’t “know” about the make_drink function and its variants; it only uses the values in the drinks_menu i.e the MFA 3tuple (usually) where the function to call in the 2nd element.

To see how the helper would be used, let briefly revisit TeaMaker’s make_earl_grey_tea variant, the one above that failed because it needed to call verbs in CoffeeMaker. Assuming the state passed to TeaMaker’s make_earl_grey_tea now holds the drinks_menu, the variant would be rewritten to use DrinkMaker.Menu.make_drink/3 (as would any other variant that needed to call make_drink variants in other drink dsl modules).

The state could hold many drinks menus and the decision on which one to use taken by the variant, the helper, whatever and based on any criteria.

The DrinkMaker GenServer

To show a testable example, I’ve created a GenServer to make drinks, holding the drinks_menu in the GenServer’s state.

(I want to have a nice easy Map interface to the drinks_menu so I’ll use my Hex package Amlapio)

Testing the DrinkMaker GenServer

In this test Jane and Lucy set their favourite drink and then make it.

How is this Polymorphism?

The GenServer example above shows that Jane’s and Lucy’s preferred drink depends on the value of the make_my_favourite_drink verb in their respective drinks_menu.

This is a example of runtime polymorphism where the decision on which function to dispatch is determined from the value of the verb (in the call to DrinkMaker.make_drink/3).

Rich’s Clojure documentation says:

The basic idea behind runtime polymorphism is that a single function designator dispatches to multiple independently-defined function definitions based upon some value of the call.

(btw the single function designator in my example is DrinkMaker.Menu.make_drink/3)

Rich goes on to say:

… go further still to allow the dispatch value to be the result of an arbitrary function of the arguments

It would be trivial to add an arbitrary function to the state, taking the state and verb as arguments, and use it to decide what make_drink variant to call, allowing arbitrarily complex multiple dispatch decisions (albeit maybe with a performance penalty).

Final Words

I’ve focused on just one dispatch function (make_drink) but it is straightforward to extend the code to other functions as well – I’ve already done so in the code for the actual project that gave rise to the idea for this blog post.

I’ve found this really very simple technique to be both flexible and powerful, especially in providing a simple, open way for multiple modules and applications to participate fully in the dsl, adding new, or modifying existing, verbs.

Code on Github

The code is on Github: