TL;DR: Elixir - a language that positively celebrates the use of macros

I’ve been picking up some Elixir so I can try my hand at a new project I want to build using the Erlang/OTP ecosystem.

Elixir uses lots and lots of compile-time macros. The language’s pervasive use of macros is unprecedented in my experience. Most (all?) of the core features of Elixir (its special forms) are (non-overridable) macros (e.g. the case statement)

Everybody likes and wants to use a language that has macros. But you are not supposed to use them right? ☺ ☺ ☺

Possibly the single most comment that has endeared me to Elixir and its community is a passage from Chris McCord’s great book Metaprogramming Elixir.

Some of the greatest insights I’ve had with macros have been with irresponsible code that I would never ship to production. There’s no substitute for learning by experimentation.

Write irresponsible code, experiment, and have fun. Use the insight you gain to drive design decisions of things you would ship to a production system.

In the same vein as Chris’s just-give-it-a-go-and-learn-some-lessons meme, I recently watched Zack Tellman’s talk Some Things that Macros do at last year’s Curry On! where he made the point that macro usage in Clojure has tended to be either at the low-end, standard fare of boilerplate removal, or up at the high-end where the doyen of Clojure macros core.async towers. Zack reckoned the gap between the two extremes offered plenty of space for people to experiment with macros.

So, with that thought, this post is my first irresponsible experiment in designing, writing and using Elixir macros. Caveat Emptor

Read more

TL;DR: threading all the things

Update: 1Sep15

Whilst writing I discovered this post has not been updated after I’d received some feedback from Shaun Parker. No idea what happened. Sorry Shaun! You might want to jump straight to the “Update 1Sep15” section, and have a look at my some-as→ macro.

Introduction

The other day I was writing a Clojure let block to transform a map. It was a pretty usual Clojure pipeline of functions, a use case Clojure excels at. The pipeline included a cond, a couple of maps, some of my own functions, and finally an assoc and dissoc to “update” the input map with the result of the pipeline and delete some redundant keys.

Even though Clojure syntax is quite spare there was quite a bit of inevitable clutter in the code and it struck me the code would be cleaner and clearer if I could use the thread first (→) macro.

If you grok macros you can probably guess the rest of this post (likely you will have seen the title and probably said to yourself “Oh yeah, that’s obvious, nothing to see here” and moved along ☺ )

Read more

TL;DR: first take on Clojure’s new Transducers

Introduction

Rich Hickey published a post a few days ago on the forthcoming Transducers.

Transducers should make their first appearance in Clojure 1.7.0 and the examples below were run using 1.7.0-alpha1

The first paragraph of Rich’s post describes Transducers as:

Transducers are a powerful and composable way to build algorithmic transformations that you can reuse in many contexts, and they’re coming to Clojure core and core.async.

Rich also says:

The other source of power comes from the fact that transducers compose using ordinary function composition.

Transducers follow on from Reducers, first announced by Rich in May 2012.

I wrote a post back in late 2012 called Some trivial examples of using Clojure Reducers in which I attempted to offer some practical, if contrived, examples of how to use reducers.

This post reprises the examples in the reducers post but now includes equivalent contrived transducers examples as well.

I’ve includes some material from the reducers post for completeness, but without any explanation, so you don’t have to flip between the two. But you may want to refer to the reducers post for reducers background.

Read more

A colleague asked me for a brief high level overview of the Big Data software ecosystem, why his company should use it, what for and what’s in it. These are the tidied up notes I put together quickly.

Big Data is a convenient mnemonic to describe an ecosystem of computing tools, techniques and technologies that has grown up over the past decade to address many of the data processing challenges faces by modern enterprises with growing data assets.

This document tales a liberal view of what is inside and outside of a strict definition of the ecosystem because, in total, it represents the most complete set of freely available software options for many enterprises’ data processing needs.

Read more

Introduction

One of the Clojure projects I’ve been working on uses multi-level (hierarchical) maps i.e. the values of the keys of a map at one level are often maps themselves.

A common enough use case and as Fogus and Chris Houser observe in their book The Joy of Clojure (section 5.6):

It’s difficult to write a program of any significant size without the need for a map of some sort.

The use of maps is ubiquitous in writing software because frankly it’s difficult to imagine a more robust data structure.

Maps have been around quite a while. In their book Introduction To Algorithms Cormen et al say Donald Knuth attributed the invention of maps to Hans Peter Luhn in early 1953.

Using maps is very natural and easy in Clojure and indeed many other languages that support them. Clojure takes Alan Perlis’s famous quote to heart by providing a rich set of functions to manage maps:

It’s better to have 100 functions operate on one data structure than 10 functions on 10 data structures.

But using deep (multi-level) maps with many levels and many keys with long-ish descriptive names can make the code look very cluttered and prone to typos, misremembered key names, juxtaposition of levels, and similar, often silent, errors as e.g. get and get-in will return a value (nil) rather than raising an error, and assoc and assoc-in wont care at all.

Its very easy in Clojure to write function(s) to encapsulate the accesses to a key’s value, especially for multi-level keys, creating “helper” putter and getter accessors.

Arguably, maybe a stretch, you could think of these accessors as _higher order_ functions where the (implicit) input functions are e.g. get or assoc.

For example, and as usual a contrived example, where a deep multi-level map holds all the details of a house:

1
2
3
(defn change-kitchen-temperature
  [house-state new-temperature]
  (assoc-in house-state [:rooms :kitchen :properties :temperature] new-temperature))

1
2
3
(defn install-kitchen-oven
  [house-state new-oven]
  (assoc-in house-state [:rooms :kitchen :appliances :cooking :oven] new-oven))

Which is nice (at least to me): semantically named accessors hiding the details of the key hierarchy. If the hierarchy needs to change for any reason, only the accessor functions have to be changed. A familiar enough paradigm from many languages.

The other thing I’ve wanted to do has been to apply contracts to the leaf values in the map. You could of course just use an assert to apply a contract to a value in “open code”. But that means the contract definition (i.e. assertion clause) is scattered over the code base, hard to change, inconsistency may creep in, etc.

Fogus’s clojure.core.contracts library uses Clojure’s :pre and post assertions to apply contracts to a function’s arguments and/or its result (return value).

In my recent sugar post on contracts, I demonstrated a new library I’ve written clojure-contracts-sugar to add some productivity macros (“sugar”) atop Fogus’s library.

In a nice bit of synergy, if you use putter and getter accessors for a map’s leaf values, you can apply also contracts to them.

Enter my new library: clojure-contracts-maps

Read more