TL;DR: threading all the things

Update: 1Sep15

Whilst writing I discovered this post has not been updated after I’d received some feedback from Shaun Parker. No idea what happened. Sorry Shaun! You might want to jump straight to the “Update 1Sep15” section, and have a look at my some-as→ macro.


The other day I was writing a Clojure let block to transform a map. It was a pretty usual Clojure pipeline of functions, a use case Clojure excels at. The pipeline included a cond, a couple of maps, some of my own functions, and finally an assoc and dissoc to “update” the input map with the result of the pipeline and delete some redundant keys.

Even though Clojure syntax is quite spare there was quite a bit of inevitable clutter in the code and it struck me the code would be cleaner and clearer if I could use the thread first (→) macro.

If you grok macros you can probably guess the rest of this post (likely you will have seen the title and probably said to yourself “Oh yeah, that’s obvious, nothing to see here” and moved along ☺ )

Read more

TL;DR: first take on Clojure’s new Transducers


Rich Hickey published a post a few days ago on the forthcoming Transducers.

Transducers should make their first appearance in Clojure 1.7.0 and the examples below were run using 1.7.0-alpha1

The first paragraph of Rich’s post describes Transducers as:

Transducers are a powerful and composable way to build algorithmic transformations that you can reuse in many contexts, and they’re coming to Clojure core and core.async.

Rich also says:

The other source of power comes from the fact that transducers compose using ordinary function composition.

Transducers follow on from Reducers, first announced by Rich in May 2012.

I wrote a post back in late 2012 called Some trivial examples of using Clojure Reducers in which I attempted to offer some practical, if contrived, examples of how to use reducers.

This post reprises the examples in the reducers post but now includes equivalent contrived transducers examples as well.

I’ve includes some material from the reducers post for completeness, but without any explanation, so you don’t have to flip between the two. But you may want to refer to the reducers post for reducers background.

Read more

A colleague asked me for a brief high level overview of the Big Data software ecosystem, why his company should use it, what for and what’s in it. These are the tidied up notes I put together quickly.

Big Data is a convenient mnemonic to describe an ecosystem of computing tools, techniques and technologies that has grown up over the past decade to address many of the data processing challenges faces by modern enterprises with growing data assets.

This document tales a liberal view of what is inside and outside of a strict definition of the ecosystem because, in total, it represents the most complete set of freely available software options for many enterprises’ data processing needs.

Read more


One of the Clojure projects I’ve been working on uses multi-level (hierarchical) maps i.e. the values of the keys of a map at one level are often maps themselves.

A common enough use case and as Fogus and Chris Houser observe in their book The Joy of Clojure (section 5.6):

It’s difficult to write a program of any significant size without the need for a map of some sort.

The use of maps is ubiquitous in writing software because frankly it’s difficult to imagine a more robust data structure.

Maps have been around quite a while. In their book Introduction To Algorithms Cormen et al say Donald Knuth attributed the invention of maps to Hans Peter Luhn in early 1953.

Using maps is very natural and easy in Clojure and indeed many other languages that support them. Clojure takes Alan Perlis’s famous quote to heart by providing a rich set of functions to manage maps:

It’s better to have 100 functions operate on one data structure than 10 functions on 10 data structures.

But using deep (multi-level) maps with many levels and many keys with long-ish descriptive names can make the code look very cluttered and prone to typos, misremembered key names, juxtaposition of levels, and similar, often silent, errors as e.g. get and get-in will return a value (nil) rather than raising an error, and assoc and assoc-in wont care at all.

Its very easy in Clojure to write function(s) to encapsulate the accesses to a key’s value, especially for multi-level keys, creating “helper” putter and getter accessors.

Arguably, maybe a stretch, you could think of these accessors as _higher order_ functions where the (implicit) input functions are e.g. get or assoc.

For example, and as usual a contrived example, where a deep multi-level map holds all the details of a house:

(defn change-kitchen-temperature
  [house-state new-temperature]
  (assoc-in house-state [:rooms :kitchen :properties :temperature] new-temperature))
(defn install-kitchen-oven
  [house-state new-oven]
  (assoc-in house-state [:rooms :kitchen :appliances :cooking :oven] new-oven))

Which is nice (at least to me): semantically named accessors hiding the details of the key hierarchy. If the hierarchy needs to change for any reason, only the accessor functions have to be changed. A familiar enough paradigm from many languages.

The other thing I’ve wanted to do has been to apply contracts to the leaf values in the map. You could of course just use an assert to apply a contract to a value in “open code”. But that means the contract definition (i.e. assertion clause) is scattered over the code base, hard to change, inconsistency may creep in, etc.

Fogus’s clojure.core.contracts library uses Clojure’s :pre and post assertions to apply contracts to a function’s arguments and/or its result (return value).

In my recent sugar post on contracts, I demonstrated a new library I’ve written clojure-contracts-sugar to add some productivity macros (“sugar”) atop Fogus’s library.

In a nice bit of synergy, if you use putter and getter accessors for a map’s leaf values, you can apply also contracts to them.

Enter my new library: clojure-contracts-maps

Read more

TL;DR: clojure-contracts-sugar - some sugar macros for clojure.core.contracts


Back in November 2012, in one of my first toe-dippings into Clojure, I wrote a post on my initial experiences with Michael Fogus’s clojure.core.contracts (hereafter just CCC).

The rest of this post assumes you have a passing familiarity with CCC. If you aren’t, you may find my original post a digestible introduction.

Although using CCC for contracts programming is self-evident, it was the opportunity to use the Clojure’s :pre and :post assertions to support the concept and potential of aspects as defined originally by Gregor Kiczales in his work on Aspect Oriented Programming (AOP).

In this post I’m following through the experiments and ideas of the original post, and illustrating with some examples using a new library I’ve written recently - clojure-contracts-sugar (CHUGAR).

In the original post I made the point that CCC could do with some productivity aids - sugar - to ease the rich usage of CCC. CHUGAR attempts to supply that sugar and you can think of this post as partly a getting started tutorial for the library.

In the simplest cases, CHUGAR reduces to CCC albeit with a slightly more flexible syntax. If your contracts needs are straightforward, you’re likely better off using CCC or even :pre and :post assertions directly.

That said, I would encourage you though to have a look at the sections on mnemonics below which offer an easy way to use (and re-use) rich and flexible aspect contracts.

Note, the usual caveat, CHUGAR is a work in progress and really is intended to be only a starting point (foundation) for explorations of contracts from Clojure. The API is “settling” but may change.

A quick note on terminology: in the original post I used suck to identify the (input) arguments of a function and spit the result (return value). That “convention” is used extensively both here and in the library’s code.

Read more