I guess I’ve always written pretty defensive code (trust nothing), insisting that e.g. passed arguments are as expected and needed. Anybody who’s seen my recent Ruby code will have found it littered with many, many assertions. Usually my methods check both the input arguments for validity and also end with a call to a final guard to ensure the returned value is also valid.

During my time writing Perl (rather a long time ago), I used my own variants of the standard carp, croak and confess functions heavily, and followed received wisdom to fail early and often.

After moving to Ruby as my primary language, my Perl style evolved into the implementation of mustbe method e.g. mustbe-hash-or-croak to ensure the argument is as expected. My mustbe methods are dynamically defined in a mixin using standard Ruby’s metaprogramming techniques and, although they commonly test whether the argument is an instance of a given class, they can apply (assert) arbitrary constraints.

In time I realised I’d fallen into a contracts style of programming. I’d not sure how or where I first came across contracts formalism but it may have been when reading about Eiffel’s Design by Contract.

Eiffel’s DBC asserts pre and post constraints on a routine’s execution; in my own informal way, I’d become strongly attracted to contract assertion, not only on entry to and exit from routines (methods, functions), but also in their bodies.

My recent excursions into writing Clojure has had me reaching for a way (technique) to apply contracts. Hence this post.

Read more

Pallet is a configuration management tool (library) for Linux infrastructures. Well known alternatives to Pallet include Chef and Puppet.

A bit more background on Pallet can be found in a previous post. That post is also worth reading for background on VMFest and how it interacts with VBox

Chef is hugely useful in defining and managing standardised, repeatable and consistent definitions and configuration of Infrastructure components such as (web servers, database serevers, etc). Picking up the basics of Chef is not a steep learning curve. I’ve learnt the 20% of Chef that does 80% of what I need to do very easily, albeit coming as I do from a recent Ruby background.

Chef is a vibrant and compelling ecosystem. Opscode are doing a fine job of taking the software forward, providing good community support, documentation, tutorials, books, etc whilst providing professional support for these organisation that need it. That said Chef has its naff bits as well. Chef is flexible but within the constraint of the “API” / “DSL”. In many ways Chef feels like Rails - its Ruby alright but in a well-defined box. Both strengths and weaknesses of course but the strengths fair outweigh the weaknesses for all practical purposes and use.

So why bother with Pallet? Maybe as a long time Infrastructure guy, its because my antennae are twitching and I can see a powerful and flexible potential addition to the armoury of tools for managing infrastructures.

It is also because Pallet “scripts” are Clojure programs and it really is Infrastructure as Code in the most explicit sense: Pallet is the library but the scripts are mine and I feel more in control of what is going on than with Chef.

Also Pallet’s composability is very attractive feature - it feels very natural (in Clojure) to build high level abstractions from lower level ones. Even with my currently limited Pallet knowledge, I can already “see” how to build hierarchical, complete, comprehensive and manageable infrastructures, something I’ve found harder to imagine with Chef. Horses for courses of course.

BTW: Although my environment is VBox 4.2 and the examples below were run using my VBox 4.2 compatible VMFest, there is no reason why they shouldn’t work with the PalletOps team’s current release for VBox 4.1 as long as my extensions to hardware models (clipboard-mode, utc-time and boot-order) are removed.

Read more

VMFest is

“… a PalletOps project to turn VirtualBox into a light-weight cloud provider.”

To understand the context and point of this post requires a minor digression into Pallet.

Pallet describes itself as

“A fresh look at cloud infrastructure automation”

and

“DevOps for the JVM”

Suitably cool aphorisms but, more prosaically, Pallet is a configuration management tool (library) for Linux infrastructures (although, in principle, there does not appear to be any reason why it wouldn’t work with Windows).

Well known alternatives to Pallet include Chef and Puppet. Currently I use Chef to manage my own development environments.

Pallet is written in Clojure and hence is part of the Java/JVM ecosystem.

Pallet supports many cloud providers though the use of the jclouds library: as long as a provider supports the jclouds API, it can be used by Pallet.

But VBox is (still?) not supported by jclouds (there doesn’t appear to be a jclouds provider).

Because there is no jclouds provider for VBox, the PalletOps team wrote pallet-vmfest to support VBox as a “local” compute service. pallet-vmfest, in turn, uses VMFest to control VBox.

Hope that’s clear. More on Pallet in a future post. Now back to VMFest.

Read more

In two previous posts I explored writing Apache Pig user defined functions (UDFs) in JRuby and Clojure to process log data, specifically an auditd log.

Pig is a very good option for extract transform and load of log files but it is by no means the only option available. Others include Fluentd, Flume, Scalding and Cascalog. Most of the others also have or use Hadoop as the processing / execution environment.

This post focuses on the same log data processing scenario as the earlier posts, this time using Cascalog, to parse the auditd log (available here) and load the fields into a HBase table.

Read more

I needed a simple http file server to serve files around my home network.

A bit of googling found this. It trivial but so, so useful.

It uses the Ruby Webrick server so you have to have Ruby installed.

Assuming the path of the filesystem you want to sere is /home/me/heretheyare, run this in a terminal:

1
ruby -rwebrick -e 'WEBrick::HTTPServer.new(:Port=>1234,:DocumentRoot=>"/home/me/heretheyare").start'

Use your browser, wget, whatever to pull the file(s) you need. Remember to include the port (if its not 80).

It should work on Windows but I haven’t tried; I use it on Ububntu.