Setting Up Development Environments With Nix

I remember my first day at Cloud9 IDE well. It was very similar to my first day Technical University of Delft as a PhD student, and my first day at LogicBlox: a day spent on installing a bunch of software, tweaking configuration files, environment variables, running services to get my system into a state so that I could actually start contributing. In some cases it took a few hours, in others it took a day. Either way, it was an utter waste of time. Worse, this setup problem often not a one-time thing. It gets really interesting if you have to develop on two projects (or two branches of the same project) that have incompatible software environment requirements. For instance, at some point, at Cloud9 we had a branch that worked only on node.js 0.4, but not on 0.6 and a separate branch that worked on 0.6. Every time I switched between these projects I had to make sure that the right version of node.js was active. Tools like nvm helped, but they’re a hassle and very specific to the particular platform. What about the Redis version that that particular project required, for instance? In aggregate weeks, months, years are wasted solving problems related to their development environment.

Setting up an environment to start developing on a project should be easy and fast. Yet in reality, all too often, it’s not. At Cloud9 IDE we developed a virtual-machine-based solution. Once we all made the move to developing on VMs in the cloud using c9.io, we started handing out EC2 VMs to employees with all software pre-installed. That was very helpful, but now you have to manage those VMs. Prepare VM images, and keep those images up-to-date. As requirements change, now you have to update all VMs and install new dependencies. If you want to work on two dev-environment incompatible projects, now you need two VMs and you have to configure them both with all your own preferences, and then there’s the cost of running multiple VMs at a cloud provider.

A tool like Vagrant helps in provisioning VMs based on a recipe and uses locally run VirtualBox VMs. Using Vagrant, you declaratively specify all the software you need for a project and it builds a VM for you to develop in. You can commit this specification in your software repository so that it’s always in sync with your project. Vagrant boots the VM locally, mounts your project directory in there, and gives you a bash prompt over SSH for you to compile and run your software. That works, but it’s a little heavy weight. Now you need to have a Linux VM booted, using your memory and CPU cycles to run and test stuff.

Isn’t there a less heavy-weight solution?

NixAt LogicBlox we are sponsoring the Nix project. Nix is a family of deployment-related products, including a package manager, operating system (NixOS), continuous integration server (Hydra) and cloud deployment tool (NixOps). Almost all of our production servers are deployed using Nix.

So why pull a deployment tool into this discussion? Because setting up a development environment is essentially a deployment problem. Basically what you want is to deploy all the services, applications and libraries to your local machine so that you can start developing on a project.

And indeed, with Nix, you can do this pretty easily. The experience is very similar to Vagrant, except that there are no virtual machines required. For this you only need the Nix package manager installed, which runs on almost all Unix-based operating systems (and if you really insist on Windows with Cygwin as well), including Linux and Mac. I won’t get into much of the technical detail of how Nix works (read more about it on the website), but in essence it’s a package manager that supports installing multiple versions of software components on a single system, stored in isolation with strict and very precisely defined dependencies defined between them.

All software managed by Nix is stored in the Nix store, a directory usually located at /nix/store. Here’s a small blip of the stuff stored there on my system (most of these are directories):

jxi3kc3h7pg9ykyixlbgnlrndvnb0rxm-bash-4.2-p42
jymhsfjv6m201q69l23c6iskng2i4cs4-groff-1.22.2k0kd4brqjm1017bhyj8rkyy36n0v5kmh-coreutils-8.21ka9xg76y827bd70hds1w6296yik0kd9b-gettext-0.18.1.1l5pdaahf6nqmpcy0gz5k2zdkj6z7dqsn-gnused-4.2.1l81rf85cabpl5phz20wqn5c16m9yi80i-libiconv-1.13.1larxfznqb48w2p8k5qr0v3kbvlplwc3v-nodejs-0.10.8yw62q6acrpjnx2r1zg9bzsn7xld1y00s-nodejs-0.6.10m6anp10748v2jmgvgjz56xxn7jizj03s-isl-0.07m9znv6pvij31vyiv0z8qvd05h1bgc67l-openssl-1.0.1emd3hd3k7igb45zxvkxb1qd6ij23v7xaj-nixops

As you can see, every entry starts with a hash. This is a hash encoding all dependencies (inputs) of the build for that component, including: the source code, the platform it was built on and software build and runtime dependencies. If a package has a dependency on a certain library it will use an absolute path linking to a location in the Nix store to that library, so that always the exact same version is used at run-time as was used at compile time. As you can see there are two versions of node.js available in my Nix store, so that if one of my projects relies on node.js version 0.6 and another 0.8, I can work on them both simultaneously without them interfering in any way. If you want to learn more about how Nix works and why it is designed the way it is, and I encourage it, have a look at the manual.

So, how does Nix help with setting up development environments? Before we get there, let’s first have a brief look at how a software component (or package) is built using Nix.

Nix defines its own purely functional programming language that is used to define components and their dependencies. Here’s a somewhat simplified version of the expression for the node.js package:

{ pkgs, stdenv, ... }:stdenv.mkDerivation rec {  version = "0.10.7";  name = "nodejs-${version}";  src = pkgs.fetchurl {    url = http://nodejs.org/dist/v0.10.7/node-v0.10.7.tar.gz;    sha256 = "1q15siga6b3rxgrmy42310cdya1zcc2dpsrchidzl396yl8x5l92";  };  buildInputs = [ pkgs.python pkgs.utillinux ];}

This piece of code, or “expression,” specifies a function with named arguments. It takes two or more arguments: pkgsstdenv and possibly more, but those are ignored. The body of the function calls a function named mkDerivation, an attribute of stdenv, which was passed in as an argument to the function. mkDerivation takes one argument: a recursive attribute set, as an argument (the rec keyword specifies that the attribute set is recursive). Why recursive? Because its attributes (versionnamesrc and buildInputs) may occasionally refer to each other. Specifically, the value of the name attribute refers to the version attribute, as you can see. So, what does the mkDerivation function return? A built package. mkDerivation assumes an autoconf-style package that can be built with the usual ./configure && make && make install. The source code for our package can be downloaded from the specified URL, and should match a particular checksum hash. By specifying this hashcode we’re sure that in the future we’ll always get exactly the same version of the tarball and that the build is reproducible and you always get the same output (so that we can do caching and we don’t have to recompile much). For the package to be built, python and some package utillinux have to be installed and available in the build environment (in the PATH). The result of the build will end up somewhere in /nix/store/<some-hash>-nodejs-0.10.7 if successful. Unlike most other package systems, the result is not installed in /usr/local/bin or the like.

Alright, now let’s look at a slightly more elaborate Nix expression:

let  pkgs = import <nixpkgs> {};  stdenv = pkgs.stdenv;in rec {  node = stdenv.mkDerivation rec {    version = "0.10.7";    name = "nodejs-${version}";    src = pkgs.fetchurl {      url = http://nodejs.org/dist/v0.10.7/node-v0.10.7.tar.gz;      sha256 = "1q15siga6b3rxgrmy42310cdya1zcc2dpsrchidzl396yl8x5l92";    };    preConfigure = stdenv.lib.optionalString stdenv.isDarwin ''export PATH=/usr/bin:/usr/sbin:$PATH'';    buildInputs = [ pkgs.python ] ++ stdenv.lib.optional stdenv.isLinux pkgs.utillinux;  };  app = stdenv.mkDerivation {    name = "application";    src = ./app;    PORT = "8888";    buildInputs = [ node ];  };}

You’ll recognize most of the part that says “node =”. This is a slightly more elaborate version of the node.js expression we just looked at, with just a few things added to also make it compile on Mac (where utillinux is not available). The first three lines basically set up some variables that are used throughout the rest of the expression. The meat of the expression is the attribute set with two attributes: node and appnode is the specific node.js build we’d like to use for our application, and app specifies the application itself. As you can see, app uses the ./app directory as source and refers to our node attribute as a build input thereby specifying that it needs this specific build of node.js to build/run. The PORT attribute is an environment variable that is used by the running application to decide what port to run the server on.

You can find the “entire source code” of this application on Github.

So, here’s where we get to the development environment part. If you have Nix installed, all you need to do to get ready to develop on this fancy node.js application is this:

$ git clone git://github.com/zefhemel/nodejs-nix.git$ cd nodejs-nix$ nix-build -A app --run-env

That last one is the magic one. What it will do is:

  1. Build and install all dependencies required to build app, if they have not been built before already. In our case: build our particular node.js version. If builds are available in the Nix store already (or are downloadable from a nix-channel), they will not be rebuilt, Nix builds are very incremental.
  2. Launch a (bash) sub-shell with all environment variables setup the way as they would be to actually build the app attribute. That is: PATH will contain the bin directory of the node.js build specified in our expression (since it was specified in buildInputs, so imagine this looking something like PATH=/nix/store/<hash>-nodejs-0.10.7/bin:…), so it will not use whatever node.js version happens to be installed elsewhere in the system. It will also set the PORT environment variable to 8888.

Here’s a typical terminal session demonstrating its use:

$ nix-build -A app --run-envbash-3.2$ echo $PORT8888bash-3.2$ node --versionv0.10.7bash-3.2$ node app/server.jsServer running at http://127.0.0.1:8888^Cbash-3.2$ exit

Of course, this is an extremely simple example, but you can imagine more complex setups. For instance, if your application depends on node.js and mongdb and some other programs, those can be specified similarly in the Nix expression. Then, all a user has to do is checkout the repo, run the nix-build command, and all those dependencies will be downloaded and setup. A script that starts required services is easy to write. As the default.nix file is part of the repository, switching to a different branch can also result in a very different set of components being made available in the shell, making switching between different projects and branches much faster.

I think setting up development environments is a very promising use case of Nix, and we’re starting to experiment with this internally at LogicBlox right now.

Because we’re essentially unifying development environments and deployment environments, we have now also specified a lot of information that NixOps needs to actually deploy this application to a server. We’ll get into that in a future post.