Declare Everything

Let me start with a brief heads up, this post starts with a rather elaborate pitch for an idea, eventually building up to what my new job is all about. If you’re in a hurry and want to go all TL;DR on me, jump to about the middle of this post to find out what I will be doing.

Still here? Fantastic. So, here goes.

Here’s a snippet of mobl code. Mobl is a programming language I developed for mobile web development during my PhD, a few years back:

list(task in Task.all() where done == false
order by dueDate) {
item {

Even though this is likely the first snippet of mobl code you’ve ever seen, you should get the gist of what it does: it defines a UI for a list of tasks only showing tasks that are not done, ordered by due date. Each item in the list represents a single task, and is rendered as a checkBox bound to the `done` property of the task object and a label with the `name` property.

Ostensibly, this looks like a loop as you would write in any imperative language. It’s not, though. It’s a declarative declaration of a user interface that automatically updates itself when:

  • A new task is added that matches the done == false filter (the task appears in the list)
  • The due date of a task is changed (tasks reorder in the list)
  • A task is marked as done (it will disappear from the list)
  • A completed task is marked as done somehow (it will appear in the list)
  • A task is deleted (it will disappear from the list)

Since this is a language with data binding, what will happen when you mark a task as done, it will be disappear from the list, as it no longer matches the filter.

So, how does this work?

Internally the query is represented as an observable collection and you can subscribe to its `change` events. So, what the `list` will do is subscribe to this collection’s change events, and adapt its DOM representation (this is a web app) as `change` events come in. How does the collection know when to trigger a `change` event? Well, it knows it’s interested only in `Task` objects, and in the value of `done` and `dueDate` properties of those objects, so it registers itself to be notified of changes in those. Whenever a `Task` object changes its `done` or `dueDate` property, the collection object checks if this impacts the items in its collections, of not, it does nothing. If it does, it triggers a `change` event. How does a `list` update its content on `change` events? There’s a number of possible implementations:

  1. The naive implementation would be to completely re-render itself every time a `change` comes in.
  2. A slightly less naive implementation delays re-rendering slightly, batching up change events to prevent re-rendering 10 times when 10 new `Task` objects are added at once.
  3. A more sophisticated implementation (that I never got to implement) actually inspects the `change` event and updates the DOM incrementally — never re-rendering anything, but just changing the parts that need to be changed.

Trust me, I left out a _lot_ of little intricacies and corner cases that are handled. If you are one of those people who tried mobl, I bet you never thought or realized what was happening behind the scenes to implement these few lines of UI code. You also probably didn’t realize that between version 0.3 and 0.4 I switched the implementation of the `list` updates from (1) to (2), and at any time (if I were still to work on mobl), I could switch to (3) without you having to worry about it. All you’d notice is an improvement in performance.

I like mobl a lot. It allows its users to write very concise programs that do a lot of things with little code — and they do not have to worry about how it’s implemented; they just have to express their intention: “I would like a list of tasks to appear here, that only shows these tasks in this order, where each task is seen as a checkbox and a label. Make it happen!”

It’s about the _what_ rather than the _how_. Mobl users don’t have to worry about the how, I spent about a year of my life putting that knowledge into the compiler machinery so that you don’t have to think about it.

This is the essence of a _declarative_ program: focus on the WHAT rather than the HOW.

You are likely thinking: “Cool story, bro, how does this apply to my life?” Declarative languages are all around these days: HTML, CSS, SQL, Ant build files, npm `package.json` files — they’re all declarative:

  • HTML: “This is what the structure and content of my webpage should be — go render it.”
  • CSS: “If you find this HTML class somewhere, make sure it has this bright yellow background — make it so!”
  • SQL: “This is the data I’m interested in — go fetch it for me!”
  • Build files: “These are my source artifacts and how they depend on each other — go and figure out a way to build them.”
  • npm’s package.json file: “I have this piece of code, it depends on these versions of these libraries — go and make sure they’re installed.”

Don’t you want more of your programming time be spent _declaring_ what you need rather than spelling out the steps to accomplish it every time?

I sure do.

So, this is a rather long introduction to what I’m sure you’ve spent numerous sleepless nights over:

Zef left Cloud9, what is he going to do next?!

Enter LogicBlox:

LogicBlox is a cloud-delivered platform-as-a-service that enables the rapid development
of adaptive and actionable Big Data enterprise-class applications.

Buzzword explosion!



“Rapid development”

“Big Data”


So, what is LogicBlox really about? LogicBlox is a (still) stealth-mode start-up (hence the amazing website). What they build is a database and application server that is programmed in a purely declarative way. _Everything_ is programmed declaratively, not just queries, but entire applications (from front to back-end).

For this purpose LogicBlox revived a previously mostly academic and as far as as the world was concerned-dead language called Datalog and hired two dozen PhDs to extend it and build a kick-ass compiler and runtime for it. And guess what, it worked!

LogicBlox makes its money by selling its technology to other companies (like Predictix who use it (thus far) primarily to do big data (multiple terabytes) crunching for big retailers. Yet, the LogicBlox vision is much bigger: they believe Datalog’s declarative programming model is superior to the imperative programming model that is currently mainstream, and a lot of benefit is to be gained by going full-on declarative with Datalog:

  1. Declaring _what_ you need, rather than _how_ to get it, enables the runtime to optimize Datalog programs in all kinds of ways. A single Datalog program could be compiled to simple JavaScript to analyze bits of information purely on the client-side, e.g. in the browser, or be run on dozens of nodes in a cloud context analyzing terabytes of data. This is possible, because Datalog programs are abstract enough to allow enormous flexibility in implementation.
  2. LogicBlox is a live environment. LogicBlox has workspaces which contain facts (think: database rows) and rules (think: ways to derive new information from existing information). You don’t really run a Datalog program like you run a Ruby program, by invoking a Ruby interpreter on it. Instead you run it by injecting new rules or facts into the database and then asking questions about it. In a sense you could compare it to a SQL database with stored procedures.
  3. A live environment allows interactive development and opens up the market to end-user programming. It is often said that Excel is the most popular IDE in the world, and it’s true. When you use Excel, enter data and formulas, you’re basically programming. The cool thing is, you get instant feedback about all of your actions. Datalog allows the same thing (we “just” need to build a good UI for it) — therefore it allows for a lot of innovation on the IDE side as well.

LogicBlox’ investment in declarative languages extends to deployment of the LogicBlox software itself. LogicBlox employs the core people behind the declarative Nix package manager and NixOS, the Linux operating system built using it. Nix enables declaratively specifying the deployment architecture of a software deployment. Then, with a single command it can create the VMs, install the software you need. This is used by LogicBlox internally, but could be used for any complex software deployment.

All in all a pretty ballsy company in my humble opinion, so I felt I had to be part of it.

I started January 1st. I will likely focus on the user-facing parts: web service access, web UIs, tooling etc. First step: immerse myself in this whole new way of programming that I had no familiarity with whatsoever up to a few weeks ago. It’s very exciting.