Objective-J and Cappuccino, with Francisco Tolmasky

280 North (bought by Motorolla) is the company behind the impressive 280 slides web-based presentation application (a la Keynote) and the framework it was built on: Cappuccino. Cappuccino is an oddball compared to other Javascript frameworks. First of all, it abstracts completely from HTML and CSS, and second of all: it’s not built using regular Javascript but with Objective-J, a strict superset of Javascript that adds support for classes and message sending to Javascript — much like Objective-C adds those features to C. Objective-J was one of the first languages that uses Javascript as a target language.

The Cappuccino framework is more or less a port of Apple’s Cocoa framework, and Cappuccino applications seem to share the same elegant design style that Mac applications have.

We talk to Francisco Tolmasky, creator of Objective-J, one of the founders of 280 Degrees North and former Apple employee.

I understand you used to work at Apple where, presumably, you picked up on the ObjC/Cocoa virus. Can you attempt to explain where the attraction comes from. Is it simply familiarity, or is it more? Is it the language, is it the framework — both?

I did in fact come from Apple, but ironically enough this was the period where I probably used ObjC/Cocoa the least. I was on the original iPhone team where I worked mainly on MobileSafari, so a lot of my work was in C++ WebKit (making it work on mobile, determining how “touch” should work on the web, etc). When I did do ObjC, it was either in WebKit bindings or in what would later become UIKit. So I didn’t touch AppKit at all during my time there (which is what most people probably consider “Cocoa”).

I think the attraction of ObjC and Cocoa is that they are just an incredibly productive environment when it comes to UI work. If you pick up Aaron Hillegass’ Programming for Mac OS X, it is astonishing how quickly you are writing real apps. I believe the first examples don’t even require any code. I think this further proved itself with the App Store. In very little time incredibly unique new applications came from almost nowhere, its just a great “getting things done” set of tools. What we wanted to do is bring that same experience to the web. This experience of course is more than just the language and framework, you have things like XCode and Interface Builder which are essential pieces of the workflow. I think Interface Builder is really unique because its one of the few visual development tools that is used both in simple and very complex apps, and produces actually good looking stuff, and has made the transition to mobile successfully, and has withstood the test of time.

So originally we weren’t sure how important ObjC and Cocoa were to this mix. The initial goal was simply to make it possible to use things like Interface Builder for the web. So the first stab we took was using pure JavaScript to make a “Cocoa-inspired” framework. As it turns out though, API design is really hard, and we were doing a lot of wheel reinventing. So the next step was to have the Cocoa APIs in pure JavaScript. Unfortunately what we found is that JavaScript makes it very hard to do things like classical inheritance and even importing code from one file into another (this is still only a half-sovled problem in “pure” JS and when we first started working on it it was like 5 years ago). Additionally, Cocoa method names don’t really work if you don’t have labels. So at that point we started experimenting with simple language translation. Our first take looked a lot like Java and was similar to how CoffeeScript handles classes in that we were just wrapping the prototype semantics up in our own custom syntax. This didn’t solve a lot of our problems though, such as the need for method missing and dynamic dispatch for a lot of Cocoa features, and once again trivial things like the lack of an “import” command.

When we were thinking about these problems we realized that we were basically already on an asymptotic path to really just re-implementing ObjC/Cocoa on the web, so we decided to stop artificially stopping ourselves half way and just go for it. This all of a sudden made things surprisingly simple for us, and it makes sense because the problems we were trying to solve were more or less the same that ObjC was trying to solve 20 years before. ObjC wanted to add a very limited set of features to, and remain backwards compatible with, C. We of course were just trying to add a very simple set of features to, and remain backwards compatible with, JavaScript, which has a very C-like syntax. So there wasn’t a lot of “philosophy” or “religion” to it, it was just a very practical solution to a set of very real and practical problems, and we ended up with this cool language that can give you all the interesting features of Objective-C (code importing that intuitively works, dynamic dispatch, method missing, classical inheritance), with none of its drawbacks, and that can inter-operate with pure JavaScript seamlessly. There are more details on the evolution of Objective-J in a blog post I wrote a while back.

Fast forward a few years and this investment has really paid off, allowing us to make full use of Xcode and Interface Builder, along with the plethora of existing Cocoa resources. Not to mention the interesting side effect that a lot of developers are able to very easily port their existing Mac and iPhone apps. It’s kind of cool that you can now pick up that same Programming for Mac OS X book and use it to make a fully functioning web app.

You mention ObjC/Cocoa are an incredibly productive environment. Do you have experience with other UI toolkit, e.g. WinForms, Java Swing, GTK? Can you identify where (and if) Apple/NeXT made different design decisions in their framework and why that worked out?

I’ve used a few other UI toolkits, including Java Swing and GTK, and have found them pretty frustrating. I think the magic of ObjC and Cocoa comes from two things: first, the very practical choices made in the framework, and second, ObjC and Cocoa’s participation in the whole stack.

The first point is more technical: I think Apple is really good at making the right compromises in their technologies. Objective-C is significantly more dynamic than C and C++ (e.g. you can treat it almost as untyped if you really want to), but it manages to do so without going overboard and really hurting performance. This is pretty subtle but really important I think. The “magic” really disappears if you have a nice language/framework and you create an app and then it feels like it will be impossible to get it fast enough. Objective-C has done a really good job of balancing these two elements, and over time evolving so that it never feels really outdated (the most recent example being the addition of blocks which are like closures).

The second point is less technical, but much of the magic comes from the surrounding environment: everything from tools to community. This is something that’s really easy to dismiss because most people developing technologies will probably consider most of this stuff “not their job”. For example, someone developing a language or framework may very well think that an IDE is someone else’s responsibility. However, especially with the App Store now, Apple really holds your hand throughout the whole process: from a great language and APi thats incredibly well documented, to a great set of tools you get for free (not just IDE, but all the performance stuff too), to finally publishing your app. The community is also focused on a lot the right things: such as good UI. It’s really hard and tangential to your normal duties to foster that kind of community, but its really important if you want to be a terrific UI framework. But I don’t want the takeaway here to be that ObjC/Cocoa are only good because of this surrounding environment, rather they are an excellent piece of a great whole. I think it would be hard to simply “swap in” a different language and framework without having to make major modifications to the other pieces as well.

When you ported AppKit and other Cocoa frameworks to the web, did you have any problems — did all the desktop metaphors easily port over to the web, or did you have to do certain things differently. Have you made improvements, particular choices different to the original Cocoa frameworks?

For the most part, most of the framework translates very nicely to the web, mainly because most of the concepts don’t change regardless of what platform you are on. A good example of this is undo: the mechanics of grouping undos and inserting them into your code is a very important and hard problem, but one that is platform agnostic. From an API standpoint most of the actual differences, while large perhaps “conceptually”, were not too difficult to handle programmatically. The most obvious I think is the asynchronous nature of the web. We had to add a lot of matching APIs for doing things asynchronously. I think where we were successful is in starting with Cocoa by default, and making changes when they made sense. This is a big win because again, most of the time the solutions that have existed for a long time just work (for example, all of Foundation works as is). That way we can focus on the really interesting problems when they come up, as opposed to having to split our time between them and the ones that are already solved. I think there’s a general and unfortunate culture of “starting from scratch” in the engineering community that really slows development down, where people try their new solution to a truly uninteresting problem first, and only revert to a tried and tested way if it fails.

I think most of the hard problems, whether technical or otherwise, had less to do with Cocoa itself and are just general problems any framework would have. A good technical example of this is the work we did to bundle all the resources of an app. On the desktop, this is not really an issue, but loading all the images and so forth needed in an online app is something everyone deals with. We had to come up with a lot of interesting solutions, like our “automagic” image spriter. A less technical example was determining the right “feel” for an app on the web. We knew we didn’t want it to feel like a web page, so we started on the opposite end of the spectrum and made it feel like a desktop app, and slowly retreated when it made sense. This is an ongoing process, and we still have a lot to learn. We just announced Aristo 2, our new UI graphics/feel designed by Sofa, which I think feels like a really good compromise. Sofa likes to call it “under-promise, over-deliver”, where your UI elements may at first glance appear more traditional web, but as soon as you interact with them you see they have all the capabilities of their desktop counterparts. In other words, the look is more web, but the feel is very desktop. I’ve played around with it on their latest Cappuccino apps and it really feels good.

The thing that people first notice when seeing ObjC/ObjJ is their extremely long message names. Programming languages vary wildly in their conciseness. Extremes never seem like a good idea: Perl is so concise nobody can read it. On the other hand, ObjC/ObjC seem very verbose when it comes to class and message naming. Can you explain why ObjJ made the right choice here?

I personally love the “long” message names, but I don’t think “long” really describes what’s going on. If you’ve never used a labeled method syntax, it’s easy to just think of a really long method which doesn’t do anyone much good. But in my experience after people try it, they really understand the benefit, especially when returning to old code. I think its also important to realize that the language is in no way “forcing” long message names, but rather they are a conscious decision of the framework designers. The fact of the matter is most programming is reading code, not writing it, so I don’t understand why you’d ever optimize for the 20% case. Objective-C/J really reads like english, you know exactly what’s going on most the time, and most importantly, you don’t have floating arguments in a method that you can only understand by looking up documentation. How many times have you seen the last argument of a method be “true” or “false”, and have had no idea what it could apply to? That never happens in Objective-C/J. Here’s another example, look at the appendChild API from JavaScript:

node.appendChild(node1, node2); I always see people mixing up whether node1 is being appended before node2, or vice versa. Again, this would never happen in Cocoa since the API would be:

[node appendChild: node1 beforeChild: node2]; Any new team member would immediately know what’s going on, which in my opinion is much more important than chasing some vague aesthetic ideal. After all, a program’s true conciseness doesn’t come from chopping chars off of method names, but from being able to express algorithms well.

Objective-J is built on top of Javascript, which is dynamically typed. Objective-J, however, requires type signatures for return types and arguments types. Why? Is this type information checked, is it for documentation purposes, IDE support?

So the “types” in Objective-J are completely optional (and they are actually optional in Objective-C methods as well). We keep them in mainly for documentation purposes, and because we do in fact have tools that catch a lot of common bugs by telling you that you are sending the wrong type to a method. I think most JavaScript languages document what types they expect, we’ve just elevated to a language level feature if you want it:

// With types  — (BOOL)containsObject:(id)anObject;

// Without types  — containsObject:anObject;

There have been rumors of Obj-J 2.0. For this you built language.js, a Javascript parser library that can handle parser errors well resulting in nicer error messages. What are the advantages of using this new library over the approach you took before? Are there any other new features coming in 2.0?

Objective-J 1 was written a long time ago when our requirements were very different. When we were making Objective-J, the only other group that had seriously tried building a language on top of JavaScript was GWT, so we were convinced we needed to nail the experience of loading an Objective-J file on the web and having it “just work” without a separate compilation step. For this reason, the ObjJ parser was designed to be incredibly fast at converting code, to the detriment of a lot of other features. SInce Objective-J is already so close to JavaScript, the parser actually identifies large pieces of code to skip altogether. This makes it quite fast, but of course means we don’t get the benefits of building a real AST and thus incorporating a lot of fancier features. That’s changing with Objective-J 2.0, where we are starting from scratch on a real parser that will allow all this. We’ll be able to support things like fast enumeration (kind of like forEach but without the slowdown of a closure, and additionally optionally “checked” for mutation during iteration), as well as some really impressive performance improvements all around (I go into the technical aspects of this in my CappCon talk that we will be posting on the website soon). And of course, the other nice benefit is really excellent error and warning support, probably even better than what the browser currently gives you.

To help me with this, I’ve started development on a new JavaScript parser generator library called language.js. Its still quite early and experimental, but language.js basically tries to solve a lot of the issues in making and, more importantly, maintaining a language that compiles to JavaScript. I think the most important aspect of this is to make it really accessible for people to be able to contribute to the language (and also of course to make new languages as well). One of the things we noticed when working on Cappuccino is that it was very hard to ever make changes to the parser. This is because parsers are generally hard to understand, and also because the performance implications were never obvious so when we did get someone who knew what was going on, we were still afraid to accept the patch. This would have been mitigated by using a parser generator like Jison, but then we would have had to sacrifice a lot of speed. Language.js thus has first class support for a lot of the things people would want to do to a language, while not hindering performance.

The best example of this is error support. I’ve added a new operator designed specifically to handle errors trivially and declaratively. If you are familiar with PEG parser generators, the “Naughty OR “operator (%) behaves just like the choice operator (/), but only gets used if the parse first completely fails. Because of this, performance is guaranteed to never be affected, regardless of how many error rules you add to the grammar. Thus, you are allowed to offer alternative “incorrect but valid” grammars to provide increasingly useful errors to your users. The idea is that as users file bugs when they get angry at experiencing an error that has a less than optimal error message, they can very easily submit patches to deal with the problems they encounter. For example, the following simplified snippet from the JavaScript grammar:

Identifier = !(ReservedWord !IdentifierPart) IdentifierName % ReservedWordIdentifier

ReservedWordIdentifier < Identifier can’t be a reserved word>= ReservedWord !IdentifierPart

generates the following error message with no additional code for “new = 5”:

new = 5; ^^^ ERROR line 0: Identifier can’t be a reserved word.

Another common error, a missing end brace, is handled just as easily:

Block = “{“ StatementList? “}” % BadBlock

BadBlock < Missing ending brace>= “{“ StatementList?

There is still a lot of work left in language.js though, which is why it hasn’t been rolled into Cappuccino proper as the official new parser yet, but if you want to contribute, you should definitely checkout the project.

Beside proper parsing and a foreach loop, what else will be new in Objective-J 2?

The feature set of Objective-J 2.0 is not set in stone yet. We are in the unique position now where all the crazy ideas we had in the past are now easily doable, so we are going to have a period where we have to weigh their pros and cons and choose only the ones that really make sense. Some of the ideas are present in Objective-C (such as properties), and others are of our own creation. Additionally, we have the choice of supporting a lot of the new JS features in browsers that don’t support them yet. One example that might be welcome to current Cocoa programmers would be the ability to enforce immutability on arrays:

immutable_array[0] = 5; // throws an error array[0] = 5; // no problem

This could be really interesting as it would allow you to return arrays from APIs safely without having to copy them, as well as possibly encouraging better programming in general. But the question arrises whether this would be confusing or not. An example of a unique feature we’ve been thinking about is much richer literal support for the Foundation types, such as CPSet. I have always been of the opinion that a lot of code abuses arrays solely because they’re “easier” to type (since they have a literal form), and this leads to a lot of misuse. Sets are a good example because people end up unknowingly writing O(n) code that should be O(1). This is something I’ve been thinking a lot about recently and will probably end up writing extensively about in a blog post at some point when I get the time. There are many more feature ideas that have been suggested and most of them have been discussed on our mailing lists so if this is something that interests you, you should check it out there.

Have you considered using Objective-J in a server context — seems like it could quite easily run on node.js — any interest in that?

Objective-J actually already works on the server, and did from the very beginning. A lot of people don’t know that the backend of 280slides.com has been Objective-J from the very start (and we did it back when basically your only option for JS on the server was Rhino). A lot of the code is shared and it’s really convenient since you can do cool stuff like archive your documents on the client and unarchive them on the server with the same model code, which is how our exporter works. You can even take it a step further and create entire graphic interfaces on the server and just send them over the wire really easily (think cibs). Luckily you don’t have to use Rhino anymore, and our tools support the CommonJS “standard”. We’re working on getting it all up and running on node too (you might have noticed language.js is all node). Its currently pretty trivial to get Objective-J running in node since it just compiles to JS at the end of the day, but we’d like to provide an official solution that makes it even easier.

During your JSConf ’09 talk you talk about working with the Webkit team to improve profiling and debug information for Objective-J by adding displayName to functions. I wonder, what else could debugging tools) do to better support non-Javascript languages running in the browser’s Javascript VM? displayName is great for profiling, but what about actual debugging (getting appropriate stack traces and line numbers, for instance)? Are you aware of any developments in that area?

I believe there is ongoing work in this field at Mozilla. Basically this provides the ability to map compiled lines to source lines. There are also hacks users can do themselves, for example a very easy one that’s popular in the Objective-J community is to edit the WebKit inspector file so that it accepts Objective-J in the console.

Every client-side web framework seems to be moving in the direction of mobile. Are there plans to move Cappuccino in that direction as well?

The mobile web, and specifically how it relates to applications, is an interesting area that I haven’t been able to really solidify my thoughts on yet. I think there are kind of two different ways to approach this question: “traditional” apps, and “magical” apps. I think Cappuccino is very capable of making some great traditional apps for the mobile web today. If you run 280slides.com on an iPad 2 today, even though it is in no way optimized for mobile, its really impressive how well it works. If we were to make it again today, I think it would be very possible to provide an awesome mobile web experience as well. The fact of the matter is that we went from Cappuccino basically not running at all to running really well in very little time, almost entirely thanks to the hardware advancing so quickly. I think very soon making these “traditional” apps will be no problem.

On the other hand, this isn’t very interesting beyond a technical and practical appreciation. What’s really interesting about mobile is the new experiences that only it can uniquely offer, the experiences that users are increasingly coming to expect. This is what I mean by “magical” apps. Even if we can solve all the performance problems (just a matter of time), and bring in all the necessary touch APIs (again, not very difficult), we still can’t give people the same “magic” as a native app. There’s no way that we the JavaScript developers can give people video recording for example, we can’t even give them access to the photos they took on their camera. Things like this will almost certainly come with time, but that is precisely the point: the mobile web isn’t blocked by any particular library, it’s simply blocked by its own lack of maturity.

GarageBand is the most exciting app I’ve seen on the iPad, and perhaps that I’ve ever seen in my life. When you use it, you really get excited about what mobile has to offer. Unfortunately today there is simply no way to make something comparable in MobileSafari. I think its a feat even in native. No JavaScript library can offer this today, and so I think its really disingenuous for anyone to really bill themselves as an honest alternative to native on mobile right now. On the other hand, I think the desktop has been ready for a long time. We see apps all the time that can very seriously compete and often outclass their desktop counterparts. The only remaining hurdle for the desktop web really seems to just be speed. But of course this wasn’t always the case, it took time for us to get to that point, and it will for the mobile web too.

Links  — — —