Javascript: A Language in Search of a Standard Library and Module System

  1. Array
  2. Boolean
  3. Date
  4. Error
  5. EvalError
  6. Function
  7. Math
  8. Number
  9. Object
  10. RangeError
  11. ReferenceError
  12. RegExp
  13. String
  14. SyntaxError
  15. TypeError
  16. URIError

Recognize that? Yes indeed, it’s the complete list of standard Javascript objects of Javascript 1.5. 16 objects, of which 7 are error objects. Of course, this is just the Javascript language by itself. In practice Javascript executes inside a browser, which gives it access to additional objects like the DOM, and all kinds of fancy HTML5 and non-standard browser-specific features.

When Javascript was just a little language that people would use to do simple mouse-over events, this was not a problem. However, now, more and more applications are written in Javascript in the browser and Javascript is used in more and more places other than the browser. Better yet I dare to bet you that in a few years time Javascript is going to be one of the most used programming languages in a growing number of domains. The growing body of Javascript code also means Javascript needs more than just those 16 objects. It needs a larger standard API and it needs a standardized way to modularize code.

At the very least it needs a de-facto standard way of doing object-oriented programming, having the choice between classical, prototypal, or, sure, why not lazy inheritance is not doing it for me. Different libraries use different styles. Worse yet, different libraries develop their own utility objects to produce classes, and if they’d at least agree on a common way of doing that, but no, there are a few dozen different way of implementing exactly the kind of inheritance you like. I’m not the biggest fan of Java, but at least they made a decision on this at the language level.

But I’m getting side tracked. The main two things Javascript needs to be a proper language that we can apply "grown-up" software engineering principles to:

  1. A standard library
  2. A module system

First, the standard library. Those 16 objects are no longer sufficient. So, what happens is that programmers step in to build a larger, more extensive library of objects. It would be great if they all teamed up to come with the one framework that everybody uses, but of course they didn’t, so we now have Prototype, MooTools, jQuery, MochiKit, ExtJS, YUI, Google Closure, and I’m probably forgetting about a dozen. This divides the Javascript world into different camps. Either you’re a jQuery guy or a Prototype guy, a ExtJS guy or a MochKit guy. There’s a YUI-y way of doing things and a Prototype way of doing things. There’s a jQuery calendar, an ExtJS calendar, and a Prototype calendar. Of course, once you picked a framework you stick to it, because if I want to use a prototype calendar from my jQuery application, I have to pull in x kilobytes of additional code that essentially duplicates jQuery functionality. A lot of effort is wasted because of this.

What Javascript’s standard library should contain is pretty tough thing to determine. First thing that comes to mind is a set of standard data structures. Of course Javascript has arrays and objects. Objects are typically also used as maps (although the keys can only be strings, and not even all strings) and sometimes even as set (where the property names represent values).

A better library of data structures is required, including a proper map and set implementation. Additionally, APIs for other common tasks such as:

  • JSON parsing and serialization
  • Testing
  • Cryptography
  • Date handling
  • DOM querying (I think most libraries agree that CSS selectors are a good way, right?)

Some stuff like DOM traversals, querying and widgets will be hard to agree on probably, but would be nice to have.

Because Javascript is increasingly being used as a non-browser language, for instance on servers, it will also need non-browser stuff like IO, inter-process communication, sockets etc. The CommonJS initiative is working on these. CommonJS is an initiative, mostly among Javascript server vendors, to agree upon certain standard interfaces to, e.g. IO, threads, sockets etc. Its main contribution to date, however, is its module system.

Javascript does not have namespaces, but you can use objects for this purpose. That’s fine. In the past, some frameworks have built their own module system around this. Dojo and Google Closure offer a remarkably similar API to export and load modules:

dojo.require('dijit.widget.Editor');
dojo.require('myproj.Something');
dojo.provide('myproj.MyObj');

myproj.MyObj = function() { };
myproj.MyObj.prototype.initialize = function() { ... };

Replace ‘dojo’ by ‘goog’ and you basically have the Google Closure version. Quite nice and reasonably clean. However, there’s also JSAN’s module system. CommonJS’s module system is really nice, however according to some reports cannot be implemented in the browser properly. There’s an asynchronous version of the CommonJS module system called RequireJS, but, well, it’s not the CommonJS standard. There’s a proposal to standardize the RequireJS system as part of CommonJS, but it’s not entirely clear what the status is of that proposal.

I feel these issues have to be resolved and the good thing is that it does not require any changes in the Javascript language itself, nor its browser support. If the different framework vendors would just agree on a single base library that they all use, because, let’s face it, everybody needs a function to trim strings and a proper set implementation, a clean way of doing inheritance, plus a module system to go with that, be it dojo/Google’s system or a CommonJS variant, I don’t care.

It would be oh-so-nice to have a de-facto standard library for this stuff.

Got something to say?
  1. Fortes says:

    FYI — document.querySelector and querySelectorAll are in the latest version of all browsers and work well for DOM querying.

  2. Dave says:

    Javascript could use a stanford library for sure, but with so many frameworks, who needs them? It's almost as if you can take your library with you anywhere you like.

  3. Zef Hemel says:

    Maybe I wasn't clear enough in my post, but I said this:

    > and the good thing is that it does not require any changes in the Javascript language itself, nor its browser support.

    As far as I'm concerned it would be fine with a standard library in a file (std.js, or base.js), that I would just include on every page, it does not have to be implemented in the browser at all. Maybe on the long term, for efficiency, but necessarily. Indeed, the nice thing about Javascript is that it is flexible enough to do this “ourselves”. So I would say the “but what about IE” discussion is kind of irrelevant, unless there's some core Javascript features it doesn't support that we absolutely need.

    Let me briefly tell you where I'm coming from. A month or two ago I started working on an O/R mapper for Javascript, built on top of the HTML5 SQL store (http://github.com/zefhemel/persistencejs). I wanted this library to be useable from any Javascript framework and supply an as “native” a Javascript API as possible. There were two problems that I came across:

    1) What is the proper way to do OO? I read “JS: The Good Parts”, but it discusses like 3 or 4 ways of implementing OO and variations on inheritance, each having different advantages and disadvantages, either using the new keyword or not etc. Personally I'm open to any OO style, just tell me which on to use. There was no obvious answer. There's a bunch of frameworks that offer utility objects to do inheritance “properly”, do mixins etc., but I didn't want to buy into any framework in particular. So eventually I went with the constructor function (function MyObj() { }, new MyObj()) approach, manually solving inheritance problems as I went along.

    If there would be a style, possibly with some utilities of doing OOP in Javascript that “everybody” would agree on, I wouldn't have had this problem. I'd just say “ok, let's rely on this std.js/base.js and it's obvious how to do it”. Everybody would understand my code, because they'd use the same style as well. Sure, it's a very powerful thing that we can do inheritance in 4 different ways, but it doesn't simplify communicating the fact that we're trying to implement inheritance, or how to use a particular constructor function (should I call new Bla() or just Bla()?).

    2) I needed to implement the observable pattern, I need to respond to changes in objects. Some JS frameworks contain an implementation of the observable pattern, but I don't want to rely on any framework in particular, so what should I do? I ended up implementing a lightweight one myself, which is included in the library. A bit of a waste of effort, plus of code size when somebody uses persistence.js together with another framework that also contains an observable pattern implementation.

    The observable pattern is another example of a construct that's common enough to be in such a standard library. If we decide on a particular implementation, we can all utilize it without being too dependent on larger frameworks. We only have a dependency on the small standard library.

    I'm currently working on a larger HTML/Javascript problem and there the modularization problem popped up. I want to declare module dependency and have them automatically be loaded. Again, Google Closure, Dojo and YUI provide ways to implement this, but I have been using jQuery so far, should I switch or add Dojo/Closure/YUI to my project, just to get the modularization features? That seems a bit excessive.

    If we would have agreed upon modularization features in a standard library that all the frameworks used, I wouldn't have this problem.

  4. Thor Larholm says:

    JSON parsing and serialization is already a native part of ECMAScript 5 and currently supported by all major browsers. IE8 was even just updated to ensure its native JSON object conforms fully to the ES5 spec.

  5. fearphage says:

    “It would be oh-so-nice to have a de-facto standard library for this stuff.”

    I disagree. I don't want to be forced to use Dojo, jQuery, Prototype, Moo, or any other library. I like the landscape now. We have options and choices. If you like X's syntax or usage pattern better, you can use that. If you prefer Y's, you can go down that path. Standardizing this would cause more hurt than harm.

  6. As an aside, you mentioned that “different libraries develop their own utility objects to produce classes.” This is a problem, but not for the reason that you state. Most, if not all of these methods have arisen as a means of providing a more familiar method of defining objects so that they look more like a class (like you would see in C++/Java/PHP/etc).

    The problem is that JavaScript doesn't have classes, but no one wants to accept that, so they make these funky constructs to force the class paradigm into the language.

  7. PS: Oh yes, and good points all around ;)

  8. asolove says:

    Many of these things would “be nice.” But “have to be resolved”? Have to be resolved or what? They have to be resolved or else JavaScript will remain the world's most widely-used programming environment?

    Pick a client library, pick a server-side library, (I recommend prototype + Node.js, but whatever floats your boat) and then enjoy.

  9. Thisisanick says:

    You should learn Javascript instead of trying to convert it into another language.
    https://developer.mozilla.org/en/Core_JavaScrip
    http://eloquentjavascript.net/chapter8.html

  10. Zef Hemel says:

    What are you talking about? Where do I try to convert it into another language? Because I want to apply the observable pattern?

  11. louis says:

    Thisisanick's comment was just rude and off base. Your the better man for even replying.

  12. louis says:

    er…”you're” the better man…sheesh. tired?

  13. dangoor says:

    CommonJS modules can be used in the browser (and we are successfully doing so in the Bespin project). The trick is that you can't just point a <script> tag at the module file. You need to either:

    1. use ajax to get the module
    2. use a build step
    3. use a tiny server side module wrapper

    The good news is that the ECMAScript working group is adding modules to the language and, from what I've seen, the current leading proposal follows a pattern that we (in the CommonJS project) were hoping for. Obviously, people tend to not get too excited about what's going on in the ECMAScript committee, because we care more about apps we can run today. But, my point is that CommonJS modules can be made to work today in the browser with minimal fuss and tomorrow's browsers with actual language-level module support will be even better.

    Also of note: the only widely implemented piece of CommonJS so far is the module spec. But, a lot of headway has been made in a few other areas (notably toward providing file access for server side/command line scripts). Non-browser contexts need a standard library for JavaScript to be a competitive language with other languages like Ruby, Python and Java.

    Regarding a larger “standard library” for the browser, that is what the ECMAScript and w3c ultimately provide. The JavaScript toolkits we all use help to inform how the standard evolves (the addition of JSON and document.querySelectorAll are good examples).

    Personally, I think that JavaScript is making nice progress as a broadly-applicable scripting language. It takes time to evolve because it has an incredibly massive user base, but luckily there's a lot we can do without changing the language (such as CommonJS modules).

    Kevin
    (Bespin product manager, CommonJS founder)

  14. Zef Hemel says:

    Interesting, so which of the three approaches for module loading do you use in Bespin?

  15. We use both 2 (building) and 3 (dynamic wrapping) for different purposes. We use the building technique for creating our standalone Bespin Embedded packages and dynamic wrapping for loading of plugins in the forthcoming “live” Bespin environment.

    We're using the Tiki loader created by Charles Jolley (as part of a forthcoming SproutCore release). The wrapper is really very simple and one could implement a wrapping routine in just about any language with very little effort.

    One final note: I don't use the ajax loading approach because I've found it hard to debug certain exceptions with Firebug in the past. This may be better now, so an ajax-driven completely client side loader may work just fine.

Comments are closed now.