I don’t mean to be mean, because I really like your blog and I read it all the time… but could you just explain to me the point of WebDSL? I honestly don’t mean it in any kind of negative way, I’m just wondering why you’re dedicating your valuable time to building something like that when RoR, php, et al. already exist.

Just from a cursory look, it seems like the syntax is halfway between Visual Basic (yikes!) and C. Why do you prefer that type of syntax to the Lisp-style syntax of Clojure/Compojure?

Good questions!

WebDSL was started about 3 years ago by Eelco Visser as an exercise the the design and implementation of domain-specific languages. His focus up to then had been on parsing and meta-programming, but it was time to focus on a new domain: the web. When he started Eelco had never built a web application. He investigated a number of Java frameworks as a basis and eventually decided to use JBoss Seam as a target.

The goal of WebDSL is to get rid of the boilerplate code you would have to write when building a Java application and raise the level abstraction. The vision was to have simple, domain-specific sub-languages that allow a programmer to specify a certain aspect of the application and the WebDSL compiler would generate all the implementation code for that aspect. Initially there were three sub-languages: a data modeling language, a user interface language and a simple action language to specify logic. As others joined the project (including myself), we added more sub-languages and more features: access control, workflow, data validation, ajax support and more recently search. Work is also done in the area of data evolution (i.e. migrating databases as you change your data model).

Although WebDSL is mainly a research project, we are increasingly working to make it useable by anybody with some programming experience. We currently have a few websites in production built using WebDSL (researchr, tweetview, webdsl.org and pil-lang.org) and the manual is growing.

The idea of building abstractions for the web itself is hardly novel. As Jay mentions, there are many web frameworks that already do this: Rails, Django and so on. There are a few things that we do differently in WebDSL, compared to existing frameworks:

• We create our own custom syntax. Whereas Rails and Django are struggling to express everything using Ruby and Python, respectively, we designed our own clean syntax. Whether you like this syntax is a matter of taste. Personally I like it, although, indeed, it inconsistent here and there.
• WebDSL is a statically typed and checked language. I wrote a number of posts about this issue and its advantages.
• WebDSL compiles to low-level Java code, which has good performance characteristics. The code we generate does not rely on run-time meta-programming and reflection features of the language which are typically rather slow.
• WebDSL is platform independent. We generate Java code now, but it can be ported relatively easily to .NET, Python or PHP. We have prototypes of this utilizing the PIL language that I developed.
• Within the next few months WebDSL will have excellent IDE support for Eclipse, built using Spoofax/IMP. My colleagues are working on this. It will feature syntax highlighting, as-you-type error reporting, code completion and eventually refactoring support.

A drawback that WebDSL has today is that it’s not trivial to install, but with the IDE plug-in and Java-version that should become a lot easier soon.

So, why am I putting so much effort into this? As you may be aware I’m doing a Ph.D. in the area of domain-specific languages, so we investigate how to best build them. WebDSL is a case study for us. Soon I intend to work on another DSL, in the domain of mobile applications (yes, a DSL to build iPhone and Android applications, people!). It’s interesting from a research perspective to see how to best do this.

In addition I regularly experiment with alternative ways of creating DSLs, like in Clojure and Scala. I’d like to see how far you can push these languages to build the DSLs you like. Clojure allows you to define your own custom syntax, in some sense, as long as you adhere to the rule of the parenthesis. Static error checking is much more problematic. Clojure is also rather tied to one platform, sure, there’s also ClojureCLR, but to write programs that work on both CLR and JVM is, well, challenging. IDE support for a Clojure DSL is also non-trivial.

On the other hand, the flexibility of a DSL like WebDSL also has its downsides. Basically you can design the language any way you like, both its syntax and semantics, you don’t get much for free. Whereas an internal DSL built on Clojure or Scala gets a lot for free: some error reporting, support for namespaces (something we still don’t have in WebDSL), a type system (in Scala’s case), an escape to a powerful language (Clojure or Scala) and a rich set of libraries you can use. In WebDSL we have to design all of this from scratch.

So in the end both approaches have their advantages and disadvantages. I intend to continue to explore them both.

This order was on purpose. I started with the most dynamic, catch-any-error-at-runtime language and ended with the most strongly typed we-try-to-find-any-mistake-at-compile-time solution. I’ve been accused of bashing these languages just for the heck of it, but there was a point to this, and it’s more than promoting WebDSL.

But let’s talk a little about WebDSL anyway. WebDSL is an external DSL for developing data-rich web applications. "External DSL" means that it is not built on top of an existing language like Ruby or Java, but that it has its own syntax and compiler. Developing such an external DSL is typically more work than developing an internal DSL, which is a disadvantage, but external DSLs have their advantages too.

Whenever we present WebDSL to people that have done some web development before, we get asked "Why is this better than Rails? Why not an internal DSL?" Although we always have problems answering this question, because none of us are very experience Rails developers, our answer comes down to a few general things that potentially apply to any external DSL:

1. We have a nicer syntax, as an external DSL we’re not limited to encoding things in a Ruby or Scala way
2. Being platform independent, we can generate Java, PHP, Python and C# code if we want to
3. An external DSL compiler can statically analyze your application to find common programmer mistakes

There’s also the runtime performance issue, but I don’t want to get into that.

DSLs built on modern languages like Ruby and Scala look pretty natural. But from time to time you gotta wonder, why do I have to put a colon in front of this word, why do I have to use quotes here? Because you are building on top of a host language, you have to obey that language’s syntax and evaluation rules, whether you like it or not. External DSLs do not have this restriction, they define their own syntax and interpretation of that syntax. 

The second advantage is clear, our language does not rely on any platform in particular and we could therefore generate code for any web platform. With Ruby on Rails you’re limited to, well, Ruby. Admittedly, there are Ruby implementations for the JVM and .NET now, but what about my little brother who can only afford cheap PHP hosting, should he just use… PHP?

We felt that the current state of the art in web frameworks is not very resilient to programmer mistakes. Typos lead to enormous stack traces and are often difficult to track down, right? That’s the thing I tried to verify last week.

It turned out we were pretty right. Although not everybody agrees.

Static verification of external DSLs

WebDSL is a statically typed language. It consists of a number of sub-languages. Currently we have a language for defining user interfaces, data models, business logic, access control, data validation and workflows.

To get a flavor of it, here’s an example entity definition. It defines a task entity:

entity Task {
  name        :: String
  description :: Text
  done        :: Bool
  archived    :: Bool
  user        -> User (inverse=User.tasks)
}

The nice thing is that we can detect all kinds of mistakes in WebDSL code at compile time. For instance, what if the inverse property of user in the Task entity does not exist?

$ webdsl build
Loading application settings (application.ini)...
[ webdslc | info ] stage 1: parsing webtasks.app
[ webdslc | info ] stage 2: importing modules
[ webdslc | info ] stage 3: typechecking
* webtasks.app:21/25: error: The field User.tasks does not exist
inverse = User.tasks

It points to exactly the right file, line number and column number. Or let’s say we misspell a property name in a page definition:

Result:

* webtasks.app:119/15: error: No property don defined for Task
task.don

Or, mistype a template name:

The error we get at compile time is as follows:

* webtasks.app:39/14: error: Template with this signature not defined 
outputt(task.done)

What if we define an access control rule for a page, or set of pages that does not exist:

Access control warning: unused rules: 
  rule page edit*(*)

We are currently working to also add more thorough checks for page element compositions. For instance, are rows defined within tables, listitems within lists and so on. We’re also working on IDE integration, so that your WebDSL programs are checked as you type.

No matter if you like the WebDSL language and its syntax or not, love or hate the formatting and wording of its error messages, abstract from all of that and the opportunity is clear:

An external DSL compiler can detect mistakes early and its error messages can be very clear and domain-specific.

Because we’re building our own syntax, compilers and typecheckers it is quite easy to add checks like these. Detecting mistakes in this manner is either not feasible, or at the very least more difficult in internal DSLs. In dynamically typed languages it’s extremely hard because of the language’s dynamic nature, in statically typed languages like Scala it may require building a compiler extension, which is also a non-trivial exercise.

Although you may say you would never make mistakes that are caught by checks like these, they have turned out to be extremely useful to us, as WebDSL users ourselves. But then again, that may be because we’re just all a bunch of NNPPs.

Criticism of the article falls into a number of categories.

Category #1: Denial

You are talking crap, those error messages are pretty good!

It saddens me that these error messages are perceived as "pretty good". It tells you something about the state of tools today. If we go back to more basic programming, for instance in Java. You write code which contains errors. You compile the code, the compiler tells you "this objects has no such method at …". "That class does not exist at…", "variable not declared at …" You may hate Java and everything it stands for, but admit it, having this type of error messages is pretty convenient, isn’t it? Wouldn’t you prefer clear to-the-point domain-specific error messages over a stack trace with an obscure error message? If you say that the error message makes perfect sense, that’s only because you’ve encountered it before and remember how to resolve it, not that the error message is in fact clear.

When I mistype the name of a controller in routes.rb and the error I get is "uninitialized constant HommeController", that is a bad error message. Better would be "At line … you refer to controller ‘homme’, but no such controller exists." Are you honestly going to argue with that? You may say, "sure that would be great, but this is enough for me, I can handle these types of error messages." And sure, you may find these errors acceptable and trade off well with the advantages of Rails as a whole, but do not tell me these error messages are perfectly fine.

Category #2: You think this is bad, try a Java framework!

Tell me about it. I have no intention of singling out Ruby on Rails here. I’m in no way a Rails hater. Rails was simply my first example to uncover there is a problem here. Later today I’ll publish a similar post about the Java Seam framework. Update: Here it is: When JBoss Seam Fails.

Category #3: You are simply a bad programmer!

I can’t help but immediately judge the author to be one of those NNPP‘s who spends days tripping over ridiculously simple issues because they never really grokked how to debug in the general sense. I’m not claiming the author is an NNPP, I’m just saying when you get hung up on the basics like this that’s the impression that is conveyed.

Another:

I hate rails for several reasons but either you just started programming, or you are trolling.

I like these, especially since I clearly pointed out:

To test this out, I followed Rails’ "Getting Started" guide and made "accidental" mistakes along the way, to see how the system would react.

I made these mistakes "accidentally" (note the quotes). Quotes imply sarcasm here. And I’m not even going to defend my programming abilities.

Category #4: Poor attention to detail is going to bite you in the ass as a programmer no matter what technology you’re using

True, but it is the tool’s job to help you as much as possible when you do screw up, which you inevitably will. Rails (as shown) does a fairly bad job at that. However, not everybody seems to agree, from the comments:

Wrong. It’s not the framework’s job to do anything. Rails isn’t for lazy people.

It’s not the framework’s job to do anything? Are you serious? Why do you use it? Isn’t the point of Rails to do less work, to make web development fun, to make work more productive? Isn’t making mistakes part of that?

Category #5: Where are your tests?

Would my error messages be less obscure if I uncovered them by writing unit tests? At most it would make localizing errors easier.

Category #6: If you don’t like it, go use Java! Screw you!

Very mature. When I post my article about Java Seam today I expect far less response, or at least far less radical responses like these. People tend to get religious about their programming language. Today Ruby on Rails is their God, so whomever says a bad thing about it, shall be damned. And screw you too, for that matter.

Category #7: Let me help you

There were a number of people that offered me help, teaching me how to debug these issues even offering to contact me and help me out. I appreciate that a lot. It’s sweet. And it’s great that there are such friendly Rails people. I am not a struggling Rails developer however. I am interested in Rails, because we’re developing WebDSL, a Rails competitor. I’m looking at various web frameworks and observed that simple programming mistakes are often fairly difficult to track down. In WebDSL we attempt to solve this problem. I appreciate the intent, though.

• The adaptive robot hand, a robot hand that can pick up any smallish object without breaking it, especially useful for picking up fruits en vegetables
• Delfi-n3xt, a mini satellite that they are going to launch a couple hundred of next year
• Bio-concrete, self-healing concrete (for streets and stuff)
• VertiGO, an electronic motorcycle that’s fast and good-looking
• DelFly Micro, a tiny electronic "insect"-like device, that can fly into dangerous areas and deliver streaming video
• DAISY, a technique to more effectively solve strabismus (cross-eyedness) in children

See what I did there? I explained in one sentence what these projects were in language that your mother would understand. Everybody will immediately see the opportunity and social impact of these inventions. Cure poor cross-eyed children, streets that frickin’ repair themselves, satellites, world peace!

Oh yeah, another project:

• WebDSL, a domain-specific language to reduce the amount of code you have to write for web applications, thereby reducing costs dramatically

Sure. That’s ok. But does it cure babies? Can you send it to space? Does it solve the climate problem?

Nah. It’s computer stuff, it’s supposed to be used by nerds that build web applications.

No, we didn’t win the award. 

A week ago we were at my parents’, a friend of theirs that I hadn’t seen for a few years was there to visit.

"So Zef, I hear you’re doing a Ph.D., what is it about?"

Now, in situations like this I no longer panic. Early on I came up with an analogy that seems to work:

You know how before before constructing a building, an architect makes an architecture of the building, right? So he draws pictures of what the building should look like and how it is structured. When this is done, the construction company takes those drawings and builders build the actual building.

Building software is much the same: people first draw pictures of how the software should work and then programmers actually implement this architecture. This programming takes up a lot of time and money, similar to the construction of buildings. So what we try to do is skip the programming/construction parts and let the computer do that for us. So we take the pictures that lay out what the software should do and generate all the programming code to make it work. It’s like letting a big machine automatically build the building for you, without human intervention.

People typically appreciate this explanation, as generic and oversimplified as it is.

"So you actually draw pictures of software?"

"Well, we represent those pictures as text in practice, that’s what many software programmers prefer."

And confused faces return.

People do not understand software, or the effort it takes to produce it.

It doesn’t speak to people, it’s too abstract. It’s not something they have been confronted with or are likely to be confronted with in the future. Unless your research focusses on preventing their Windows to crash or killing computer viruses. And even if it doesn’t, they’ll conclude: "Right… so you know about computers, right? Because I have this problem with Word…"

A project like ours is never going to win a design & engineering award because it’s too abstract, people simply do not care.