Author Archives


30
Aug 10

Moving the Server to the Browser

I think that it’s well-established by now that the majority of desktop software will move to the browser at some point. It happened to e-mail, calendar, word processing and many more applications already. But here’s a crazy idea: what if we move the server to the browser as well?

The past week I’ve been playing with socket.io in mobl, my language for developing mobile web apps. Socket.io is a library that greatly simplifies building real-time web applications — applications that keep a connection (a socket) open to the server, allowing the server to push content to the client and vice versa. It was pretty straight-forward to build a library enabling use of socket.io from mobl.

The first thing I built was mobl draw (only works in modern versions of Chrome, Safari or Firefox). The client code, written in mobl, is pretty straight-forward. It uses the HTML5 canvas and responds to touch/mouse events. When you click/tap and drag, it will draw a line at that position and buffers your movements locally. A couple of times per second it pushes the buffer of movements to the server. The server broadcasts the update to all other connected clients. The result: everybody connected at the same time, can draw together in real-time. A pretty cool demo. In addition, the server keeps track of the entire drawing history, resulting in a quick replay of all the drawing that happened when you first open the page. The client part of mobl draw is written in mobl, of course, but the server component (albeit small) is written in server-side Javascript, using node.js.

Then, a colleague suggested building a game using mobl. I decided to build a multi-player snake variant. The client code was quick enough to build. The server-component, however, contained all the “interesting” stuff. The game logic was all written as part of the server, written using Javascript. Wasn’t this supposed to be a mobl exercise? Mobl currently focusses purely on the client-side of things, it does not (yet) have a server-side component. If I wanted to build the server in mobl as well, I had two options: (1) add a server-component to mobl, or (2) build a generic light-weight peer-to-peer message relay server in node.js once, enabling servers to also run in the browser.

I decided to do the latter. This brings back memories from networked games I used to play, where you could either join or host a server that would run on your computer.

So, in mobl there’s now a library called mobl::peersocket that has two types:

  • ServerSocket, to instantiate a server
  • Socket, to connect to a server

To start a server, you use ServerSocket.create:

var ss = ServerSocket.create("my-server",
             onconnect=handleConnect,
             ondisconnect=handleDisconnect,
             onmessage=receiveMessage);
ss.broadcast("Waddup!?");

Then, you can connect to it:

var cs = Socket.join("my-server",
             onmessage=clientReceiveMessage);
cs.send("Hello");

What this will do is establish a connection with the node.js relay server and register this mobl application as a server with the name “my-server”. Whenever a client connects to this server (using Socket.join("my-server")), the server app will be notified (onconnect will be triggered), and any messages sent to my-server, will be relayed to the mobl server app. All the node.js relay server does is handle the creation of servers, handle connection to servers and pass messages around between clients and servers. It does not contain any game logic whatsoever, all of that is in the mobl server code.

The result is running here. To start, it presents you with a list of currently running servers (if any), and a link to the page to host your own server. After picking a server you pick a player name and start playing. Multiple servers can be run at the same time and each has its own playing field. An issue seems to be latency. If the server does not have a low-latency internet connection, it can take a while for it to register left/right movements, which is sort of annoying. Of course, the latency is higher that typical, because every message between client and server is relayed through the node.js server. It’s a quite fun game though, one of my colleagues even built a simple bot using Greasemonkey.

Although it’s a thought-provoking experiment, what’s the point of running a server in your browser? In my case the reason was to be able to write the server in mobl as well, but how does that help anybody else? To be honest I’m not entirely sure yet.

It could be a secure way to let third parties host their own code on your servers. A third party would upload their code to your servers and they would run the uploaded server software in their own browser, which is completely secure — the server code can only communicate with its clients through your relay server and use resources available in the browser (e.g. a local database and CPU cycles). But who knows, there may be other reasons to do this as well. Any thoughts?


14
Jul 10

How Git Encourages Open Source Contribution

It is almost exactly 10 years ago that I released my first piece of open source software. The name of the project was YaBB and it was the first open source bulletin board/forum software written in Perl. Really — Perl? Yeah, that was the lingua franca of the web at that time — CGI baby!

A few months after YaBB’s initial release the team of contributors grew to about a dozen or so people. Our main collaboration tool was the YaBB software itself and e-mail. We did not use a version control system (who knew how to use CVS anyway). The main “repository” of source code was my hard drive, or even the FTP where the YaBB website was hosted. Some people had “commit” access i.e. had the FTP username and password and they contributed by directly uploading their changed files straight to the FTP — any changes were live immediately. Once in a while we would make a release (in the beginning about a few times a day, later less frequently).

Outsiders could contribute by submitting patches to us, or send us modified files via e-mail. But frankly, that did not happen often. The barrier to contribute was pretty high. People essentially had to be invited to join the team and get FTP access to contribute. Looking back, I think this greatly reduced the amount of outside contributions. And in case you were wondering, the project is still around — not very popular anymore, it seems — but its successor SMF seems to be doing pretty well.

Later on, as a consumer of open source software I encountered this issue from the other end. Even if a project did use a version control system, like CVS or subversion, it was mostly useful to stay up to date with developments of the project team — the people with commit access. If I made any changes myself, there was no light-weight way to contribute them back. Sure, I could make a patch and send it by e-mail, but that’s a lot of hassle. Who does that? In effect, my changes only lived on my hard drive and ultimately disappeared when cleaning stuff up, after I didn’t need them anymore.

About a year ago I started to use Git, a distributed version control system. That means that, rather than having a centralized source repository, every developer keeps his or her own repository locally. Locally, but maybe also remotely someplace, so that others can access it. For open source, a popular place to host such remote repositories is GitHub. I host a number of repositories at github myself.

So, how does “git encourage open source contribution” then? Well, in Git, forking a project is a very common thing to do. Effectively, as soon as you clone a repository (a “checkout” in subversion-speak), you keep your own copy locally. You can commit to that copy as much as you want, create branch, tags etc. Effectively you have now created a fork of the project, which you can push (upload) to your own project repository, on GitHub for instance. As an example, let’s consider persistence.js — my Javascript ORM library. According to GitHub, there are currently 7 forks of this project. The main one lives at http://github.com/zefhemel/persistencejs, but there’s another one at http://github.com/fgrehm/persistencejs, and yet another one at http://github.com/eegg/orm.js. That is, there are 7 people who made changes locally and pushed them to their own GitHub project repository.

So, what would you fork a project for, is that like a hostile take-over? Not really. People usually do it if they found some kind of bug, or want to add some kind of feature and publish those changes (or they want to port the whole thing to Coffeescript, lord knows why). They can do that by simply forking and pushing it to GitHub. If they want, the can send me a pull request, which is a request for me to pull in their changes into my repository, these changes can be merged into my main repository. I say main repository, but there is really no such thing.

So, does that actually work? Yes, it turns out that, indeed, it does. In the case of persistence.js, people have contributed bug fixes and the persistence.migrations.js plug-in (to handle changes in your data model) has been contributed by a “stranger”. With stranger I mean somebody who did not request to be “part of the team” (there’s not really a team), or to get commit access to anything. Just a somebody that happens to use persistence.js, and needed this plug-in. I have contributed fixes to a couple of projects started by others as well, such as nodejs-mysql-native, TouchScroll and congomongo. Doing so was a breeze.

If your open source project is still running on subversion — or lord forbid, CVS — and you feel you do not get a lot of outside contributions, Git (or Mercurial for that matter) is definitely something to consider. GitHub is a great place to host your code, but if you need more space (you only get about 300MB for all your repos together), Gitorious is excellent as well (you get virtually unlimited space there). Or use both!


2
Jul 10

Back on Android

When my iPhone was unusable last week, I had to temporarily switch back to Android. At that time an early build of Android 2.2 Froyo came out and I decided to install it, which was, when you think of it, kind of stupid — upgrading your back-up phone when the other one is unusable because of an upgrade — but I did it anyway. This upgrade succeeded, thankfully, and I must say that I like it a lot.

Android feels a lot snappier since the upgrade. I also love the fact that you can tether via wifi without having to hack your phone (as I did with the iPhone). It made me reconsider using the Nexus One as my main phone, and for the past week and a half, I have. To be frank, I’m pretty happy with it.

Sure, the iPhone is more polished and slightly more responsive. It is more user friendly. Android is more like Linux and Windows, and iPhone is more like the Mac. The iPhone is simpler, but with that also more limited. The Android is more complex — you actually have to learn to use it effectively, but when you do, it is more powerful.

Multi-tasking, for instance, is touted as the main new feature in iPhone. Except, it’s not really multi-tasking — it doesn’t do two things at the same time hardly ever (with a few exceptions such as playing music in the background), it just enables faster switching between multiple applications. Does that distinction matter? I thought it didn’t — my main annoyance on the iPhone was when I had to e.g. copy and past stuff from one application to the other; and in that case fast app switching is all you need. However, now that I’m on Android I experienced the advantage of true multi-tasking.

When I wake up in the morning my phone has automatically downloaded new podcast episodes to my phone. I do not have to connected it to my computer to sync them. When I have time to read some tweets, my twitter application has already loaded them; ready for me to read. That is something you simply will not get on the iPhone, currently. On the iPhone you launch or switch to the Twitter app, wait for it to connect to twitter and fetch new tweets, then you read them.

If you’re a (new) Android user, be sure to check the following applications out (just search for them on the Marketplace):

  • CallTrack, to log your phone calls to a Google Calendar.
  • Android Scripting Environment, run Python, Ruby and Javascript scripts on your phone. Program on the REPL!
  • Audible for Android (beta), are you and Audible subscriber? They got a great Android app now.
  • DiskUsage, where did all your storage go?
  • File Manager, a nice clean file manager.
  • Dropbox, are you a dropbox user? This application allows you to upload, download and manage files on your dropbox account (and edit them using other applications)
  • Fring, call other people, e.g. on skype. Also features video chat.
  • IMDB, a nice application for the IMDB movie database
  • Kindle, read your kindle books on Android. Includes whisper sync.
  • Listen, to download and listen to podcasts.
  • QuickSSHd (not free), a nice little SSH server so you can remotely log-on to your phone.
  • Rom buddy/SNesoid (not free), play all those nice old super nintendo games
  • Spotify, are you a Spotify Premium subscriber? Listen to all music on the go.
  • Terminal emulator, access your phone’s command line
  • Twitter, Twitter’s official twitter client.
  • Smart keyboard pro (there’s also a free version), a great keyboard replacement. The key feature of it is that there’s Dutch and Polish dictionaries available for it so I can finally benefit from auto corrections.

And as far as the paid applications goes. Did you know you can return a paid app within 24 hours and get a refund? No reason not to try these apps.


29
Jun 10

CSS3 Makes DOM Unnecessary?

Google to use HTML5 in Gmail (emphasis added):

Google’s current goal is to get Gmail to load in under a second. “Speed is a feature,” he said.

Early tests have proved promising. For instance, Gmail looks for those browsers that can work with version 3 of the Cascading Style Sheets (CSS), a standard closely related to HTML5. If the browser supports CSS3, Gmail will render the pages using these specifications, rather than its traditional approach of using the Document Object Model (DOM). The company has found that using CSS3 can speed the rendering time by 12 percent.

Glad to be finally get rid of that DOM.


22
Jun 10

iOS4: Upgrading Hangs; and How to Fix It

Yesterday evening, the evening before my birthday (which is today, yes, thank you!) I tried to upgrade my iPhone to the newly released iOS 4 (formerly known as iPhone OS 4.0). However, when after an hour and a half the upgrade was still at around 60% (according to my iPhone) and about 99% (according to iTunes), I got worried. I unplugged my iPhone, restarted iTunes. My iPhone was unusable and asking to be plugged into iTunes.

Crap.

Restarting the restore didn’t help. Same problem. Today after some searching I found other people are having the same problem. In case you find this post before the one I just linked to, here’s how I solved it: reinstall iTunes. First completely uninstall, and also clear out your Library/iPhone Software Upgrades/ (or something) directory. Then download and install iTunes again. That solved it for me, hope it will for you too.

Happy iOS4-ing!


8
Jun 10

Safari 5: HTML5, Faster JS, Reader

Apple released Safari 5 yesterday. Although I probably won’t use it beyond the testing of mobl web apps (yes, that’s what they’re called these days), it does have a couple new features. There’s the improved support for HTML5 features (including the Ruby tag, geolocation and websockets), a 20-25% faster javascript engine. The most important feature from a user perspective, however, will be “Reader”. What does it do?

Well, it will turn this:

Into this:

A cleaner, no nonsense reading experience. Very much like what instapaper does.


2
Jun 10

Internal DSL Implementation Techniques

Two weeks ago I gave a lecture about internal DSLs and techniques to implement them. Here are the slides, sadly I don’t have a video/audio recording of this one:


31
May 10

iPad: a Great Holiday Computer

Yesterday we came back from a 10 day trip to Coma-ruga, Spain. We stayed in a nice hotel, but like any hotel, you don’t really want to leave expensive stuff there while you’re out on the beach. Usually we take a laptop to watch movies and TV series during the evening, this time we decided to take my iPad, which conveniently fit into our room’s little locker.

The iPad is great for media playback, we managed to take about 10 movies and a dozen or so episodes from TV series we watch (I got a 16GB version). I bought a leather case that wraps around the screen and can also be folded over to the back to let it stand up, for instance on the bed, which worked out great.

Not only is the iPad great for watching video, it’s great for games too. My wife finally beat the adventure mode of Plants versus Zombies, and I played a bit of Asphalt 5 HD and Orbital HD.

When I finished the two physical books I took I also read the 97-paged sample of “Under the Dome” that I got through iBooks (I think I’ll have to purchase the whole thing now). And yes, I read it outside, albeit in the shadow. The screen brightness is good enough to read outside. Unless you’re reading in the sun, that probably won’t work so well.

But maybe my favorite feature was that although it can do all of the above, it is not a full-fledged computer that would invite me to do some work. If you’re carrying around a laptop, why not open up Eclipse and do some programming? On the iPad you can’t and when you’re on holiday that’s a feature for sure. I could have read one of the few dozen academic papers that I have stored on it, but I resisted. Who wants to read about functional reactive programming when you’re outside on the beach anyway?


19
May 10

mobl 101

Yesterday I posted the slides of my 41 minute long lecture about mobl. I also recorded video of it (through my MacBook’s webcam) and the result is acceptable enough to publish. So, if you have some time to spare, here’s me presenting mobl to a group of our students, a demo is included:

Mobl Lecture from Zef Hemel on Vimeo.

In a nutshell, mobl is a new language to develop mobile applications using web technologies. It’s a statically typed language that compiles down to Javascript and HTML.

Some interesting features:

Slides:

mobl
View more presentations from zefhemel.

There is no release yet, if you feel very adventurous, you can install Spoofax and then clone the mobl git repo to try it out.


18
May 10

Slides From My Mobl Lecture

I gave a lecture about mobl today. Here are the slides. I also recorded video, but I have to check if it is of sufficient quality.

mobl
View more presentations from zefhemel.