Reinventing Web Storage

I had this great idea a month or two ago: We are storing more and more data on servers on the web, be it email, pictures, music, calendars or documents. And most of it is scattered all around on different services with different interfaces. Can't we come up with a unified interface to storage and make our data mobile again? That was the secret project I have been working on, which I dubbed WebFS. It was a very simple REST webservice, which basically allowed you to GET and POST data items (documents). Each item had metadata associated with it (in RDF). There is a special kind of item, called the Container, which has a directory-like function. Every service supporting WebFS should allow the user to store arbitrary RDF metadata for its items, which would allow all kinds of neat services (such as synchronization) to be built.

I got quite a bit of it to work. I wrote a little bash-like shell (webshell) which allowed you to browse through a WebFS share and allowed you to copy items, view them, and get and set metadata. I built a WebFS interface for a simple local filesystem, but also for Jungle Disk. So I could access my files on Amazon's S3 (where JungleDisk stores the files) through WebFS. The idea was that I would also build a FlickrFS which exposed flickr albums as a WebFS system and so you can easily copy files from there to your local driver, or other WebFS-enabled storage systems. In the future other systems could be built, like IMAPFS, CalendarFS, and so on and so forth. WebFS would be the great unifier.

Nice idea huh?

Yeah, so great that it already exists. WebDAV does almost exactly this. It wasn't really built to create wrappers around existing systems, but it does allow you to access files on remote webservers pretty much in the way WebFS allows you to do that. In addition it allows you to store arbitrary XML data as meta data for your files. And the best thing, it's already integrated in all major operating systems. OS X allows you to mount WebDAV shares (actually that's how Jungle Disk works) and so does Windows and Linux.

Crap.

There are differences between WebDAV and WebFS, and choices in WebFS which can be considered nicer, but they surely do not constitute a new protocol while WebDAV is already well established. So basically I spent dozens of hours implementing something really elegant that already existed.

Great huh.

But what it did learn me is that we should have a better look at WebDAV for our services. Many services have, beside the standard web interface, some kind of API (usually a REST one). Wouldn't it be an idea to also add a WebDAV interface to those? This way data is truly open, truly free to the user and we wouldn't have a data lock-in, which I think nobody likes and is not very web-2.0-ish. Another possibility is to allow to store the data on a WebDAV store of choice for the user. Flickr then would just be a front-end to the Pictures directory I have somewhere on a WebDAV server. This is kinda what Zoho and Omnidrive are doing. It is possible to open word documents stored in Omnidrive directly in Zoho Writer (an online word processor). Problem is that they use Omnidrive's non-standard protocols. Not that I have anything against Omnidrive, but as of yet I have no reason to believe Omnidrive will be the ultimate storage solution for everyone.