Defining a WebOS API

It has been a couple of years since everyone came around to realize that the web is the most important emerging application platform. All hands are now on-deck to fight the resulting API war. Because I am an embedded software developer, you might think I wouldn't have much of a role to play in this war. I hope you'd be wrong.

What might seem simpler to some might be harder to understand for others
Back in the day when computers were hard to use, many of the non-engineers that I know would make use of computers by writing small programs. Those programs might print their names across the screen or play sequences of musical notes. On occasion, those programs would be borrowed from a magazine and might encompass an entire adventure or arcade game. I wouldn't necessarily call this programming, but it meant that they understood how to edit a source file, invoke the compiler or interpreter, and run an application they could actually dissect themselves. How many of your friends not involved in writing software for a living can do this today?

Those were also days when people made backups of their floppy disks. Sure, part of that was because they had personal experience with data loss, but they also would lose smaller bits of data at a time to learn this lesson. What's more, their computers would still work without any time lost to restore a backup to a hard drive. How many of your computer using friends create a backup of their data today, let alone a bootable image with the programs required for using that data?

See also:
Microsoft BASIC Wikipedia entry
TRS-80 Model I Level II BASIC page

Don't get me wrong, I dread the idea of turning on my computer and seeing a BASIC "Ready" prompt looking back at me. I hope you'll admit, if not reminisce, that prompt let you at least know what state your computer was in and what it was "Ready" to do. The really good news is web browsers have given much of that know-what-state information back to computer users through the beauty of a hyperlink or URL. Typing a URL into the address bar of your favorite browser typically results in something you expect.

Since running web applications is easy, programming them should be easy too. If we can stay a little focussed, we might just find that is possible.

What is an API?
In the embedded world, I get pretty frustrated with how often a function call is confused with an API. I can accept that a collection of function call definitions could constitute an API, but a good API is often much more than that. An API is everything an application programmer requires to make use of a bundle of software that was created by someone else. So, a documented set of function calls into a library could be an API, but so could documented interrupt traps, messages, or even URLs. What is important is that you can give enough information to the application programmer to make use of your software bundle.

In the web application and web services worlds, there are lots of possible starting points for defining an API, all with various benefits and pitfalls. On the web services side, there are SOAP, XML-RPC, and many more. On the web application client side, there are Mochikit, Dojo, DOM, GWT, and many more. On the web application server side, there are Zope, Zend, Ruby on Rails, Mediawiki, Twiki, SharePoint, Blogger, Drupal, and many, many more. Gather it up and there is a lot of room for confusion.

Fortunately, as complicated as all of the support layers have gotten, the fundamentals of how most of the web works have remained relatively consistent. Data is fetched from URLs using HTTP's GET method and manipulated using two or three other HTTP methods. This thinking has been captured in the representational state transfer architectural style, or REST, as described by Roy Fielding's doctoral dissertation. Given average computer user's familiarity with fetching data using URLs and providing input using on-line forms, there is some potential for creating an API that people can understand.

A web service or web operating system REST API is somewhat akin to a desktop operating system call or software trap. In a system call, the application prepares some aspects of the CPU state then executes a CPU instruction to enter a more privileged set of code defined by the operating system. It is an important feature to not make these calls transparent, because this is where potentially sensitive information is moved to the network. These HTTP transactions are abstracted network transactions that most computer users can understand, because they already make these transactions frequently.

An early entry
YouOS has created something of an API for applications running within their WebOS environment. This API includes web application client library functions and server functions. YouOS has communicated their goals in the YouOS manifesto.

YouOS strives to bring the web and traditional operating systems together to form a shared virtual computer. To you, it's one giant computer that you and your friends can work on. To us, it's all the servers, routers, software, bandwidth, and engineers to keep this grand experiment in collaborative computing running.

I agree this is a noble vision for a web operating system, but this isn't necessarily in the best interest of consumers. At the least, some off-line capability, such as that described for Scrybe, should be included. Ultimately, there is also little reason to pay for storage and bandwidth if your home ISP connection is sufficient. I love the Amazon web services that YouOS utilizes to provide the scalability for Internet-wide software deployments, but locking customers into a single source provider for YouFS isn't in a consumer's best interest. Big players won't see a reason to use the service defined by YouOS because they can invest in their own infrastructure and new developers can't start small enough, which is confined to their desktop.

Provide an open source server implementation and client library
One answer is to provide an implementation of the server that is hosted locally. By continuing to use the HTTP interface definition, the location of the service, primarily data storage and retrieval, can instantly be made irrelevant. Further, it defines the necessary interface for authentication and provides a sandbox for web applications. A new developer who learns this environment can be given most of the lessons necessary to deploy their application across the Internet.

A pitfall that must be avoided is the creation of too many abstractions in the client library that are potentially more leaky than the HTTP interface. The programmer should be kept aware of the HTTP transactions that need to occur to perform their desired tasks. By providing a local server, significant services can be kept out of the client library.

I'm not saying that the client library should be avoided all together, but it shouldn't be seen as the entry to the operating system. The client library is more akin to libc for C programs or the standard template library for C++ programs, but with less need for portability. As long as a web application makes only the HTTP transactions supported by the web operating system, it should run just fine, no matter what language or client library is used.

Use microformatted XHTML, not a new XML schema
What shouldn't be avoided is defining the inputs and outputs of those HTTP transactions to be extensible, forwards compatible, and backwards compatible. A simple approach to this is to make use of microformatted XHTML for all queries and results. The server should still support HTML for robustness, but only generate XHTML. XHTML is something that every major web browser already knows how to parse and present. Microformats extend XHTML in a way that doesn't break that compatibility.

One huge lift from using microformatted XHTML is the ability to provide a service like Amazon's AWS Zone directly on-top of the web operating system services, without needing to understand the syntax of WSDL and SOAP. This self-documenting style is exactly what is necessary for those new to programming. This allows for cut-and-paste style experimentation to produce results that can be expected, such as how many people initially learned to produce HTML documents.

What all should a WebOS do?
Certainly it should provide data management so that you can get to your data wherever you are. I'd say it should go a bit further and define the paradigms for protecting and sharing that data with other users. Sharing content, especially media, is an area where Parakey seems a little bit more focused than the other WebOS entries. A WebOS should help you not only get to your data from everywhere, but publish and synchronize it with other users. (See: RSS, JXTA, and SyncML.)

A WebOS might need to handle some tasks where the programming environment of a web browser cannot execute tasks quickly enough. I've done plenty of assembly language programming in my career, but it is almost always only needed in a very small part of even complex algorithms. For the vast majority of software, the environment should optimize for ease of development. Dynamic type checking is better than static. Automatic memory management is better than manual. Programmers should only need to focus on the problem at hand. If the problem is the need to express some control flow, then just think about the software techniques required for that. If the problem is to create a high-performance video codec, something I do frequently, then worry about the individual machine cycles in your tight loops. Don't make programmers think about all of the problems at once.

The case of media manipulation seems to me to be one of those areas that is simply too intense for coding in JavaScript with today's interpreters. This is an instance where a proprietary interface, Adobe's Flex, seems to have an advantage over existing open source options. Hopefully, this is an issue that Parakey's JUL will address.

Should it also provide window management? This seems to be a popular function of the early WebOS entries, but I don't necessarily see the point. Web browsers already support multiple windows or tabs, though the idea of bringing up a desktop on remote computers in a predefined state has some appeal to me. What should be done by a web operating system is to clarify and distinguish the roles of software modules along the lines of the model-view-controller, or simiarly useful, pattern. This would allow web applications written against a WebOS to be immediately more accessible.

Conclusion
Some say that programming will always be difficult. This may be true for highly-optimized, widely-deployed, well-structured, and cutting-edge applications, but the definition of programming is a bit of a moving target. I'd be happy to get more non-engineering people introduced to a programming environment where creating the "Hello World" application is truly trivial and creating the next YouTube is truly possible. That environment, where you can be self-taught and all of the tools are at your fingertips, should be among the goals for any WebOS application programmer's interface definition.