2007-12-13

Criticism of the OLPC XO-1 concept

In John C. Dvorak's PC Magazine article "One Laptop per Child Doesn't Change the World", he writes:

Does anyone but me see the OLPC XO-1 as an insulting "let them eat cake" sort of message to the world's poor?
I like Dvorak and I often follow him on the TWiT podcast and CrankyGeeks on TiVo, but he polarizes issues in ways that sometimes aren't that useful, except for bringing attention to an issue. Hopefully the audience is paying enough attention to think for themselves, but that has proven repeatedly not to be the case.

In my view, if some service like "Mechanical Turk" pays living wages for these folks, then it was worth it. On average and over a lifetime, each of these students should be able to earn more than the cost/value of the computer.

Yes, the literacy rates and language barriers are an issue in making the computers useful at all. There would be, however, huge motivation to focus on literacy and additional languages, if some people are able to earn money with these machines.

So, Dvorak has given all of us XO enthusiasts a mission: enable students to make money using these machines by providing services like Mechanical Turk in the languages of the students and figure out how they can collect the resulting goods.

OK, I admit, this isn't a perfect idea. I've heard concerns that these laptops will be stolen if a market emerges for them and having them be a source of money would certainly make them valuable. This is also, to a degree, advocating some sort of child labor, which is a reality, despite the many objections we have in the developed world.

Better ideas?

2007-12-01

Blogger Beta Ships OpenID

Google has added new login options to Blogger, including OpenID. This is an important additional baby step towards a web with a single sign-on that allows you to have better control of your identity information.

read more | digg story

2007-11-20

N810 demo video from Nokia

Nothing too special about the web page, but the video gives lots of good angles and pictures of accessories. Sure would be nice if they used that GPS and a database to help find some WiFi hotspots. Too bad it doesn't have WiMax or EV-DO.

read more | digg story

2007-10-01

Accessibility is the killer mashup/webos application

After watching Douglas Crockford (of Yahoo and JavaScript fame) plea for Google, Microsoft, and others to participate in a mashup summit and reading some of the feedback around the web, I realized the critical application use-case is still missing.

Someone in the Q&A brought up a good example of a mashup implemented today in an undesirable method due to security reasons: Facebook accessing your GMail/MSN/... contacts to request more members. Contact sharing between applications is an excellent use-case for mashups, but I don't see it as a driving application. Certainly it gets to the heart of Crockford's talk: security is an excellent application for the Google Gears WorkerPools. If you are like me, you'll still be left thinking about how everyday web consumers will be motivated to download Gears, instead of walking down the questionable path of simply giving applications like Facebook access to all of your potentially private information.

Mashups are the most interesting innovation in software development in decades. Unfortunately, the browser's security model did not anticipate this development, so mashups are not safe if there is any confidential information in the page. Since virtually every page has at least some confidential information in it, this is a big problem. Google Gears may lead to the solution.
Security is important and is critical to the growth of new mashup applications and I'll be happy if that alone brings us worker threads and off-line support, but I think the killer mashup is the one that makes all of this great data exposed through APIs and structured web pages and makes it accessible in new ways.

Mark, of diveintoaccessibility and diveintogreasemonkey fame, who I admire for his vision of accessibility wrote in his blog post "if wishes were iPhones":
I don’t understand this continuing obsession with buying things that you need to break before they do what you want.
And with this thought I am reminded that the killer mashup/webos application is the one that takes all of those immensely useful web services out there and makes them measurably usable. And by usable, I mean giving the user control.

2007-09-10

Mobile 2.0 Conference

I'm thinking about trying to attend the Mobile 2.0 Conference on October 15th in San Francisco. Anyone have comments about this event? If there were a handful of people interested in chatting about how to enable creation of scalable web services generated from mobile/embedded devices, I'd make a point of going.

2007-09-06

No server, no satisfaction

There's just no satisfaction for the casual home user who wants to collaborate with friends. Even when dealing with a problem that has been solved many times over, it is really difficult without a server of your own and a fair amount of programming. The problem I'm talking about is planning events on a group calendar.

There is a group of somewhat over 25 people near where I live that frequently gets together to play outdoor roller hockey. We play in a parking lot in one of the area parks or offices. We have a mailing list on Yahoo, but most people are just copied on a repeatedly used e-mail thread. On that thread, the subject line is typically changed to match the proposed day and time. Every week, we all bombard each other with e-mails to make sure that enough of us are coming out to play. This has actually worked fairly well, but there have been some significant exceptions.

Sometimes we don't meet our threshold of 6 players and additional e-mails go out to entice people to sign-up to play. Calls are made. Threats are discussed. People who previously agreed to go attend might decline since they don't want to risk trying to play hockey with only 3 people. Chaos ensues.

One of they guys who used to come out regularly created a really simple sign-up page on a website. The site accepted a name, e-mail address, and phone number to sign-up for a given game. The name is listed beside the entry for that game. The e-mail was used to send out the "game-on" or "need-more-players" notification a few hours before the prospective game. Phone numbers were included to speed up the communication.

The application worked quite well and was simple-minded. Entering the same sign-up information twice would result in being removed from the list. The phone number and e-mail information had to match, providing a tiny amount of security from folks simply removing everyone from the list. No verification of the entry was done, but you can imagine a simple verification code being provided via SMS to the mobile phone number if we ever started to have problems with that. Life was good.

He stopped playing and his site stopped working. We were back to using e-mail. A few folks reminisced about the good 'ol days when we had our own web server. I own the domain name, so I decided to bring back the sign-up sheet. Where should I host it?

I don't really like the idea of pointing people to my home computer, so that's not my first choice. I don't really like the idea of paying for a full-featured (LAMP and/or Ruby enabled, ie. scripting and a database) hosting service just for this hobby. This is just calendar data! Why should I need a web server to do something that Yahoo and Google provide for free?

Our mailing list is on Yahoo, so I looked first at their group calendar. It was a disastrously complex to use and didn't provide any of the custom features we had with the much simpler web app. Similar problems were had with Google's calendar and Evite. The most fundamental issue with all of these calendaring solutions: they required account creation and login to utilize.

I tried to see if I could do something with static hosting, but it seems even Amazon's simple queuing service doesn't seem to work without having a dynamic host. At this point I gave up, but I'll get back to this application yet.

2007-09-03

Still don't get the whole WebOS thing?

I've gotten a bit smarter about explaining why there will be a sort of emerging web operating system to the people who inquire. For example, I've started calling it a "web services kit", instead of an operating system. Today's tech savvy minds can accept the idea of yet-another-SDK, whereas the idea of a web operating system is either tainted by the webtops or seen as inconceivable and unnecessary delusions to compete with Windows, Linux, or OSX.

What I haven't leveraged enough is the great summary of web service APIs provided by ProgrammableWeb. From their simple scorecard, you can get a quick overview of the categories of popular services and some of the key players. Ask yourself what sustainable advantage do any of these players have within their service space. Don't get fooled, it isn't an easy question. Keep in mind that standard service definitions are coming into existence for most of these services, such as XMPP for chatting and OpenID for identity. Take up the exercise to look across these service APIs, look for winners, and look for emerging standards.

Comfortable? Now, realize that it is only a matter of time before there are standards-based implementations of all of these services. Sure, it might take a while, but it'll happen.

If you are quick, you might be sighing and thinking to yourself, "what about the data?". I'm glad you asked, because that is really the point. These services are all about controlling access to data and looking for ways to monetize it. You might stumble over the idea that on-line office applications involve an incredibly complex pile-o-code, but then you'll remember that you already have 2-3 other viable choices of office applications to which you already had access. Over the long-term, it is all about the data.

Still don't feel like you're any closer to accepting the idea of a web operating system? That's okay, as long as you recognize the benefit of something that provides you with the capability to control and monetize access to data and some sort of well-understood integration layer back into your application. You'll come around when you start thinking about who you want to own your data.

2007-08-25

BarCampHouston

I'm heading up to BarCampHouston today. I don't know what to expect and I haven't had any free time whatsoever, so I won't put on much of a demo or presentation. I'll be happy to share any info on P2P collaboration and my Maemo SDK EC2 image, if anyone is interested. I'm also looking forward to simply discussing toolkits for building web services (web operating systems).

By the way, I've moved my EC2 script over to http://sdk-ami.garage.maemo.org.

2007-08-18

Boot your N800 Maemo SDK today with Amazon's EC2

I really appreciate that someone has created VMWare and QEMU images for running the Maemo SDK. Unfortunately, my machines, both Mac and PC, are typically too busy with other stuff to allow me to quickly fire-up a virtual machine image that will chew up all my computing resources. Instead, booting up a machine from Amazon for about $0.15/hour or so is affordable enough for my N800 development. No more downloading a 1.5+GB image; EC2 users can instead just share an image and a boot script and be up-and-running at a known-good starting point.

Of course, the first step was to get an EC2 (and S3) account. I waited almost a month for my EC2 account. At some point I'll figure some way to let other people just rent a machine from me to make it easy, but that'll require a bit of thought and management. For now, head on over to Amazon, request an account, and they'll get to you eventually. There isn't any monthly fee or hidden costs; you just pay for the time, bandwidth, and storage you use. As long as you copy your work off somewhere else, such as by using Subversion hosted by code.google.com, you can shutdown without having any recurring fees.

Once you have an EC2 and S3 account, you'll want to download the EC2 command-line tools and your access identifiers.

The next step was to choose the Linux image I wanted to use as my starting point. Personally, I'm a Gentoo fan because I think Linux has an excessive number of binary-compatible dependencies on the C library and Gentoo solves that by recompiling every new application, instead of needing to update your C library to match the binary-compatibility requirements of all your applications. Of course, that makes application installation slow and Maemo itself uses the Debian package model, so Debian or Ubuntu make the most sense. However, Amazon supplies some nice reference images on Fedora Core 4 that might simplify my life around issues like ssh login security when the root account password can't be secret. Nothing is easy, so I found an Ubuntu image that has reasonable documentation on how it was created such that someone could redo this all with a better supported AMI in the future.

At this point, the issues started to pile up and I decided my best hope was to document my steps in scripts so that I can reproduce them with a better starting image and make corrections that people point out to me. I decided to the Google Subversion server I mentioned earlier to host my script at http://code.google.com/p/maemo-sdk-image/.

I used a Mac to run my script, but I plan to eventually make it run on the N800 itself or on a Windows PC. Right now, the script uses bash, which isn't natively on either the N800 or Windows. Also, the Unix-style version of the EC2 command-line tools also utilizes bash. I think the solution for both is likely to install bash.

The access identifier information ended up placed in a subdirectory called 'secrets' under where I ran the script. These secrets end up getting copied temporarily to the EC2 images for the purpose of bundling them up. I exclude that directory from the bundle.

I say 'images', because I end up working with four different EC2 images in the script. The first one is the base Fedora Core image that Amazon makes public. The second one is a bare-bones Ubuntu Feisty image that can be boot on EC2. The third one is patched to be self-bundling. The fourth one actually contains the Maemo SDK. The third and fourth could easily be combined, but I am still inching along.

In theory, you can run all of the steps using 3 separate calls to the script:

  1. ./build_maemo_api build-feisty
  2. ./build_maemo_api patch-feisty
  3. ./build_maemo_api install-maemo
I think the first two should work reasonably well to create a self-bundling Ubuntu image, but I haven't run them exactly like that to test them out yet.

The third step certainly won't work. One issue is that the Nokia binaries require you to agree to a license, so you'll need to do that part manually. This also only gets you to version 3.1 of the SDK, so you'll need to update that as well. There are also several steps missing before the image is really usable, such as setting up the X server and VNC server to allow you to view the emulated N800 screen remotely.

Please feel free to post your comments here or on the wiki on how to improve the script. I won't hesitate to utilize your inputs on the script hosted on the Subversion server.

2007-07-27

Working on an Amazon EC2 AMI for the Maemo SDK (scratchbox on Ubuntu)

I'll get into why I want to create this Amazon EC2 AMI thing later, but I thought I'd get information out there on a problem I'm having. Tve did a nice write-up on RightScale on bundling-up an ubuntu EC2 instance. This is a really helpful write-up, but I get a shell script error and a hang when I try to bundle my Ubuntu image.

root@domU-...:~# ec2-bundle-vol -d /mnt -k ~root/pk-....pem -c ~root/cert-....pem -u ...
Copying / into the image file /mnt/image...
Excluding:
/sys
/var/lock
/dev/shm
/proc
/dev/pts
/proc/sys/fs/binfmt_misc
/var/run
/dev
/dev
/media
/mnt
/proc
/sys
/mnt/image
/mnt/img-mnt
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.003492 seconds, 300 MB/s
mke2fs 1.39 (29-May-2006)
warning: 256 blocks unused.

Bundling image file...
sh: Syntax error: Bad substitution
The script keeps running. I don't know Ruby well, but the script seems to be stuck in a shell call to 'openssl'. This seems to occur in bundle.rb line 51. It looks like the 'tar' call on line 57 that is meant to feed the pipe being read by the running 'openssl' died, but I don't see where the "Bad substitution" might be.

Tve didn't have a place to make a comment on the blog entry. Time to debug...

Quick update on when I canceled the process (need to think if this confirms or denies what I was thinking):
sh: Syntax error: Bad substitution
sh: cannot open /tmp/bundleimage-pipe1: Interrupted system call
error executing tar -chS -C /mnt image | tee /tmp/bundleimage-pipe1 |gzip | openssl enc -e -aes-128-cbc -K ... -iv ... > /mnt/image.tar.gz.enc; for i in ${PIPESTATUS[@]}; do [ $i == 0 ] || exit $i; done, exit status code 2
ec2-bundle-vol failed
...need to take a break and get back to my real work.

Update #2: The problem turned out to be assumption in Amazon's Ruby script that 'bash' would be the shell executed by default. The answer was within a script found on the Amazon developer's forum. I've created my first image and I'll be hosting my script using Google's open source code repository when it is done at http://code.google.com/p/maemo-sdk-image/.

2007-04-27

Judgment vs. Jealousy: Jadon on Twitter

After hearing so much talk about Twitter on the blogs, podcasts, and vlogs I read, hear, and watch, I decided I had to know what the fuss was about: http://twitter.com/Jadon. How do I manage to find time for this sort of waste?

2007-04-19

Google AJAX Feed API

I read Udi Dahan's post on how Google's Ajax API Simplifies Safe Mashups. It didn't take 5 minutes to take Google's "Hello World" AJAX Feed API example and embed into a web page of my own. This is fun stuff allowing me to finally be able to manipulate feeds across sites without injecting any server-side code.

What should be noted is that this is relying on Google services and must follow their terms. It means that they monitor all the content this API brings into my page while other search engines cannot. (Crawlers do not typically execute the JavaScript, so that's why they wouldn't see the results of the API calls.)

Certainly I'm not the only one disturbed by this, right? Where is the counter-movement to give the every-blogger ownership of his own services in just as simple a manner? Is there a better answer than Amazon's web services? (For those that don't know, Amazon provides hosting solutions that are API-driven, highly customizable, scalable, and billed-by-use.)

Google's solution is so simply by comparison that I am struggling to remember why this even matters to me. Something in my gut just keeps telling me it is wrong to rely on services where I can't understand the business model.

Also distracting from this actually-quite-cool service from Google is the fact that Yahoo! Pipes already offered this feature and many more. Additionally, I believe that Yahoo! doesn't require you to register for an API key against your URL, though there is a need to register to create a pipe (feed) if you aren't using one someone else already created.

P.S. If you are listening, sorry I'm not keeping up and haven't even uploaded the rest of my notes on my P2P collaboration presentation. I have lot's of activity at work these days keeping my creative energy going without having to resort to my blog rantings. :-) But, this was such a quick post.


Powered by ScribeFire.



Technorati Tags: , , ,

2007-03-17

A CIO that "gets" open source and collaboration tools

I've been really busy lately and will get back to the P2P Collaboration tool post soon. I've really been enjoying my new job and have been spending a bit too much time at it. :)

I was trying to catch up on a bit of my reading and I ran across a fantastic 50 minute video on open source in the enterprise when reading Stu's blog that I had to stop and share it really quickly. It is really worth the time. Be patient. If your job is related to information technology, you must hear this and consider it.

CIO JP Rangaswami breaks down the economic justification for using open source in the enterprise and many of the reasons we need to tear down the walled gardens. There are some specifics he gives for banks, but most technology companies will have very similar issues working with other companies. Any company taking an early stand for open source collaboration tools can gain the benefits of better recruiting as he explains. He also explains how the various tools are consolidating to standards and a small number of fundamental operations.

When I listen, it is more and more justification for the coming WebOS.

Again, the link is http://www.infoq.com/presentations/jp-rangaswami-open-source.

2007-02-08

Peer-to-peer Collaboration Tools

I gave this presentation at an internal company conference last week. Large corporations suffer from different collaboration issues than the open source world, but we also have much in common. My hope is to show folks at my company some tricks that we can learn from the open source world. Open source development is almost certain to be globally distributed and using on-line tools for almost all of the communication.

[Introduction]

It is rare that I get to speak about a topic for which I have such a great interest and I know will have such a great impact. The scale of the impact is on par with the emergence of e-mail as a technology. In fact, I don’t believe it will be very long before peer-to-peer-supported blog-like technology replaces e-mail as the primary communication mechanism over the Internet and corporate networks. Perhaps 3-5 years.

The title of my presentation is "overview of peer-to-peer collaboration technologies" and is sub-titled "managing communications with a global workforce". I chose this subtitle to emphasize that these technologies will help you address some of the many challenges of working in global teams. Communication is a fundamental problem of business that is complicated by globalization. I hope to show you here what is happening now to solve the communication requirements.

Purpose of this presentation
The purpose of this presentation is to illustrate that the emergence of peer-to-peer-based collaboration software is inevitable, show why [our company] should embrace peer-to-peer software architectures, and give an overview of some current peer-to-peer, metadata, and collaboration technologies. Some peer-to-peer collaboration architectures, such as Microsoft Office Groove 2007, will quickly and significantly alter the client-server architecture used for most existing content management systems, such as SharePoint, TWiki, and [our proprietary systems].

These peer-to-peer tools will allow the elimination of many IT dependencies. IT will obviously have significant roles to play, but those roles will change. With the emergence of these tools, team productivity can start almost immediately. Additional productivity and stability can then follow with more formalized IT involvement. For example, someone needs to create bridges between all the products and protocols. The various tools need to be combined into a single product with a consistent look and feel. The costs of deploying a collaboration solution can be optimized. Additional points of access and different accessibility features can be provided. My suggestion is that checkpoints requiring resource allocation, however, can be eliminated until there is a business benefit for IT involvement.

This isn't as much a choice I'm suggesting we make as it is the recognition that existing forces will drive acceptance of peer-to-peer related technology. On-the-go collaboration is a critical requirement of global business communication tools. Information locked into what can be called "walled gardens" or server silos cannot benefit all of the necessary people at the necessary times. This is certainly true in a global workforce of partnerships, third-parties, outsourcing, and ODMs. While business communications regulations, such as Sarbanes-Oxley and other E-Discovery requirements, will make the process more complex; they make it no less inevitable. The required technologies will survive by natural selection.

Tasks, e-mails, notes, and calendar items are mostly the same: they are a bit of collateral communication along with some specific metadata to allow the tool to know how to advise the user. There should be a tight association between the collateral, or content, or microcontent, and the metadata. These digital communications must be preserved in context. New metadata types, such as test case and product requirement associations, must also be supported. Making special tools to handle these bits of metadata is a formula for disaster.

The madness of creating new centralized walled gardens of highly specialized data repositories must be replaced with a vision of building new interoperable data definitions, widely accessible visualization tools, and improvements to existing communications infrastructure.

In this presentation, I will cover:

  • how I got interested in this problem,
  • why we need collaboration tools,
  • why collaboration will go peer-to-peer,
  • what a peer-to-peer collaboration tool might look like,
  • why should [our company] continue to evaluate peer-to-peer collaboration technologies, and
  • how you can get involved.

How did I get interested in this problem?
[Our company] entered the portable media player market in 1999. This market largely grew out of the popular usage of peer-to-peer file sharing applications, such as the old Napster. These peer-to-peer file sharing applications provided a source for content that established distributors weren't yet willing to provide. Learning about the architecture of these systems convinced me that they were useful for far more productive arenas than the sharing of pirated music.

The most critical aspect of making both peer-to-peer file sharing networks and portable media players work for people is the management of metadata. Metadata is the information associated to the content, such as the artist, title, genre, format, or rating. With all of the possible content available for download and playback, user satisfaction is driven by the ability to find desirable content quickly and easily using metadata. Apple's combination of the iPod scroll-wheel and the iTunes Music Store, along with a bit of slick advertising, gave people answers to where to get to music they wanted quickly. It led them to great success, without necessarily relying on peer-to-peer file swapping. Still, file swapping networks are still the quickest and easiest way to get to some content and they remain in wide use.

Failed attempts to defeat the networks convinced me that the technology will continue to be widely adopted, despite organized attempts to the contrary. The file swapping networks that are still in existence survived because they use peer-to-peer architectures. In a peer-to-peer application, there isn't a central server that can be shutdown to disable the network. Peers directly share metadata with each other, providing a path for sharing content. To build a peer-to-peer network, you only need peers speaking the same protocols and a willingness to participate.

The thing to remember here is that intelligent managing of metadata is how content is found, no matter what the platform.

Another experience I had was when this team went global. My job quickly shifted from a focus on media player technology to a focus on information management. I spent my time worrying about version control, status reports, bug tracking, requirements management, and portfolio management. The communication was less-than-efficient and the results were less-than-desired. This had nothing to do with the talent of the team, but had much to do with the communication mechanisms and processes that were in place. An 11 hour time-zone difference is literally one world away.

I saw that my experience in peer-to-peer technologies and metadata management could be applied to solve my new information management problems. Just like getting to media content quickly and easily helps media consumers, the product developers need quick and easy access to information.

Why do we need collaboration tools?
Communication in a global team cannot be handled entirely face-to-face. No matter how many worldwide face-to-face meetings you hold, when you break up and get back to work, there is always something left unsaid. Having frequent teleconferences helps close communication gaps, but they impact the work-flow of team members and never provide the depth of conversation required to establish a cohesive team.

While there is no substitute for strong individual communication skills, collaboration tools provide a path for competing with otherwise more convenient information sources and distractions. On-line tools provide opportunities for collaboration between people where distance, time, cultural, language, experience, and ability barriers exist.

You can see this in the success of open source projects like FireFox, which is a web browser that competes with Microsoft's Internet Explorer. Contributions to open source projects come from people all over the world in many different situations. To make open source projects work, the participants make extensive use of on-line collaboration tools.

Also note, the communication in these tools isn't one-way. The messages being communicated aren't dictates from a single expert telling everyone how to solve the problems of the whole. Requirements and solutions come from every member of the team and beyond.

Why will collaboration go peer-to-peer?
First, what do I mean by peer-to-peer? Back in 2001, Daniel Bricklin gave a speech at the O'Reilly P2P Conference on "The Cornucopia of the Commons". That speech was later printed in a great
O'Reilly book. He quotes a 1968 essay on "The Tragedy of the Commons" summarizing a commonly expressed problem:

Therein is the tragedy. Each man is locked into a system that compels him to increase his herd without limit--in a world that is limited. Ruin is the destination toward which all men rush, each pursuing his own best interest in a society that believes in the freedom of the commons. Freedom in a commons brings ruin to all.

That isn't such a pleasant or welcome idea, but we can all see some degree of reality to that view.

Dan goes on to offer another view in the light of successful peer-to-peer architectures and their failings:

In the case of certain ingeniously planned services, we find a contrasting cornucopia of the commons: use brings overflowing abundance. Peer-to-peer architectures and technologies may have their benefits, but I think the historical lesson is clear: concentrate on what you can get from users, and use whatever protocol can maximize their voluntary contributions. That seems to be where the greatest promise lies for the new kinds of collaborative environments.

That is the thinking that launched the Web 2.0 explosion with businesses like Flickr, MySpace, and YouTube. Similarly, when I'm referring to peer-to-peer technologies, I'm talking about creating protocols that maximize the contributions from the edges. Those contributions generate a wealth of information and content highly valued by users.

The elimination of a required centralized server is one common technology applied in peer-to-peer architectures. The benefits that approach provides to IT may be a bit counter-intuitive. I've done a small amount of examination of four benefit areas to collaboration tools by elimination of a required centralized server: security, reliability, interactivity, and efficiency.

The elimination of a centralized server is sometimes required for security purposes. We occasionally enter contracts with other corporations that give very explicit rules about who can have certain data on their hard drives. A centralized server would violate that policy, whereas it may still be acceptable to share the data with specific peers. Ultimately, security is best provided if information managers control the access to data, rather than IT.

There is also the occasional need to utilize modern tools that aren't yet available on our internal servers. Some of those tools are available on external servers, but it isn't secure to place [our company's] private data on those servers. A tool that does not rely on a centralized server could be more easily deployed. This doesn't eliminate the need for security audits on the tool, but it does eliminate the overhead of evaluating the impact of that tool on other tools running on a common server.

Despite on-going improvements to server reliability, there doesn't ever seem to be an end to the reasons why a server must occasionally be shutdown or relocated. With an architecture that doesn't rely on a single centralized server, the reliability is certain to be increased. An application network that requires a high degree of reliability should still make use of high-reliability servers, but it makes sense to architect those networks to not rely on them exclusively when possible.

Efficiency benefits from eliminating requirements on a central server come from the ability to rapidly deploy a new peer-to-peer network. A business could deploy a new network right away to gain the benefits of collaboration without waiting for a high-reliability server to come on-line. Such a server could then later be deployed to increase the reliability of the network without creating downtime. Further, an information manager within the business can be directly responsible for adding and removing users, machines, and services on the network.

Efficiency benefits also come from more general aspects of peer-to-peer architecture. By allowing users to organize their own data, they can access it more efficiently. In most cases, there isn't one ideal solution to data organization. A peer-to-peer architecture allows everyone to organize their own data and allow others to benefit from that organization.

By interactivity, I mean both the possibility of working with other tools and the way the tool works for its users. Interoperability and connectivity to other tools doesn't necessarily require the elimination of a central server, but tools that don't attempt to create a centralized repository for information are more likely to support the standards required for interoperability. By creating protocols for use in a peer-to-peer network, some of the requirements for interacting with other tools are necessarily met.

One particular interactivity benefit of eliminating requirements on a central server is the possibility of providing "on-the-go" or disconnected collaboration. You experience this today with Outlook when you are on a plane. You can read your e-mail that is cached locally, create your responses off-line, and synchronize your mail once you are connected to the network again.

E-mail must be the most common on-line tool for collaboration. It is simple, universal, "on-the-go", search-able, you know who is on both ends, you get notification, and for the most part it just works.

Being such an effective collaboration tool, e-mail has many peer-to-peer architecture characteristics. Foremost, it is distributed and resilient. It relies on DNS, the domain name service that is used to look-up the address of servers on the Internet. DNS is distributed and resilient, allowing for server failures at many points in the system. Further, SMTP, the protocol used to forward e-mail messages, can be run on just about any computer. You could almost decide to run SMTP on your own desktop machine, but you'd quickly find that getting the DNS record to point to your machine is a bit of a hassle when you don't leave your machine running all of the time. This doesn't matter that much, though, since [our company] provides you with a high-reliability server to collect your e-mail and clients that cache the e-mail for off-line use.

So, what is wrong with e-mail as a collaboration tool? First of all, the information is trapped into little personal "silos" that no one else on your team can search or access. Certainly you don't want to share all of the information provided to you by e-mail, but some of it you do. Some of it you just want other people to know you have, but not necessarily give it to them without your approval. You can forward e-mail to individuals or mailing lists. You can archive e-mail sent to mailing lists on a website. You can even create shared mailboxes, though those are a bit complex for many people to handle. In the end, you are left with a large number of small silos of data that can't be organized as part of a larger body of information.

E-mail is not secure. Sure, there are tools for encrypting e-mail, but they are practically only ever used on the most sensitive data. E-mail encryption tools are simply too difficult to use and you cannot yet create encrypted messages for the vast majority of your e-mail address book and expect the recipients to be able to perform the decryption.

Monitoring e-mail for sensitive data is virtually impossible. E-mail is sent from all levels of the organization all across the world, without any approval of information managers. When inappropriate e-mail is detected, there is no realistic mechanism for confirming retraction of that data from recipients. Outlook has a "recall" feature, but it often fails and cannot be confirmed outside of our organization. E-mail must be the most important and most dangerous of all the collaboration tools available today.

Perhaps the worst aspect of e-mail is the lack of efficiency. Information coming in e-mails cannot be easily categorized. Creation of e-mail filters often makes the situation worse by creating yet more silos of data. Fields that would allow for some categorization, such as priority, action required flags, and deadlines, are typically never used and are often misused. Misuse is sometimes the result of spam, which is a problem that cannot easily be avoided. By one study, 94% of e-mail last month was spam.

What about TWiki or SharePoint? These tools are often called "content management systems". They can be quite effective in improving communication within a global team, but they have their own issues. Perhaps the best summary comes from a success story taken from the TWiki website from a company that deployed TWiki to improve support to field engineers:

People in the field were used to using email for communicating with the factory. Email is a one to one communication, a mailing list a one to many. The problem with email is that useful information does not reach everybody, email is not easy to search and email gets lost over time. Collaborating the Wiki way solves these problems, however changing habits is a difficult issue that needed to be coached.

Initially we also had a chicken-and-egg problem, i.e. voices like "why should I use this collaboration tool, the content is so limited". The solution was to assign a support engineer who monitored the mailing lists and entered useful information into TWiki.

Successful deployment took over 6 month[s], [which was] longer [than] expected. But now everybody is used to browse, search, collaborate and document the Wiki way.

The result was that customer satisfaction with the field support improved. The effort was a real success.

This sort of dedicated information management may not be something we can easily commit in our environment. If deployment took 6 months, how are we supposed to keep up with frequent release cycles? How do we convince managers to commit resources to TWiki when current searches for me today often return my own weekly reports, rather than something of valuable interest? With so little attention, even the top-level structure of the TWiki sites today can't even keep up with our organizational structure. I believe in TWiki, but we can only overcome this "chicken-and-egg" problem in each team by strongly evangelizing its use during the painful learning stages.

SharePoint has similar problems, but is a bit of a different beast. SharePoint is particularly well-suited for collaborating on Microsoft Office documents. It uses a standard protocol called WebDAV that allows for folder views in Windows Explorer. Most importantly, [our company] supports a mechanism for accessing SharePoint sites to customers and partners from outside the firewall.

The biggest problem with SharePoint, beyond the problems it shares with TWiki, is its complexity. The user permissions tables are extremely convoluted. Editing the content of any one page requires extensive knowledge of the overall system, rather than simply clicking an "edit" button and changing some text. I’m not saying that TWiki markup is trivial, but it doesn’t require learning specialized tools or extensive training. The help system is built right into the tool. Also, the SharePoint version control system is somewhat less than reliable because it allows overwriting and elimination of old document revisions.

We are still learning about how best to use TWiki and SharePoint on our projects and the best standard practices are not obvious with either tool. Neither provides great search solutions for the data you need on their own, especially if the data is in mixed formats. Instead of providing a complete knowledge picture, the combined usage of e-mail, TWiki, and SharePoint creates islands of information that must each be explored separately.

[Description of an internal collaboration tool]

Efforts like this should continue, but it is best to break up the platform into interoperable components. Consider interaction with a tool such as Microsoft Live Clipboard. Live Clipboard is a mechanism for users to initiate sharing of data between websites without requiring development of web services scripts or other complicated programming. By supporting such a feature in all of our collaboration tools, the islands of information can be bridged.

The same guy at Microsoft who dreamed up Live Clipboard, Ray Ozzie, has also brought us one of the more compelling peer-to-peer collaboration tools already available, Microsoft Office Groove 2007. Recently we made use of Groove on one of our projects. The project spanned two partner companies and two contractors. Some folks on the team were able to start collaborating on the very first day by simply installing the tool. It took a few more days for others to overcome some minor installation headaches that were likely related to the tool being in beta. The product will be released with the 2007 version of Microsoft Office.

Groove works across firewalls, provides account management, secures communications, provides synchronization for off-line usage, includes instant messaging with some voice capability, and has some limited integration with Office applications and SharePoint. When I talk about the integration being limited, however, it needs some emphasis. All of your important e-mail, calendar, and contact information isn't easily shared in Groove with something as simple as a single click on a category. To get that information into Groove, a user must jump through many hoops. A major concern for the team was lack of support for maintaining old versions of documents. Groove did a great job of ensuring everyone had a copy of the latest version of a document, but SharePoint was required to maintain historical copies.

Groove also lacks a client for any platform besides Windows and the client can be a bit slow because it consumes a large amount of memory at times. There is no way to see the data in Groove by simply logging into a web page. You can synchronize a SharePoint with Groove, but it is a tool separate from the other tools in Groove which all seem to act as more islands unto themselves.

Ultimately, usage of Groove suffers for many of the same reasons as the web-based content management tools. Some folks wouldn't use it regularly, instead using familiar tools such as e-mail. It never became part of the team's everyday work flow, partially because other tools were required to author rich documents and manage code. The client tool was seen as painful to start-up or leave running. Ultimately, nothing was pushing users to actively communicate using Groove.

Certainly there are some dangers with this going unchecked, primarily related to it being difficult for IT to log file exchanges. Exchanges over SSL secured websites or, to a lesser degree, encrypted e-mails offer similar challenges. Ultimately, there are always ways for employees to circumvent security and, in some cases, the risk of not progressing business is worse than the risk of compromising security.

What might a peer-to-peer collaboration tool look like?
Groove offers a good starting point for describing the peer-to-peer collaboration tools of the future, but it is not alone in its class. What I'd like now is to describe for you some of the building blocks for creating a tool like Groove and some of the building blocks that could be used to make a better tool. I won't draw you a complete picture of the ideal peer-to-peer collaboration tool, but I hope to point you in that direction.

I'll finish typing this up when I get back from vacation next week. I need to scrub and upload the pictures...

2007-02-07

Ending walled gardens with microformatted microcontent

Forgive me for stating the obvious today, but I'm surprised how little I seem to see this point repeated.

I continue to be surprised by the continual emergence of Web 2.0 companies putting up walled gardens to try to hold onto users. I think MySpace is the current prototypical example. They could use a lesson from AOL on how likely they are to be successful over the long term. When AOL offered ease-of-use, content, and community features that simply weren't available on the Internet at the time, AOL grew at enormous rates. Once the Internet was well established outside of AOL's walls, AOL quickly tumbled.

You can expect the same results with MySpace. From outside MySpace today, it is possible to link to MySpace, be linked by MySpace pages, and consume MySpace blog entries using an RSS feed. Interaction to the broader Internet is roughly limited to just those features. Once there is broader access to video/photo/blog/music hosting, friend linking, trusted commenting, shared calendars, templates/widgets, and all of the other MySpace features, people will start wondering why they put up with all of the advertising and spam on MySpace. MySpace is "free", but there are many degrees of freedom. Person A might even want to control how Person B's sites appear when Person A reads them. This freedom is established with microformatted microcontent over RSS/Atom feeds and aggregated feed readers.

Microcontent is simply each of the entries you make on your MySpace pages (or other people's pages) today, but placed into small, consistent containers. Your preferences could be stored in a microcontent article. Certainly each of your blog entries, friend relationships, and everything else you communicate on MySpace could also be placed into microcontent articles once the right microformats are defined. Those microformats provide the protocol for interoperability in whatever RSS reader you use.

Of course, the RSS readers need to be smarter than they are today. Bloglines and Google Reader are fantastic compared to what we had before they existed. They are, however, still significantly limited, providing almost no microformat or customization support at all. This will change quickly, however, and the garden walls of social networking will fall before people start trying to build walls around something totally new.

So, Web 2.0 companies, I guess you can't help but build your pretty little walled gardens while you can. If I was a VC, I wouldn't give you a dime unless you had an exit plan.

2007-01-19

Is the WebOS revolution over?

Muli Koppel wrote a nice piece on why the web2.0 revolution has failed. By the same measure, I believe we'd need to conclude that the WebOS revolution, which hasn't even taken off yet, has also failed. Before a real WebOS could even be created to provide the people with a viable web development environment, the term has acted as a gravity point for commercially-minded folks trying to capture hold of people's valuable data. (Correction: Thanks Benjamin) At least the Exo Platform gives you the server code , but the release I've played with doesn't give all of the source and is available under GPL license. EyeOS is also open source and includes a "mini-server" that provides LAMP-like support on your PC, though a database is not required.

Muli's examples of "failed" revolutions, punk, open source, and Second Life, do give room for hope. Each has failed to reach their ideals, but each has also made improvements to our culture along the paths of those ideals.

Perhaps the real revolution is still in the works. Maybe the only way to succeed is to avoid naming the revolution completely until it is already won. Besides, no one has offered me any useful names for the product that will ultimately emerge.

Update (Jan 20): Keep the feedback coming. Let me know why you think the WebOS revolution is on!

2007-01-12

Poisoning the WebOS well

The term 'WebOS', or web operating system, has been useful to gain attention to the idea of creating a framework for rapid web application development and making those applications available from any computer with a web browser and Internet access. This is a noble and important goal. Making web applications can otherwise be quite difficult. We should take advantage of the web to simplify creation, distribution, and interaction of our applications.

The windowing glitz of the current so-called WebOS offerings has bloggers questioning the point. Windows within windows just seem silly. They solve a problem of maintaining the state of an entire collection of applications in an environment, but they they recreate a paradigm that users where there is already a significant amount of frustration: the desktop. I'm not ruling out the possibility of using this paradigm, but it distracts from the more fundamental problems of rapid application development and distributed access.

It would be great if this web-based desktop-like, or webtop, functionality could be isolated from the rest of the WebOS discussion. As long as these webtop offerings are being called WebOSes, as long as they are being reviewed only on their webtop merits, the WebOS well will continue to be poisoned. There really isn't much that can be done in the short term. Eventually, some new terminology will emerge to differentiate web application frameworks that provide glitzy windowing and ones that allow the rich array of web services to be used easily and transparently by even the most novice of developers. Sometimes they might be the same framework.

Read more on this blog about what a WebOS could be, then tell me what I should be calling this sort of web application development framework.

2007-01-09

Droolling over the Apple iPhone

The inclusion of OS X, full-blown Safari, and 802.11 support has got me. Finally, *nix in a widely available US phone. I wish it had EVDO in addition to EDGE, so it will be slow compared to my Treo 700p. Quad-band GSM--nice, should be a good international phone.

Anyone know if it will support A2DP so I can use my stereo Bluetooth headphones?

Update: It looks like it does according to Engadget.

2007-01-04

New microformat, not viagra

Posting comments on blogs is still a pain. I just tried to post the following comment to a post on APIs being the next site map:

I agree with you, APIs are the new site map. [comment about site rejecting previous post]

There is still extensive work to be done to eliminate the silos. First, everyone has to have their own web server. The technology isn't really there for that today, unless you count projects like Paper Airplane (which isn't a complete product). Next, service providers need to provide a common set of services that will allow everyone the freedom to move from their limited function server onto the full-scale Internet.

I call that common set of services a WebOS API (http://clamoring.blogspot.com/2006/12/defining-webos-api.html). A new name is probably needed, since folks seem to have stolen that name for webtops.
Do you think a microformat that describes that site map and API would be useful? It would need to replace something machine oriented that is already useful today, such as WSDL.
The following response was given:

Comment Posting Guidelines

Sorry but it looks as though your comment has been filtered out to avoid blog spam. You weren't trying to sell me viagra were you?
Does that look like a viagra advertisement to you? Oh well, I understand the need for comment filters. It would have been nice if it at least indicated that it was going into a moderation queue.

Selling out

I've added Google AdSense to my site. This site is largely an experiment for me to best understand the technology of blogging, so I'm really not in it for the money. If the ads bother you, just let me know and I'll remove them.

The Snap previews are there to help you get an idea what is being linked, but they can be annoying at times as well. Please give me your feedback on those too.

Also, note that there are now 4 feeds on this site. The original two from Blogger and two from FeedBurner. The main Feedburner link adds my del.icio.us and Flickr posts. The other Feedburner link is just to my Google Reader shared items.

2007-01-03

Step #1 to Creating a WebOS: Proxy Server

Reading blogs this morning, I ran across Brad Neuberg's announcement that he is working for SitePen to create an off-line Dojo toolkit. I have to get ready for my day job now, but I'm excited enough that had to write something first. This is really step #1 to creating a WebOS.

I've mentioned in the past how a client-side server is required to create a proper WebOS. That seems to be exactly the approach Brad is taking.

An off-the-shelf, open source (GPL), C-based web-proxy will be used, named Polipo, saving months of development time creating a custom HTTP/1.1 proxy.
Deploying with a C-based solution seems like the wrong idea to me. I've toyed with something similar using Java and JavaScript to create a thin web server/proxy. I haven't been able to dedicate enough time to get it to the point I can demonstrate it. Keeping Dojo almost entirely in JavaScript, and using Java server implementations like Tomcat or Jetty, would seem more portable and supportable.

I also commented to Brad that Scrybe seems to do the off-line work without a proxy, though they may just be leaving that out of the demo information.

The great news, however, is that Dojo is a really popular open source toolkit and should get a lot of traction, even if it isn't on as many platforms as possible from the start. Brad also seems to be a really smart guy.

2007-01-02

Increasing traffic randomly

I was reading Mike-O-Matic and ran across his Click Pyramid site.

This is how it works:

  1. You visiting each link below in turn and collect the codes that are presented.
  2. These codes are entered into the form below, along with the website information you would like added to our database.
  3. You are then given a link to a new list, which will have your site first on the list and each one moved down one slot. The bottom site will drop off.
  4. You can then place that link in web advertisements or on your website/blog. Any users who click on it will go through a similar process to add their link, and will spread your link even further.
I figured it couldn't do any harm, so I created my own ID, #208. Only 208! Oh well, I guess I'm spreading good karma for someone who created a couple of blog entries that I found interesting.

Of course, if I advertise that link, wouldn't I just be better off advertising my own site instead of adding it to a list with 4 other sites? I guess that I could get a few people to check out my site simply because they want to advertise theirs, but doesn't allowing them to leave a comment on my blog accomplish roughly the same thing?

The thought is still somewhat interesting, but someone will have to turn it around somehow to make it work.

2007-01-01

WebOS killer app

The VC guys are saying a WebOS or Browser OS is on the way, but web developers are asking what will be the killer app to bring about a standard? I think we've already seen "alphas" of the killer app in MySpace, Facebook, YouTube, etc.. Privacy, interoperability, ease-of-use, and desktop-integration will drive standardization of those services, starting in the open source community.