"Dreaming in Code" by Scott Rosenberg

Scott Rosenberg's "Dreaming in Code", despite its rather evocative title, is the mundane story of an ongoing software project called "Chandler", a tool for Personal Information Management (PIM). Wait, that's not right. It is a vividly written but altogether-familiar story of software development. A story of requirements ("specs") and disagreements, of delays and deadlines, of plans, of changes in plans and of more changes in plans, of ideals and of pragmatism, and of course, of bugs that make you tear your hair out in agony. The sort of things that are all too familiar, when it comes to coding.

Why then, was it written? Over to Rosenberg :

Chandler offered a look at the technical, cultural and psychological dimension of making software, liberated from the exigencies of the business world. [pg 54]

But I am getting ahead of my tale.

In the post-war era, a project that took on a special meaning with the development of computing machines: the augmentation of human intellect. Here, many people thought, was a tool to rival language, and writing, a tool perhaps to re-invent man himself. The vision has been expressed eloquently: most notably by Vannevar Bush ("As We May Think") and Douglas Engelbart ("Augmenting Human Intellect"). Chandler, an open-ended open-source project, follows in that same tradition.

Mitchell Kapor, the owner of OSAF, and the hero of our story, was the designer behind Lotus 1-2-3, the first widely successful spreadsheet used in the business world. Lotus 1-2-3 made Kapor a millionaire several times over. His next product called "Agenda", was supposed to take personal information management to the next level and from the reviews, it did! -- but it never really took off and Lotus dropped it like a hot potato (see this review by James Fallows, that praises Agenda but also gives a good idea of its difficulties). But Agenda remained on Kapor's mind and in Chandler, he tried to go back to the same feeling that inspired him to build it.

What was this spirit? PIMs like Microsoft Outlook separate their content into silos: there's email, there are tasks, there are lists, there are action items, and so on. But of course there are no such neat categories in human activity where everything is also something else. My email is also a task (not the least because I actually have to type a reply to it), a task or a project involves emails, emails are parts of projects, emails are a way to store and access files, documents. In other words, the decomposition of the artifacts of human activity, while convenient, is also just that: an analytic convenience. Agenda aimed to go beyond silos -- as does Chandler. A personal tool that would not be caught into silos, something that could truly capture the way humans actually worked, and thereby help them do their tasks better.

Of course, all this is easier said than done. Programming itself is all about silos. Formalization, which is after all what programs are all about, require us to logically decompose processes and methods into categories. The better the categories are defined, the better a program will work. Overlapping categories and amorphous boundaries, while not unimplementable, are almost guaranteed to break down in some scenario or the other. Still, Chandler was almost a romantic project, a project that followed in the tradition of Bush and Engelbart so it had an open-ended vision, something most software projects can't really afford.

That Chandler could do so was because it was administered by Kapor's OSAF, the Open Source Applications Foundation. This was to be a non-profit project, a project solely instigated by what is called "the programmer's itch", an urge to get something done that starts off with a personal wish or frustration ("I wish I had a software that did this" or "Damn, why can't this calendar software do that??" etc). The non-profit part (which of course depended heavily on Mitch Kapor's deep pockets) meant that the Chandler team was liberated from constraints that come when you program for profit.

But there was a third component that made Chandler special. Chandler was to be open-source and therefore it was meant to harness the forces of peer production. This meant that a core Chandler team would work on the code, while at the same time, at least in theory, relying on a vast array of programmers all over the world. In short it would take the best of the two modes of production, made famous by Eric Raymond: "the Cathedral and the Bazaar". In his essay, Raymond postulates that there are two modes of production. One of them, embodied by the cathedral, is a top-down, command-and-control approach, where a plan is built up and then systematically carried out. The other, embodied by the bazaar, is bottom-up: a group of people find each other and self-organize, without any command-and-control structure. The development of Linux followed the bazaar model -- and the main reason for its success that the internet offered a wide range of tools for a lot of people, all over the world, to collaborate. Raymond's conclusion is what he called Linus' Law: "Given enough eyeballs, all bugs are shallow".

So, did Chandler's combination of the cathedral and the bazaar work? Its still too early to tell, Rosenberg's book ends with the release of Chandler 0.5 (the most recent version is 0.7 -- the fully functional Chandler release is 1.0, still a long way off). But the book does illustrate the problems of having an open-ended software project. Because the Chandler team could not agree on a set of requirements, they could not get a batch of workable code out immediately -- the release of workable code being a crucial component in harnessing the forces of peer production. But Chandler never quite managed to get the participation that a peer-produced project generally needs and slowly its team slogged on, making compromises, slowly and steadily.

I could go on and on but I'll stop now. Try Chandler out, I liked it, even if I don't quite visualize using it just yet. What did I take away from Dreaming in Code? Perhaps, if nothing else, this quote by Linus Torvalds, his advice for people starting large open-source projects, burned into my brain (pg 174):

"Nobody should start to undertake a large project, " Torvalds snapped. "You start with a small trivial project, and you should never expect it to get large. If you do, you'll just overdesign and generally think it is more important than it likely is at that stage. Or, worse, you might be scared away by the sheer size of the work you envision. So start small and think about the details. Don't think about some big picture and fancy design. If it doesn't solve some fairly immediate need, it's almost certainly overdesigned."

Words from the wise, indeed.