This is one of my Web Based Projects. It should be considered a work in progress.
Stew is a collaboration system based on a E-mail, revision management, and Wiki. By blending the best features of each of these systems together, a coherent collaboration system results. Fundamental to the workings of Stew is being able to author material in via either plain text E-mail or web pages. A very simple markup language called PTML (Plain Text Markup Language) is used to accomplish this task. Lastly, by appointing a editor (called a chef) to oversee a collaboration area, it is hoped to improve the overall organization of the resulting collaboration information.
Stew is a collaboration system. It is meant to help people who are separated in both space (i.e. different continents) and time (i.e. different time zones.) It is an evolutionary system that is built on top of E-mail, revision control, and Wiki, rather than a revolutionary system that intends to supplant preexisting systems. Stew accomplishes this by working within some of the severe limitations imposed by both E-mail and Web technologies.
There are three broad areas of collaboration systems that are relevant to Stew. The first is the suprisingly successful collaboration via E-mail and network news groups. The second is collaboration that occurs during software developement via revision control systems. The third is a fairly new and suprisingly simple collaboration model called Wiki. All three of these areas are discussed in the sections that immediately follow.
E-mail started out as just a simple mechanism to send a plain text message from one person to another. Over time, its capabilities have evolved into a fairly simple and but widely used collaboration system. The evolved features that are worthy of discussion are:
List servers are programs that more or less automatically manage lists of E-mail addresses The two dominate list servers are majordomo [ MajorDomo ] and LISTSERV [ LISTSERV ]. When E-mail started out, it was point to point delivery of plain text messages. Very early in its inception, it supported multi-cast, the ablity to send the same message to more than one recipiant. Quickly people became very tired of typing in all those E-mail address and set up a level of indirection whereby somebody could just type in the list once and use it over and over again. Eventually, the amount of network traffic due to multi-cast mail messages became so great, the network news groups were created along with a specialized protocol NNTP [ NTTP ] to service them. Over time the hassle of dealing with additions and deletions to the mail lists became automated with list servers like majordomo and LISTSERV.
The key feature about list servers and network news groups is that they allow groups of people from all over the world to exchange information on topics of common interest. The topics can be quite broad (e.g. metal working) or extremely narrow (e.g. signaling for model railroads.)
One of the important reasons why E-mail is working so well for collaboration is because of the widely adopted strategy of quoting somebody else's work via indentation prior to commenting on it. The most common form of indentation is to preceed each line of the quoted material with "> ". This quoting stategy is so common that many E-mail client directly support the quotation of previous E-mail via indentation. Immediately below is a short example of the practice:
> According to the environmentalits CO2
> is the most powerful greenhouse gas and
> it must be dealt with immediately.
Actually, both methane (CH4) and water (H2O)
are more powerful greenhouse gases. H2O is
far more prevasive in the atmosphere
than CO2. And while CO2 is more common
than MH4, MH4 is a more powerful gas
and its effects can not be neglected.
Indeed, all of us have probably seen E-mails with
so much indented quotation, that it is hard to
figure out who is saying what.
While quoted indentation may not be as technically elegant as using hypertext links, it has the characteristic that it works, people understand it, and the various mail clients have variying levels of support for it.
One of the major problems with collaborative E-mail exchanges is that they tend to be temporary (i.e. ephemeral) in nature. After a while, the same question will come up again, and a similar (but not identical) discussion will occur. There are two strategies that have come up to deal with the ephemeral nature of E-mail -- E-mail archives and FAQ's.
An E-mail archive tool takes an archive of E-mail messages and compiles it into a set of web accessable pages. This allows people who have the time and inclination to read past exchanges on a given topic. The dominant E-mail archive tool is currently HyperMail [ Hypermail ]. Unfortunately, it takes a fairly dedicated person to read through all of the E-mail in an E-mail archive.
FAQ stands for Frequently Asked Questions and more properly should be called something like Recurring Issues. However, FAQ is close enough and we will continue to use the term. Basically, a FAQ tries to organize the common issues into a more or less coherent document that people who are new to a mail list can read to get themselves up to speed. Not all discussion groups have a FAQ because it takes some significant amount of effort on the part of someone to organize the data. Since the FAQ's are not universally available, many people who newly join a list simply ask the same old questions and start the same old discussions.
Before I leave the topic of E-mail, it should be noted that E-mail has fairly strongly resisted a transition from plain text to a richer format such as HTML [ HTML ]. WHile Multi-media In e-Mail Extensions (MIME) [ MIME ] do allow E-mail to be authored in both plain and rich text formats, the follow through issues were not handled very well. Conceptually, user A should be able to author a rich text E-mail, send it to a mail list, and each person on the mail list would see the message in the format of their choice -- either plain text or rich text. Instead, the people with plain text E-mail clients usually get to see both versions and they complain vociferously when it happens. In addition, the people who have 56Kbps modems complain that rich text E-mails take over two times longer to down load. The net result is most E-mail is still taking place in plain text.
One aspect of MIME that has been more successful has been E-mail attachments. It is now more common for people to attach small pictures and the like to an E-mail message. The people at the end of 56Kps modems still complain, but the situation still persists.
As E-mail is currently used, it is still a fairly effective collaboration system that suffers from a lack of longer term memory.
Programmers have been using revision management systems to share code for a rather long time. The first revision management system I am aware of is SCCS [ SCCS ]. In the open source community the dominate revision management system is a combination of RCS [ RCS ] and CVS [ CVS ]. There are numerous commercial revision management systems out there with Perforce [ Perforce ] being my current commercial favorite. Interestingly enough, Perforce is available to Open Source projects for free; however, I am unaware of any projects that taken up that offer.
While revision management systems are primarily used for computer source code managment, they are also used for document management as well (e.g. manual pages, tutorials, reference manuals, etc.) While I have seen some instances of people successfully doing collaboration using a revision management system, the collaboration tends to be more of a editorial flavor than a idea generation and brain storming flavor.
The older revision management systems worked by allowing only one person at a time to edit an individual document. This strategy worked, but sometimes caused bottlenecks as more than one waited to add their edits to a document. The more modern revision management systems like CVS and Perforce use a copy, modify, merge strategy for document editing. Each person gets to edit their own copy of the document, which eventually merged back into main document.
What makes a revision management systems interesting is that conceptually, there is only one copy of a document. All of the changes eventually get merged back into the original. If there are conflicts, the revision management system helps to identify them and resolve the differences. This is in contrast to an E-mail collaboration system where everything is done by simple copying, thereby eliminating the need to detect conflicts.
A more recent tool in the collaboration arena is Wiki [ Wiki ]. In Wiki, the key concept is to lower the barriers to editing. Anybody in the community can edit any page and they are encouraged to do so. If two people edit the same page at the same time, the last person to check their page in "wins". While the utter lack of access control is completely contrary to the mind set of revision control systems, Wiki uses social conventions to accomplish more or less the same thing.
The original Wiki has been used to develop extensive ideas on the idea of programming patterns. As one person came up with an idea, other interested people would chim in and their comments to the page. What is neat about Wiki is that it tends to avoid the ephemeral problems associated with E-mail. The people who are working on a page tend to care about keeping it up-to-date and coherent.
Another key component to the Wiki system is that all of the pages are edited using a plain text editor. This allows people to use HTML web forms with a TextArea <Input> element. Thus, there are some simple rules for editing text and entering hypertext links and not much else.
{Comparison goes here}
{There is now a separate PTML Manual.}
The basic concept behind Stew is to try and take the best features of E-mail, revision control, and Wiki and blend them together into a coherent collaboration system. The name Stew came about because a real stew is made out of a bunch of different ingrediants which when combined together tend to be tastier than each of the ingredients separately. That is the hope for Stew, by using the best features of E-mail, revision control and Wiki, the result will better than each of the systems by themselves. Lastly, the final ingredient to Stew (i.e. the secret sauce) is some social engineering to cause discussed issues to get summarized in a timely fashion (more on that later.)
The initial version of stew is a conglomeration of preexisting tools -- an Apache web server, a majordomo E-mail list server, a hypernews E-mail archive processor, and stew specific programs that run every 15 minutes or so from a cron tab entry. Over time, it is anticipated that some of these features will become more and more integrated (in particular the E-mail archive processor.) For the purpose of this overview, I will treat them as an integrated whole rather than the collection of tools that they will initially start out as.
The top level page of a stew server is a web page is a list of collaboration projects. These projects are partitioned into those that active vs. those that have been deactivated. Of course, when a Stew server is first started up both the active and inactive lists will be empty.
Stew starts out as a web page being served up by the Web server of your choice, for example Apache [ Apache ]. This top level page lists the various ongoing Stew projects. I'm sorry, but I can not resist abusing the stew metaphor and have decided to call these project pages stew pots. Each stew pot page lists the colloboration topic name, gives directions for signing up for the project E-mail list server, provides an entry point for the E-mail list mail archives, and provides an entry point for the project pages area (i.e. wiki area.) A short summary of the project can be found on the stew pot page.
The person who sets up the initial Stew pot is called the chef (sorry, again, I can't resist the tempatation to abuse the metaphor.) Besides some of the more mundane adminstration tasks, the chef has the important task of trying to keep the collaboration effort on task and moving forward. There will be more on this important task later on.
When the chef is setting up the stew pot, he or she gets to set some overall stew pot policies. The first policy decision is whether the stew pot is open or closed. An open stew pot allows anybody that wants to join the stew pot mailing list by simply send a subscribe request to E-mail list server. This is the standard majordomo policy. A closed stew pot requires that chef approve each subscription request. An open stew pot entails less adminstrative hassle over the long haul.
After someone subscribes to a stew pot, a member page is created and a short message is sent to the subscriber asking them to put some relevant information on their member page. There is a link from the stew pot page to the list of members. When somebody decides to desubscribe, their member page is not deleted, it is merely shuffled onto a list of prior members. A member page can only be edited by the person who created the page in the first place. By the way, the chef is just like any other member and has to subscribe and set up their member page as well.
The goal of having member pages is to start the process of building a community. While the page may just contain a hypertext link off to the person's real home page on another system; many people do not have such a page and will use the Stew member page as their `home' for the collaboration effort.
The next configuration task is to set up the E-mail archive software. Eventually, Stew needs to have its own customized software. Initially, the intention is to not reinvent the wheel and use preexisting one such as HyperMail.
All Stew collaboration starts via the E-mail discussion list for the Stew Pot. As each message comes in, it is assigned a unique message identification number starting from 1. The E-mail digest tools, convert the E-mail to a single HTML page with a fixed URL that does not change -- http://host/.../pot_name/messages/number. In addition, the E-mail digest tools generate index lists of the E-mail sorted by Date, Thread, and Author. These summary pages are maintained for each day, week, month, quarter and year.
When someone signs up for the discussion list, they can choose between immediate delivery or digest format. For digest format, they can choose to get a digest once a day, once a week, or once a month. The digest can be sent in either plain text, or HTML. The messages can either be directly included in the digest, or they can simply contain a list of hypertext links to the appropriate message or index page.
If a Stew pot member prefers to use their favorate E-mail tool for collaboration, they can simply hit the reply button and type in their comments. The automatic indentation supported by most E-mail clients is automatically decoded by the Stew E-mail digest software.
If a Stew pot member prefers to use web pages for collaboration, they can just use a web interface to reply to a message.
As the discussion preceeds, the E-mail archive is starting to collect useful amounts of material. At some point in time, the Stew pot chef sends out a message that says something like, `We have talked about this issue for quite a while, who would like to assemble a summary document?' If there are multiple volunteers, the chef picks one volunteer. If their are no volunteers, the chef either has to cajole somebody into volunteering, assign somebody the task, or undertakes the task by himself (or herself.)
Once the initial summary document is generated, it is posted by the Stew server and an E-mail message is generated announcing its availablity. Either the entire summary or a simple link to the document can be can be E-mailed out.
The summary discussion can take place either via the standard E-mail, or people can directly edit the summary document. Their comments are edited in as indented block comments.
In many situations, the summary document will eventually reach a form that all stew pot members are agreeable with.
It is the Stew pot chef's job to keep the summary pages organized so that people can easily find them. Thus, as new members jump on board, they can easily read the shorter summary documents to come up to speed. If they disagree with a given summary document, they can read the original discussion in the E-mail archives. If they still disagree with the summary, they can post their disagreement to the list and reopen discussion.
{More goes here.}
{Summary goes here}
First and foremost, I would like to thank my wife and son for having patience with me as I type away in the office on documents like this. They don't always understand what it is I am doing, but they do understand that it is important to me. For that I am most thankful.
A great deal of the insight for the Stew design came from some hard learned lessons from the people who worked on the CritSuite [ CritSuite ] project. At the beginning of that project we knew that the key to successful collaboration was a good implementation of back link technology; by the end of the project we had learned that E-mail was actually the more powerful strategy. Of particular note, are Ka Ping Yee, Mark Miller and Christine Peterson; there are others who are being slighted and for that I offer my apologies in advance.
For a brief period of time, there was a proposal by Eric Armstrong to build a fairly comprehensive collaboration system. While that work never seemed to go anywhere, I did gain a bunch of insight from it.
Lastly, I am also greatful that I am able to stand on the shoulders of some true giants in the field -- Ted Nelson, Doug Englebart, and Tim-Berners Lee. I just wish that so many other people who clambered on top of their shoulders had been wearing soft soled shoes rather than cleats for extra Internet traction action.