Tag Archives: reinventing the wheel

Wheel Reinvention and Geek Culture

In mainstream North American society, “reinventing the wheel” (investing efforts on something which has already been done) is often seen as a net negative.  “Don’t waste your time.” “It’s all been done.” “No good can come out of it.”

In geek culture, the mainstream stigma on wheel reinvention has an influence. But many people do spend time revisiting problems which have already been solved. In this sense, geek culture is close to scientific culture. Not everything you do is completely new. You need to attempt things several times to make sure there isn’t something you missed. Like scientists, geeks (especially engineering-type ones) need to redo what others have done before them so they can “evolve.” Geeks are typically more impatient in their quest for “progress” than most scientists working in basic research, but the connection is there.

Reasons for wheel reinvention abound. The need to practice before you can perform. The burden of supporting a deprecated approach. The restrictions placed on so-called “intellectual property.” The lack of inspiration by some people. The (in)famous NIH (“Not Invented Here”) principle.  The fact that, as Larry Wall say, “there is always another way.”

Was thinking about this because of a web forum in which I participate. Although numerous web forum platforms exist as part of “Content Management Systems,” several of them free of charge, this web developer created his own content management system, including forum support.

Overall, it looks like any other web forum.  Pretty much the same features. The format tags are somewhat non-standard, the “look-and-feel” is specific, but users probably see it as the exact same as any other forum they visit. In fact, I doubt that most users think about the forum implementation on a regular basis.

This particular forum was created at a time when free-of-charge Content Management Systems were relatively rare.  The site itself was apparently not meant to become very big. The web developer probably put together the forum platform (platforum?) as an afterthought since he mostly wanted to bring people to his main site.

Thing is, though, the forums on that particular site seem to be the most active part of the site. In the past, the developer has even referred to this situation as a problem. He would rather have his traffic go to the main pages on the site than to the forums. Several “bridges” exist between the forums and the main site but the two seem rather independent of one another. Maybe the traffic issue has been solved in the meantime but the forums remain quite active.

My perception is that the reasons for the forums’ success include some “social” dimensions (the forum readership) and technical dimensions (the “reinvented” forum platform). None of these factors could explain the forums’ success but, taken together, they make it easy to understand why the forums are so well-attended.

In social terms, these forums reach something of a niche market which happens to be expanding. The niche itself is rather geeky in the passion for a product category as well as in the troubleshooting approach to life. Forum readers and participants are often looking for answers to specific questions. The signal to noise ratio in most of the site’s forums seems, on average, particularly high. Most moderation happens seamlessly, through the community. While not completely invisible, the site’s staff is rarely seen in most forum threads. Different forums, addressing different categories of issues, attract different groups of people even though some issues cross over from one forum to another. The forum users’ aggregate knowledge on the site’s main topic is so impressive as to make the site look like the “one-stop shop” for any issue related to the site’s topic. At the same time, some approaches to the topic are typically favored by the site’s members and alternative sites have sprung up in part to counterbalance a perceived bias on that specific site. A sense of community has been built among some members of several of the forums and the whole forum section of the site feels like a very congenial place.

None of this seems very surprising for any successful web forum. All of the social dynamics on the site (including the non-forum sections) reinforce the idea that a site’s succes “is all about the people.”

But there’s a very simple feature of the site’s forum platform which seems rather significant: thread following through email. Not unique to this site and not that expertly implemented, IMHO. But very efficient, in context.

At the end of every post is a checkbox for email notification. It’s off by default so the email notification is “opt-in,” as people tend to call this. There isn’t an option to “watch” a thread without posting in it (that is, only people who write messages in that specific thread can be notified directly when a new message appears). When a new message appears in a specific thread, everyone who has checked the mail notification checkbox for a message in that thread receives a message at the email address they registered with the site. That email notification includes some information about the new forum post (author’s username, post title, thread title, thread URL, post URL) but not the message’s content. That site never sends any other mail to all users. Private mail is done offsite as users can register public email addresses and/or personal homepages/websites in their profiles.

There’s a number of things I don’t particularly enjoy about the way this email notification system works. The point is, though, it works pretty well. If I were to design a mail notification system, I would probably not do it the same way.  But chances are that, as a result, my forums would be less successful than that site’s forums are (from an outsider’s perspective).

Now, what does all this have to do with my original point, you ask? Simple: sometimes reinventing the wheel is the best strategy.

Homebrewing Knowledge-Base from HBD Archives?

Uh-oh!

Started thinking again. This time about a way to repurpose messages on the HomeBrew Digest into a kind of database of brewing knowledge. I can just see it. It’d be ah-some!

Anybody knows how to transform email messages from well-structured digests into database entries? Seems to me that it should be a trivial task, especially for someone well-versed in Perl and/or PHP. But what do I know?
That venerable HBD mailing-list contains a wealth of information about pretty much every single dimension of beer homebrewing. For a large number of reasons, content from the HBD.org site turns up quite often in Web searches for brewing terms.

One issue with the HBD, though, is that it’s a bit hard to search. There used to be a custom-built search feature on the site but we now need to rely on Google and AltaVista. This wouldn’t be too much of an issue if not for the fact that those engines search complete digests instead of individual messages. So the co-occurrence of two terms in the same digest can be due to two messages on completely different subjects.

Another issue with the HBD (as with many other mailing-lists) is the relatively high redundancy in message content. Some topics came cyclically on the mailing-list and though some kind souls were gracious enough to respond to the same queries over and over again, the mailing-list often looks like an outlet for FAQs. Among HBD “perennials” (or cyclical topics) are discussions of the effects of HSA (hot-side aeration), decoction mashing, and batch sparging, to name but a few technical issues.

Unfortunately, it looks like the HBD might need to be retired at some point in the not-so-distant future, at least for lack of sponsorship. Also, Pat Babcock, the digest’s “janitor,” recently asked for mirror space and announced the retrieval of some of the older digests (from the late 1980s).

Of course, there are lots of other brewing resources out there. So many, in fact, that it can be overwhelming to the newbie brewer. One impact of having so much information so easily available about homebrewing (and commercial brewing, for that matter) is a “democratization of beer knowledge.” Contrary to brewing guilds of medieval times, brew groups are open and free. Yet a side-effect of this is that there isn’t a centralized authority to prevent disinformation. Also, because the accumulated knowledge is difficult to peruse, people tend to “reinvent the wheel.”

In Internet terms, the HBD is the closest equivalent to a historical source. Few other mailing-lists have been running continuously since 1986.

Luckily, all the digests since October 1988 are available as HTML files. And the digest format has remained almost unchanged since that time.
All of the content is in plain ASCII. Messages never exceed a certain
length. IIRC, line length is also controlled. And HTML was officially
not admitted. Apparently, some messages did contain a bit of HTML
code
, but that shouldn’t be an issue.

Here’s what I imagine could be done:

  1. “Burst” out digests into individual messages (with each message containing digest information)
  2. Put all the individual messages (350MB worth) into a Content Management System
  3. Host the archived messages in the form of a knowledge-base
  4. Process those entries for things like absolute links and line breaks
  5. Collect messages in threads
  6. Add relevant del.icio.us-like tags and slashdot- or digg-like ratings
  7. Use this knowledge-base for wiki-like collaborative editing
  8. Assess some key issues to be taken up by brewing communities
  9. Add to the brewing knowledge-base
  10. Build profiles for major contributors and major groups

Because I couldn’t help it, I started writing down some potential tags I might use to label messages on the HBD. It could be part “folksonomy,” part taxonomy. For one thing, it’d be useful to distinguish messages based on “type” (general queries about a brewing technique vs. recipe posted after a competition) since many of the same terms and tags would be found in radically different messages.