1-13-04 Rethinking weblogging (and everything else)

Feedback

It seems I have to struggle with a problem for awhile before "the simplest thing" begins to occur to me. It's also helpful to study solutions that other people have developed, to see what works and what doesn't. Perhaps there is a maxim "you can't be both simplest and first." Corrolaries would be "don't make it so perfect you can never deliver it," "worse is better," "stop after the first 90% (because the last 10% will take the other 90% of the time)," or simply "just solve the core problem." I guess that's the thing, though: knowing what the core problem is can be the major part of your struggle (This is certainly the case with design patterns). Elegance is inversely proportional to sweat.

Because elegance is difficult and time to market is crucial, many have come to believe that the former should be sacrificed for the latter. It's hard to argue against this. The rabbits that rush a half-working piece of garbage to market are usually the ones that are wildly successful, whereas the careful tortoises are never noticed. Of course, this is a description of an industry that is fueled with startups and innovation, rather than a mature industry. You're probably not going to make a dent in the world of tires or tv dinners without a lot of up-front research and investment. Perhaps there are businesses other than software with enough wildcards that it's possible to get a brainstorm and suddenly create a breakthrough product with very little investment, but I can't think of any.

My belief is still that "Elegance always pays off." The payoff comes after the initial flush of market success, when you are improving the product and making it more complex. The architecture and implementation techniques will then either spur you on to more greatness, or sink you. But it seems rare for a company to become successful with a bad design and implementation, and then stop and refactor the whole thing rather than rushing to add more features. So from the acorn grows the oak.

What is the balance between simplicity and expedience? "Do the simplest thing that could possibly work" is certainly not saying "do the most elegant thing" because the goal is to get something working, without too much effort, so that you can try it out and see if it solves any portion of the problem. "Trying it out" is what will produce the valuable information that can be fed back into the next iteration, and will also begin to tell you what's most important about the problem.

All this musing was precipitated by a mental retrospective about my weblogging experiences. I suppose it all started when I decided to stop creating articles from scratch for print magazines. I had begun to notice that people refer to a magazine article by mentioning the magazine, not the author, but with a book they typically don't remember the publisher, but only the author (O'Reilly has been the exception, but they've been sliding towards ordinary-ness over the last few years). So my premise was that (A) I want people to come to my seminars and workshops (B) they must remember my name in order for it to have any value (C) I should focus on books, not magazine articles. At that point, the only articles I wrote were adaptations from whatever book I was working on; some of you may remember a long series on C++ in Embedded Systems Programming that was taken from "Thinking in C++" as it was being developed. Since then, of course, the Internet has been strangling most computer print magazines.

As an aside, the fascinating effect of newsreader software is that it allows each individual to create their own magazine, filled with their own favorite columnists. This was predicted by futurists, but those predictions were inevitably a vision of some central newspaper-like authority assembling and delivering your customized paper to you. We have been so conditioned to think in terms of Big Authority Figures that the concept of assemble-your-own directly from the producer didn't occur to anyone until after it already happened. In hindsight, it's obvious: it's not only the cheapest approach, but also the most flexible and open, and serves the consumer best.

Putting "Thinking in Java" up on the web while I was developing it was the next step in the process. The myth that "all you have to do is put up a website and customers will flock to it" had passed by then (At the time, I was hoping that the web would go away, or at least get better, before I had to figure out the technology). My friend Richard Hale Shaw and I had done a series of seminars through the Software Development Conference which were marketed by them, through their mailing list, with lots of money spent on printing and mailing brochures, direct phone solicitation, and magazine ads. Richard and I weren't happy with the situation, but it seemed a very risky proposition to undertake on our own. However, if I could get people to come to the web site, I reasoned, I could tell them about the seminars without much risk at all. I decided that even if I never sold a single book, if people came to seminars it would be more than worth the loss. The publisher was very uncomfortable about this and did a small initial print run, which sold out quickly. Whew (they thought)! Print media has not been extinguished by the internet, and printing presses and trucks are still a viable business, even if people are able to download and print a book on their own. The convenience of the package is still valuable.

So that became my model: continually publish a developing book on the web, and publish it in print when it becomes finished and stable. All the while, give seminars based on the material. But a big problem with this approach is one of granularity: everything is now in book and seminar-sized chunks. Books are big; associated seminars are 5 days long. There's no room for other forms of expression that are smaller and/or more agile and flexible than books + 5-day seminars.

Several years ago I began seeking such forms of expression: how can I talk about things in units smaller and more nimble than books/seminars? Writing articles on the web seemed like a logical first step, and it also seemed logical to use someone else's article-publishing package rather than inventing my own ("buy don't build"). After a bit of looking, it seemed that the SlashCode software would do the trick, and it was freely available. However, it ended up being messy to set up and configure, and I think over a year of potential publishing time was lost while fooling around with it, and by the time I actually started using it I discovered that the whole thing was very awkward and unpleasant. This is a case where using someone else's solution was a costly mistake – If I had simply started putting up articles right away, working within the Zope system that I already had running, I could have been publishing and experimenting during that whole period, rather than being stuck. Because I was unable to do the experiment, I couldn't move forward.

Not that Zope is the perfect solution. I like Zope a lot because I've learned a great deal from it. I think that building an app server on top of an object- oriented database and integrated security, using URLs for object locators, using a single consistent, powerful language for application development, and the "product" concept for extensibility is all brilliant and has great potential (and making it free is especially attractive). But everything in Zope is most definitely invented from scratch, and not all the ideas have worked out. The Zope Corporation knows this and the whole point of rearchitecting Zope 3 is to make it easy and expedient to program. But as powerful and useful as it's been, I've also started to think that it's been holding me back in a number of places. For a long time I couldn't understand why other people were creating new application servers based on Python – why not just use Zope? But after all this time, I don't "Think in Zope" yet because I don't do it all the time and there are too many conceptual hurdles to absorb if I don't do it all the time. So while I'm sure I will not stop using Zope as an app server, I think I also need another Python-based app server running alongside which is much easier to program (possibly until Zope 3 finally appears and overwhelms me with its simplicity). The Twisted Matrix looks like a very promising candidate.

Part of what I'm trying to convey in this article is the subtle threshold beyond which a thing is just a bit too complicated to absorb into your toolbox, along with all the other tools. The subtlety is that it's possible to use that tool, but beyond a certain vague point it's just a little bit too hard to put down and then easily pick up again later, and for me, that's the crux – not whether it's usable at all, but how easy is it to put down and pick up? If I have some fantastic wrench that requires me to consult an instruction manual every time I use it (Java and IOstreams come to mind here – I know of colleges that have gone back to C++ as a first language just because teaching "Hello, world" in Java is so complex), I'll probably just hunt around for the plain old fixed-size wrench, ashamed that I'm not sophisticated enough to use the fancier tool. Making things effortless by lowering the complexity of accomplishment really does make a difference. This is why build systems are so important, for example. For me, the most enticing feature of the Mac notebook computers is the "instant on" feature, a perfect example of making something easy to put down and pick up (of course, running Unix is a big plus, too). Python's dramatic reduction in both finger-typing and type-check typing make it a far quicker and more functional tool than most other programming languages. And right now, if I were wireless and could drift around the house while connected to the net, that could seem like an excessive luxury but it would be helpful enough to make it worthwhile.

So phase two was to use Zope with a few enhancements (to create a master list, list the most recent articles, and create an RSS feed). I did this last February and started logging, reasonably happy with the system. But as Volume 2 of "Thinking in C++" became more intense, the tool became just a bit too hard to use – the aforementioned threshold of easyness got moved, in this case. Then I went to Burning Man (I will eventually collect my scribblings on that experience into an article) and had my head rearranged about what the world might look like, and after that it was nonstop C++ until the book was done, and in the meantime I fell back to maybe one entry per month (and suspended the newsletter for quite awhile, as well). It's not that I didn't have lots of ideas during that time, it's that even the small ceremony necessary to put them up had moved me into thinking in terms of articles (worth the effort to publish) vs. ideas (not worth the effort to publish using my Zope system).

After going to Gerald Weinberg's "Amplifying Your Effictiveness" (AYE) conference in Phoenix this past November, as well as creating the article on UML drawing tools where reader feedback was so useful, I found myself wanting more feedback. Initially, I tried making wiki pages and asking appropriate friends and acquantances to comment upon my chosen topic, but it wasn't always interesting to them and most of these conversations drifted to a stop before I was satisfied. Also, the effort to start such a topic puts in too much of a threshold. The obvious alternative is newsgroups, but these have always been far too noisy and meandering to me. Because they must be under one vast umbrella, they are not topic-oriented enough and it's too hard to follow the thread of a conversation that interests you. This assertion is more than supported by all the thread-management software that's been developed for newsgroups.

For each of the last couple of articles I've posted (and notice that I'm calling them articles, because it's clear to me now that's what they are), I've added a simple wiki page (using the Zope ZWiki product), which just allows adding comments and not editing the page (I'll consider the latter if the need appears). This seems to have worked quite well, and the article becomes the topic and all the comments are directly associated with it, so there's no effort to read the full conversation about a particular topic. I suppose a logical extension of this is to use a single wiki page with the article at the top and the comments following. This would end up looking much like Ward's Wiki at C2, and right now it's a little depressing to think that all of my mental effort would simply bring me back to using an idea and structure that they thought of years ago, so I won't go there just yet (I also think it's helpful to separate the topic from the comments, so you don't have to see all the comments if you don't want to).

I now believe there are three modes of written communication: books, articles, and ideas. The first two I have long experience with, but I lack a medium for ideas. My friend Bill Venners, who created Artima.com, notes that his weblogging structure at Artima also seems to motivate people to produce articles rather than ideas. Daniel Will-Harris observes that the weblogs he finds most interesting are the ones that have small idea-sized entries appended together on a single page.

That's what I'm missing, I think. I will certainly continue to have article- sized entries that come up from time to time (Bill dubbed these "blarticles"), but I'd like a medium of expression for ideas that is so simple that I won't hesitate to put something up. This will be more about brainstorming than producing fully-developed concepts. So I think I'd like it to be email-based, so I can just email entries to my weblog. I think the mailblog could enable weblogging for a lot of other people, as well.

Some time ago we figured out how to add FAQ entries to Zope via email, so that may be the right place to start. However, I'd be interested in knowing if there are any systems that already do this out of the box (as noted above, I'm not interested in learning a whole new world view in the process).

I also wonder if, in addition to an RSS feed, some people might find it more helpful to just add their names to a list and get an email containing each new idea (and a link back to the associated wiki page if they want to make comments). Most of the people I know haven't acquired RSS readers, and the few readers that I've tried have been remarkably awkward so I'm still in wait-and- see mode on that (if Mozilla incorporated a slick RSS reader into their browser it could become a must-have that Microsoft would have to catch up with).

A related problem I've been struggling with is to create other new modes of expression. The seminar and the seminar-on-CD have been versions of this and certainly effective in their ways, but are notably high-ceremony operations that don't lend themselves to being easily thrown together as experiments to see if there's anything to a particular concept. I've been looking into Flash development tools to see if there's an easier and better way to turn a slide- based lecture into a Flash presentation, as a way to distribute "article-sized" multimedia experiences, but so far the amount of effort required to produce these things seems quite high; not something you can just dash off in a few minutes. Still looking for the magic solution there. Daniel and I have discussed ways to use a phone bridge for distance learning, along with HTML slides and something like NetMeeting (yes, a phone bridge rather than streaming audio, which has never worked satisfactorily. Sometimes old technology is the best solution).

More important to me is live experiences. I've been struggling to escape the straightjacket of the full-scale seminar for years now. After organizing seminars through the web, I began to imagine what I called a "self-organizing conference," one that would involve the speakers in the structuring of the conference, and use web technology to do most of the rest. Although I still think this idea has potential, Martin Fowler rescued me from the throes of my imaginings when he suggested that we try something like it on a small scale, which we have done with great success for several years now and are beginning to expand (we call them "summits"). This takes self-organization to a whole new level by making it spontaneously self-organizing. You get a bunch of interesting people together and decide what to talk about. At the Python conference last February I was exposed to a variation of the "open space," where someone puts up a topic and people show up to participate in the discussion. It's like a BOF, but it doesn't require so much planning; you can come up with the idea right then rather than being required to submit your BOF for approvial weeks or months in advance. At the Python conference, they only had a single big room for this (usually the spaces are physically distinct), which actually made it more interesting because people could easily drift around the room from topic to topic. Another activity that I first saw at a Python conference was the "lightning talk" where people could get up and speak about what they were working on or interested in for 3 minutes, just long enough to get the ideas across but not long enough to get boring. Notice there is some parallel between these structures and written "articles" and "ideas."

Over the past number of years I've also been evolving my seminars towards being less "teacher talking, class listening" and more interactive experiences for everyone. The more I do this, the more that I find that people learn much more and we all have a much better time (my friend Matt, a physics teacher, is fond of telling me how studies show again and again that lecturing is the single worst way of transferring information). The closest I've gotten to a good mix, I think, is the design patterns seminar, where I briefly describe a single pattern and then we turn around and do an exercise that develops that pattern. I find that we not only move forward faster, but that everyone's energy level (most notably my own) keeps increasing because there are no long, drawn-out lectures (I've heard there is an ideal length for lectures, something under 15 or 20 minutes).

What I'm looking for, I think, is a spectrum of in-person experiences, just like ideas and articles and books are a spectrum of writing experiences, and the Web and MP3 and Flash and phone bridges are a spectrum of media experiences. One dimension of this spectrum includes working with other people, friends whose minds I enjoy interacting with, such as Bill and Richard and Daniel and Martin and Dave Bartlett and Jeremy Meyer and Scott Meyers, and some of the people I've met at the summits. I think it would be stimulating and educational for people to see and participate when it's more than just me there. In fact, I know this is true because when Bill and I have done seminars together and have gotten rolling back and forth, people have invariably commented that it's entertaining and stimulating.

Speaking at user groups is certainly one end of this spectrum, and the user group experience seems in general to be beneficial (although I have never found one to belong to myself, probably because I have this tendency to move around and live in obscure areas). But I would also like to create events that have various levels of remuneration to help support my other habits. One of the wiki-discussions I launched was based on an idea of having myself and one other person available in a hotel suite in a city for a few days for team consulting. The idea was that if we were already going to be there for some reason we could stay a few extra days and save the travel costs for potential customers. Also, customers could easily book a few hours or a couple of days, depending on their needs and budgets. Although I think there's a germ of a good idea buried somewhere in there, the general concensus among the discussion group was that this would be too hard to package in an understandable way.

However, the concept of the package did get me thinking about what works for customers. A seminar is a clearly-defined package with a lot of structure so people see exactly what they are getting, and it's much easier to sell to managers. But consulting – what does that mean? Everyone who isn't an employee says they're a consultant, even if what they're doing is contract programming. If I say "I consult," it always requires further explanation ("I come in for a few days and do a design or code review, or help solve a specific problem"). Perhaps what I need to do is create a "consulting package" with clearly defined activities and durations (and the ability to customize). That gives people a much better idea of what I do and what the possibilities are. I am currently ruminating on how this should look.

In the grand scale, I still think of books and seminars and conferences, of course, but I'd like to think that tools and modes of experimentation that allow thinking-in-the-small-and-experimental could allow me to create more interesting and exciting experiences (both for me and for you), and to develop these experiences in a more organic and evolutionary fashion. I am coming to believe that both the results and the process will be not just more desirable, but more exciting and energizing for everyone concerned.

Feedback