8-16-04 Process Patterns
Last week, Bill Venners and I gave the Designing Objects &
Systems seminar. Each time we've given it, it has been unique, and one of
the hard things about marketing the seminar is explaining just what kind of
experience it is. This version was even more interesting than usual. Partway
through the seminar, after we had been introducing and discussing any number of
different ways that various people have invented to discover the objects and
their structure when designing a system, a couple of people observed that all of
these hand-waving descriptions of other people's design tools were not helping
them with their team project. In storytelling parlance, we were "telling" rather
than "showing." In order to "show don't tell," how about if Bill and I came up
with a design for a problem that we had not seen before, live, in front of the
class?
This turned out to be quite stimulating for us, and it seem to give everyone
a boost. One of the important things that we had been discussing is the idea
that there was "no right way," and sure enough, we both attacked the problem in
different ways. Bill is prone to thinking in terms of Java interfaces, and the
problem statement we were given did in fact request interfaces as the
deliverable. However, it also gave a lot of detail about the model (which is
what the students who created the spec were actually looking for, it turned out,
despite their specification of interfaces), and I went in that direction,
because that is my bent.
With all OO analysis techniques, the goal is to (at least) produce objects,
their interfaces, and their relationships. Bill and I did that, in our different
ways. All the different techniques that everyone else comes up with produces
those three things (and sometimes a lot more). We both have different
experiences and backgrounds and those have combined to influence the way we
approach problems.
Switching to a different example, consider the discussion that I and others
have been having about static vs. dynamic typing (see earlier weblogs). If we
can smooth over some of the bumps and say that C++, Java, Python, Smalltalk,
Lisp and similar languages are strongly typed (yes, I know it's possible
to use casts to push both C++ and Java into runtime exceptions, but in the
absence of such casts they tend to behave themselves), and that the issue at
hand is when such typing occurs (static -> compile time, dynamic -> run
time), then the arguments seem to come down into two distinctly different camps:
- I need every possible type check that I can get, imposed as early as
possible. Even with all those type checks, problems slip through, so if it's
possible to add even more compile-time checking, that would be better.
- The extra verbiage that must be added in order to make the compiler happy
doesn't really accomplish that much and has a severe impact on both productivity
and code maintenance. We have to write unit tests anyway, so the illusion of
safety comes at far too high a cost. Ned
Batchelder put it another way: "Static typing prevents certain kinds of
failures. Unfortunately, it also prevents certain kinds of successes."
I started out in the first camp, seeing all the problems that C (especially
pre-ANSI C) allowed, being firmly staunched by C++. I stayed in that mindset with
Java, which seemed to do a better job than C++ (with things like checked
exceptions). After a brief bout of Perl (great for quickly writing code, bad for
scalability and maintainability) I moved to Python and began puzzling through
both dynamic and latent typing (see previous weblogs), wondering why these
things which went against my experience nonetheless seemed to work so well,
eventually moving into camp #2 and finally understanding what the Smalltalk
programmers had been saying all these years.
When I had the internship program a couple of years ago I moved back towards
camp #1. The interns were generally inexperienced and they needed rules. Even
then, they would sometimes actively circumvent the the rules. Only the rules
that were built into the compiler couldn't be circumvented, and thus took on the
nature of "received wisdom" to the interns who could then believe it. The
additional rules that I either added as written guidelines or tried to impose
via CVS checkin rules were sometimes seen as my arbitrary constraints and did
not have the religious weight of being part of "the language," and so were
sometimes ignored or disabled.
On web discussions, this is sometimes referred to as "having a bunch of
programmers with a few weeks of Visual Basic training." BASIC was my first
language and over the years I accomplished many things with it, so I don't think
that this is saying VB is bad. (Indeed, I am far too experienced to imagine that
I shall never be in a position where I must use it again). Instead, I think the
phrase refers to programmers with very limited experience and world view. That
is the big difference between the first and second camps -- they are dealing
with different kinds of programmers.
My father was a building contractor until he retired. I saw many different
kinds of subcontractors, all of whom would say that they "worked on houses." But
when I worked for my father (an experience that not only motivated me to stay in
college for a rather long time, but also to develop a different profession; the
point of the exercise, his chief carpenter explained to me), I put up sheet rock
(drywall) just like the sheetrock subcontractors would. I took apart old
plumbing but I never put together new plumbing. I did a little bit of carpentry
but never finish work. And yet I could say that I "worked on houses," much the
same way that I could say that I "programmed computers" sometime after I began
to do that. But for a long time I was more of a ditch-digging, sheetrock-hanging
kind of a computer programmer, who needed a lot of direction. You wouldn't want
me anywhere near a tool when the finish work was going on.
In Tom DeMarco's book Slack: Getting Past Burnout, Busywork, and the Myth
of Total Efficiency, he talks about the illusion of treating people as
"fungible resources," where one person can easily be replaced with another. I
suspect this idea is more predominant in programming, where the mystery of the
craft prevents the average manager from having any concept of what programmers
do. Programmers type code, and code looks like a bunch of words. Why can't one
programmer be replaced with another?
But as in people who work on building houses, there are vast differences
between programmers. I would say the main division is between those folks who
treat programming as a saleable job skill (and who have other lives and
interests, which is certainly admirable) and those for whom it is a profession,
a craft, an interest, a pursuit on its own, and a passion.
This is only a rough distinction, of course. People who consider programming
"just a job" may still dedicate themselves to doing the best they can while they
are on the job, and people for whom it is a passion may still fritter away their
time being passionate about their favorite language on the web, or playing
games.
A second division is experience. The maturation of a programmer is almost
certainly linked to their personal maturation. Life experience influences the
way you think. Our ability to model, and to implement that model, is all
about thinking, perception, and the ability to organize and manipulate thoughts
and perceptions. And of course, to interact with other people. These lessons
don't come easy.
A third important issue is the ability to understand that -- even though we
ultimately work in a world of ones and zeroes -- things are often not what they
seem, and that an open mind along with a well-developed sense of critical
thinking are the most powerful tools you can have. I don't know if it's simply
experience that teaches this, or if additional factors like training and
mentoring are involved (although it is certainly tempting, given what I do, to
assert the latter, but that would suggest that I myself have not yet risen above
the one/zero mentality, and I like to think that I have).
Is there some way to manufacture experienced programmers more quickly? A
group of folks who write, consult, and educate about programming met in
Portland, Oregon January 15-17, 2003 for the "Writing Better Code" summit, where
we tried to struggle with the issues of why programming continues to be
practiced in a poor fashion, and ways that the situation might be improved. I
think because the problem is more about psychology and individual growth than
anything you can apply rules and guidelines to (as we digital types are wont to
do) we didn't achieve much. The only resulting artifacts that I know of are an article on Artima and
Chuck Allison's address to the computer science class at his college (which will
appear on the upcoming "Thinking in Code" audio interview CD; subscribe to the
newsletter to be notified when that becomes available).
What I want to explore here, however, is the corporate cultures that promote
or demote good programming practice, and support or discourage learning. More
specifically, not so much whether you are a "learning organization" (to use a
business-fad word from some time in the past few years) but how you got the way
you are ("One logical step at a time," according to Gerald Weinberg in
Secrets of Consulting).
The question I'm asking is this: is it possible that the apparently endless
variety of organizations that we see are based on permutations of a finite set
of fundamental patterns, and that if we can discover these patterns we may be
able to better understand and even improve these organizations?
Here's a candidate pattern partly based on the above static-vs-dynamic typing
discussion. We can take a first shot at naming: "Programmers as Commodities" or
possibly "Programmer Experience Clumping." A business begins and programming is
some fundamental part of the products that they create. However, there is no
programming experience in the "C" level of the corporation (CEO, CFO, CTO,
etc.). Programmers are seen as a commodity, and with no other criterion to go
on, cost becomes the primary decision point. Inexperienced programmers are
willing to work for this company at low cost, in order to gain experience, while
experienced craftsmen can see what is happening and thus go elsewhere, seeking a
place where programming is better understood and appreciated. This produces a
"clumping" effect, where inexperience is clumped together within the company,
and the experienced craftsmen may or may not clump together elsewhere. With only
the guidance of other inexperienced programmers to go by, a superstitious mob of
programmers evolves, most likely just doing whatever the inexperienced
programming manager tells them, which is likely to be the current fad. No one in
this organization knows what a good programmer looks like except that such a
person is likely to be too expensive, and so the hiring process is unlikely to
produce any good programmers unless something changes.
Note the dependence on initial conditions, one of the big issues in
complexity theory. The same equations produce radically different results given
(sometimes very small) differences in initial conditions. In the above example,
the company might introduce a CTO with software experience, or at least software
awareness coupled with an open mind, and dramatically change the resulting
software culture of the company. Even one or more timely visits from the
appropriate consultant (yes, yes ... I do this but I'm being serious here, not
just trying to market) could help establish conditions that grow a much better
software culture.
Ultimately, I think it takes good programmers to recognize other good
programmers, and also to recognize inexperienced programmers that have the
potential of becoming good programmers, given guidance and a supportive
environment.
A second pattern could be called "Faux Chief Architect," and it is one I have
seen more than once. A company is aware that there are important differences
between programmers. However, the idea of creating a corporate software culture
sounds like too much effort and expense, and so instead they decide to find a
really smart chief architect and let that person guide the process. The
assumption is that the architect (A) knows what s/he is doing and (B)
understands people and organizations, not just software. Because the "C" levels
in the organization don't understand the issues themselves, but they think they
do, they are impressed with a particular architect who interviews very
convincingly. Alas, that architect is not truly experienced, and primarily wants
the feeling of power. S/he assures the "C" levels that they no longer have to
worry or pay attention to software, and then proceeds to make dramatic decisions
which cannot be questioned by subordinates without being fired. All along, the
"C" levels are assured that software issues are humming along, while the project
becomes more and more topheavy with the decisions passed down by the
architect/demigod, decisions which sound crazy to the people working on the
project. Perhaps it should be called "mad emperor syndrome." By the time the "C"
levels begin to realize that things have gone horribly wrong (often by calling
in consultants to evaluate the situation), the project is a foundering
shipwreck, and most or all of the time that went into it is lost.
Of course, I have no idea whether these two examples are really patterns, and
I note that both of them are negative, so-called "antipatterns." The
antipatterns seem to appear far more often than the positive patterns and so
would fall under the category of problem discovery. But like Tolstoy's famous
first sentence in Anna Karenina ("Happy families are all alike; every unhappy
family is unhappy in its own way"), the likelihood of things going right is
quite rare and the results would seem to be quite consistent, whereas there are
myriad ways for things to go wrong.
A positive pattern might be "The Gelled Team." Someone in the company is able
to hire an energetic, experienced programmer who enjoys interacting with people,
and to give that person a key role in hiring other programmers for the team, and
a reasonable budget to do so. This person is instrumental in hiring people that
s/he wants to work with, and as a result a team is formed that really enjoys
being together and working together. The team becomes remarkably productive and
effective, and the company doesn't break up the team in a mistaken attempt to
distribute the seeds of its success to other teams.
So, is there the kernel of an idea here? If you can think of other "Process
Patterns," Suggest them here.
