11-16-04 Static vs. Dynamic

I'm a professor at Trinity University teaching a course in programming languages. Static typing is something that we have talked about a fair bit and one of my students came across your web page which we used quite nicely to fuel a discussion in one class. Personally I lean toward static typing and I was wondering if you could comment on two things for me.

The first point is my general reason for liking static typing. Basically, static typing provides a proof that some aspect of the program works. Granted, the type system doesn't prove everything is OK and testing is still needed, but at least some aspect is proven correct. Why this seems to matter to me is that complete testing requires the number of tests grow at least exponentially with code size. Anything less will leave many paths through the code untested. No matter how productive one can be in a certain language, writing an exponential number of test cases will overwhelm that. Given this, isn't it worth the reliability to have some aspects proven correct by the compiler?

The more interesting question I have is what you think about ML and its derivatives in regard to the strong testing vs. static typing argument. These languages have the advantage of dynamically-typed languages in that they rarely require the user to specify types, however, they infer the types for all expressions and are strongly statically typed. In this regard, their typing system is far safer than a language where dynamic checks are required (Java) or where users can coerce the type system into errors (C/C++).

The comparison of ML to Scheme came immediately to my mind when I read your article on strong testing vs. strong typing because Scheme does all of its type checking dynamically and gives the user complete flexibility when writing code. However, in my experience, it is very hard to write large programs in and the fact that Scheme accepts a chunk of code doesn't mean it is anywhere close to being correct. ML is the opposite in that is does static type checking on everything even without the user specifying the type of anything. When I get my ML code to compile, it works the majority of the time and one or two simple tests will uncover any bugs that remain.

With Microsoft releasing F# into the .NET framework, it is likely that many more programmers will be exposed to the ML style of coding. I just have to wonder if you would feel that having strong static typing without the coder specifying the types leads to the best of both worlds.

First, I'm at least partially responsible for the mix-up in terminology that has propagated around this, and I have not had the time nor inclination to go back into previous articles and fix this up. It also seems that things are not as clearly defined in the computer world around these terms. So to try to help rectify the situation:

Weak typing: Allows incorrect messages to be sent to objects. C and C++ allow you to successfully cast to the wrong type, and are thus viewed as having weak typing (although Stroustrup once said that "C++ was strongly typed with a couple of holes in the type mechanism"). I have argued in the past that "weakly typed" (a term I originally used to mean "latent typing") was distinct from "weak typing," but I think the terminology differences are far too subtle and not worth arguing over. In addition, the idea of latent typing confuses the issue; C++ (static typing, arguably weak typing), Python and Ruby (dynamically typed) all support latent typing (Ruby calls it "duck typing").

Dynamic typing: Typically still strongly typed, but the type is not checked until runtime. Type checking still happens, and it can be strong typing, but it happens at runtime rather than at compile-time. A strongly-typed dynamic language still only allows you to send valid messages to objects.

To clarify an important point: I'm not against static type checking. The problems as I see them are that:

1. There is an illusion that static type checking can solve all of your problems, followed by the conclusion that more static type checking is always better.

2. Additional forms of static type checking are often added to a language without regard to the actual cost. In extreme cases you spend all your time arguing with the compiler.

In general, my attitude is that static typing is desirable as long as it doesn't cost you too much. As you point out, type inference as found in ML gives you static type checking without requiring that you give extra information that the compiler can figure out itself. The new Nice language also uses type inference.

I do seek the "best of all worlds," as you suggest. The question is whether a particular implementation of static type checking is helping more than it is impeding. I am often accused of just complaining about "finger typing," but what I am observing is much more than just the extra carpal assaults (much of which, it has been pointed out, can be automatically generated by tools such as Eclipse and IntelliJ).

The real issue is the limits of the human mind to manage complexity. The hard boundary is the famous number "seven plus or minus two," the number of things that we can hold in our mind at any one moment. I assert that all progress in computer programming is progress in improving the mental model in order to make it simpler for us to manipulate the essential concepts. The reason I find Python so fascinating is that it seems to regularly grasp and incorporate new perspectives on "simpler" and "essential concepts." Much of my fascination is in how this seems to happen, and I currently have little or no sense of the secret behind it. The language design seems to incorporate the psychology of computer programming.

Despite this, I've had some leanings back in the direction of static type checking. As you point out, the goal is to create solid components – the question is how to accomplish that? In a dynamic language you have the flexibility to do rapid experimentation which is highly productive, but to ensure that your code is airtight you must be both proficient and diligent at unit testing. In a language that leans towards static type checking, the compiler will ensure that certain things will not slip through the cracks, and this is helpful, although the resulting language will typically make you work harder for a desired result, and the reader must also work harder to understand what you've done. I think the impact of this is much greater than we imagine.

In addition, I think that statically typed languages give the illusion of program correctness. In fact, they can only go so far in determining the correctness of a program, by checking the syntax. But I think such languages encourage people to think everything is OK, when in fact the requirement for unit testing is just as important. I also suspect that the extra effort required to run the gauntlet of the compiler saps some of the energy required to do the unit testing. And you bring up an interesting question in suggesting that a dynamically-typed language may require more unit testing than a statically typed language. Of this I am not convinced; I suspect the amount may be roughly the same and if I am correct it implies that the extra effort required to jump through the static type-checking hoops may be less fruitful than we might believe.

What is the best of both worlds? In my own experience, it's very helpful to create models in a dynamic language, because there is a very low barrier to redesigning as you learn. Possibly more important, you're able to quickly try out your ideas to see how they work with actual data, to get some real feedback about the veracity of the model, and change the model rapidly to conform to your new understanding.

I think this approach has great benefits over simply modeling with UML, or this new fad of model-driven architecture. Those produce a model that is a fantasy and only after some complex transformations do you have something that expresses and tests your ideas. My experience with complex transformations of this kind is that they weigh you down and discourage you from experimenting and making changes. With a dynamic language, on the other hand, the model becomes the code and vice-versa, and so you're able to experiment without the inertia of something like MDA. I think this lightness is very important, because it is far closer to the way our brains work.

Once you've worked out and tested a model using a dynamic language, is it then worth transforming it into a statically-typed language? My experience with Python has not compelled me to do this for two reasons:

1. Once I get something working in Python, it seems to work pretty well. The benefits of transforming the model into a statically-typed language at that point are few. Other's experience with significant Python programs seems to support this – large Python programs seem to be surprisingly bug-free.

2. I usually find that any program that is used regularly will be changed. It is much easier to change the program in Python than it is in a statically typed language, so I have further incentive to leave it in Python.

I certainly don't mean to imply that it never makes sense to transform a debugged Python model into Java, just that I haven't been compelled to. I could easily see many business situations where:

1. You begin by "sketching out" a preliminary model, using "UML lite" and/or CRC cards.

2. You implement this model in Python or Ruby, whichever language the developers are comfortable with. At this point you leave the paper model behind, and the code becomes your model. Python is often described as "executable pseudocode" and this becomes very helpful during modeling and experimentation.

3. You experiment and evolve this "live" model until it seems like you've worked out the kinks and it will do the trick.

4. You then translate into Java, C++, C# or some other language that the project constraints dictate.

By developing the model in a language that encourages change, my experience is that you end up with a better model, and this produces a distinct benefit when that model is translated to your implementation language.

In the end, my primary interest is in productivity and scalability. Visual Basic is a very productive language for small projects (and there are lots of those, so it solves lots of problems) but it doesn't scale well. Perl also falls into this category, although its apparent successor, Ruby, seems to implement the object paradigm reasonably well and this suggests that it may scale to larger projects (these may already be in existence; I have not been tracking it). Python has been used on a number of significantly large projects and despite its lack of static type checking, the results appear to have very low bug counts.

This last point is a major puzzle – we believe that static type checking prevents bugs, and yet a dynamically-typed language produces very good results anyway. As I have tried to delve more deeply into this mystery, many of my preconceptions – the major one being that static type checking is essential – have been challenged. An initial response to this is often to simply deny that it's the case, but once you begin denying evidence your theories rapidly become nothing more than fantasies. In my own experience it can be very hard to put my finger on exactly why Python works so well. However, in trying to do so I have discovered many things and gained greater understanding about the other languages I use (see, for example, my recent articles about latent typing and Java generics).

My guess is that Python allows me to think more clearly about the concepts of the problem that I'm trying to solve. It is less distracting because it doesn't force me to think so much about the rules imposed by the language – rules that are basically arbitrary when I'm trying to produce an effective model of my problem space. By getting out of the way, Python and similar dynamic languages allow me to spend more of my brain's "seven plus or minus two" items on the problem itself, and less on the details of the language implementation. I have had this experience more than once, for example when going from C++ to Java where I no longer had to worry about operator= and the copy-constructor. Having Java take care of more things for you definitely seems to improve programmer productivity, and I've had the same experience with Python.

In the end, I can still see that there could be some value in taking a model that was evolved using Python or Ruby and reimplementing it in Java (and note that, with Jython, this could even be an evolutionary process which would allow you to slowly morph your Jython code into Java). Much of this value is in fitting into an existing Java development environment, but it also seems likely that you might discover some bugs because of static type checking. It would be interesting to hear experiences and bug counts from people who have done this experiment.

You asked a question about whether type inference a la ML might give us the best of both worlds. From what I've seen, type inference is certainly an interesting addition to a language but it only touches on the different benefits offered by a dynamic language. In addition, type inference may actually exclude what I consider to be an important feature: latent typing, which puts less of a constraint on the types used by a function rather than more, and thus allows the creation of truly generic code. (Although I describe it in other articles, in brief: latent typing means that a piece of code can say "I don't care what type I'm working with as long as it has these methods." Latent typing is still strongly typed, however – you cannot send incorrect messages to objects). I do not know whether there is a proof one way or another about whether latent typing and type inference are exclusive features, but my impression is that it would be rather difficult to implement both in one language. Nice, for example, has type inference but does not have latent typing.

We need help in order to create accurate programs, and it's pretty hard to argue against strong typing. The type system is the water in which our object- oriented fish swim, and to allow holes and back doors says "these things are true about a type, but you can only rely on it if people know what they are doing and are behaving well," which is not very reassuring. Strong typing actually helps us think about our object models by ensuring their proper behavior.

The question is not whether strong typing is a good thing. I'd say it unequivocally is. The question is when does the type checking occur, and how much does it cost to have it occur statically vs. dynamically. In C++, effectively all the type checking is static. In Java, it is both static and dynamic (I think any language that tries to achieve thorough type checking will require some dynamic type checking), and in Python and other dynamic languages it is predominantly dynamic. As you note, in most dynamic languages not all execution paths are tested by the compiler, and this can be somewhat of a problem, but how bad is it? I suspect that if we applied the same unit tests to both a Java program and its equivalent Python version we would exercise all the execution paths; I think that the unit tests are testing at a high enough level in both cases that you end up with the same number. Put another way, I don't think you will need extra unit tests in order to produce syntax checking, but that the syntax checking will fall out naturally when the unit tests exercise the class interfaces. I don't have direct evidence for this other than not having to do this extra work myself.

One of the things that Java Generics accomplishes is the static type checking of collection classes. This prevents ClassCastExceptions at runtime, which is somewhat useful, but if that were the only reason for Java Generics it wouldn't be worth the complexity of the syntax (fortunately, with some effort they can be used to create generic code, as I've shown in previous articles). The resulting ClassCastExceptions don't happen that often, and are not difficult to find when they do. This is a case where dynamic type checking is adequate.

I have also pointed out in previous articles that checked exceptions do not scale well, and can easily get in the way even for small programs. The fact that no language designed since Java has duplicated this experiment makes, I think, a strong argument against it. However, I have found that the new (as of JDK 1.4) RuntimeException(Throwable cause) constructor eliminates most of the complaints that I have about checked exceptions – if they get in the way, I can turn them off with only a minor amount of coding. Thus, I can choose to leave them on or turn them off.

These are a couple of examples where too much static type checking, no matter how well-intentioned, gets in the way of both the creation and the examination of code. Despite that, I actually like static type checking, appropriately used. For example, many of the features of the Nice language are very appealing to me, and in general it seems to be designed to do work for you while staying out of your way. I have even considered suggesting an interface mechanism for Python to be used for static checking. But I resist any implications that "all static checking is good, so more is always better." If you think about this you can easily imagine it taken to extremes, and I think that in a number of cases Java has done exactly that, without doing a cost-benefit analysis of the results. It is the illusion that there is no cost to static type checking that I argue against.

Feedback Wiki Page