9-14-03 Solve the Real Problem

Note: Because I've started treating this weblog as a place for essays that take a long time to write – which is fine, sometimes, but that attitude often stifles the easy flow of words – I decided to try an experiment, prompted by a reader question. Occasionally I get a reader question that stimulates some thoughts that might qualify as a non-essay weblog; some ideas that might be worth sharing.

I received an email this morning from a reader struggling with a project where he has already made a certain number of baseline architectural decisions, without, I think, really considering whether they are appropriate or necessary:

I'm trying to develop a business application in C++. I want to be able to build a framework of the business objects which can be extended later without touching the framework source.

My main problem (one of them anyway) has been designing a way for these objects to be persistent. I want to use a relational database product. The headache comes from being able to do polymorphic reads of the data from the database.

I have made various attempts to design a way around this but have just been running around in circles with factories, abstract factories etc and still no luck. I have read in your book Thinking in C++ about virtual constructors which I hoped would do it for me.

Problem Description

If have a class hierarchy of classes that looks like this.

Animal
  |
  +---Vertebrate
  |   |
  |   +---Mammal
  |   |
  |   +---Reptile
  |   |
  |   +---Fish
  |   |
  |   +---Bird
  |    
  +---InVertebrate
      |
      +---Insect

I want to be able to load from the database from any node in the hierarchy and have it polymorphically load the correct object types. E.g. If I load all animals from the database I want the set to contain instances of Reptiles, Fish, Insect as appropriate. I also want to be able to go down the hierarchy and load all instances of Vertebrates and have it load the correct instances of its subclasses.

Many of the solutions I've seen require the supertype to know of its subtypes and sometimes even requires instantiating an instance of a factory instance of each of the types even if it may not be used.

I want the superclasses to be built into a framework like for example a biology framework or accounting framework. Applications built on the framework would now extend the framework by creating new subclasses of the classes provided in the framework. In this situation it is not possible to go and modify the factory classes in the framework.

I hope you can help me with a solution to this problem.

You are trying to solve the problem of the "Object-Relational Mapping" (OR Mapping). Research that and you'll begin to understand the complexity of this problem. If you really need an OO database or the appearance of one, you should probably try to use an existing OO database rather than trying to build your own.

In addition, I suspect you have prematurely decided that you must build a framework. Read Martin Fowler on this, as well as some of his (and other folks) other writings on frameworks.

Finally, it sounds like you need to do what Dave Thomas and Andy Hunt call Tracer Bullets. You need to build an end-to-end solution, doing (from Extreme Programming, a.k.a. XP) "The simplest thing that could possibly work," and then decide what problems you really need to solve, rather than deciding up front that you have to have an OR Mapping, a framework, or even that you have to use C++.

One of the biggest reasons for software failures is not understanding what problem you really need to solve: the customer's problem. And even if you understand that it's about the customer, you are only part way there – you have to understand what the customer's actual problems are, and these are usually different than what the customer thinks their problems are. Often, the best way to flush out the true needs of the customer is to interview them, caputure some of the basic "User Stories" (another XP term) and implement those stories in the simplest and fastest way possible. Consider using a rapid development language like Python on an existing application server like Zope (which already has a built-in OO database) to build something fast and present it to the customer, just to see if you are even trying to solve the right problem.

He replied:

Thanks for replying really quickly and thanks for the slap on the wrist. I think your answer was really what I needed. The project I'm working on has been delayed for about 2 months now and up till now there seems to be no visible progress on it.

It has been a long and torturous chain of decisions that finally led me to even try to develop the application in C++ at all. I had for the last couple of years abandoned my C++ roots for Visual Basic, mainly because most of the requirements of our clients did not require using C++. However I decided to come back to C++ because VB just seemed to be a waste of time. Every project had to be rewritten again despite all the work you had done in a previous project trying to build some reusable components. I just got frustrated trying to do things in VB I could easily have done in C++. Years before that I had been working with a RAD tool called Magic which allowed you to deploy applications really fast, but that again drove me up the wall. It's implementation of even the simplest things like Comboboxes was dreadful. I could probably have lived with it if I didn't know any better. Alas here I am again thinking of dragging my self back to VB. Maybe I will eventually and maybe not. But one thing I will definitely do is concentrate on the user perspective by doing some UIs and slowly and gently start doing my domain classes and their implementation.

Thank you again

The problem with Visual Basic (pre-.NET, anyway) is that it tends to encourage a bad style of programming; in particular, mixing the UI and the business logic together, which discourages the creation of maintainable programs in favor of rapidly-developed, but hard-to-maintain programs. Thus, as you observe, you end up disposing of old programs rather than maintaining or reusing them. See Martin Fowler's article on Technical Debt.

Visual Basic doesn't have to work like this, despite the language and development environment. If you know that your goal is to separate business logic from UI to promote maintainable, reusable code you can do it. In addition, VB.NET has true OO support (along with all the other useful .NET doodads) and so it may satisfy your longings for C++ features. If you have deep knowledge of VB and it solves a lot of your problems for you (not, however, the OO database problem as far as I know), then it might be the most expedient solution to developing your system.

Project development is risk management, and depending on your needs this might be the best way to minimize your risks. I have often made the mistake of premature assumptions and only much later have come around to seeing (once again) that I should have tried "The simplest thing that could possibly work," even if that only gave me enough information to move on to a more complex solution. Most of the time, the simplest thing was all that I needed.