home > writings > On Software Design
2008 May 06
Much has been said about the right way to design and implement software. It is quite surprising that many still follow a failed model that has already resulted in a flood of bad software.
Here I would like to suggest a rather underrated approach to software design. Surprisingly this method has been suggested by quite a few master programmers and ignored by almost everyone else. When I say master programmers, I have men like Donald Knuth and Richard Stallman in mind. They created software that endured the test of time and won the admiration of millions.
Knuth's Tex is an example of excellent software engineering. It has been in use for nearly thirty years. Knuth did not make any major release of the software since 1989 because the product has become feature complete and virtually bug free! There are very few software systems of similar perfection, longevity and usability.
What is the secret behind the success of these great hackers? When I say success, I don't mean to say that they became millionaires by writing software. That is a legitimate way to measure success, though, here we are concerned only about quality. They are men of extraordinary talent is one reason. The other reason is that they followed a rather simple method of software design. Knuth gives this short recipe for writing complex programs in his Art of Computer Programming vol 1:
Step 0: Initial idea. Decide vaguely upon the general plan of attack the program will use.
Step 1: Draw a rough sketch of the program in any 'convenient language'. Write the main program first. Refine the sketch by breaking up the tasks between functions.
Step 2: First working program. We now write in a computer language. This step goes in the opposite direction from step 1. We start with the lowest functions first and write the main program last. (Knuth uses the term subroutine, but we shall call the smallest units of a program 'functions' as it is more natural for high-level programming languages in vogue) Knuth also makes a useful suggestion here. After an individual function is coded, we should immediately prepare a complete description of what it does, its calling sequence etc. In other words comment the code!
Step 3: Reexamination. The result of step 2 will be an almost working program. But it might be possible to improve on it. Study each function along with the calls to it. A function can be refined more, or two or more functions can be merged into one. At this point you might even want to scrap the entire program and go back to step 1!
Step 4: Debugging. After step 3 it is time to study the program in the order the computer will inspect it. Debugging can be done by hand or with the aid of a software designed for that purpose. A good start toward effective debugging is the preparation of appropriate test data. The most effective debugging techniques are often built into the program itself. These takes the form of clear readable routines, logging statements etc. Another good debugging practice is to keep a record of all mistakes made. This will be a great help in avoiding similar mistakes in future.
Unlike the rather rigid waterfall model, Knuth suggests an iterative process for software design. Again, unlike the waterfall model, Knuth's method has resulted in great software! I trust this method. I would like to make a few points on how we can practically apply this method using the tools available today.
Knuth suggests to write a rough sketch of the program in any convenient language (Step 1). What can this be? English? Or is it some notation specifically created for software design, like UML? I don't know what was in his mind. But I guess he was not referring to a programming language, as in Step 2 he suggests to write the real program in a computer language. I would suggest to draw the sketch in a computer language itself. Use a dynamically typed, high-level, interpreted language like Python or Scheme. Then Step 2 will be a mere refinement of what we already did in Step 1. We can follow Step 3 and Step 4 without much changes. But if we find some areas that need improved performance, we can just re-write those parts in a systems language like C or C++ and glue them to the main code. Both Python and most implementations of Scheme facilitate gluing together components written in C/C++. As we had already written the same component in the high-level language we know what we had done wrong and how we can do it better. This will naturally result in a component of high quality. This approach works well for projects of any size. We can create high-quality software in less time.
In summary here is my favored method of Software Engineering:
I do not intend to suggest that following a particular methodology in itself will result in good software. There are many things involved. The personal talent and taste of the programmer is an important factor. I would like to conclude this post with a list of skills that always go with a good programmer: