Illusory Follies

There has got to be a better way of doing Software Engineering than what we’re currently slogging through at work. I heard a guy on NPR talking about Software Engineering broadly and mentioning that one of the problems is that there really aren’t any established standards. Why is software so much harder to build than “real stuff” like bridges and buildings and highways. Granted, there are plenty of examples of poorly constructed bridges, buildings, and highways but it seems like there are more standards and better practices applied in those disciplines. Maybe it’s because people’s lives depend on quality design and construction. But then again, much software out there can be “life saving” in similar ways. The software than runs medical systems or military systems is life altering. Maybe that’s why so many of those systems are out-dated technologically simply because no quality product (no matter how much newer) has been successfully developed.

So, what’s the problem and why hasn’t it been solved? Maybe it’s because software is too easy to build (hey, it’s just rearranging rust on a metal plate). It’s easy to release something, hope for the best and just fix problems when they come. A lot easier than constantly taking a bridge apart to fix engineering faults. So maybe the issue is that because software is “so easy” to fix, few people take the time to get it right first when they can almost just as easily get it wrong the first 20 or so times. Why debug a program when the public does this for free by submitting trouble tickets? Of course, some products actually work remarkably well and seem to have been tested and released in an environment that really was focused on quality. But even these products DO have major problems and DO have major releases that fix problems and create new ones.

In building bridges, we have physics to contend with and although it’s difficult to beat and it’s amazing what humans can accomplish to “beat physics”, it’s a relatively simple set of rules that will determine whether the design will work or not. Civil engineers don’t get involved in string theory to build a bridge, they assume a pretty normal set of conditions — the same sort of conditions that allowed Greek builders 2,000+ years ago to build beautiful and functional buildings. In software, we don’t really have a set of rules. We’re trying to make the customer happy and so we have to modify it to meet their needs. Their needs (and desires) change daily so it’s an always moving target. A bridge might be built a certain way but will only be rebuilt when the desire is so great that it warrants the amazingly high cost to redo the work. Software suffers from the problem that every change is just so darn easy — or at least that’s the belief.

Or maybe it has to do with what we’re building. We’re not trying to build something that exists, we’re trying to optimize (automate and streamline) the flow of information. When you’re building a bridge, you’re concerned (usually) with making a platform that things can move across to get to the other side of the bridge. When you build a software program you’re concerned with making stuff move quicker and quicker and better and better to more and more places. It has no solid end-goal. “Better” is pretty much what Version 2.0 of all software products advertise. Better because it moves information faster, or more efficiently to more places or presents it differently (in some abstractly “better” way). It’s all so subjective. If you have an existing software program that runs “slowly” you could rewrite it specifying that it should run “twice as fast”. But this is a silly requirement if current technology allows it to run 200 times as fast. Things change so often and so quickly that what you want at the time of the writing of the software is “the best”. But the best will have changed between the time that the project requirements were written and the time that the software product is released. It seems just as silly to say “200 times as fast” at the beginning of the project (because the engineers assure you it can be done) and meet that goal only to find out that your system is now out of date because it can now be done 400 times as quickly with only minimally more effort.

A bridge has subjective elements of design. It should be “beautiful” or “classic” or something like that. These details are worked through with the architect and the customer who nail down what that means exactly. Functionality is essential though. With regards to these subjective aspects, the customer has to decide how much beauty can be got for how much money and come to a decision on that. There’s going to be a trade-off on the beauty part. In software, although I rarely would call one of my own creations “beautiful”, most of the design ideas are subjective. “We want it better” means little. “We want it faster” means more but doesn’t tell me as an engineer what the final product should look like. “We want it 100 times faster” means a lot but either it’s not feasible to do this at this time, or if it IS feasible, it’s unlikely that the customer really wanted it that much faster (i.e. it was probably the engineers idea to put the number “100 times” into the requirement since they knew that was what WAS feasible). If this is the case, the customer might end up with a product 100 times faster but won’t be happy to find that this puts them behind the curve when the product is done. When you’re talking about a highly integrated product or a “system” of multiple software packages, hardware, and processes, it’s almost impossible to ever deliver a “modern” system.

What’s the answer? I have no idea. It seems like there’s got to be some sort of way structuring a design and development process to focus on the unique aspects of software products. Check this space for more ramblings.