Illusory Follies

My typical work day is a mostly mental exercise. Most of my work falls into these categories:

• Pushing or pulling ideas between various people ( normally we call these conversations but that’s what it feels like)
• Attempting to create abstractions of concrete examples
• Attempting to make concrete examples of abstract ideas
• Designing optimized interactions and processes for defined workflows
• Implementing software applications that meet the design
• Thinking of ways that the implementation can be broken or hacked
• Outlining and documenting all of the above in a way that almost anyone can understand it

I’m a software engineer and that’s what I do. It’s almost entirely a mental exercise. There are very parts of the day where I just click through something or enter data over and over or do anything else where I can zone out. I think a lot of non-techies think that computers are all about following some steps in a process (e.g. “Push that button, then click here, then push the other button.”). I’m sure to offend someone here, but I’ll pick an example of a task that at first requires some mental exercise but then really doesn’t: assembling furniture from Ikea. So, when you first get a box and pull out the pieces you have to apply some thought. Hopefully you skim through the directions, make sure you have the parts that you expect and then start on page 1. You look at the pieces, look at the drawing and start working. Two points to make on this: first, the thought is pretty basic. You’ve been given everything you need to work, you just have to follow the steps. Second, if you buy more than one item (or if your Ikea furniture wears out like ours does) you may find yourself assembling the second unit without even referencing the book. You essentially have to be smart enough to identity what object is in your hand and what parts in front of you fit where on that part.

Software engineering is almost never like this. To be sure there are processes and steps and flow in software. There is a typical software requirements lifecycle that is composed of Planning, Design, Development, Testing, Deployment, and Maintenance [although sometimes the order and categories are described differently, but it’s still basically the same thing] but this just a huge framework that outlines the overall process. In fact, in many software methodologies the lifecycle is intentionally NOT a straightforward process but rather an iterative one. You start with requirements, do some design, write some tests, write code, do more design, refine some requirements, write more code, etc. Most of the process is up to you. Gathering requirements for a software project can be aided by good tools and by good frameworks. However, it’s still up to smart people to ask good questions and translate sometimes misleading responses and then continue to ask questions that flesh out the real desired functionality. Software design has a number of tools and frameworks as well. Just as in real-world architecture, there are numerous design patterns that can be leveraged to achieve a solid and good design but no one but a skilled engineer can make the call on what design pattern is appropriate. We have great modeling tools like UML which can help in developing a [mostly] unambiguous representation of a software component but someone has to construct the diagrams. The coding or implementation itself is probably the least “mental” part of the work. Granted, there’s a large amount of material that a developer needs to understand how to implement a design in code but most of the material is available easily from reference books or more commonly, the Internet. Even so, a developer must spend time understanding the design and ensuring that the code does in fact implement it. It’s difficult to ensure that your implementation is in the line with the requirements so testing is necessary. Testing is hard. I’m not even for a moment suggesting that it is too hard and should be ignored. In fact, you can’t really separate testing from implementation since there’s no way of knowing if what you wrote works without testing. But good, solid testing can be quite difficult. Modern tools are wonderful for testing — it’s easy to write unit tests for code to verify that code does what it’s supposed to do. The tools make life easier but there’s still an enormous amount of thought that has to go into testing. Edge cases and complex use case scenarios are common; it’s often very hard to have good tests that adequately cover all the requirements rigorously. Often software is written that supports an infinite variability of input. In order to write adequate tests, a developer must have intimate knowledge of the requirements but also the language, frameworks, operating system and other parameters that are part of the software environment. The tests themselves are easy to implement. It’s knowing what we should test that’s hard. Without even going into deployment, maintenance, and other related tasks, it’s easy to see that most of this process is a thinking process. In fact, really good software can be nearly complete before any software has been written.

My point of all this is to indicate that software is a lot of brain power. And more than just brain power, it requires a lot of creativity. A lot of software has been written in the last 60 years. New software isn’t just about reinventing existing software but making things work better than before. Perhaps this means finding a creative way to tweak more performance out of the same hardware, or maybe constructing a more efficient interface that humans can spend less time learning and more time using, or maybe finding a way to make disparate systems talk together smoothly and correctly. All of these are valuable but often we’re not trying to find a solution but to find the best solution. Requirements have many non-functional elements that describe constraints on the system. We might be building a system that searches an airline reservation system for available seats but our constraint might be that we need to retrieve this data from the current system in less than 1 second. That part gets tricky.

This wears me out mentally. I find that by the end of the day I rarely want to spend time on the computer (unless it’s doing something mindless). In fact, even on the weekends I greatly prefer activities that are manual. I enjoy do-it-yourself projects around the house that involve punching holes in walls and cutting down trees. That sort of thing is easy — it takes some minimal thought and care but it’s basically doing stuff not thinking about stuff. I rarely do work on evenings or weekends that’s very creative — I’ve struggled just to write on this blog let alone writing out some stories that I’ve had floating in my head for years. Any project around the house that requires a good amount of planning or preparation tends to not happen because my brain sees the looming creative task and tries to shut down.

So finally, one observation, and one idea.

In the last several weeks I’ve transitioned in my full-time work to doing very repetitive (even boring) work that’s much more akin to pushing buttons and clicking the mouse. There’s a large backlog of system configuration tasks that I’ve been working through. The work is easy in one sense. Almost all of it is well-defined and easily understood. It just takes time. Sometimes I have to chase down problems and troubleshoot things that aren’t working, but it’s mostly clicking, typing, waiting, and repeating. What I’ve observed is that after about 3 days of “time off” from the creative process of software engineering I had a large boost in creativity. During the day, in the evening, on the weekend — I suddenly was interested in doing creative tasks. My brain had realized that it wasn’t getting any action and started asking for attention. I spent a few late nights working on some tasks that I’ve wanted to do for a long time (including some software development for a personal project that’s been on the back burner for literally years). All of it felt not only good, but great. I felt like I had time to think through things and although it’s a little subjective, I think that the quality of my work was superior than it normally is. I’ve had more “ah ha” moments and more “outside the box” solutions than I normally do. It’s been wonderful.

This probably isn’t too much of a surprise to anyone who thinks about it. I’m sure there are studies and articles on the subject (I’ve seen some). However, I’m interested in the idea that there may be optimal ways of mixing work for professions that require large amounts of creativity. For example, it might make sense that the best activity for a group of software engineers isn’t to have them spend an extra day each week on a project of their choosing (although I think it’s better than nothing) but rather to have them spend time working on running cable in the corporate offices or doing rack wiring or adding components to computer cases. These are all worth-while activities and although they take some skill, most technically inclined people find these things to be fairly simple mentally (although not always physically). A company doesn’t normally pay the same salary to people who do these tasks, but I think it could provide real value to the company in the long run. By turning off the creative process for a little bit, software engineers could [perhaps] be much more creative when they come back to their regular work. I’m not sure if there have been attempts to do things like this in a company that employees software engineers, but I’m curious to hear from my readers on whether they think that the idea has any merit. Anecdotal evidence as well as real studies on the subject are all welcome!