Software Is Not A Term PaperPosted: May 19, 2008
I’ve been thinking a lot lately, and having lots of discussions, about what our development process should be for the next release of PolicyCenter. I’ve taken the potentially-controversial position that date targets and deadlines are detrimental to developing a software project, and it’s a position that I feel many non-developers don’t really understand.
The intuitive position that I’m arguing against is what I think of as the term paper philosophy; having the deadline there forces you to get it together to crank through the thing, whereas without any firm deadline you’d keep working on it forever (or keep putting off working on it), and moreover you wouldn’t really work very hard on it either. Some people need that deadline pressure and the rush of adrenaline that accompanies the fear of not getting things done, but even people that don’t thrive on it often benefit from having a deadline to focus their efforts.
What I’m saying is that that theory doesn’t apply very well to long-lived software projects. It might apply to toy projects in college that you never work on again, but the two fundamental differences between software and a term paper (or nearly anything else you try to build on a deadline) are that you keep having to work on the software project and there are far more corners to cut in software development.
It’s hard to over-emphasize the significance of that statement, and I think that non-developers really don’t have a very good analog of what that means. There just aren’t that many other fields or endeavors in which something is built up successively over years or even decades of work. There are even fewer where you can release an intermediate product off of that tree periodically that people will actually use (and expect support and upgrades for), and fewer still where you can “successfully” cut as many corners as you can in software development without having things completely fall apart in the short term. The only analogy I can think of might be construction work, where if you rush the foundations of a building then the future stories won’t fare so well, but it might not be obvious that’s the case until you start on those future stories or until an earthquake or some other disaster hits. But most sorts of projects or tasks that people do are kind of one-time things; if you keep using them, you use them as they were when they were finished rather than attempting to build more and bigger things on top of your initial work. If you rush building a chair, maybe the chair’s a little off-balance, but it’s not going to affect future chairs that you build. And of course, that’s all combined with the fact that software estimation for anything other than the immediate task at hand is basically impossible, meaning it’s hard to avoid deadline pressure by just estimating accurately, since you’re always trying to estimate something that’s fundamentally pretty unknowable.
What sorts of corners can you cut that will make the product look “done” but will come back to haunt you later? Here are a few examples:
- No or incomplete tests
- Ignoring or not considering edge cases or error conditions
- Ignoring or not considering feature interactions
- Poor UI design
- Poor datamodelling
- Poor API design
- Improper encapsulation, decomposition, or otherwise messy code
- Inconsistency between different areas of the code
- No, poor, or inaccurate documentation and specs
All of those things will inevitably bite you later down the line; some of them will bite your customers too, some of them will make future changes nearly impossible or far more difficult, some will cause your development effort to grind to a halt in the future, and others will be fixable but just require more work to do later than they would have to do right initially. Nearly all of them will eventually slow down development, putting even more pressure on future deadlines and leading to a vicious cycle where even more short-term hacks enter the system.
So let’s imagine a real-world scenario. You’ve got what you think is 3 months worth of work to do, but one month in less than 1/3 of it (as far as you can tell) is done. What do you think will happen? People will feel behind, and they’ll pull whatever strings they can to try to try to make up the lost time, either consciously or unconsciously. The same thing will happen with more short-term deadlines if you ask someone to try to get something done by the end of the week: they’ll find a way to try to make it happen, even if they should really take more time. In the long run, you’re going to pay dearly for those sorts of decisions.
So what’s the solution? I’m honestly not sure, though I have some theories I want to try out. There are two obvious problems with not having deadlines at all. First of all, some people really do need deadline pressure to get things done, and others simply will go on working on less important things forever in that case. My personal opinion is that those issues can be dealt with without formal deadlines, by constantly re-estimating and re-prioritizing work instead. The second problem is harder to escape, at least if you’re selling your products to people: everyone you’re selling to expects you to tell them what you’ll have done and when it will be done, so it’s inescapable that you’ll end up making some level of date-based feature commitments that then have the potential to put deadline pressure on your team and cause them to cut corners. Some types of agile methodologies can hopefully mitigate those problems for at least part of your development cycle, but most of those work better for consulting-style projects that actually have an end date at some point, and I honestly don’t feel like it’s a solved problem yet for our type of development work. As a company, we do a very good job of (and take immense pride in) standing behind the commitments to our customers, but we can definitely do a better job internally of mitigating the pressure that such commitments put on the development team. I’m hopeful we’ll be trying some new things out in the next release cycle; whether those experiments work out or not, hopefully I’ll get a chance to write about them and tell everyone whether they worked out and why.