How Much Do You Sweat The Details?

I tend to try to make parallels between my work and whatever else is going on in my life, even when they’re inappropriate.  Case in point:  I got married at the end of July (yay!) which meant that my wife and I spent a large percentage of our free time for the prior year or so planning the wedding (d’oh!).  As anyone who’s gone through that knows, it’s kind of a never-ending list of things to do and, more specifically, of decisions to be made.  Not only that, but each decision is about something that’s costing you hundreds or even thousands of dollars.  Normally, when making decisions involving large sums of money, I tend to be pretty careful; I read product reviews online, I comparison shop, etc.  But that all takes time, and when you have to make several such decisions per day for months and months, you just can’t put that level of effort into each and every decision.  In the end, if you don’t want to go crazy you have to just accept that you’re going to make some mistakes, and that some of those decisions will turn out to be sub-optimal.

So what in the world does that have to do with software development?  Perhaps this is a stretch, but as I reflected on that experience I started to realize that most development work is a similar never-ending series of decisions, each of which could potentially have a large long-term impact if it’s done wrong, or at least could require a large amount of effort to fix.  What do I name this method?  Where do I put this class?  Do I store this value as two booleans or a tri-state enum?  Do I add this extra field to the object, or should it logically be its own object/row in the database?  Do I mock this out in testing, or use the production version of the class?  Do I use a factory to build this or just a normal constructor?  Do I make this method public, protected, or private?  Is this class properly decomposed?  Should this class really be split into multiple pieces?  On and on it goes.  Even if all those decisions are relatively neutral in the sense that you can make any of them work right now, each one of those decisions can have a large downstream impact on your ability to test, maintain, extend, and understand the code base.  One bad decision could literally be the difference between making a future feature take 15 minutes and making it take 15 hours, or between making debugging easy or turning it into a multi-day expedition.

One interesting complication with software, though, is that the time to make the decision itself is a cost, and with software development time is usually everything (assuming you take correctness for granted, i.e. that you will spend whatever time is necessary to test and debug and ensure correctness).  So even though the name of a method is critical to people’s understanding of it, their ability to read code that calls it and their ability to discover it, how much time do you spend making the decision?  Is the name that you come up with after fifteen minutes of deliberation likely to, on average, save you fifteen minutes of development time compared to the name you came up with right away?  Well, it depends on how bad your initial name is, and how much better the other name could be.

As developers, we all make hundreds of those meta-decisions a day:  do I make a snap judgment and keep going, or do I spend a bunch of time thinking or talking or prototyping?  It’s almost like you need to draw a three-dimensional curve, plotting, for each amount of time that could be spent deliberating, what the the probability is for different kinds of time savings over, say, the next five years:  if we spend 15 minutes on this, there’s a 5% chance we’ll come up with a decision that’s worse by 1 hour over five years and a 50% chance we’ll find something that’s 1 hour better and a 45% chance we’ll come up with something exactly as good, but if we spend 30 minutes on it we now have a 60% chance of saving 1 hour . . . then all you have to do is do a little calculus to find the amount of deliberation time that maximizes the expected savings – that deliberation time . . . and then throw in some extra variables to account for the potential that your probability graph itself has a certain probability of being wrong in certain ways . . . not to mention a discount factor on the value of time now versus the value of your time five years from now to represent the importance of shipping on time . . .

Clearly, I’ve thought about this way too much.

The same reasoning applies to decisions beyond just the “do we spend time talking about this” decision.  Do I just hack this in?  How much will it cost us down the line?  Do I refactor this?  Will the time spent refactoring be paid back in the long run?

By now, you’re probably wondering where I’m going with this.  Honestly, as much as I wish I did, I don’t have an answer as to how to determine when to spend the time and when to just hack away.  My point is merely that it’s a subject worth thinking about.  Most of us have an internal dial that’s set one direction or the other, where we’re either biased to just hack away or we’re biased to deliberate a ton and refactor a ton and try to make everything perfect.  And I think much of the time we take for granted that our dial is set correctly, and that we’re making the right tradeoffs.  If we’re just hacking away, it’s obviously because we just need to get this done and this is probably close enough to optimal anyway, so why bother talking about it?  And if we’re refactoring or talking about how to do things, it’s because it’s critical to contain complexity and avoid technical debt so it’s really important to get things as right as we can.

Every so often, it’s good to do a little meta-cognition and ask yourself if you have the dial cranked to the appropriate place given your project and all the other constraints.  Are you bounding ahead just because you want to, when some extra time and discussion would lead to a better decision?  Or are you overthinking decisions that are so speculative or so inconsequential that you should just make the decision and move on?


4 Comments on “How Much Do You Sweat The Details?”

  1. Alex Naddaff says:

    I was browsing through my email and my Google alert picked this article up. Whenever I see one of our development posts I read it because it gives me a chance to gain insight into people that I really admire. This is not exception. Reading this article makes me think about one of the most popular issues that Guidewire consulting focuses on with our customers. We typically are in a position of explaining our early estimates and how we know the effort and duration needed for a project. One of the biggest factors that drives actual effort and duration is time to decision. So we assess the size of a company and both their desire and their ability to come to decision and how much a company will take advantage of our repetitive project experience. Sometimes customers openly welcome a drive towards efficiency and at other times there is not such a warm reception. In the end this factor has a much bigger impact on implementation and timeline and effort than the capability of the technology. There has to be an effective balance between timely decisions and deliberate decisions. Here are some considerations when thinking about how to drive towards efficiency when making decisions.
    • Do your best to make sure that all members of the team are empowered and that only decisions that get improved by collaboration are expanded to others on the team. Basically avoid posturing between parties when making decisions. If the single empowered person on the team will make most decisions just as well as if 4 people debated the issue let the one team member make the decision. Good people learn from mistakes and if they make a mistake it can be corrected!
    • If a particular issue has more than one almost right answer and the differences are minor in value than pick any one of the almost right answers and move forward.
    • Make sure that the issues that receive the most time are the issues that can have the most impact on Value
    This all seems so basic and you say ok so tell me something that I don’t already know “Sherlock”. Well while this seems basic the common sense points above are not always observed behaviors when working on an implementation. Heck many times I fall into the trap of worrying about the victory of debate instead of the value of the victory of the debate. This goes a step further to look at why Guidewire emphasizes the use of Agile on projects. Agile techniques allow us to encourage debate time for valuable topics as opposed to all topics. So you may say how does that make sense? Well in my experience many projects that follow Waterfall methodologies spend time defining requirements that will never be implemented because they either are not affordable or not valuable enough to want to afford when comparing to other areas of opportunity. So if you can avoid debate by isolating the work you will do vs. the work you will theoretically do and you will end up saving significant time through avoidance of work for the team.
    So maybe this is ringing true to your own experiences. Well now you say how can you make this happen? You can’t completely solve for this in my opinion. You can reduce wasted think time by having a couple of things.
    • One is a clear focus which in our terms means clear scope
    • The second is connected sponsors. The connected sponsor can provide guidance and encouragement when the clear focus sometimes loses its target.
    • The other item is an experienced team. Experience is an amazing contributor to efficient decision making
    Now to be completely honest I don’t envy the people solving the development issue that is discussed in the article. Since our developers are developing software from scratch the number of possibilities is endless when they are faced with decisions. I marvel at how well the Guidewire development team does when faced with decision time and collaboration. Our implementation projects are slightly easier. For the most part we implement the same product over and over again. So the real challenge for the consulting force is to learn how to effectively listen to new customers and to differentiate the items that truly affect a customer’s uniqueness from the items that have a marginal impact on a customer’s competitive advantage. The best warning sign of a weak item is when a team member says “we do it that way because we always did it that way”. Really is that something that you want to rely on. So an early efficiency tool is to let the team know that they cannot ever justify a point by saying “we do it that way because we always did it that way”. Once understood the best consultants can help customer teams to benefit from the multiple implementations and to focus their time on value added decisions.
    But alas this is not all magic and the success of any single project is based on a combination of the quality of the implementation team, the quality of the sponsors and the desire to achieve efficiency. I think this is true when making personal decisions like which washing machine to buy and when making professional decision like which integration to include in phase one scope.
    I do have one more piece of advice for the Newlywed. I am married 25 years. I have learned a good lesson. It is better if my wife makes all of the important decisions at home. She is really in charge and any time I spend pretending otherwise is really inefficient.  Besides she is right the many more times than she is wrong.
    Having said all of this I am wondering if my rambling actually added any value to this topic. Well who knows maybe it will be a catalyst for more thoughts?

    Cheers from Alex

  2. dw says:

    …and how do you know when thinking about the
    way you develop (as opposed to actually developing)
    is the best use of your time? 🙂

    David

  3. Minh Vu says:

    🙂 If I get into Alan’s head, I probably won’t find my way out. This is an interesting thought which holds many implications. I doubt that complex decision like this should be made at any regular basis. This is like socialist government trying to set price for thousand of products on the market, no ways they gonna know how much a product will cost until someone pay for it. Upfront, you cannot tell how much time you can save, nor can you predict the possibility of such saving. We need a simple idea like capitalism, let supply and demand decide the price and in turn allocate scared resources efficiently.

    My personal take on this is to go with your first instinct, if you think there is a chance that your first instinct may be different from others, double check with them. If it’s clear in your mind and that of your collages, go with it, if it would not works, everyone will understand why and will fix it. If you don’t even have the first instinct, go ask Alan K. 🙂

  4. jeffgrigg says:

    Yes, these are good questions to ask; there is a trade-off between current and future costs. I find that with Test-Driven Development, there are substantial benefits writing the simplest and most maintainable code now, knowing that if a more sophisticated solution is needed later, we can always introduce the change at that time, supported by the tests.

    I find it helpful to always improve the code that I happen to be working on. To achieve the greatest benefit, we need to have the most frequently maintained code be highly maintainable. I would argue that if we are always making small improvements to whatever code we happen to be working on at the time, then without putting any effort into planning, we will find ourselves investing most of our effort where it is most highly needed. Add a little planning to that, and very good things can happen.

    Re: “We have always done it that way.”
    There may be a value to respecting tradition. There may even have been a good reason for doing things that way, at one time. But I think that if we want things to improve, it may be time for a change.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s