I know it’s a bit harsh to insult you in the title of this post, but the odds are, you really stink at estimation. Even if you know you’re bad at it, you’re probably even worse than you think.

Don’t take it so personally – I’m certainly not saying I’m perfect at it either! But you’re among friends here, so let’s have an honest discussion.

While you will never be perfect at estimation (it is after little more than prediction of the future), you can get better at building and communicating estimates if you remember a few key reasons why you’re so bad at it. You’re still human after all, and so I guarantee you will fall into some of these traps.

Reason #1: You’re doing it alone

Too often, estimates are done alone. Typical scenarios include one person (such as the team lead) estimating an entire project, or a project manager going to a specific developer and asking them for an estimate. One person giving an estimate by themselves is one of the most unreliable ways to give an estimate, because there is no one to check you against common biases. And since all initial estimates tend to be very optimistic, you are almost guaranteed to set unrealistic expectations.

Reason #2: You’re doing it for others

When the team lead gives an estimate on behalf of their team, several problems ensue. First, you are relying solely on the judgment of one person (the team lead), and no matter how smart that person thinks they are, they are susceptible to biases when they go it alone. Second, when the rest of the team is told what the estimate for the project is, they had no voice in it, and therefore, they have no sense of ownership in that estimate. A lack of ownership means they will have no sense of accountability, and they will not go the extra mile to stick to the estimate.

Reason #3: You’re being way too optimistic

Very few of us give pessimistic estimates. For one reason, we want to please the customer. When they say they want something in two weeks, we make mental leaps in how we will build the software to try and get them what they want. Or we discount the likelihood of obstacles, and hope for the best case scenario.

Reason #4: You think you’re an expert

There’s a great chapter in the book “The Black Swan” about the problems with predictions, and one problem described is “the expert problem.” Studies have shown that experts tend to underestimate their own margins of error on predictions, because they are overconfident in their expert status. As software developers, we fall into this trap regularly when we say things like “oh yeah, that will be easy, I’ve done something similar before.” When we do remember that task last time was not as easy as we thought it would be, we often discount those obstacles we encountered as outliers. We assume that “now that we know how to do it”, we won’t encounter those obstacles again, and no other obstacles will arise. We are usually wrong.

Reason #5: You’re using single point estimates

Using single point estimates makes all of these previous reasons worse. Single point estimates encourage you to go with your gut and the first number you think of, or the number that the customer wants to hear. Think how ridiculous it would be someone asked you “How long does it take to get to your house?”, and you answered “Oh, about 25.6735 minutes”. By including all those extra decimal points, you are creating a false impression of accuracy. Single point estimates are the same idea. If I say “it takes 25 minutes to get home”, then you will expect to be there in exactly 25 minutes. But what about traffic, or other variables? A more accurate answer is to say that it takes “25-30 minutes, depending on traffic.”

A solution to estimation biases: Range estimation in Scrum Poker

Using a range instead offers a lot of benefits. By saying “that will take me 2-5 days” instead of “I can get that to you in two days”, you are communicating the uncertainty that is inherent in all software development. The size of the range you provide communicates risk. And it prepares the customer for a more realistic schedule, instead of setting you up for failure when they expect completion on the most optimistic schedule possible.

Combining range estimation with the Agile practices of team estimation is doubly effective. In Agile teams we often play “Scrum Poker.” I’ll avoid the long description of it here, but in short, each team member gets a deck of cards. When you are estimating a story or a task, each team member privately chooses a card (or range of cards) from their deck. When everyone has chosen, they all show their cards at the same time.

Showing the cards at the same time combats a number of the reasons I list above, because you get a true sense of the variety of estimates across the team without introducing the biases of the team lead or architect. Those who picked the lowest or highest estimates are not discounted as outliers, but encouraged to describe to the team why they chose that number. This encourages the team to discuss the idea until they have reached a common vision for the complexity of that story or task.

Agile team estimation is great, but one big problem I still have with it is the reliance on single point estimates. That’s why we use range estimation in our Scrum Poker at OpenSource Connections, and we find it to be more effective. By holding two cards instead of one, each team member is indicating a range estimate and communicating lot more about the risk and complexity that they see in that project.

Next Tuesday (August 10th, from 11am-noon in room Asia5) I’ll be presenting at the Agile 2010 conference on “Building a More Accurate Burndown: Using Range Estimation in Scrum”. If you’re going to be in Orlando, I hope you’ll stop by to hear more! You can also see my slides on slideshare.

Arin Sime is a Senior Consultant with OpenSource Connections, specializing in Agile process consulting, Solr search, and development consulting. You can follow Arin’s tweets at ArinSime

I’m looking forward to speaking next Tuesday night at AgileCville about range estimation in Scrum. It’s been a while since I’ve been able to attend AgileCville meetings regularly, so I’m excited to reconnect with everyone. I hope to see you there!

AgileCville
Date: Tuesday, June 15th, 2010
Time: 6 PM to 7:30 PM
Venue: OpenSpace
455 Second Street SE, Suite 100
Charlottesville, VA 22902

I am looking forward to presenting at Agile 2010 this year, and I was very pleased to find out that my session proposal on Range Estimation in Scrum has been accepted. Here is info about the session, and I hope to see you at the conference in August!

Building a More Accurate Burndown: Using Range Estimation in Scrum

Traditional Scrum burndowns are based on single point estimates of how long a task will take. However, single point estimates are inherently faulty and inaccurate, and they encourage underestimation. Learn how to incorporate range based estimation techniques into your Scrum burndown, and better communicate to your boss or clients what a project is really going to take. Arin will back up this thesis with academic and industry research, real world examples, and an engaging presentation style. Participants will leave with concrete tips & templates for using range estimates in their projects.

Process/Mechanics

The first part of the session will focus on the pitfalls of traditional estimation techniques, in particular single point estimates. This will involve audience interaction as well as citing industry and academic research. Next the session will move into a discussion of range estimation and how that leads to better accuracy. Then I will present specific advice for how to use range estimates in common Scrum practices like Scrum poker and the burndown. Along the way I will show examples of how our company has been using this practice with clients over the last year, and some best practices that have come out of it. The techniques will be fairly simple and easy to adopt, but provide power results in estimating more accurately and better communication with managers and clients.

Potential pitfalls and best practices of range estimation will also be discussed. For example, some managers are reluctant to accept range estimates, but they can be convinced when a demonstration is made to them of how it better communicates risk and allows them to make better financial decisions on the viability of a project.

This topic started for me with a paper I wrote in my Masters in Management of IT program in 2009 at the University of Virginia, and now has become regular practice at OpenSource Connections. I am also pursuing further research on the topic with two professors from the University of Virginia (Professors Ryan Nelson and Mike Morris of the McIntire School of Commerce). Any preliminary findings or papers from that research will also be woven into the presentation.

I have proposed this as a 60 minute talk, but it could also be made into a 90 minute Tutorial with relative ease. In that scenario, I would bring more hands on activities to encourage participants to really see the value of range estimation in their projects. I have experience with those types of sessions from helping to teach corporate education classes at Virginia Commonwealth University, and I would use that experience to make sure things are kept interesting.

It’s my intention for this session to take the relatively easy-to-use but underutilized practice of range estimation, and make it so compelling that participants will want to apply it right away on their projects. This will be done through a combination of good research, real world examples, engaging presentation style, and practical tips.

Learning outcomes

* Why single point estimates lead to underestimation
* Why range estimation reduces natural biases
* How range estimates better communicate a project’s impact
* How to use range estimation in Scrum Poker
* How to incorporate range estimates into a Scrum burndown
* How range estimates impact project decision making

For those registered on the Agile 2010 site, you can also see this session here.

So, is James Bach (@jamesmarcusbach) the bad boy of testing?

I flew up to Boston on Monday to lead some workshops on Continuous Integration. I checked into my room at the Hyatt and then went downstairs to see who was around. I ran into a couple of speakers milling about, and eventually joined one of them, and we headed over to the MIT Press bookstore, me to look for my Solr book. I wasn’t too sure of the name of the other speaker I was with (I asked once, but couldn’t quite remember what it was…). So we got to the book shop, I ask my fellow speaker again: James Bach. The name I was familiar with, but couldn’t quite place it… I ended up buying Parentonomics, and then we go for coffee.

So, over coffee, he asks me about what my topic is, and I gave him the brief summary of my two CI related workshops. Wow.. Little did I realize that I was sitting with the guy who rails against the “fetish” that Agile folks have for automated testing! That his entire approach to “testing” is to use skilled, motivated folks who do “sapient testing”. And I’m the guy who’s selling an approach that REQUIRES automated tests! That encourages expanding the use of automated testing!

He actually walked me through a process of talking about how to “think like a tester”, and it was really great mini-workshop.. He definitely subscribes to the socratic approach, and believes in his message, I was sweating at the end of it! That chat probably sparked more ideas in less time over that coffee then anything else this week. I also heard a lot of ideas and phrases that were echoed in Michael Bolton’s keynote later on in the week. Clearly a lot of collaboration between the two!

Probably the biggest idea that James and chatted about was the idea that automated tests really aren’t automated tests, they are automated checks. They verify that the expected behavior of the code was met. His argument that if you want to do testing, real testing, then computers, automated processes can’t meet that need, only people can.

Now, I don’t know if I believe that is completely true, but I am very aware that the “manual testing” where long test scripts written as Word documents are executed by human beings by hand are really a waste of human potential. And that those test scripts are really, to use James terms, “check scripts” because the people are not using any creativity! In fact, a lot of my interest in CI comes from the idea that people should not do monkey testing, that machines can do it much better, and my frustation with the perception that testing is a low value activity and can be easily shipped off to low skilled folks.

I think that this shift away from the term “test” for automated tests is actually happening in many places. In the Ruby world, we have libraries like Shoulda that are moving from using words like assert to other words like should. A Cucumber test really shows how controlled the space that a test needs to be to work well in an automated fashion:

Scenario: See all vendors
Given I am logged in as a user in the administrator role
And There are 3 vendors
When I go to the manage vendors page
Then I should see the first 3 vendor names

So while I don’t know if I am bought in on the idea that only people can do “testing”, and machines can only do “checking”. Tools like Heckle try to simulate aspects of what a human can do. While not suggesting that we can automate the “does my website look okay after someone changed the CSS” type of work today, in the future our automated testing will be more capable then just “checking” because we will move beyond the very constrained tests we have today to ones that mimicing the richness of the simulators that Airline Pilots use. Instead of testing the training given to pilots, we’ll be testing the robustness of software via simulations!

At any rate, James Bach, while taking a rather provocative approach to sharing his ideas, does subscribe to my favorite bullet in the Agile Manifesto: Individuals and interactions over processes and tools.

Here is him giving a great presentation with the subversive title of “How to Fake a Project” that was incredible entertaining, and also quite thought provoking:

What do you think? Automated testing is a fetish of the Agile community?

From the Meetup site:

We’ll look at the thriving Ruby ecosystem that has grown up around integrating with Solr. From Ruby gems that integrate with Solr like solrb and rsolr, to general search solutions like acts_as_solr and sunspot. We’ll also look at a complete “shrink wrapped” catalog solution for Solr using BlacklightOPAC.

You’ll lean the basics of getting started with Solr, and an understanding of what Ruby solutions are available to simplifying adding great search to your site!

As usual, food and beverages will be provided.

I’m very pleased to be speaking to Richmond SPIN (Software Process Improvement Network) next Wednesday October 14th, at 6:00 pm at Anthem BCBS in Richmond.  The address is 2015 Staples Mill Road in Richmond.  You can find more details and register online at:

http://www.richmondspin.org/home22222

Here’s the brief on my talk, which is based on my recent talk at Agile 2009.  I hope to see you there!

“Strategies for Persuasion on Adopting Agile”
By Arin Sime, Senior Consultant with OpenSource Connections

Do you want to know why Agile is sometimes a preferred methodology over a Waterfall method?  Arin will present 12 different methods for convincing a “traditional client” to use an Agile project plan. Based upon his own experiences with a wide variety of project styles, he will also present several definitions of each of the environments, well as results gathered from a survey on how to sell Agile. Some stories and best practices from other sources will also be covered.

Arin Sime’s presentation last week at Agile 2009 on how to sell a traditional client on an agile plan made a good impression, apparently.  First, his presentation was featured on the front page of SlideShare.

Now, he’s received an excellent writeup from Adam Goucher (we were pointed to Adam’s writeup from here).  Many thanks to Adam for the kind words about Arin’s writeup!

Arin’s next speaking engagement is at the University of Virginia’s EdUI conference on September 22, covering the Facebook API.  He’d love to see you there!

This fall I will present at the following events:

• October 3rd – Richmong Code  Camp
• October 8th – Roanoke Valley .NET User Group
• October 15th – Charlottesville .NET User Group

Hope to see y’all there. Thank You