I know it’s a bit harsh to insult you in the title of this post, but the odds are, you really stink at estimation. Even if you know you’re bad at it, you’re probably even worse than you think.

Don’t take it so personally – I’m certainly not saying I’m perfect at it either! But you’re among friends here, so let’s have an honest discussion.

While you will never be perfect at estimation (it is after little more than prediction of the future), you can get better at building and communicating estimates if you remember a few key reasons why you’re so bad at it. You’re still human after all, and so I guarantee you will fall into some of these traps.

Reason #1: You’re doing it alone

Too often, estimates are done alone. Typical scenarios include one person (such as the team lead) estimating an entire project, or a project manager going to a specific developer and asking them for an estimate. One person giving an estimate by themselves is one of the most unreliable ways to give an estimate, because there is no one to check you against common biases. And since all initial estimates tend to be very optimistic, you are almost guaranteed to set unrealistic expectations.

Reason #2: You’re doing it for others

When the team lead gives an estimate on behalf of their team, several problems ensue. First, you are relying solely on the judgment of one person (the team lead), and no matter how smart that person thinks they are, they are susceptible to biases when they go it alone. Second, when the rest of the team is told what the estimate for the project is, they had no voice in it, and therefore, they have no sense of ownership in that estimate. A lack of ownership means they will have no sense of accountability, and they will not go the extra mile to stick to the estimate.

Reason #3: You’re being way too optimistic

Very few of us give pessimistic estimates. For one reason, we want to please the customer. When they say they want something in two weeks, we make mental leaps in how we will build the software to try and get them what they want. Or we discount the likelihood of obstacles, and hope for the best case scenario.

Reason #4: You think you’re an expert

There’s a great chapter in the book “The Black Swan” about the problems with predictions, and one problem described is “the expert problem.” Studies have shown that experts tend to underestimate their own margins of error on predictions, because they are overconfident in their expert status. As software developers, we fall into this trap regularly when we say things like “oh yeah, that will be easy, I’ve done something similar before.” When we do remember that task last time was not as easy as we thought it would be, we often discount those obstacles we encountered as outliers. We assume that “now that we know how to do it”, we won’t encounter those obstacles again, and no other obstacles will arise. We are usually wrong.

Reason #5: You’re using single point estimates

Using single point estimates makes all of these previous reasons worse. Single point estimates encourage you to go with your gut and the first number you think of, or the number that the customer wants to hear. Think how ridiculous it would be someone asked you “How long does it take to get to your house?”, and you answered “Oh, about 25.6735 minutes”. By including all those extra decimal points, you are creating a false impression of accuracy. Single point estimates are the same idea. If I say “it takes 25 minutes to get home”, then you will expect to be there in exactly 25 minutes. But what about traffic, or other variables? A more accurate answer is to say that it takes “25-30 minutes, depending on traffic.”

A solution to estimation biases: Range estimation in Scrum Poker

Using a range instead offers a lot of benefits. By saying “that will take me 2-5 days” instead of “I can get that to you in two days”, you are communicating the uncertainty that is inherent in all software development. The size of the range you provide communicates risk. And it prepares the customer for a more realistic schedule, instead of setting you up for failure when they expect completion on the most optimistic schedule possible.

Combining range estimation with the Agile practices of team estimation is doubly effective. In Agile teams we often play “Scrum Poker.” I’ll avoid the long description of it here, but in short, each team member gets a deck of cards. When you are estimating a story or a task, each team member privately chooses a card (or range of cards) from their deck. When everyone has chosen, they all show their cards at the same time.

Showing the cards at the same time combats a number of the reasons I list above, because you get a true sense of the variety of estimates across the team without introducing the biases of the team lead or architect. Those who picked the lowest or highest estimates are not discounted as outliers, but encouraged to describe to the team why they chose that number. This encourages the team to discuss the idea until they have reached a common vision for the complexity of that story or task.

Agile team estimation is great, but one big problem I still have with it is the reliance on single point estimates. That’s why we use range estimation in our Scrum Poker at OpenSource Connections, and we find it to be more effective. By holding two cards instead of one, each team member is indicating a range estimate and communicating lot more about the risk and complexity that they see in that project.

Next Tuesday (August 10th, from 11am-noon in room Asia5) I’ll be presenting at the Agile 2010 conference on “Building a More Accurate Burndown: Using Range Estimation in Scrum”. If you’re going to be in Orlando, I hope you’ll stop by to hear more! You can also see my slides on slideshare.

Arin Sime is a Senior Consultant with OpenSource Connections, specializing in Agile process consulting, Solr search, and development consulting. You can follow Arin’s tweets at ArinSime

Everybody loves employees and consultants who can get stuff done. Especially if you’ve made a commitment to a client or your boss, it can be very difficult to ask for an extension to that timeline. It’s much easier to ask your team to put in the extra hours to get the job done, even if that means the job won’t be done well. Sometimes leaders will even resort to claiming the product is “done”, when it was never truly done at all, and needs a lot more work.

Your development team will likely respond to your calls with heroic effort. And you might actually make the deadline. But if you do it regularly, your reputation as a leader may actually suffer, and your team will probably burn out unless you can pour on enough accolades and spot-bonuses.

At some point, almost all IT leaders, including myself, have done it. That is unfortunately not much of a secret and it’s why IT teams often develop a bad reputation. The bigger secret is that a lot of developers not only regularly take on the hero role, they actually like it.

This creates a dangerous situation, where IT leaders ask for heroic efforts because it is the path of least resistance. And the development teams respond with grumbles and late nights, but also with the misconception of job security and the expectation that they will receive a lot of praise when they come in on Monday bleary-eyed (but with at least part of the job done).

Why is there so much danger in encouraging your developers to be heroes? Do a google search on “cowboy coders” or “programmer heroes” and you’ll see other blog posts that describe the whys and the pitfalls of reliance on hero development. In short, you’ll get inconsistent and buggy code that may not fit your architecture, won’t meet the customer’s needs, is not very flexible, creates technical debt you will complain about for years, and will probably need to be reworked eventually.

So how do you avoid heroes? As a CIO or technical lead, the tools of Agile offer many solutions:

• Sprint planning – restricting the work that is done to the current sprint helps discourage sudden changes of priority. Most tasks can in reality wait until the next sprint to be added, especially if the customer understands that it means the current sprint has to be put on hold or canceled.
• Daily standups and regular customer interaction – Chronic hero programmers often like to hideaway on their work, and so they can be more easily detected when you are interacting with the customer each day. There is less opportunity for the developer to hide away and not give status updates, and the standups provide the customer more immediate feedback on everything that is being held up because they sent the hero on a distracting quest.
• Team estimation – When the team estimates tasks for a sprint together, before they are assigned to people, that encourages open discussion and debate on how long something will take.
• Continuous Integration – Maintaining rules about daily code check-ins and regular integration and unit testing prevents the hero from going too far away from the mainstream code base, and gives visible signs of progress. When done properly, it also helps to reduce the bugs delivered in the rushed code.

In addition, in our OpenApproach implementation of Agile and Scrum, we use range estimation of tasks. Asking people to give you a single-point estimate of how long something will take encourages them to be overly optimistic. “Oh, I can get that done in about 2 hours.”

Really? What if it’s not as simple as you or the developer thought? Then the only solution is to put in a heroic effort if the timeline must be met.

A simple range estimate will provide a much more realistic view, and encourages a more thorough thought process: “It should only take about 2 hours of actual coding, but I’ll have to switch projects and get that site running again on my machine. Plus, sometimes those java script issues can be really though to debug, so it could take as long as 8 hours of work.”

As an Agile CIO, it’s your job to not only be understanding of your developers when they give you a range estimate, but encourage it. Don’t settle for the heroic answer of “I’ll just put in some extra time this weekend.” Get to the root cause of the problems, and make sure the customer or your boss is aware of the delays earlier rather than later. It will make all of your lives easier, and help you to avoid the pitfalls of programmer heroes.

The Agile CIO is a series of blog posts advising IT leaders on how best to incorporate Agile techniques into their organizations. For more information about OpenSource Connections’ Agile process consulting services, please contact the author at ASime@OpenSourceConnections.com.

Last Friday OpenSource Connections held one of our regular “hackathons”, where OSC developers get together for a day and work on a development project of our choosing, and then at the end of the day, present our work to each other. A hackathon is a fun event where we each get to explore some technology we are interested in, and see how far we can get in one day with it.

This time, we decided to mix up the way we do hackathons. We added in a theme, voting on the best project, celebrity judges, a gift certificate to Best Buy, and even a trophy! The usual elements of time pressure, creativity, fun, caffeine, and a beer at the end of the day were still present.

At the end of the day: Scott Stults was crowned the inaugural OSC “Code Ninja”. Congratulations Scott! In a moment I will describe a little more about the hackathon, but first, let’s all sit back and enjoy this impressive photo of Scott with his prized trophy. Scott is making a feeble attempt to strike the same pose as the karate-guy on the trophy.

For the hackathon, we started the day by meeting at the Nook for breakfast. Then we headed over to a conference room at CitySpace where we set up camp for the day.

The theme of the hackathon was “time”, which meant that you could build any application you wanted, as long as there was some element of time in it. The app had to be built in the 7 or so hours of the hackathon, and judging began at 4:30pm. The applications were judged on three axes: Innovation, Business Utility/Usefulness, and Completeness. Each person voted on all the projects except for their own, giving 1-5 stars for each app on each of the three axes. The project with the highest total number of points was the winner.

To add another dynamic to the voting, we invited “celebrity judges” to join us at 4:30 and they had votes as well. Our celebrity judges were Glenn Wasson and Brian Wheeler. Glenn is an architect at SAIC, and is perhaps most famous for being one of the founders of “The Oracle of Bacon” website. Brian is famous for the Charlottesville Tomorrow website.

Both judges were excellent, and very gracious with their time since we kept them there until nearly 6pm on a Friday night. Thanks Glenn and Brian!

We had a variety of applications built. Caleb worked with Html5 to build a dynamic graph for tracking stocks. Michael worked on extending unit testing frameworks in Visual Studio. Eric built a website where you can send your server logs to, and it will stream out audio sounds that you can listen to which indicate what types of messages are being logged (errors, info, etc). Youssef and I teamed up to build a droid app and deploy it to my phone. We had some cool ideas, but ended up scaling them down to building an Agile standup meeting timer. It made my phone vibrate, which was of course extremely cool.

Despite that coolness, and despite a strong challenge from Eric, the inaugural code ninja is Scott! Scott didn’t just win because he was the only one of us to prepare a powerpoint presentation about his app (that actually lost him a few votes I’m guessing – yuk yuk), but because he made wiki’s exciting! Scott used SPARQL in Semantic Media Wiki SMW+ to show how we could track our projects at OSC and see timelines of who is working on what project.

Thanks to everybody for a great time – I’m looking forward to the next hackathon! I understand that Scott has removed family photos in order to place the trophy in a place of honor at his home, and so I wish the next Code Ninja the best of luck in wresting it away from him.

What they are

Annotation is the process of selecting portions of an original document and adding additional information. These additions are not necessarily saved with the original and are commonly used by editors and scholars to include background citations. In Computer Science, annotations are used in Semantic Web and work collaboration technologies. The Comment function in MS Word is an example of using annotations for collaboration.

Business uses

Aside from these functions I think that there is a whole class of business processes that could benefit from annotation technology. For example, we often copy line items from a Request For Proposal (RFP) and paste them into our proposal. Later, these line items may get pasted into a Product Backlog, Production Readiness Checklist, or ticketing system.

As a programmer, I shudder whenever I see so much copy-and-pasting. If a block of text is important enough to be present in multiple documents, it should be extracted as its own entity and reused. This eliminates the problem of maintaining that block in multiple places and ensures that each instance of the block is accurate. More importantly, it helps you concentrate on adding value rather than repeating yourself (see the DRY principle.)

Categories of technologies

Annotation technologies can be split into two broad categories based on where the annotations are saved: In a server or in the document. Annotations that are saved centrally in a server lend themselves to collaboration because multiple people can update the annotations without having to modify a master document. On the other hand, document control may be more important than collaboration, in which case storing the annotations within the document would be more appropriate.

Example technologies

While Googling for what’s around the web regarding Annotations, I ran across Ian Lumb’s blog.  He’s got a number of excellent posts on the subject.  I also found a number of different technologies listed over at the Semantic Web portal, but the list appears to be dated (several were private projects, there were a couple broken links, and a few of the projects appear to be unmaintained.) However, there were enough working projects there for me to get an idea of what people are doing in the field of Annotations. The promising projects I saw there were:

• Annotea
• Annozilla
• GATE Teamware
• Semantic MediaWiki+
• KIM Semantic Annotation Platform
• Nuxeo Document Management

Annotea is more of a protocol than software, but there were a number of client and server implementations listed (Annozilla was one of them). The Zope server product ZAnnot was a breeze to install on top of a fresh download of Zope, and I was able to get it working with Annozilla pretty quickly. Annozilla itself, though, needs a little TLC before I can incorporate it into a working system.

First of all, Annozilla hasn’t been updated in about nine months and requires FireFox 3.5 (current version is 3.6 and I reinstalled with version 3.5 just to try it out.) Secondly, the annotations themselves are free-form and cannot be constrained or reused (such as with an ontology). This is a well-known problem in the world of tagging content, where search-as-you-type tagging helps you avoid multiple tags that are almost but not exactly the same (such as the tags “annotations” and “annotation”.)

GATE (General Architecture for Text Engineering) is an amazing collection of projects centered around doing things with text. I found only one technology in there that was relevant to annotations, and that was regarding automatic annotations (examining text and feeding annotations into Annotea.) There were a lot of other interesting libraries in there and I hope to check them out later.  Specifically, they announced a Teamware application would be forthcoming which would incorporate annotation technology and workflows.

Likewise, the KIM Semantic Annotation Platform seemed to be oriented more toward automatic annotation rather than streamlining human annotation.

Nuxeo Document Management is a Java-based product very similar to Alfresco.  It has a document preview feature that shows you an HTML version of Word or PDF documents it stores, and also has an annotation module that lets you annotate that preview.  [edit:  When I originally installed it I rushed through and misread some of the documentation.  Stefane Fermigier, the founder of Nuxeo saw this post and corrected me, but I have yet to revisit Nuxeo.  In the interim I've edited this post to remove my misinterpretations.  Hopefully soon I'll be able to follow up with a more in-depth look at Nuxeo.]

Lastly I gave SMW+ a try.  I’ve used Semantic MediaWiki before and I think it has great promise.  I especially like the rich report formats that you can generate with semantic queries (like showing a SIMILE timeline as the result of a time series query.)  There was a long list of extensions to install, but I eventually got to the point where I could copy and paste a whole RFP as a wiki page and annotate it.  The annotation tool was AJAX-based and a little clunky, but once I was done I had a nice report of each annotation in the original document.  It wasn’t clear how to bind an ontology to it, so I have the same complaint with it as I do with Annozilla.

Parting Thoughts

The kind of annotation I want to do seems possible using some of the WYSIWYG editors embedded in most blogs and CMS (like kupu or TinyMCE). A colleague of mine noted that you could simply supply a custom CSS style that would be included in the menu of available styles, but refrain from adding any style changes to it. So in theory any CMS which provides document preview ala Nuxeo should be able to supply an annotation editor which wouldn’t change the original. That to me is the most promising direction.

If you have any experience with similar technology please let me know in the comments.

This is like the state of the union address, except in mid march, and the only thing I’m president of is my current residence. If you have ever studied Science, you know about potential energy vs kinetic, Or maybe a better metaphor for the reality TV generation, is the Swan.

Opensource software on the Microsoft stack has tons of stored potential, even some movement, but it is still left wanting. The evil empire has embraced opensource software, releasing jQuery with Visual Studio, starting codeplex.com and the Codeplex Foundation.

The are tons of abandoned projects or ones that have gone stale (log4net anyone?). Compared to java or even the new kid ruby, we’re lagging behind. We don’t even have a fully managed opensource enterprise web server, compared to Java’s n-th variety of containers to pick and use. Even rails has a built in server.

Of course there is kayak http web server framework and webserver on codeplex, but they’re new, far from mature and their not an industry defacto standard like jboss or tomcat.

Don’t get me wrong there are some amazing opensource projects out there. Gallio, db4o, Subtext, blogengine.net, facebook developer toolkit to name a few.

However, there are gaps in having a full opensource Microsoft stack. We have plenty of unit testing and mock testing libraries, but with NDoc gone, left to SandCastles release schedule, libraries left to rust like log4net wit not even .net 4.0 beta build or silverlight build. CruiseControl.Net is in dire need of revamp and version 2, at the very least it needs some decent competition that isn’t java.

With plenty of single person projects out there who just end up getting burnt out, seemingly stagnated public dialog from the likes of the code plex foundation, its hard to really get developers to rally and get some much needed things done.

The community need some decent leaders, organizers, and some company backing. Most software vendors and clients get a great productivity boost from these projects. It would only make sense to invest in their growth, even pool resources for joint projects.

Organize some hack-a-thons days with some cool prizes for work top-notch work. Even put together a small guild of programmers, just have 20 or so companies pitch in, put their banners on a website and churn out some decent open source projects that everyone can use.

.Net isn’t going away and its time the community and companies invest more into the opensource community instead of letting all that potential go to waste.

Last week was the crucial week on my current Lucene -> Solr project for making our goals.  A lot of work the previous couple of weeks came together.  I wanted to take a couple of minutes and just record some of the little things that I’ve been learning about:

Solr

Sunspot is the up and coming solution for integrating Solr into Ruby on Rails, and fortunately enough, the 1.0 release (followed quickly by 1.0.1!) has just come out last week.  Between acts_as_solr and Sunspot, Sunspot wins hands down for it’s support of a master/slave Solr configurations, embedded Solr for testing, richer indexing semantics, and not being tied to ActiveRecord.  The companion sunspot_rails gem does give wonderful ActiveRecord integration however.

Solr cores are the bees knees!  We’ve built a simple RoR webapp using HTTParty and the Solr API that allows you to perform all the admin functions for cores, and allows you to quickly clone a core for your own nefarious purposes!  Simplifies hacking around with a new schema or configuration without having a local copy of Solr running.  Allows multiple QA environments to potentially share a single Solr infrastructure.

Solr master and slave setup in a single VM.  While pointless from a scaling perspective, it’s a really great way to work out the kinks!  It’s funny to see a slave core polling the same Solr VM its in for updated segments!

JRuby

Doesn’t suck after all.  Actually, maybe I should say that JBoss, when combined with JRuby, means that JBoss doesn’t suck so much.  I had the aforementioned Solr core admin tool bundled up as a WAR file with JRuby, and was able to deploy it to an existing environment that had JBoss installed!  I didn’t have to install ruby on the box, (or JRuby for that matter!)  I just deployed the WAR file and bamn, off to the races.  Ops folks get the JBoss they love, I get the Ruby on Rails that I love.

And on a related note, Warbler was the key to thinking JRuby is cool.  I’d never actually had to package up a RoR app, so Warbler came to the rescue.  And you know what?  It was nice to build a single file that I knew had everything that I needed in it that could be scp’ed around!  And thanks to some cool code in the environment.rb, my app was able to load up the right configuration file for the environment based on an environmental variable set in JBoss.

Virtual Machines

I recently migrated a Linux VPS based RoR + Solr app (see a trend in tech choices )  to a Windows environment.  And to deliever the new Windows environment, I used VirtualBox to host the Windows Vista environment on my Mac laptop.

A couple of notes:

• VirtualBox may not have all the snazzy integration points of Parallels with the host computer like seamless application sharing, but it seems to be much lighter weight.  Starts up quicker, and I don’t get the spinning beach ball of death as much.
• If you are shipping a 11 GB file, you can’t use a 16 GB USB Memory Stick…  Turns out the biggest file is 4 GB.  (Although I never tried formatting the stick as NTFS, maybe that would have allowed a single 11 GB file???)
• Uploading 11 GB to a remote out on the internet server will take a long long long time.  Even on a really fast network. connection.
• If you need to format an external USB hard drive as NTFS on a Mac, it is possible!  Just fire up your trusty Windows Vista image in Parallels, plug the USB drive in, download and install the correct USB drivers so the drive doesn’t show up as a network share mapped to the Mac, and then use the built in reformatting tools!  Warning: This will take a loooong time!
• Lastly, if you are using VirtualBox, and you attempt to create a Windows XP machine, and attach a Windows Vista hard disk image to it, VirtualBox will let you!  And then Windows won’t start.  sigh.

Recap of First Two Weeks post is out: http://whichrubycmsshouldiuse.com/2010/01/12/recap-of-first-two-weeks.html

Day Two with adva-cms is out:  http://whichrubycmsshouldiuse.com/2010/01/08/day-two-with-adva-cms.html