So, is James Bach (@jamesmarcusbach) the bad boy of testing?

I flew up to Boston on Monday to lead some workshops on Continuous Integration. I checked into my room at the Hyatt and then went downstairs to see who was around. I ran into a couple of speakers milling about, and eventually joined one of them, and we headed over to the MIT Press bookstore, me to look for my Solr book. I wasn’t too sure of the name of the other speaker I was with (I asked once, but couldn’t quite remember what it was…). So we got to the book shop, I ask my fellow speaker again: James Bach. The name I was familiar with, but couldn’t quite place it… I ended up buying Parentonomics, and then we go for coffee.

So, over coffee, he asks me about what my topic is, and I gave him the brief summary of my two CI related workshops. Wow.. Little did I realize that I was sitting with the guy who rails against the “fetish” that Agile folks have for automated testing! That his entire approach to “testing” is to use skilled, motivated folks who do “sapient testing”. And I’m the guy who’s selling an approach that REQUIRES automated tests! That encourages expanding the use of automated testing!

He actually walked me through a process of talking about how to “think like a tester”, and it was really great mini-workshop.. He definitely subscribes to the socratic approach, and believes in his message, I was sweating at the end of it! That chat probably sparked more ideas in less time over that coffee then anything else this week. I also heard a lot of ideas and phrases that were echoed in Michael Bolton’s keynote later on in the week. Clearly a lot of collaboration between the two!

Probably the biggest idea that James and chatted about was the idea that automated tests really aren’t automated tests, they are automated checks. They verify that the expected behavior of the code was met. His argument that if you want to do testing, real testing, then computers, automated processes can’t meet that need, only people can.

Now, I don’t know if I believe that is completely true, but I am very aware that the “manual testing” where long test scripts written as Word documents are executed by human beings by hand are really a waste of human potential. And that those test scripts are really, to use James terms, “check scripts” because the people are not using any creativity! In fact, a lot of my interest in CI comes from the idea that people should not do monkey testing, that machines can do it much better, and my frustation with the perception that testing is a low value activity and can be easily shipped off to low skilled folks.

I think that this shift away from the term “test” for automated tests is actually happening in many places. In the Ruby world, we have libraries like Shoulda that are moving from using words like assert to other words like should. A Cucumber test really shows how controlled the space that a test needs to be to work well in an automated fashion:

Scenario: See all vendors
Given I am logged in as a user in the administrator role
And There are 3 vendors
When I go to the manage vendors page
Then I should see the first 3 vendor names

So while I don’t know if I am bought in on the idea that only people can do “testing”, and machines can only do “checking”. Tools like Heckle try to simulate aspects of what a human can do. While not suggesting that we can automate the “does my website look okay after someone changed the CSS” type of work today, in the future our automated testing will be more capable then just “checking” because we will move beyond the very constrained tests we have today to ones that mimicing the richness of the simulators that Airline Pilots use. Instead of testing the training given to pilots, we’ll be testing the robustness of software via simulations!

At any rate, James Bach, while taking a rather provocative approach to sharing his ideas, does subscribe to my favorite bullet in the Agile Manifesto: Individuals and interactions over processes and tools.

Here is him giving a great presentation with the subversive title of “How to Fake a Project” that was incredible entertaining, and also quite thought provoking:

What do you think? Automated testing is a fetish of the Agile community?

Why do we need to sell Agile?

For those of us who know and love Agile, the benefits of Agile seem obvious.  It can be frustrating when someone doesn’t inherently see those benefits, and we sometimes forget that we need to sell our clients or our managers on our process.

For some perspective on the gut reaction many people have to Agile, consider the following quote.  John Zachman penned a famous paper on software architecture design in the 1980’s, where he said:

“Some kind of structure (or architecture) is imperative because decentralization without structure is chaos.” – J.A. Zachman, 1987, “A framework for information systems architecture”

As practitioners of Agile, we need to remember that someone’s first reaction may be that we are engaging in chaos, and so we need to consider beforehands the needs of our client or manager and how we can convince them to follow a better process.

In this post, I will briefly present 12 techniques for selling a traditional client on an Agile project plan.  This post is based on a talk I gave at Agile 2009 and to a number of other user groups, and which you can see on Slideshare here.

As part of preparing this presentation, I created a survey of fellow grad students in UVa’s MSMIT program and other colleagues in the field.  I asked them how they have sold Agile or been sold on Agile.  Not everyone who answered the survey was a fan of agile and many of their quotes are interesting.

You can see the survey yourself here, and add your own feedback to it:
http://www.tinyurl.com/SellingAgileSurvey/

Those who answered the survey are all experienced IT veterans who have worked for a wide range of organizations, including Booz Allen Hamilton, SAIC, Capitol One, the International Monetary Fund, Fannie Mae, Freddie Mac, and more.

First, a couple of general comments from the survey that I found interesting:

“Agile seems to carry the connotation of ‘codelike-hell’ or just, ‘work faster’.”

“I am skeptical of any methods that that could be interpreted as ‘cutting corners’”

These are common first impressions of Agile, and I hope the following 12 techniques will give you ideas how to convince your clients or managers that Agile is the way to go and their first impressions are not accurate.  These techniques are based on a survey of current literature, my own consulting experiences at OpenSource Connections, and the feedback I received from the survey mentioned above.

1. Trial by Sprint

“You need to show a success to get adoption.”

Consider giving your boss an option.  Ask them to give you a sprint or two to try Agile out, and if they don’t like it, then you will go back to the old ways of doing things.  Make sure that you pick tasks for those trial sprints which can be successfully accomplished, and will show your boss the value of the incremental delivery of Agile.  You should also encourage your boss to play a role in the daily standups and see how communication in your team improves.

2.  Case Studies in Success

Do some research on similar companies to yours, and what methodologies they are using.  By googling around online, you should be able to find some case studies on Agile success and present those to your boss.

3.  Client Testimonials

For consultants, make sure you go back to your clients after a project and interview them.  Ask them how your process worked, and gather some quotes you can use in future proposals or client discussions.  I did this for one of our recent projects, and the customer gave us some great feedback like:

“Certainly one of the most successful projects ever here … Scrum eliminated my biases of what developers could do by letting them self-select.”

4. Find a Champion

One approach is to let others do the selling for you.  This is particularly useful for consultants who already have a relationship with a client, and you can let some of the clients’ employees talk about how they want you to use Scrum.  Or, as one of my classmates did, he became the internal champion for Agile:

“I highlighted the benefits to the Project Manager: higher productivity and less team management stuff since the team will take care of lots of team-management and updating (burn charts) instead of PM’s managing those details.”

Basically, identify a stakeholder in the project who is most in need of the benefits of Agile, and enlist their support to help you sell management on the use of Agile.

5. Use Metrics of Success

Many traditional clients like their current methods because they provide metrics that they can graph and look at.  They don’t always immediately understand how they can measure a project using methodology without a lot of documents.

But Agile has its own metrics too, and perhaps you can use those to sway the client.  Consider presenting them with information on velocity, story points completed vs waiting, test coverage, unit test success, etc.  Some Agile tools will provide these in pretty reports, but you can also fashion them yourself.  And don’t forget to show them the benefits of the burndown chart if you’re using Scrum, and emphasize how this gives them a real time view into the health of a sprint without encumbering the developers with lots of reporting.

6. Show how Agile combats common IT failures

One of my professors, Ryan Nelson, has been running a study of project retrospectives over the last 10 years, and has published articles in MIS Quarterly Executive detailing the Top Classic Mistakes in IT projects.  The top 3 are consistently:  Poor estimation and scheduling, Ineffective Stakeholder Management, and Insufficient Risk Management.  Agile can be used to address all those classic mistakes.

Consider why projects normally fail in your organization, and present the ways that Agile combats those common failures.

7. Examples of government/industry leaders using Agile

As one of my classmates pointed out:

“Clients, especially the military, are wary of catch phrases and sometimes unwilling to change their habits.”

If your client or boss fits that description, then you need to convince them they are not the first to try Agile, and that it has been used elsewhere successfully.  Try to find examples of their peers in industry or government using Agile, and point to them as an example.

One example you can use is my previous blog post about the CIA using Agile and Scrum.

8. Comparison to other methodologies

Compare how Agile/Scrum work compared to areas that your client or boss is already familiar with.  Try to point out both the similarities and the differences.  For example, one of my classmates took the following approach:

“I gave an overview of the Scrum process and highlighted the ease of transition since iterative/incremental development has been in practice for a long time (in other forms such as a spiral approach)”

9.  Listen to their needs and address them.

“I am always skeptical of anything that promises it is the ‘only’ or the ‘best’ [methodology].” – Comment from a development manager

Instead of going into the meeting and kicking off the discussion with your rant on why Agile is the best development methodology, take time to listen to your client or manager first.

Ask them how their projects have been going, what challenges they face, and take notes on their common problems.  Then suggest how Agile will address some of these issues.

The key here is to spend most of your time listening, and only then talk up the benefits of Agile.  This will assure your audience that you have heard their concerns, and that you are trying to present positive solutions to their problems.  This can be much more effective than trying to force a methodology on them.

10.  Sneak it in

This is a very common approach, and one that I have done before too.  One of my classmates is a technology program manager, and they commented that:

“I make sure I utilize agile practices where ever I can – I just don’t use the agile terminology.”

The point here is not to be deceptive, it’s just that sometimes you don’t need the sign off from the boss to do something.  After you successfully conclude your project, you can then point out to your boss that “by the way, all that stuff we did that really impressed you – that was Agile.”  Now that they have a positive impression and have witnessed some success, you can work on letting them formally adopt the process.

Note that doing this might require you to temporarily also fill out the same reports or status updates your current process requires, but hopefully you can get those dropped after having showed the benefits of Agile.

11.  Compromise

An agile purist may not like the idea of compromising, but I found the following sentiment to be pretty common:

“The methodology that has worked in my experience has been to incrementally introduce Agile … Start using a limited set of the practices and gradually start bringing in more.”

Similar to the “Sneak it in” approach, this can be done with or without your boss’s knowledge.

12.  Agile Project Management Office

One other thought to consider is setting up an Agile Project Management Office, or PMO.  A traditional PMO is a very document and regulation intense group, but some Agile practitioners have started to promote the idea of a lighter weight PMO.  An Agile PMO focuses less on micro managing teams, and more on providing an interface to traditional processes.  The Agile PMO can help Agile development teams by taking ownership of any burdensome compliance requirements which go against Agile processes.  This allows the developers to concentrate on developing code how they want, and not filling out checklists of documents.

They can also serve as an educator and coach to the client.

Conclusion

There are many variants on the above ideas, but I hope this quick list has given you some direction on different strategies you can use for convincing people to adopt Agile.  Keep in mind that each of these strategies has pros and cons, and you should consider the needs of your situation before adopting any particular strategy.

If you have other suggestions, please leave a comment and I look forward to hearing your feedback!

I’m very pleased to be speaking to Richmond SPIN (Software Process Improvement Network) next Wednesday October 14th, at 6:00 pm at Anthem BCBS in Richmond.  The address is 2015 Staples Mill Road in Richmond.  You can find more details and register online at:

http://www.richmondspin.org/home22222

Here’s the brief on my talk, which is based on my recent talk at Agile 2009.  I hope to see you there!

“Strategies for Persuasion on Adopting Agile”
By Arin Sime, Senior Consultant with OpenSource Connections

Do you want to know why Agile is sometimes a preferred methodology over a Waterfall method?  Arin will present 12 different methods for convincing a “traditional client” to use an Agile project plan. Based upon his own experiences with a wide variety of project styles, he will also present several definitions of each of the environments, well as results gathered from a survey on how to sell Agile. Some stories and best practices from other sources will also be covered.

Scott Ambler wrote a very interesting article for Dr Dobbs Journal recently entitled “RFPs the Agile Way — or — Fear and Loathing in the Procurement Department.”  I first came across the article when Deborah Hartman Preuss wrote an excellent commentary on InfoQ about Scott’s article, entitled “Using the RFP Process to Hire Agile”.

As Ambler points out, the use of a formal RFP often implies that the client uses a traditional development method like Waterfall.  Procurement writes an RFP, and then the consultant responds with a detailed proposal dictating timelines and cost estimates, and perhaps an up front design.  All before ever meeting with the client.

The RFP process is a necessary evil of the consulting world, particularly in government contracting.  But RFPs don’t lend themselves well to the very collaborative discussions you can have with a client when you are using an Agile process, or if you are securing the contract in person rather than electronically.

This topic relates somewhat to my upcoming presentation at Agile 2009, “How to sell a traditional client on an Agile project plan.”  Although a lot of the advice I give in that talk can be done in proposals, most of it works better in person.  And so I think that’s a real challenge to accurately respond (in an Agile way) to an RFP that assumes a waterfall development model.

At first I didn’t quite find the answers I was looking for in Ambler’s article.  I don’t say that as a knock against the article at all, because it is well worth the read and I think it frames very well the problems with RFP processes.  However, Ambler is mainly encouraging procurement people to change the way they structure RFP’s so that Agile people can respond to it better.

I’m not sure the procurement people are going to read his column or see the incentive to do so (which he acknowledges, and asks readers to send this article to procurement people they know).  And so while I believe my Agile 2009 talk offers some strategies for how to positively discuss Agile in a proposal, I’m still in search of the holy grail.  How exactly do you respond to an RFP “the Agile Way”?

But after reading Ambler’s article again, perhaps he does lay out some of the answers I’m looking for.  He just does it from the opposite perspective.  He suggests this advice for what procurement people should look for in an Agile response to an RFP:

(rather than lift all his text, I’ve just included the highlights below.  I suggest you read his article for the full explanation)

The supplier should provide resumes

Follow a reasonable set of rights and responsibilities for both the customer and the supplier to adhere to … how they will collaborate together.

Propose how they will work with the customer.

Produce potentially shippable software to the customer every X weeks.

Allow the customer to monitor their work and work products.

Follow the customer’s corporate development guidelines.

Do full regression testing throughout the lifecycle.

Provide a minimal set of deliverable documentation, particularly user documentation, operations documentation, support documentation, and high-level systems overview documentation. The customer should provide examples of such documentation if they’re available.

Shared-risk strategy [for payment] which is fair to both customer and supplier.

Indicate rough estimates and schedules, in the +/- 30% range.

That’s a great list of ideas for consultants to incorporate into proposals, even if the customer is not looking for them.  I’m glad to see that we already do many of them in our standard proposal template at OpenSource Connections.  In particular, we always talk about our “Open Approach” to projects, ie, the highly collaborative and open way that we work with our customers using Scrum.

This is all a sign of the increasing maturity and adoption of Agile processes, and if nothing else, that’s a good thing.  Perhaps the procurement people will sign on eventually after all.

Last Thursday I had a great opportunity to speak at AgileCville, and give a dry run of my upcoming presentation at Agile 2009.

Eric took this photo of me speaking.  It looks like I’m doing The Robot.  I’m not, but that would have been cool.

I’d like to thank all those who attended. I really enjoyed the chance to share my thoughts on “How to sell a traditional client on an Agile project plan.” I got a lot of great feedback from everybody. I’m flattered how much of it was positive, and I also got a lot of very helpful constructive feedback that I will definitely incorporate as I polish my slides for Agile 2009. Most notably perhaps, I’m going to take a crack at improving the quality and consistency of the graphics in the slides.

The slides are available online here. I also brought my voice recorder with me, and I’ve posted very roughly edited audio from the meeting here. The presentation was about 35 minutes long, and then we had a very interesting and valuable discussion session for about an hour after that. If you try to listen to the audio, be forewarned that not everyone’s voice comes through clear in the second half when we’re having our discussion.

I’m really looking forward to seeing everyone at Agile 2009!

I’ll be speaking to this Thursday’s #agilecville group meeting, about “How to sell a traditional client on an Agile project plan.” I have 11 strategies that I will go over for how to convince your clients that Agile methodologies are the way to go, and I think they cover a variety of different styles so there should be something for everyone.

When: Thursday July 16th, 6:00 PM

Where: Jefferson Room, 3rd Floor, Charlottesville Central Library (http://www.jmrl.org/li-meeting.htm)

More info and signups are on the AgileCville Google groups page.

This is all part of my lead up to speaking at Agile2009, and I’m really looking forward to tapping into the experiences and brilliance of my fellow Agile practitioners here in Charlottesville.  If you haven’t already, please take a moment to fill out my survey on how to sell Agile.  I’ll use the information I gather from this informal survey as part of my presentation.

I hope to see you there!  OpenSource Connections will provide pizza and soda!

At OSC, we have a well developed methodology that we apply to our client work, and one of the core tenets is using Continuous Integration to ensure our code behaves the way we intend it to.

However, recently we’ve had two projects were the usual CI solutions such as CruiseControl etc haven’t worked out well, and we had to develop our own internal CI tool that we are ready to publish to the world called MonkeyCI.

On the first project, which was a PHP based application with 5 full time developers, we used CruiseControl with the phpUnderControl addon. However, we were running CruiseControl on what turned out to be an underpowered hosted Windows server, and we kept getting build failure errors related to environmental difficulties. Now, if you’ve seen my talk about CI, you know how big I am on speccing a beefy server for CI, and this experience reinforced that lesson. We decided that migrating the CI environment to a bigger server was something that we felt was in the “nice to have” category, and that it could wait till the next iteration. But we needed something immediate. Enter MonkeyCI.

The heart of CI is all about building the code, running the tests, and publishing the results frequently. Everything else, the reports, the red/green lava lamps, the pretty JavaDocs etc are all gravy. To meet the needs of your developers, you need to know if the “bar is green”. So MonkeyCI does that in a decidedly low tech way:

Everytime someone runs the full suite of unit tests they record the day and time, and put their initials. If the build is failing then they immediately fixed it. We’ve played with writing the results in red for failing builds and green for successful builds as well. Then, each day at the standup someone highlights how the CI is doing, and verifies that multiple folks are initialing, which means that the tests are running on multiple systems successfully.

While this does mean you have an additional manual process, it’s also really easy to do, requiring just a whiteboard! And for small project teams, the overhead of maintaining a reliable CI system is too much.

We’re doing another two developer project right now, and at least so far MonkeyCI has been great. We haven’t seen integration issues yet such as database scripts that don’t run, or busted code being checked in. I’ll post a picture of our whiteboard once we have a bunch of checkoffs recorded!

We call this simple low tech process MonkeyCI because typically we refer to anything manual, such as testing by pounding keyboards as Monkey testing. Also, somewhat of a reference to the great developers at the Primate Programming Institute who I am sure would use this approach to CI!:

This past weekend was my first as a graduate student at the University of Virginia’s McIntire School of Commerce, where I am working on a Masters degree in Management of Information Technology.  It was a great 3 days of intensive classes on IT strategy, and I really enjoyed it.  Over the next year as I continue my studies, I will try to blog regularly about topics we are learning about.

As part of the program, the faculty regularly bring in interesting guest speakers with CIO experience.  This Saturday was a great example, since Jill Singer joined us.  Ms. Singer was formerly the Vice President for Project Management at SAIC, and is now the deputy CIO of the CIA.  She gave a great presentation on the role of the CIO, and the process they use at the CIA for evaluating, architecting and implementing their internal IT projects.

The CIA, despite the mystique and the fact that Ms Singer was not free to answer all the questions we asked, is still a lot like any other IT shop.  The process they follow for IT initiatives could easily be found in any Fortune 500 company.  In short, they follow these steps: understand the mission, establish the vision, develop the architecture, define plans, resource plans, execute plans, and measure progress.

Sounds pretty traditional, right?  Many other federal agencies probably follow a similar approach, which sounds a lot like a spiral development method.  But even within the constraints of this process, I was very pleased to hear Ms. Singer talk about regularly using Agile methodologies.

According to Ms. Singer, the CIA regularly uses Scrum, most often in 4 week development cycles.  Their customers, which would generally be some sort of internal analyst, really like the fact that Scrum encourages regular and tangible deliveries.  This allows them to try out the prototypes, and their customers also enjoy being able to add features and change priorities during each iteration.

This has worked very well for them on many projects, and Ms. Singer feels that the move from a more waterfall style to Scrum has really helped them improve many of their projects, though with an interesting side effect.

The biggest challenge she has seen on their is knowing when they are done, or as she put it, “defining what 1.0 is.”  They can’t fund their projects forever, just like any other IT shop.  Sometimes they end up doing iterations indefinitely though, and then realize they have gone longer than they originally thought because they keep adding features.  But unlike most project methodologies, if they slip on a project schedule using Scrum, Ms. Singer has found their customers are much more forgiving then when a waterfall project is late.

The reason for this is simple, and it is one of the fundamental advantages of Scrum and Agile, regardless of whether you are a start up company, a government agency, or even the CIA.  By engaging your business owners with burndowns, daily stand ups, and short iterations for which the customer helps set the priorities, you are empowering your customer.  It’s important to note that this is done in a way that does not infringe on the creativity of your development team.  Your developers are likewise empowered by choosing what they work on and in what order within a sprint, setting their own estimates, and providing regular feedback and ideas directly to the customers.

It’s never good when a project is late, but if the customer has seen constant progress along the way, and they are empowered to help decide what features should be added or removed, then you have successfully created a collaborative environment between your customers and your IT staff.

Determining “what 1.0 is” can be a real challenge – we just had a good discussion on that today with one of our current clients.  By employing Scrum, it sounds like the CIA has also learned the advantages of a highly iterative and collaborative process, and it is helping them to stay efficient and productive.  By the very nature of what they do, the CIA must be innovative, and so it should come as no surprise that they are using the latest in software development methodology.  I hope that other federal agencies will follow their lead.