Where are Eric and Youssef?

This year two OSC folks are working remotely for extended periods of time. Youssef spent 6 weeks in Lebanon earlier this year, and I am spending the month of July in the mountains of Western North Carolina. We had talked about some of the rewards as well as challenges of working effectively when you are in a new/fun/exciting/[insert superlative here] place. Youssef and I came up with a few tips to make balancing work/play easier!

Tip 1: Establish a Routine

Youssef says: There will be no shortage of distractions, specially during the World Cup season. A trip to the mountains, a music festival or a friend’s unannounced (or announced) visit are all good reasons to drop what you’re doing and enjoy the place you are in. The only way to both get your work done and have fun is to know when is the appropriate time for each. I found that establishing a routine worked best for me. I woke up early every morning and took advantage of the early quiet hours of the day. Started work at 8am and got plenty done early on. Then I gave myself a couple of hours of break time for unscheduled events (a friend’s visit, a trip to the store to get souvenirs, or even a couple of hours at the beach). Then I resumed work till the evening when I had a conference call scheduled every day at 5:45pm. After the call I was free to do whatever I wanted and that gave me plenty of time, specially in a city like Beirut that does not sleep.
Eric says: I know that I sometimes struggle to get started working in the morning. And being in the office makes it simpler because everyone else is working. So having a specific schedule of when I am working helps me shift my brain out of family/vacation mode and into work mode!
(more…)

Agile teams should be based on talented development staff with multi disclipinary skill sets. This allows team members to easily trade tasks and keep the iteration moving forward. As a consultant, I have seen and worked in many environments where development staff are very silo’d in their skill sets. In most cases they are very good at what they do, but at various times each team member can become a bottleneck because they are the only person who knows how to do a particular task or work on a certain legacy system.

But multi-disclipinary skill sets are not the only key to fluidly switching people between tasks. You must also be able to change technology or development environments easily as well, and this is an area that I find almost all development environments suffer from.

As an IT leader, when you bring in a team of consultants to augment your team and give their velocity a shot in the arm, you expect them to integrate quickly and be productive right away. And that’s what the consultant wants too. But are you giving them the tools they need to be effective right away? Or do you expect each of them to go through the same development environment setup that you ask new hires to do?

When you hire a new person, perhaps it’s worth the time to have them setup everything about their development environment. They learn about the nuances of your system, and a few days or a week of their time is nothing compared to the long term investment you expect from having them work for you for several years.

When you bring in a consultant, it’s different. You may only expect to have them for a few months, and their time is more expensive then your internal staff. And they are not going to benefit much from the learning experience of setting up a dev environment.

And even for your full time staff, what happens when they have to upgrade their machine a year from now, or their system dies and they have to rebuild from scratch? Do you really want to have them spend another week trying to remember how to get past that one install bug that they encountered a year or more before?

One answer to this problem is to develop code using virtual machines. Instead of having your first developer start by setting up the toolsets and plugins on their machine, have them start with a clean virtual image, using a tool like VMWare Player. Keep a bare bones image around with your basic development tools, or start with one from a similar project. After the developer gets the code started and everything basically working, then they should make a copy of that image and distribute it to the rest of the development team.

A backup copy of this image should also be kept on a network share or external drive so that when you add in consultants, or bring in a new hire, they can get started on code just as soon as they copy the image over to their machine. Instead of spending a week getting everything installed and picking the brain of your development staff, or Googling on obscure error messages, the only time delay is how long it takes the image to copy to their local machine (probably no more than an hour in most cases).

Now you can spend their time where it’s valuable: Bringing them up to speed on the nuances of your code itself, not the right service pack or magic incantations to get the development tools working.

You might be thinking that with cloud computing and languages like Ruby on Rails, shouldn’t this be a non issue? I just pull down the code from svn or github, and I’m ready to go! In my experience, while that is usually the case with relatively small projects or with projects started in the last year or two, it’s not always the case with lots of other code that exists in the world and that we have to work on.

I was reminded of this because of a client I recently worked with is still using version 1.1 of the .NET framework, and so is required to run Microsoft Visual Studio 2003. As part of the project we’ll probably end up doing an upgrade to more recent versions, but we still have work we have to do first. And unfortunately I discovered on my first day on the project that Windows 7 does not play well with Visual Studio 2003. Once it installed, it wouldn’t connect to IIS in debug mode. After doing some googling and seeing that this is a common problem with no easy solution (I tried many of the recommended solutions to no avail), I decided it was time to just get an image of Windows XP going. The rest of the dev team is on XP, but they are not using virtual machines to do it unfortunately.

So I started creating a new XP image under Virtual PC, only to have it started failing on certain files. Even though the install continued, I ended up with an image that could not talk to the outside world and was clearly half-baked.

So then I started over with VMWare Player, and as I write this, my XP image with Visual Studio 2003 already installed is nearly done. 3 days later. Before I start changing code on it, I’ll be setting the image aside to give to another consultant joining the team in a few weeks, so he doesn’t have to go to the same level of pain. I’ll also make sure the client has a copy of that image to make their life easier in the future the next time they bring us in or make a new hire.

Before you attribute the delays I experienced to Microsoft, I’ve seen similar challenges on Php projects and to a lesser degree on Groovy on Grails projects.

Virtualization is great for reducing the size of your server room, cutting down on power bills, and becoming more green. But don’t let it stop there. A little bit of time spent up front on virtualizing your development environment will save you a lot of heartache and expenses later down the road, and make your IT shop much more agile.

The Agile CIO is a series of blog posts advising IT leaders on how best to incorporate Agile techniques into their organizations. For more information about OpenSource Connections’ Agile process consulting services, please contact the author at ASime@OpenSourceConnections.com.

Everybody loves employees and consultants who can get stuff done. Especially if you’ve made a commitment to a client or your boss, it can be very difficult to ask for an extension to that timeline. It’s much easier to ask your team to put in the extra hours to get the job done, even if that means the job won’t be done well. Sometimes leaders will even resort to claiming the product is “done”, when it was never truly done at all, and needs a lot more work.

Your development team will likely respond to your calls with heroic effort. And you might actually make the deadline. But if you do it regularly, your reputation as a leader may actually suffer, and your team will probably burn out unless you can pour on enough accolades and spot-bonuses.

At some point, almost all IT leaders, including myself, have done it. That is unfortunately not much of a secret and it’s why IT teams often develop a bad reputation. The bigger secret is that a lot of developers not only regularly take on the hero role, they actually like it.

This creates a dangerous situation, where IT leaders ask for heroic efforts because it is the path of least resistance. And the development teams respond with grumbles and late nights, but also with the misconception of job security and the expectation that they will receive a lot of praise when they come in on Monday bleary-eyed (but with at least part of the job done).

Why is there so much danger in encouraging your developers to be heroes? Do a google search on “cowboy coders” or “programmer heroes” and you’ll see other blog posts that describe the whys and the pitfalls of reliance on hero development. In short, you’ll get inconsistent and buggy code that may not fit your architecture, won’t meet the customer’s needs, is not very flexible, creates technical debt you will complain about for years, and will probably need to be reworked eventually.

So how do you avoid heroes? As a CIO or technical lead, the tools of Agile offer many solutions:

• Sprint planning – restricting the work that is done to the current sprint helps discourage sudden changes of priority. Most tasks can in reality wait until the next sprint to be added, especially if the customer understands that it means the current sprint has to be put on hold or canceled.
• Daily standups and regular customer interaction – Chronic hero programmers often like to hideaway on their work, and so they can be more easily detected when you are interacting with the customer each day. There is less opportunity for the developer to hide away and not give status updates, and the standups provide the customer more immediate feedback on everything that is being held up because they sent the hero on a distracting quest.
• Team estimation – When the team estimates tasks for a sprint together, before they are assigned to people, that encourages open discussion and debate on how long something will take.
• Continuous Integration – Maintaining rules about daily code check-ins and regular integration and unit testing prevents the hero from going too far away from the mainstream code base, and gives visible signs of progress. When done properly, it also helps to reduce the bugs delivered in the rushed code.

In addition, in our OpenApproach implementation of Agile and Scrum, we use range estimation of tasks. Asking people to give you a single-point estimate of how long something will take encourages them to be overly optimistic. “Oh, I can get that done in about 2 hours.”

Really? What if it’s not as simple as you or the developer thought? Then the only solution is to put in a heroic effort if the timeline must be met.

A simple range estimate will provide a much more realistic view, and encourages a more thorough thought process: “It should only take about 2 hours of actual coding, but I’ll have to switch projects and get that site running again on my machine. Plus, sometimes those java script issues can be really though to debug, so it could take as long as 8 hours of work.”

As an Agile CIO, it’s your job to not only be understanding of your developers when they give you a range estimate, but encourage it. Don’t settle for the heroic answer of “I’ll just put in some extra time this weekend.” Get to the root cause of the problems, and make sure the customer or your boss is aware of the delays earlier rather than later. It will make all of your lives easier, and help you to avoid the pitfalls of programmer heroes.

The Agile CIO is a series of blog posts advising IT leaders on how best to incorporate Agile techniques into their organizations. For more information about OpenSource Connections’ Agile process consulting services, please contact the author at ASime@OpenSourceConnections.com.

This past Monday, OSC had a company hackathon day. It is not our first, but we have been trying to make this sort of activity a more regular thing for us. Unlike our previous hackathons, this one was more flexible on the rules. Previously we had the entire team work on the same technology or goal whereas this time was more free form. We each had the opportunity to work on either a new or old technology (by new and old I mean something we’ve worked with before or not), on new or old projects and to collaborate or work individually.

The day featured work on things like Google App Engine and the Android application platform. I took this chance to get myself out of the PHP world I’ve been stuck in for about a year now and get back into some Rails development. I had a little tool in mind that I wanted to develop which would help us internally do some things a bit more efficiently. I used that as an excuse to try out some of the Rails plugins or gems that I have wanted to use for a while, which are:

• Mocha (http://github.com/mislav/rspec-rails-mocha http://mocha.rubyforge.org/)
• Factory Girl (http://github.com/thoughtbot/factory_girl)
• GChart (http://github.com/jbarnette/gchart)
• Open Flash Chart (http://github.com/pullmonkey/open_flash_chart)

I have to say, the buzz around Mocha and Factory Girl is very old and I wish I had the opportunity to use these tools before now. New things come out everyday and it is tough to keep up with everything. My advise to you is to dedicate a “playing time” often for you to try out the newest and greatest thing that comes out. Do this in groups where each one can be working on a different things and share your findings at the end of the day. Even though I didn’t work on the Android app, I did get a feel for it from Arin through his experience with it. One of the things that helped take this experience further is our presence all together in one conference room feeding out of each other.

Looking forward to the next OSC hackathon.

Working at a small company like OSC allows for group bonding “work” activities such as RailsRumble, which create a good working environment and helps develop our skills. If you don’t know what RailsRumble is, it’s a competition where teams of 1 up to 4 members develop a Ruby on Rails application in 48 hours from scratch. If you don’t know what Ruby on Rails is, you can’t be saved . The competition is fun, or at least the OSC team is fun (no bias what so ever) because you get together with a group of people, sometimes teams share office space, and work your brain until you pass out. One of the things this competition teaches us is managing time. What you can do in 48 hours might vary from person to person, but it’s still 48 hours people!

Our entry for 2008, see blog posts here and here, used a SMS service called Zeep Mobile. We needed to be able to communicate with our users, specially those who would be using our app on the go, and texting was the best solution. We came up with the idea for our app the day the competition started, so we didn’t have time to do a lot of research on providers and available APIs for such a service. In retrospect, Zeep Mobile still seems the best free solution there. The one problem with this service is that the learning curve turned out to be too steep for the 48 hours we had.

One of my teammates was charged with handling the integration between Zeep Mobile and our application. That task took about a day, half the time we had to build EVERYTHING! That is too much time. So a light bulb lit above my head. I always wanted to contribute to the rails community but have never had the way to do it. I have no experience building plugins or gems, so this was a good project to learn. Unfortunately, I discovered what my teammate had a year ago, how hard it is to integrate Zeep Mobile!!!

No Worries! ZeepIt is here to help

My first plugin, ZeepIt, is designed to get your app ready to go with Zeep Mobile. All you have to do is install the plugin and you automatically have a URL for Zeep Mobile to forward SMS to and a way to parse those texts. The plugin uses the zeep/messaging gem provided by Zeep Mobile to provide a the MVC structure needed to leverage the service.

The plugin is available on GitHub: http://github.com/ychaker/zeep_it/tree/master
Documentation is available on Rdoc.info: http://rdoc.info/projects/ychaker/zeep_it
References:

• RailsCasts
• Ruby on Rails guides
• RailsMagazine
• Zeep Mobile
• Google (of course)

If you have any comments, questions or concerns please feel free to contact me which ever way you like. I would love to hear your feedback. ZeepIt is also on twitter @ZeepIt. (by the way, as I said, it’s my first plugin, so be gentle)

At OSC, we have a well developed methodology that we apply to our client work, and one of the core tenets is using Continuous Integration to ensure our code behaves the way we intend it to.

However, recently we’ve had two projects were the usual CI solutions such as CruiseControl etc haven’t worked out well, and we had to develop our own internal CI tool that we are ready to publish to the world called MonkeyCI.

On the first project, which was a PHP based application with 5 full time developers, we used CruiseControl with the phpUnderControl addon. However, we were running CruiseControl on what turned out to be an underpowered hosted Windows server, and we kept getting build failure errors related to environmental difficulties. Now, if you’ve seen my talk about CI, you know how big I am on speccing a beefy server for CI, and this experience reinforced that lesson. We decided that migrating the CI environment to a bigger server was something that we felt was in the “nice to have” category, and that it could wait till the next iteration. But we needed something immediate. Enter MonkeyCI.

The heart of CI is all about building the code, running the tests, and publishing the results frequently. Everything else, the reports, the red/green lava lamps, the pretty JavaDocs etc are all gravy. To meet the needs of your developers, you need to know if the “bar is green”. So MonkeyCI does that in a decidedly low tech way:

Everytime someone runs the full suite of unit tests they record the day and time, and put their initials. If the build is failing then they immediately fixed it. We’ve played with writing the results in red for failing builds and green for successful builds as well. Then, each day at the standup someone highlights how the CI is doing, and verifies that multiple folks are initialing, which means that the tests are running on multiple systems successfully.

While this does mean you have an additional manual process, it’s also really easy to do, requiring just a whiteboard! And for small project teams, the overhead of maintaining a reliable CI system is too much.

We’re doing another two developer project right now, and at least so far MonkeyCI has been great. We haven’t seen integration issues yet such as database scripts that don’t run, or busted code being checked in. I’ll post a picture of our whiteboard once we have a bunch of checkoffs recorded!

We call this simple low tech process MonkeyCI because typically we refer to anything manual, such as testing by pounding keyboards as Monkey testing. Also, somewhat of a reference to the great developers at the Primate Programming Institute who I am sure would use this approach to CI!:

This past weekend was my first as a graduate student at the University of Virginia’s McIntire School of Commerce, where I am working on a Masters degree in Management of Information Technology.  It was a great 3 days of intensive classes on IT strategy, and I really enjoyed it.  Over the next year as I continue my studies, I will try to blog regularly about topics we are learning about.

As part of the program, the faculty regularly bring in interesting guest speakers with CIO experience.  This Saturday was a great example, since Jill Singer joined us.  Ms. Singer was formerly the Vice President for Project Management at SAIC, and is now the deputy CIO of the CIA.  She gave a great presentation on the role of the CIO, and the process they use at the CIA for evaluating, architecting and implementing their internal IT projects.

The CIA, despite the mystique and the fact that Ms Singer was not free to answer all the questions we asked, is still a lot like any other IT shop.  The process they follow for IT initiatives could easily be found in any Fortune 500 company.  In short, they follow these steps: understand the mission, establish the vision, develop the architecture, define plans, resource plans, execute plans, and measure progress.

Sounds pretty traditional, right?  Many other federal agencies probably follow a similar approach, which sounds a lot like a spiral development method.  But even within the constraints of this process, I was very pleased to hear Ms. Singer talk about regularly using Agile methodologies.

According to Ms. Singer, the CIA regularly uses Scrum, most often in 4 week development cycles.  Their customers, which would generally be some sort of internal analyst, really like the fact that Scrum encourages regular and tangible deliveries.  This allows them to try out the prototypes, and their customers also enjoy being able to add features and change priorities during each iteration.

This has worked very well for them on many projects, and Ms. Singer feels that the move from a more waterfall style to Scrum has really helped them improve many of their projects, though with an interesting side effect.

The biggest challenge she has seen on their is knowing when they are done, or as she put it, “defining what 1.0 is.”  They can’t fund their projects forever, just like any other IT shop.  Sometimes they end up doing iterations indefinitely though, and then realize they have gone longer than they originally thought because they keep adding features.  But unlike most project methodologies, if they slip on a project schedule using Scrum, Ms. Singer has found their customers are much more forgiving then when a waterfall project is late.

The reason for this is simple, and it is one of the fundamental advantages of Scrum and Agile, regardless of whether you are a start up company, a government agency, or even the CIA.  By engaging your business owners with burndowns, daily stand ups, and short iterations for which the customer helps set the priorities, you are empowering your customer.  It’s important to note that this is done in a way that does not infringe on the creativity of your development team.  Your developers are likewise empowered by choosing what they work on and in what order within a sprint, setting their own estimates, and providing regular feedback and ideas directly to the customers.

It’s never good when a project is late, but if the customer has seen constant progress along the way, and they are empowered to help decide what features should be added or removed, then you have successfully created a collaborative environment between your customers and your IT staff.

Determining “what 1.0 is” can be a real challenge – we just had a good discussion on that today with one of our current clients.  By employing Scrum, it sounds like the CIA has also learned the advantages of a highly iterative and collaborative process, and it is helping them to stay efficient and productive.  By the very nature of what they do, the CIA must be innovative, and so it should come as no surprise that they are using the latest in software development methodology.  I hope that other federal agencies will follow their lead.

Last month Scott, Arin, and myself road tripped to attend the Richmond SPIN meeting, where Kris Puthucode of the Software Quality Center gave a talk on “Metrics and Analysis in Companies at Different Maturity Levels of the CMMI
Model”. The focus on the talk was what results you can expect out of metrics if you do the work of performing the analysis required.

I was attending the presentation with a certain sense of trepidation… I consider myself a hard core developer( despite my “Test Obsessed” armband) who doesn’t have time or patience for pointy headed manager paperwork. But I am also someone who focuses on process improvement and honing my craft of software development, so where can metrics inspired by CMMI be useful to a fast moving Agile team cranking out functioning software every three weeks? So I listened to Kris and tried to think about what he said in the context of Scrum.

The first slide pointed out that there are three kinds of lies: “lies, damn lies, and statistics”. You need to be careful about what your data says. When you are measuring velocity in Scrum, you need to have a couple of sprints, at least 3 to have any sense of your progress. If you say “we get X done” based on the first sprint, well often the first one is very conservative sprint. And, as you look at your average burndown, you need to have a couple of days of information before you can get a sense of average burndown as the first days often you find as many tasks as you accomplish. And, over the 15 days of your iteration, progress can be pretty spiky… Many teams have pretty flat burn down at the beginning, and then some steep drops… Ideally you are looking to see progress per day become flatter, less spiky, which would indicate that your estimating is improving. Or your team is being less affected by external factors that might hamper their productivity.

He talked about whether to use average or median numbers in looking at series of numbers, such as looking at your burndown.. If you have a lot of outliers, then use median to get a better view, otherwise use average. So if you burndown 20, 25, 22, 40, 25 then using the average is good. But if you have 8, 20, 25, 22, 40, 12 then maybe median would be better.

Kris talked about the cultural challenge of convincing people to provide the data required to build metrics. People need to know why the numbers matter, and even better why it will help then. It’s why developer’s hate filling out TPS reports! And why I like Scrum and it’s low overhead, as well as more passive measurement tools like HackyStat and 6thSense(now part of RallyDev). I feel like mandating one metric, say your basic time tracking is viable, but if you add more on you start getting more push back or gaming of the metrics.

He talked about getting metrics like Project Start Date and Project End Date. A local company splits up it’s year as 3 week iterations, and then apportions iterations across competing projects. These iterations then feed the project start and end dates.

He stressed that you need to have a shared vision on metrics, and a shared vision of what “on time, in budget, with quality” really means. I know we the other day had a scrum team debate how to track found tasks. And this was a set of people that had worked together previously on different project having different visions of tracking found tasks and how they should affect the “ideal burndown” line! Covering periodically what that shared vision, and ensuring your team and stakeholders are all on the same page is very valuable.

He stressed that your metrics need to be actual “measurable” things. You need to be able to quantify the metrics that you are using using a shared basis so that when you compare two things using the same metric that you are doing an apples to apples comparison, not an apples to kiwi comparison! For Scrum, it means you need to use the same time frame for iterations, and you can compare one sprint to another for a specific team on a specific project, but not across teams or projects.

Scrum for us provides a very standarized metrics across the OSC organization, regardless of client or specific technology. As long as we are sharing the same vision for our metrics!

When we do a retrospective, and look back at our burndowns, we are doing “Progress Indicators” that are lagging indicators. One of the things that the speaker was advocating was to look at forward looking indicators that predict into the future where we will be. But of course, identifying and seeing a leading indicator is difficult, and takes a lot more analysis. In scrum we would have to tie our tasks to various sprint goals which are appropriately estimated against to provide our velocity.

Did highlight that you need multiple projects happening to be able to gather the variety of data points to be able to compare data points. Compare two scrum teams together and it’s tough to compare them because you can’t see the outliers. But, if you have 10 scrum teams, who have a shared vision of the metrics, then you can start comparing them together. You may have to normalize across the teams, but with enough iterations you can compare and predict.

So some things to show would be a histgram of how much the team burns down a day. Highlight what kind of deviation we have in our progress per day. Over multiple sprints you can maybe see what the first third, middle third, and final third look like. Can we characterize “At OSC, we typically see this kinda of result in the thirds of the project?” Hey, does this feed into our Waterfall projects in 3 weeks?

First week is requirements solidification. Second week is development. Third week is testing and polish.

Can we figure out how to predict the results for sprint 3? We could do this for sprint 1 and 2 and predict sprint 3.

We measure progress per day as a ratio, and then sum it over a week. With that progress per week, then we can see what a sprint 3 would do.