This past Monday, OSC had a company hackathon day. It is not our first, but we have been trying to make this sort of activity a more regular thing for us. Unlike our previous hackathons, this one was more flexible on the rules. Previously we had the entire team work on the same technology or goal whereas this time was more free form. We each had the opportunity to work on either a new or old technology (by new and old I mean something we’ve worked with before or not), on new or old projects and to collaborate or work individually.

The day featured work on things like Google App Engine and the Android application platform. I took this chance to get myself out of the PHP world I’ve been stuck in for about a year now and get back into some Rails development. I had a little tool in mind that I wanted to develop which would help us internally do some things a bit more efficiently. I used that as an excuse to try out some of the Rails plugins or gems that I have wanted to use for a while, which are:

• Mocha (http://github.com/mislav/rspec-rails-mocha http://mocha.rubyforge.org/)
• Factory Girl (http://github.com/thoughtbot/factory_girl)
• GChart (http://github.com/jbarnette/gchart)
• Open Flash Chart (http://github.com/pullmonkey/open_flash_chart)

I have to say, the buzz around Mocha and Factory Girl is very old and I wish I had the opportunity to use these tools before now. New things come out everyday and it is tough to keep up with everything. My advise to you is to dedicate a “playing time” often for you to try out the newest and greatest thing that comes out. Do this in groups where each one can be working on a different things and share your findings at the end of the day. Even though I didn’t work on the Android app, I did get a feel for it from Arin through his experience with it. One of the things that helped take this experience further is our presence all together in one conference room feeding out of each other.

Looking forward to the next OSC hackathon.

Working at a small company like OSC allows for group bonding “work” activities such as RailsRumble, which create a good working environment and helps develop our skills. If you don’t know what RailsRumble is, it’s a competition where teams of 1 up to 4 members develop a Ruby on Rails application in 48 hours from scratch. If you don’t know what Ruby on Rails is, you can’t be saved . The competition is fun, or at least the OSC team is fun (no bias what so ever) because you get together with a group of people, sometimes teams share office space, and work your brain until you pass out. One of the things this competition teaches us is managing time. What you can do in 48 hours might vary from person to person, but it’s still 48 hours people!

Our entry for 2008, see blog posts here and here, used a SMS service called Zeep Mobile. We needed to be able to communicate with our users, specially those who would be using our app on the go, and texting was the best solution. We came up with the idea for our app the day the competition started, so we didn’t have time to do a lot of research on providers and available APIs for such a service. In retrospect, Zeep Mobile still seems the best free solution there. The one problem with this service is that the learning curve turned out to be too steep for the 48 hours we had.

One of my teammates was charged with handling the integration between Zeep Mobile and our application. That task took about a day, half the time we had to build EVERYTHING! That is too much time. So a light bulb lit above my head. I always wanted to contribute to the rails community but have never had the way to do it. I have no experience building plugins or gems, so this was a good project to learn. Unfortunately, I discovered what my teammate had a year ago, how hard it is to integrate Zeep Mobile!!!

No Worries! ZeepIt is here to help

My first plugin, ZeepIt, is designed to get your app ready to go with Zeep Mobile. All you have to do is install the plugin and you automatically have a URL for Zeep Mobile to forward SMS to and a way to parse those texts. The plugin uses the zeep/messaging gem provided by Zeep Mobile to provide a the MVC structure needed to leverage the service.

The plugin is available on GitHub: http://github.com/ychaker/zeep_it/tree/master
Documentation is available on Rdoc.info: http://rdoc.info/projects/ychaker/zeep_it
References:

• RailsCasts
• Ruby on Rails guides
• RailsMagazine
• Zeep Mobile
• Google (of course)

If you have any comments, questions or concerns please feel free to contact me which ever way you like. I would love to hear your feedback. ZeepIt is also on twitter @ZeepIt. (by the way, as I said, it’s my first plugin, so be gentle)

At OSC, we have a well developed methodology that we apply to our client work, and one of the core tenets is using Continuous Integration to ensure our code behaves the way we intend it to.

However, recently we’ve had two projects were the usual CI solutions such as CruiseControl etc haven’t worked out well, and we had to develop our own internal CI tool that we are ready to publish to the world called MonkeyCI.

On the first project, which was a PHP based application with 5 full time developers, we used CruiseControl with the phpUnderControl addon. However, we were running CruiseControl on what turned out to be an underpowered hosted Windows server, and we kept getting build failure errors related to environmental difficulties. Now, if you’ve seen my talk about CI, you know how big I am on speccing a beefy server for CI, and this experience reinforced that lesson. We decided that migrating the CI environment to a bigger server was something that we felt was in the “nice to have” category, and that it could wait till the next iteration. But we needed something immediate. Enter MonkeyCI.

The heart of CI is all about building the code, running the tests, and publishing the results frequently. Everything else, the reports, the red/green lava lamps, the pretty JavaDocs etc are all gravy. To meet the needs of your developers, you need to know if the “bar is green”. So MonkeyCI does that in a decidedly low tech way:

Everytime someone runs the full suite of unit tests they record the day and time, and put their initials. If the build is failing then they immediately fixed it. We’ve played with writing the results in red for failing builds and green for successful builds as well. Then, each day at the standup someone highlights how the CI is doing, and verifies that multiple folks are initialing, which means that the tests are running on multiple systems successfully.

While this does mean you have an additional manual process, it’s also really easy to do, requiring just a whiteboard! And for small project teams, the overhead of maintaining a reliable CI system is too much.

We’re doing another two developer project right now, and at least so far MonkeyCI has been great. We haven’t seen integration issues yet such as database scripts that don’t run, or busted code being checked in. I’ll post a picture of our whiteboard once we have a bunch of checkoffs recorded!

We call this simple low tech process MonkeyCI because typically we refer to anything manual, such as testing by pounding keyboards as Monkey testing. Also, somewhat of a reference to the great developers at the Primate Programming Institute who I am sure would use this approach to CI!:

This past weekend was my first as a graduate student at the University of Virginia’s McIntire School of Commerce, where I am working on a Masters degree in Management of Information Technology.  It was a great 3 days of intensive classes on IT strategy, and I really enjoyed it.  Over the next year as I continue my studies, I will try to blog regularly about topics we are learning about.

As part of the program, the faculty regularly bring in interesting guest speakers with CIO experience.  This Saturday was a great example, since Jill Singer joined us.  Ms. Singer was formerly the Vice President for Project Management at SAIC, and is now the deputy CIO of the CIA.  She gave a great presentation on the role of the CIO, and the process they use at the CIA for evaluating, architecting and implementing their internal IT projects.

The CIA, despite the mystique and the fact that Ms Singer was not free to answer all the questions we asked, is still a lot like any other IT shop.  The process they follow for IT initiatives could easily be found in any Fortune 500 company.  In short, they follow these steps: understand the mission, establish the vision, develop the architecture, define plans, resource plans, execute plans, and measure progress.

Sounds pretty traditional, right?  Many other federal agencies probably follow a similar approach, which sounds a lot like a spiral development method.  But even within the constraints of this process, I was very pleased to hear Ms. Singer talk about regularly using Agile methodologies.

According to Ms. Singer, the CIA regularly uses Scrum, most often in 4 week development cycles.  Their customers, which would generally be some sort of internal analyst, really like the fact that Scrum encourages regular and tangible deliveries.  This allows them to try out the prototypes, and their customers also enjoy being able to add features and change priorities during each iteration.

This has worked very well for them on many projects, and Ms. Singer feels that the move from a more waterfall style to Scrum has really helped them improve many of their projects, though with an interesting side effect.

The biggest challenge she has seen on their is knowing when they are done, or as she put it, “defining what 1.0 is.”  They can’t fund their projects forever, just like any other IT shop.  Sometimes they end up doing iterations indefinitely though, and then realize they have gone longer than they originally thought because they keep adding features.  But unlike most project methodologies, if they slip on a project schedule using Scrum, Ms. Singer has found their customers are much more forgiving then when a waterfall project is late.

The reason for this is simple, and it is one of the fundamental advantages of Scrum and Agile, regardless of whether you are a start up company, a government agency, or even the CIA.  By engaging your business owners with burndowns, daily stand ups, and short iterations for which the customer helps set the priorities, you are empowering your customer.  It’s important to note that this is done in a way that does not infringe on the creativity of your development team.  Your developers are likewise empowered by choosing what they work on and in what order within a sprint, setting their own estimates, and providing regular feedback and ideas directly to the customers.

It’s never good when a project is late, but if the customer has seen constant progress along the way, and they are empowered to help decide what features should be added or removed, then you have successfully created a collaborative environment between your customers and your IT staff.

Determining “what 1.0 is” can be a real challenge – we just had a good discussion on that today with one of our current clients.  By employing Scrum, it sounds like the CIA has also learned the advantages of a highly iterative and collaborative process, and it is helping them to stay efficient and productive.  By the very nature of what they do, the CIA must be innovative, and so it should come as no surprise that they are using the latest in software development methodology.  I hope that other federal agencies will follow their lead.

Last month Scott, Arin, and myself road tripped to attend the Richmond SPIN meeting, where Kris Puthucode of the Software Quality Center gave a talk on “Metrics and Analysis in Companies at Different Maturity Levels of the CMMI
Model”. The focus on the talk was what results you can expect out of metrics if you do the work of performing the analysis required.

I was attending the presentation with a certain sense of trepidation… I consider myself a hard core developer( despite my “Test Obsessed” armband) who doesn’t have time or patience for pointy headed manager paperwork. But I am also someone who focuses on process improvement and honing my craft of software development, so where can metrics inspired by CMMI be useful to a fast moving Agile team cranking out functioning software every three weeks? So I listened to Kris and tried to think about what he said in the context of Scrum.

The first slide pointed out that there are three kinds of lies: “lies, damn lies, and statistics”. You need to be careful about what your data says. When you are measuring velocity in Scrum, you need to have a couple of sprints, at least 3 to have any sense of your progress. If you say “we get X done” based on the first sprint, well often the first one is very conservative sprint. And, as you look at your average burndown, you need to have a couple of days of information before you can get a sense of average burndown as the first days often you find as many tasks as you accomplish. And, over the 15 days of your iteration, progress can be pretty spiky… Many teams have pretty flat burn down at the beginning, and then some steep drops… Ideally you are looking to see progress per day become flatter, less spiky, which would indicate that your estimating is improving. Or your team is being less affected by external factors that might hamper their productivity.

He talked about whether to use average or median numbers in looking at series of numbers, such as looking at your burndown.. If you have a lot of outliers, then use median to get a better view, otherwise use average. So if you burndown 20, 25, 22, 40, 25 then using the average is good. But if you have 8, 20, 25, 22, 40, 12 then maybe median would be better.

Kris talked about the cultural challenge of convincing people to provide the data required to build metrics. People need to know why the numbers matter, and even better why it will help then. It’s why developer’s hate filling out TPS reports! And why I like Scrum and it’s low overhead, as well as more passive measurement tools like HackyStat and 6thSense(now part of RallyDev). I feel like mandating one metric, say your basic time tracking is viable, but if you add more on you start getting more push back or gaming of the metrics.

He talked about getting metrics like Project Start Date and Project End Date. A local company splits up it’s year as 3 week iterations, and then apportions iterations across competing projects. These iterations then feed the project start and end dates.

He stressed that you need to have a shared vision on metrics, and a shared vision of what “on time, in budget, with quality” really means. I know we the other day had a scrum team debate how to track found tasks. And this was a set of people that had worked together previously on different project having different visions of tracking found tasks and how they should affect the “ideal burndown” line! Covering periodically what that shared vision, and ensuring your team and stakeholders are all on the same page is very valuable.

He stressed that your metrics need to be actual “measurable” things. You need to be able to quantify the metrics that you are using using a shared basis so that when you compare two things using the same metric that you are doing an apples to apples comparison, not an apples to kiwi comparison! For Scrum, it means you need to use the same time frame for iterations, and you can compare one sprint to another for a specific team on a specific project, but not across teams or projects.

Scrum for us provides a very standarized metrics across the OSC organization, regardless of client or specific technology. As long as we are sharing the same vision for our metrics!

When we do a retrospective, and look back at our burndowns, we are doing “Progress Indicators” that are lagging indicators. One of the things that the speaker was advocating was to look at forward looking indicators that predict into the future where we will be. But of course, identifying and seeing a leading indicator is difficult, and takes a lot more analysis. In scrum we would have to tie our tasks to various sprint goals which are appropriately estimated against to provide our velocity.

Did highlight that you need multiple projects happening to be able to gather the variety of data points to be able to compare data points. Compare two scrum teams together and it’s tough to compare them because you can’t see the outliers. But, if you have 10 scrum teams, who have a shared vision of the metrics, then you can start comparing them together. You may have to normalize across the teams, but with enough iterations you can compare and predict.

So some things to show would be a histgram of how much the team burns down a day. Highlight what kind of deviation we have in our progress per day. Over multiple sprints you can maybe see what the first third, middle third, and final third look like. Can we characterize “At OSC, we typically see this kinda of result in the thirds of the project?” Hey, does this feed into our Waterfall projects in 3 weeks?

First week is requirements solidification. Second week is development. Third week is testing and polish.

Can we figure out how to predict the results for sprint 3? We could do this for sprint 1 and 2 and predict sprint 3.

We measure progress per day as a ratio, and then sum it over a week. With that progress per week, then we can see what a sprint 3 would do.

Yesterday Arin Sime and I saw Jean Bauer’s presentation on developing the Early American Foreign Service Database that is part of her disseration attempting to identify why the Foreign Service was so effective when it was intrinsicaly a very chaotic unorganized period for what we now call the Foreign Service.

Jean gave a great brief on some of the challenges the Foreign Service faced in the late 1790’s, and walked us through her database for modeling the social network of the Foreign Service. Source code available at http://projectquincy.rubyforge.org. She modeled both traditional relational data such as individuals and where they are posted, letters they wrote to who, as well as more GIS oriented data, such as what states belong to what countries and what countries were part of what empires (ie British Empire, Ottoman Empire).

One of the things that struck me is that while the schema was very clean, all done up to 3rd normal form standards with good use of primary and foreign keys, what I didn’t like was that many of the relationship tables such as the assignments table for various folks had begin and end dates. And it struck me that many of the queries that you would want to issue are all based around time spans. I want all individuals posted to Barbadoes between 1770 and 1780. Which can easily be done with a select statement that looks something like:

SELECT * from INDIVIDUALS i, POSTINGS p, LOCATION l where l.name = 'barbados' and l.id = p.location_id and p.individual_id = i.id and p.start_date > 1770 and p.end_date < 1781

However it strikes me that this extra section at the end “p.start_date > 1770 and p.end_date < 1781” is really the key part of the relationships, potentially on the level of the primary keys and foreign keys for identifying the data. How would I model a single posting to a country that was interrupted for a small time period? I guess the traditional answer is to have another table called something like “Residency” that would track actual time periods as well but that seems icky and roundabout adding extra layers of complexity to my data model.

So part of me wonders, why don’t the Primary Keys and Foreign Keys contain the time axis of data as well as the relationship between individual tables? Why don’t we have a Time Based Database for storing data whose primary relationship is time? Document Based Databases have joined our pantheon of data stores, so what would a Time Based Database look like? I think it would deal with various calendars very simply.. Tell me when Rome was founded in Gregorian or Julian calendar. Or, what year did Rosa Park’s stage her protest according to the Jewish calendar?

I can imagine that a TBDB would need to understand the degree of confidence/accuracy in dates. If I selected when all the worlds cities were founded, it would natively let me know that Rome was founded middle of the 8th century BC, but that New York was founded in 1624 AD.

And lastly, it would naturally handle time zones. No need to tediously convert all dates into UTC, just use whatever and know the TBDB would handle the selects.

I can see this being very useful for data such as flight schedules, and figuring out the best route between cities. How do I get from Charlottesville Virginia to Brisbane Australia with multiple stops on the way? A TBDB would handle all that icky date handling so that a stop in Dubai wouldn’t overlap with the departing flight from Dubai.

While there don’t seem to be any candidates yet, feel free to comment on what you would want in your TBDB!

Every year people tend to spend the first month or two making promises they are going to break a few moments, maybe weeks later.  We tend to lack resolve and have seasonal habits.  We act nicer in December to our fellow man (woman), unless it comes to taking the last tickle-me elmo off the shelf, then its every person for themselves.  We pull out the big grill  that you probably won’t use for the rest of the summer for labor day weekend that you got for xmas to outshine your neighbor’s 2k stainless steal beast he got the year before, all the while making tim the tool man tailor grunts while cooking, .  And of course, its January, its time for a new diet fad (this year it seems to be acri berry diets that boasts you don’t need to really diet or exercise, all thanks to Opera Winfrey having a show about the most nutritional foods with acri berry being #1).

But I digress.  However I do have some suggestions for the new year because more than likely people are more open to ideas this time of year and hopefully maybe one or two will stick, thereby making your life that much more productive and a little easier, or fun.

Optimize the way you work on a pc, Pimp your desktop.

You don’t need to be a power user like on linux or have to buy a mac to have an intuitive and shiny UI (User Interface, in this case, desktop) to be productive. Its taken me some time, but I’ve finally found the tools that really help clean up the desktop. 

First off clean up your desktop, its rare that you really need those icons, especially if you have windows vista with the new start menu. If you like the mac’s quicksilver application, but like me, tend to work on a windows os, you can try launchy, skylight, or slickrun. Right now I prefer skylight since its written using .NET WPF and its extendable.  With Alt + Spacebar, the window appears and you can quickly find whatever program you want. 

Take time to find a desktop wall paper you like, deviant art is good place to start.  People in general tend to enjoy working with things that are appealing to eye. Everyone has different tastes and different styles of work flow. Experiment not only with the wall paper, but also where the taskbar is placed and what toolbars are shown, etc.  Also play with the visual styles of windows as well. Right now I tend to like the vista glass with a lil black in it.  If you have xp, you can get vista glass.  

Add lifehacker.com to your RSS feed, they always have some pretty nifty desktop ideas, tools, and even lists of pimped/tricked out desktops, and just good productivity tips in general. 

If you need icons or if you like the Mac’s Dock, give rocket dock a try, its pretty customizable and you can replace the icons for whatever your trying to open. 

Gadgets.  If you have windows vista, you can try the gallery for finding some halfway useful gadgets. If you’re using xp or want better gadgets on vista you can try yahoo widgets or google gadgets.  Though gadgets are a cool desktop concept, they have really yet to take off with usefulness or high end eye candy, but you might find one or two to meet your needs.

Practical websites

Live Mocha – This is actually social networking language learning website that really has nothing to do with coffee other than the colors of the website. Its a decent learning tool and not to mention a good way of finding pen pals or other people interested in the same languages that you can practice with.

Live Strong – Calorie Counter that has tons of foods, and exercises taken into account? check. Must have information on the latest diet fads?  check. Must have it as an IPhone or ITouch application? check.  Must be a social network so you don’t feel alone… loser, i mean check.

Mint – looking for a good online free way to manage/budget/invest your money with a ton of awesome tools and that you can take with you on your IPhone or Itouch, then mint.com is the place for you

Life Hacker – need ways to be productive, find new gadgets that are worth the money, or just like to learn new ways of doing things…

Mozy – online data/documents/files backup that is unlimited for just 5$ a month isn’t bad, especially if you’re the tech geek for your family. If they have under 2 gigs of stuff, its free.

Remember the milk – Into the GTD (getting things done) way of doing things? Then check out remember the milk for your task lists. They also provide google gears, iphone application, gadgets and many other ways of entering and keeping track of your tasks. 

Doing dot .net?

Pimp your color themes…    Visual Studio Color Themes and Is your IDE hot or not?

T4 (Text Template Transformation Toolkit) – Visual Studio has code generation that has been there for while under the covers, not really used, but very useful.  Why are people spending money on code gen tools when you already spent a fortune on visual studio and comes with this gem?

Gallio – The one stop shop for running your unit tests and it comes packages with the killer mbunit 3.0. 

Moq – need an mock library that doesn’t suck, isn’t confusing and uses lambda? then take a look moq (mock you). 

Test Driven .net – I’ve been using this on my opensource project.  This is one of the very few add ons that I install for visual studio. It lets you run tests, code coverage reports and other nifty things right inside of Visual Studio. A time saver.

Reflector – Red gate now owns and updates reflector, but its still free, want to look at the source code for a dll, check this out.

Linqpad – need to write some linq queries and want to make sure they execute correctly, check out linqpad.

General Coding.

E-Texteditor – The power of text mate on windows.  This is pretty my replacement of note pad.  If you have mac font envy, you can grab monaco for windows. Or you can just use Consolas.

Stylizer – css styling for ajax applications, even with tools like firebug, can be a pain, cause that doesn’t exactly help you with IE and it doesn’t always help with doing heavy ajax applications with pops up and such. Enter stylizer (also coming to macs soon), which lets you style with a real time preview of how it would show in the actual browser (IE and firefox). 

FTP

winscp – you can ftp down, edit in e-texteditor and ftp the file right back when you save changes to the file.  nifty. 

filezilla – slick fast opensource ftp client and server. 

Instant Messenger?

pidgin – all your fav instant messengers and even ones you don’t know wrapped into one program.  

skype – voip, phone calls, sms texts, chats.  Definitely the professional grade instant messenger/ communications program. calls are even encrypted.

meebo – you online web page instant messaging client, in case your away and can’t download a client. 

Challenge.

Anything that will help you to be more productive always costs something. It could be money, but most of the time, its costs time. But invest some time now that might reap huge benefits later.  So I challenge you to take some time, get organized, find new efficient ways of doing things that fit your way of life and if you know of anything, programs or otherwise, feel free to add those in the comments below.