For a time, my job involved improving the user experience of a Windows desktop application that had been in development without qualified design input for two years. Given that a beta release was imminent, I decided to give the application the papercut treatment, cleaning up visual assets, hiring visual designers to freshen the identity, reverting disgustingly-skinned controls to their default appearance, correcting typos, removing technical jargon, etc.

Initially, each refinement took an inordinate amount of time; it took a good week from the time I specified the simplest change to when the change was running on my laptop for review. I evaluated our development workflow and found that we scored a low 3 on the Joel Test. Rather than recount all of the depressing details, I'll present a few choice morsels of schadenfreude:

  • There was no preexisting peer review process, meaning that basic tools and cultural norms required for agile design were missing.

  • Source was kept in a 3GB perforce repo containing years of deprecated projects, build artifacts, and the skeleton of a programmer everyone thought had left for grad school in 2006.

  • The build process required two versions of Visual Studio, and it wasn't until months after I had joined that I could reliably build the product on my laptop.

  • 'Automated' builds were run by a 2,000 line script that nobody understood, and took 45 minutes to run on a beefy server. This same script also served as a dumping ground for random maintenance code, half of which was commented out. It was like one of those medical oddity museums but with really bad Perl instead of pickled heads.

Fast, easy feedback from anyone.

I wanted to iterate much faster than one cycle per week. My personal goal was to provide design feedback within ten minutes of code change (bear in mind that we're a distributed team). My broader objective was for non-technical collaborators to be able to test the product as it was being developed. This may be the norm for web products, but desktop software is not often seen as amenable to real-time feedback, especially from remote, non-technical team members such as visual designers.

Decentralize

First of all, we migrated from Perforce to Git, abandoning our 3GB-per-branch code graveyard for a shiny, new 100MB repo with cheap branches. This meant that developers could commit, share, and collaborate on features without merging unfinished work into the product.

Next, I made product development visible to all members of the company by hosting our code on GitHub and signing up for HipChat. GitHub integrates with HipChat, so new branches, merge requests, comments, issues, etc. are announced, with links, right under our noses. Previously, few members of our organization knew how to install the Perforce client, tunnel into our VPN, and review changes to the product; now, anyone logged into our company chat can monitor development as it happens. For example, every morning, I open HipChat and see everything my overseas colleague worked on while I was asleep:

GitHub activity in HipChat - Original image has been lost

Automate

Once we had a branch-based workflow with high visibility of work-in-progress, it was time to focus on automating builds for review.

I used Rake and the excellent albacore library for building .NET projects, superseding the 2,000-line ancient Perl script with a 67-line Rakefile written in a declarative DSL. At this point, as long as you had two versions of Visual Studio, Git, and Ruby installed; and you had Bash knowledge and you'd configured your public key properly on GitHub, you could test work in progress like this:

$ git fetch && git checkout some-feature && rake build run

Setting up this build environment was still a manual, error-prone process, and using it required too much technical knowledge. To make it simpler, I created a preconfigured virtual machine with everything required to obtain and build the product. Using VirtualBox, anyone in the company could build the product in a few steps:

  • Install VirtualBox.

  • Download and run the virtual build machine.

  • Double-click the 'Build' script on the desktop.

After this, building the product required only a few clicks, and it could be done by non-technical collaborators on any platform. One of the designers we work with uses OS X, and now he can easily review his work in vivo. There are still a few drawbacks, however:

  • Downloading a 10GB virtual machine isn't fun, not even once.

  • The product should build no more than once per commit, not once per commit per reviewer.

  • The review process was still pull-based, in that the reviewer had to manually pull and build new changes before a review.

Automate More

I setup a Jenkins server to build every branch on every push, and I wrote an Express app that comments on open pull requests with links after every build:

Jenkins build comment on pull request - Original image has been lost

Now, every open pull request has a link to a build of its latest commit. Builds happen incrementally, meaning that pull requests are updated within a couple minutes of a push. This is finally starting to feel slick!

The final piece I automated was obtaining builds for review. Although links on pull requests worked fairly well, I wanted to be able to open new builds immediately after being notified of changes in chat, without navigating a pull request or downloading anything manually. I configured Jenkins to copy successful builds into our company's shared Dropbox folder, so everyone has the latest successful build on each branch automatically synced to their desktop.

Iterate

Now if I see a pull request, or even just a new commit announced in chat, I can simply double-click the build and take a look; anyone in the company who reports an issue can verify its fix; and even non-technical remote collaborators who don't use Windows can run the product and participate in tight feedback loops.

If you're building a product on a distributed team, with or without non-technical collaborators, I strongly recommend adopting some of these automations!