I was thinking about Eric Lippert’s blog post How many Microsoft employees does it take to change a lightbulb?
So, our customer found a bug in the last released version of our software. For various reason’s it’s only reproducible on one of their customer’s sites (we’re an OEM). It’s an important customer, and our customer couldn’t wait until the next release for the fix (due in a month or so).
It took two of us three days to track down the cause of the bug and the fix resulted in a one line change.
The one line change was in one of the top level assemblies in our shared base library. The change had to be back-ported. We had to create a new branch in our source control system, back port the change, and get a new build of our platform library that is shared between multiple products. We (er, I) decided that we would just drop the DLL into the release build of our software and edit the assembly redirection in our App.config file and provide them with instructions to manually copy the two files (a DLL and the App.config file) into the installation directory of their software.
So, when I got the build, I installed our old version and dropped in the new assembly into the install directory and tried to run the software. As expected it crashed on start-up. So I edited the App.config to point the old version of the assembly to the new version. Start again.. crash … scratch head for few minutes. Since the new assembly references all it’s dependent assemblies, they also need redirection. Twelve (yes twelve) assemblies.
Next write the instructions, send them to Tech Comm for the formal Service Note write up.
Send the instructions to the testers so they can test the patch. Our customer is in Japan, we need to test on a Japanese OS. They’ve also done the localization themselves. We need to test with our test satellite assemblies. They’ve also re-branded our software. We need to test a re-branded, localized version on a Japanese OS.
Next, we need to make sure everything is in change control, get Quality Assurance and Regulatory Affairs to review our docs, prepare the release request, release notes, review the service note prepared by Tech Comm, get Customer Support to execute the Mini Acceptance Test Plan on the release CD, have the release meeting collect signatures, dot the i’s, cross the t’s, and finally send the CD to manufacturing so they can label it, post it to the ftp site and send an email to our external customer.
Maintenance is the most expensive phase of software, ’cause this whole process was cause by a one line change.