Having been through my own continuous deployment journey with Kanbanery, neuropathologist I’ve become very curious about the experiences of others who have tried to achieve fearless continuous deployment. I haven’t met one yet, but amoung those who have tried one story comes up again and again.
The primary goal of continuous deployment is not to deploy continuously. It is to be able to deploy continuously. Doing that requires a dramatic shift in mindset and practices. In addition to learning to create efficient and stable automated background deployment scripts, teams also need to learn to commit smaller amounts of higher quality work with better automated testing then they might consider possible. Learning to do that is the real value of continuous deployment. The mindset shift required is even harder to achieve. They must learn to trust themselves to deliver the level of quality that is required for continuous deployment. In teams that have made the most progress toward continuous deployment, it is this mindset shift that is the hardest barrier to overcome, and I have a solution to propose.
The scenario I often hear is that teams achieve automated deployment to staging or testing environments, but never become comfortable enough with their standards and practices to take the next step and automate production releases. For these teams, I propose borrowing, as we in software so often do, from manufacturing.
My suggestion is to modify the well-known safety signage of the shop floor, X Days Since Last Accident, which you’ll recognize from the opening to The Simpsons, on which manufacturing teams can proudly increment the number of days without an injury until some failing of safety precautions requires that they re-set the count to zero. If your software team has gotten stuck at the stage of your continuous deployment project at which you’re automatically deploying small stable bits of code to a staging server, consider adding a sign to your team room like this – X Days Since the Last Build We’re Really Glad We Didn’t Deploy to Production. For every day that the team finds nothing embarassing on the staging server, you can proudly increment that count by one. Depending on your level of risk aversion and the nature of your product and business environment, if you are continuously improving, then some day you’ll look at that number and find the courage to take the last step and automate your production deploys.
Let me know what you think in the comments.
Comments on this entry are closed.
I like this idea, but want to play Devil’s advocate.
Part of a continuos deployment plan seems to be a cautious entry into production – deploy such that only a fraction of your traffic goes to the new version. That, coupled with really good monitoring to retreat quickly.
It’s not so much about having zero accidents, but ensuring that whatever accidents you do have are do little damage.
In Ops terms MTTR is more important than MTBF
Eric, you’re absolutely right. Clearly I’m not trying to teach anyone how to do continuous integration in this post, as that would require much more detail and is usually very context-specific. I’m just trying to help teams who are doing it right get over the last hurdle. Minimizing the scale of the risk exposure is a powerful tool in the continuous deployment toolkit, but even so, even great teams still get stuck on their own fear of failure.
I agree that fear does hold a lot of teams back. Love the article!