How to Know When Continuous Delivery is Too 'Continuous'

Continuous delivery is usually a good thing. But, like most good things, it becomes a bad thing in excess. If you deliver too continuously, you risk undercutting the core benefits that continuous delivery is supposed to enable. (And, yes, by the dictionary definition, “continuous” is an absolute, but we’re going by the continuous delivery definition, where continuous is not really continuous.)

Here’s a look at why too much continuous delivery is problematic, and how to tell if you’ve gone overboard in your CI/CD (continuous integration/continuous delivery) process.

The Goals of Continuous Delivery

Continuous delivery refers to a software development and deployment strategy in which application updates are issued in small, frequent batches. It’s the opposite of so-called waterfall delivery, a method in which updates are few and far between, but involve major changes when they do appear.

During the past decade, the DevOps philosophy has enticed about two-thirds of development teams to embrace continuous delivery in one form or another. They typically do this in the hope of achieving several goals:

• Making it faster and easier to push out software updates
• Allowing applications to evolve gradually, instead of foisting major changes on users all at once
• Making better use of development resources by avoiding situations where one development task can’t be completed before another, which leads to delays
• Enabling fast rollbacks to an older application version in the event that something goes wrong with a new release
• Increasing the ability to take advantage of advanced software testing features, like feature flags and A/B testing

It’s worth noting that “continuous” is a relative (and ambiguous) term in this context. There’s no official release frequency that you need to achieve to qualify as doing continuous delivery. Companies like Netflix and Facebook report deploying new releases more than once a day, but for a majority of companies that have embraced DevOps and CI/CD, releasing a few times a week is a more realistic goal to target.

Even just a few releases a month could count as continuous delivery, as long as they introduce small, incremental changes, rather than the monumental updates associated with the waterfall methodology.

How Continuous Is Too Continuous?

To put this another way, the real goal of continuous delivery should be to issue releases steadily and regularly, but not necessarily as fast as possible. Indeed, there is danger in trying to maximize your deployment rate just for the sake of maximizing it. There is such a thing as too much continuous delivery.

How do you know if you’re releasing too often? Here are some signs.

1. There is no real difference between your releases.

Just because you’ve written some new code somewhere within your app doesn’t mean you need to release it right away. If you release every time you update anything, you increase your risk (because more releases means more opportunities for something to go wrong) without a good reason. You also waste resources by deploying more releases than are necessary.

You need not have a major update in each new release, but each release should introduce a new feature, bug fix or other meaningful change of some kind--minor though it may be. If you can fit multiple meaningful changes into each release while still releasing frequently and steadily, that’s even better.

2. You deploy according to an arbitrary schedule.

Along similar lines, releases should only be deployed when there is a reason to deploy a new release, not because it’s a certain day of the week or time of day. Deciding that you’re going to deploy whatever working code you have once a day at midnight, or every Tuesday and Friday, for example, would be a mistake. It may make your delivery schedule continuous and consistent, but it’s pointless if the releases are not tied to meaningful application change.

3. You deploy a new release before you’re confident in the stability of your last one.

Even if you deploy multiple times per day, you shouldn’t deliver a new release until you’re at least relatively confident that the previous one is working as expected. If you do, you run the risk that you’ll discover a problem with your production code, but be unable to roll it back efficiently because you won’t know which release introduced it.

If you let your drive to delivery continuously take precedence over your commitment to stability, you’re doing it wrong.

On a related note, you should never deploy faster than you can complete software testing for each new release. Even if testing takes longer than you would like, thorough testing is more important than rapid releases.

4. Your users are confused by frequent application changes.

Even if each new release is thoroughly vetted and works exactly as you intended, changes that come too frequently may be a bad thing for your users. Users don’t like when their interface changes constantly or they are inundated with minor new features every time they refresh your app. It’s true that, in general, continuous delivery leads to happier end users by putting new functionality into their hands faster. But you shoot yourself in the foot if you issue changes so fast that your users are overwhelmed or confused.

Conclusion

When it comes to continuous delivery, everyone’s mileage varies. Just because one company deploys twice a day doesn’t mean you have to do the same to succeed at DevOps. On the contrary, striving to maximize deployment rates at the expense of software quality, the end user experience and your ability to link releases to positive application change does more harm than good.