LAB
DevelopmentLAB

A Rollback Button Is Not a Recovery Plan: Include Data Changes in Releases

Published:

7 min read

Before approving a release that changes persistent data, ask which version can be restored and whether it can read what the new version has written. Replacing application code does not restore the previous data state. Restoring a backup may exclude legitimate work completed after that backup was taken. The release plan therefore needs to cover code, configuration, data and integrations, with an understandable recovery decision for each failure condition.

The objective is not to prohibit change or attach generic restoration instructions. It is to establish the real rollback boundary and exercise it in an authorized environment. If safe reversal is unavailable, the decision owner needs to know before deployment and consider containment or a forward fix rather than relying on a button that cannot address the problem.

Separate four different recovery actions

Code rollback runs a previous application version. Configuration rollback restores earlier values or routing. Data restoration recovers a saved state, with consequences for later records. A forward fix addresses the problem in a new release. A recovery may combine these actions, but they are not interchangeable.

AWS discusses rollback safety in terms of compatibility between coexisting versions and testing movement in both directions, rather than merely testing each version on its own. That principle helps define the evidence needed here; it does not mean every system should copy a provider-specific design. AWS on rollback safety.

Change Acceptance question before release
New field How does the old version behave when reading and writing it?
Changed state meaning Can every consumer still interpret it correctly?
Existing-data conversion How are unconverted or incorrectly converted records detected?
Removed field or route What proves all consumers have stopped depending on it?
New external interaction What happens to operations already sent if code is rolled back?

Map readers and writers

Naming the database is insufficient. Identify applications, background jobs, reports and integrations that read or write the changing data. A nightly report may remain on old behavior after the user interface has changed. Approval from the application owner does not establish that an external provider can interpret a new state.

Cloud Architecture Design & Review can clarify dependencies and recovery boundaries. Required application changes belong in App Development, particularly when temporary compatibility requires changes to reading or writing behavior. Those are implementation tasks, not free footnotes at the end of a deployment checklist.

Hypothetical example: splitting an order state

Suppose an ordering system uses “in progress” for both preparation and handover to shipping. A new release needs to distinguish the two stages. A staff interface, a nightly job and an external integration all read the state. This is a fictional design scenario, not an instruction to modify a production system.

If the new application immediately writes new values, the old application may be unable to interpret them after rollback. The proposed plan first introduces a transitional version that understands both representations without immediately removing the old one. It then tests all readers and writers before activating the new behavior.

A successful interface test is not enough: the nightly job and integration must behave as agreed. Acceptance includes an older order, an order created during transition and an exceptional record that cannot be converted confidently. The exception is held for review rather than assigned a guessed value.

After activation, the available rollback target is the transitional version that understands newly written data, not any historic application build. Removing the old representation is a later decision requiring its own evidence and the end of the agreed rollback window. This staged example describes a decision pattern; the appropriate implementation still depends on the actual data model and system constraints.

A backup needs a restoration exercise and a business decision

Establish when the backup was taken, what it includes, who may restore it and where restoration has been tested. Request verification using permitted data in an isolated environment, with differences from production documented. Do not copy sensitive records merely to make rehearsal convenient, and do not test restoration over the production database without specific authorization.

Most importantly, decide how legitimate work after the backup would be handled. Can it be replayed? How will duplication be prevented? Who reconciles the results? Avoid promising a recovery duration or amount of data loss without evidence, an exercise and agreement. Dependencies on another provider must be visible to the decision owner rather than hidden in a technical note.

Make the recovery decision operational

The authorized owner should know when to stop further rollout, which observation requires containment and how an application defect differs from a suspected data-integrity issue. The plan should not depend on an on-call person inventing a solution under pressure. It also should not give implementers blanket permission to alter customer records.

Infrastructure as Code & CI/CD can connect release stages to approved gates and recovery steps. Software Quality Assurance can exercise the specified scenarios: mixed versions, partial transition, rollback and data completeness. Screenshots alone are weaker than a test record identifying version, environment, inputs and outcome.

A practical acceptance checklist

  • Map all relevant readers and writers of the changing data.
  • State the oldest permitted rollback version and why that boundary exists.
  • Test transition and reversal with records created during deployment, not only older data.
  • Verify conversion results, record counts and exceptions without exposing sensitive details.
  • Define restoration and reconciliation of later work where needed.
  • Name the decision owner, executor, escalation route and authority limits.
  • Record untested areas and unresolved conditions that block release approval.

Avoid treating this as a paperwork exercise. The purpose of the checklist is to establish what the owner can safely decide when the release does not behave as expected. An untested claim should remain an assumption, not be promoted into a recovery commitment.

Frequently asked questions

Does every release need a data-recovery plan?

Assess every release. One that does not affect persistent data may need a simpler plan, but adding a state or changing a message format can affect consumers that are not visible in the main interface.

Can rollback be tested against an empty database?

Not adequately when the risk involves data written by the new version. Include authorized examples representing transition states and exceptions.

Is a forward fix evidence that the recovery plan failed?

No. It may be the responsible choice when rollback would make matters worse, provided the boundary and decision are understood and containment has been considered.

What should we supply before scoping?

Provide the proposed data change, reader and writer inventory, current deployment approach and available restoration evidence. Scope follows these dependencies, not a general assurance that a backup exists.

Reference

AWS: ensuring rollback safety during deployments, reviewed 26 October 2026. The example and acceptance checklist are original scoping guidance.

DevelopmentLAB

Need help with this?

Explore LAB's Infrastructure as Code & CI/CD service.