All runbooks
Under construction · working draft requires a release manifest

Technical runbook

Upgrade and rollback

Controlled version change without relying on the latest tag and without irreversible retry in production.

Audience
Implementer, operations, DBA, product owner and change committee
Outcome
The upgrade has a verified input state, backup, test, decision point, and realistic rollback.

01

Before the change

  1. Verify the supported path from the current version and the compatibility of all enabled services.
  2. Review breaking changes, configuration, schema, API, and template changes.
  3. Back up your data and perform or document a current recovery test.
  4. Download new images before the maintenance window and record digests.
  5. Establish the last safe moment to return and the person responsible for the go/no-go decision.

02

Execution

  1. Stop new inputs or switch the system to runbook mode.
  2. Verify that the integration queues are empty or safely preserved.
  3. Run the migrations in the prescribed order and monitor their results.
  4. Launch services with new digests and perform technical and process smoke tests.
  5. Only resume full operation after acceptance.

03

Rollback is not just an old image

If the upgrade changed the data or schema, rolling back the container alone is not enough.

  • Use only the described reverse migration, or restore a consistent backup.
  • Return a compatible set of all services together.
  • Consider the documents and messages created after the backup.
  • After returning, repeat the smoke test and reconciliation of integration queues.
  • Record the incident and the decision to return in the change log.