Skip to content

Upgrading SerOps

How to safely upgrade your SerOps installation to a new version.


Pre-Upgrade Checklist

Before upgrading, complete the following steps:

  • Back up the database: Follow the Backup & Restore procedure, or run the quick-start rotation script
    ./scripts/backup.sh
    
  • Note the current version: Check your .env file for SEROPS_VERSION
  • Review the changelog: Read the Changelog for breaking changes, new required environment variables, or migration notes
  • Check the compatibility matrix below — some upgrades require intermediate stops
  • Review new environment variables: Compare .env.example with your .env for any new settings
  • Notify your team: Inform users of a brief maintenance window

Always back up your database before upgrading.

Database migrations are irreversible. If an upgrade introduces problems, you will need a backup to roll back. See Backup & Restore for the full procedure including secrets and PVCs.


Version compatibility matrix

SerOps follows Semantic Versioning. Same major version means safe rolling upgrades; a major bump means review the migration notes.

From To Supported path Notes
1.0.x 1.0.y (patch) Direct rolling upgrade Zero-downtime with Helm
1.0.x 1.x.0 (minor) Direct upgrade Read changelog for new envs
1.x 2.0 Via last 1.x release Never skip the last minor of a major
Older than N-2 minor Latest Via intermediate release Migrations accumulate; step through

Kubernetes / Helm chart compatibility

Since 1.1.0 the Helm chart is versioned with the app: chart 1.2.0 deploys SerOps 1.2.0.

Chart version App version Kubernetes tested
Same as the app (1.1.0 and later) 1.1.0 and later 1.28, 1.29, 1.30, 1.31
0.1.x 1.0.x 1.28, 1.29, 1.30, 1.31

We test the newest patch of each supported Kubernetes minor against helm template + helm install on a KinD cluster. Older Kubernetes minors may work but aren't gated by CI.

PostgreSQL compatibility

SerOps version Postgres major Notes
1.x 14, 15, 16, 17 16 recommended; bundled image = 16

Upgrading Postgres itself (e.g. 15 → 16) is out of scope for the SerOps upgrade path — dump + reload into the new Postgres major, then run the SerOps upgrade. See Backup & Restore.

Redis compatibility

SerOps version Redis version
1.0.x 6.x, 7.x

Redis holds only ephemeral state (Celery queue, cache) so version upgrades are safe with a brief restart.


Upgrade notes

Releases that need something from you after upgrading. Migrations run on their own (docker compose up / the Helm migrate job).

1.2.2: security release

  • New migrations: Data 0051–0053, Projects 0015, Organisations 0013, Users 0012.
  • Re-run Setup Webhook on each repository (SCM) binding: bindings now have a webhook secret, and the receiver refuses payloads without it. Until you do, pushes from that repository are ignored.
  • Django admin is off by default: set DJANGO_ADMIN_ENABLED=True to use /api/admin/.
  • TRUSTED_PROXY_COUNT (default 1): the number of reverse proxies in front of SerOps, used for login rate limits and audit IPs. Raise it if several proxies append to X-Forwarded-For.
  • Superusers no longer reach other organisations' data through the API: sign in as a member of an organisation to work in it.
  • Helm: /metrics is no longer published on the ingress. Scrape it in the cluster (serviceMonitor.enabled: true), or add the path back with basic auth (see Self-metrics).
  • Organisation logos must be uploaded again (they were never stored durably before); profile photos are imported automatically.

Automated Upgrade

The simplest way to upgrade is using the included upgrade script:

./scripts/upgrade.sh [version]

For example:

# Upgrade to a specific version
./scripts/upgrade.sh 1.5.0

# Upgrade to the latest version
./scripts/upgrade.sh

The script automatically:

  1. Creates a pre-upgrade database backup
  2. Pulls new images for the specified version
  3. Restarts the backend and frontend (PostgreSQL and Redis keep running)
  4. Reports which images were updated

Manual Upgrade Procedure

If you prefer to upgrade manually:

Step 1: Back Up the Database

./scripts/backup.sh

Step 2: Update the Version (Optional)

If upgrading to a specific version, edit .env:

SEROPS_VERSION=1.5.0

Step 3: Pull New Images

docker compose pull

Step 4: Restart Services

docker compose up -d

Note

Database migrations run automatically when the backend container starts. There is no need to run them manually.

Step 5: Verify the Upgrade

# Check container status
docker compose ps

# Check application logs for errors
docker compose logs --tail 50 backend

# Run the health check
./scripts/healthcheck.sh

Log in and verify core functionality:

  • Dashboard loads with data
  • Batch list and analysis results display correctly
  • AI chat responds to queries
  • Notifications are being delivered

Rolling Back

If an upgrade causes issues, roll back to the previous version:

Step 1: Stop the Application

docker compose down

Step 2: Restore the Database

docker exec -i serops-postgres pg_restore \
  -U serops -d serops --clean --if-exists \
  < backups/serops_db_YYYYMMDD_HHMMSS.dump

Step 3: Start the Previous Version

Set the previous version in .env and start:

# Edit .env to set SEROPS_VERSION to the previous version
SEROPS_VERSION=1.4.0 docker compose up -d

Step 4: Verify the Rollback

Confirm the application is running on the previous version and all data is intact.

Warning

Database rollback restores data to the point of the backup. Any data created between the backup and the rollback will be lost.


Migration Troubleshooting

Migrations run automatically on container startup. If you encounter issues:

Migration Fails on Startup

Check the backend container logs for the specific error:

docker compose logs backend

Check Migration State

# See which migrations Django thinks are applied
docker exec serops-backend python manage.py showmigrations

# Check actual database columns
docker exec serops-backend python manage.py dbshell -c "\d tablename"

State Desync

If Django reports a migration as applied but the database schema does not reflect it, you may need to fake the migration backward and re-run:

# Mark migration as unapplied (does NOT touch the database)
docker exec serops-backend python manage.py migrate app_name 0003 --fake

# Re-run the migration
docker exec serops-backend python manage.py migrate app_name

unique=True Constraint Errors

If a migration adding a unique constraint fails because of duplicate values:

  1. Identify duplicate rows in the database
  2. Resolve duplicates manually (delete or update)
  3. Restart the backend container: docker compose restart backend

Duplicate Index Errors

If Django's deferred SQL creates duplicate indexes:

django.db.utils.ProgrammingError: relation "index_name" already exists

This typically happens when AddField and AlterField appear in the same migration. Check if the index already exists and fake the migration if so:

docker exec serops-backend python manage.py migrate app_name 000X --fake

Then restart the backend container.


Zero-Downtime Upgrades

For deployments that require high availability, consider the following approach:

  1. Run migrations before deploying new code -- most Django migrations are backward-compatible with the old code
  2. Use a blue-green deployment -- bring up a new set of containers alongside the old ones, switch traffic after verification
  3. Use a load balancer health check -- point your load balancer at /api/health/ and let it drain connections from old containers

Tip

Database migrations that add columns (without NOT NULL constraints) or add new tables are safe to run while the old code is still serving traffic. Migrations that rename or drop columns require a maintenance window.


Next Steps