Upgrades and Migrations

Upgrade Principles

Treat three things separately:

• Binary or image version: controls code behavior and APIs.
• Database migrations: control schema and may be irreversible.
• Configuration and deployment templates: control network surfaces, secrets, dependencies, cache, cluster, and observability.

Caution

An image rollback is not always a system rollback. If a migration changed the database schema, verify that the old version can still read it.

Preflight

1. Read release notes or change logs for the target version.
2. Back up PostgreSQL and production secrets.
3. Restore production-like data into a test environment, start the target version, and verify automatic migrations and business reads/writes.
4. Run synctv config validate.
5. Save synctv config show --output yaml as the current effective config snapshot.
6. Compare against the Configuration Index and Full Configuration Example.
7. For Helm, run helm lint and helm template.
8. For Compose, run docker compose config.

Database Migrations

SyncTV runs embedded SQLx migrations automatically during service startup. Migration state is stored in _sqlx_migrations, and SQLx uses database-side locking so the same migration set is not executed concurrently by multiple replicas. Multi-replica startup does not require a Job to manually serialize migrations, and administrators do not need to decide whether an upgrade needs migrations.

synctv db migrate remains available as an explicit operations command for release preflight, CI, test environments, and troubleshooting. It runs the same embedded migrations that service startup runs automatically.

Check status:

synctv db status

Optional preflight:

synctv db migrate

Compose example:

docker compose run --rm synctv synctv db migrate

Running synctv db migrate separately in production is a release-process choice, not a correctness requirement. Its value is observability and failure isolation: it can surface dirty migrations, checksum drift, or database privilege issues before the new version receives traffic. If it is not run separately, service startup still applies pending migrations automatically.

Docker Compose Upgrade

1. Pull the new image.
2. Back up database and secrets.
3. Run docker compose config.
4. Optionally run synctv db migrate as a release preflight.
5. Restart the service. Startup applies pending migrations automatically.
6. Check /health/ready, login, WebSocket, providers, and metrics.

Common commands:

docker compose pull
docker compose up -d
docker compose logs -f synctv

Helm Upgrade

Render Check

helm lint ./helm/synctv
helm template synctv ./helm/synctv -f values.production.yaml

Check that HTTP Service and gRPC Service are separate; HTTP Ingress and gRPC Ingress are separate; the gRPC Ingress sets nginx.ingress.kubernetes.io/backend-protocol: "GRPC"; and secrets, volumes, env, ServiceMonitor, or VMServiceScrape match expectations.

Release

helm upgrade --install synctv ./helm/synctv \
  -n synctv \
  -f values.production.yaml

After release:

kubectl -n synctv rollout status deploy/synctv
kubectl -n synctv logs deploy/synctv --tail=200
kubectl -n synctv get ingress,svc,pod

Rolling Updates and Long Connections

SyncTV has WebSocket, HTTP-FLV, HLS, gRPC streams, and streaming proxy requests. Rolling updates need enough drain time.

Recommendations:

• Align server.shutdown_drain_timeout_seconds with Kubernetes terminationGracePeriodSeconds.
• Make readiness fail quickly once shutdown starts.
• Do not set terminationGracePeriodSeconds shorter than the drain timeout.
• For livestream or heavy WebSocket deployments, release during low-traffic windows.

Configuration Change Strategy

Do not copy the full default template blindly. First save the effective configuration:

synctv --config /etc/synctv/synctv.yaml config show --output yaml > effective-before.yaml

Then modify only values you intentionally own:

• Secrets and secret file paths.
• Database, Redis, SMTP, OAuth2, WebAuthn.
• Ports, domains, CORS, trusted proxies.
• Cluster, shared HLS storage, file-backed slice cache.
• Metrics, log format, and rate limits.

Rollback

Before rolling back, classify the situation:

Situation	Guidance
Only config or image changed and the target version has no new migrations	Roll back through the deployment system
The new version started and ran automatic migrations, but the old version supports the new schema	Roll back image and watch logs closely
The new version ran an irreversible migration	Prefer forward fix; if rollback is required, restore database backup
Secret was rotated incorrectly	Restore the old secret and restart service

After rollback:

synctv db status
synctv config validate
curl -fsS http://localhost:8080/health/ready

Post-Release Validation

Confirm at least:

• New pods or containers are ready.
• /health/ready succeeds.
• Root/admin users can log in.
• 2FA-enabled users can complete MFA.
• OAuth2, WebAuthn, and email codes work if enabled.
• Room WebSocket sync works.
• Provider direct and proxy paths work.
• Metrics are scraped.
• Logs do not show sustained migration, Redis, database, or provider errors.