[Ops] Run database migrations automatically and ordered on deploy #89

New issue

Closed

opened 2026-07-14 19:47:27 +00:00 by claude-bot · 0 comments

claude-bot commented

2026-07-14 19:47:27 +00:00

Contributor

Context

The documented upgrade flow is docker compose up -d --build followed by a separate manual make migrate-safe (README.md:89-90, docs/OPERATIONS.md:88-89). Nothing enforces ordering: depends_on for backend (docker-compose.yml:49-53) and worker (:72-76) gates only on db/redis health, so both boot and serve against the old schema until the operator remembers to migrate.

Current behavior

Between up and migrate-safe, the app runs on a mismatched schema — new code querying columns/tables that don't exist yet, surfacing as 500s. A fresh-clone first boot also requires the manual step before the app works at all.

Fix / Spec

Add a one-shot migrate service to docker-compose.yml:
- Uses the backend image; command runs the existing guarded preflight then alembic upgrade head, using DATABASE_MIGRATE_URL.
- restart: "no", depends_on: db: condition: service_healthy.
Give backend and worker (and beat) depends_on: migrate: condition: service_completed_successfully in addition to their current conditions.
Keep make migrate-safe as the manual + pre-migration-backup flow for major upgrades. Decide and document one of:
- the auto path does NOT take the pre-migration pg_dump (state this clearly in OPERATIONS.md and recommend migrate-safe for majors), or
- the migrate service performs the dump itself when backup config exists.
Belt-and-suspenders (optional but preferred): a startup schema-revision check in the backend — compare alembic_version against the code's head revision and refuse to serve (clear log naming alembic upgrade head / make migrate-safe) on mismatch.
Verify docker-compose.prod.yml overlays cleanly (the migrate service must exist/behave there too).

Acceptance criteria

Fresh clone: docker compose up -d --build reaches healthy with a fully migrated schema, zero manual steps.
Upgrade test: seed a volume with an old-schema database, start the new stack — migrations run before backend serves; backend/worker never run against the old schema.
make migrate-safe still works and is documented for major upgrades.
Docs updated (README.md, docs/OPERATIONS.md) to reflect the new flow and the backup decision.

References

README.md:89-90, docs/OPERATIONS.md:88-94
docker-compose.yml:49-53 (backend depends_on), :72-76 (worker depends_on)

Filed from the July 2026 full-project review.

## Context The documented upgrade flow is `docker compose up -d --build` followed by a **separate manual** `make migrate-safe` (`README.md:89-90`, `docs/OPERATIONS.md:88-89`). Nothing enforces ordering: `depends_on` for backend (`docker-compose.yml:49-53`) and worker (`:72-76`) gates only on db/redis health, so both boot and serve against the old schema until the operator remembers to migrate. ## Current behavior Between `up` and `migrate-safe`, the app runs on a mismatched schema — new code querying columns/tables that don't exist yet, surfacing as 500s. A fresh-clone first boot also requires the manual step before the app works at all. ## Fix / Spec 1. Add a one-shot **`migrate` service** to `docker-compose.yml`: - Uses the backend image; command runs the existing guarded preflight then `alembic upgrade head`, using `DATABASE_MIGRATE_URL`. - `restart: "no"`, `depends_on: db: condition: service_healthy`. 2. Give `backend` and `worker` (and `beat`) `depends_on: migrate: condition: service_completed_successfully` in addition to their current conditions. 3. Keep `make migrate-safe` as the manual + pre-migration-backup flow for major upgrades. **Decide and document one of:** - the auto path does NOT take the pre-migration pg_dump (state this clearly in OPERATIONS.md and recommend migrate-safe for majors), or - the migrate service performs the dump itself when backup config exists. 4. Belt-and-suspenders (optional but preferred): a startup schema-revision check in the backend — compare `alembic_version` against the code's head revision and refuse to serve (clear log naming `alembic upgrade head` / `make migrate-safe`) on mismatch. 5. Verify `docker-compose.prod.yml` overlays cleanly (the migrate service must exist/behave there too). ## Acceptance criteria - [ ] Fresh clone: `docker compose up -d --build` reaches healthy with a fully migrated schema, zero manual steps. - [ ] Upgrade test: seed a volume with an old-schema database, start the new stack — migrations run before backend serves; backend/worker never run against the old schema. - [ ] `make migrate-safe` still works and is documented for major upgrades. - [ ] Docs updated (`README.md`, `docs/OPERATIONS.md`) to reflect the new flow and the backup decision. ## References - `README.md:89-90`, `docs/OPERATIONS.md:88-94` - `docker-compose.yml:49-53` (backend depends_on), `:72-76` (worker depends_on) _Filed from the July 2026 full-project review._

claude-bot added the

backend

enhancement

labels

2026-07-14 19:47:33 +00:00

claude-bot added this to the v3.4.0 - Reliability, Operations & Self-Hosting milestone

2026-07-14 19:47:34 +00:00