rbrooks/Circa
Watch
1
0
Fork
You've already forked Circa
 0
Code Issues 66 Pull requests 1 Projects Releases 2 Packages Wiki Activity Actions

Create the production deployment and packaging story #50

New issue
Open
opened 2026-07-28 04:59:58 +00:00 by claude-bot · 1 comment
claude-bot commented 2026-07-28 04:59:58 +00:00
Copy link

Context

The app currently runs from a development checkout: a venv, uvicorn --reload, and a Vite
dev server. There is no deployment story, which means there is no supported way to
actually run Circa for its intended purpose over the years a family collection takes to work through.

Scope

A reproducible deployment: containerization, production configuration, and an upgrade path.

Implementation notes

  • Containerize backend, worker, and frontend, with a compose file bringing them up together.
    The worker is a separate process from the API and must be deployed as such.
  • Serve the built frontend properly in production rather than through the Vite dev server.
  • Run migrations as an explicit deployment step, not implicitly at application startup —
    implicit migration on boot makes rollback and multi-process startup hazardous.
  • Persist the SQLite database and the storage root on volumes, and document backup
    implications for both.
  • Health and readiness endpoints suitable for orchestration; a health endpoint already exists.
  • Reverse proxy and TLS guidance, given that the OAuth redirect URI must be HTTPS in production.
  • Document resource expectations and an upgrade procedure including the rollback path.
  • Verify the whole thing on the Linux dev server at 10.1.1.14, which already runs Docker.
  • Pin versions for reproducibility, and let Renovate keep them current.

Done when

  • Backend, worker, and frontend run from containers via a single compose file
  • Migrations are an explicit, documented deployment step
  • Data and storage persist across container replacement
  • TLS and reverse proxy setup are documented
  • Upgrade and rollback procedures are documented and tested
  • The deployment is verified end to end on the dev server

References

  • README.md (development-only instructions today)
  • AGENTS.md (dev server at 10.1.1.14, Docker available)

Depends on: #14 (fail fast on insecure configuration).

## Context The app currently runs from a development checkout: a venv, `uvicorn --reload`, and a Vite dev server. There is no deployment story, which means there is no supported way to actually run Circa for its intended purpose over the years a family collection takes to work through. ## Scope A reproducible deployment: containerization, production configuration, and an upgrade path. ## Implementation notes - Containerize backend, worker, and frontend, with a compose file bringing them up together. The worker is a separate process from the API and must be deployed as such. - Serve the built frontend properly in production rather than through the Vite dev server. - Run migrations as an explicit deployment step, not implicitly at application startup — implicit migration on boot makes rollback and multi-process startup hazardous. - Persist the SQLite database and the storage root on volumes, and document backup implications for both. - Health and readiness endpoints suitable for orchestration; a health endpoint already exists. - Reverse proxy and TLS guidance, given that the OAuth redirect URI must be HTTPS in production. - Document resource expectations and an upgrade procedure including the rollback path. - Verify the whole thing on the Linux dev server at `10.1.1.14`, which already runs Docker. - Pin versions for reproducibility, and let Renovate keep them current. ## Done when - [ ] Backend, worker, and frontend run from containers via a single compose file - [ ] Migrations are an explicit, documented deployment step - [ ] Data and storage persist across container replacement - [ ] TLS and reverse proxy setup are documented - [ ] Upgrade and rollback procedures are documented and tested - [ ] The deployment is verified end to end on the dev server ## References - `README.md` (development-only instructions today) - `AGENTS.md` (dev server at `10.1.1.14`, Docker available) Depends on: #14 (fail fast on insecure configuration).
claude-bot added this to the v1.0.0 milestone 2026-07-28 04:59:58 +00:00
claude-bot commented 2026-08-06 18:43:42 +00:00
Author
Copy link

Substantially done across two repos. Leaving this open on the last box, which is the one that needs a real run.

This repo (8239ae9, c45d4a7, released as v0.3.1): a backend image and a frontend image, and a publish job that pushes both on a version tag. The deployment itself lives in Rhoving/iac-repo #262 / PR #263 — a circa-ansible project on that repo's role pattern, deploying dev to docker-test and production to docker-host from the same pinned tag.

Against the "Done when" list

  • Backend, worker, and frontend run from containers via a single compose file. Three containers, two images: the worker is the same backend image with a different command, because it is a separate process by design and is what executes the OCR jobs ingest queues. The compose file is in iac-repo, deliberately — a production compose in this repo would be a second source of truth for something the fleet already models as data.
  • Migrations are an explicit, documented deployment step. Not at startup: two processes start from one image, which is exactly where implicit-on-boot bites. The iac role quiesces, snapshots the database into a dated directory, runs the upgrade, then brings the stack up behind a health gate.
  • Data and storage persist across container replacement. Host bind mounts rather than named volumes, so down -v cannot take the two things worth keeping, and a future backup job can reach them without going through Docker.
  • TLS and reverse proxy setup are documented. Both instances are entries in the fleet's Caddy routing table, without forward-auth — Circa does its own OIDC login and has its own role model.
  • Upgrade and rollback procedures are documented and tested — with one deliberate exception worth stating: there is no automatic image rollback on a failed health gate. Migrations have already run by then, so restoring the old image without the old schema would put old code on a new database and report success. It fails loudly and names the snapshot directory instead.
  • The deployment is verified end to end on the dev server. Not yet. Needs the two Authentik applications and a play run, both of which are @rbrooks's.

Verified rather than assumed

  • tesseract --version → 5.5.0 inside the backend image, and the full OCR chain across two containers: ingest a front/back pair, worker claims the job, evidence row lands. Without it every job #144 queues fails permanently, per photograph, silently.
  • alembic upgrade head from an empty volume → 000 → 017, and the API started on a fresh volume without migrating, confirming migrations really are not implicit.
  • Through nginx: /api/health → 200 with only the backend's headers, SPA fallback serves index.html, /api/nope returns the backend's JSON 404 rather than being swallowed, and a query string arrives byte-for-byte.
  • client_max_body_size 220m. The nginx default is 1 MB and a 4×6 print at 400dpi is about 2.8 — the browser upload path (#142) this deploy exists for would have refused every scan.
  • Forced the backend onto a new container address; nginx kept answering. A literal proxy_pass caches the address for the life of the process, so every upgrade would otherwise leave the frontend 502ing until someone restarted it too.
  • Both images confirmed present in the registry at 0.3.1, with :latest moved.

Two things this turned up

A green run that built nothing. The first v0.3.1 tag run passed every gate and published no image — the runner skipped publish on an if: expression that works in shelf-life but not under this workflow's filtered triggers. Fixed by deciding in the shell against $GITHUB_REF and printing all three ref variables unconditionally (c45d4a7). Recorded because "CI is green" was, for about an hour, a true statement about nothing.

#145 — EXIF extraction fails on every TIFF, found while building the backend image and confirmed on the host, so not container-induced. Worth fixing before a real collection goes in rather than after: EXIF is read at ingest, so anything imported before the fix carries no exif evidence, and re-deriving means re-reading every original.

Substantially done across two repos. Leaving this open on the last box, which is the one that needs a real run. **This repo** (8239ae9, c45d4a7, released as v0.3.1): a backend image and a frontend image, and a `publish` job that pushes both on a version tag. **The deployment itself lives in `Rhoving/iac-repo` #262 / PR #263** — a `circa-ansible` project on that repo's role pattern, deploying dev to `docker-test` and production to `docker-host` from the same pinned tag. ## Against the "Done when" list - [x] **Backend, worker, and frontend run from containers via a single compose file.** Three containers, two images: the worker is the *same* backend image with a different command, because it is a separate process by design and is what executes the OCR jobs ingest queues. The compose file is in iac-repo, deliberately — a production compose in this repo would be a second source of truth for something the fleet already models as data. - [x] **Migrations are an explicit, documented deployment step.** Not at startup: two processes start from one image, which is exactly where implicit-on-boot bites. The iac role quiesces, snapshots the database into a dated directory, runs the upgrade, then brings the stack up behind a health gate. - [x] **Data and storage persist across container replacement.** Host bind mounts rather than named volumes, so `down -v` cannot take the two things worth keeping, and a future backup job can reach them without going through Docker. - [x] **TLS and reverse proxy setup are documented.** Both instances are entries in the fleet's Caddy routing table, without forward-auth — Circa does its own OIDC login and has its own role model. - [x] **Upgrade and rollback procedures are documented and tested** — with one deliberate exception worth stating: there is **no automatic image rollback** on a failed health gate. Migrations have already run by then, so restoring the old image without the old schema would put old code on a new database *and report success*. It fails loudly and names the snapshot directory instead. - [ ] **The deployment is verified end to end on the dev server.** Not yet. Needs the two Authentik applications and a play run, both of which are @rbrooks's. ## Verified rather than assumed - `tesseract --version` → **5.5.0** inside the backend image, and the full OCR chain across two containers: ingest a front/back pair, worker claims the job, evidence row lands. Without it every job #144 queues fails permanently, per photograph, silently. - `alembic upgrade head` from an empty volume → `000 → 017`, and the API started on a fresh volume **without** migrating, confirming migrations really are not implicit. - Through nginx: `/api/health` → 200 with only the backend's headers, SPA fallback serves `index.html`, `/api/nope` returns the backend's JSON 404 rather than being swallowed, and a query string arrives byte-for-byte. - `client_max_body_size 220m`. The nginx default is 1 MB and a 4×6 print at 400dpi is about 2.8 — the browser upload path (#142) this deploy exists for would have refused every scan. - Forced the backend onto a new container address; nginx kept answering. A literal `proxy_pass` caches the address for the life of the process, so every upgrade would otherwise leave the frontend 502ing until someone restarted it too. - Both images confirmed present in the registry at `0.3.1`, with `:latest` moved. ## Two things this turned up **A green run that built nothing.** The first v0.3.1 tag run passed every gate and published no image — the runner skipped `publish` on an `if:` expression that works in shelf-life but not under this workflow's filtered triggers. Fixed by deciding in the shell against `$GITHUB_REF` and printing all three ref variables unconditionally (c45d4a7). Recorded because "CI is green" was, for about an hour, a true statement about nothing. **#145 — EXIF extraction fails on every TIFF**, found while building the backend image and confirmed on the host, so not container-induced. Worth fixing before a real collection goes in rather than after: EXIF is read at ingest, so anything imported before the fix carries no `exif` evidence, and re-deriving means re-reading every original.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
renovate/lock-file-maintenance
v0.3.3
v0.3.2
v0.3.1
v0.3.0
v0.2.0
Labels
Clear labels   
ai
AI / OCR evidence backends

  
backend
Backend / API / services

  
bug
Something is not working

  
docs
Documentation

  
duplicate
This issue or pull request already exists

  
enhancement
New feature

  
frontend
React UI

  
help wanted
Need some help

  
infra
CI, deployment, tooling

  
invalid
Something is wrong

  
question
More information is needed

  
security
Auth, authorization, data safety

  
testing
Automated tests and QA

  
wontfix
This won't be fixed

  
worker
Background jobs and worker runtime

No labels
ai
backend
bug
docs
duplicate
enhancement
frontend
help wanted
infra
invalid
question
security
testing
wontfix
worker
Milestone
Clear milestone
No items
No milestone
v1.0.0
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Circa#50
No description provided.