feat(ops): back up by default, somewhere the operator can see (#414) #446

Merged

claude-bot merged 1 commit from fix/414-backups-on-by-default into main 2026-08-29 22:26:48 +00:00

Conversation 0 Commits 1 Files changed 7 +198 -9

claude-bot commented 2026-08-29 18:33:15 +00:00

Contributor

Copy link

First half of #414. Addresses the part a self-hoster feels immediately; the off-host half is deliberately separate, for a reason set out below.

What changed

Enabled by default. Backups shipped off, and production ran for months with backup_logs empty and not one scheduled dump ever taken — every dump that existed was made by hand, mid-incident, by someone who already knew the feature was there. Off is the wrong default for the one feature whose whole purpose is to be present before anyone thinks to want it. An explicitly stored enabled: false is still honoured; this changes the default, not a decision anyone made.

Named volume → ./backups bind mount. The old location lives under /var/lib/docker/volumes — exactly the path people don't include when they back up a machine. Operators back up /home and their project directories, so the dumps sat on the same disk and outside whatever backup habit already existed, findable only via docker volume inspect. ./backups sits beside the compose file and can be handed to rclone, restic or rsync by someone who knows nothing about Docker.

Ownership handled, so there's nothing to prepare. Docker creates a missing bind-mount source as root, and the app runs as appuser (uid 999) — verified on prod. Without a fix, the first scheduled backup on a fresh install fails on EACCES, silently, forever. The migrate one-shot now does it: it already runs before backend and worker, so the long-running services keep their non-root posture rather than starting as root to chown a directory. Alembic uses its own database credentials, so root in a throwaway container grants nothing extra.

Why the audio volume isn't here

The issue asks for "an equivalent backup path for the audio volume", and I started building it before concluding it would be close to useless in this form.

Copying audio into ./backups puts a second copy on the same disk, doubling consumption — potentially by gigabytes per session — while protecting against nothing the database dump doesn't already cover. The failure audio needs protecting from is losing the host, and only an off-host copy addresses that.

So audio coverage isn't a peer of the database dump; it's part of the off-host work, and I'd rather ship that once than ship a local copy that costs disk and buys nothing. #414 stays open for it.

What this explicitly does not do

Stated in docs/OPERATIONS.md as a table rather than left to be discovered — the media volume, the audio volume, and any off-host copy are all still uncovered. A backup on the same disk survives a bad migration, not a dead machine.

That honesty matters here: #429 was a backup that reported success and could not be restored, and the lesson is that a backup you believe in but haven't checked is worse than none.

Upgrading

Existing dumps stay in the orphaned quest-board_backups volume. Nothing moves or deletes them — relocating somebody's only backups unasked is not something a deploy should do quietly. OPERATIONS.md has the copy command and says to verify the files before removing the old volume.

Verification

• Mutation-checked: with the old default restored, test_admin_get_backup_config_default fails on a plain assert False.
• Two new tests: a fresh install with no stored row backs up; an explicit opt-out is still respected.
• 1,409 backend tests pass; ruff clean at CI's pinned 0.4.4.
• docker compose config validates, with all three services (migrate, backend, worker) resolving ./backups → /app/backups and migrate as root.

🤖 Generated with Claude Code

First half of #414. Addresses the part a self-hoster feels immediately; the off-host half is deliberately separate, for a reason set out below. ## What changed **Enabled by default.** Backups shipped off, and production ran for months with `backup_logs` empty and not one scheduled dump ever taken — every dump that existed was made by hand, mid-incident, by someone who already knew the feature was there. Off is the wrong default for the one feature whose whole purpose is to be present *before* anyone thinks to want it. An explicitly stored `enabled: false` is still honoured; this changes the default, not a decision anyone made. **Named volume → `./backups` bind mount.** The old location lives under `/var/lib/docker/volumes` — exactly the path people don't include when they back up a machine. Operators back up `/home` and their project directories, so the dumps sat on the same disk *and* outside whatever backup habit already existed, findable only via `docker volume inspect`. `./backups` sits beside the compose file and can be handed to `rclone`, `restic` or `rsync` by someone who knows nothing about Docker. **Ownership handled, so there's nothing to prepare.** Docker creates a missing bind-mount source as root, and the app runs as `appuser` (uid 999) — verified on prod. Without a fix, the first scheduled backup on a fresh install fails on `EACCES`, silently, forever. The `migrate` one-shot now does it: it already runs before backend and worker, so the long-running services keep their non-root posture rather than starting as root to chown a directory. Alembic uses its own database credentials, so root in a throwaway container grants nothing extra. ## Why the audio volume isn't here The issue asks for "an equivalent backup path for the audio volume", and I started building it before concluding it would be close to useless in this form. Copying audio into `./backups` puts a second copy **on the same disk**, doubling consumption — potentially by gigabytes per session — while protecting against nothing the database dump doesn't already cover. The failure audio needs protecting from is *losing the host*, and only an off-host copy addresses that. So audio coverage isn't a peer of the database dump; it's part of the off-host work, and I'd rather ship that once than ship a local copy that costs disk and buys nothing. #414 stays open for it. ## What this explicitly does not do Stated in `docs/OPERATIONS.md` as a table rather than left to be discovered — the media volume, the audio volume, and any off-host copy are all still uncovered. **A backup on the same disk survives a bad migration, not a dead machine.** That honesty matters here: #429 was a backup that reported success and could not be restored, and the lesson is that a backup you believe in but haven't checked is worse than none. ## Upgrading Existing dumps stay in the orphaned `quest-board_backups` volume. Nothing moves or deletes them — relocating somebody's only backups unasked is not something a deploy should do quietly. `OPERATIONS.md` has the copy command and says to verify the files before removing the old volume. ## Verification - **Mutation-checked**: with the old default restored, `test_admin_get_backup_config_default` fails on a plain `assert False`. - Two new tests: a fresh install with no stored row backs up; an explicit opt-out is still respected. - **1,409 backend tests pass**; ruff clean at CI's pinned 0.4.4. - `docker compose config` validates, with all three services (`migrate`, `backend`, `worker`) resolving `./backups` → `/app/backups` and `migrate` as root. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 2 commits 2026-08-29 18:33:16 +00:00

chore: allowlist the test-fixture secret so the scanner stays trustworthy (#132) ... 0e42e4490b

The fleet security scanner has reported exactly one finding for some time: a
`generic-api-key` at webapp/backend/tests/test_config.py:8. It is
`VALID_SECRET = "0123456789abcdef0123456789abcdef"`, in a file whose docstring
reads "Tests for startup validation of placeholder and weak secrets" — a
deliberately fake value that exists to prove the app ACCEPTS a well-formed key
while rejecting weak ones. It has never been a live credential.

The point is not the noise, it is what permanent noise does. A scanner that
always says "1 secret" is a scanner nobody reads, and this week #428 found a
real credential in plaintext at rest that had gone unnoticed for months. The
next finding needs to mean something.

Allowlisted by path rather than by an inline `gitleaks:allow` comment, because
the finding is reported against a historical commit (f25ee12096, 2026-07-14)
and an inline marker in the working tree would not suppress it.

NOTE: the scanner lives in the fleet repo, not here (`security-scan.yml` is not
in .forgejo/workflows). This config only takes effect if gitleaks is invoked
without an explicit --config, in which case it picks up .gitleaks.toml from the
scanned repository. If the fleet runner passes its own config, the allowlist
has to move there instead — worth confirming on the next scheduled scan.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

feat(ops): back up by default, somewhere the operator can see (#414) ... b3a4ed49da

Two changes, both aimed at the same failure: an instance that has no backups
because nobody found the setting.

Enabled by default. It shipped off, and production ran for months with
backup_logs empty and not one scheduled dump ever taken — every dump that
existed had been made by hand, mid-incident, by someone who already knew the
feature was there. Off is the wrong default for the one feature whose entire
purpose is to be present before anyone thinks to want it. An explicitly stored
`enabled: false` is still honoured; this changes the default, not a decision.

Moved from a named volume to a ./backups bind mount beside the compose file.
A named volume lives under /var/lib/docker/volumes, which is exactly the path
people do not include when they back up a machine — operators back up /home
and their project directories. So the dumps sat on the same disk AND outside
whatever backup habit already existed, discoverable only via
`docker volume inspect`. ./backups is visible and can be handed to rclone,
restic or rsync by someone who knows nothing about Docker.

The bind mount needs its ownership fixed, because Docker creates a missing
source directory as root and the app runs as appuser (uid 999) — otherwise the
first scheduled backup on a fresh install fails on EACCES, silently, forever.
The migrate one-shot does it: it already runs before backend and worker, so
the long-running services keep their non-root posture instead of starting as
root to chown a directory. Alembic uses its own database credentials, so root
in a throwaway container grants nothing.

Existing installs keep their dumps in the orphaned quest-board_backups volume.
Deliberately not migrated automatically — relocating somebody's only backups
unasked is not something a deploy should do quietly. OPERATIONS.md has the
copy command and says to verify before removing the old volume.

What this does NOT do, stated plainly in the docs rather than left to be
discovered: the media and audio volumes are still uncovered, and there is no
off-host copy. A backup on the same disk survives a bad migration, not a dead
host. Off-host and audio remain open on #414 — see the PR for why a local copy
of the audio volume would be near-pointless and belongs with the off-host work.

Verified by mutation: with the old default restored,
test_admin_get_backup_config_default fails on a plain `assert False`.
1,409 backend tests pass; compose validated with `docker compose config`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

rbrooks force-pushed fix/414-backups-on-by-default from b3a4ed49da to aac322b693 2026-08-29 19:18:36 +00:00 Compare

claude-bot referenced this pull request 2026-08-29 19:19:50 +00:00

fix(bot): never sweep a directory that holds a real recording (#399) #447

claude-bot merged commit e1819f80c9 into main 2026-08-29 22:26:48 +00:00

claude-bot referenced this pull request from a commit 2026-08-29 22:26:48 +00:00

Merge PR #446: back up by default, somewhere the operator can see (#414)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

backend

Issue related to the backend

  

bot

Issue related to the bot

  

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

frontend

Issue related to the frontend

  

help wanted

Need some help

  

invalid

Something is wrong

  

question

More information is needed

  

recording

Issue related to the recording system

  

security

Security hardening / vulnerability work

  

wontfix

This won't be fixed

No labels

backend

bot

bug

duplicate

enhancement

frontend

help wanted

invalid

question

recording

security

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

rbrooks/Quest-Board!446

No description provided.