rbrooks/Iris-WLED

Watch

Fork 0

Mirror the SQLite backups to an off-site target (#18) #123

Merged

claude-bot merged 1 commit from feat/18-offsite-backup into main

2026-09-04 23:07:05 +00:00

claude-bot commented

2026-09-04 23:01:49 +00:00

Contributor

Closes #18.

The existing snapshots live on the same volume as the database they protect — a defence against a bad import, not against losing the host. Two targets, both off by default: a mounted directory (BACKUP_REMOTE_PATH) for a NAS or second disk, and any S3-compatible endpoint (BACKUP_S3_*) for MinIO, Backblaze or AWS. Restore is documented in docs/troubleshooting.md.

Credentials are environment-only and never enter the settings table, so they can't reach the config bundle from #23 — a file whose whole purpose is to be committed to a repository.

A real bug in my first draft

Remote retention originally matched on an "iris_" filename prefix. Every daily backup is named iris_daily_..., so that prefix also matches the dailies — manual pruning would have deleted daily backups early. It now shares the exact _KIND_GLOBS pattern local pruning uses, deliberately the same rule rather than a parallel one that has to be kept in step. There's a test for it.

Why no boto3

S3 here is three operations — PUT, list, DELETE — and boto3 with botocore is ~50MB in an image we deliberately slimmed in #60. So the SigV4 signature is computed against httpx directly.

That's normally a bad trade, and it's worth being explicit about why it's acceptable here: SigV4 is request authentication, not encryption. There's no key exchange and no confidentiality property to get subtly wrong — a mistake produces a 403 and a loudly broken feature, not a silently insecure one.

Two things the review caught, both mine

The security hook flagged stdlib XML parsing, correctly. S3 listing bodies are remote input and xml.etree expands entities, so a hostile or compromised endpoint could answer with a billion-laughs payload. I'd argued against dependencies for this feature — but defusedxml is ~30KB against boto3's ~50MB, so refusing that would have been false economy. Added, plus a size bound on the body.

I asserted a SigV4 test vector from memory and it was wrong. The test failed; I "fixed" it by parameterising the service to match AWS's iam example, and it failed again — because the hex constant itself was fabricated. I then fetched AWS's actual documentation: it states the derivation as an algorithm and gives no worked numeric example, so there is no vendor vector to cite.

The chain I implemented matches the spec exactly (AWS4+secret → date → region → service → aws4_request), but the test now reimplements those four steps longhand from the specification rather than calling the module and comparing it to itself. I removed the service parameter that existed only to serve the bogus test, and confirmed the remaining test catches a region/service swap.

I'd rather say plainly that there's no vendor vector than leave a comment claiming an authority the test doesn't have.

Failure behaviour

An unreachable target never fails a backup. The local snapshot is already written and is what actually stands between the user and data loss, so a dead NAS is logged and the bulk operation that triggered it still completes. That's documented in troubleshooting, along with the corollary: check the logs rather than assuming silence means success.

Uploads to a mounted path go to a staging name and are renamed, so a reader never sees a half-copied backup that looks complete.

Verification

551 backend tests pass (20 new), ruff, mypy clean
uvx pre-commit run --all-files passes
Chain-order mutation checked by hand: swapping region and service fails the derivation test

Nothing here touches the controller or the frontend.

🤖 Generated with Claude Code

Closes #18. The existing snapshots live on the same volume as the database they protect — a defence against a bad import, not against losing the host. Two targets, both **off by default**: a mounted directory (`BACKUP_REMOTE_PATH`) for a NAS or second disk, and any S3-compatible endpoint (`BACKUP_S3_*`) for MinIO, Backblaze or AWS. Restore is documented in `docs/troubleshooting.md`. Credentials are environment-only and never enter the settings table, so they can't reach the config bundle from #23 — a file whose whole purpose is to be committed to a repository. ### A real bug in my first draft Remote retention originally matched on an `"iris_"` filename prefix. Every daily backup is named `iris_daily_...`, so that prefix **also matches the dailies** — manual pruning would have deleted daily backups early. It now shares the exact `_KIND_GLOBS` pattern local pruning uses, deliberately the same rule rather than a parallel one that has to be kept in step. There's a test for it. ### Why no boto3 S3 here is three operations — PUT, list, DELETE — and boto3 with botocore is ~50MB in an image we deliberately slimmed in #60. So the SigV4 signature is computed against httpx directly. That's normally a bad trade, and it's worth being explicit about why it's acceptable here: **SigV4 is request authentication, not encryption.** There's no key exchange and no confidentiality property to get subtly wrong — a mistake produces a 403 and a loudly broken feature, not a silently insecure one. ### Two things the review caught, both mine **The security hook flagged stdlib XML parsing**, correctly. S3 listing bodies are remote input and `xml.etree` expands entities, so a hostile or compromised endpoint could answer with a billion-laughs payload. I'd argued against dependencies for this feature — but `defusedxml` is ~30KB against boto3's ~50MB, so refusing *that* would have been false economy. Added, plus a size bound on the body. **I asserted a SigV4 test vector from memory and it was wrong.** The test failed; I "fixed" it by parameterising the service to match AWS's `iam` example, and it failed again — because the hex constant itself was fabricated. I then fetched AWS's actual documentation: it states the derivation as an algorithm and gives **no worked numeric example**, so there is no vendor vector to cite. The chain I implemented matches the spec exactly (`AWS4+secret → date → region → service → aws4_request`), but the test now reimplements those four steps longhand from the specification rather than calling the module and comparing it to itself. I removed the `service` parameter that existed only to serve the bogus test, and confirmed the remaining test catches a region/service swap. I'd rather say plainly that there's no vendor vector than leave a comment claiming an authority the test doesn't have. ### Failure behaviour An unreachable target **never fails a backup**. The local snapshot is already written and is what actually stands between the user and data loss, so a dead NAS is logged and the bulk operation that triggered it still completes. That's documented in troubleshooting, along with the corollary: check the logs rather than assuming silence means success. Uploads to a mounted path go to a staging name and are renamed, so a reader never sees a half-copied backup that looks complete. ### Verification - 551 backend tests pass (20 new), `ruff`, `mypy` clean - `uvx pre-commit run --all-files` passes - Chain-order mutation checked by hand: swapping region and service fails the derivation test Nothing here touches the controller or the frontend. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit

2026-09-04 23:01:49 +00:00

Mirror the SQLite backups to an off-site target (#18 )

CI / Dockerfile lint (pull_request) Successful in 7s

Details

CI / Alembic migration check (pull_request) Successful in 38s

Details

CI / Pre-commit hooks (pull_request) Successful in 51s

Details

CI / Python lint & type-check (pull_request) Successful in 1m5s

Details

CI / Frontend lint, test & build (pull_request) Successful in 1m41s

Details

CI / Python tests (pull_request) Successful in 2m32s

Details

CI / Docker build, health smoke & E2E (pull_request) Successful in 1m53s

Details

04cbc5f36b

The existing snapshots live on the same volume as the database they
protect, which defends against a bad import but not against losing the
host. Approved schemes are real curation effort, so they deserve to
survive the disk.

Two targets, both off by default: a mounted directory
(BACKUP_REMOTE_PATH) for a NAS or second disk, and any S3-compatible
endpoint (BACKUP_S3_*) for MinIO, Backblaze or AWS. Retention is applied
remotely using the same globs local pruning uses -- deliberately the same
rule rather than a parallel one, because the two kinds are not
prefix-separable: every daily backup is named iris_daily_..., so matching
on an "iris_" prefix would have swept the dailies into the manual
retention and deleted them early. That was a real bug in the first draft.

Credentials are environment-only and never enter the settings table, so
they cannot reach the config bundle from #23 -- a file whose whole purpose
is to be committed to a repository.

No boto3. S3 here is PUT, list, DELETE, and boto3 with botocore is ~50MB
in an image deliberately slimmed in #60, so the SigV4 signature is
computed against httpx directly. That is normally a bad trade and it is
worth being explicit about why it is acceptable: SigV4 is request
authentication, not encryption, so a mistake produces a 403 and a loudly
broken feature rather than a silently insecure one.

defusedxml is added (~30KB) for the listing response. Avoiding that would
have been false economy: the body is remote input and the stdlib parser
expands entities, so a hostile or compromised endpoint could answer a
listing with a billion-laughs payload. The response is size-bounded too.

On the signing test: I first asserted a hex vector recalled from memory,
which was wrong, and then "fixed" it by matching AWS's iam example, which
was also wrong. AWS's current documentation states the derivation as an
algorithm and gives no worked numeric example, so there is no vendor
vector to cite. The test now reimplements the four-step HMAC chain
longhand from the specification instead of calling the module and
comparing it to itself, and it catches a region/service swap.

An unreachable target never fails a backup: the local snapshot is already
written and is what stands between the user and data loss, so a dead NAS
is logged and the operation that triggered it still completes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

claude-bot merged commit a250b7e038 into main

2026-09-04 23:07:05 +00:00

claude-bot deleted branch feat/18-offsite-backup

2026-09-04 23:07:06 +00:00

claude-bot referenced this pull request

2026-09-04 23:08:35 +00:00

Off-site / object-storage backup target for the SQLite DB #18