fix(webapp): pack parsing off the event loop; a server error isn't a sign-out (#1239)

claude-bot commented

2026-10-30 20:49:30 +00:00

Collaborator

Closes #1239. On prod (26 Oct), fetching both starter packs at once ran CPU-bound pack parsing on the event loop. Both uvicorn workers missed the 5 s healthcheck and were killed, and everyone looked signed out.

Backend (app/reference/import_service.py):

read_meta (in _prepare_system and on the queued path) and read_pack (in import_from_path) run in asyncio.to_thread.
normalise's per-record walk is pure CPU: adapters parse lazily as they're iterated, and every record is sanitised. It's split into a sync _normalise_records run in a thread, with the DB lookups kept in the async prelude. This goes beyond the issue's list, because without it most of the parsing still blocked the loop.
The upload's sha256 (source_record) runs in a thread too.
Routes need no change: they all go through import_pack / import_from_path. The Celery task gets the same change, which is harmless there.
New test test_content_pack_off_loop.py: a slow read_pack runs off the main thread, while a ticker coroutine keeps ticking during the admin import POST.

Frontend:

useAuth: only a 401 from /api/me means signed out. A 5xx or a network failure keeps loading up and retries, backing off from 1 s to 15 s and cancelled on unmount. refreshUser clears the user only on a 401.
retrying is exposed. AuthGuard's wait then says "Quest Board can't reach its server just now. Trying again…" instead of "Signing you in…" for as long as the server is down (added on review).
Tests: a 503 then success, a network failure then success, and the retrying flag. AuthGuard tests cover both texts.

Wording, held for the owner (Wording Review part i, i-003): the retry line above. Don't merge until it's decided.

Checked:

On docker-test: lint clean, version sync and changelog checks OK. Content-pack modules: 126 passed.
Full backend: 10,390 passed, with 1 failure in test_exif_orientation.py[WEBP]. It's unrelated and passed on rerun (222). It's a load flake, worth its own issue if it recurs.
vitest src/hooks + AuthGuard: 40 passed. eslint: 0 errors.

Not covered: the new off-loop test wasn't run against the unfixed code to prove it fails there. export_pack's zip building is still CPU work on the loop, a smaller and separate case.

🤖 Generated with Claude Code

Closes #1239. On prod (26 Oct), fetching both starter packs at once ran CPU-bound pack parsing on the event loop. Both uvicorn workers missed the 5 s healthcheck and were killed, and everyone looked signed out. **Backend (`app/reference/import_service.py`):** - `read_meta` (in `_prepare_system` and on the queued path) and `read_pack` (in `import_from_path`) run in `asyncio.to_thread`. - `normalise`'s per-record walk is pure CPU: adapters parse lazily as they're iterated, and every record is sanitised. It's split into a sync `_normalise_records` run in a thread, with the DB lookups kept in the async prelude. This goes beyond the issue's list, because without it most of the parsing still blocked the loop. - The upload's sha256 (`source_record`) runs in a thread too. - Routes need no change: they all go through `import_pack` / `import_from_path`. The Celery task gets the same change, which is harmless there. - New test `test_content_pack_off_loop.py`: a slow `read_pack` runs off the main thread, while a ticker coroutine keeps ticking during the admin import POST. **Frontend:** - `useAuth`: only a 401 from `/api/me` means signed out. A 5xx or a network failure keeps `loading` up and retries, backing off from 1 s to 15 s and cancelled on unmount. `refreshUser` clears the user only on a 401. - `retrying` is exposed. `AuthGuard`'s wait then says **"Quest Board can't reach its server just now. Trying again…"** instead of "Signing you in…" for as long as the server is down (added on review). - Tests: a 503 then success, a network failure then success, and the `retrying` flag. AuthGuard tests cover both texts. **Wording, held for the owner** (Wording Review part i, i-003): the retry line above. **Don't merge until it's decided.** **Checked:** - On docker-test: lint clean, version sync and changelog checks OK. Content-pack modules: 126 passed. - Full backend: 10,390 passed, with 1 failure in `test_exif_orientation.py[WEBP]`. It's unrelated and passed on rerun (222). It's a load flake, worth its own issue if it recurs. - vitest `src/hooks` + AuthGuard: 40 passed. eslint: 0 errors. **Not covered:** the new off-loop test wasn't run against the unfixed code to prove it fails there. `export_pack`'s zip building is still CPU work on the loop, a smaller and separate case. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 2 commits

2026-10-30 20:49:30 +00:00

fix(webapp): pack parsing off the event loop; only a 401 means signed out (#1239 ) dd70097278

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

fix(webapp): the sign-in wait says when the server can't be reached (#1239 )

CI / Summarisation accuracy eval harness (stub provider) (pull_request) Successful in 53s

Details

CI / Bot/backend version sync (pull_request) Successful in 23s

Details

CI / Which journeys does this change need? (pull_request) Successful in 22s

Details

CI / Does this change touch the recording path? (pull_request) Successful in 25s

Details

CI / Synthetic session harness (no GPU, no LLM) (pull_request) Has been skipped

Details

CI / Backend lint (ruff) (pull_request) Successful in 29s

Details

CI / Feedback site tests (pull_request) Successful in 44s

Details

CI / Bot tests and audit (pull_request) Successful in 2m1s

Details

CI / Docker image build (pull_request) Successful in 2m33s

Details

CI / Frontend (shard 2/2) (pull_request) Successful in 8m29s

Details

CI / Frontend (shard 1/2) (pull_request) Successful in 10m17s

Details

CI / End-to-end journeys (desktop) (pull_request) Successful in 13m18s

Details

CI / Backend migration, tests, and audit (pull_request) Successful in 40m24s

Details

677604a235

A failed /api/me that is not a 401 is retried, and AuthGuard's wait then
says so instead of "Signing you in…" for as long as the server is down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rbrooks added 1 commit

2026-10-30 23:02:13 +00:00

Merge remote-tracking branch 'origin/main' into fix/1239-pack-fetch-off-loop

CI / Summarisation accuracy eval harness (stub provider) (pull_request) Successful in 1m3s

Details

CI / Backend lint (ruff) (pull_request) Successful in 52s

Details

CI / Feedback site tests (pull_request) Successful in 54s

Details

CI / Bot/backend version sync (pull_request) Successful in 25s

Details

CI / Does this change touch the recording path? (pull_request) Successful in 21s