test(backend): assert process_audio threads the game-system context into summarise; share same_task_session (#457) #646

Merged
claude-bot merged 2 commits from test/457-process-audio-harness into main 2026-09-14 04:51:09 +00:00
Collaborator

Test-only. Nothing under webapp/backend/app/ changes.

1. One same_task_session fixture

Celery task bodies open their own session with task_session(). The db fixture never really commits (commit is aliased to flush, then rolled back), so without a patch a task can't see any row the test set up. It returns early and the test passes without reaching the code under test. That's how #400 and #401 went wrong.

The patch was hand-copied into 14 files:

  • 10 module-level same_task_session fixtures
  • 7 per-test closures in test_reminder_tasks.py / test_previously_on_recap.py
  • 2 more under a different name (_yield_db) in test_availability.py and test_summary_recompose.py, which a search for the fixture name missed

All were behaviourally identical: each patches only app.database.task_session with a bare yield db. They're now one fixture in conftest.py, with a docstring explaining the silent-pass failure it prevents.

Refactor mutation check: with the shared fixture made a no-op, 14 tests across 5 converted files went red; they're green again with it restored. The at-risk tests in test_availability.py guard this by construction: with no patch, the poller finds no session and call_count == 1 fails.

2. summarise gets the resolved game-system context through process_audio

test_process_audio_system_context.py drives process_audio.run(...) end to end, with only the provider calls stubbed (the seam from test_hallucination_filter.py). It intercepts audio_service.summarise and asserts on the argument it was actually called with.

  • Free-form campaign: receives "Homebrew Steampunk System" unchanged.
  • Linked registry campaign: receives the dnd5e prompt_hint, and not the raw column.

Both cases run the real resolve_system_prompt_context against an in-memory Campaign (and, for the linked case, GameSystem); neither branch needs a database. Together they tell resolving the context apart from simply passing campaign.game_system through.

Mutation check: passing None at the summarise(...) call site in reminder_tasks.py fails both tests (assert None == 'Homebrew Steampunk System', assert None == 'D&D 5th Edition characters are built from…'); restored, git diff webapp/backend/app is empty.

3. The gap note is gone

The "summarise() call site" note in test_system_aware_prompts.py is removed, and the module docstring now points to the new test.

Verification

  • Full backend suite: 3476 passed, 13 skipped.
  • The two follow-up files rerun: 83 passed.
  • ruff format --check / ruff check clean under CI's pinned 0.4.4.

Against #457's acceptance criteria

  • A shared fixture patches task_session; the hand-copied duplicates are removed.
  • A test drives process_audio end to end with external seams stubbed. The seam fixture in test_failed_windows.py already did this; the new module adds a case aimed at summarise.
  • summarise receives the resolved context, for linked and free-form campaigns.
  • The note in test_system_aware_prompts.py is removed.

Item 4 (sequencing guards for #421/#425/#431/#432/#397) was optional and isn't in this PR.

Closes #457

🤖 Generated with Claude Code

Test-only. Nothing under `webapp/backend/app/` changes. ## 1. One `same_task_session` fixture Celery task bodies open their own session with `task_session()`. The `db` fixture never really commits (`commit` is aliased to `flush`, then rolled back), so without a patch a task can't see any row the test set up. It returns early and the test passes without reaching the code under test. That's how #400 and #401 went wrong. The patch was hand-copied into **14 files**: - 10 module-level `same_task_session` fixtures - 7 per-test closures in `test_reminder_tasks.py` / `test_previously_on_recap.py` - 2 more under a different name (`_yield_db`) in `test_availability.py` and `test_summary_recompose.py`, which a search for the fixture name missed All were behaviourally identical: each patches only `app.database.task_session` with a bare `yield db`. They're now one fixture in `conftest.py`, with a docstring explaining the silent-pass failure it prevents. **Refactor mutation check:** with the shared fixture made a no-op, 14 tests across 5 converted files went red; they're green again with it restored. The at-risk tests in `test_availability.py` guard this by construction: with no patch, the poller finds no session and `call_count == 1` fails. ## 2. `summarise` gets the resolved game-system context through `process_audio` `test_process_audio_system_context.py` drives `process_audio.run(...)` end to end, with only the provider calls stubbed (the seam from `test_hallucination_filter.py`). It intercepts `audio_service.summarise` and asserts on the argument it was actually called with. - **Free-form campaign:** receives `"Homebrew Steampunk System"` unchanged. - **Linked registry campaign:** receives the dnd5e `prompt_hint`, and not the raw column. Both cases run the **real** `resolve_system_prompt_context` against an in-memory `Campaign` (and, for the linked case, `GameSystem`); neither branch needs a database. Together they tell resolving the context apart from simply passing `campaign.game_system` through. **Mutation check:** passing `None` at the `summarise(...)` call site in `reminder_tasks.py` fails both tests (`assert None == 'Homebrew Steampunk System'`, `assert None == 'D&D 5th Edition characters are built from…'`); restored, `git diff webapp/backend/app` is empty. ## 3. The gap note is gone The "summarise() call site" note in `test_system_aware_prompts.py` is removed, and the module docstring now points to the new test. ## Verification - Full backend suite: **3476 passed, 13 skipped**. - The two follow-up files rerun: 83 passed. - `ruff format --check` / `ruff check` clean under CI's pinned 0.4.4. ## Against #457's acceptance criteria - [x] A shared fixture patches `task_session`; the hand-copied duplicates are removed. - [x] A test drives `process_audio` end to end with external seams stubbed. The `seam` fixture in `test_failed_windows.py` already did this; the new module adds a case aimed at `summarise`. - [x] `summarise` receives the resolved context, for linked and free-form campaigns. - [x] The note in `test_system_aware_prompts.py` is removed. Item 4 (sequencing guards for #421/#425/#431/#432/#397) was optional and isn't in this PR. Closes #457 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Adds test_process_audio_system_context.py, which drives process_audio.run
through a provider-stubbed seam (copied from test_hallucination_filter.py)
with audio_service.summarise intercepted instead of stubbed, and asserts the
game-system argument it receives for both a free-form campaign and one linked
to the registry (D&D 5e's prompt_hint). Both cases let the real
resolve_system_prompt_context run against an in-memory Campaign — no database
needed, since the free-form branch reads campaign.game_system directly and
the linked branch reads campaign.game_system_ref once it is set, satisfying
sqlalchemy.inspect's loaded check without a session. Verified with a
mutation check against reminder_tasks.py (restored after, no diff under
webapp/backend/app). Removes the corresponding gap note in
test_system_aware_prompts.py.

Also consolidates the ten near-identical module-level `same_task_session`
fixtures (all patched app.database.task_session identically) into one shared
fixture in conftest.py, and converts the equivalent inline closures in
test_reminder_tasks.py and test_previously_on_recap.py to use it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
test(backend): fold the last two task_session copies into the shared fixture (#457)
All checks were successful
CI / Summarisation accuracy eval harness (stub provider) (pull_request) Successful in 55s
CI / Backend lint (ruff) (pull_request) Successful in 27s
CI / Bot/backend version sync (pull_request) Successful in 27s
CI / Bot tests and audit (pull_request) Successful in 1m58s
CI / Frontend tests, audit, and build (pull_request) Successful in 2m27s
CI / Docker image build (pull_request) Successful in 4m46s
CI / Backend migration, tests, and audit (pull_request) Successful in 8m42s
CI / Synthetic session harness (no GPU, no LLM) (pull_request) Successful in 16m23s
7951cce5c2
test_availability.py and test_summary_recompose.py carried the same
patch under a different local name (_yield_db), so a search for
same_task_session missed them. Same behaviour, now the shared fixture.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
claude-bot deleted branch test/457-process-audio-harness 2026-09-14 04:51:09 +00:00
Sign in to join this conversation.
No description provided.