fix(webapp): a suggested fact's quote must say the change, not only mention the page (#1280) #1336

Merged

claude-bot merged 1 commit from fix/1280-quote-must-say-it into main

2026-11-04 23:29:01 +00:00

claude-bot commented

2026-11-04 21:45:31 +00:00

Collaborator

Part of #1280. Found by the dev rehearsal of 4.7.0-rc2.

What dev showed. The past-sessions pass over 44 real sessions (176 pages asked) gave 8 suggestions, 4 distinct (two Amberwake campaigns). Every quote really was in the transcript, but three of the four didn't say the change:

Sallow Reach, Ruler → Tamsin Reed: "Tamsin Reed doesn't so much greet you as size you up"
Kettlebridge Assay Office, Leader → Petronella Vay: "Petronella Vay, Kettlebridge Assay Office"
The Gull's Wake, Leader → Jory Callander: "Jory Callander doesn't anchor anywhere he hasn't paid to be safe."
The Drowned Lantern, Owner → Orla Brennock: "Orla Brennock is behind it", the one that does.

The change

The prompt asks for the line that itself states the change, containing the new value and the word that says it ("Tamsin Reed rules Sallow Reach now", "they buried Harrow"). A line that only mentions the page or the person isn't enough.
supports() checks it in code. The quote must name the new value (any word of it longer than two letters that isn't a small word) and hold a word for the fact:
- Ruler: rules, lord, mayor, council…
- Leader: leads, head, captain…
- Owner: owns, bought, keeper, behind…
- Allegiances: swore, serves, joined, betrayed…
- Status: a word for that status (died, buried, drowned… / vanished, missing… / alive, rescued…).
Counting: a suggestion that fails is dropped and counted as dropped_unsupported on the run, next to dropped_no_quote.

Tests

The four dev cases are encoded: Orla is kept, the other three are dropped.
Outright statements are kept.
A value without a fitting word, or a fitting word without the value, is dropped.
The counter is recorded on the run.
The test fixtures that leaned on weak quotes now quote lines that say the change.
Results on the dev host: the fact-suggestion, lifecycle and migration tests plus the three reader guards, 70 passed. ruff format and check are clean, and the changelog check is OK.

To see it against the real model: this needs an rc with the change. On dev, clear the 8 rc2 suggestions and their run counts, then rerun the past-sessions pass and compare.

🤖 Generated with Claude Code

Part of #1280. Found by the dev rehearsal of 4.7.0-rc2. **What dev showed.** The past-sessions pass over 44 real sessions (176 pages asked) gave 8 suggestions, 4 distinct (two Amberwake campaigns). Every quote really was in the transcript, but three of the four didn't say the change: - Sallow Reach, Ruler → Tamsin Reed: *"Tamsin Reed doesn't so much greet you as size you up"* - Kettlebridge Assay Office, Leader → Petronella Vay: *"Petronella Vay, Kettlebridge Assay Office"* - The Gull's Wake, Leader → Jory Callander: *"Jory Callander doesn't anchor anywhere he hasn't paid to be safe."* - The Drowned Lantern, Owner → Orla Brennock: *"Orla Brennock is behind it"*, the one that does. **The change** - **The prompt** asks for the line that itself states the change, containing the new value and the word that says it ("Tamsin Reed rules Sallow Reach now", "they buried Harrow"). A line that only mentions the page or the person isn't enough. - **`supports()` checks it in code.** The quote must name the new value (any word of it longer than two letters that isn't a small word) **and** hold a word for the fact: - Ruler: rules, lord, mayor, council… - Leader: leads, head, captain… - Owner: owns, bought, keeper, behind… - Allegiances: swore, serves, joined, betrayed… - Status: a word for that status (died, buried, drowned… / vanished, missing… / alive, rescued…). - **Counting:** a suggestion that fails is dropped and counted as `dropped_unsupported` on the run, next to `dropped_no_quote`. **Tests** - The four dev cases are encoded: Orla is kept, the other three are dropped. - Outright statements are kept. - A value without a fitting word, or a fitting word without the value, is dropped. - The counter is recorded on the run. - The test fixtures that leaned on weak quotes now quote lines that say the change. - Results on the dev host: the fact-suggestion, lifecycle and migration tests plus the three reader guards, 70 passed. ruff format and check are clean, and the changelog check is OK. **To see it against the real model:** this needs an rc with the change. On dev, clear the 8 rc2 suggestions and their run counts, then rerun the past-sessions pass and compare. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit

2026-11-04 21:45:31 +00:00

fix(webapp): a suggested fact's quote must say the change, not only mention the page (#1280 )

CI / Bot/backend version sync (pull_request) Successful in 28s

Details

CI / Backend lint (ruff) (pull_request) Successful in 34s

Details

CI / Feedback site tests (pull_request) Successful in 49s

Details

CI / Does this change touch the recording path? (pull_request) Successful in 22s

Details

CI / Which journeys does this change need? (pull_request) Successful in 22s

Details

CI / Synthetic session harness (no GPU, no LLM) (pull_request) Has been skipped

Details

CI / Summarisation accuracy eval harness (stub provider) (pull_request) Successful in 1m1s

Details

CI / Bot tests and audit (pull_request) Successful in 2m7s

Details

CI / Docker image build (pull_request) Successful in 2m2s

Details

CI / Frontend (shard 2/2) (pull_request) Successful in 9m22s

Details

CI / Frontend (shard 1/2) (pull_request) Successful in 10m37s

Details

CI / End-to-end journeys (desktop) (pull_request) Successful in 13m9s

Details

CI / Backend migration, tests, and audit (pull_request) Successful in 37m41s

Details

f4f33f24cc

The dev rehearsal of 4.7.0-rc2 ran the past-sessions pass over 44 real
sessions: 8 suggestions, every quote in the transcript, but three of the
four didn't say the change ("Tamsin Reed doesn't so much greet you as
size you up" offered as Sallow Reach's Ruler).

- The prompt asks for the line that itself states the change, holding
  the new value and the word that says it.
- supports() checks it: the quote must name the new value and hold a
  word for the fact (rules, leads, owns, swore to…) or, for Status, for
  that status (died, buried, vanished…). Others are dropped and counted
  as dropped_unsupported on the run.
- Tests: the four dev cases (one kept, three dropped), outright
  statements kept, and the counter on the run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude-bot referenced this pull request

2026-11-04 21:45:35 +00:00