Fact suggestions: also ask about pages named in the transcript, not only pages linked to the session #1342

New issue

Closed

opened 2026-11-06 01:04:58 +00:00 by claude-bot · 0 comments

claude-bot commented

2026-11-06 01:04:58 +00:00

Collaborator

Follow-up to #1280, found when the backfill ran on prod after the 4.7.0 deploy (2026-11-06). The owner asked for this to be scheduled.

Problem

suggest_for_session (webapp/backend/app/services/fact_suggestions.py) asks only about wiki pages joined through lore_session_appearances. On prod, "A Merry Band of Thieves" has 63 live pages but only one appearance link in the whole campaign. None of sessions 13, 14 or 16 (1,244 to 1,691 transcript lines each) has a linked page, so the backfill finished with pages_asked: 0 on all three and never asked the AI anything. Dev found candidates only because its pages were linked.

Proposed change

Candidates: the session's linked pages, plus every live, non-proposed page in the campaign with an allowed type (allowed_for) whose title or alias appears in the session's transcript lines. Find mentions with the existing mention_pattern / excerpts_for, not a new matcher.
Order and cap: linked pages first, then the others by number of mentions, with a per-session cap (a constant such as 12) so a long session doesn't turn into dozens of AI calls. Count the pages skipped by the cap as skipped_cap in the counts, so the run records it.
Unchanged: quote validation (match_quote, supports), the written-label guard, the bin and erasure rules, and one pending suggestion per session, page and label.
Short names: a page whose title is a common word, or shorter than about 3 characters, must not match everywhere. Look at how mention_pattern handles this already and add a test for one-word titles.

Tests

A session with no appearance links, where the transcript names a page: it is asked about and gets a suggestion.
The cap: with more mentioned pages than the cap allows, linked pages come first, then the most-mentioned ones, and skipped_cap is counted.
Trashed pages, pages proposed by the bot, and pages from another campaign are never candidates.

After it ships

Re-run the backfill on prod (celery -A app.tasks.celery_app call app.tasks.fact_tasks.suggest_fact_changes_backfill). It skips sessions that already have a run unless that is changed. Check sessions_to_backfill and decide whether the re-run needs a "force" argument for sessions whose earlier run asked about 0 pages. Prod runs only on the owner's say.

Follow-up to #1280, found when the backfill ran on prod after the 4.7.0 deploy (2026-11-06). The owner asked for this to be scheduled. ## Problem `suggest_for_session` (`webapp/backend/app/services/fact_suggestions.py`) asks only about wiki pages joined through `lore_session_appearances`. On prod, "A Merry Band of Thieves" has 63 live pages but only **one** appearance link in the whole campaign. None of sessions 13, 14 or 16 (1,244 to 1,691 transcript lines each) has a linked page, so the backfill finished with `pages_asked: 0` on all three and never asked the AI anything. Dev found candidates only because its pages were linked. ## Proposed change - **Candidates:** the session's linked pages, **plus** every live, non-proposed page in the campaign with an allowed type (`allowed_for`) whose title or alias appears in the session's transcript lines. Find mentions with the existing `mention_pattern` / `excerpts_for`, not a new matcher. - **Order and cap:** linked pages first, then the others by number of mentions, with a per-session cap (a constant such as 12) so a long session doesn't turn into dozens of AI calls. Count the pages skipped by the cap as `skipped_cap` in the counts, so the run records it. - **Unchanged:** quote validation (`match_quote`, `supports`), the written-label guard, the bin and erasure rules, and one pending suggestion per session, page and label. - **Short names:** a page whose title is a common word, or shorter than about 3 characters, must not match everywhere. Look at how `mention_pattern` handles this already and add a test for one-word titles. ## Tests - A session with no appearance links, where the transcript names a page: it is asked about and gets a suggestion. - The cap: with more mentioned pages than the cap allows, linked pages come first, then the most-mentioned ones, and `skipped_cap` is counted. - Trashed pages, pages proposed by the bot, and pages from another campaign are never candidates. ## After it ships Re-run the backfill on prod (`celery -A app.tasks.celery_app call app.tasks.fact_tasks.suggest_fact_changes_backfill`). It skips sessions that already have a run unless that is changed. Check `sessions_to_backfill` and decide whether the re-run needs a "force" argument for sessions whose earlier run asked about 0 pages. Prod runs only on the owner's say.

claude-bot added this to the v4.8.0 - A Living World Fills In milestone

2026-11-06 01:05:03 +00:00