[Backend] Bound the four remaining unbounded LLM contexts #340

New issue

Closed

opened 2026-08-25 20:38:58 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-08-25 20:38:58 +00:00

Contributor

Severity: MEDIUM. Found in the August 2026 session lifecycle review (#319).

The defect

The #271 fix (generation_service.py:57-119 — per-position budgets with middle-out truncation) is the right pattern and the workbench tools follow it. Four places do not, and each grows without limit:

Bot /ask (routers/bot.py:1145-1196 → audio_service.py:2189-2237) — ten full untruncated session summaries plus every approved lore entry with its full body. Grows with the wiki forever.
Session-title suggestions (reminder_tasks.py:4424-4466) — the comment claims each context is truncated, but storyline.body is included whole (line 4433), and the storyline is the concatenation of every session summary. Roughly 35k tokens by session 50.
lore_match_category (reminder_tasks.py:841-846 → audio_service.py:1239-1292) — every approved entry of the type at 800 chars each. Bounded per entry, unbounded in count; 200 NPCs is ~45k tokens.
Single-pass lore mode (lore_service.py:24-26, 932-938) — the transcript is head-truncated at 16,000 characters, keeping the first ~20 minutes of a 3.5 h session. This is the exact pre-#271 mistake, still live in a non-default mode.

Proposed fix

Apply the #271 per-position-budget pattern to the first three. Change single-pass lore's head truncation to middle-out, or better, route it through the same window-sized chunking as the summariser.

Acceptance criteria

/ask context is bounded by count and per-item length
Session-title context truncates storyline.body
lore_match_category is bounded by entry count
Single-pass lore no longer head-truncates
Each has a test asserting the bound holds as the corpus grows

**Severity: MEDIUM.** Found in the August 2026 session lifecycle review (#319). ## The defect The #271 fix (`generation_service.py:57-119` — per-position budgets with middle-out truncation) is the right pattern and the workbench tools follow it. Four places do not, and each grows without limit: - **Bot `/ask`** (`routers/bot.py:1145-1196` → `audio_service.py:2189-2237`) — ten full untruncated session summaries **plus every approved lore entry with its full body**. Grows with the wiki forever. - **Session-title suggestions** (`reminder_tasks.py:4424-4466`) — the comment claims each context is truncated, but `storyline.body` is included **whole** (line 4433), and the storyline is the concatenation of every session summary. Roughly 35k tokens by session 50. - **`lore_match_category`** (`reminder_tasks.py:841-846` → `audio_service.py:1239-1292`) — every approved entry of the type at 800 chars each. Bounded per entry, unbounded in count; 200 NPCs is ~45k tokens. - **Single-pass lore mode** (`lore_service.py:24-26, 932-938`) — the transcript is **head-truncated at 16,000 characters**, keeping the *first* ~20 minutes of a 3.5 h session. This is the exact pre-#271 mistake, still live in a non-default mode. ## Proposed fix Apply the #271 per-position-budget pattern to the first three. Change single-pass lore's head truncation to middle-out, or better, route it through the same window-sized chunking as the summariser. ## Acceptance criteria - [ ] `/ask` context is bounded by count and per-item length - [ ] Session-title context truncates `storyline.body` - [ ] `lore_match_category` is bounded by entry count - [ ] Single-pass lore no longer head-truncates - [ ] Each has a test asserting the bound holds as the corpus grows

claude-bot added this to the v4.0.0 - Session Pipeline Accuracy milestone

2026-08-25 20:38:58 +00:00

claude-bot added the

bug

backend

labels

2026-08-25 20:38:58 +00:00

rbrooks referenced this issue from a commit

2026-08-26 02:22:21 +00:00

fix(backend): bound the four prompt contexts that grew without limit (#340)

rbrooks referenced this issue from a commit

2026-08-27 15:40:12 +00:00

feat(backend): size extraction windows for attention, not for context (#423)

claude-bot commented

2026-08-27 23:27:08 +00:00

Author

Contributor

Verified against the acceptance criteria before closing. All four contexts are bounded, each with a corpus-growth test.

/ask — summaries truncated to 1,200 chars via truncate_keeping_ends, lore bounded to 40 entries via rank_by_word_overlap with bodies capped at 400 chars (audio_service.py:3892-3941). Test: test_ask_prompt_stops_growing_with_the_wiki.
Session-title context — truncate_keeping_ends(storyline.body, 4000) (reminder_tasks.py:4579-4590). Test: test_storyline_context_is_bounded_as_chapters_accumulate.
lore_match_category — rank_by_word_overlap(..., 60) (reminder_tasks.py:863-882). Test: test_lore_matching_compares_against_the_entries_most_like_the_candidates.
Single-pass lore — head-truncation replaced by sample_evenly (lore_service.py:987-998). The primitive is well covered; there is no integration test wiring a large transcript through that specific call site, which is a minor gap given the primitive is solid and the wiring is one line.

Worth noting the ranking choice: entries are dropped by relevance rather than arbitrarily, because a dropped wiki entry becomes a duplicate wiki page.

Closing. Part of a full acceptance-criteria pass across the v4.0.0 milestone.

Verified against the acceptance criteria before closing. All four contexts are bounded, each with a corpus-growth test. - **`/ask`** — summaries truncated to 1,200 chars via `truncate_keeping_ends`, lore bounded to 40 entries via `rank_by_word_overlap` with bodies capped at 400 chars (`audio_service.py:3892-3941`). Test: `test_ask_prompt_stops_growing_with_the_wiki`. - **Session-title context** — `truncate_keeping_ends(storyline.body, 4000)` (`reminder_tasks.py:4579-4590`). Test: `test_storyline_context_is_bounded_as_chapters_accumulate`. - **`lore_match_category`** — `rank_by_word_overlap(..., 60)` (`reminder_tasks.py:863-882`). Test: `test_lore_matching_compares_against_the_entries_most_like_the_candidates`. - **Single-pass lore** — head-truncation replaced by `sample_evenly` (`lore_service.py:987-998`). The primitive is well covered; there is no integration test wiring a large transcript through that specific call site, which is a minor gap given the primitive is solid and the wiring is one line. Worth noting the ranking choice: entries are dropped by *relevance* rather than arbitrarily, because a dropped wiki entry becomes a duplicate wiki page. Closing. Part of a full acceptance-criteria pass across the v4.0.0 milestone.

claude-bot closed this issue

2026-08-27 23:27:22 +00:00

claude-bot referenced this issue

2026-08-27 23:28:53 +00:00

[Backend] /ask is unprotected on every provider — no context budget, no num_ctx, no truncation check #426

rbrooks referenced this issue from a commit

2026-08-28 18:46:37 +00:00

fix(backend): stop the chunker recursing forever, and brief it on what it holds

claude-bot referenced this issue

2026-08-28 19:15:39 +00:00

[Backend] Chunk the transcript by time window, sized from the provider's declared context #331

rbrooks referenced this issue from a commit