Beats gathered across extraction passes reach compose unsorted, though the prompt says they are chronological #566

New issue

Closed

opened 2026-09-09 04:47:27 +00:00 by claude-bot · 0 comments

claude-bot commented

2026-09-09 04:47:27 +00:00

Contributor

Symptom. COMPOSE_SYSTEM_PROMPT tells the model the verified events are "already in chronological order", and render_beats_for_compose says "in time order" in its docstring, but the list it renders is only sorted within each extraction pass. validate_beats sorts one pass's beats; gather_validated_beats then does dedupe_beats(validations + new), and dedupe_beats returns kept in list order without sorting. Since MIN_BEAT_EXTRACTION_PASSES = 2, every production run has at least two passes, so every compose prompt since #423 has been a concatenation of per-pass sorted runs, not one chronological list.

Seen. Replaying the 2026-09-09 session ("The Northward Labyrinth") through the real pipeline code with four models standing in for the LLM: Haiku's 21 verified beats reached compose as …00:41:18, 00:43:33, 00:14:42, 00:15:16, 00:29:11, 00:41:16, 01:03:20, …, 00:31:24, 00:48:25, 01:11:59. Sonnet's 93-beat list produced a summary whose fifth paragraph is a dumping ground of out-of-order leftovers ("Elsewhere in their explorations…") that re-narrates the fog and the squirrel after the journal. Haiku's summary places Detect Magic (outside the cave) among the mushroom chamber events. The model is doing what it was told: trusting an order that is not there.

Fix. Sort in one place the compose path cannot bypass: render_beats_for_compose should iterate sorted(validations, key=lambda r: (r.beat.t_start, r.beat.t_end)) (which is what render_summary_anchors already does), and gather_validated_beats could sort after each merge for good measure. One test: two passes whose beats interleave in time must render interleaved.

Found while comparing summariser models for the same session; the run stats and the driver that reproduced it are in the session notes.

**Symptom.** `COMPOSE_SYSTEM_PROMPT` tells the model the verified events are "already in chronological order", and `render_beats_for_compose` says "in time order" in its docstring, but the list it renders is only sorted *within* each extraction pass. `validate_beats` sorts one pass's beats; `gather_validated_beats` then does `dedupe_beats(validations + new)`, and `dedupe_beats` returns `kept` in list order without sorting. Since `MIN_BEAT_EXTRACTION_PASSES = 2`, every production run has at least two passes, so every compose prompt since #423 has been a concatenation of per-pass sorted runs, not one chronological list. **Seen.** Replaying the 2026-09-09 session ("The Northward Labyrinth") through the real pipeline code with four models standing in for the LLM: Haiku's 21 verified beats reached compose as `…00:41:18, 00:43:33, 00:14:42, 00:15:16, 00:29:11, 00:41:16, 01:03:20, …, 00:31:24, 00:48:25, 01:11:59`. Sonnet's 93-beat list produced a summary whose fifth paragraph is a dumping ground of out-of-order leftovers ("Elsewhere in their explorations…") that re-narrates the fog and the squirrel after the journal. Haiku's summary places Detect Magic (outside the cave) among the mushroom chamber events. The model is doing what it was told: trusting an order that is not there. **Fix.** Sort in one place the compose path cannot bypass: `render_beats_for_compose` should iterate `sorted(validations, key=lambda r: (r.beat.t_start, r.beat.t_end))` (which is what `render_summary_anchors` already does), and `gather_validated_beats` could sort after each merge for good measure. One test: two passes whose beats interleave in time must render interleaved. Found while comparing summariser models for the same session; the run stats and the driver that reproduced it are in the session notes.

claude-bot referenced this issue

2026-09-09 06:29:43 +00:00

Beat extraction silently loses whole windows on the local Qwen router, and logs the failure with an empty message #572