fix(backend): pick the summary's events by time, and check for an answer-shaped beat against what was spoken (#589, #584) #591

Merged

claude-bot merged 2 commits from fix/589-selection-and-detector into main

2026-09-10 07:30:58 +00:00

claude-bot commented

2026-09-10 07:09:43 +00:00

Contributor

Closes #589. Advances #584. Both found by re-running a real session on dev after this morning's merges.

#589 — the summary was opening mid-scene. select_beats_for_compose ranked two kind tiers and took thirty, and on a session with 81 verified events that cut everything typed other — which on an exploration session is the establishing material. The summary opened at 02:38 with a character moving to a lower level; waking up displaced was gone. Selection is now time-stratified: the session's span is divided into as many buckets as the cap allows, each non-empty bucket contributes its best beat (kind tier only as the within-bucket tiebreak), any slack is filled by rank, and the result is re-sorted chronologically. Identity below the cap is unchanged.

The honest cost, which is tested rather than hidden: with 81 beats over fifty minutes a bucket is 99 seconds wide, so a session whose first three events land inside 77 seconds gets one of them, not three. test_the_session_opens_where_the_session_opened pins that the prose now starts where the session started, test_a_dense_opening_still_costs_its_second_and_third_beat pins the cost with the arithmetic, and test_a_shorter_session_keeps_the_whole_opening shows the case where all three survive. One #568 test asserted the old behaviour and was rewritten; the rest pass untouched.

#584 — the detector was looking at the wrong text. It ran against the compacted transcript, where consecutive same-speaker lines merge, so the "short answer" test that gives it its precision could not fire. It now runs against the raw lines behind each merged line. The mapping is built by walking the two line lists in step and matching on second and speaker rather than by stamp arithmetic, because on the golden transcript pure arithmetic yields an empty window for four lines that collide on the same second — a new blind spot inside the fix for a blind spot. The detector still only reports, still logs counts only.

Three things I had wrong, corrected here by measurement:

The precision figure was already a raw-transcript measurement, not a compacted one. It is 1 of 71 GM lines, and the docstring now says which text it refers to.
Merging GM lines cannot separate a question from its answer; a player line always breaks a GM run. The real second mechanism is that a player's question merges with their own next remark, so the merged line no longer ends in a question mark. Both mechanisms have tests, each paired with an assertion that the compacted path reports nothing.
The test I asked for, that all three opening events survive, is arithmetically impossible at that session's density. It is asserted in the session shape where it does hold.

The golden fixture cannot demonstrate the underlying bug — it is authored as strict turn-taking, so only 9 of its 209 lines merge — which is stated in the docstring and is another reason the detector still reports rather than rejects.

Eval scores identical before and after on both fixture sets. Full backend suite 3,137 passed; formatted with CI's ruff.

🤖 Generated with Claude Code

Closes #589. Advances #584. Both found by re-running a real session on dev after this morning's merges. **#589 — the summary was opening mid-scene.** `select_beats_for_compose` ranked two kind tiers and took thirty, and on a session with 81 verified events that cut everything typed `other` — which on an exploration session is the establishing material. The summary opened at 02:38 with a character moving to a lower level; waking up displaced was gone. Selection is now time-stratified: the session's span is divided into as many buckets as the cap allows, each non-empty bucket contributes its best beat (kind tier only as the within-bucket tiebreak), any slack is filled by rank, and the result is re-sorted chronologically. Identity below the cap is unchanged. The honest cost, which is tested rather than hidden: with 81 beats over fifty minutes a bucket is 99 seconds wide, so a session whose first three events land inside 77 seconds gets one of them, not three. `test_the_session_opens_where_the_session_opened` pins that the prose now starts where the session started, `test_a_dense_opening_still_costs_its_second_and_third_beat` pins the cost with the arithmetic, and `test_a_shorter_session_keeps_the_whole_opening` shows the case where all three survive. One #568 test asserted the old behaviour and was rewritten; the rest pass untouched. **#584 — the detector was looking at the wrong text.** It ran against the compacted transcript, where consecutive same-speaker lines merge, so the "short answer" test that gives it its precision could not fire. It now runs against the raw lines behind each merged line. The mapping is built by walking the two line lists in step and matching on second and speaker rather than by stamp arithmetic, because on the golden transcript pure arithmetic yields an empty window for four lines that collide on the same second — a new blind spot inside the fix for a blind spot. The detector still only reports, still logs counts only. **Three things I had wrong**, corrected here by measurement: - The precision figure was already a raw-transcript measurement, not a compacted one. It is 1 of 71 GM lines, and the docstring now says which text it refers to. - Merging GM lines cannot separate a question from its answer; a player line always breaks a GM run. The real second mechanism is that a player's question merges with their *own* next remark, so the merged line no longer ends in a question mark. Both mechanisms have tests, each paired with an assertion that the compacted path reports nothing. - The test I asked for, that all three opening events survive, is arithmetically impossible at that session's density. It is asserted in the session shape where it does hold. The golden fixture cannot demonstrate the underlying bug — it is authored as strict turn-taking, so only 9 of its 209 lines merge — which is stated in the docstring and is another reason the detector still reports rather than rejects. Eval scores identical before and after on both fixture sets. Full backend suite 3,137 passed; formatted with CI's ruff. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 2 commits

2026-09-10 07:09:43 +00:00

fix(backend): choose the summary's thirty events by time, not by kind (#589 ) 5279ed4431

#568 capped compose at thirty verified beats and picked which thirty by
ranking two kind tiers — reveal/decision/combat/travel above social/other.
On the first real session it ran on, that cut the session's beginning off.

Everything establishing that session came back typed "other": the long rest,
waking up somewhere else, realising the caverns had shifted. All of it was
demoted below every reveal in the following fifty minutes and cut, and the
summary opened at 02:38 on one character walking down a ramp. Of 81 verified
beats, 35 were "other" and 15 "social" — 62% of the session in the losing
tier, because the extractor's six labels do not fit an exploration session.

So selection is stratified by time instead: the span is divided into `limit`
equal buckets, each non-empty bucket contributes one beat, and the slots empty
buckets leave over are filled from the rest. The two-tier kind priority stays
as the *within-bucket* tiebreak, with time breaking that tie — kind now decides
which beat represents a stretch of the session, never which stretch survives.
A kind-ranked cut is only as good as the model's labels; this one is not.

Unchanged: the identity property below the cap (byte-for-byte the same summary
for a normal session), the chronological re-sort before returning, purity and
determinism — the selection runs twice per session, for compose and for the
dropped-event check, and the two must agree.

A dense opening still costs its second and third beat: thirty slots over fifty
minutes is one beat per hundred seconds. The difference is where the cost
falls — evenly across the session rather than entirely on its start. Both
halves of that are tests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

fix(backend): check for an answer-shaped beat against the lines that were spoken (#584 )

CI / Backend lint (ruff) (pull_request) Successful in 52s

Details

CI / Summarisation accuracy eval harness (stub provider) (pull_request) Successful in 1m46s

Details

CI / Bot/backend version sync (pull_request) Successful in 43s

Details

CI / Frontend tests, audit, and build (pull_request) Successful in 4m29s

Details

CI / Docker image build (pull_request) Successful in 1m14s

Details

CI / Bot tests and audit (pull_request) Successful in 3m4s

Details

CI / Backend migration, tests, and audit (pull_request) Successful in 8m20s

Details

CI / Synthetic session harness (no GPU, no LLM) (pull_request) Successful in 17m34s

Details

f505f25961

The detector added with #585 logged nothing on the run it was written for,
while two verified beats asserted a false location. It was not merely narrow;
it was structurally unable to fire.

The pipeline prompts from the compacted transcript (#341), where a speaker's
consecutive lines merge into one line carrying the run's first stamp. So a GM
who answers in three short bursts is one line several times the length of any
of them, and the "<= 8 words" bound that gives this predicate its precision
can never match it. A player whose question is followed by one more remark of
their own is, in the same view, a line that no longer ends in a question mark,
so the lookback finds no question either. Both were true of the exchange this
was reported from.

`summarise` now keeps the pre-compaction text and carries it down to this one
check. A cited stamp resolves to the *span* of raw lines the compacted line
covers, and the word-count and preceding-question tests are applied to those.
Spans come from `merged_line_spans`, which aligns the two line lists by
walking them in step and matching (stamp, speaker) rather than doing
arithmetic on stamps: a stamp-bounded window is empty whenever the next
compacted line shares this one's second, which is 4 of 209 lines on the golden
transcript, each of which would have silently stopped being checked. The
result is a partition of the raw lines — asserted in a test, because a line in
two spans is word-counted twice and a line in none is a line nothing checks.

The contract is unchanged: it reports, it never rejects, it logs counts and
never beat or transcript text, and every raw line in a cited span must be
reply-shaped — loosening that to "any of them" would report a beat drawn from
the narration it happens to be filed with.

Precision re-measured on raw lines, per the issue. On
evals/fixtures/synthetic_session/golden/full.transcript.txt the shape matches
**1 of 71 raw GM lines**, and that one is table talk — the same figure the
docstring already quoted, which turns out to have been measured on raw lines
all along (218 lines, 71 GM). The compacted view of the same file matches 1 of
66. The two agree because that fixture is authored as strict turn-taking and
barely compacts: 9 of its 209 lines merge two raw lines, none merge three. It
therefore cannot demonstrate this bug at all, and the agreement is a property
of the fixture rather than evidence the change is a no-op. On real speech,
where roughly a quarter of lines merge, the false-positive rate remains
unmeasured — which is why this still reports rather than rejects.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

claude-bot scheduled this pull request to auto merge when all checks succeed

2026-09-10 07:09:51 +00:00

claude-bot merged commit 56b017570a into main

2026-09-10 07:30:58 +00:00

claude-bot deleted branch fix/589-selection-and-detector

2026-09-10 07:30:59 +00:00

claude-bot referenced this pull request from a commit

2026-09-10 07:30:59 +00:00

Merge PR #591: fix(backend): pick the summary's events by time, and check for an answer-shaped beat against what was spoken (#589, #584)

claude-bot referenced this pull request

2026-09-11 03:31:47 +00:00

Summary ends with an empty "What's next?" heading on Gemma 4 #595