rbrooks/Quest-Board

Watch

Fork 0

Compose selection fills spare slots earliest-first, so a session's ending and its dense stretches lose out #603

New issue

Closed

opened 2026-09-11 14:36:06 +00:00 by claude-bot · 2 comments

claude-bot commented

2026-09-11 14:36:06 +00:00

Contributor

Found while measuring #600. Owner approved the fix on 2026-09-11.

What happens

beat_service.select_beats_for_compose (#589) splits the session into limit time buckets and takes one beat from each non-empty bucket. It then fills the leftover slots from sorted(rest, key=rank), where rank = (kind tier, t_start, t_end, position). So spare slots go to story kinds first, then the earliest in the session. Two consequences:

A late event the model labelled other comes last, whatever the cap. On run 7 (b0613da6), a read-only replay of the real selector showed the Thieves' Cant (01:08:55) and the mid-sized city (01:11:26) were not picked at 30 or at 36. Both are real reveals that the model labelled other.
The session's ending loses every tie. That's also why run 7's "What's next?" was weak: the events it needed weren't narrated.

Raising the cap (#600) only extended the same earliest-first queue. The extra slots bought chatter such as "The players engage in off-topic chatter about the game Phasmophobia".

A second, separate cause of that chatter: the extraction prompt tells the model to leave out talk that isn't about the game, and it records it anyway, as a story kind. This is #596's lesson again: labelling holds where omission doesn't.

Fix

Fill across time. After one-per-bucket, give the leftover slots to buckets in proportion to how many unpicked story events each still has (largest-remainder or equivalent). Dense stretches get more, every non-empty bucket keeps at least one, and kind stays a tiebreak within a bucket only, as #589 intended.
- It must stay deterministic, the result re-sorted chronologically, and the identity function under the cap.
Label chatter. Off-topic talk (other games, food, real life) gets recorded with type table_admin instead of being expected to disappear. The kind means "not part of the story", so compose, dedupe and the event log already handle it. The event log's folded line says "table notes" so it fits both.

How it's measured

Fill, with no GPU: replay the new selector on the stored beats of runs 5, 6 and 7 (scratchpad/summ/replay_selection.py). Are the Cant and city picked, and how are picks spread between the first and second half of the session?
Chatter and the summary as a whole: one live summary-only rerun on the dev replay against run 7.

Found while measuring #600. Owner approved the fix on 2026-09-11. ## What happens `beat_service.select_beats_for_compose` (#589) splits the session into `limit` time buckets and takes one beat from each non-empty bucket. It then fills the leftover slots from `sorted(rest, key=rank)`, where `rank = (kind tier, t_start, t_end, position)`. So spare slots go to **story kinds first, then the earliest in the session**. Two consequences: 1. **A late event the model labelled `other` comes last, whatever the cap.** On run 7 (`b0613da6`), a read-only replay of the real selector showed the Thieves' Cant (01:08:55) and the mid-sized city (01:11:26) were **not picked at 30 or at 36**. Both are real reveals that the model labelled `other`. 2. **The session's ending loses every tie.** That's also why run 7's "What's next?" was weak: the events it needed weren't narrated. Raising the cap (#600) only extended the same earliest-first queue. The extra slots bought chatter such as "The players engage in off-topic chatter about the game Phasmophobia". A second, separate cause of that chatter: the extraction prompt tells the model to leave out talk that isn't about the game, and it records it anyway, as a story kind. This is #596's lesson again: labelling holds where omission doesn't. ## Fix 1. **Fill across time.** After one-per-bucket, give the leftover slots to buckets **in proportion to how many unpicked story events each still has** (largest-remainder or equivalent). Dense stretches get more, every non-empty bucket keeps at least one, and kind stays a tiebreak **within** a bucket only, as #589 intended. - It must stay deterministic, the result re-sorted chronologically, and the identity function under the cap. 2. **Label chatter.** Off-topic talk (other games, food, real life) gets recorded with type `table_admin` instead of being expected to disappear. The kind means "not part of the story", so compose, dedupe and the event log already handle it. The event log's folded line says "table notes" so it fits both. ## How it's measured - **Fill, with no GPU:** replay the new selector on the stored beats of runs 5, 6 and 7 (`scratchpad/summ/replay_selection.py`). Are the Cant and city picked, and how are picks spread between the first and second half of the session? - **Chatter and the summary as a whole:** one live summary-only rerun on the dev replay against run 7.

claude-bot added the

backend

bug

labels

2026-09-11 14:36:27 +00:00

claude-bot commented

2026-09-11 14:37:28 +00:00

Author

Contributor

Picking this up (2026-09-11), as one backend-only branch (fix/compose-fill-and-chatter) with two parts, measured separately.

Fill by density. Leftover slots go to buckets in proportion to their unpicked events, with a tie rule that doesn't favour early buckets. Kind stays a tiebreak inside a bucket only.
- Measured without the GPU: the old and new selectors are replayed on the real stored events of runs 5, 6 and 7. That covers whether the Cant and city are picked, the first-half and second-half split, and how many picks land after 01:05.
- The replay data is private campaign content, so it stays in a scratch directory and never enters the repo.
Label chatter as table_admin ("not part of the story"), instead of expecting it to be left out. This changes what the model produces, so it's measured with one live rerun against run 7.

Deferred: the event log's "table notes" wording is a frontend change, left until #602 (which rewrites that component) has merged, to avoid a conflict. It's a one-line follow-up.

**Picking this up** (2026-09-11), as one backend-only branch (`fix/compose-fill-and-chatter`) with two parts, measured separately. 1. **Fill by density.** Leftover slots go to buckets in proportion to their unpicked events, with a tie rule that doesn't favour early buckets. Kind stays a tiebreak inside a bucket only. - **Measured without the GPU:** the old and new selectors are replayed on the real stored events of runs 5, 6 and 7. That covers whether the Cant and city are picked, the first-half and second-half split, and how many picks land after 01:05. - The replay data is private campaign content, so it stays in a scratch directory and never enters the repo. 2. **Label chatter as `table_admin`** ("not part of the story"), instead of expecting it to be left out. This changes what the model produces, so it's **measured with one live rerun** against run 7. **Deferred:** the event log's "table notes" wording is a frontend change, left until #602 (which rewrites that component) has merged, to avoid a conflict. It's a one-line follow-up.

rbrooks referenced this issue from a commit

2026-09-11 14:56:16 +00:00

fix(backend): share the compose fill across the clock, and label chatter (#603)

rbrooks referenced this issue from a commit

2026-09-11 15:10:40 +00:00

fix(backend): spread a busy bucket's picks and anchor the session's ending (#603)

claude-bot referenced this issue from a pull request that will close it,

2026-09-11 15:11:35 +00:00

fix(backend): pick summary events across the whole session, and label chatter (#603) #605

claude-bot closed this issue

2026-09-11 15:30:44 +00:00

claude-bot referenced this issue from a commit

2026-09-11 15:30:45 +00:00

Merge pull request 'fix(backend): pick summary events across the whole session, and label chatter (#603)' (#605) from fix/compose-fill-and-chatter into main

claude-bot commented

2026-09-11 15:47:11 +00:00

Author

Contributor

Merged (PR #605, main 137bfe0), deployed to dev, and measured live: run 8 (00e7962d, Gemma 4 26B, 2026-09-11 15:46 UTC), against run 7 (b0613da6).

	Run 7 (before)	Run 8 (after)
Verified events / story events / table_admin	88 / 54 / 34	96 / 65 / 31
Summary length	2,675 chars	2,416 chars
Thieves' Cant in the prose	no	yes
Mid-sized city in the prose	no	yes
Opening (woke somewhere else)	yes	yes
Phasmophobia chatter	narrated (`other`)	labelled `table_admin`, kept out
Table admin still reaching the prose	3–4 phrases	2 ("searched for a map…", "learned about the nature of a thresher")

It's the first run whose prose carries both targets: "Through the use of Thieves' Cant and other methods, various characters were able to discern details from the journal, including the location of a mid-sized city situated a few miles from an exit on the map." The hook now reaches the city too, although it still repeats the Ironroot slip. That one is left to #424's correction.

What's left is table admin given a story label: "7 beats look like table admin but were labelled other". That's the model's labelling, not the picker.

Closing: the picker bias this issue describes is fixed and measured.

**Merged (PR #605, main `137bfe0`), deployed to dev, and measured live:** run 8 (`00e7962d`, Gemma 4 26B, 2026-09-11 15:46 UTC), against run 7 (`b0613da6`). | | Run 7 (before) | Run 8 (after) | |---|---|---| | Verified events / story events / table_admin | 88 / 54 / 34 | 96 / 65 / 31 | | Summary length | 2,675 chars | 2,416 chars | | Thieves' Cant in the prose | no | **yes** | | Mid-sized city in the prose | no | **yes** | | Opening (woke somewhere else) | yes | yes | | Phasmophobia chatter | narrated (`other`) | **labelled `table_admin`**, kept out | | Table admin still reaching the prose | 3–4 phrases | 2 ("searched for a map…", "learned about the nature of a thresher") | **It's the first run whose prose carries both targets:** *"Through the use of Thieves' Cant and other methods, various characters were able to discern details from the journal, including the location of a mid-sized city situated a few miles from an exit on the map."* The hook now reaches the city too, although it still repeats the Ironroot slip. That one is left to #424's correction. **What's left** is table admin given a story label: "7 beats look like table admin but were labelled other". That's the model's labelling, not the picker. Closing: the picker bias this issue describes is fixed and measured.

rbrooks referenced this issue from a commit

2026-09-11 15:54:57 +00:00

ui(webapp): "table notes", now that the fold holds chatter too (#603)

claude-bot referenced this issue

2026-09-11 15:55:06 +00:00

ui(webapp): "table notes", now that the fold holds chatter too (#603) #606