Beat extraction records the GM's table admin as story events

claude-bot commented

2026-09-11 03:32:01 +00:00

Collaborator

What happens

Run 3 of the dev replay (Gemma 4 26B, 2026-09-11 03:28 UTC, run a0784433, at 6533fa1) verified 83 events. Several of them are the GM running the table, not anything that happened in the story:

00:20:31 "The GM fixes Wyatt's character position on the map."
00:20:37 "The GM moves Idani to the upper layer of the map."
00:21:21 "The GM adjusts Idani's position on the map to resolve a movement issue."
00:54:16 and 00:55:05 "The GM explains what a thresher is." (twice)
01:02:47 "The GM asks the players for their character backgrounds."
01:07:00 to 01:07:31: three events about which skill check reading the journal will need
01:12:17 to 01:13:08: three events about adding a skill to Wyatt's sheet

They verify because they're true. The GM did say those lines, so the validator (which checks that the speaker and cited lines exist) passes them. Most stay in the event log (#568), which is noise. At least one reached the prose summary: "the group received an explanation regarding what a thresher is".

Related but not the same: #584's answer-shaped check catches a short GM reply to a player's question. These are mostly long GM lines, and nobody asked a question.

Why it matters

Each one competes for the 30 events the summary is written from (#589).
Players reading the event log see Foundry housekeeping mixed in with the story.
Local models reproduce the pattern reliably. #584 showed that a prompt rule alone didn't stop Gemma.

Options

A beat kind for this. Add a table kind (or meta), which the extraction prompt is told to use for rules calls, token moves, sheet admin and term explanations. Compose selection skips it, and the event log shows it folded by default. This needs no correct judgement from code. The model only has to label, not decide to omit, and labelling held up better than omission under #584's prompt change.
A cheap deterministic backstop: a beat whose actors are only the GM, whose summary starts "The GM (asks|explains|clarifies|allows|agrees|rules|decides|moves|adjusts|fixes)", and whose kind is other gets marked table admin. Report it on the run record first, as #584's check does, before it's allowed to change anything.
Add a table-talk fixture to the eval harness, which ties in with #586.

Recommend 1 with 2 as a report-only check, measured on this replay before and after.

## What happens Run 3 of the dev replay (Gemma 4 26B, 2026-09-11 03:28 UTC, run `a0784433`, at 6533fa1) verified 83 events. Several of them are the GM running the table, not anything that happened in the story: - 00:20:31 "The GM fixes Wyatt's character position on the map." - 00:20:37 "The GM moves Idani to the upper layer of the map." - 00:21:21 "The GM adjusts Idani's position on the map to resolve a movement issue." - 00:54:16 and 00:55:05 "The GM explains what a thresher is." (twice) - 01:02:47 "The GM asks the players for their character backgrounds." - 01:07:00 to 01:07:31: three events about which skill check reading the journal will need - 01:12:17 to 01:13:08: three events about adding a skill to Wyatt's sheet **They verify because they're true.** The GM did say those lines, so the validator (which checks that the speaker and cited lines exist) passes them. Most stay in the event log (#568), which is noise. At least one reached the prose summary: *"the group received an explanation regarding what a thresher is"*. **Related but not the same:** #584's answer-shaped check catches a short GM reply to a player's question. These are mostly long GM lines, and nobody asked a question. ## Why it matters - Each one competes for the 30 events the summary is written from (#589). - Players reading the event log see Foundry housekeeping mixed in with the story. - Local models reproduce the pattern reliably. #584 showed that a prompt rule alone didn't stop Gemma. ## Options 1. **A beat kind for this.** Add a `table` kind (or `meta`), which the extraction prompt is told to use for rules calls, token moves, sheet admin and term explanations. Compose selection skips it, and the event log shows it folded by default. **This needs no correct judgement from code.** The model only has to label, not decide to omit, and labelling held up better than omission under #584's prompt change. 2. **A cheap deterministic backstop:** a beat whose actors are only the GM, whose summary starts "The GM (asks|explains|clarifies|allows|agrees|rules|decides|moves|adjusts|fixes)", and whose kind is `other` gets marked table admin. Report it on the run record first, as #584's check does, before it's allowed to change anything. 3. Add a table-talk fixture to the eval harness, which ties in with #586. Recommend 1 with 2 as a report-only check, measured on this replay before and after.

claude-bot added the

backend

enhancement

labels

2026-09-11 03:32:24 +00:00

claude-bot referenced this issue

2026-09-11 04:46:05 +00:00

Summary ends with an empty "What's next?" heading on Gemma 4 #595

claude-bot commented

2026-09-11 04:46:14 +00:00

Author

Collaborator

Picking this up (2026-09-11) with option 1, plus option 2 as a check that only reports. The owner agreed the order: #595 and #588 land and are measured first, then this one, measured on its own. It changes what the model produces, and Gemma's run-to-run variance (59, 81 and 83 verified events across the last three runs) would hide its effect if everything moved at once.

The kind: a new table_admin kind in the extraction prompt, the json_schema enum, validation, and the event log. The model labels these events; it doesn't drop them, so a mislabel costs one folded row, never a lost event.
Compose: selection leaves table_admin events out before stratifying, so they don't take time buckets from story events.
Event log: these rows fold behind one line, "N table-admin notes".
The check: counts GM-only other events that start "The GM asks, explains, …" and logs the count without changing anything. That shows whether the model is using the label.

**Picking this up** (2026-09-11) with option 1, plus option 2 as a check that only reports. The owner agreed the order: #595 and #588 land and are measured first, then this one, measured on its own. It changes what the model produces, and Gemma's run-to-run variance (59, 81 and 83 verified events across the last three runs) would hide its effect if everything moved at once. - **The kind:** a new `table_admin` kind in the extraction prompt, the json_schema enum, validation, and the event log. The model **labels** these events; it doesn't drop them, so a mislabel costs one folded row, never a lost event. - **Compose:** selection leaves `table_admin` events out before stratifying, so they don't take time buckets from story events. - **Event log:** these rows fold behind one line, "N table-admin notes". - **The check:** counts GM-only `other` events that start "The GM asks, explains, …" and logs the count without changing anything. That shows whether the model is using the label.

claude-bot referenced this issue

2026-09-11 05:12:03 +00:00

fix(backend): no empty "What's next?", and collapse exact-duplicate beats (#595, #588) #598

rbrooks referenced this issue from a commit

2026-09-11 05:13:19 +00:00

feat(webapp): label the GM running the table, and never narrate it (#596)

rbrooks referenced this issue from a commit

2026-09-11 05:16:06 +00:00

fix(webapp): offer table_admin last in the beat kinds (#596)

claude-bot referenced this issue

2026-09-11 05:46:58 +00:00

Beat dedupe misses same-moment duplicates and split moments, inflating the event log #588

rbrooks referenced this issue from a commit

2026-09-11 06:00:33 +00:00

feat(webapp): label the GM running the table, and never narrate it (#596)

rbrooks referenced this issue from a commit

2026-09-11 06:00:33 +00:00

fix(webapp): offer table_admin last in the beat kinds (#596)

rbrooks referenced this issue from a commit

2026-09-11 06:09:57 +00:00

fix(backend): dedupe keeps the story copy over a table_admin one (#596, #588)

claude-bot referenced this issue from a pull request that will close it,

2026-09-11 06:10:33 +00:00

feat(backend): label the GM's table admin instead of narrating it (#596) #599

claude-bot referenced this issue from a commit

2026-09-11 06:26:58 +00:00

Merge pull request 'feat(backend): label the GM's table admin instead of narrating it (#596)' (#599) from feat/596-table-admin-beats into main

claude-bot closed this issue

2026-09-11 06:26:59 +00:00

claude-bot commented

2026-09-11 06:54:21 +00:00

Author

Collaborator

Merged (PR #599, main 504f446) and measured on dev over two summary-only reruns of the replay session (Gemma 4 26B), against run 4 (6d3e6f54), which had #598 but not this.

	Run 4 (before)	Run 5 (`ace1414f`)	Run 6 (`03b3acfa`)
Verified events	76	95	85
…labelled `table_admin`	—	35	42
Story events left for the prose	76	60	43
"Looks like table admin but labelled other" (the check that only reports)	—	1	1
Answer-shaped (#584)	4	2	2
Table-admin phrases in the prose	3	5	2

The event log is the clear win. 35 to 42 table-side events per run are folded now, and the labels are good.

Folded correctly: every Foundry and token fix ("fixes Wyatt's character position on the map", "attempts to fix a Foundry bug affecting the map layers"), every "which check is it" call, the thresher explanation, the Cave-skill sheet admin, and "The GM decides to end the session".
Folded but carrying story: a few GM lines that hold fiction were also labelled. Examples are the perception results and "explains the distance between the current location and the entry point", which is a teleport clue. They're folded, not lost, and the story side was still narrated from other events.

The prose improves, but not cleanly. Run 6 is the best summary so far: it has the opening, the fall and the rope catch, the skeleton credited correctly, the circus rumour, the mid-sized city, and a real hook. Run 5 lost the opening from its prose, although the waking-up events were stored as story events.

What still reaches the prose is table admin given a story label, not missed labels:

"The GM describes the current positions of the party members" (other)
"The GM asks if anyone else wants to examine the journal" (decision)
"Idani decides to change the bulk of the mushrooms in their inventory" (decision)
"The GM reveals the perception results…" (reveal)

The report-only check only looks at other with a fixed verb list, so it counts 1 of these, not 4 or 5.

Ironroot is unchanged. Both runs labelled "The GM reveals the party's current location is Ironroot Hold" as reveal. Run 5 then built its hook on it ("navigate their way through Ironroot Hold"). That's the GM slip described on #424: only a GM correction fixes it.

Possible follow-ups, not filed yet:

Broaden the report-only check to every kind and more verbs (states, clarifies, searches, asks whether), so it measures the real mislabel rate.
A rule in the compose prompt against narrating the GM's process ("the GM asked, clarified, confirmed, searched"). It's about phrasing, not about leaving events out, so it may hold where extraction rules didn't. It needs measuring.
Revisit the 30-event cap (#589). Table admin no longer competes for it, and the Thieves' Cant keeps falling out of the dense 01:07–01:11 stretch.
#588's judgement half: the GM-side and player-side retellings of one moment.

**Merged (PR #599, main `504f446`) and measured on dev over two summary-only reruns of the replay session** (Gemma 4 26B), against run 4 (`6d3e6f54`), which had #598 but not this. | | Run 4 (before) | Run 5 (`ace1414f`) | Run 6 (`03b3acfa`) | |---|---|---|---| | Verified events | 76 | 95 | 85 | | …labelled `table_admin` | — | 35 | 42 | | Story events left for the prose | 76 | 60 | 43 | | "Looks like table admin but labelled other" (the check that only reports) | — | 1 | 1 | | Answer-shaped (#584) | 4 | 2 | 2 | | Table-admin phrases in the prose | 3 | 5 | 2 | **The event log is the clear win.** 35 to 42 table-side events per run are folded now, and the labels are good. - **Folded correctly:** every Foundry and token fix ("fixes Wyatt's character position on the map", "attempts to fix a Foundry bug affecting the map layers"), every "which check is it" call, the thresher explanation, the Cave-skill sheet admin, and "The GM decides to end the session". - **Folded but carrying story:** a few GM lines that hold fiction were also labelled. Examples are the perception results and "explains the distance between the current location and the entry point", which is a teleport clue. They're folded, not lost, and the story side was still narrated from other events. **The prose improves, but not cleanly.** Run 6 is the best summary so far: it has the opening, the fall and the rope catch, the skeleton credited correctly, the circus rumour, the mid-sized city, and a real hook. Run 5 lost the opening from its prose, although the waking-up events were stored as story events. **What still reaches the prose is table admin given a *story* label, not missed labels:** - "The GM describes the current positions of the party members" (*other*) - "The GM asks if anyone else wants to examine the journal" (*decision*) - "Idani decides to change the bulk of the mushrooms in their inventory" (*decision*) - "The GM reveals the perception results…" (*reveal*) The report-only check only looks at `other` with a fixed verb list, so it counts 1 of these, not 4 or 5. **Ironroot is unchanged.** Both runs labelled "The GM reveals the party's current location is Ironroot Hold" as `reveal`. Run 5 then built its hook on it ("navigate their way through Ironroot Hold"). That's the GM slip described on #424: only a GM correction fixes it. **Possible follow-ups, not filed yet:** 1. **Broaden the report-only check** to every kind and more verbs (states, clarifies, searches, asks whether), so it measures the real mislabel rate. 2. **A rule in the compose prompt** against narrating the GM's process ("the GM asked, clarified, confirmed, searched"). It's about phrasing, not about leaving events out, so it may hold where extraction rules didn't. It needs measuring. 3. **Revisit the 30-event cap (#589).** Table admin no longer competes for it, and the Thieves' Cant keeps falling out of the dense 01:07–01:11 stretch. 4. **#588's judgement half:** the GM-side and player-side retellings of one moment.

claude-bot referenced this issue

2026-09-11 13:40:23 +00:00

Raise the compose cap from 30 to 36 now that table admin no longer competes for it #600

rbrooks referenced this issue from a commit

2026-09-11 13:44:16 +00:00

tune(backend): narrate up to 36 events, not 30 (#600)

claude-bot referenced this issue

2026-09-11 13:44:30 +00:00

tune(backend): narrate up to 36 events, not 30 (#600) #601

claude-bot referenced this issue

2026-09-11 14:15:57 +00:00

Raise the compose cap from 30 to 36 now that table admin no longer competes for it #600