[Backend] Resolve every speaker label to a character in code, and delete the prompt-time rename #329

Closed
opened 2026-08-25 20:38:48 +00:00 by claude-bot · 1 comment
Contributor

Severity: HIGH. Found in the August 2026 session lifecycle review (#319). Prerequisite for the rest of this milestone — it shrinks the attribution error surface under every design that follows, at zero token cost.

The defect

Speaker-to-character resolution happens twice, half in code and half in the prompt, and the two halves disagree.

_apply_character_names (reminder_tasks.py:2121-2131) relabels segments for linked players who have a character_name set — before the transcript is built. Unlinked players and players without a character keep their Discord display names; GM lines keep the GM's display name. The result is a mixed transcript.

The prompt then still lists "X is playing Y" and instructs "Use character names (not player names) when referring to player actions" (audio_service.py:927) — an instruction that is redundant where the code already renamed, and where it is not redundant forces the model to perform a global rename across thousands of lines. Separately, _format_gm_context (audio_service.py:1103-1115) tells the model "The GM may also voice NPCs, so use context to distinguish narration from roleplay" — an explicit invitation to reassign the speaker of GM-labelled lines by guesswork.

There is also a trap: if the GM's own member row has a character_name, _apply_character_names relabels all GM narration as that character while gm_names still lists the display name — so the GM-context block points at a name that no longer appears in the transcript.

Proposed fix

Resolve every label in code, and take naming out of the model's hands entirely:

  • Linked player with a character → the character name, introduced once in a legend
  • Unlinked player, or no character set → display name, listed in the legend as "player, character unknown"
  • GM → a reserved GM label, always, regardless of whether the GM row has a character

Then delete the "use character names" instruction and replace the GM-guessing invitation with a hard rule:

Lines labelled GM include narration and NPC dialogue. Never attribute a GM line's action to a player character. Attribute an action only to the speaker of the line that performs it; treat suggestions and table talk as not performed.

Emit a machine-readable legend (speaker label → role, character, user id) alongside the transcript. The beat validator in this milestone depends on it.

Acceptance criteria

  • Every transcript line's label is resolved deterministically in code; no label is left for the model to translate
  • The GM is always labelled GM, and a GM with a character set does not break the legend
  • The "use character names" instruction and the NPC-guessing sentence are removed
  • A machine-readable legend is produced and persisted with the transcript
  • Tests cover: linked+character, linked without character, unlinked, GM with and without a character, and two members sharing a display name
**Severity: HIGH.** Found in the August 2026 session lifecycle review (#319). Prerequisite for the rest of this milestone — it shrinks the attribution error surface under every design that follows, at zero token cost. ## The defect Speaker-to-character resolution happens **twice**, half in code and half in the prompt, and the two halves disagree. `_apply_character_names` (`reminder_tasks.py:2121-2131`) relabels segments for linked players who have a `character_name` set — *before* the transcript is built. Unlinked players and players without a character keep their Discord display names; GM lines keep the GM's display name. The result is a **mixed** transcript. The prompt then *still* lists "X is playing Y" and instructs **"Use character names (not player names) when referring to player actions"** (`audio_service.py:927`) — an instruction that is redundant where the code already renamed, and where it is not redundant forces the model to perform a global rename across thousands of lines. Separately, `_format_gm_context` (`audio_service.py:1103-1115`) tells the model *"The GM may also voice NPCs, so use context to distinguish narration from roleplay"* — an explicit invitation to reassign the speaker of GM-labelled lines by guesswork. There is also a trap: if the GM's own member row has a `character_name`, `_apply_character_names` relabels **all GM narration** as that character while `gm_names` still lists the display name — so the GM-context block points at a name that no longer appears in the transcript. ## Proposed fix Resolve **every** label in code, and take naming out of the model's hands entirely: - Linked player with a character → the character name, introduced once in a legend - Unlinked player, or no character set → display name, listed in the legend as "player, character unknown" - GM → a reserved `GM` label, always, regardless of whether the GM row has a character Then **delete** the "use character names" instruction and replace the GM-guessing invitation with a hard rule: > Lines labelled GM include narration and NPC dialogue. Never attribute a GM line's action to a player character. Attribute an action only to the speaker of the line that performs it; treat suggestions and table talk as not performed. Emit a machine-readable legend (speaker label → role, character, user id) alongside the transcript. The beat validator in this milestone depends on it. ## Acceptance criteria - [ ] Every transcript line's label is resolved deterministically in code; no label is left for the model to translate - [ ] The GM is always labelled `GM`, and a GM with a character set does not break the legend - [ ] The "use character names" instruction and the NPC-guessing sentence are removed - [ ] A machine-readable legend is produced and persisted with the transcript - [ ] Tests cover: linked+character, linked without character, unlinked, GM with and without a character, and two members sharing a display name
Author
Contributor

Verified. Four of five criteria were genuinely met; the fifth was half-done and is fixed in 152048e.

Criteria

  • Every transcript line's label is resolved deterministically in coderesolve_speakers produces the label, process_audio builds tracks from it, and transcribe_track stamps speaker/track_owner_id onto every segment. As of #342 the server's echoed label is not consulted on any path, so this is now structural rather than a convention.
  • The GM is always GM, and a GM with a character set does not break the legend — a GM's characters can never enter the legend.
  • The "use character names" instruction and the NPC-guessing sentence are removed — both survive only in docstrings describing their removal.
  • A machine-readable legend is produced and persisted with the transcript — see below.
  • Tests cover linked+character, linked without character, unlinked, GM with and without a character, two members sharing a display name — all present.

The legend was persisted only for runs that produced beats

_persist_summarisation_run returned early whenever a run had no beats, so a run that fell back to prose stored no legend at all — and those are exactly the sessions where attribution was trusted rather than checked, i.e. the ones whose legend is most worth having.

Every run now gets a row (#333), so the legend is persisted unconditionally. used_beats distinguishes the two cases, which is what made the empty row unambiguous enough to write in the first place.

One thing worth noting for later

The legend lives on summarisation_runs.speaker_legend — persisted with the run, not with the transcript. transcript_segments carries (track_owner_id, speaker_label) per line, which is the label↔owner half only: no role, no character list. Nothing stores a legend against sessions itself.

That satisfies the criterion as written — the legend is produced and persisted, stored as it was used rather than recomputed, which is the property #330 makes matter. But if something later wants "the legend for this transcript" without going through a summarisation run, it does not exist yet. Recording it rather than leaving it to be rediscovered.

Also strengthened here

_best_display_name (#344) means a speaker the bot could not name is now resolved from the linked account rather than carrying a placeholder into the transcript — one more label decided in code instead of left as a stand-in.

Closing.

**Verified. Four of five criteria were genuinely met; the fifth was half-done and is fixed in `152048e`.** ## Criteria - [x] **Every transcript line's label is resolved deterministically in code** — `resolve_speakers` produces the label, `process_audio` builds tracks from it, and `transcribe_track` stamps `speaker`/`track_owner_id` onto every segment. As of #342 the server's echoed label is not consulted on *any* path, so this is now structural rather than a convention. - [x] **The GM is always `GM`, and a GM with a character set does not break the legend** — a GM's characters can never enter the legend. - [x] **The "use character names" instruction and the NPC-guessing sentence are removed** — both survive only in docstrings describing their removal. - [x] **A machine-readable legend is produced and persisted with the transcript** — see below. - [x] **Tests cover linked+character, linked without character, unlinked, GM with and without a character, two members sharing a display name** — all present. ## The legend was persisted only for runs that produced beats `_persist_summarisation_run` returned early whenever a run had no beats, so a run that fell back to prose stored **no legend at all** — and those are exactly the sessions where attribution was trusted rather than checked, i.e. the ones whose legend is most worth having. Every run now gets a row (#333), so the legend is persisted unconditionally. `used_beats` distinguishes the two cases, which is what made the empty row unambiguous enough to write in the first place. ## One thing worth noting for later The legend lives on `summarisation_runs.speaker_legend` — persisted with the *run*, not with the transcript. `transcript_segments` carries `(track_owner_id, speaker_label)` per line, which is the label↔owner half only: no role, no character list. Nothing stores a legend against `sessions` itself. That satisfies the criterion as written — the legend is produced and persisted, stored as it was *used* rather than recomputed, which is the property #330 makes matter. But if something later wants "the legend for this transcript" without going through a summarisation run, it does not exist yet. Recording it rather than leaving it to be rediscovered. ## Also strengthened here `_best_display_name` (#344) means a speaker the *bot* could not name is now resolved from the linked account rather than carrying a placeholder into the transcript — one more label decided in code instead of left as a stand-in. Closing.
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#329
No description provided.