rbrooks/Quest-Board

Watch

Fork

You've already forked Quest-Board

Code Issues 58 Pull requests 7 Projects Releases 76 Packages Wiki Activity Actions

[Backend] Resolve every speaker label to a character in code, and delete the prompt-time rename #329

New issue

Closed

opened 2026-08-25 20:38:48 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-08-25 20:38:48 +00:00

Collaborator

Copy link

Severity: HIGH. Found in the August 2026 session lifecycle review (#319). Prerequisite for the rest of this milestone — it shrinks the attribution error surface under every design that follows, at zero token cost.

The defect

Speaker-to-character resolution happens twice, half in code and half in the prompt, and the two halves disagree.

_apply_character_names (reminder_tasks.py:2121-2131) relabels segments for linked players who have a character_name set — before the transcript is built. Unlinked players and players without a character keep their Discord display names; GM lines keep the GM's display name. The result is a mixed transcript.

The prompt then still lists "X is playing Y" and instructs "Use character names (not player names) when referring to player actions" (audio_service.py:927) — an instruction that is redundant where the code already renamed, and where it is not redundant forces the model to perform a global rename across thousands of lines. Separately, _format_gm_context (audio_service.py:1103-1115) tells the model "The GM may also voice NPCs, so use context to distinguish narration from roleplay" — an explicit invitation to reassign the speaker of GM-labelled lines by guesswork.

There is also a trap: if the GM's own member row has a character_name, _apply_character_names relabels all GM narration as that character while gm_names still lists the display name — so the GM-context block points at a name that no longer appears in the transcript.

Proposed fix

Resolve every label in code, and take naming out of the model's hands entirely:

Linked player with a character → the character name, introduced once in a legend
Unlinked player, or no character set → display name, listed in the legend as "player, character unknown"
GM → a reserved GM label, always, regardless of whether the GM row has a character

Then delete the "use character names" instruction and replace the GM-guessing invitation with a hard rule:

Lines labelled GM include narration and NPC dialogue. Never attribute a GM line's action to a player character. Attribute an action only to the speaker of the line that performs it; treat suggestions and table talk as not performed.

Emit a machine-readable legend (speaker label → role, character, user id) alongside the transcript. The beat validator in this milestone depends on it.

Acceptance criteria

Every transcript line's label is resolved deterministically in code; no label is left for the model to translate
The GM is always labelled GM, and a GM with a character set does not break the legend
The "use character names" instruction and the NPC-guessing sentence are removed
A machine-readable legend is produced and persisted with the transcript
Tests cover: linked+character, linked without character, unlinked, GM with and without a character, and two members sharing a display name

**Severity: HIGH.** Found in the August 2026 session lifecycle review (#319). Prerequisite for the rest of this milestone — it shrinks the attribution error surface under every design that follows, at zero token cost. ## The defect Speaker-to-character resolution happens **twice**, half in code and half in the prompt, and the two halves disagree. `_apply_character_names` (`reminder_tasks.py:2121-2131`) relabels segments for linked players who have a `character_name` set — *before* the transcript is built. Unlinked players and players without a character keep their Discord display names; GM lines keep the GM's display name. The result is a **mixed** transcript. The prompt then *still* lists "X is playing Y" and instructs **"Use character names (not player names) when referring to player actions"** (`audio_service.py:927`) — an instruction that is redundant where the code already renamed, and where it is not redundant forces the model to perform a global rename across thousands of lines. Separately, `_format_gm_context` (`audio_service.py:1103-1115`) tells the model *"The GM may also voice NPCs, so use context to distinguish narration from roleplay"* — an explicit invitation to reassign the speaker of GM-labelled lines by guesswork. There is also a trap: if the GM's own member row has a `character_name`, `_apply_character_names` relabels **all GM narration** as that character while `gm_names` still lists the display name — so the GM-context block points at a name that no longer appears in the transcript. ## Proposed fix Resolve **every** label in code, and take naming out of the model's hands entirely: - Linked player with a character → the character name, introduced once in a legend - Unlinked player, or no character set → display name, listed in the legend as "player, character unknown" - GM → a reserved `GM` label, always, regardless of whether the GM row has a character Then **delete** the "use character names" instruction and replace the GM-guessing invitation with a hard rule: > Lines labelled GM include narration and NPC dialogue. Never attribute a GM line's action to a player character. Attribute an action only to the speaker of the line that performs it; treat suggestions and table talk as not performed. Emit a machine-readable legend (speaker label → role, character, user id) alongside the transcript. The beat validator in this milestone depends on it. ## Acceptance criteria - [ ] Every transcript line's label is resolved deterministically in code; no label is left for the model to translate - [ ] The GM is always labelled `GM`, and a GM with a character set does not break the legend - [ ] The "use character names" instruction and the NPC-guessing sentence are removed - [ ] A machine-readable legend is produced and persisted with the transcript - [ ] Tests cover: linked+character, linked without character, unlinked, GM with and without a character, and two members sharing a display name

claude-bot added this to the v4.0.0 - Session Pipeline Accuracy milestone

2026-08-25 20:38:48 +00:00

claude-bot added the

bug

backend

labels

2026-08-25 20:38:48 +00:00

rbrooks referenced this issue from a commit

2026-08-26 01:02:23 +00:00

feat(backend): resolve speaker attribution deterministically in code (#329, #342, #344)

rbrooks referenced this issue from a commit

2026-08-26 01:02:23 +00:00

feat(backend): summarise long transcripts in ordered windows (#331, #338)

rbrooks referenced this issue from a commit

2026-08-28 00:45:01 +00:00

feat(backend): persist the events a summary was built from (#332, #333, #329)

claude-bot commented

2026-08-28 19:15:21 +00:00

Author

Collaborator

Copy link

Verified. Four of five criteria were genuinely met; the fifth was half-done and is fixed in 152048e.

Criteria

Every transcript line's label is resolved deterministically in code — resolve_speakers produces the label, process_audio builds tracks from it, and transcribe_track stamps speaker/track_owner_id onto every segment. As of #342 the server's echoed label is not consulted on any path, so this is now structural rather than a convention.
The GM is always GM, and a GM with a character set does not break the legend — a GM's characters can never enter the legend.
The "use character names" instruction and the NPC-guessing sentence are removed — both survive only in docstrings describing their removal.
A machine-readable legend is produced and persisted with the transcript — see below.
Tests cover linked+character, linked without character, unlinked, GM with and without a character, two members sharing a display name — all present.

The legend was persisted only for runs that produced beats

_persist_summarisation_run returned early whenever a run had no beats, so a run that fell back to prose stored no legend at all — and those are exactly the sessions where attribution was trusted rather than checked, i.e. the ones whose legend is most worth having.

Every run now gets a row (#333), so the legend is persisted unconditionally. used_beats distinguishes the two cases, which is what made the empty row unambiguous enough to write in the first place.

One thing worth noting for later

The legend lives on summarisation_runs.speaker_legend — persisted with the run, not with the transcript. transcript_segments carries (track_owner_id, speaker_label) per line, which is the label↔owner half only: no role, no character list. Nothing stores a legend against sessions itself.

That satisfies the criterion as written — the legend is produced and persisted, stored as it was used rather than recomputed, which is the property #330 makes matter. But if something later wants "the legend for this transcript" without going through a summarisation run, it does not exist yet. Recording it rather than leaving it to be rediscovered.

Also strengthened here

_best_display_name (#344) means a speaker the bot could not name is now resolved from the linked account rather than carrying a placeholder into the transcript — one more label decided in code instead of left as a stand-in.

Closing.

**Verified. Four of five criteria were genuinely met; the fifth was half-done and is fixed in `152048e`.** ## Criteria - [x] **Every transcript line's label is resolved deterministically in code** — `resolve_speakers` produces the label, `process_audio` builds tracks from it, and `transcribe_track` stamps `speaker`/`track_owner_id` onto every segment. As of #342 the server's echoed label is not consulted on *any* path, so this is now structural rather than a convention. - [x] **The GM is always `GM`, and a GM with a character set does not break the legend** — a GM's characters can never enter the legend. - [x] **The "use character names" instruction and the NPC-guessing sentence are removed** — both survive only in docstrings describing their removal. - [x] **A machine-readable legend is produced and persisted with the transcript** — see below. - [x] **Tests cover linked+character, linked without character, unlinked, GM with and without a character, two members sharing a display name** — all present. ## The legend was persisted only for runs that produced beats `_persist_summarisation_run` returned early whenever a run had no beats, so a run that fell back to prose stored **no legend at all** — and those are exactly the sessions where attribution was trusted rather than checked, i.e. the ones whose legend is most worth having. Every run now gets a row (#333), so the legend is persisted unconditionally. `used_beats` distinguishes the two cases, which is what made the empty row unambiguous enough to write in the first place. ## One thing worth noting for later The legend lives on `summarisation_runs.speaker_legend` — persisted with the *run*, not with the transcript. `transcript_segments` carries `(track_owner_id, speaker_label)` per line, which is the label↔owner half only: no role, no character list. Nothing stores a legend against `sessions` itself. That satisfies the criterion as written — the legend is produced and persisted, stored as it was *used* rather than recomputed, which is the property #330 makes matter. But if something later wants "the legend for this transcript" without going through a summarisation run, it does not exist yet. Recording it rather than leaving it to be rediscovered. ## Also strengthened here `_best_display_name` (#344) means a speaker the *bot* could not name is now resolved from the linked account rather than carrying a placeholder into the transcript — one more label decided in code instead of left as a stand-in. Closing.

claude-bot closed this issue

2026-08-28 19:16:13 +00:00

No Branch/Tag specified

main

feat/1328-fill-it-in

feat/1328-fill-frontend

feat/1328-fill-backend

feat/9-foundry-push

fix/9-foundry-live

feat/9-foundry-push-frontend

renovate/celery-5.x

feat/5-foundry-connect

feat/9-foundry-push-backend

feat/9-foundry-push-module

feat/5-foundry-frontend

feat/5-foundry-backend

feat/5-foundry-module

release/v4.8.0-rc1

docs/d341-convert-draft

feat/958-update-to-version

feat/958-convert-sheet

feat/958-pathbuilder-import-frontend

fix/1360-heading-order

feat/958-pathbuilder-import-backend

feat/11-weekly-digest

feat/1341-auto-update

docs/d332-superseded

chore/1345-names-cleanup

feat/1342-mentioned-pages

feat/1328-recipes

fix/1346-pgdg-key

perf/1247-shared-recording-stream

chore/132-gitleaks-allowlist

release/v4.7.1

spike/foundry-module

release/v4.7.1-rc1

docs/decisions-d332-d337

fix/bot-certifi

chore/release-4.7.0

chore/release-4.7.0-rc4

chore/deps-4.7.0

chore/release-4.7.0-rc3

fix/1280-quote-must-say-it

fix/1257-names-row-squeeze

chore/release-4.7.0-rc2

fix/1245-pg-upgrade-count

chore/release-4.7.0-rc1

feat/1257-names-in-prep

feat/1257-be

feat/1257-fe

feat/1257-rating-fixes

design/1328-fill-it-in

feat/1257-name-splits

fix/1326-redis-503

feat/1280-fact-suggestions

feat/1280-be

feat/1280-fe

design/1280-fact-suggestions

feat/1279-infobox

feat/1279-be

feat/1279-fe

feat/1321-shelf-copies

design/1279-infobox

feat/1243-table-find

feat/1257-who-rated

feat/1243-prep-drawer

feat/1243-sheet-pickers

design/1243-browse-in-place

feat/1315-hotwords-default

feat/1242-system-filters

feat/1257-more-name-styles

feat/506-leg3-hotwords

design/1242-system-filters

fix/1302-packs-polish

feat/1241-rules-library

design/1241-rules-library

feat/1245-postgres-18

integ/4.6.2

fix/1263-map-ui-2

fix/1263-map-server

fix/1263-map-ui-1

fix/1261-editor-link-menu

fix/1229-axe-4.14-name-mismatch

renovate/axllent-mailpit-1.x

renovate/eslint-monorepo

renovate/authlib-1.x

chore/release-4.6.0

chore/release-4.6.0-rc2

fix/460-rc2-screens

fix/460-rc2-server

fix/release-prerelease-exit

chore/release-4.6.0-rc1

feat/125-campaign-templates

feat/126-calendar

feat/1181-pictures-and-files

feat/1190-event-people

integ/4.6.0

feat/1191-line-of-the-night

feat/1200-starter-packs

chore/bot-multidict-and-playwright-group

fix/1202-this-week-answered-count

fix/122-124-leftovers

fix/1178-get-routes-that-write

fix/1203-journeys-check-names

design/125-campaign-templates

feat/126-calendar-engine

design/126-in-world-calendar

fix/1189-cited-stamp-in-range

fix/1189-v4-looser-check

fix/1189-richer-beats

fix/1113-export-bind-lists

fix/1154-release-notes-by-file

chore/release-4.5.0

chore/release-4.5.0-rc12

fix/1135-wiki-markup-as-text

fix/demo-usage-cost-in-window

chore/release-4.5.0-rc11

fix/1121-sheet-consistency

fix/1121-sheet-5e-terms-and-card-hp

copy/1122-busiest-pages

fix/1121-relevant-choices

fix/1121-wiki-create-npc-save-sheet-links

fix/1029-menu-stays-on-screen

docs/prototype-transcript-bin-wording

fix/1023-editor-save-row-back-scroll

fix/1020-search-exact-first

fix/1020-table-prep

fix/1020-shell-wordmark-footer-bell

fix/1020-wiki-article-party

fix/872-admin-kept-until-phone-tables

feat/l13-milestone-level-waiting

fix/1000-1001-bell-older-setup-context

fix/999-pronouns-in-prompts

fix/l19-other-class-feats

fix/1010-transcript-window-ordinary-confirm

docs/d254-d269-decisions

fix/702-member-search

fix/680-transcript-bin

fix/993-unidentified-prompts

fix/841-redis-timeouts

fix/990-binned-not-exported

fix/830-export-prep-shelf

fix/deps-audit-2026-10-01

fix/877-title-reads

fix/798-audio-received-at

build/lane11-17-j7

build/lane11-16-levelup

build/lane11-14-seams

build/lane11-14-seed

build/lane11-15-start

build/lane11-13-old-block-out

fix/980-review-r3

build/lane11-12-block

build/lane11-8-packs

fix/demo-seed-canonical-names

build/lane11-11-hot-half

docs/l11-decisions

build/lane11-7-convert

build/lane11-10-editing

fix/npm-audit-brace-expansion

build/lane11-9-sheet-page

fix/bot-pyjwt

build/lane11-6-content

fix/lane11-rank-names

build/lane11-2b-language

build/lane11-4-js-evaluator

build/lane11-3-server

build/lane11-5a-pf2e

build/lane11-5b-5e

feat/design-admin

build/lane11-2-evaluator

build/lane11-1-sheet-pieces

build/lane11-0-drawing

fix/943-quest-objectives-fk

fix/prepsheet-shelf-flake

fix/945-leftovers

build/demo-gm-copy

fix/901-email-landings

build/lane10-8-journeys

build/demo-seed-lanes-8-10

fix/887-gm-own-confirmation

build/lane9-5-j1-admin

fix/918-audience-click

fix/useasync-overtaken

ci/939-critical-path

test/937-backend-suite-speed

chore/claude-orchestration

fix/924-xp-gm-only

fix/session-stale-poll

docs/decisions-d239-d242

fix/930-pypdf

build/lane8-journeys

build/925-quest-editor

fix/894-private-page-pictures

build/lane8-p4-drafts

build/lane8-p7-map

build/lane8-p6-handouts-rules

fix/wiki-fixture-names

build/lane10-d227-test-usage

fix/rsvp-horizon-test-tuesday

fix/media-volume-owner

build/lane8-p3-editor

fix/912-unidentified-rename

build/lane8-p5-party-quests

build/lane8-handout-takeback

build/lane10-pickers

build/lane8-p2-article

build/lane8-6-map

build/lane10-7b-email-pages

build/lane8-5-handouts

build/lane10-7a-test-the-bot-pages

build/lane8-p1-pages

build/lane8-4-quests

build/lane8-2b-feeds

build/lane8-3-parties

build/lane8-2a-links

build/lane10-5-commands

build/lane9-6-seed

build/lane10-4b-reactions

build/lane9-4d-pages

build/lane9-4c-pages

build/lane10-4a-posts

build/lane10-3-email

fix/891-web-fonts

build/lane9-4b-pages

build/lane10-6-test-the-bot

build/lane10-2-notify-out

build/lane8-1-wiki-text

build/lane9-4a-frame

design/lane10-drawings

fix/j6-budget

fix/j5-phone-flake

fix/869-test-leaks

fix/843-record-start-followups

build/lane7-all

build/lane6-all

build/lane9-b2-server

chore/changelog-fragments

docs/decisions-d218

build/lane9-a-owner-overview

fix/rule-card-level-label

fix/834-retention-semantics

fix/o3c-record-without-announcement

docs/decisions-d179

docs/decisions-d131-d170

fix/851-commit-before-response

build/lane6-j3-seed

build/lane9-b1-d67-identity

fix/841-blocking-loop

build/lane9-c-backups-audit

fix/vote-race-and-start-commit

fix/recording-announcement

fix/proposal-read-leak

fix/last-admin-guard

build/lane6-p4c-planning

build/lane6-p4b-margin

build/lane6-p4a-sheet

build/lane5-j4-seed

build/lane6-base

build/lane7-pages-talk

build/lane5-pages

build/lane7-pages-safety

build/lane7-p3-safety

build/lane7-p2-notify

build/lane6-p3-planning

build/lane6-p2-history

build/lane6-p1b-items

build/lane7-p1-threads

build/lane5-p6-faces

build/lane6-p1a-sheet

build/lane5-p5-find

build/lane5-p4a-board

build/lane5-p4b-scenes

build/lane5-p3-live

build/lane5-base

build/lane5-p2-cards

docs/lane5-decisions

build/review-r2-wide

fix/review-r1

build/lane5-p1-scenes

build/demo-seed

build/lane4-journeys-j5

build/lane4-page-review-b

build/lane4-page-review-a

build/lane4-page-read

build/lane4-journeys

build/lane4-page-sess2

build/lane4-page-sess1

build/lane4-notes

build/lane4-discord-on-approval

build/lane4-scheduling

build/lane4-versions

build/760-session-number

build/lane4-decisions

build/797-erase-no-regenerate

build/test-race-audit

build/lane3-completions

build/794-dark-only

build/796-titles-scroll

build/791-retention-anchor

build/a11y-sweep

build/shell-followups

build/retention-preview

build/753-backup-readback

docs/d91-d93

build/lane3-j1

build/ci-harness-paths-selftest

fix/useauth-test-race

build/ci-harness-paths

build/777-ai-settings-read

build/lane3-settings-c

build/lane3-settings-b

build/lane3-j6

build/lane3-sessionform

build/lane3-settings-a

build/lane3-members

build/lane3-thisweek

build/lane3-story

chore/87-align-python-3.12

v4.8.0-rc1

v4.7.1

v4.7.1-rc1

v4.7.0

v4.7.0-rc4

v4.7.0-rc3

v4.7.0-rc2

v4.7.0-rc1

v4.6.2

v4.6.2-rc4

v4.6.2-rc3

v4.6.2-rc2

v4.6.2-rc1

v4.6.1

v4.6.1-rc1

v4.6.0

v4.6.0-rc2

v4.6.0-rc1

v4.5.3

v4.5.3-rc1

v4.5.2

v4.5.2-rc1

v4.5.1

v4.5.1-rc2

v4.5.1-rc1

v4.5.0

v4.5.0-rc12

v4.5.0-rc11

v4.5.0-rc10

v4.5.0-rc9

v4.5.0-rc8

v4.5.0-rc7

v4.5.0-rc6

v4.5.0-rc5

v4.5.0-rc4

v4.5.0-rc3

v4.5.0-rc2

v4.5.0-rc1

v4.4.1

v4.4.0

v4.3.0

v4.2.3

v4.2.2

v4.2.1

v4.2.0

v4.1.2

v4.1.1

v4.1.0

v4.0.1

v4.0.0

v3.11.4

v3.11.3

v3.11.2

v3.11.1

v3.11.0

v3.10.0

v3.9.1

v3.9.0

v3.8.0

v3.7.0

v3.6.1

v3.6.0

v3.5.0

v3.4.1

v3.4.0

v3.3.0

v3.2.0

v3.1.0

v3.0.0

v2.1.0

v2.0.0

v1.1.0

v1.0.0

v0.12.0

v0.11.0

v0.10.0

v0.9.1

v0.3.3

v0.9.0

v0.8.0

v0.3.2

v0.3.1

v0.3.0

v0.7.0

v0.6.0

v0.5.0

v0.4.0

v0.2.1

v0.2.0

v0.1.1

v0.1.0

Labels

Clear labels

backend

Issue related to the backend

bot

Issue related to the bot

bug

Something is not working

duplicate

This issue or pull request already exists

enhancement

New feature

frontend

Issue related to the frontend

More information is needed

recording

Issue related to the recording system

security

Security hardening / vulnerability work

No labels

Milestone

Clear milestone

No items

No milestone

v4.0.0 - Session Pipeline Accuracy

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

rbrooks/Quest-Board#329

Reference in a new issue

Repository

rbrooks/Quest-Board

Title

Body

No description provided.

Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?

Rows
Columns