[Backend] Feed campaign proper nouns to the ASR as vocabulary bias #355

Closed
opened 2026-08-25 20:39:17 +00:00 by claude-bot · 2 comments
Contributor

Found in the August 2026 session lifecycle review (#319).

Why

Fantasy proper nouns are precisely what a generic speech model gets wrong, and no amount of downstream prompting repairs a name the ASR never heard. Quest Board is unusually well placed here: it already holds every PC name, NPC name, place and faction for the campaign in its own wiki.

Most providers accept some form of vocabulary bias — keyterms, keyword boosting, custom dictionaries, or an initial prompt. The provider contract already carries a vocabulary input; this issue fills it.

Proposed fix

Assemble a per-campaign term list from the wiki: character names, NPC names, places, factions, and aliases, ranked by recency and appearance count so the list stays inside provider limits (commonly a couple of hundred terms). Pass it on every transcription request for that campaign. Refresh it as the wiki grows.

Measure the effect on the golden corpus rather than assuming it, and record the result — this is the kind of change that is easy to believe in and hard to verify.

Acceptance criteria

  • Term list assembled per campaign from wiki entities and aliases
  • Ranked and truncated to the provider's declared limit
  • Passed through the contract's vocabulary input on every request
  • Refreshed as the wiki changes
  • Measured effect recorded against the golden corpus
Found in the August 2026 session lifecycle review (#319). ## Why Fantasy proper nouns are precisely what a generic speech model gets wrong, and no amount of downstream prompting repairs a name the ASR never heard. Quest Board is unusually well placed here: it already holds every PC name, NPC name, place and faction for the campaign in its own wiki. Most providers accept some form of vocabulary bias — keyterms, keyword boosting, custom dictionaries, or an initial prompt. The provider contract already carries a `vocabulary` input; this issue fills it. ## Proposed fix Assemble a per-campaign term list from the wiki: character names, NPC names, places, factions, and aliases, ranked by recency and appearance count so the list stays inside provider limits (commonly a couple of hundred terms). Pass it on every transcription request for that campaign. Refresh it as the wiki grows. Measure the effect on the golden corpus rather than assuming it, and record the result — this is the kind of change that is easy to believe in and hard to verify. ## Acceptance criteria - [ ] Term list assembled per campaign from wiki entities and aliases - [ ] Ranked and truncated to the provider's declared limit - [ ] Passed through the contract's `vocabulary` input on every request - [ ] Refreshed as the wiki changes - [ ] Measured effect recorded against the golden corpus
Author
Contributor

Landed in PR #491 (merged 2026-09-05, CI green, 1750 tests).

  • app/services/vocabulary_service.py assembles the list per session from three sources in survival tiers — party character names, then wiki titles of proper-noun types plus NPC cards ranked by session appearances and recency, then aliases — deduped after ranking so the better-ranked spelling wins; trashed entries and unaccepted bot proposals excluded so the pipeline cannot teach itself its own misspelling.
  • reminder_tasks.resolve_asr_vocabulary reads the resolved ASR provider's declared vocabulary style and limit, and the list travels on every request: plain, VAD, per-span, and the language probe. No cache; it is rebuilt from the wiki each time a recording is processed.

Not claimed: "measured effect against the golden corpus" — the #433 harness that can measure proper-noun survival is being built now; the measurement is a follow-up there. Also note: the bundled WhisperX server ignored initial_prompt until its API v2 (Rhoving/iac-repo#396, deploying today), so the bias only takes effect against the updated server.

Landed in **PR #491** (merged 2026-09-05, CI green, 1750 tests). - `app/services/vocabulary_service.py` assembles the list per session from three sources in survival tiers — party character names, then wiki titles of proper-noun types plus NPC cards ranked by session appearances and recency, then aliases — deduped after ranking so the better-ranked spelling wins; trashed entries and unaccepted bot proposals excluded so the pipeline cannot teach itself its own misspelling. - `reminder_tasks.resolve_asr_vocabulary` reads the resolved ASR provider's declared vocabulary style and limit, and the list travels on every request: plain, VAD, per-span, and the language probe. No cache; it is rebuilt from the wiki each time a recording is processed. **Not claimed:** "measured effect against the golden corpus" — the #433 harness that can measure proper-noun survival is being built now; the measurement is a follow-up there. **Also note:** the bundled WhisperX server ignored `initial_prompt` until its API v2 (Rhoving/iac-repo#396, deploying today), so the bias only takes effect against the updated server.
Author
Contributor

Measured effect of the vocabulary bias (PR #505; synthetic-session harness ASR leg against the bundled WhisperX server, API v2, large-v3-turbo, the invented campaign's marker script).

The #433 calibration had been run with vocabulary off, so that run is the no-bias baseline. Three conditions, each run twice:

condition marker detection WER markers from the other assembly fabricated names
control, no vocabulary (reproduces the calibration) 72.2 % baseline 0 0
the 24 script markers as the term list (ceiling) 96.3 % unchanged 0 0
the 24 markers diluted in a 100-term list (realistic) 81.5 % unchanged 0 0
  • The diluted row is the one to plan around: a real campaign's vocabulary carries far more names than any one session mentions, and the +9.3 pp there is what a GM should expect. The ceiling row shows what a short, session-specific list would buy.
  • WER did not move in any condition and no marker leaked in from the other assembly, so the bias is not buying recall with hallucinated proper nouns.
  • Run-to-run spread was zero and the control reproduced the calibration run bit-identically, so the harness is stable enough to carry comparisons of this size.

Two things to try next, tracked separately: WhisperX v2 also honours hotwords, which was not exercised here, and the marker scorer's denominator needs a small fix before the two assemblies can be compared against each other (filed as a harness follow-up).

**Measured effect of the vocabulary bias** (PR #505; synthetic-session harness ASR leg against the bundled WhisperX server, API v2, `large-v3-turbo`, the invented campaign's marker script). The #433 calibration had been run with vocabulary **off**, so that run is the no-bias baseline. Three conditions, each run twice: | condition | marker detection | WER | markers from the other assembly | fabricated names | |---|---|---|---|---| | control, no vocabulary (reproduces the calibration) | 72.2 % | baseline | 0 | 0 | | the 24 script markers as the term list (ceiling) | **96.3 %** | unchanged | 0 | 0 | | the 24 markers diluted in a 100-term list (realistic) | **81.5 %** | unchanged | 0 | 0 | - The diluted row is the one to plan around: a real campaign's vocabulary carries far more names than any one session mentions, and the +9.3 pp there is what a GM should expect. The ceiling row shows what a short, session-specific list would buy. - WER did not move in any condition and no marker leaked in from the other assembly, so the bias is not buying recall with hallucinated proper nouns. - Run-to-run spread was zero and the control reproduced the calibration run bit-identically, so the harness is stable enough to carry comparisons of this size. Two things to try next, tracked separately: WhisperX v2 also honours `hotwords`, which was not exercised here, and the marker scorer's denominator needs a small fix before the two assemblies can be compared against each other (filed as a harness follow-up).
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#355
No description provided.