feat(asr): feed campaign proper nouns to the transcriber as vocabulary bias (#355) #491

Merged

claude-bot merged 3 commits from feat/355-asr-vocabulary into main 2026-09-05 02:03:06 +00:00

Conversation 0 Commits 3 Files changed 8 +1096 -14

claude-bot commented 2026-09-05 01:55:43 +00:00

Contributor

Copy link

Closes #355 (all criteria except the golden-corpus measurement — see below).

Every transcription request for a session now carries the campaign's proper nouns through the ASR contract's vocabulary input (#350), which LocalWhisperProvider maps to Whisper's initial_prompt.

What

• app/services/vocabulary_service.py — campaign_vocabulary(db, campaign_id, *, limit) over three sources, ranked in tiers so a whole tier survives truncation before the next starts: (0) party character names, (1) wiki entry titles of proper-noun types + NPC cards, ranked by session-appearance count then recency, (2) aliases. Case-insensitive dedupe after ranking so the higher-ranked spelling wins. Excludes trashed entries (#408) and unaccepted bot proposals, so the pipeline cannot teach itself its own misspelling. Cleaning drops junk (single lowercase words, >6-word titles, edge punctuation) and is scoped to cased scripts.
• reminder_tasks.resolve_asr_vocabulary — resolves the ASR provider, reads its declared vocabulary style and limit, returns nothing without querying when the provider has no vocabulary support, logs the count only.
• Threaded through every request in audio_service: plain, VAD, per-span, and the language-detection probe (whose transcript is kept as that span's result, so skipping it would leave one span per track unbiased). asr_provider now travels with it — it was read from settings and dropped before.
• No cache, by design: three small SELECTs once per session versus an invalidation rule that must fire on rename, alias, trash and restore.

Not claimed

"Measured effect recorded against the golden corpus" — the synthetic-audio harness (#433) that can measure proper-noun survival is not built yet; the measurement is a follow-up on that issue. Also note the bundled WhisperX server currently ignores initial_prompt; the server-side #352 change (in flight in the infrastructure repo) makes it honour it.

Verification

• Backend suite: 1750 passed, 0 failed (34 new tests: ranking, sourcing against real rows, and the multipart field actually sent, on both the plain and VAD paths).
• ruff format/check clean; version sync OK; no bot contract change.

🤖 Generated with Claude Code

Closes #355 (all criteria except the golden-corpus measurement — see below). Every transcription request for a session now carries the campaign's proper nouns through the ASR contract's `vocabulary` input (#350), which `LocalWhisperProvider` maps to Whisper's `initial_prompt`. ## What - **`app/services/vocabulary_service.py`** — `campaign_vocabulary(db, campaign_id, *, limit)` over three sources, ranked in tiers so a whole tier survives truncation before the next starts: (0) party character names, (1) wiki entry titles of proper-noun types + NPC cards, ranked by session-appearance count then recency, (2) aliases. Case-insensitive dedupe *after* ranking so the higher-ranked spelling wins. Excludes trashed entries (#408) and unaccepted bot proposals, so the pipeline cannot teach itself its own misspelling. Cleaning drops junk (single lowercase words, >6-word titles, edge punctuation) and is scoped to cased scripts. - **`reminder_tasks.resolve_asr_vocabulary`** — resolves the ASR provider, reads its declared vocabulary style and limit, returns nothing without querying when the provider has no vocabulary support, logs the *count* only. - Threaded through every request in `audio_service`: plain, VAD, per-span, and the language-detection probe (whose transcript is kept as that span's result, so skipping it would leave one span per track unbiased). `asr_provider` now travels with it — it was read from settings and dropped before. - No cache, by design: three small SELECTs once per session versus an invalidation rule that must fire on rename, alias, trash and restore. ## Not claimed "Measured effect recorded against the golden corpus" — the synthetic-audio harness (#433) that can measure proper-noun survival is not built yet; the measurement is a follow-up on that issue. Also note the bundled WhisperX server currently ignores `initial_prompt`; the server-side #352 change (in flight in the infrastructure repo) makes it honour it. ## Verification - Backend suite: **1750 passed**, 0 failed (34 new tests: ranking, sourcing against real rows, and the multipart field actually sent, on both the plain and VAD paths). - ruff format/check clean; version sync OK; no bot contract change. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 3 commits 2026-09-05 01:55:43 +00:00

feat(asr): feed campaign proper nouns to the transcriber as vocabulary bias ... a75893dbcc

Invented names are the one class of word a general-purpose speech model has
never encountered, and they are also the words a session turns on. Nothing
downstream can repair one: the summariser, the beat extractor and the wiki
proposer all read the transcript rather than the audio, so a name the ASR heard
as two ordinary words is wrong everywhere it appears, permanently.

The provider contract (#350) has carried a `vocabulary` input since it landed —
the bundled Whisper adapter spells it as `initial_prompt`, a managed one will
spell it as keyterms — and nothing has ever filled it. This fills it, from the
campaign's own wiki.

`vocabulary_service.campaign_vocabulary` assembles the list from three sources
in a deliberate order: members' character names first, then wiki titles for the
entry types whose titles are names people say out loud, then aliases. Within
the wiki, session appearances rank the list, because that is the nearest thing
the schema has to "how often is this actually spoken", and recency breaks ties.
The order is load-bearing rather than cosmetic — the local adapter accepts 100
terms — so the rank/clean/dedupe/truncate step is a pure function, testable
without a database.

Cleaning is deliberately aggressive: a term that is a single all-lowercase word
is dropped, because Whisper reads the prompt as style as well as vocabulary and
"the" in a proper-noun list spends budget making the transcript worse. Trashed
entries and unaccepted bot proposals are excluded — feeding back a spelling the
GM never approved would let the pipeline teach itself its own mistake.

No cache. It is three small SELECTs once per recorded session, against a
transcription that takes minutes, so "refreshed as the wiki changes" is true by
construction instead of by an invalidation rule someone has to remember.

Wired through every request rather than the first: `transcribe_session`,
`transcribe_session_vad`, `transcribe_track_vad`, `_transcribe_span_resiliently`
and the language probe all carry it, because a VAD-trimmed track is hundreds of
separate requests and the provider holds no state between them. The probe
carries it precisely because its transcript is kept and reused.

`asr_provider` is threaded down the same paths. The configured provider name
was being read from settings and then dropped before transcription, which was
harmless with one adapter and would not have been the moment a second arrived —
`process_audio` now reads the vocabulary limit from the capabilities of the
adapter that will actually run.

Counts only in the logs. The terms are the campaign's own content.

Refs #355

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

test(asr): pin the vocabulary list's ranking, sourcing and transport ... cb4725e2f4

Three layers, because three separate things can fail independently and only one
of them is visible from the outside.

The selection policy is tested as a pure function: what survives truncation,
that a bare dictionary word is not a proper noun, that an uncased script is not
mistaken for one, that a name is kept whole, and that a rejected term does not
consume a slot the provider was willing to fill.

The sourcing is tested against a real campaign with the exclusions present in
the database rather than merely absent — a trashed entry, an unaccepted bot
proposal, a free-form concept and an event title all sit there and all stay out.

The transport is tested at the wire: the constructor of `httpx.AsyncClient` is
replaced and the multipart form fields actually sent are asserted on, for both
the plain and the VAD path. The VAD assertion is per span rather than per
session — a vocabulary attached once would bias one clip out of dozens and look
identical to a working one from every angle except the transcript.

The existing VAD doubles grow the new keyword arguments. Two of them take a
catch-all, being doubles for unrelated properties; the two in test_vad_trim
name them, and assert the terms arrive, since forwarding is what that file is
about.

Refs #355

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

docs(changelog): note that transcription now knows your campaign's names ... 3c659ab70c

Refs #355

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

claude-bot scheduled this pull request to auto merge when all checks succeed 2026-09-05 01:55:53 +00:00

claude-bot merged commit a00a827529 into main 2026-09-05 02:03:06 +00:00

claude-bot deleted branch feat/355-asr-vocabulary 2026-09-05 02:03:07 +00:00

claude-bot referenced this pull request from a commit 2026-09-05 02:03:07 +00:00

Merge PR #491: feed campaign proper nouns to the transcriber as vocabulary bias (#355)

claude-bot referenced this pull request 2026-09-05 02:27:34 +00:00

[Backend] Feed campaign proper nouns to the ASR as vocabulary bias #355

claude-bot referenced this pull request 2026-09-05 06:04:34 +00:00

chore(release): v4.2.0 — AI Provider Abstraction & BYO-AI #504

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

backend

Issue related to the backend

  

bot

Issue related to the bot

  

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

frontend

Issue related to the frontend

  

help wanted

Need some help

  

invalid

Something is wrong

  

question

More information is needed

  

recording

Issue related to the recording system

  

security

Security hardening / vulnerability work

  

wontfix

This won't be fixed

No labels

backend

bot

bug

duplicate

enhancement

frontend

help wanted

invalid

question

recording

security

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

rbrooks/Quest-Board!491

No description provided.