Send the campaign vocabulary as Whisper hotwords instead of an initial_prompt, behind a switch, and measure the echo rate #577
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why. The bundled WhisperX server (v2 API, #352) advertises both
initial_promptandhotwords, butproviders/asr.pyonly knows how to sendinitial_prompt:resolve_capabilitiesmaps a declaredinitial_prompttoVocabularyStyle.INITIAL_PROMPTand there is noHOTWORDSstyle at all. On the 2026-09-09 session the 36-term prompt biased the names only weakly (the transcript still has Adani, Bidani, Parahark, Nick) while Whisper echoed fragments of the prompt on silent stretches (#569, #576). faster-whisper'shotwordsparameter is the mechanism designed for term biasing; whether it echoes less and biases more is an empirical question this issue exists to answer on the dev replay.Change.
VocabularyStyle.HOTWORDS; the local adapter sends the terms as thehotwordsform field when that style is selected and the server declareshotwords: true; the server wrapper in iac-repo already accepts it.initial_prompt): an environment variable on the worker is enough for the experiment (QB_ASR_VOCABULARY_STYLE=hotwords|initial_prompt|none), with the Admin → Bot Settings field to follow only if hotwords wins.ASR vocabulary:log line already prints the style; the run's usage/summarisation record should carry it too so a comparison can be read back later.Measure on dev, same audio as the baseline (session
5b0d6ac1…): name spellings for the five characters in the transcript (count of correct vs variant spellings per name), thehallucination_dropssplit, and the fragment-echo count from #576's query. Baseline numbers are on #569.Out of scope: changing the vocabulary content (#574 covers the parenthesis).
Measured on the dev replay (same audio, VAD defaults, filter on),
initial_promptvshotwords. The worker loggedstyle=hotwordson every track and the run record stampsvocabulary_style = hotwords, so the arm ran as intended.Verdict: no measurable difference. The outputs are near-identical, which is consistent with faster-whisper implementing
hotwordsby feeding the terms through the same previous-text slot the prompt uses; the echo rate is the same because the same words reach the decoder the same way. The switch stays (it is documented, default unchanged, and the stamp on the run makes future comparisons free), but it is not a lever for this problem.What the numbers say the real problem is. "Viq" is heard as "Vic" every single time, with or without bias. No prompt will overturn a common English word the acoustics already match. The fix is deterministic and GM-controlled: let a character carry "heard as" aliases (Vic, Vick) and snap whole-word matches to the character's name after transcription, the way the lore-entry aliases already exist for wiki linking. Filed as #579.
Dev is left on
hotwordsfor now (harmless, equivalent); prod is unchanged.