[Backend] Let GMs set a campaign's spoken language, and pass it to transcription #419

New issue

Closed

opened 2026-08-26 03:37:02 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-08-26 03:37:02 +00:00

Contributor

Severity: MEDIUM. Follow-up to the 2026-08-26 production incident (see #347).

Why

On 2026-08-26 a 77-minute six-speaker session was lost because WhisperX identified three seconds of English as Icelandic at 0.95 confidence, then raised ValueError: No default align-model for language: is. #347 fixed the mechanism — the language is now detected once from the longest span in the session and pinned for every other span, instead of being guessed per 1-3 second clip.

That makes the default path safe. It does not make it certain, and the failure this protects against is worse than the one we actually saw:

The 500 was the lucky outcome. It was loud, and it stopped the pipeline. WhisperX ships aligners for German, Spanish, Dutch, French and others. Had the misdetection landed on one of those rather than Icelandic, that clip would have returned a confident, wrong transcription and flowed straight into the transcript, the summary and the lore wiki with nothing marking it — undetectable after the fact. Detection is a heuristic, and a heuristic in the attribution path is exactly what the v4.0.0 milestone exists to remove.

A GM knows what language their table speaks. Asking them beats guessing.

Design

An override, not a replacement. Detection stays the default so the product works before anyone finds a setting.

campaigns.transcription_language — nullable text, ISO 639-1 (en, pt, de, …). NULL means "detect", which is the default and current behaviour.
Campaign settings UI: a language select with an explicit "Detect automatically (default)" option, not a blank.
process_audio reads it and passes it through. The backend already accepts it — transcribe_session_vad(..., language=...) and transcribe_with_optional_vad(..., language=...) skip detection entirely when supplied (#347).

The default must stay "detect". Defaulting to en would silently mangle every non-English table until they found the setting — trading a loud failure for a quiet one, which is the wrong direction and the whole point of the paragraph above.

Worth considering while in here

Restrict the select to languages WhisperX has an alignment model for, or warn on ones it does not — otherwise a GM can configure the exact crash #347 just fixed.
Log the language actually used per session, so a bad transcript can be diagnosed after the fact rather than guessed at.

Acceptance criteria

campaigns.transcription_language exists, nullable, defaulting to NULL
The campaign settings UI offers "Detect automatically" plus an explicit language list
process_audio passes the configured language through to transcription
When set, no detection probe is issued at all
When unset, behaviour is byte-identical to #347's detect-once-and-pin
The language used is recorded in the logs for the session
Tests cover: configured language wins; unset falls back to detection; a language with no aligner is surfaced rather than silently accepted

**Severity: MEDIUM.** Follow-up to the 2026-08-26 production incident (see #347). ## Why On 2026-08-26 a 77-minute six-speaker session was lost because WhisperX identified three seconds of English as **Icelandic** at 0.95 confidence, then raised `ValueError: No default align-model for language: is`. #347 fixed the mechanism — the language is now detected once from the longest span in the session and pinned for every other span, instead of being guessed per 1-3 second clip. That makes the default path safe. It does not make it *certain*, and the failure this protects against is worse than the one we actually saw: **The 500 was the lucky outcome.** It was loud, and it stopped the pipeline. WhisperX ships aligners for German, Spanish, Dutch, French and others. Had the misdetection landed on one of those rather than Icelandic, that clip would have returned a confident, wrong transcription and flowed straight into the transcript, the summary and the lore wiki with nothing marking it — undetectable after the fact. Detection is a heuristic, and a heuristic in the attribution path is exactly what the v4.0.0 milestone exists to remove. A GM knows what language their table speaks. Asking them beats guessing. ## Design An **override, not a replacement**. Detection stays the default so the product works before anyone finds a setting. - `campaigns.transcription_language` — nullable text, ISO 639-1 (`en`, `pt`, `de`, …). `NULL` means "detect", which is the default and current behaviour. - Campaign settings UI: a language select with an explicit **"Detect automatically (default)"** option, not a blank. - `process_audio` reads it and passes it through. The backend already accepts it — `transcribe_session_vad(..., language=...)` and `transcribe_with_optional_vad(..., language=...)` skip detection entirely when supplied (#347). **The default must stay "detect".** Defaulting to `en` would silently mangle every non-English table until they found the setting — trading a loud failure for a quiet one, which is the wrong direction and the whole point of the paragraph above. ## Worth considering while in here - Restrict the select to languages WhisperX has an alignment model for, or warn on ones it does not — otherwise a GM can configure the exact crash #347 just fixed. - Log the language actually used per session, so a bad transcript can be diagnosed after the fact rather than guessed at. ## Acceptance criteria - [ ] `campaigns.transcription_language` exists, nullable, defaulting to NULL - [ ] The campaign settings UI offers "Detect automatically" plus an explicit language list - [ ] `process_audio` passes the configured language through to transcription - [ ] When set, no detection probe is issued at all - [ ] When unset, behaviour is byte-identical to #347's detect-once-and-pin - [ ] The language used is recorded in the logs for the session - [ ] Tests cover: configured language wins; unset falls back to detection; a language with no aligner is surfaced rather than silently accepted

claude-bot referenced this issue

2026-08-26 03:37:25 +00:00

fix: a misdetected language on a 3-second clip must not cost a session (#347) #420

claude-bot added this to the v4.0.0 - Session Pipeline Accuracy milestone

2026-08-26 03:37:37 +00:00

claude-bot added the

backend

enhancement

labels

2026-08-26 03:37:41 +00:00

rbrooks referenced this issue from a commit

2026-08-26 05:29:07 +00:00

feat(backend,frontend): let a GM pin their table's spoken language (#419)

claude-bot commented

2026-08-27 23:27:40 +00:00

Author

Contributor

Verified against the acceptance criteria before closing. All seven met, with direct test coverage — the cleanest issue in this milestone's verification pass.

campaigns.transcription_language, nullable, defaults to NULL — migration b3c4d5e6f7a9. Pinned by test_the_default_is_detect_not_english (tests/test_transcription_language.py:98), which asserts both CampaignCreate(name="Test").transcription_language is None and language_service.DETECT is None. This was the criterion I most wanted checked: a default of en would have silently mistranscribed every non-English group until someone found the setting, replacing a loud failure with a quiet one.
UI offers "Detect automatically" plus an explicit list — CampaignDetail.jsx:2204-2215.
process_audio passes it through — reminder_tasks.py:2192-2207.
When set, no detection probe is issued — test_a_configured_language_suppresses_detection_entirely asserts probes == [].
When unset, the #347 detect-once-and-pin path is unchanged — test_an_unset_language_still_detects; the underlying logic is gated by a parameter defaulting to None, not rewritten.
Language used is logged — per session (reminder_tasks.py:2195-2201) and per track when detection runs (audio_service.py:896-906).
Unsupported language surfaced — test_a_language_the_aligner_cannot_handle_is_refused uses Icelandic, the exact #347 failure, rejected at the Pydantic boundary rather than at the aligner.

That last one is the good part: the crash that cost a session is now unreachable by configuration.

Closing. Part of a full acceptance-criteria pass across the v4.0.0 milestone.

Verified against the acceptance criteria before closing. All seven met, with direct test coverage — the cleanest issue in this milestone's verification pass. - **`campaigns.transcription_language`, nullable, defaults to NULL** — migration `b3c4d5e6f7a9`. Pinned by `test_the_default_is_detect_not_english` (`tests/test_transcription_language.py:98`), which asserts both `CampaignCreate(name="Test").transcription_language is None` and `language_service.DETECT is None`. This was the criterion I most wanted checked: a default of `en` would have silently mistranscribed every non-English group until someone found the setting, replacing a loud failure with a quiet one. - **UI offers "Detect automatically" plus an explicit list** — `CampaignDetail.jsx:2204-2215`. - **`process_audio` passes it through** — `reminder_tasks.py:2192-2207`. - **When set, no detection probe is issued** — `test_a_configured_language_suppresses_detection_entirely` asserts `probes == []`. - **When unset, the #347 detect-once-and-pin path is unchanged** — `test_an_unset_language_still_detects`; the underlying logic is gated by a parameter defaulting to `None`, not rewritten. - **Language used is logged** — per session (`reminder_tasks.py:2195-2201`) and per track when detection runs (`audio_service.py:896-906`). - **Unsupported language surfaced** — `test_a_language_the_aligner_cannot_handle_is_refused` uses Icelandic, the exact #347 failure, rejected at the Pydantic boundary rather than at the aligner. That last one is the good part: the crash that cost a session is now unreachable by configuration. Closing. Part of a full acceptance-criteria pass across the v4.0.0 milestone.

claude-bot closed this issue

2026-08-27 23:27:52 +00:00

rbrooks referenced this issue from a commit

2026-08-28 19:03:44 +00:00

fix(backend): remove the unverifiable attribution join entirely (#342)

claude-bot referenced this issue

2026-08-28 19:14:01 +00:00

[Backend] Stop the ASR server's speaker label overriding the known track owner #342