[Backend] Fix per-span language detection and lost context on the VAD path #347

New issue

Closed

opened 2026-08-25 20:39:03 +00:00 by claude-bot · 0 comments

claude-bot commented

2026-08-25 20:39:03 +00:00

Contributor

Severity: MEDIUM. Found in the August 2026 session lifecycle review (#319). Becomes live as soon as VAD is enabled by default in v3.11.5.

The defect

The two transcription paths use different endpoints: VAD transcribes each span independently via /transcribe, while the non-VAD path sends whole tracks to /transcribe/session (audio_service.py:747-749 vs :474-546).

Independent per-span requests mean independent language detection per span. A two-second "mm-hm" span can come back detected as a different language and transcribed as garbage. It also loses Whisper's conditioning context across span boundaries, which degrades proper nouns and mid-sentence continuations exactly where a fantasy campaign needs them most.

The code comment claiming the VAD path is "indistinguishable from transcribing the untrimmed track" (:726-727) is true for timeline arithmetic only — that part is verified correct — not for transcription output.

Proposed fix

Pin the language explicitly rather than detecting per span, from a campaign-level setting. Pass an initial prompt or conditioning context carrying the previous span's tail plus campaign proper nouns, so spans are not decoded blind. Where the server supports it, batch a track's spans into one request so conditioning is shared.

Correct the misleading comment while in here.

Acceptance criteria

Language is pinned per campaign, not detected per span
Spans receive conditioning context from the preceding span
Campaign proper nouns are supplied as bias where the provider supports it
The equivalence comment is corrected to say "timeline arithmetic only"
A short span between two long ones no longer detects a different language in tests

**Severity: MEDIUM.** Found in the August 2026 session lifecycle review (#319). Becomes live as soon as VAD is enabled by default in v3.11.5. ## The defect The two transcription paths use different endpoints: VAD transcribes each span independently via `/transcribe`, while the non-VAD path sends whole tracks to `/transcribe/session` (`audio_service.py:747-749` vs `:474-546`). Independent per-span requests mean **independent language detection per span**. A two-second "mm-hm" span can come back detected as a different language and transcribed as garbage. It also loses Whisper's conditioning context across span boundaries, which degrades proper nouns and mid-sentence continuations exactly where a fantasy campaign needs them most. The code comment claiming the VAD path is "indistinguishable from transcribing the untrimmed track" (`:726-727`) is true for **timeline arithmetic only** — that part is verified correct — not for transcription output. ## Proposed fix Pin the language explicitly rather than detecting per span, from a campaign-level setting. Pass an initial prompt or conditioning context carrying the previous span's tail plus campaign proper nouns, so spans are not decoded blind. Where the server supports it, batch a track's spans into one request so conditioning is shared. Correct the misleading comment while in here. ## Acceptance criteria - [ ] Language is pinned per campaign, not detected per span - [ ] Spans receive conditioning context from the preceding span - [ ] Campaign proper nouns are supplied as bias where the provider supports it - [ ] The equivalence comment is corrected to say "timeline arithmetic only" - [ ] A short span between two long ones no longer detects a different language in tests

claude-bot added this to the v4.0.0 - Session Pipeline Accuracy milestone

2026-08-25 20:39:03 +00:00

claude-bot added the

bug

backend

recording

labels

2026-08-25 20:39:03 +00:00