[Backend] Stop the ASR server's speaker label overriding the known track owner

claude-bot commented

2026-08-25 20:39:00 +00:00

Collaborator

Severity: HIGH. Found in the August 2026 session lifecycle review (#319).

The defect

The backend knows exactly who owns each track — it sent one file per speaker — and then discards that knowledge in favour of whatever the remote server echoes back.

audio_service.py:535 — transcribe_session takes seg.get("speaker", "Unknown") with no cross-check
audio_service.py:460 — on the VAD path, seg.get("speaker") or display_name gives the server's value precedence over the known owner

If the server runs diarization or normalises to a label like SPEAKER_00, that silently replaces the real name. _apply_character_names (reminder_tasks.py:2121-2130) then fails to match, and the character mapping is lost for that whole track. If the server associates results by completion order rather than by the order files were sent, every line of a track is attributed to the wrong player — wholesale, plausible-looking, and undetectable from inside the backend.

Proposed fix

The track owner is authoritative. Make the fallback unconditional: use display_name always, and treat any server-supplied speaker label as advisory metadata at most. On the session endpoint, zip results back to tracks by an explicit identifier rather than by position.

This is a one-character change on the VAD path and a small one on the session path, and it removes an entire class of silent misattribution.

Acceptance criteria

Speaker labels derive from track ownership, never from the server echo
Results are associated with tracks by explicit id, not by ordering
A server returning SPEAKER_00 or a wrong name does not change attribution
Tests cover a server echoing a diarization label and a server returning results out of order

**Severity: HIGH.** Found in the August 2026 session lifecycle review (#319). ## The defect The backend knows exactly who owns each track — it sent one file per speaker — and then **discards that knowledge in favour of whatever the remote server echoes back**. - `audio_service.py:535` — `transcribe_session` takes `seg.get("speaker", "Unknown")` with no cross-check - `audio_service.py:460` — on the VAD path, `seg.get("speaker") or display_name` gives the **server's value precedence** over the known owner If the server runs diarization or normalises to a label like `SPEAKER_00`, that silently replaces the real name. `_apply_character_names` (`reminder_tasks.py:2121-2130`) then fails to match, and the character mapping is lost for that whole track. If the server associates results by completion order rather than by the order files were sent, every line of a track is attributed to the wrong player — wholesale, plausible-looking, and undetectable from inside the backend. ## Proposed fix The track owner is authoritative. Make the fallback unconditional: use `display_name` always, and treat any server-supplied speaker label as advisory metadata at most. On the session endpoint, zip results back to tracks by an explicit identifier rather than by position. This is a one-character change on the VAD path and a small one on the session path, and it removes an entire class of silent misattribution. ## Acceptance criteria - [ ] Speaker labels derive from track ownership, never from the server echo - [ ] Results are associated with tracks by explicit id, not by ordering - [ ] A server returning `SPEAKER_00` or a wrong name does not change attribution - [ ] Tests cover a server echoing a diarization label and a server returning results out of order

claude-bot added this to the v4.0.0 - Session Pipeline Accuracy milestone

2026-08-25 20:39:00 +00:00

claude-bot added the

bug

backend

recording

labels

2026-08-25 20:39:00 +00:00