[Backend] Fail processing when per-speaker track durations do not match the recorded session length #324

Closed
opened 2026-08-25 20:38:44 +00:00 by claude-bot · 1 comment
Contributor

Severity: HIGH. Found in the August 2026 session lifecycle review (#319).

The defect

Nothing anywhere compares the audio it received against the session it was told about. process_audio already holds the wall-clock duration_seconds as a task argument — supplied by the bot, logged into the Discord embed — and compares it against nothing.

That is why the capture bug ran undetected for months. The corruption was arithmetically obvious from data the backend already had.

It is also worth recording why the other obvious check would not have helped: the transcript's timestamps are perfectly monotonic, but merge_attributed_transcript sorts by start (audio_service.py:846), so output stamps are non-decreasing for any input whatsoever. Monotonicity validates the sorted() call, not the data. The detectable signal was always magnitude.

Evidence

  • webapp/backend/app/tasks/reminder_tasks.py:1987-1994duration_seconds arrives and is unused for validation
  • webapp/backend/app/tasks/reminder_tasks.py:2046-2060 — track discovery globs *.wav and reads speakers.json with no length check
  • webapp/backend/app/routers/bot.py:857-864 — the bot supplies the duration at intake
  • webapp/backend/app/services/audio_service.py:846 — the sort that makes monotonicity tautological

Proposed fix — two guards, both failing loud

Guard 1, pre-transcription (cheapest — it saves the wasted GPU run). After track discovery, for each WAV: if duration_seconds > 600 and wav_duration < 0.9 * duration_seconds, raise, naming the track and both durations. On the 2026-08-11 incident this fires on every track (607-3287 s against 10800 s) before a byte reaches Whisper.

Guard 2, pre-merge (belt and braces). If duration_seconds > 600 and max(segment.end) < 0.6 * duration_seconds, refuse to merge with a message naming the percentage covered.

Both must route into the existing failure path — audio_processing_error set, status failed, bot notified — so the GM gets a real error and a retry button rather than a confidently wrong summary.

The zero-duration caveat

Manual retries pass duration_seconds=0 (routers/sessions.py:937, routers/admin.py:216), which would skip both guards. Skipping at 0 is the minimum fix; the better one is to persist the bot-supplied duration on the session at intake (bot_upload_audio already has it) so retries keep the check.

Acceptance criteria

  • Pre-transcription guard rejects any track shorter than 90% of the session duration
  • Pre-merge guard rejects a transcript covering less than 60% of the session duration
  • Both guards are skipped when the duration is unknown, and the duration is persisted at intake so retries retain it
  • Both failures set audio_processing_error with a message a GM can act on, and notify the bot
  • Tests cover: a good session passes, a compressed-track session fails guard 1, a short-transcript session fails guard 2, a zero-duration retry is not blocked
**Severity: HIGH.** Found in the August 2026 session lifecycle review (#319). ## The defect Nothing anywhere compares the audio it received against the session it was told about. `process_audio` **already holds** the wall-clock `duration_seconds` as a task argument — supplied by the bot, logged into the Discord embed — and compares it against nothing. That is why the capture bug ran undetected for months. The corruption was arithmetically obvious from data the backend already had. It is also worth recording why the *other* obvious check would not have helped: the transcript's timestamps are perfectly monotonic, but `merge_attributed_transcript` sorts by start (`audio_service.py:846`), so output stamps are non-decreasing **for any input whatsoever**. Monotonicity validates the `sorted()` call, not the data. The detectable signal was always magnitude. ## Evidence - `webapp/backend/app/tasks/reminder_tasks.py:1987-1994` — `duration_seconds` arrives and is unused for validation - `webapp/backend/app/tasks/reminder_tasks.py:2046-2060` — track discovery globs `*.wav` and reads `speakers.json` with no length check - `webapp/backend/app/routers/bot.py:857-864` — the bot supplies the duration at intake - `webapp/backend/app/services/audio_service.py:846` — the sort that makes monotonicity tautological ## Proposed fix — two guards, both failing loud **Guard 1, pre-transcription** (cheapest — it saves the wasted GPU run). After track discovery, for each WAV: if `duration_seconds > 600` and `wav_duration < 0.9 * duration_seconds`, raise, naming the track and both durations. On the 2026-08-11 incident this fires on **every** track (607-3287 s against 10800 s) before a byte reaches Whisper. **Guard 2, pre-merge** (belt and braces). If `duration_seconds > 600` and `max(segment.end) < 0.6 * duration_seconds`, refuse to merge with a message naming the percentage covered. Both must route into the existing failure path — `audio_processing_error` set, status `failed`, bot notified — so the GM gets a real error and a retry button rather than a confidently wrong summary. ## The zero-duration caveat Manual retries pass `duration_seconds=0` (`routers/sessions.py:937`, `routers/admin.py:216`), which would skip both guards. Skipping at 0 is the minimum fix; the better one is to **persist the bot-supplied duration on the session at intake** (`bot_upload_audio` already has it) so retries keep the check. ## Acceptance criteria - [ ] Pre-transcription guard rejects any track shorter than 90% of the session duration - [ ] Pre-merge guard rejects a transcript covering less than 60% of the session duration - [ ] Both guards are skipped when the duration is unknown, and the duration is persisted at intake so retries retain it - [ ] Both failures set `audio_processing_error` with a message a GM can act on, and notify the bot - [ ] Tests cover: a good session passes, a compressed-track session fails guard 1, a short-transcript session fails guard 2, a zero-duration retry is not blocked
Author
Contributor

Shipped — closing as part of a v3.11.5 bookkeeping sweep.

The invariants live in webapp/backend/app/services/audio_service.py under a block headed "── Duration invariants (#324) ──", implemented as check_tracks_cover_session and check_transcript_covers_session with DURATION_GUARD_MIN_SECONDS, TRACK_COVERAGE_MIN and TRANSCRIPT_COVERAGE_MIN.

The capture side is written against them: recording.py:321,567,841 each reference "the backend's duration invariant (#324)" when explaining tail-padding and the single-origin clock.

Two later corrections are worth recording here, since both were found because this guard existed:

  • #421 — reprocess used to pass duration=0, which silently disabled both coverage checks on every retry. Reprocess now derives the real duration from the tracks.
  • #431check_transcript_covers_session measured against the recording's wall clock, and every track is tail-padded to it, so a session that played for two hours and was left recording for another fifty had a complete transcript rejected. It now measures against the audible extent of the tracks. Fixed and deployed in v4.0.0.

The guard did its job in the end: it is what would have caught #320 on first occurrence.

Shipped — closing as part of a v3.11.5 bookkeeping sweep. The invariants live in `webapp/backend/app/services/audio_service.py` under a block headed "── Duration invariants (#324) ──", implemented as `check_tracks_cover_session` and `check_transcript_covers_session` with `DURATION_GUARD_MIN_SECONDS`, `TRACK_COVERAGE_MIN` and `TRANSCRIPT_COVERAGE_MIN`. The capture side is written against them: `recording.py:321,567,841` each reference "the backend's duration invariant (#324)" when explaining tail-padding and the single-origin clock. Two later corrections are worth recording here, since both were found because this guard existed: - **#421** — reprocess used to pass `duration=0`, which silently disabled *both* coverage checks on every retry. Reprocess now derives the real duration from the tracks. - **#431** — `check_transcript_covers_session` measured against the recording's wall clock, and every track is tail-padded to it, so a session that played for two hours and was left recording for another fifty had a complete transcript rejected. It now measures against the audible extent of the tracks. Fixed and deployed in v4.0.0. The guard did its job in the end: it is what would have caught #320 on first occurrence.
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#324
No description provided.