rbrooks/Quest-Board

Watch

Fork 0

[Backend] Fail processing when per-speaker track durations do not match the recorded session length #324

New issue

Closed

opened 2026-08-25 20:38:44 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-08-25 20:38:44 +00:00

Collaborator

Severity: HIGH. Found in the August 2026 session lifecycle review (#319).

The defect

Nothing anywhere compares the audio it received against the session it was told about. process_audio already holds the wall-clock duration_seconds as a task argument — supplied by the bot, logged into the Discord embed — and compares it against nothing.

That is why the capture bug ran undetected for months. The corruption was arithmetically obvious from data the backend already had.

It is also worth recording why the other obvious check would not have helped: the transcript's timestamps are perfectly monotonic, but merge_attributed_transcript sorts by start (audio_service.py:846), so output stamps are non-decreasing for any input whatsoever. Monotonicity validates the sorted() call, not the data. The detectable signal was always magnitude.

Evidence

webapp/backend/app/tasks/reminder_tasks.py:1987-1994 — duration_seconds arrives and is unused for validation
webapp/backend/app/tasks/reminder_tasks.py:2046-2060 — track discovery globs *.wav and reads speakers.json with no length check
webapp/backend/app/routers/bot.py:857-864 — the bot supplies the duration at intake
webapp/backend/app/services/audio_service.py:846 — the sort that makes monotonicity tautological

Proposed fix — two guards, both failing loud

Guard 1, pre-transcription (cheapest — it saves the wasted GPU run). After track discovery, for each WAV: if duration_seconds > 600 and wav_duration < 0.9 * duration_seconds, raise, naming the track and both durations. On the 2026-08-11 incident this fires on every track (607-3287 s against 10800 s) before a byte reaches Whisper.

Guard 2, pre-merge (belt and braces). If duration_seconds > 600 and max(segment.end) < 0.6 * duration_seconds, refuse to merge with a message naming the percentage covered.

Both must route into the existing failure path — audio_processing_error set, status failed, bot notified — so the GM gets a real error and a retry button rather than a confidently wrong summary.

The zero-duration caveat

Manual retries pass duration_seconds=0 (routers/sessions.py:937, routers/admin.py:216), which would skip both guards. Skipping at 0 is the minimum fix; the better one is to persist the bot-supplied duration on the session at intake (bot_upload_audio already has it) so retries keep the check.

Acceptance criteria

Pre-transcription guard rejects any track shorter than 90% of the session duration
Pre-merge guard rejects a transcript covering less than 60% of the session duration
Both guards are skipped when the duration is unknown, and the duration is persisted at intake so retries retain it
Both failures set audio_processing_error with a message a GM can act on, and notify the bot
Tests cover: a good session passes, a compressed-track session fails guard 1, a short-transcript session fails guard 2, a zero-duration retry is not blocked

**Severity: HIGH.** Found in the August 2026 session lifecycle review (#319). ## The defect Nothing anywhere compares the audio it received against the session it was told about. `process_audio` **already holds** the wall-clock `duration_seconds` as a task argument — supplied by the bot, logged into the Discord embed — and compares it against nothing. That is why the capture bug ran undetected for months. The corruption was arithmetically obvious from data the backend already had. It is also worth recording why the *other* obvious check would not have helped: the transcript's timestamps are perfectly monotonic, but `merge_attributed_transcript` sorts by start (`audio_service.py:846`), so output stamps are non-decreasing **for any input whatsoever**. Monotonicity validates the `sorted()` call, not the data. The detectable signal was always magnitude. ## Evidence - `webapp/backend/app/tasks/reminder_tasks.py:1987-1994` — `duration_seconds` arrives and is unused for validation - `webapp/backend/app/tasks/reminder_tasks.py:2046-2060` — track discovery globs `*.wav` and reads `speakers.json` with no length check - `webapp/backend/app/routers/bot.py:857-864` — the bot supplies the duration at intake - `webapp/backend/app/services/audio_service.py:846` — the sort that makes monotonicity tautological ## Proposed fix — two guards, both failing loud **Guard 1, pre-transcription** (cheapest — it saves the wasted GPU run). After track discovery, for each WAV: if `duration_seconds > 600` and `wav_duration < 0.9 * duration_seconds`, raise, naming the track and both durations. On the 2026-08-11 incident this fires on **every** track (607-3287 s against 10800 s) before a byte reaches Whisper. **Guard 2, pre-merge** (belt and braces). If `duration_seconds > 600` and `max(segment.end) < 0.6 * duration_seconds`, refuse to merge with a message naming the percentage covered. Both must route into the existing failure path — `audio_processing_error` set, status `failed`, bot notified — so the GM gets a real error and a retry button rather than a confidently wrong summary. ## The zero-duration caveat Manual retries pass `duration_seconds=0` (`routers/sessions.py:937`, `routers/admin.py:216`), which would skip both guards. Skipping at 0 is the minimum fix; the better one is to **persist the bot-supplied duration on the session at intake** (`bot_upload_audio` already has it) so retries keep the check. ## Acceptance criteria - [ ] Pre-transcription guard rejects any track shorter than 90% of the session duration - [ ] Pre-merge guard rejects a transcript covering less than 60% of the session duration - [ ] Both guards are skipped when the duration is unknown, and the duration is persisted at intake so retries retain it - [ ] Both failures set `audio_processing_error` with a message a GM can act on, and notify the bot - [ ] Tests cover: a good session passes, a compressed-track session fails guard 1, a short-transcript session fails guard 2, a zero-duration retry is not blocked

claude-bot added this to the v3.11.5 - Recording Clock Hotfix milestone

2026-08-25 20:38:44 +00:00

claude-bot added the

bug

recording

backend

labels

2026-08-25 20:38:44 +00:00

rbrooks referenced this issue from a commit

2026-08-25 22:24:18 +00:00

fix(backend): enable VAD, guard the duration invariant, stop logging dialogue (#323, #324, #325)

claude-bot referenced this issue

2026-08-25 22:29:56 +00:00

v3.11.5 — per-speaker recordings never shared a session clock (#320) #418

claude-bot referenced this issue from a commit

2026-08-25 23:11:52 +00:00

Merge PR #418: v3.11.5 — per-speaker recordings never shared a session clock (#320)

claude-bot referenced this issue

2026-08-26 03:50:05 +00:00

[Backend] The audio retry endpoint disarms both duration guards #421

claude-bot commented

2026-08-29 07:57:13 +00:00

Author

Collaborator

Shipped — closing as part of a v3.11.5 bookkeeping sweep.

The invariants live in webapp/backend/app/services/audio_service.py under a block headed "── Duration invariants (#324) ──", implemented as check_tracks_cover_session and check_transcript_covers_session with DURATION_GUARD_MIN_SECONDS, TRACK_COVERAGE_MIN and TRANSCRIPT_COVERAGE_MIN.

The capture side is written against them: recording.py:321,567,841 each reference "the backend's duration invariant (#324)" when explaining tail-padding and the single-origin clock.

Two later corrections are worth recording here, since both were found because this guard existed:

#421 — reprocess used to pass duration=0, which silently disabled both coverage checks on every retry. Reprocess now derives the real duration from the tracks.
#431 — check_transcript_covers_session measured against the recording's wall clock, and every track is tail-padded to it, so a session that played for two hours and was left recording for another fifty had a complete transcript rejected. It now measures against the audible extent of the tracks. Fixed and deployed in v4.0.0.

The guard did its job in the end: it is what would have caught #320 on first occurrence.

Shipped — closing as part of a v3.11.5 bookkeeping sweep. The invariants live in `webapp/backend/app/services/audio_service.py` under a block headed "── Duration invariants (#324) ──", implemented as `check_tracks_cover_session` and `check_transcript_covers_session` with `DURATION_GUARD_MIN_SECONDS`, `TRACK_COVERAGE_MIN` and `TRANSCRIPT_COVERAGE_MIN`. The capture side is written against them: `recording.py:321,567,841` each reference "the backend's duration invariant (#324)" when explaining tail-padding and the single-origin clock. Two later corrections are worth recording here, since both were found because this guard existed: - **#421** — reprocess used to pass `duration=0`, which silently disabled *both* coverage checks on every retry. Reprocess now derives the real duration from the tracks. - **#431** — `check_transcript_covers_session` measured against the recording's wall clock, and every track is tail-padded to it, so a session that played for two hours and was left recording for another fifty had a complete transcript rejected. It now measures against the audible extent of the tracks. Fixed and deployed in v4.0.0. The guard did its job in the end: it is what would have caught #320 on first occurrence.

claude-bot closed this issue

2026-08-29 07:57:52 +00:00

rbrooks referenced this issue from a commit

2026-09-01 00:04:39 +00:00

perf(recording): convert to 16 kHz mono during capture (#176 item 1)

claude-bot referenced this issue

2026-09-01 00:05:13 +00:00

perf(recording): convert to 16 kHz mono during capture (#176 item 1) #474

claude-bot referenced this issue

2026-09-01 00:20:38 +00:00

[Recording] Follow-ups to #82: in-flight downsample, crash-resume salvage, doc cleanup #176