rbrooks/Quest-Board

Watch

Fork 0

[Backend] Enable VAD by default and drop the noise floor to -45 dB #323

New issue

Closed

opened 2026-08-25 20:38:43 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-08-25 20:38:43 +00:00

Collaborator

Severity: HIGH. Found in the August 2026 session lifecycle review (#319). This is a required companion to the wall-clock capture fix — shipping that fix without this is a regression.

Why this is coupled to the capture fix

VAD is currently off in production: there is no vad_config row in app_settings, so VAD_DEFAULT_ENABLED = False applies (webapp/backend/app/services/settings_service.py:136).

That is survivable today only because the capture bug deletes all silence before the audio ever reaches Whisper. Once capture is corrected, every track runs the full wall-clock length and three things change at once:

ASR input grows roughly ninefold — from ~2 track-hours for a 3.5 h/5-speaker session to ~17.5. On a hosted tier that is a direct cost multiplier; on the GPU box it is queue time.
Whisper's hallucination-on-silence failure re-arms. Per-speaker tracks become mostly silence, which is the documented worst case — phantom repeated text emitted with timestamps inside the silent regions and attributed to that track's speaker. That is an independent misattribution vector, arriving exactly as we fix the first one.
Long-form timestamp drift becomes relevant, since tracks go from 10-55 minutes to multiple hours with no client-side chunking or re-anchoring on the default path.

The VAD pre-pass addresses all three: it cuts each track into speech spans, transcribes each span, and re-anchors timestamps by adding the span's own start offset. Its arithmetic is the most rigorous code in the transcription layer — spans, offsets, trailing-silence and fully-silent cases are all correct and covered by tests/test_vad_trim.py.

The noise floor is wrong for this input

VAD_DEFAULT_NOISE_FLOOR_DB = -30 (settings_service.py:138) is aggressive. silencedetect marks anything below -30 dBFS for at least min_silence_ms (default 2000) as silence, so a soft-spoken player, a whispered in-character aside, or an un-normalised quiet mic can be cut and never transcribed — silently, since only aggregate kept_seconds is logged.

Discord per-user tracks are digitally silent (-inf dB) when that user is not transmitting, so a much lower floor loses nothing real. Set the default to -45 dB.

Note that cutting quiet speech would be a new way to lose data, so this is the one change in the hotfix that should be validated against a real recording before it goes to production.

Acceptance criteria

VAD_DEFAULT_ENABLED = True
VAD_DEFAULT_NOISE_FLOOR_DB = -45
Existing installs with no vad_config row pick up the new defaults; an explicit row still wins
Per-span trim decisions are logged (count, total trimmed, per-track kept ratio) so a track that trims suspiciously hard is visible
Validated against at least one real recorded session: compare the transcript with VAD at -30 and at -45 and confirm no speech is lost at -45
Admin → Bot Settings exposes both values with an explanation of the trade-off

**Severity: HIGH.** Found in the August 2026 session lifecycle review (#319). This is a **required companion** to the wall-clock capture fix — shipping that fix without this is a regression. ## Why this is coupled to the capture fix VAD is currently **off in production**: there is no `vad_config` row in `app_settings`, so `VAD_DEFAULT_ENABLED = False` applies (`webapp/backend/app/services/settings_service.py:136`). That is survivable today only because the capture bug deletes all silence before the audio ever reaches Whisper. Once capture is corrected, every track runs the full wall-clock length and three things change at once: 1. **ASR input grows roughly ninefold** — from ~2 track-hours for a 3.5 h/5-speaker session to ~17.5. On a hosted tier that is a direct cost multiplier; on the GPU box it is queue time. 2. **Whisper's hallucination-on-silence failure re-arms.** Per-speaker tracks become mostly silence, which is the documented worst case — phantom repeated text emitted with timestamps inside the silent regions and attributed to that track's speaker. That is an independent misattribution vector, arriving exactly as we fix the first one. 3. **Long-form timestamp drift becomes relevant**, since tracks go from 10-55 minutes to multiple hours with no client-side chunking or re-anchoring on the default path. The VAD pre-pass addresses all three: it cuts each track into speech spans, transcribes each span, and re-anchors timestamps by adding the span's own start offset. Its arithmetic is the most rigorous code in the transcription layer — spans, offsets, trailing-silence and fully-silent cases are all correct and covered by `tests/test_vad_trim.py`. ## The noise floor is wrong for this input `VAD_DEFAULT_NOISE_FLOOR_DB = -30` (`settings_service.py:138`) is aggressive. `silencedetect` marks anything below -30 dBFS for at least `min_silence_ms` (default 2000) as silence, so a soft-spoken player, a whispered in-character aside, or an un-normalised quiet mic can be **cut and never transcribed** — silently, since only aggregate `kept_seconds` is logged. Discord per-user tracks are *digitally* silent (-inf dB) when that user is not transmitting, so a much lower floor loses nothing real. **Set the default to -45 dB.** Note that cutting quiet speech would be a *new* way to lose data, so this is the one change in the hotfix that should be validated against a real recording before it goes to production. ## Acceptance criteria - [ ] `VAD_DEFAULT_ENABLED = True` - [ ] `VAD_DEFAULT_NOISE_FLOOR_DB = -45` - [ ] Existing installs with no `vad_config` row pick up the new defaults; an explicit row still wins - [ ] Per-span trim decisions are logged (count, total trimmed, per-track kept ratio) so a track that trims suspiciously hard is visible - [ ] Validated against at least one real recorded session: compare the transcript with VAD at -30 and at -45 and confirm no speech is lost at -45 - [ ] Admin → Bot Settings exposes both values with an explanation of the trade-off

claude-bot added this to the v3.11.5 - Recording Clock Hotfix milestone

2026-08-25 20:38:43 +00:00

claude-bot added the

bug

recording

backend

labels

2026-08-25 20:38:43 +00:00

claude-bot referenced this issue

2026-08-25 20:45:22 +00:00

[Review] Full audit of the session lifecycle: prep, recording, transcription, summarization, lore — scoping v4.0.0 #319

rbrooks referenced this issue from a commit

2026-08-25 22:24:18 +00:00

fix(backend): enable VAD, guard the duration invariant, stop logging dialogue (#323, #324, #325)

claude-bot referenced this issue

2026-08-25 22:29:56 +00:00

v3.11.5 — per-speaker recordings never shared a session clock (#320) #418

claude-bot referenced this issue from a commit

2026-08-25 23:11:52 +00:00

Merge PR #418: v3.11.5 — per-speaker recordings never shared a session clock (#320)

rbrooks referenced this issue from a commit

2026-08-28 17:01:54 +00:00

fix(backend): close two attribution gaps found by verifying #342 and #344

rbrooks referenced this issue from a commit

2026-08-28 19:03:44 +00:00

fix(backend): remove the unverifiable attribution join entirely (#342)

claude-bot referenced this issue

2026-08-28 19:14:01 +00:00

[Backend] Stop the ASR server's speaker label overriding the known track owner #342

claude-bot commented

2026-08-29 07:57:06 +00:00

Author

Collaborator

Shipped — closing as part of a v3.11.5 bookkeeping sweep.

Both halves are in webapp/backend/app/services/settings_service.py:

VAD_DEFAULT_ENABLED = True          # "Enabled by default since #323" (:193)
VAD_DEFAULT_NOISE_FLOOR_DB = -45

get_vad_config's docstring carries the reasoning — with per-speaker tracks correctly spanning the whole session, each one is mostly silence, which is Whisper's documented worst case for hallucinating phantom speech.

Worth noting it has been load-bearing since: transcribe_session's #342 rework leaned on VAD already being the default, since the VAD path had always sent one file per request and so was already free of the echo-based attribution join. audio_service.py:1012 also warns when the floor looks like it is clipping quiet speech, which is the follow-through this issue's floor change needed.

Shipped — closing as part of a v3.11.5 bookkeeping sweep. Both halves are in `webapp/backend/app/services/settings_service.py`: ```python VAD_DEFAULT_ENABLED = True # "Enabled by default since #323" (:193) VAD_DEFAULT_NOISE_FLOOR_DB = -45 ``` `get_vad_config`'s docstring carries the reasoning — with per-speaker tracks correctly spanning the whole session, each one is mostly silence, which is Whisper's documented worst case for hallucinating phantom speech. Worth noting it has been load-bearing since: `transcribe_session`'s #342 rework leaned on VAD already being the default, since the VAD path had always sent one file per request and so was already free of the echo-based attribution join. `audio_service.py:1012` also warns when the floor looks like it is clipping quiet speech, which is the follow-through this issue's floor change needed.

claude-bot closed this issue

2026-08-29 07:57:50 +00:00