rbrooks/Quest-Board

Watch

Fork 0

perf(recording): convert to 16 kHz mono during capture (#176 item 1) #474

Merged

claude-bot merged 1 commit from feat/176-in-capture-downsample into main

2026-09-01 00:20:02 +00:00

claude-bot commented

2026-09-01 00:05:13 +00:00

Contributor

Item 1 of #176. Item 3 landed in #473; item 2 was already done by #399.

What it does

The sink streamed Discord's native 48 kHz stereo to disk and handed the result to FFmpeg after the recording stopped: 192 kB/s per speaker, 4.15 GB for a six-hour session. That size is what pushed the raw tracks past WAV's 4 GB field and forced them headerless in #321. Converting as the audio arrives cuts it 6× — to 0.69 GB — and removes the post-stop pass entirely.

It is not two lines of `audioop`, and that is the whole story

#176 proposed resampling each frame with audioop.ratecv. That is quietly wrong: ratecv interpolates with no anti-aliasing filter, so everything above the output's 8 kHz Nyquist limit folds back into the speech band instead of being removed.

Measured on a 12 kHz tone:

method	output RMS	spurious 4 kHz
`audioop`	8485	6000.0
ffmpeg	39	0.3
this PR	17	0.4

Speech has real energy above 8 kHz in sibilants and fricatives, so the naive version would lay phantom tones under every voice and feed them to Whisper — with the files the right size and format, and every existing test still green. Nothing would have reported it.

utils/resample.py filters properly first: a Hamming-windowed sinc low-pass, odd tap count so the group delay is an exact integer sample rather than a half (a half-sample delay smears the cross-speaker alignment that #320 was about), then takes every third sample. Filter state and decimation phase carry across frames — without that, every 20 ms boundary is a click and the tracks drift apart over hours.

Why numpy

A 159-tap FIR at 50 frames/sec/speaker is ~15M multiply-accumulates per second. Pure Python cannot do that on the event loop. With numpy it is 78 µs per 20 ms frame — 0.39% of one core per speaker, 3.9% at ten.

The dependency is added to requirements.in and the lockfile regenerated with pip-compile, per the pip-compile manager note in renovate.json. The diff to requirements.txt is two lines.

The raw tracks are now self-describing, and they had to be

This is the part that was not in the issue, and it is the reason the work is bigger than "resample in write()".

These files are headerless — nothing in one says what rate it is. The format lived entirely in the constants this PR changes. And since #399, the bot automatically recovers leftover raw tracks at startup. So a bot upgraded between a crash and a restart would have read 48 kHz stereo bytes using 16 kHz mono constants and reported six times the real duration — silently, and straight into the backend's duration invariant (#324).

So:

the suffix names the format (.16k-mono.s16le), and RAW_TRACK_FORMATS maps suffix → rate/channels
every consumer resolves per track, not per constant: duration, the MIN_SPEECH_BYTES threshold (16 000 bytes is half a second of 16 kHz mono but a twelfth of a second of 48 kHz stereo), and the finalise step, which declares the track's own rate to FFmpeg
a track whose format cannot be established is refused, never guessed
legacy 48 kHz tracks are still found, still read, still converted exactly as before

Finalising no longer shells out

The sink's output is already Whisper's format, so _process wraps it in a WAV header rather than re-encoding a file into the format it is already in. FFmpeg stays for the MP3 mixdown and for legacy tracks.

Test changes worth a look

_FRAME_BYTES was doing double duty as "what the fake decoder returns" and "what lands on disk". Those are now different numbers, so they are separate constants and the on-disk one derives from BYTES_PER_SECOND. Four sink tests asserted byte-equality against the pre-resample pattern, which cannot hold once the bytes are filtered — they assert length and not-silence instead.

test_recording_recovery's autouse FFmpeg stand-in became inert when the main path stopped calling FFmpeg: an autouse fixture patching a function nothing calls any more, with assertions inside it that could never fire — silently asserting nothing on every test in the file. It is inverted now, and fails if anything re-encodes a current-format track.

Verification

274 bot tests pass. 13 mutations, all caught:

removing the low-pass (i.e. reproducing the audioop failure mode)
dropping filter state between frames
losing the decimation phase across chunks
skipping the stereo downmix
a reset() that does not clear history
an even tap count
measuring legacy tracks at today's byte rate — the 6× hazard
a non-format-aware speech threshold
dropping the legacy format from the table
finalise ignoring the track's format, or declaring today's rate for all
guessing at an unknown format instead of refusing
grouping that only finds the current format

The bot-prod image builds with numpy, and docs/ plus both CLAUDE.md files are updated to describe the new pipeline.

🤖 Generated with Claude Code

Item 1 of #176. Item 3 landed in #473; item 2 was already done by #399. ## What it does The sink streamed Discord's native 48 kHz stereo to disk and handed the result to FFmpeg after the recording stopped: **192 kB/s per speaker, 4.15 GB for a six-hour session.** That size is what pushed the raw tracks past WAV's 4 GB field and forced them headerless in #321. Converting as the audio arrives cuts it 6× — to **0.69 GB** — and removes the post-stop pass entirely. ## It is not two lines of `audioop`, and that is the whole story #176 proposed resampling each frame with `audioop.ratecv`. That is quietly wrong: `ratecv` interpolates with **no anti-aliasing filter**, so everything above the output's 8 kHz Nyquist limit folds back into the speech band instead of being removed. Measured on a 12 kHz tone: | method | output RMS | spurious 4 kHz | |---|---|---| | `audioop` | 8485 | **6000.0** | | ffmpeg | 39 | 0.3 | | **this PR** | **17** | **0.4** | Speech has real energy above 8 kHz in sibilants and fricatives, so the naive version would lay phantom tones under every voice and feed them to Whisper — with the files the right size and format, and every existing test still green. Nothing would have reported it. `utils/resample.py` filters properly first: a Hamming-windowed sinc low-pass, **odd tap count** so the group delay is an exact integer sample rather than a half (a half-sample delay smears the cross-speaker alignment that #320 was about), then takes every third sample. Filter state and decimation phase carry across frames — without that, every 20 ms boundary is a click and the tracks drift apart over hours. ### Why numpy A 159-tap FIR at 50 frames/sec/speaker is ~15M multiply-accumulates per second. Pure Python cannot do that on the event loop. With numpy it is **78 µs per 20 ms frame — 0.39% of one core per speaker, 3.9% at ten.** The dependency is added to `requirements.in` and the lockfile regenerated with `pip-compile`, per the pip-compile manager note in `renovate.json`. The diff to `requirements.txt` is two lines. ## The raw tracks are now self-describing, and they had to be This is the part that was not in the issue, and it is the reason the work is bigger than "resample in `write()`". These files are **headerless** — nothing in one says what rate it is. The format lived entirely in the constants this PR changes. And since #399, the bot **automatically recovers leftover raw tracks at startup**. So a bot upgraded between a crash and a restart would have read 48 kHz stereo bytes using 16 kHz mono constants and reported **six times the real duration** — silently, and straight into the backend's duration invariant (#324). So: - the suffix names the format (`.16k-mono.s16le`), and `RAW_TRACK_FORMATS` maps suffix → rate/channels - every consumer resolves **per track**, not per constant: duration, the `MIN_SPEECH_BYTES` threshold (16 000 bytes is half a second of 16 kHz mono but a *twelfth* of a second of 48 kHz stereo), and the finalise step, which declares the track's own rate to FFmpeg - a track whose format cannot be established is **refused, never guessed** - legacy 48 kHz tracks are still found, still read, still converted exactly as before ## Finalising no longer shells out The sink's output is already Whisper's format, so `_process` wraps it in a WAV header rather than re-encoding a file into the format it is already in. FFmpeg stays for the MP3 mixdown and for legacy tracks. ## Test changes worth a look `_FRAME_BYTES` was doing double duty as "what the fake decoder returns" and "what lands on disk". Those are now different numbers, so they are separate constants and the on-disk one derives from `BYTES_PER_SECOND`. Four sink tests asserted byte-equality against the *pre-resample* pattern, which cannot hold once the bytes are filtered — they assert length and not-silence instead. `test_recording_recovery`'s autouse FFmpeg stand-in became **inert** when the main path stopped calling FFmpeg: an autouse fixture patching a function nothing calls any more, with assertions inside it that could never fire — silently asserting nothing on every test in the file. It is inverted now, and fails if anything re-encodes a current-format track. ## Verification **274 bot tests pass. 13 mutations, all caught:** - removing the low-pass (i.e. reproducing the `audioop` failure mode) - dropping filter state between frames - losing the decimation phase across chunks - skipping the stereo downmix - a `reset()` that does not clear history - an even tap count - measuring legacy tracks at today's byte rate — the 6× hazard - a non-format-aware speech threshold - dropping the legacy format from the table - finalise ignoring the track's format, or declaring today's rate for all - guessing at an unknown format instead of refusing - grouping that only finds the current format The `bot-prod` image builds with numpy, and `docs/` plus both `CLAUDE.md` files are updated to describe the new pipeline. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit

2026-09-01 00:05:14 +00:00

perf(recording): convert to 16 kHz mono during capture (#176 item 1)

CI / Bot/backend version sync (pull_request) Successful in 40s

Details

CI / Backend lint (ruff) (pull_request) Successful in 52s

Details

CI / Frontend tests, audit, and build (pull_request) Successful in 1m37s

Details

CI / Summarisation accuracy eval harness (stub provider) (pull_request) Successful in 1m56s

Details

CI / Docker image build (pull_request) Successful in 2m11s

Details

CI / Bot tests and audit (pull_request) Successful in 3m35s

Details

CI / Backend migration, tests, and audit (pull_request) Successful in 5m43s

Details

75ee479c31

The sink streamed Discord's native 48 kHz stereo to disk and handed the result
to FFmpeg after the recording stopped: 192 kB/s per speaker, 4.15 GB for a
six-hour session. That size is what pushed the raw tracks past WAV's 4 GB field
and forced them headerless in #321. Converting as the audio arrives cuts it 6x,
to 0.69 GB, and removes the post-stop pass entirely.

Not two lines of audioop, and that is the whole story here
----------------------------------------------------------
#176 proposed resampling each frame with audioop.ratecv. That is quietly wrong:
ratecv interpolates with no anti-aliasing filter, so everything above the
output's 8 kHz Nyquist limit folds back into the speech band instead of being
removed. Measured on a 12 kHz tone:

    method     out RMS   spurious 4 kHz
    audioop       8485           6000.0     <- full-amplitude phantom tone
    ffmpeg          39              0.3
    ours            17              0.4

Speech has real energy above 8 kHz in sibilants and fricatives, so the naive
version would lay phantom tones under every voice and feed them to Whisper,
with the files the right size and format and every existing test still green.

utils/resample.py filters properly first — a Hamming-windowed sinc low-pass,
odd tap count so the group delay is an exact integer sample rather than a half
(a half-sample delay smears the cross-speaker alignment #320 was about) — then
takes every third sample. Filter state and decimation phase carry across
frames, or every 20 ms boundary becomes a click and the tracks drift.

This needs numpy: a 159-tap FIR at 50 frames/sec/speaker is ~15M MACs/sec,
which pure Python cannot do on the event loop. With numpy it is 78 us per
frame — 0.39% of one core per speaker, 3.9% at ten.

The raw tracks are now self-describing, and they had to be
----------------------------------------------------------
These files are headerless, so nothing in one says what rate it is; the format
lived entirely in the constants that just changed. #399 recovers leftover
tracks automatically at startup, so a bot upgraded between a crash and a
restart would have read 48 kHz stereo bytes with 16 kHz mono constants and
reported six times the real duration — silently, and straight into the
backend's duration invariant (#324).

So the suffix names the format (".16k-mono.s16le"), RAW_TRACK_FORMATS maps
suffix to rate, and every consumer resolves per track rather than per constant:
duration, the MIN_SPEECH_BYTES threshold (16 000 bytes is half a second of
16 kHz mono but a twelfth of a second of 48 kHz stereo), and the finalise step,
which declares the track's own rate to FFmpeg. A track whose format cannot be
established is refused, never guessed. Legacy tracks are still read and
converted exactly as before.

Finalising no longer shells out
-------------------------------
The sink's output is already Whisper's format, so _process wraps it in a WAV
header instead of re-encoding a file into the format it is already in.
FFmpeg stays for the MP3 mixdown and for legacy tracks.

Test changes worth noting
-------------------------
_FRAME_BYTES was doing double duty as "what the fake decoder returns" and "what
lands on disk"; those are now different numbers, so they are separate constants
and the on-disk one derives from BYTES_PER_SECOND. Four sink tests asserted
byte-equality against the pre-resample pattern, which cannot hold once the
bytes are filtered — they assert length and not-silence instead.

test_recording_recovery's autouse FFmpeg stand-in became inert when the main
path stopped calling FFmpeg: an autouse fixture patching a function nothing
calls, with assertions that could never fire. It is inverted now — it fails if
anything re-encodes a current-format track.

Verification: 274 bot tests pass. 13 mutations, all caught, including removing
the low-pass (the audioop failure mode), dropping filter state, losing the
decimation phase, an even tap count, measuring legacy tracks at today's rate,
a non-format-aware speech threshold, and guessing at an unknown format. The
bot-prod image builds with numpy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>