[Recording] Follow-ups to #82: in-flight downsample, crash-resume salvage, doc cleanup #176
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Context
#82 moved recording capture to stream per-speaker PCM straight to on-disk WAVs (bounded memory). It deliberately left three follow-ups out of scope; this tracks them.
1. In-flight 48 kHz-stereo → 16 kHz-mono downsample (optimization)
Today the sink writes a 48 kHz/2ch raw WAV to disk, then a post-stop FFmpeg step (
utils/audio.py convert_to_whisper_wav) resamples each to 16 kHz mono. Downsampling inwrite()would cut the on-disk footprint ~6× and let us drop the FFmpeg pass entirely.Steps:
PerUserPCMSink, resample each incoming Opus/PCM frame to 16 kHz mono beforewriteframesraw(open the wave writer with the new params)._session_bytes,_write_silence, DTX frame size) against the new sample rate/width so cross-speaker timeline alignment stays exact.ffprobe), and the bot suite stays green.2. Crash-resume / salvage of orphaned recordings (resilience)
Now that audio survives a bot crash on disk, an interrupted session leaves finalizable
*_raw.wavfiles instead of losing everything.Steps:
audio_tempfor orphaned session dirs with*_raw.wavand no completed upload.close(); an orphan may be missing that), run the normal convert +speakers.json+ upload path, or surface them for GM review.3. Doc drift cleanup (trivial)
bot/CLAUDE.mdand thePerUserPCMSinkmodule docstring still describe the old "accumulates each speaker's PCM in memory" behaviour in one spot. Update to the streaming-to-disk description.Notes
Items 1 and 2 are independent and can ship separately; item 3 can ride along with either (or a standalone docs PR).
All three items are done. Closing.
ad6097f)26d4328)3a43e1b)Item 2 was already finished before this issue was picked up
recover_interrupted_recordings(#399) converts and hands off raw tracks left by a mid-session crash, on the next start. Its description here had also decayed: it says to scan for*_raw.wavin "orphaned session dirs", but tracks have been headerless.s16lein the audio_temp root since #321, and a crash during capture leaves no session directory at all — which is precisely what makes their presence a reliable signal.Item 3 was partly already fixed
The
PerUserPCMSinkmodule docstring named here was accurate by the time the work started. BothCLAUDE.mdfiles were not — they still said the sink "accumulates PCM in memory", untrue since #82, and materially misleading: the reason a crashed session is recoverable at all is that the audio was already on disk.Item 1's proposed approach was wrong, and quietly so
This is the part worth reading if you come back to this issue later.
The issue said to resample each frame with
audioop.ratecv.ratecvinterpolates with no anti-aliasing filter, so everything above the output's 8 kHz Nyquist limit folds back into the speech band. Measured on a 12 kHz tone:audioop.ratecvSpeech has real energy above 8 kHz in sibilants and fricatives. The naive version would have laid phantom tones under every voice and fed them to Whisper — with files of the right size and format, and every existing test still green. Nothing anywhere would have reported it. The acceptance criterion in the issue ("verify produced WAVs are byte-format-identical … via ffprobe") would have passed: the container format was never the thing at risk.
utils/resample.pylow-passes before decimating, which needs numpy — a 159-tap FIR at 50 frames/sec/speaker is ~15M MACs/sec, beyond pure Python on the event loop. It costs 78 µs per frame, 0.39% of a core per speaker.What the issue did not anticipate
The raw tracks are headerless, so their sample format lived entirely in the constants item 1 changes — and since #399 the bot recovers leftover tracks automatically at startup. A bot upgraded between a crash and a restart would have read 48 kHz stereo bytes with 16 kHz mono constants and reported six times the real duration, silently, into the backend's duration invariant (#324).
So the tracks are now self-describing: the suffix names the format,
RAW_TRACK_FORMATSmaps it, and duration, the speech threshold and the finalise step all resolve per track. An unrecognised format is refused rather than guessed. Legacy tracks are still read and converted as before.Results
Disk per speaker-hour: 192 kB/s → 32 kB/s. A six-hour session goes from 4.15 GB to 0.69 GB — back under the 4 GB WAV field whose overflow forced these files headerless in #321, though they stay headerless deliberately (a header written at open time has the wrong length in it if the bot dies, and recovery would rather measure bytes than repair a file). The post-stop FFmpeg pass is gone from the normal path; finalising now just writes a WAV header.
One operational note: this format change is effectively one-way. A rollback to a previous image would find
.16k-mono.s16letracks it does not recognise. Recovery refuses an unknown format rather than misreading it — the safe failure — but audio captured by this build would be stranded until someone rolled forward or renamed the files.