[Ops] Stream recording audio to disk instead of buffering whole sessions in RAM #82

New issue

Closed

opened 2026-07-14 19:46:02 +00:00 by claude-bot · 0 comments

claude-bot commented

2026-07-14 19:46:02 +00:00

Contributor

Context

PerUserPCMSink (bot/questboard_bot/cogs/recording.py:54) accumulates decoded 48kHz stereo PCM per user in io.BytesIO buffers (recording.py:75, allocated in write() at recording.py:103). Silence is padded in so each speaker's buffer grows toward full session length regardless of how much they talk (recording.py:104-113). Buffers are only flushed at stop, where buf.getvalue() (recording.py:555) copies the entire buffer before writing the WAV (recording.py:556), doubling peak memory.

Current behavior

48kHz × 2ch × 2 bytes ≈ 675 MB/hour/speaker. A 5-person, 6-hour session ≈ 20 GB resident — the bot OOMs long before the max_recording_hours: int = 6 cap (bot/questboard_bot/config.py:36) ever fires. A crash mid-recording also loses all audio, since nothing touches disk until stop.

Fix / Spec

In write(), stream each user's PCM incrementally to an on-disk WAV instead of BytesIO:
- On first packet from a user, open a wave.Wave_write for them in the session directory and write silence frames to align with session start (reusing the existing padding logic at recording.py:104-113).
- Each subsequent packet: write silence frames for any gap (existing timeline logic), then writeframes the packet data. DTX silence packets keep writing their one 20ms silence frame.
- At stop: close the wave writers. No getvalue(), no full-buffer copy anywhere.
Remove the in-memory _buffers dict / audio_data accessor (recording.py:172-173) or repoint them at the on-disk files, updating all consumers in the stop/upload path.
Optional but recommended: downsample to 16kHz mono in-flight (Whisper gains nothing from 48kHz stereo; ~6x size reduction). If done, all silence/gap arithmetic must use the new sample rate so timestamps stay consistent.
Preserve the existing upload contract exactly: per-user WAV files in the session directory, same naming, accepted unchanged by the backend pipeline.
Note for a follow-up issue (do not implement here): since audio now survives a bot crash, a resume/salvage-on-restart flow becomes possible.

Acceptance criteria

Memory stays bounded during recording — document a measured before/after (e.g. RSS during a multi-minute, multi-speaker test recording) in the PR.
Produced WAVs are byte-format-compatible with the existing backend pipeline (transcription of a test recording succeeds end to end).
The stop/finalize path no longer performs a full-buffer copy (buf.getvalue() on session audio is gone).
Timeline alignment across speakers is unchanged (silence gaps still positioned correctly).

References

bot/questboard_bot/cogs/recording.py:54-173 (sink, buffers, write path)
bot/questboard_bot/cogs/recording.py:555-556 (stop-path full copy)
bot/questboard_bot/config.py:36 (max_recording_hours)

Filed from the July 2026 full-project review.

## Context `PerUserPCMSink` (`bot/questboard_bot/cogs/recording.py:54`) accumulates decoded 48kHz stereo PCM per user in `io.BytesIO` buffers (`recording.py:75`, allocated in `write()` at `recording.py:103`). Silence is padded in so each speaker's buffer grows toward full session length regardless of how much they talk (`recording.py:104-113`). Buffers are only flushed at stop, where `buf.getvalue()` (`recording.py:555`) copies the entire buffer before writing the WAV (`recording.py:556`), doubling peak memory. ## Current behavior 48kHz × 2ch × 2 bytes ≈ 675 MB/hour/speaker. A 5-person, 6-hour session ≈ 20 GB resident — the bot OOMs long before the `max_recording_hours: int = 6` cap (`bot/questboard_bot/config.py:36`) ever fires. A crash mid-recording also loses all audio, since nothing touches disk until stop. ## Fix / Spec 1. In `write()`, stream each user's PCM incrementally to an on-disk WAV instead of `BytesIO`: - On first packet from a user, open a `wave.Wave_write` for them in the session directory and write silence frames to align with session start (reusing the existing padding logic at `recording.py:104-113`). - Each subsequent packet: write silence frames for any gap (existing timeline logic), then `writeframes` the packet data. DTX silence packets keep writing their one 20ms silence frame. - At stop: close the wave writers. No `getvalue()`, no full-buffer copy anywhere. 2. Remove the in-memory `_buffers` dict / `audio_data` accessor (`recording.py:172-173`) or repoint them at the on-disk files, updating all consumers in the stop/upload path. 3. **Optional but recommended**: downsample to 16kHz mono in-flight (Whisper gains nothing from 48kHz stereo; ~6x size reduction). If done, all silence/gap arithmetic must use the new sample rate so timestamps stay consistent. 4. Preserve the existing upload contract exactly: per-user WAV files in the session directory, same naming, accepted unchanged by the backend pipeline. 5. Note for a follow-up issue (do not implement here): since audio now survives a bot crash, a resume/salvage-on-restart flow becomes possible. ## Acceptance criteria - [ ] Memory stays bounded during recording — document a measured before/after (e.g. RSS during a multi-minute, multi-speaker test recording) in the PR. - [ ] Produced WAVs are byte-format-compatible with the existing backend pipeline (transcription of a test recording succeeds end to end). - [ ] The stop/finalize path no longer performs a full-buffer copy (`buf.getvalue()` on session audio is gone). - [ ] Timeline alignment across speakers is unchanged (silence gaps still positioned correctly). ## References - `bot/questboard_bot/cogs/recording.py:54-173` (sink, buffers, write path) - `bot/questboard_bot/cogs/recording.py:555-556` (stop-path full copy) - `bot/questboard_bot/config.py:36` (`max_recording_hours`) _Filed from the July 2026 full-project review._

claude-bot added the

bot

enhancement

recording

labels

2026-07-14 19:46:14 +00:00

claude-bot added this to the v3.4.0 - Reliability, Operations & Self-Hosting milestone

2026-07-14 19:46:15 +00:00