[Recording] Follow-ups to #82: in-flight downsample, crash-resume salvage, doc cleanup

claude-bot commented

2026-07-16 02:14:49 +00:00

Collaborator

Context

#82 moved recording capture to stream per-speaker PCM straight to on-disk WAVs (bounded memory). It deliberately left three follow-ups out of scope; this tracks them.

1. In-flight 48 kHz-stereo → 16 kHz-mono downsample (optimization)

Today the sink writes a 48 kHz/2ch raw WAV to disk, then a post-stop FFmpeg step (utils/audio.py convert_to_whisper_wav) resamples each to 16 kHz mono. Downsampling in write() would cut the on-disk footprint ~6× and let us drop the FFmpeg pass entirely.

Steps:

In PerUserPCMSink, resample each incoming Opus/PCM frame to 16 kHz mono before writeframesraw (open the wave writer with the new params).
Critical: recompute ALL silence/gap arithmetic (_session_bytes, _write_silence, DTX frame size) against the new sample rate/width so cross-speaker timeline alignment stays exact.
Remove/short-circuit the post-stop FFmpeg resample once the sink already emits 16 kHz mono; keep the same output path/naming so the backend pipeline is unchanged.
Verify produced WAVs are byte-format-identical to today's post-FFmpeg output (ffprobe), and the bot suite stays green.

2. Crash-resume / salvage of orphaned recordings (resilience)

Now that audio survives a bot crash on disk, an interrupted session leaves finalizable *_raw.wav files instead of losing everything.

Steps:

On bot startup, scan audio_temp for orphaned session dirs with *_raw.wav and no completed upload.
Finalize the WAV headers (the streaming writer patches sizes on close(); an orphan may be missing that), run the normal convert + speakers.json + upload path, or surface them for GM review.
Guard against racing an in-progress recording; add a retention bound so genuinely-abandoned scratch is still cleaned up.
Test with a simulated mid-session kill + restart.

3. Doc drift cleanup (trivial)

bot/CLAUDE.md and the PerUserPCMSink module docstring still describe the old "accumulates each speaker's PCM in memory" behaviour in one spot. Update to the streaming-to-disk description.

Notes

Items 1 and 2 are independent and can ship separately; item 3 can ride along with either (or a standalone docs PR).

## Context #82 moved recording capture to stream per-speaker PCM straight to on-disk WAVs (bounded memory). It deliberately left three follow-ups out of scope; this tracks them. ### 1. In-flight 48 kHz-stereo → 16 kHz-mono downsample (optimization) Today the sink writes a 48 kHz/2ch raw WAV to disk, then a post-stop **FFmpeg** step (`utils/audio.py convert_to_whisper_wav`) resamples each to 16 kHz mono. Downsampling *in `write()`* would cut the on-disk footprint ~6× and let us drop the FFmpeg pass entirely. Steps: - In `PerUserPCMSink`, resample each incoming Opus/PCM frame to 16 kHz mono before `writeframesraw` (open the wave writer with the new params). - **Critical:** recompute ALL silence/gap arithmetic (`_session_bytes`, `_write_silence`, DTX frame size) against the new sample rate/width so cross-speaker timeline alignment stays exact. - Remove/short-circuit the post-stop FFmpeg resample once the sink already emits 16 kHz mono; keep the same output path/naming so the backend pipeline is unchanged. - Verify produced WAVs are byte-format-identical to today's post-FFmpeg output (`ffprobe`), and the bot suite stays green. ### 2. Crash-resume / salvage of orphaned recordings (resilience) Now that audio survives a bot crash on disk, an interrupted session leaves finalizable `*_raw.wav` files instead of losing everything. Steps: - On bot startup, scan `audio_temp` for orphaned session dirs with `*_raw.wav` and no completed upload. - Finalize the WAV headers (the streaming writer patches sizes on `close()`; an orphan may be missing that), run the normal convert + `speakers.json` + upload path, or surface them for GM review. - Guard against racing an in-progress recording; add a retention bound so genuinely-abandoned scratch is still cleaned up. - Test with a simulated mid-session kill + restart. ### 3. Doc drift cleanup (trivial) `bot/CLAUDE.md` and the `PerUserPCMSink` module docstring still describe the old "accumulates each speaker's PCM in memory" behaviour in one spot. Update to the streaming-to-disk description. ## Notes Items 1 and 2 are independent and can ship separately; item 3 can ride along with either (or a standalone docs PR).

claude-bot added the

bot

enhancement

recording

labels

2026-07-16 02:15:23 +00:00

claude-bot referenced this issue

2026-08-25 19:32:48 +00:00

[Review] Full audit of the session lifecycle: prep, recording, transcription, summarization, lore — scoping v4.0.0 #319

claude-bot added this to the v4.1.0 - Data Durability & Recovery milestone

2026-08-25 20:39:57 +00:00

claude-bot referenced this issue

2026-08-25 20:45:22 +00:00

[Review] Full audit of the session lifecycle: prep, recording, transcription, summarization, lore — scoping v4.0.0 #319

rbrooks referenced this issue from a commit

2026-08-31 23:15:58 +00:00

docs: recording pipeline no longer buffers in memory (#176 item 3)

claude-bot referenced this issue

2026-08-31 23:16:12 +00:00

docs: recording pipeline no longer buffers in memory (#176 item 3) #473

claude-bot referenced this issue from a commit

2026-09-01 00:00:29 +00:00

Merge PR #473: recording pipeline no longer buffers in memory (#176 item 3)

rbrooks referenced this issue from a commit

2026-09-01 00:04:39 +00:00

perf(recording): convert to 16 kHz mono during capture (#176 item 1)

claude-bot referenced this issue

2026-09-01 00:05:13 +00:00

perf(recording): convert to 16 kHz mono during capture (#176 item 1) #474

claude-bot referenced this issue from a commit

2026-09-01 00:20:04 +00:00

Merge PR #474: convert to 16 kHz mono during capture (#176)

claude-bot commented

2026-09-01 00:20:37 +00:00

Author

Collaborator

All three items are done. Closing.

Item	Where
1. In-flight 48 kHz stereo → 16 kHz mono downsample	#474 (`ad6097f`)
2. Crash-resume / salvage of orphaned recordings	Already done by #399 (`26d4328`)
3. Doc drift cleanup	#473 (`3a43e1b`)

Item 2 was already finished before this issue was picked up

recover_interrupted_recordings (#399) converts and hands off raw tracks left by a mid-session crash, on the next start. Its description here had also decayed: it says to scan for *_raw.wav in "orphaned session dirs", but tracks have been headerless .s16le in the audio_temp root since #321, and a crash during capture leaves no session directory at all — which is precisely what makes their presence a reliable signal.

Item 3 was partly already fixed

The PerUserPCMSink module docstring named here was accurate by the time the work started. Both CLAUDE.md files were not — they still said the sink "accumulates PCM in memory", untrue since #82, and materially misleading: the reason a crashed session is recoverable at all is that the audio was already on disk.

Item 1's proposed approach was wrong, and quietly so

This is the part worth reading if you come back to this issue later.

The issue said to resample each frame with audioop.ratecv. ratecv interpolates with no anti-aliasing filter, so everything above the output's 8 kHz Nyquist limit folds back into the speech band. Measured on a 12 kHz tone:

method	output RMS	spurious 4 kHz
`audioop.ratecv`	8485	6000.0
ffmpeg	39	0.3
what shipped	17	0.4

Speech has real energy above 8 kHz in sibilants and fricatives. The naive version would have laid phantom tones under every voice and fed them to Whisper — with files of the right size and format, and every existing test still green. Nothing anywhere would have reported it. The acceptance criterion in the issue ("verify produced WAVs are byte-format-identical … via ffprobe") would have passed: the container format was never the thing at risk.

utils/resample.py low-passes before decimating, which needs numpy — a 159-tap FIR at 50 frames/sec/speaker is ~15M MACs/sec, beyond pure Python on the event loop. It costs 78 µs per frame, 0.39% of a core per speaker.

What the issue did not anticipate

The raw tracks are headerless, so their sample format lived entirely in the constants item 1 changes — and since #399 the bot recovers leftover tracks automatically at startup. A bot upgraded between a crash and a restart would have read 48 kHz stereo bytes with 16 kHz mono constants and reported six times the real duration, silently, into the backend's duration invariant (#324).

So the tracks are now self-describing: the suffix names the format, RAW_TRACK_FORMATS maps it, and duration, the speech threshold and the finalise step all resolve per track. An unrecognised format is refused rather than guessed. Legacy tracks are still read and converted as before.

Results

Disk per speaker-hour: 192 kB/s → 32 kB/s. A six-hour session goes from 4.15 GB to 0.69 GB — back under the 4 GB WAV field whose overflow forced these files headerless in #321, though they stay headerless deliberately (a header written at open time has the wrong length in it if the bot dies, and recovery would rather measure bytes than repair a file). The post-stop FFmpeg pass is gone from the normal path; finalising now just writes a WAV header.

One operational note: this format change is effectively one-way. A rollback to a previous image would find .16k-mono.s16le tracks it does not recognise. Recovery refuses an unknown format rather than misreading it — the safe failure — but audio captured by this build would be stranded until someone rolled forward or renamed the files.

All three items are done. Closing. | Item | Where | |---|---| | 1. In-flight 48 kHz stereo → 16 kHz mono downsample | #474 (`ad6097f`) | | 2. Crash-resume / salvage of orphaned recordings | Already done by #399 (`26d4328`) | | 3. Doc drift cleanup | #473 (`3a43e1b`) | ## Item 2 was already finished before this issue was picked up `recover_interrupted_recordings` (#399) converts and hands off raw tracks left by a mid-session crash, on the next start. Its description here had also decayed: it says to scan for `*_raw.wav` in "orphaned session dirs", but tracks have been headerless `.s16le` in the audio_temp **root** since #321, and a crash during capture leaves no session directory at all — which is precisely what makes their presence a reliable signal. ## Item 3 was partly already fixed The `PerUserPCMSink` module docstring named here was accurate by the time the work started. Both `CLAUDE.md` files were not — they still said the sink "accumulates PCM in memory", untrue since #82, and materially misleading: the reason a crashed session is recoverable at all is that the audio was *already on disk*. ## Item 1's proposed approach was wrong, and quietly so This is the part worth reading if you come back to this issue later. The issue said to resample each frame with `audioop.ratecv`. `ratecv` interpolates with **no anti-aliasing filter**, so everything above the output's 8 kHz Nyquist limit folds back into the speech band. Measured on a 12 kHz tone: | method | output RMS | spurious 4 kHz | |---|---|---| | `audioop.ratecv` | 8485 | **6000.0** | | ffmpeg | 39 | 0.3 | | what shipped | 17 | 0.4 | Speech has real energy above 8 kHz in sibilants and fricatives. The naive version would have laid phantom tones under every voice and fed them to Whisper — with files of the right size and format, and every existing test still green. Nothing anywhere would have reported it. The acceptance criterion in the issue ("verify produced WAVs are byte-format-identical … via ffprobe") would have **passed**: the container format was never the thing at risk. `utils/resample.py` low-passes before decimating, which needs numpy — a 159-tap FIR at 50 frames/sec/speaker is ~15M MACs/sec, beyond pure Python on the event loop. It costs 78 µs per frame, 0.39% of a core per speaker. ## What the issue did not anticipate The raw tracks are **headerless**, so their sample format lived entirely in the constants item 1 changes — and since #399 the bot recovers leftover tracks *automatically at startup*. A bot upgraded between a crash and a restart would have read 48 kHz stereo bytes with 16 kHz mono constants and reported **six times the real duration**, silently, into the backend's duration invariant (#324). So the tracks are now self-describing: the suffix names the format, `RAW_TRACK_FORMATS` maps it, and duration, the speech threshold and the finalise step all resolve per track. An unrecognised format is refused rather than guessed. Legacy tracks are still read and converted as before. ## Results Disk per speaker-hour: **192 kB/s → 32 kB/s**. A six-hour session goes from 4.15 GB to 0.69 GB — back under the 4 GB WAV field whose overflow forced these files headerless in #321, though they stay headerless deliberately (a header written at open time has the wrong length in it if the bot dies, and recovery would rather measure bytes than repair a file). The post-stop FFmpeg pass is gone from the normal path; finalising now just writes a WAV header. **One operational note:** this format change is effectively one-way. A rollback to a previous image would find `.16k-mono.s16le` tracks it does not recognise. Recovery refuses an unknown format rather than misreading it — the safe failure — but audio captured by this build would be stranded until someone rolled forward or renamed the files.

claude-bot closed this issue

2026-09-01 00:20:39 +00:00

claude-bot referenced this issue

2026-09-01 17:03:10 +00:00

[Ops] Live-stack verification of v3.4.0 reliability changes #175

rbrooks referenced this issue from a commit

2026-09-05 03:50:26 +00:00

feat(evals): assembler and seeding — clips to the bot's own directory contract (#433)

rbrooks referenced this issue from a commit

2026-09-05 03:50:27 +00:00

docs: what the synthetic session harness proves, and what it does not (#433)

rbrooks referenced this issue from a commit