Transcription cuts a speaker's phrases twenty-five at a time, not one ffmpeg process each (#684)

claude-bot commented

2026-10-02 03:22:53 +00:00

Collaborator

Closes #684.

With voice-activity detection on, transcribe_session_vad spawned one ffmpeg process per speech span, in sequence: about 213 for the 30-minute fixture scene, on the order of 1,300 for a three-hour, five-speaker session. The cost was process start-up, not decoding.

What changes

One ffmpeg run cuts up to 25 spans, with one input per span:
ffmpeg -ss a -t d -i track.wav -ss b -t e -i track.wav … -map 0:a:0 … out0 -map 1:a:0 … out1.
Each span keeps its own input seek, decoder and resampler and the same -ss/-t strings, so each output is what _cut_segment writes.
The obvious form, one input with many outputs, was measured and rejected: on ffmpeg 7 it is three times slower than today on a 30-minute track, and its bytes differ on a resampled source.
Batch size 25 is chosen for the worker's memory, not speed: a batch peaks at 63–81 MB against 56 MB for a single cut (50 would be 74–103 MB), under a 1 GB worker limit with a history of memory deaths (#841). A test pins the constant with the reason.
Failure is as before. A batch that fails, cannot be spawned or leaves a file unwritten is cleaned up, logged once, and its spans are cut one at a time by the unchanged _cut_segment, so bad audio raises the same error from the same place. A timed-out batch propagates rather than multiplying the wait. Temp files have the same directory and lifetime; cleanup is tested on every path, cancellation included.
The language probe is left on the single cut. _cut_segment is unchanged.

Proof the output is the same

Generated PCM at 16 kHz mono and 48 kHz stereo, real ffmpeg, whole files compared header included: sample 0, sub-millisecond boundaries, a 1 ms span, overlaps, a span ending on the last sample and one running past the end.
Every one of the fixture scene's 208 spans cut both ways: all identical.
A sampled version of that comparison now runs in the harness's guards step, and fails if any batch fell back.
Every golden transcript, fingerprint and guard passes unchanged. No golden was regenerated; no audio was added.

Measured (local, Docker)

	Before	After
Transcribing the scene with VAD	17.6–20.1 s, 213 processes	4.7 s, 16 processes
`tests/test_synthetic_session_guards.py`	173 s, 1,471 processes	83 s, 371 processes

A test-only memo of silencedetect per unchanged track (keyed on path, inode, size, mtime and thresholds) is part of the second row. The issue's "17 minutes" was stale: the module was already near three.

Checked

Full backend suite: 6800 passed, 14 skipped; ruff 0.4.4 clean. tests/test_vad_batch_cut.py: 28 passed.
The harness, with CI's commands: manifest and scorers 74 passed; assembly, fingerprints and goldens 21 passed; guards 12 passed (84 s); self-check ok; stub eval leg 1.000 on every score.
Backend only: no migration, no bot change, no workflow change. docs/OPERATIONS.md and the changelog fragment say what a fallback looks like in the log.

Left for later

Session assembly in the harness (one Opus decode per clip, 224 processes) is now half the guards module and the largest harness step; the same batching would help, with the same byte-compare discipline, as its own change. xdist is not worth it here (a module-scoped fixture rebuilt per worker; ci.yml already records -n 4 as a regression). The comment above the guards step in ci.yml quotes an old timing.

🤖 Generated with Claude Code

Closes #684. With voice-activity detection on, `transcribe_session_vad` spawned one ffmpeg process per speech span, in sequence: about 213 for the 30-minute fixture scene, on the order of 1,300 for a three-hour, five-speaker session. The cost was process start-up, not decoding. ## What changes - **One ffmpeg run cuts up to 25 spans**, with one input per span: `ffmpeg -ss a -t d -i track.wav -ss b -t e -i track.wav … -map 0:a:0 … out0 -map 1:a:0 … out1`. Each span keeps its own input seek, decoder and resampler and the same `-ss`/`-t` strings, so each output is what `_cut_segment` writes. - The obvious form, one input with many outputs, was measured and rejected: on ffmpeg 7 it is three times **slower** than today on a 30-minute track, and its bytes differ on a resampled source. - **Batch size 25 is chosen for the worker's memory**, not speed: a batch peaks at 63–81 MB against 56 MB for a single cut (50 would be 74–103 MB), under a 1 GB worker limit with a history of memory deaths (#841). A test pins the constant with the reason. - **Failure is as before.** A batch that fails, cannot be spawned or leaves a file unwritten is cleaned up, logged once, and its spans are cut one at a time by the unchanged `_cut_segment`, so bad audio raises the same error from the same place. A timed-out batch propagates rather than multiplying the wait. Temp files have the same directory and lifetime; cleanup is tested on every path, cancellation included. - The language probe is left on the single cut. `_cut_segment` is unchanged. ## Proof the output is the same - Generated PCM at 16 kHz mono and 48 kHz stereo, real ffmpeg, whole files compared header included: sample 0, sub-millisecond boundaries, a 1 ms span, overlaps, a span ending on the last sample and one running past the end. - Every one of the fixture scene's 208 spans cut both ways: all identical. - A sampled version of that comparison now runs in the harness's guards step, and fails if any batch fell back. - Every golden transcript, fingerprint and guard passes unchanged. No golden was regenerated; no audio was added. ## Measured (local, Docker) | | Before | After | |---|---|---| | Transcribing the scene with VAD | 17.6–20.1 s, 213 processes | 4.7 s, 16 processes | | `tests/test_synthetic_session_guards.py` | 173 s, 1,471 processes | 83 s, 371 processes | A test-only memo of `silencedetect` per unchanged track (keyed on path, inode, size, mtime and thresholds) is part of the second row. The issue's "17 minutes" was stale: the module was already near three. ## Checked - Full backend suite: **6800 passed, 14 skipped**; ruff 0.4.4 clean. `tests/test_vad_batch_cut.py`: 28 passed. - The harness, with CI's commands: manifest and scorers 74 passed; assembly, fingerprints and goldens 21 passed; guards 12 passed (84 s); self-check ok; stub eval leg 1.000 on every score. - Backend only: no migration, no bot change, no workflow change. `docs/OPERATIONS.md` and the changelog fragment say what a fallback looks like in the log. ## Left for later Session assembly in the harness (one Opus decode per clip, 224 processes) is now half the guards module and the largest harness step; the same batching would help, with the same byte-compare discipline, as its own change. xdist is not worth it here (a module-scoped fixture rebuilt per worker; `ci.yml` already records `-n 4` as a regression). The comment above the guards step in `ci.yml` quotes an old timing. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 4 commits

2026-10-02 03:22:54 +00:00

Cut a track's speech spans fifty to an ffmpeg process, not one each (#684 ) 3a85d18e60

One input per span in a single command, so each output is byte for byte
what _cut_segment writes; a failed batch falls back to the single cut.
The guards module also stops re-scanning unchanged tracks (test-only).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

The batched cut's fallback, against real ffmpeg (#684 ) a221d960b6

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Merge remote-tracking branch 'origin/main' into perf/684-batch-ffmpeg-cuts f5f12c31b2

Cut spans twenty-five to a process, not fifty: the worker's memory (#684 )

CI / Backend lint (ruff) (pull_request) Successful in 43s

Details

CI / Summarisation accuracy eval harness (stub provider) (pull_request) Successful in 1m7s

Details

CI / Feedback site tests (pull_request) Successful in 48s

Details

CI / Does this change touch the recording path? (pull_request) Successful in 39s

Details

CI / Docker image build (pull_request) Successful in 50s

Details

CI / Bot tests and audit (pull_request) Successful in 2m10s

Details

CI / Bot/backend version sync (pull_request) Successful in 1m17s

Details

CI / Frontend (shard 2/2) (pull_request) Successful in 7m48s

Details

CI / Frontend (shard 1/2) (pull_request) Successful in 9m29s

Details

CI / Synthetic session harness (no GPU, no LLM) (pull_request) Successful in 8m4s

Details

CI / End-to-end journeys (phone) (pull_request) Successful in 16m52s

Details

CI / End-to-end journeys (desktop) (pull_request) Successful in 18m38s

Details

CI / Backend migration, tests, and audit (pull_request) Successful in 26m2s

Details

CI / End-to-end journeys (keyboard) (pull_request) Successful in 17m52s

Details

526e636d1b

Fifty saved about two seconds per 300 spans for 10-20 MB more at ffmpeg's
peak, in a worker limited to 1 GB that has died under memory pressure
before (#841). The measurements are beside the constant, and a test pins it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

claude-bot referenced this pull request

2026-10-02 03:48:47 +00:00

Level up's and Start a character's content pickers empty their list during every search #1030

claude-bot commented

2026-10-02 03:50:08 +00:00

Author

Collaborator

CI on 526e636d: every job is green except CI / End-to-end journeys (keyboard) (run https://git.rhoving.com/rbrooks/Quest-Board/actions/runs/1790).

The log: e2e/j7-a-character.spec.js:440 "Priya levels up", chooseFeat at line 655, getByRole('button', { name: 'Choose Sudden Charge' }) not found after 10 s. The same journey passed on desktop and phone in this run. This PR changes only audio_service.py's span cutting and backend tests, so it cannot have caused it.

Cause and fix are in #1030 (the feat picker empties its list while a search is in flight, and a key pressed on a row in that moment is lost). Not re-pushed: this wants a re-run of that one job, then it merges.

CI on 526e636d: every job is green except **CI / End-to-end journeys (keyboard)** (run https://git.rhoving.com/rbrooks/Quest-Board/actions/runs/1790). The log: `e2e/j7-a-character.spec.js:440` "Priya levels up", `chooseFeat` at line 655, `getByRole('button', { name: 'Choose Sudden Charge' })` not found after 10 s. The same journey passed on desktop and phone in this run. This PR changes only `audio_service.py`'s span cutting and backend tests, so it cannot have caused it. Cause and fix are in #1030 (the feat picker empties its list while a search is in flight, and a key pressed on a row in that moment is lost). Not re-pushed: this wants a re-run of that one job, then it merges.

claude-bot referenced this pull request

2026-10-02 04:16:21 +00:00

Menus stay on the screen, and the content pickers keep their list while they search (#1029, #1030) #1032

claude-bot referenced this pull request from a commit

2026-10-02 04:43:00 +00:00

The content pickers keep their list while the next search is asked for (#1030)

claude-bot referenced this pull request

2026-10-02 04:43:34 +00:00

Level up's and Start a character's content pickers empty their list during every search #1030

claude-bot merged commit 96cf88cf98 into main

2026-10-02 20:58:11 +00:00

claude-bot referenced this pull request from a commit