[Backend] Keep self-hosted ASR first-class — VAD-cut with forced alignment #352

Closed
opened 2026-08-25 20:39:15 +00:00 by claude-bot · 2 comments
Contributor

Found in the August 2026 session lifecycle review (#319). Self-hosted is a first-class target, not a fallback — it is the configuration the project owner will run for the foreseeable future.

Why

The bundled self-hosted path is plain Whisper, which carries exactly the failure modes this review exists to fix: long-form timestamp drift and hallucination on silence. Once the capture fix lands and tracks become mostly silence, that is the documented worst case for Whisper.

The good news is that the fix costs nothing but engineering: WhisperX-style VAD-cut-and-merge with forced phoneme alignment addresses both, and produces the word-level timestamps the provider contract normalises on. A self-hoster should get a genuinely accurate pipeline, not a degraded one.

Proposed fix

Move the bundled path from plain Whisper webservice to faster-whisper or WhisperX with VAD-cut-and-merge plus forced alignment. Emit word-level timestamps so the local adapter satisfies the same contract as a managed one, with no downstream branching.

Verify against the golden corpus (see the eval harness in v4.0.0) so "self-hosted is first-class" is a measured claim rather than an aspiration.

Acceptance criteria

  • Bundled ASR uses VAD-cut-and-merge with forced alignment
  • The local adapter returns word-level timestamps satisfying the shared contract
  • No downstream code branches on local vs managed
  • Accuracy measured against the golden corpus and recorded
  • Upgrade path documented for existing self-hosters, including what changes about their existing transcripts
Found in the August 2026 session lifecycle review (#319). **Self-hosted is a first-class target, not a fallback** — it is the configuration the project owner will run for the foreseeable future. ## Why The bundled self-hosted path is plain Whisper, which carries exactly the failure modes this review exists to fix: long-form timestamp drift and hallucination on silence. Once the capture fix lands and tracks become mostly silence, that is the documented worst case for Whisper. The good news is that the fix costs nothing but engineering: WhisperX-style VAD-cut-and-merge with forced phoneme alignment addresses both, and produces the word-level timestamps the provider contract normalises on. A self-hoster should get a genuinely accurate pipeline, not a degraded one. ## Proposed fix Move the bundled path from plain Whisper webservice to faster-whisper or WhisperX with VAD-cut-and-merge plus forced alignment. Emit word-level timestamps so the local adapter satisfies the same contract as a managed one, with no downstream branching. Verify against the golden corpus (see the eval harness in v4.0.0) so "self-hosted is first-class" is a measured claim rather than an aspiration. ## Acceptance criteria - [ ] Bundled ASR uses VAD-cut-and-merge with forced alignment - [ ] The local adapter returns word-level timestamps satisfying the shared contract - [ ] No downstream code branches on local vs managed - [ ] Accuracy measured against the golden corpus and recorded - [ ] Upgrade path documented for existing self-hosters, including what changes about their existing transcripts
Author
Contributor

Picking this up. Grounding first, because the premise turned out to be half wrong in a useful way.

What the bundled server actually is

It is not "plain Whisper webservice". whisper_config.endpoint_url points at http://10.3.0.28:8091, a ~130-line FastAPI wrapper around WhisperX 3.8.5 (large-v3-turbo, CUDA, faster-whisper 1.2.1) in Rhoving/iac-repo (whisperx-ansible/roles/whisperx/files/api/main.py). It already runs WhisperX's VAD-cut transcription and whisperx.align forced alignment on every request — and then threw the word timings away at the API boundary, returning only {start, end, text, speaker}. It also had no initial_prompt field, so the vocabulary bias from #355 was silently dropped.

So the first two criteria were true server-side and false at the interface, and the fix was an API change, not an engine change.

Server half — done and deployed (Rhoving/iac-repo#396, PR #397)

API v2, backward compatible: segments[].words[] {word, start, end, score} (null where WhisperX could not align a word; words may be empty for a segment), optional initial_prompt/hotwords per request (applied by swapping pipeline.options under a lock, WhisperX's own idiom), one lock around transcribe+align with concurrency_limit: 1 advertised, and /health now carries api_version: 2 and capabilities: {word_timestamps, initial_prompt, hotwords, vad, alignment, concurrency_limit} computed from the live pipeline. Requirements pinned to what was running. Deployed to ai-host 2026-09-05 02:25 UTC (20 ok / 5 changed / 0 failed) after confirming prod had nothing in flight.

Verified live from the dev box: /healthapi_version: 2, all six capabilities true/1; a silent-WAV POST /transcribe with initial_prompt → 200, {"segments": [], "language": "en", "api_version": 2} in 7.4 s including model load.

Client half — next, after #357 lands

LocalWhisperProvider will read /health.capabilities (cached) and declare word_timestamps=True against a v2 server, consume words as real spans (interpolated=False) with interpolation only for null timings, keep the current interpolation for a v1 server, and declare concurrency_limit from the server. Words get persisted on transcript segments (one nullable JSONB column — sequenced after #357's migration to avoid a second Alembic head) and used by anchor_highlights to tighten anchors when present, so the capability is not moot. Upgrade path for self-hosters goes in OPERATIONS. Accuracy against the golden corpus is #433's live leg.

Picking this up. Grounding first, because the premise turned out to be half wrong in a useful way. ## What the bundled server actually is It is not "plain Whisper webservice". `whisper_config.endpoint_url` points at `http://10.3.0.28:8091`, a ~130-line FastAPI wrapper around **WhisperX 3.8.5** (large-v3-turbo, CUDA, faster-whisper 1.2.1) in `Rhoving/iac-repo` (`whisperx-ansible/roles/whisperx/files/api/main.py`). It already runs WhisperX's VAD-cut transcription **and** `whisperx.align` forced alignment on every request — and then threw the word timings away at the API boundary, returning only `{start, end, text, speaker}`. It also had no `initial_prompt` field, so the vocabulary bias from #355 was silently dropped. So the first two criteria were true server-side and false at the interface, and the fix was an API change, not an engine change. ## Server half — done and deployed (Rhoving/iac-repo#396, PR #397) API v2, backward compatible: `segments[].words[] {word, start, end, score}` (`null` where WhisperX could not align a word; `words` may be empty for a segment), optional `initial_prompt`/`hotwords` per request (applied by swapping `pipeline.options` under a lock, WhisperX's own idiom), one lock around transcribe+align with `concurrency_limit: 1` advertised, and `/health` now carries `api_version: 2` and `capabilities: {word_timestamps, initial_prompt, hotwords, vad, alignment, concurrency_limit}` computed from the live pipeline. Requirements pinned to what was running. Deployed to ai-host 2026-09-05 02:25 UTC (20 ok / 5 changed / 0 failed) after confirming prod had nothing in flight. Verified live from the dev box: `/health` → `api_version: 2`, all six capabilities true/1; a silent-WAV `POST /transcribe` with `initial_prompt` → 200, `{"segments": [], "language": "en", "api_version": 2}` in 7.4 s including model load. ## Client half — next, after #357 lands `LocalWhisperProvider` will read `/health.capabilities` (cached) and declare `word_timestamps=True` against a v2 server, consume `words` as real spans (`interpolated=False`) with interpolation only for `null` timings, keep the current interpolation for a v1 server, and declare `concurrency_limit` from the server. Words get persisted on transcript segments (one nullable JSONB column — sequenced after #357's migration to avoid a second Alembic head) and used by `anchor_highlights` to tighten anchors when present, so the capability is not moot. Upgrade path for self-hosters goes in OPERATIONS. Accuracy against the golden corpus is #433's live leg.
Author
Contributor

Client half landed in PR #499 (merged 2026-09-05, CI green; 2087 backend tests). With the server half (Rhoving/iac-repo#396, deployed 02:25 UTC) this closes the issue.

  • Criterion 1 (VAD-cut with forced alignment): the bundled server always did both; API v2 stops discarding the result at the interface.
  • Criterion 2 (word-level timestamps satisfying the contract): LocalWhisperProvider discovers the server's capabilities from /health (cached per endpoint, 10 min) and declares word_timestamps=True against v2; real spans are parsed with interpolated=False, null-timed words placed proportionally between aligned neighbours and flagged, empty words falling back to whole-segment interpolation. Against a v1 or unreachable server it declares no word timestamps and no vocabulary support, so #355 skips the query and the #358 panel says "update the bundled server" rather than pretending.
  • Criterion 3 (no downstream branching): words travel on the segment dicts, offset on the VAD path, persisted as nullable JSONB transcript_segments.words (migration c7d8e9fa0b1c), and a test proves the local v2 adapter and the stub adapter produce identical rows from identical timings. anchor_highlights now anchors to the first measured word of a matched phrase.
  • Criterion 4 (accuracy measured): #433's harness against this server, three identical runs — session WER 3.73 %, 0 foreign markers, onset drift median 0.07 s; marker detection 72 % without vocabulary bias, which is the next measurement queued (#355's criterion).
  • Criterion 5 (upgrade path): OPERATIONS "Upgrading the bundled transcription server" — how to read the API version, that word timings and vocabulary bias apply to new sessions only, and that nothing is reprocessed.

Live: a real submit with a vocabulary returned {"word": "Bramblewick", "start": 0.031, "end": 0.253} — the v2 parser on a real body and the server honouring initial_prompt in one request.

Client half landed in **PR #499** (merged 2026-09-05, CI green; 2087 backend tests). With the server half (Rhoving/iac-repo#396, deployed 02:25 UTC) this closes the issue. - **Criterion 1 (VAD-cut with forced alignment):** the bundled server always did both; API v2 stops discarding the result at the interface. - **Criterion 2 (word-level timestamps satisfying the contract):** `LocalWhisperProvider` discovers the server's capabilities from `/health` (cached per endpoint, 10 min) and declares `word_timestamps=True` against v2; real spans are parsed with `interpolated=False`, `null`-timed words placed proportionally between aligned neighbours and flagged, empty `words` falling back to whole-segment interpolation. Against a v1 or unreachable server it declares no word timestamps **and no vocabulary support**, so #355 skips the query and the #358 panel says "update the bundled server" rather than pretending. - **Criterion 3 (no downstream branching):** words travel on the segment dicts, offset on the VAD path, persisted as nullable JSONB `transcript_segments.words` (migration `c7d8e9fa0b1c`), and a test proves the local v2 adapter and the stub adapter produce identical rows from identical timings. `anchor_highlights` now anchors to the first measured word of a matched phrase. - **Criterion 4 (accuracy measured):** #433's harness against this server, three identical runs — session WER 3.73 %, 0 foreign markers, onset drift median 0.07 s; marker detection 72 % *without* vocabulary bias, which is the next measurement queued (#355's criterion). - **Criterion 5 (upgrade path):** OPERATIONS "Upgrading the bundled transcription server" — how to read the API version, that word timings and vocabulary bias apply to new sessions only, and that nothing is reprocessed. Live: a real `submit` with a vocabulary returned `{"word": "Bramblewick", "start": 0.031, "end": 0.253}` — the v2 parser on a real body and the server honouring `initial_prompt` in one request.
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#352
No description provided.