rbrooks/Quest-Board

Watch

Fork

You've already forked Quest-Board

Code Issues 58 Pull requests 7 Projects Releases 76 Packages Wiki Activity Actions

[Backend] Drop all-silence tracks before transcription #348

New issue

Closed

opened 2026-08-25 20:39:04 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-08-25 20:39:04 +00:00

Collaborator

Copy link

Severity: MEDIUM. Found in the August 2026 session lifecycle review (#319). New failure mode created by the capture fix.

The defect

Once capture fills gaps with silence, a user who joins voice but never actually speaks — or whose client emits only DTX comfort-noise frames — produces a full-length, entirely silent track. That track is then sent to Whisper, which is well documented to hallucinate repeated phantom text on silence. Because attribution is per-track, those phantom lines are attributed to that user.

That is an independent misattribution vector, arriving as a direct consequence of fixing the first one.

There is a related trap already in the code: _peak_amplitude (recording.py:243-254) reads only the first 0.5 s of a file, which is guaranteed silence for every padded speaker. Any check built on it reports zero for every non-first speaker and would mislead exactly this investigation.

Proposed fix

At close(), scan the whole file (not the first half second) for peak amplitude and total non-silent duration. Drop tracks with no real speech before upload, and record them in the existing uncaptured_member_ids field so the GM sees "present, never captured" rather than a phantom transcript.

Add a backend-side guard too, since the bot cannot be the only line of defence: a track whose speech duration is effectively zero should not be sent for transcription.

Acceptance criteria

_peak_amplitude scans the full file, not the first 0.5 s
All-silence tracks are dropped before upload and reported as uncaptured
The backend independently skips tracks with no detectable speech
The GM can see which participants were present but never captured
Tests cover a silence-only track and a track with one brief utterance

**Severity: MEDIUM.** Found in the August 2026 session lifecycle review (#319). New failure mode created by the capture fix. ## The defect Once capture fills gaps with silence, a user who joins voice but never actually speaks — or whose client emits only DTX comfort-noise frames — produces a **full-length, entirely silent track**. That track is then sent to Whisper, which is well documented to hallucinate repeated phantom text on silence. Because attribution is per-track, those phantom lines are attributed to that user. That is an independent misattribution vector, arriving as a direct consequence of fixing the first one. There is a related trap already in the code: `_peak_amplitude` (`recording.py:243-254`) reads only the first 0.5 s of a file, which is guaranteed silence for every padded speaker. Any check built on it reports zero for every non-first speaker and would mislead exactly this investigation. ## Proposed fix At `close()`, scan the **whole** file (not the first half second) for peak amplitude and total non-silent duration. Drop tracks with no real speech before upload, and record them in the existing `uncaptured_member_ids` field so the GM sees "present, never captured" rather than a phantom transcript. Add a backend-side guard too, since the bot cannot be the only line of defence: a track whose speech duration is effectively zero should not be sent for transcription. ## Acceptance criteria - [ ] `_peak_amplitude` scans the full file, not the first 0.5 s - [ ] All-silence tracks are dropped before upload and reported as uncaptured - [ ] The backend independently skips tracks with no detectable speech - [ ] The GM can see which participants were present but never captured - [ ] Tests cover a silence-only track and a track with one brief utterance

claude-bot added this to the v4.0.0 - Session Pipeline Accuracy milestone

2026-08-25 20:39:04 +00:00

claude-bot added the

bug

backend

recording

labels

2026-08-25 20:39:04 +00:00

rbrooks referenced this issue from a commit

2026-08-26 14:48:12 +00:00

fix(bot,backend): never transcribe a track with no speech in it (#348)

rbrooks referenced this issue from a commit

2026-08-26 15:00:30 +00:00

fix(backend): fail when a speaker we sent comes back with nothing (#343)

claude-bot referenced this issue

2026-08-27 23:25:20 +00:00

[Backend] The silence guard can delete a quiet player from a full-length session #425

claude-bot referenced this issue

2026-08-28 00:22:15 +00:00

[Backend] The silence guard can delete a quiet player from a full-length session #425

rbrooks referenced this issue from a commit

2026-08-28 16:47:37 +00:00

fix(backend): tell the GM when the backend drops a silent speaker (#425)

claude-bot referenced this issue

2026-08-28 16:47:54 +00:00

[Backend] The silence guard can delete a quiet player from a full-length session #425

claude-bot commented

2026-08-28 16:48:10 +00:00

Author

Collaborator

Copy link

Verified against acceptance criteria. Four met, one unmet-as-worded but superseded. The gap that mattered was the fourth, and it was fixed in 540b0a2 under #425.

Criteria

_peak_amplitude scans the full file, not the first 0.5 s — unmet as worded, and deliberately so. It still samples five 64 KB windows (recording.py:374-400).

But the criterion was written when _peak_amplitude was going to be the drop decision. It isn't. The decision now uses speech_bytes_written() (recording.py:887) — a running counter of decoded audio accumulated during capture, with no sampling blind spot at all. _peak_amplitude survives only as a log diagnostic, and its docstring says so and explains why it samples spread through the file rather than at the head.

Making it scan exhaustively now would add cost to a log line. The intent behind the criterion — "do not decide this from a head-only read" — is satisfied by something stronger than what was asked for. Recording it as superseded rather than ticking it.
All-silence tracks are dropped before upload and reported as uncaptured — recording.py:969-978, merged into uncaptured_member_ids and sent via api_client.post_audio_tracks.
The backend independently skips tracks with no detectable speech — drop_silent_tracks, called unconditionally in process_audio. Its detection was rewritten under #425 after the original sampling version turned out to drop quiet players from long sessions.
→ [x] The GM can see which participants were present but never captured — was met for bot-dropped tracks only. The backend's own drops went to a logger.info and never reached uncaptured_member_ids, which was built solely from presence.json. Since that file carries only what the bot declined to upload, a track dropped by the backend guard was invisible to the GM. Fixed in 540b0a2; full reasoning on #425.
Tests cover a silence-only track and a track with one brief utterance — test_silent_tracks.py, and the brief-utterance case is now tested at realistic session length (4,639 s) with speech scattered, not contiguous. I confirmed those fail against the pre-#425 sampler rather than trusting their docstrings.

Worth noting

This issue's own body contains the warning that caught #425: "_peak_amplitude reads only the first 0.5 s of a file... Any check built on it reports zero for every non-first speaker and would mislead exactly this investigation." The backend guard then shipped with a different version of the same defect — sampling that degrades with file length instead of position — and the test written to prove otherwise used a 60-second fixture.

Closing.

**Verified against acceptance criteria.** Four met, one unmet-as-worded but superseded. The gap that mattered was the fourth, and it was fixed in `540b0a2` under #425. ## Criteria - [ ] **`_peak_amplitude` scans the full file, not the first 0.5 s** — **unmet as worded, and deliberately so.** It still samples five 64 KB windows ([recording.py:374-400](bot/questboard_bot/cogs/recording.py#L374-L400)). But the criterion was written when `_peak_amplitude` was going to *be* the drop decision. It isn't. The decision now uses `speech_bytes_written()` ([recording.py:887](bot/questboard_bot/cogs/recording.py#L887)) — a running counter of decoded audio accumulated during capture, with no sampling blind spot at all. `_peak_amplitude` survives only as a log diagnostic, and its docstring says so and explains why it samples *spread through* the file rather than at the head. Making it scan exhaustively now would add cost to a log line. The intent behind the criterion — "do not decide this from a head-only read" — is satisfied by something stronger than what was asked for. Recording it as superseded rather than ticking it. - [x] **All-silence tracks are dropped before upload and reported as uncaptured** — [recording.py:969-978](bot/questboard_bot/cogs/recording.py#L969-L978), merged into `uncaptured_member_ids` and sent via `api_client.post_audio_tracks`. - [x] **The backend independently skips tracks with no detectable speech** — [`drop_silent_tracks`](webapp/backend/app/services/audio_service.py#L1312), called unconditionally in `process_audio`. Its detection was rewritten under #425 after the original sampling version turned out to drop quiet players from long sessions. - [ ] → [x] **The GM can see which participants were present but never captured** — **was met for bot-dropped tracks only.** The backend's own drops went to a `logger.info` and never reached `uncaptured_member_ids`, which was built solely from `presence.json`. Since that file carries only what the *bot* declined to upload, a track dropped by the backend guard was invisible to the GM. Fixed in `540b0a2`; full reasoning on #425. - [x] **Tests cover a silence-only track and a track with one brief utterance** — [test_silent_tracks.py](webapp/backend/tests/test_silent_tracks.py), and the brief-utterance case is now tested at realistic session length (4,639 s) with speech *scattered*, not contiguous. I confirmed those fail against the pre-#425 sampler rather than trusting their docstrings. ## Worth noting This issue's own body contains the warning that caught #425: *"`_peak_amplitude` reads only the first 0.5 s of a file... Any check built on it reports zero for every non-first speaker and would mislead exactly this investigation."* The backend guard then shipped with a different version of the same defect — sampling that degrades with file length instead of position — and the test written to prove otherwise used a 60-second fixture. Closing.

claude-bot closed this issue

2026-08-28 16:48:16 +00:00

claude-bot referenced this issue

2026-08-28 22:16:55 +00:00

[Backend] #425's uncaptured-member fix is overridden — a backend-dropped silent speaker is still proposed as "spoke" #432

claude-bot referenced this issue

2026-09-29 21:07:08 +00:00

Lane 10 PR 8: J2 and J4 walk outside the app; a mail app's Unsubscribe works (#736) #945

No Branch/Tag specified

main

feat/1328-fill-it-in

feat/1328-fill-frontend

feat/1328-fill-backend

feat/9-foundry-push

fix/9-foundry-live

feat/9-foundry-push-frontend

renovate/celery-5.x

feat/5-foundry-connect

feat/9-foundry-push-backend

feat/9-foundry-push-module

feat/5-foundry-frontend

feat/5-foundry-backend

feat/5-foundry-module

release/v4.8.0-rc1

docs/d341-convert-draft

feat/958-update-to-version

feat/958-convert-sheet

feat/958-pathbuilder-import-frontend

fix/1360-heading-order

feat/958-pathbuilder-import-backend

feat/11-weekly-digest

feat/1341-auto-update

docs/d332-superseded

chore/1345-names-cleanup

feat/1342-mentioned-pages

feat/1328-recipes

fix/1346-pgdg-key

perf/1247-shared-recording-stream

chore/132-gitleaks-allowlist

release/v4.7.1

spike/foundry-module

release/v4.7.1-rc1

docs/decisions-d332-d337

fix/bot-certifi

chore/release-4.7.0

chore/release-4.7.0-rc4

chore/deps-4.7.0

chore/release-4.7.0-rc3

fix/1280-quote-must-say-it

fix/1257-names-row-squeeze

chore/release-4.7.0-rc2

fix/1245-pg-upgrade-count

chore/release-4.7.0-rc1

feat/1257-names-in-prep

feat/1257-be

feat/1257-fe

feat/1257-rating-fixes

design/1328-fill-it-in

feat/1257-name-splits

fix/1326-redis-503

feat/1280-fact-suggestions

feat/1280-be

feat/1280-fe

design/1280-fact-suggestions

feat/1279-infobox

feat/1279-be

feat/1279-fe

feat/1321-shelf-copies

design/1279-infobox

feat/1243-table-find

feat/1257-who-rated

feat/1243-prep-drawer

feat/1243-sheet-pickers

design/1243-browse-in-place

feat/1315-hotwords-default

feat/1242-system-filters

feat/1257-more-name-styles

feat/506-leg3-hotwords

design/1242-system-filters

fix/1302-packs-polish

feat/1241-rules-library

design/1241-rules-library

feat/1245-postgres-18

integ/4.6.2

fix/1263-map-ui-2

fix/1263-map-server

fix/1263-map-ui-1

fix/1261-editor-link-menu

fix/1229-axe-4.14-name-mismatch

renovate/axllent-mailpit-1.x

renovate/eslint-monorepo

renovate/authlib-1.x

chore/release-4.6.0

chore/release-4.6.0-rc2

fix/460-rc2-screens

fix/460-rc2-server

fix/release-prerelease-exit

chore/release-4.6.0-rc1

feat/125-campaign-templates

feat/126-calendar

feat/1181-pictures-and-files

feat/1190-event-people

integ/4.6.0

feat/1191-line-of-the-night

feat/1200-starter-packs

chore/bot-multidict-and-playwright-group

fix/1202-this-week-answered-count

fix/122-124-leftovers

fix/1178-get-routes-that-write

fix/1203-journeys-check-names

design/125-campaign-templates

feat/126-calendar-engine

design/126-in-world-calendar

fix/1189-cited-stamp-in-range

fix/1189-v4-looser-check

fix/1189-richer-beats

fix/1113-export-bind-lists

fix/1154-release-notes-by-file

chore/release-4.5.0

chore/release-4.5.0-rc12

fix/1135-wiki-markup-as-text

fix/demo-usage-cost-in-window

chore/release-4.5.0-rc11

fix/1121-sheet-consistency

fix/1121-sheet-5e-terms-and-card-hp

copy/1122-busiest-pages

fix/1121-relevant-choices

fix/1121-wiki-create-npc-save-sheet-links

fix/1029-menu-stays-on-screen

docs/prototype-transcript-bin-wording

fix/1023-editor-save-row-back-scroll

fix/1020-search-exact-first

fix/1020-table-prep

fix/1020-shell-wordmark-footer-bell

fix/1020-wiki-article-party

fix/872-admin-kept-until-phone-tables

feat/l13-milestone-level-waiting

fix/1000-1001-bell-older-setup-context

fix/999-pronouns-in-prompts

fix/l19-other-class-feats

fix/1010-transcript-window-ordinary-confirm

docs/d254-d269-decisions

fix/702-member-search

fix/680-transcript-bin

fix/993-unidentified-prompts

fix/841-redis-timeouts

fix/990-binned-not-exported

fix/830-export-prep-shelf

fix/deps-audit-2026-10-01

fix/877-title-reads

fix/798-audio-received-at

build/lane11-17-j7

build/lane11-16-levelup

build/lane11-14-seams

build/lane11-14-seed

build/lane11-15-start

build/lane11-13-old-block-out

fix/980-review-r3

build/lane11-12-block

build/lane11-8-packs

fix/demo-seed-canonical-names

build/lane11-11-hot-half

docs/l11-decisions

build/lane11-7-convert

build/lane11-10-editing

fix/npm-audit-brace-expansion

build/lane11-9-sheet-page

fix/bot-pyjwt

build/lane11-6-content

fix/lane11-rank-names

build/lane11-2b-language

build/lane11-4-js-evaluator

build/lane11-3-server

build/lane11-5a-pf2e

build/lane11-5b-5e

feat/design-admin

build/lane11-2-evaluator

build/lane11-1-sheet-pieces

build/lane11-0-drawing

fix/943-quest-objectives-fk

fix/prepsheet-shelf-flake

fix/945-leftovers

build/demo-gm-copy

fix/901-email-landings

build/lane10-8-journeys

build/demo-seed-lanes-8-10

fix/887-gm-own-confirmation

build/lane9-5-j1-admin

fix/918-audience-click

fix/useasync-overtaken

ci/939-critical-path

test/937-backend-suite-speed

chore/claude-orchestration

fix/924-xp-gm-only

fix/session-stale-poll

docs/decisions-d239-d242

fix/930-pypdf

build/lane8-journeys

build/925-quest-editor

fix/894-private-page-pictures

build/lane8-p4-drafts

build/lane8-p7-map

build/lane8-p6-handouts-rules

fix/wiki-fixture-names

build/lane10-d227-test-usage

fix/rsvp-horizon-test-tuesday

fix/media-volume-owner

build/lane8-p3-editor

fix/912-unidentified-rename

build/lane8-p5-party-quests

build/lane8-handout-takeback

build/lane10-pickers

build/lane8-p2-article

build/lane8-6-map

build/lane10-7b-email-pages

build/lane8-5-handouts

build/lane10-7a-test-the-bot-pages

build/lane8-p1-pages

build/lane8-4-quests

build/lane8-2b-feeds

build/lane8-3-parties

build/lane8-2a-links

build/lane10-5-commands

build/lane9-6-seed

build/lane10-4b-reactions

build/lane9-4d-pages

build/lane9-4c-pages

build/lane10-4a-posts

build/lane10-3-email

fix/891-web-fonts

build/lane9-4b-pages

build/lane10-6-test-the-bot

build/lane10-2-notify-out

build/lane8-1-wiki-text

build/lane9-4a-frame

design/lane10-drawings

fix/j6-budget

fix/j5-phone-flake

fix/869-test-leaks

fix/843-record-start-followups

build/lane7-all

build/lane6-all

build/lane9-b2-server

chore/changelog-fragments

docs/decisions-d218

build/lane9-a-owner-overview

fix/rule-card-level-label

fix/834-retention-semantics

fix/o3c-record-without-announcement

docs/decisions-d179

docs/decisions-d131-d170

fix/851-commit-before-response

build/lane6-j3-seed

build/lane9-b1-d67-identity

fix/841-blocking-loop

build/lane9-c-backups-audit

fix/vote-race-and-start-commit

fix/recording-announcement

fix/proposal-read-leak

fix/last-admin-guard

build/lane6-p4c-planning

build/lane6-p4b-margin

build/lane6-p4a-sheet

build/lane5-j4-seed

build/lane6-base

build/lane7-pages-talk

build/lane5-pages

build/lane7-pages-safety

build/lane7-p3-safety

build/lane7-p2-notify

build/lane6-p3-planning

build/lane6-p2-history

build/lane6-p1b-items

build/lane7-p1-threads

build/lane5-p6-faces

build/lane6-p1a-sheet

build/lane5-p5-find

build/lane5-p4a-board

build/lane5-p4b-scenes

build/lane5-p3-live

build/lane5-base

build/lane5-p2-cards

docs/lane5-decisions

build/review-r2-wide

fix/review-r1

build/lane5-p1-scenes

build/demo-seed

build/lane4-journeys-j5

build/lane4-page-review-b

build/lane4-page-review-a

build/lane4-page-read

build/lane4-journeys

build/lane4-page-sess2

build/lane4-page-sess1

build/lane4-notes

build/lane4-discord-on-approval

build/lane4-scheduling

build/lane4-versions

build/760-session-number

build/lane4-decisions

build/797-erase-no-regenerate

build/test-race-audit

build/lane3-completions

build/794-dark-only

build/796-titles-scroll

build/791-retention-anchor

build/a11y-sweep

build/shell-followups

build/retention-preview

build/753-backup-readback

docs/d91-d93

build/lane3-j1

build/ci-harness-paths-selftest

fix/useauth-test-race

build/ci-harness-paths

build/777-ai-settings-read

build/lane3-settings-c

build/lane3-settings-b

build/lane3-j6

build/lane3-sessionform

build/lane3-settings-a

build/lane3-members

build/lane3-thisweek

build/lane3-story

chore/87-align-python-3.12

v4.8.0-rc1

v4.7.1

v4.7.1-rc1

v4.7.0

v4.7.0-rc4

v4.7.0-rc3

v4.7.0-rc2

v4.7.0-rc1

v4.6.2

v4.6.2-rc4

v4.6.2-rc3

v4.6.2-rc2

v4.6.2-rc1

v4.6.1

v4.6.1-rc1

v4.6.0

v4.6.0-rc2

v4.6.0-rc1

v4.5.3

v4.5.3-rc1

v4.5.2

v4.5.2-rc1

v4.5.1

v4.5.1-rc2

v4.5.1-rc1

v4.5.0

v4.5.0-rc12

v4.5.0-rc11

v4.5.0-rc10

v4.5.0-rc9

v4.5.0-rc8

v4.5.0-rc7

v4.5.0-rc6

v4.5.0-rc5

v4.5.0-rc4

v4.5.0-rc3

v4.5.0-rc2

v4.5.0-rc1

v4.4.1

v4.4.0

v4.3.0

v4.2.3

v4.2.2

v4.2.1

v4.2.0

v4.1.2

v4.1.1

v4.1.0

v4.0.1

v4.0.0

v3.11.4

v3.11.3

v3.11.2

v3.11.1

v3.11.0

v3.10.0

v3.9.1

v3.9.0

v3.8.0

v3.7.0

v3.6.1

v3.6.0

v3.5.0

v3.4.1

v3.4.0

v3.3.0

v3.2.0

v3.1.0

v3.0.0

v2.1.0

v2.0.0

v1.1.0

v1.0.0

v0.12.0

v0.11.0

v0.10.0

v0.9.1

v0.3.3

v0.9.0

v0.8.0

v0.3.2

v0.3.1

v0.3.0

v0.7.0

v0.6.0

v0.5.0

v0.4.0

v0.2.1

v0.2.0

v0.1.1

v0.1.0

Labels

Clear labels

backend

Issue related to the backend

bot

Issue related to the bot

bug

Something is not working

duplicate

This issue or pull request already exists

enhancement

New feature

frontend

Issue related to the frontend

More information is needed

recording

Issue related to the recording system

security

Security hardening / vulnerability work

No labels

Milestone

Clear milestone

No items

No milestone

v4.0.0 - Session Pipeline Accuracy

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

rbrooks/Quest-Board#348

Reference in a new issue

Repository

rbrooks/Quest-Board

Title

Body

No description provided.

Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?

Rows
Columns