feat(ops): reclaim audio directories no session refers to (#406) #466

Merged

claude-bot merged 1 commit from fix/406-reconcile-orphaned-audio into main 2026-08-31 16:19:06 +00:00

Conversation 0 Commits 1 Files changed 4 +389

claude-bot commented 2026-08-31 05:45:27 +00:00

Contributor

Copy link

Closes #406 (MEDIUM). The last of #416's cross-referenced stuck states — that issue can close once this and #409 are both in.

The defect

Neither existing sweep can see these directories. The backend's retention passes iterate database rows, so a directory whose row is gone is invisible by construction. The bot's startup sweep skips anything marked handed-off "regardless of age" — correct for a live session, wrong forever afterwards. A hard-deleted campaign leaves its recordings on the volume permanently, with no operator-visible signal.

What ships

A daily Beat task reconciles the volume against the sessions table, removes directories nothing refers to after a 7-day grace, and writes an audit entry naming what went.

That grace is also what satisfies the fourth criterion. Rather than adding cleanup to the campaign-delete path — a second mechanism to keep correct — a delete is simply reclaimed within a bounded time by the one mechanism that already has to exist.

This task deletes, so most of the design is about refusing to

An empty sessions table deletes nothing. "Remove everything not in the database" removes everything the instant that set comes back empty — a failed query, a half-migrated schema, a database pointed at the wrong host. Directories on disk with no sessions at all is a database problem, not a pile of orphans, and acting on it automatically would destroy every recording on the volume. It logs loudly and stops.

This is the same shape as #399, whose sweep destroyed real recordings while acting with complete confidence on a premise ("the backend never received the upload so there is nothing to preserve") that was exactly backwards. That is the failure this milestone exists to stop repeating, so the guard is a first-class part of the task rather than a defensive afterthought.

Only directories named like a session id are candidates. A lost+found, a mount point, or an operator's scratch copy on the same volume was not created by this pipeline and is not ours to delete.

Recent directories are left alone regardless of whether a row exists. The bot creates the directory before the backend records the path, and during a long recording that window is hours.

One evidence bullet in the issue is now stale

It lists a session stuck in processing as permanently exempt from every sweep. #398's watchdog now moves those to failed within 8 hours, so they reach a sweepable state on their own. No special handling is needed, and adding some would have been dead code.

Verification

Mutation-checked: removing the empty-table guard fails test_an_empty_sessions_table_deletes_nothing, which asserts all three seeded directories survive.

Nine tests, of which four are about not deleting — a reclaimer that passes only its happy path is the dangerous kind. There is also a test that an idle run writes no audit entry, since an entry per quiet day buries the ones that mean something.

1,512 backend tests pass (was 1,503). Lint clean at pinned ruff 0.4.4.

🤖 Generated with Claude Code

Closes #406 (MEDIUM). The last of #416's cross-referenced stuck states — that issue can close once this and #409 are both in. ## The defect Neither existing sweep can see these directories. The backend's retention passes iterate **database rows**, so a directory whose row is gone is invisible by construction. The bot's startup sweep skips anything marked handed-off **"regardless of age"** — correct for a live session, wrong forever afterwards. A hard-deleted campaign leaves its recordings on the volume permanently, with no operator-visible signal. ## What ships A daily Beat task reconciles the volume against the sessions table, removes directories nothing refers to after a **7-day grace**, and writes an audit entry naming what went. That grace is also what satisfies the fourth criterion. Rather than adding cleanup to the campaign-delete path — a second mechanism to keep correct — a delete is simply reclaimed within a bounded time by the one mechanism that already has to exist. ## This task deletes, so most of the design is about refusing to **An empty sessions table deletes nothing.** "Remove everything not in the database" removes *everything* the instant that set comes back empty — a failed query, a half-migrated schema, a database pointed at the wrong host. Directories on disk with no sessions at all is a database problem, not a pile of orphans, and acting on it automatically would destroy every recording on the volume. It logs loudly and stops. This is the same shape as **#399**, whose sweep destroyed real recordings while acting with complete confidence on a premise (*"the backend never received the upload so there is nothing to preserve"*) that was exactly backwards. That is the failure this milestone exists to stop repeating, so the guard is a first-class part of the task rather than a defensive afterthought. **Only directories named like a session id are candidates.** A `lost+found`, a mount point, or an operator's scratch copy on the same volume was not created by this pipeline and is not ours to delete. **Recent directories are left alone regardless of whether a row exists.** The bot creates the directory *before* the backend records the path, and during a long recording that window is hours. ## One evidence bullet in the issue is now stale It lists a session stuck in `processing` as permanently exempt from every sweep. **#398's watchdog** now moves those to `failed` within 8 hours, so they reach a sweepable state on their own. No special handling is needed, and adding some would have been dead code. ## Verification Mutation-checked: removing the empty-table guard fails `test_an_empty_sessions_table_deletes_nothing`, which asserts all three seeded directories survive. Nine tests, of which four are about *not* deleting — a reclaimer that passes only its happy path is the dangerous kind. There is also a test that an idle run writes no audit entry, since an entry per quiet day buries the ones that mean something. **1,512 backend tests pass** (was 1,503). Lint clean at pinned ruff 0.4.4. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 2 commits 2026-08-31 05:45:27 +00:00

feat(ops): size the bot event stream from evidence, not a constant (#415) ... acbf1eada3

BOT_EVENTS_MAXLEN was a hardcoded 10,000, chosen for one self-hosted group
and never revisited for multi-tenant volume where every tenant's events
share the one stream. XADD MAXLEN drops the oldest entries read or not, and
the stream is the durable delivery path — a trimmed event has no catch-up.

Now settings.bot_events_maxlen, read at call time so a deployment can raise
it without a code change.

A configurable number nobody can size is still a guess, so the third
acceptance criterion is the substance here. detect_unread_trim compares the
consumer group's last-delivered-id against the stream's oldest surviving
entry: if the group never reached what is now the first entry, everything
between was trimmed unread. An hourly Beat task publishes that as
bot_events_trimmed_unread_total.

Two details that are load-bearing:

Stream ids are `<ms>-<seq>` and must be compared numerically — "9-0" sorts
after "10-0" as text, which inverts the exact comparison being made and
would report loss on a healthy stream. Mutation-checked.

"Nothing to compare" returns None, not 0, and leaves the gauge untouched. A
zero the detector has not earned is indistinguishable from a genuine all-
clear, and that is the more dangerous of the two answers.

15 new tests; the numeric-comparison one fails against a lexical mutant.
1,503 backend tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

feat(ops): reclaim audio directories no session refers to (#406) ... dd7df59e0f

Neither sweep could see them. The backend's retention passes iterate DB
rows, so a directory whose row is gone is invisible by construction; the
bot's startup sweep skips anything marked handed-off "regardless of age".
A hard-deleted campaign left its recordings on the volume permanently.

A daily Beat task reconciles the volume against the sessions table and
removes directories nothing refers to, after a 7-day grace, with an audit
entry naming what went. That grace is also what makes criterion 4 hold: a
campaign delete is reclaimed within a bounded time, so the delete path
needs no cleanup of its own.

This task deletes, in the milestone about not losing data, so most of the
design is about when it must refuse:

- An empty sessions table deletes nothing. "Remove everything not in the
  database" removes *everything* the instant that set comes back empty — a
  failed query, a half-migrated schema, a database pointed at the wrong
  host. Directories with no sessions at all is a database problem, not a
  pile of orphans. This is the same shape as #399, whose sweep acted with
  complete confidence on a premise that was simply false.
- Only directories named like a session id are candidates. A lost+found or
  an operator's scratch copy on the same volume is not ours to delete.
- Recent directories are left alone regardless. The bot creates the
  directory before the backend records the path, and during a long
  recording that window is hours.

One evidence bullet in the issue is now stale: stuck-processing sessions
were listed as permanently exempt from every sweep, but #398's watchdog
moves them to `failed` within 8h, so they reach a sweepable state on their
own. No special handling needed.

Mutation-checked: removing the empty-table guard fails the test that pins
it. 1,512 backend tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

rbrooks force-pushed fix/406-reconcile-orphaned-audio from dd7df59e0f to 108d730862 2026-08-31 16:12:41 +00:00 Compare

claude-bot merged commit 44742a4b6b into main 2026-08-31 16:19:06 +00:00

claude-bot deleted branch fix/406-reconcile-orphaned-audio 2026-08-31 16:19:06 +00:00

claude-bot referenced this pull request from a commit 2026-08-31 16:19:08 +00:00

Merge PR #466: reclaim audio directories no session refers to (#406)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

backend

Issue related to the backend

  

bot

Issue related to the bot

  

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

frontend

Issue related to the frontend

  

help wanted

Need some help

  

invalid

Something is wrong

  

question

More information is needed

  

recording

Issue related to the recording system

  

security

Security hardening / vulnerability work

  

wontfix

This won't be fixed

No labels

backend

bot

bug

duplicate

enhancement

frontend

help wanted

invalid

question

recording

security

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

rbrooks/Quest-Board!466

No description provided.