An exact preview of what a retention change would delete, an exact erase count, and a fix for over-cap campaigns (#778) #792

Merged

claude-bot merged 4 commits from build/retention-preview into main 2026-09-24 00:21:10 +00:00

Conversation 1 Commits 4 Files changed 17 +2601 -445

claude-bot commented 2026-09-23 23:08:41 +00:00

Collaborator

Copy link

Closes #778. Owner, 2026-09-23 (B3): build it before v4.5.0.

This is the confirmation in front of deleting people's recorded voices, so the rule throughout is: the thing that counts and the thing that deletes are the same function.

The retention preview

GET /api/campaigns/{id}/retention-preview?audio_mode=&audio_days=&transcript_mode=&transcript_months= is GM-only and takes a proposed pair, the value the GM has typed and not saved yet. It returns, per side, the sessions that would actually lose audio and those that would lose a transcript, newest first. Nothing is written.

The preview and the nightly sweep share one selection. app/retention.py now holds select_retention_catch, and enforce_retention's audio and transcript passes simply act on what it returns. The preview calls the same function, with:

• the same cap refusal: an over-cap proposal gets a 400 whose body is exactly the PATCH's, and a test asserts the two bodies are equal;
• the same resolve-then-clamp: effective_retention_from, split out of get_effective_retention, which now calls it.

The agreement is tested, not asserted. test_retention_preview.py:

• pins the clock and seeds eight sessions, one per branch of the selection;
• then for six pairs, capped and uncapped (12 runs), previews, PATCHes the same pair, runs the real sweep, and checks that what was actually binned and cleared equals what the preview listed;
• has one case guarding against both sides being empty.

Mutation-checked: making the preview skip the clamp fails the capped case.

"Has audio" and "has a transcript" mean what the sweep keys on: the database columns (audio_file_path, transcript and their anchors), not files on disk. The trashed-campaign cache the sweep keeps is untouched.

On the page (IS-313), the hedge is gone. It now reads, for example, "Three sessions have audio older than that — Kettlebridge, … and …". It asks about a side only if that side changed, previews as the GM types (250 ms debounce, abortable), and shows the over-cap refusal before anyone presses Save.

An exact count for erasing a member's recordings

GET /api/campaigns/{id}/members/{user_id}/recordings → {sessions, recordings, transcripts, quotes}, behind the erase's own authorisation.

Same principle: run_member_erasure no longer decides as it goes. plan_member_erasure builds a plan, reading only. The erase applies exactly that plan, and the count counts it. Tested: count, then erase and confirm the erase matches the count exactly, then count again and get all zeros.

The warning on Members now reads: "It cannot be undone: it takes Ade's audio from 11 sessions, their lines from 9 transcripts and 2 quotes attributed to them."

A bug in a page shipped this week, fixed here

A campaign whose stored retention is above a cap set later couldn't save anything on the Recording page. The page PATCHed all four retention fields on every save, and the cap check treated a present field as a change, so flipping a toggle got a 400. D24 deliberately keeps such an override for the day the cap is raised, so this state is expected, not rare.

Fix: the server checks a retention field against the cap only if it differs from the stored value. An over-cap value still can't be newly set, and the stored one is still clamped on read, which is what the sweep obeys. The page also sends only the fields the GM changed.

Tests:

• unchanged over-cap fields plus a toggle → 200;
• the same over-cap value again → 200, still clamped;
• a new over-cap value → still 400 with the same sentence;
• refusal parity with the preview, in both directions.

Also

The member search ("Add someone", "Attach to an account") is debounced 250 ms and abortable, and never shows a stale answer for the current text.

Checks

• Backend: retention, cap, preview and campaign suites 270 passed; full suite 4043 passed. The one failure reads repo-root files the backend-only mount lacks, and CI mounts the full checkout. ruff 0.4.4 clean.
• Frontend: vitest 1941 passed (--maxWorkers=2); eslint 0 errors, and the warning count didn't rise.

Reported, not changed

• Erasing one member resets everyone else's retention clock in those sessions (transcript_updated_at = now), so their audio and words are kept longer. That's the wrong direction for a privacy control. Filed as #791.
• "Bin it once transcribed" in the nightly sweep uses the audio-days window, so switching a campaign to it bins backlog older than the instance default rather than all already-transcribed audio. New audio is binned at processing time either way. The preview shows the truth; whether the mode should also clear the backlog at once is a product question.
• The preview measures from now, and the sweep runs nightly, so a session crossing the window in between isn't listed. The gap is under a day.
• The erase endpoint doesn't refuse a trashed campaign. That's unchanged behaviour.

🤖 Generated with Claude Code

Closes #778. Owner, 2026-09-23 (B3): build it before v4.5.0. This is the confirmation in front of deleting people's recorded voices, so the rule throughout is: **the thing that counts and the thing that deletes are the same function.** ## The retention preview `GET /api/campaigns/{id}/retention-preview?audio_mode=&audio_days=&transcript_mode=&transcript_months=` is GM-only and takes a **proposed** pair, the value the GM has typed and not saved yet. It returns, per side, the sessions that would actually lose audio and those that would lose a transcript, newest first. Nothing is written. **The preview and the nightly sweep share one selection.** `app/retention.py` now holds `select_retention_catch`, and `enforce_retention`'s audio and transcript passes simply act on what it returns. The preview calls the same function, with: - the same cap refusal: an over-cap proposal gets a 400 whose body is **exactly** the PATCH's, and a test asserts the two bodies are equal; - the same resolve-then-clamp: `effective_retention_from`, split out of `get_effective_retention`, which now calls it. **The agreement is tested, not asserted.** `test_retention_preview.py`: - pins the clock and seeds eight sessions, one per branch of the selection; - then for six pairs, capped and uncapped (12 runs), previews, PATCHes the same pair, runs the real sweep, and checks that what was actually binned and cleared **equals** what the preview listed; - has one case guarding against both sides being empty. Mutation-checked: making the preview skip the clamp fails the capped case. "Has audio" and "has a transcript" mean what the sweep keys on: the database columns (`audio_file_path`, `transcript` and their anchors), not files on disk. The trashed-campaign cache the sweep keeps is untouched. **On the page (`IS-313`)**, the hedge is gone. It now reads, for example, *"Three sessions have audio older than that — Kettlebridge, … and …"*. It asks about a side only if that side changed, previews as the GM types (250 ms debounce, abortable), and shows the over-cap refusal **before** anyone presses Save. ## An exact count for erasing a member's recordings `GET /api/campaigns/{id}/members/{user_id}/recordings` → `{sessions, recordings, transcripts, quotes}`, behind the erase's own authorisation. Same principle: `run_member_erasure` no longer decides as it goes. `plan_member_erasure` builds a plan, reading only. The erase applies exactly that plan, and the count counts it. Tested: count, then erase and confirm the erase matches the count exactly, then count again and get all zeros. The warning on Members now reads: *"It cannot be undone: it takes Ade's audio from 11 sessions, their lines from 9 transcripts and 2 quotes attributed to them."* ## A bug in a page shipped this week, fixed here **A campaign whose stored retention is above a cap set later couldn't save anything on the Recording page.** The page PATCHed all four retention fields on every save, and the cap check treated a present field as a change, so flipping a toggle got a 400. D24 deliberately keeps such an override for the day the cap is raised, so this state is expected, not rare. **Fix:** the server checks a retention field against the cap only if it **differs from the stored value**. An over-cap value still can't be newly set, and the stored one is still clamped on read, which is what the sweep obeys. The page also sends only the fields the GM changed. Tests: - unchanged over-cap fields plus a toggle → 200; - the same over-cap value again → 200, still clamped; - a new over-cap value → still 400 with the same sentence; - refusal parity with the preview, in both directions. ## Also The member search ("Add someone", "Attach to an account") is debounced 250 ms and abortable, and never shows a stale answer for the current text. ## Checks - **Backend:** retention, cap, preview and campaign suites 270 passed; full suite 4043 passed. The one failure reads repo-root files the backend-only mount lacks, and CI mounts the full checkout. ruff 0.4.4 clean. - **Frontend:** vitest 1941 passed (`--maxWorkers=2`); eslint 0 errors, and the warning count didn't rise. ## Reported, not changed - **Erasing one member resets everyone else's retention clock** in those sessions (`transcript_updated_at = now`), so their audio and words are kept longer. That's the wrong direction for a privacy control. Filed as **#791**. - **"Bin it once transcribed" in the nightly sweep** uses the audio-days window, so switching a campaign to it bins backlog older than the instance default rather than *all* already-transcribed audio. New audio is binned at processing time either way. The preview shows the truth; whether the mode should also clear the backlog at once is a product question. - **The preview measures from now**, and the sweep runs nightly, so a session crossing the window in between isn't listed. The gap is under a day. - **The erase endpoint doesn't refuse a trashed campaign.** That's unchanged behaviour. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 2 commits 2026-09-23 23:08:42 +00:00

feat(webapp): an exact preview of what a retention change would delete (#778, #729) ... e43da4d626

The confirmation in front of deleting people's recorded voices stops
hedging. IS-313 used to count sessions "old enough to be caught" and say
"any audio they still have goes to the bin", because nothing on the wire
could say what a session still held.

GET /api/campaigns/{id}/retention-preview takes the pair a GM has typed
and not saved and answers with the sessions that would actually lose
audio and those that would actually lose a transcript. It is the sweep's
own selection: enforce_retention and the preview now both call
retention.select_retention_catch, the policy is resolved and clamped by
the same settings_service.effective_retention_from, and an over-cap pair
gets the PATCH's own 400 and sentence (the same _check_retention_within_cap
call). A test previews, saves, runs the sweep at the same pinned instant,
and checks both took the same sessions, across capped and uncapped pairs.

The member erase is counted the same way. run_member_erasure now applies
a plan built by plan_member_erasure, which decides and changes nothing;
GET .../members/{user_id}/recordings counts that plan, so "Ade's audio
from 11 sessions" is the erase's own number. A test counts, erases,
compares, and counts again to find nothing left.

The page asks while the GM types (debounced like the app bar's search),
shows an over-cap refusal before anyone saves, asks only about the side
that changed, and says exactly what goes, naming it as the drawing does.
The member search on Members is debounced the same way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

fix(webapp): a campaign kept above a later cap can save its settings again (#778, D24) ... 59c1fdc165

A campaign whose stored retention override outlives a cap set after it
was written keeps that preference on purpose (D24). But the Recording
settings page sent all four retention fields on every save, and
_check_retention_within_cap treated any present field as touched, so
not one toggle on that page could be saved.

The server now checks only a retention side whose value actually
changes: a field sent equal to what the campaign stores is not a change.
An over-cap value still cannot be newly set (same 400, same sentence),
and the stored one is still clamped on read by get_effective_retention,
so the sweep obeys the cap either way. The page, belt and braces, sends
only the retention fields the GM changed, as the preview already only
asks about a changed side.

The preview/PATCH refusal-parity test is unchanged and passes; a new one
checks parity the other way, that a stored over-cap pair left as it is is
refused by neither.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude-bot commented 2026-09-23 23:19:43 +00:00

Author

Collaborator

Copy link

Owner, 2026-09-23: Option A for "Bin it once transcribed", the second item under Reported, not changed above.

Choosing trash_after_processing now also clears all already-transcribed audio at the next sweep, not only backlog older than the instance's default days. "Already transcribed" means exactly what the post-processing step would have binned at the time. Audio from a failed or unfinished transcription is not caught, because it's kept so it can be reprocessed.

It goes in the shared select_retention_catch, so the sweep and the preview change together and the agreement test covers it. It's being added to this PR as a further commit, with the test cases: a transcribed session younger than the window is now caught; a failed run and already-trashed audio are not.

🤖 Generated with Claude Code

**Owner, 2026-09-23: Option A** for "Bin it once transcribed", the second item under *Reported, not changed* above. Choosing `trash_after_processing` now also clears **all already-transcribed audio** at the next sweep, not only backlog older than the instance's default days. "Already transcribed" means exactly what the post-processing step would have binned at the time. Audio from a failed or unfinished transcription is **not** caught, because it's kept so it can be reprocessed. It goes in the shared `select_retention_catch`, so the sweep and the preview change together and the agreement test covers it. It's being added to this PR as a further commit, with the test cases: a transcribed session younger than the window is now caught; a failed run and already-trashed audio are not. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot referenced this pull request 2026-09-23 23:32:23 +00:00

Erasing one member's recordings restarts everyone else's retention clock #791

rbrooks added 1 commit 2026-09-23 23:32:30 +00:00

feat(backend): "Bin it once transcribed" bins every transcribed recording at the next sweep (#778) ... 276e3c6559

The owner's decision, 2026-09-23 (option A): choosing
trash_after_processing says "don't keep audio". The success path of
process_audio already bins each new recording as its transcript is
written, but audio transcribed while the campaign was under another mode
waited out the instance's default days first. It now goes at the next
sweep, with no age window.

"Already transcribed" is exactly the state the success path leaves, and
the only state in which apply_post_processing_audio_retention bins:
status ready with transcript_updated_at set (retention.
transcribed_successfully). Both halves matter: an admin restore from the
bin also leaves status ready, and that bin can hold a failed run's audio,
which has no transcript_updated_at. Anything else, above all a failed run
kept for reprocessing, keeps exactly the windowed behaviour it had.
Trashed audio stays skipped; transcripts are unaffected.

It lives in retention.audio_window_catches, so the sweep and the preview
change together. The agreement test now asserts, for this mode chosen or
inherited, that a transcribed session inside the default window goes and
a recent failed run, a restored-never-transcribed run and already-trashed
audio do not, on both sides. Disabling the rule fails 7 tests.

The page says "One session's audio goes to the bin" as before, and when
nothing is caught under this mode says "No transcribed recording is still
being kept, so nothing goes" rather than "nothing is that old yet".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

rbrooks added 1 commit 2026-09-23 23:56:11 +00:00

Merge main into build/retention-preview (#778) ... 4ea36d75cc

#790 (backups) landed underneath, and both change `reminder_tasks.py`: #790
the backup task, this branch the retention sweep. It auto-merged; the only
conflict was `CHANGELOG.md`, where each side held only its own entries.

Verified rather than assumed: the full backend suite on the merged tree,
with the whole repo mounted — 4096 passed, 13 skipped, 0 failed.

Worth knowing for anyone re-running a subset: five tests in
`test_backup_tasks.py` fail when that file runs on its own (identically on
main), because they need a table another test file creates first. They
pass in a full run. That is a test-isolation weakness, not a regression.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

claude-bot merged commit 7dc3296772 into main 2026-09-24 00:21:10 +00:00

claude-bot referenced this pull request from a commit 2026-09-24 00:21:11 +00:00

Merge PR #792: an exact retention preview, an exact erase count, over-cap campaigns can save again, and bin-once-transcribed clears the backlog (#778)

claude-bot referenced this pull request 2026-09-24 00:22:48 +00:00

Nothing can preview a retention sweep, so IS-313 has to hedge #778

rbrooks referenced this pull request from a commit 2026-09-24 00:48:10 +00:00

Merge main into build/shell-followups (#780, D93)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

backend

Issue related to the backend

  

bot

Issue related to the bot

  

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

frontend

Issue related to the frontend

  

help wanted

Need some help

  

invalid

Something is wrong

  

question

More information is needed

  

recording

Issue related to the recording system

  

security

Security hardening / vulnerability work

  

wontfix

This won't be fixed

No labels

backend

bot

bug

duplicate

enhancement

frontend

help wanted

invalid

question

recording

security

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

rbrooks/Quest-Board!792

No description provided.