Erasing one member's recordings restarts everyone else's retention clock #791

Closed
opened 2026-09-23 22:51:45 +00:00 by claude-bot · 4 comments
Collaborator

Found building the retention-sweep preview (#778, B3).

run_member_erasure sets transcript_updated_at = now on every session it scrubs. That field is the anchor for both audio and transcript retention (app/retention.py: audio_retention_anchor uses transcript_updated_at, and the transcript window measures from it).

So erasing one member's recordings from a session restarts both retention windows for everybody else's audio and words in that session. A campaign set to keep audio 30 days, whose GM erases Ade from a 25-day-old session, now keeps everyone else's audio from that session for another 30 days.

That's the wrong direction for a privacy control. Erasing someone should never cause more to be kept.

Reprocessing (reminder_tasks.py, around line 3372) resets the same field. There it's arguably right, because the transcript really is new, but it's worth deciding deliberately rather than inheriting.

Fix

Retention needs an anchor that means "when this session's content was made", separate from "when its transcript was last touched". Either:

  • the erase stops touching transcript_updated_at, and whatever relied on that bump (a cache, a "last edited" display) gets its own field; or
  • retention anchors on a field that erasing doesn't move, e.g. the session's processing time.

Whichever is chosen, select_retention_catch in app/retention.py is now the one place retention decides what is caught (#778), so the change lands in one spot and the sweep and the preview stay in agreement.

A test: erase a member from an old session, and assert that the other members' audio and transcript are still caught by the sweep on the original schedule.

🤖 Generated with Claude Code

Found building the retention-sweep preview (#778, B3). `run_member_erasure` sets `transcript_updated_at = now` on every session it scrubs. **That field is the anchor for both audio and transcript retention** (`app/retention.py`: `audio_retention_anchor` uses `transcript_updated_at`, and the transcript window measures from it). So erasing *one* member's recordings from a session **restarts both retention windows for everybody else's audio and words in that session**. A campaign set to keep audio 30 days, whose GM erases Ade from a 25-day-old session, now keeps everyone else's audio from that session for another 30 days. That's the wrong direction for a privacy control. Erasing someone should never cause *more* to be kept. Reprocessing (`reminder_tasks.py`, around line 3372) resets the same field. There it's arguably right, because the transcript really is new, but it's worth deciding deliberately rather than inheriting. ## Fix Retention needs an anchor that means "when this session's content was made", separate from "when its transcript was last touched". Either: - the erase stops touching `transcript_updated_at`, and whatever relied on that bump (a cache, a "last edited" display) gets its own field; or - retention anchors on a field that erasing doesn't move, e.g. the session's processing time. Whichever is chosen, `select_retention_catch` in `app/retention.py` is now the **one** place retention decides what is caught (#778), so the change lands in one spot and the sweep and the preview stay in agreement. A test: erase a member from an old session, and assert that the other members' audio and transcript are still caught by the sweep on the original schedule. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Collaborator

A second symptom of the same root cause, found while building Option A (#792).

A session being reprocessed is in status processing while still carrying the old transcript_updated_at from its earlier successful run. Under any windowed audio mode, if that old anchor is past the window, the nightly sweep can catch it — and bin the audio while it is being reprocessed, failing the reprocess.

It's pre-existing, not introduced by #792: Option A only changes what ready sessions do under "bin once transcribed", and processing is never ready.

Both symptoms come from retention anchoring on transcript_updated_at, which means "when the transcript was last touched" rather than "when this content was made". Whatever this issue settles on for the anchor should also make the selection skip a session that is mid-processing. select_retention_catch in app/retention.py is now the one place that decides, so the fix lands there, and the preview and the sweep stay in agreement.

🤖 Generated with Claude Code

**A second symptom of the same root cause**, found while building Option A (#792). A session being **reprocessed** is in status `processing` while still carrying the old `transcript_updated_at` from its earlier successful run. Under any windowed audio mode, if that old anchor is past the window, the nightly sweep can catch it — and bin the audio **while it is being reprocessed**, failing the reprocess. It's pre-existing, not introduced by #792: Option A only changes what `ready` sessions do under "bin once transcribed", and `processing` is never `ready`. Both symptoms come from retention anchoring on `transcript_updated_at`, which means "when the transcript was last touched" rather than "when this content was made". Whatever this issue settles on for the anchor should also make the selection skip a session that is mid-processing. `select_retention_catch` in `app/retention.py` is now the one place that decides, so the fix lands there, and the preview and the sweep stay in agreement. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Collaborator

Owner, 2026-09-24 (R1): option (a), and fix it now, before lane 4.

Retention counts from when the session's audio was first processed. That's a new, write-once timestamp that neither a member erase nor a reprocess moves. It measures "how long have we held this recording", which is what the setting promises. The alternative, the session's played date, would count a late upload as old from its first day.

As part of the same fix, a session that is mid-processing is never caught, whatever its dates.

The change lands in app/retention.py's select_retention_catch, so the nightly sweep and the preview (#778) move together, and the agreement test keeps them honest.

🤖 Generated with Claude Code

**Owner, 2026-09-24 (R1): option (a), and fix it now, before lane 4.** Retention counts from **when the session's audio was first processed**. That's a new, write-once timestamp that neither a member erase nor a reprocess moves. It measures "how long have we held this recording", which is what the setting promises. The alternative, the session's played date, would count a late upload as old from its first day. As part of the same fix, a session that is **mid-processing is never caught**, whatever its dates. The change lands in `app/retention.py`'s `select_retention_catch`, so the nightly sweep and the preview (#778) move together, and the agreement test keeps them honest. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Collaborator

Fixed by #800. Retention now counts from sessions.audio_processed_at, a write-once stamp set the first time a session's audio is processed successfully. A member erase or a reprocess no longer restarts anyone's clock. A session mid-processing is never caught.

The migration (e1f2a3b4c5d6) backfills the stamp from each row's existing anchor, so nothing changes on upgrade day. Sessions erased or reprocessed before the upgrade keep the later date they already had.

One open question for the owner. A forced re-upload of new audio to an already-transcribed session keeps the first stamp. The new recording's clock therefore starts from the original processing, not from the re-upload. This follows "write-once" literally. It still goes through the bin's grace period, never straight to deletion. If you meant per recording, it's a small change: reset the stamp on a forced upload.

Follow-ups: #798 (a failed run's clock) and #797 (the erase dialog's regenerate option).

🤖 Generated with Claude Code

Fixed by #800. Retention now counts from `sessions.audio_processed_at`, a write-once stamp set the first time a session's audio is processed successfully. A member erase or a reprocess no longer restarts anyone's clock. A session mid-processing is never caught. The migration (`e1f2a3b4c5d6`) backfills the stamp from each row's existing anchor, so nothing changes on upgrade day. Sessions erased or reprocessed before the upgrade keep the later date they already had. **One open question for the owner.** A forced re-upload of new audio to an already-transcribed session keeps the *first* stamp. The new recording's clock therefore starts from the original processing, not from the re-upload. This follows "write-once" literally. It still goes through the bin's grace period, never straight to deletion. If you meant per recording, it's a small change: reset the stamp on a forced upload. Follow-ups: #798 (a failed run's clock) and #797 (the erase dialog's regenerate option). 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Collaborator

The re-upload question is settled: leave it. In practice a forced re-upload never happens. Nothing sends force=true: the bot never sets it, and the web app has no control for it. A second recording on an already-transcribed session is refused with a 409 by #397's guard, so the case can only be reached by a hand-made API call with the bot key. A reprocess of the same audio is correctly anchored on the first processing.

Revisit only if a product flow ever starts sending force, for example recording a session's second half after the break. Such a flow should reset the stamp. The owner asked how often it would happen, and was told this on 2026-09-24.

🤖 Generated with Claude Code

**The re-upload question is settled: leave it.** In practice a forced re-upload never happens. Nothing sends `force=true`: the bot never sets it, and the web app has no control for it. A second recording on an already-transcribed session is refused with a 409 by #397's guard, so the case can only be reached by a hand-made API call with the bot key. A reprocess of the *same* audio is correctly anchored on the first processing. Revisit only if a product flow ever starts sending `force`, for example recording a session's second half after the break. Such a flow should reset the stamp. The owner asked how often it would happen, and was told this on 2026-09-24. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#791
No description provided.