[Privacy] Per-player erasure from transcripts and recordings #118

Closed
opened 2026-07-14 19:53:38 +00:00 by claude-bot · 1 comment
Contributor

Context / Motivation

A player leaving a group can reasonably ask to be scrubbed from recorded material. This is tractable because content is speaker-attributed end to end:

  • Transcripts are stored as a single Text column of attributed lines [HH:MM:SS] Speaker: text (Session.transcript, webapp/backend/app/models/session.py:114; built by merge_attributed_transcript, services/audio_service.py:401, from per-speaker segments {start, end, text, speaker}). Speaker labels are character names resolved from speakers.jsonPlatformLinkCampaignMember (tasks/reminder_tasks.py:1541-1584).
  • Per-speaker audio: one WAV per Discord user in the session's audio_temp directory; the directory is retained until GM approval + retention expiry (cleanup_trashed_audio, reminder_tasks.py:1715 — deletes whole dirs after audio_trashed_at + retention days).
  • Transcript search: transcript_search_vector / summary_search_vector are generated TSVECTOR columns with GIN indexes (models/session.py:116-131, migration alembic/versions/d4e5f6g7h8i9_session_summary_transcript_search.py) — rewriting transcript automatically reindexes them.
  • Pipeline intermediates: LoreExtractCache rows hold per-chunk extracted content (models/lore_extract_cache.pysession_id, chunk_index, extracts JSONB).

Spec

Trigger: GM (per their campaign) or instance admin action: "Erase member from recordings" — endpoint like POST /campaigns/{campaign_id}/members/{member_id}/erase-recordings, with type-to-confirm UI on the campaign member management page.

Erasure task (Celery, per campaign member):

  1. Transcripts: for every session in the campaign, rewrite Session.transcript, replacing the text of each line attributed to that member's speaker label(s) with [removed] — preserve the [HH:MM:SS] Speaker: skeleton so the timeline stays readable. Resolve all labels the member may appear under: current character_name, and their Discord display name (the pre-character_map fallback). Generated tsvector columns reindex automatically — verify with a search test.
  2. Audio: delete the member's per-speaker WAV file (matched by their Discord user id from PlatformLink) from any session audio directory still on the audio_temp volume; leave other speakers' files.
  3. Pipeline artifacts: delete or scrub LoreExtractCache rows for affected sessions (simplest: delete rows for those sessions — they are cache).
  4. Quotes/highlights: if the highlights feature (Table Tools milestone) has shipped, delete that member's attributed quotes.
  5. Summaries: optionally enqueue summary regeneration for affected sessions (flag on the request, default off — regeneration costs LLM calls and may alter approved text; if off, summaries are left as-is and the erasure note flags them).
  6. Marking: set an erasure note on affected sessions (e.g. "Contributions from a former member were removed on ") visible on the session page.
  7. Audit: one member.recordings_erased entry via audit_service.log_event (services/audit_service.py:12) with session counts in context.

Out of scope

  • Full account deletion (separate issue in this milestone).
  • Editing Discord-side messages/embeds already posted.
  • Voice-print detection for unattributed audio (mixed MP3s from the pre-per-speaker era, if any, can only be deleted wholesale — surface this to the GM if such artifacts exist).

Acceptance criteria

  • After erasure, searching the erased member's lines via transcript search returns nothing (FTS verified against the generated tsvector), and the transcript view shows [removed] placeholders with an intact timeline.
  • The member's WAVs are gone from any retained audio dirs; other speakers' WAVs untouched.
  • LoreExtractCache rows for affected sessions are purged.
  • Affected sessions display the erasure note; exactly one audit entry exists.
  • Idempotent: running erasure twice is safe and produces no second audit spam (or a clearly-marked repeat entry — decide and test).

References

  • webapp/backend/app/models/session.py:114 (transcript), :116-131 (generated tsvector columns)
  • webapp/backend/alembic/versions/d4e5f6g7h8i9_session_summary_transcript_search.py
  • webapp/backend/app/services/audio_service.py:401 (merge_attributed_transcript)
  • webapp/backend/app/tasks/reminder_tasks.py:1541-1584 (speaker→member resolution), :1715 (cleanup_trashed_audio)
  • webapp/backend/app/models/lore_extract_cache.py
  • webapp/backend/app/services/audit_service.py:12 (log_event)

Filed from the July 2026 full-project review.

## Context / Motivation A player leaving a group can reasonably ask to be scrubbed from recorded material. This is tractable because content is speaker-attributed end to end: - Transcripts are stored as a single Text column of attributed lines `[HH:MM:SS] Speaker: text` (`Session.transcript`, `webapp/backend/app/models/session.py:114`; built by `merge_attributed_transcript`, `services/audio_service.py:401`, from per-speaker segments `{start, end, text, speaker}`). Speaker labels are character names resolved from `speakers.json` → `PlatformLink` → `CampaignMember` (`tasks/reminder_tasks.py:1541-1584`). - Per-speaker audio: one WAV per Discord user in the session's audio_temp directory; the directory is retained until GM approval + retention expiry (`cleanup_trashed_audio`, `reminder_tasks.py:1715` — deletes whole dirs after `audio_trashed_at` + retention days). - Transcript search: `transcript_search_vector` / `summary_search_vector` are **generated** TSVECTOR columns with GIN indexes (`models/session.py:116-131`, migration `alembic/versions/d4e5f6g7h8i9_session_summary_transcript_search.py`) — rewriting `transcript` automatically reindexes them. - Pipeline intermediates: `LoreExtractCache` rows hold per-chunk extracted content (`models/lore_extract_cache.py` — `session_id`, `chunk_index`, `extracts` JSONB). ## Spec **Trigger**: GM (per their campaign) or instance admin action: "Erase member from recordings" — endpoint like `POST /campaigns/{campaign_id}/members/{member_id}/erase-recordings`, with type-to-confirm UI on the campaign member management page. **Erasure task** (Celery, per campaign member): 1. **Transcripts**: for every session in the campaign, rewrite `Session.transcript`, replacing the text of each line attributed to that member's speaker label(s) with `[removed]` — preserve the `[HH:MM:SS] Speaker:` skeleton so the timeline stays readable. Resolve all labels the member may appear under: current `character_name`, and their Discord display name (the pre-`character_map` fallback). Generated tsvector columns reindex automatically — verify with a search test. 2. **Audio**: delete the member's per-speaker WAV file (matched by their Discord user id from `PlatformLink`) from any session audio directory still on the audio_temp volume; leave other speakers' files. 3. **Pipeline artifacts**: delete or scrub `LoreExtractCache` rows for affected sessions (simplest: delete rows for those sessions — they are cache). 4. **Quotes/highlights**: if the highlights feature (Table Tools milestone) has shipped, delete that member's attributed quotes. 5. **Summaries**: optionally enqueue summary regeneration for affected sessions (flag on the request, default **off** — regeneration costs LLM calls and may alter approved text; if off, summaries are left as-is and the erasure note flags them). 6. **Marking**: set an erasure note on affected sessions (e.g. "Contributions from a former member were removed on <date>") visible on the session page. 7. **Audit**: one `member.recordings_erased` entry via `audit_service.log_event` (`services/audit_service.py:12`) with session counts in context. ## Out of scope - Full account deletion (separate issue in this milestone). - Editing Discord-side messages/embeds already posted. - Voice-print detection for unattributed audio (mixed MP3s from the pre-per-speaker era, if any, can only be deleted wholesale — surface this to the GM if such artifacts exist). ## Acceptance criteria - After erasure, searching the erased member's lines via transcript search returns nothing (FTS verified against the generated tsvector), and the transcript view shows `[removed]` placeholders with an intact timeline. - The member's WAVs are gone from any retained audio dirs; other speakers' WAVs untouched. - `LoreExtractCache` rows for affected sessions are purged. - Affected sessions display the erasure note; exactly one audit entry exists. - Idempotent: running erasure twice is safe and produces no second audit spam (or a clearly-marked repeat entry — decide and test). ## References - `webapp/backend/app/models/session.py:114` (`transcript`), `:116-131` (generated tsvector columns) - `webapp/backend/alembic/versions/d4e5f6g7h8i9_session_summary_transcript_search.py` - `webapp/backend/app/services/audio_service.py:401` (`merge_attributed_transcript`) - `webapp/backend/app/tasks/reminder_tasks.py:1541-1584` (speaker→member resolution), `:1715` (`cleanup_trashed_audio`) - `webapp/backend/app/models/lore_extract_cache.py` - `webapp/backend/app/services/audit_service.py:12` (`log_event`) _Filed from the July 2026 full-project review._
Author
Contributor

Done — merged in PR #202 (backend 653dacc + frontend). CI green.

Shipped: run_member_erasure scrubs a member's transcript lines to [HH:MM:SS] <label>: [removed] (skeleton preserved; FTS reindexes → erased words unsearchable), deletes their {discord_id}.wav (verified PlatformLink), purges LoreExtractCache, deletes their attributed quote highlights (#116), appends a session erasure_notes entry (migration a2b3c4d5e6f7), and writes one member.recordings_erased audit entry attributed to the acting GM/admin. Idempotent. GM/admin endpoint; frontend type-to-confirm (type the member's name) + erasure notice on affected sessions.

Two review fixes: threaded the actor into the audit entry (was system-actor — a destructive action needs accountability); fixed a test whose FTS token sat in the preserved speaker label rather than the redacted spoken text (scrub code was correct).

Tests: backend 561 (+11, incl. FTS-unsearchable + idempotency + authz); frontend 296 (+10). All acceptance criteria met. Closing.

Done — merged in PR #202 (backend `653dacc` + frontend). CI green. **Shipped:** `run_member_erasure` scrubs a member's transcript lines to `[HH:MM:SS] <label>: [removed]` (skeleton preserved; FTS reindexes → erased words unsearchable), deletes their `{discord_id}.wav` (verified PlatformLink), purges `LoreExtractCache`, deletes their attributed quote highlights (#116), appends a session `erasure_notes` entry (migration `a2b3c4d5e6f7`), and writes one `member.recordings_erased` audit entry **attributed to the acting GM/admin**. Idempotent. GM/admin endpoint; frontend type-to-confirm (type the member's name) + erasure notice on affected sessions. **Two review fixes:** threaded the actor into the audit entry (was system-actor — a destructive action needs accountability); fixed a test whose FTS token sat in the *preserved speaker label* rather than the redacted spoken text (scrub code was correct). **Tests:** backend 561 (+11, incl. FTS-unsearchable + idempotency + authz); frontend 296 (+10). All acceptance criteria met. Closing.
rbrooks referenced this issue from a commit 2026-07-18 09:02:57 +00:00
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#118
No description provided.