[Privacy] Configurable retention policies for audio and transcripts #119
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Context / Motivation
Retention is currently implicit. Inspected behavior today:
process_audio(webapp/backend/app/tasks/reminder_tasks.py:1461) explicitly keeps the session directory on both success and failure (docstring at:1483: the directory "stays until a GM approves it").cleanup_trashed_audio(reminder_tasks.py:1715): sessions whoseaudio_processing_statusistrashed(i.e. GM approved) and whoseaudio_trashed_atis older than the retention window (get_audio_trash_retention_days,services/settings_service.py:243, default 7 days) getshutil.rmtreeon the audio dir, then status →approved.Session.transcript(models/session.py:114) lives forever.Spec
Settings — instance-level defaults (Admin UI, stored via
settings_serviceinapp_settingslike existing keys) + per-campaign overrides (nullable columns = inherit instance default):audio_retention:delete_after_processing(recommended default) |retain_days_N|retain_indefinitelydelete_after_processingchanges the pipeline contract: delete the WAV dir at the end of successfulprocess_audio(transcript is the artifact of record; note this removes the ability to reprocess — surface that in the settings help text). Failed processing keeps audio for debugging until N days.retain_days_Ngeneralizes today's behavior but anchored on processing completion, not GM approval, so unapproved sessions can't hoard audio forever.transcript_retention:retain_indefinitely(default) |retain_months_N— expiry clearsSession.transcript(and per-issue-#118 artifacts likeLoreExtractCacherows); summaries and wiki/lore entries are unaffected.Enforcement — extend/replace
cleanup_trashed_audiowith a general retention Beat task: computes overdue artifacts from settings, hard-deletes (rmtree for audio;NULL-out transcript columns — the generated tsvector columns (models/session.py:116-131) empty automatically), and logs a summary line of what it removed (counts per campaign). Idempotent and exactly-once per artifact: deletion is driven by current state, so a rerun finds nothing to delete.Audit: one audit entry per enforcement run that deleted anything (
retention.enforced, counts in context) viaaudit_service.log_event.Docs: README privacy stance section — "transcript-only by default": recommended config keeps transcripts, deletes raw audio after processing.
Out of scope
Acceptance criteria
delete_after_processing, the WAV dir is gone immediately after a successful pipeline run; transcript intact.retain_days_N, audio expires N days after processing regardless of approval state (test the never-approved case).LoreExtractCache; summary untouched.References
webapp/backend/app/tasks/reminder_tasks.py:1461(process_audio; retention docstring:1483),:1715(cleanup_trashed_audio)webapp/backend/app/services/settings_service.py:243(get_audio_trash_retention_days)webapp/backend/app/models/session.py:114(transcript),:116-131(generated tsvector),:143-156(audio status/trash columns)webapp/backend/app/models/lore_extract_cache.pywebapp/backend/app/services/audit_service.py:12(log_event)Filed from the July 2026 full-project review.
Done — merged in PR #201 (backend
f31d7f8+ frontend). CI green (first try).Shipped: instance-default + per-campaign audio/transcript retention (
get_effective_retentionresolves per-field, override-wins/null-inherits); migrationf1a2b3c4d5e6.delete_after_processingdeletes the WAV dir inline at the end of a successfulprocess_audio(defensive, → terminalapproved);enforce_retentionBeat task anchors audio deletion ontranscript_updated_atregardless of approval (fixes the never-approved hoarding), andretain_monthsclears transcript+FTS + purgesLoreExtractCache(summaries untouched); one idempotentretention.enforcedaudit entry. Admin "Data retention" tab + per-campaign overrides; README privacy section.Tests: backend 550 (+9), frontend 286 (+10). All acceptance criteria met (delete_after_processing immediacy, never-approved retain_days, transcript+FTS+cache expiry with summary intact, idempotency, override resolution). Closing.