rbrooks/Quest-Board

Watch

Fork 0

[Privacy] Complete the audit log: coverage, campaign scoping, admin UI, pruning #120

New issue

Closed

opened 2026-07-14 19:54:40 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-07-14 19:54:40 +00:00

Contributor

Context / Motivation

An audit log already exists but is narrow. Current state (verified):

Model AuditLog (webapp/backend/app/models/audit_log.py): id, created_at (indexed), actor_id (nullable UUID, indexed, no FK — None for system/bot), event (String(100), indexed), description (Text), context (JSONB).
Service (webapp/backend/app/services/audit_service.py): log_event(db, event, description, *, actor_id, context) — flush-only, one-line call sites — and get_recent_events(limit, event_filter, actor_id).
API: GET /admin/audit-log (admin_get_audit_log, routers/users.py:267, admin-only, event-prefix + actor filters, limit ≤ 1000).
Only 7 write sites: admin.role.{granted|revoked} (users.py:217), admin.settings.notifications.updated (:359), admin.settings.bot.updated (:482), admin.bot_key.regenerated (:511), campaign.archived (campaigns.py:797), campaign.restored (:821), campaign.member.removed (:1148).

Missing: campaign scoping, target identification, most destructive actions, any frontend, and pruning.

Spec

Model extension (migration): add nullable campaign_id (UUID, indexed), target_type (String, e.g. "campaign"/"user"/"session"/"member"), target_id (UUID/Text). Composite index (campaign_id, created_at); actor_id already indexed. Keep the existing event string convention (domain.action), no enum table.

Coverage — wire these call sites (one log_event line each; keep the list tight — destructive/administrative only, explicitly NOT a general activity feed):

campaign delete (hard delete, if/where exposed) — campaign.deleted
backup config changes — admin.settings.backup.updated
Whisper/LLM (AI) settings changes — admin.settings.ai.updated
platform-link admin recovery/unlink — admin.platform_link.recovered
account deletion — user.deleted (from the account-deletion issue in this milestone)
erasure/retention actions — member.recordings_erased, retention.enforced (from the sibling issues)
(already covered: role grants, bot settings, bot key, archive/restore, member remove)

Admin UI: new Admin section "Audit log" — table over GET /admin/audit-log with filters (actor, event prefix, date range — extend the endpoint with since/until and campaign_id params) and pagination. Render context as an expandable JSON blob.

Retention/pruning: keep entries 2 years (instance setting, range 1-2y); a Beat task (pattern: cleanup_trashed_audio, tasks/reminder_tasks.py:1715) deletes older rows monthly and logs a count — the pruning run itself is NOT audit-logged (avoid self-noise), just app-logged.

Out of scope

A general user-activity feed (views, edits of ordinary content).
Log shipping / SIEM export (the table + API is the interface).
Tamper-proofing/signatures.

Acceptance criteria

Each newly wired action produces exactly one entry with actor, event, target, campaign (where applicable), and useful context (before/after for settings — redact secrets: never log key material, mirroring the existing admin.settings.bot.updated practice).
Admin UI lists, filters (actor/event/date/campaign), and paginates.
Pruning removes only rows older than the configured window and is idempotent.
Existing 7 call sites keep working unchanged (backward-compatible migration — new columns nullable).

References

webapp/backend/app/models/audit_log.py (existing model)
webapp/backend/app/services/audit_service.py (log_event, get_recent_events)
webapp/backend/app/routers/users.py:267 (admin_get_audit_log), :217/:359/:482/:511 (existing write sites)
webapp/backend/app/routers/campaigns.py:797/:821/:1148 (existing write sites)
webapp/backend/app/tasks/reminder_tasks.py:1715 (Beat-task pruning pattern)

Filed from the July 2026 full-project review.

## Context / Motivation An audit log **already exists** but is narrow. Current state (verified): - Model `AuditLog` (`webapp/backend/app/models/audit_log.py`): `id`, `created_at` (indexed), `actor_id` (nullable UUID, indexed, no FK — None for system/bot), `event` (String(100), indexed), `description` (Text), `context` (JSONB). - Service (`webapp/backend/app/services/audit_service.py`): `log_event(db, event, description, *, actor_id, context)` — flush-only, one-line call sites — and `get_recent_events(limit, event_filter, actor_id)`. - API: `GET /admin/audit-log` (`admin_get_audit_log`, `routers/users.py:267`, admin-only, event-prefix + actor filters, limit ≤ 1000). - Only 7 write sites: `admin.role.{granted|revoked}` (`users.py:217`), `admin.settings.notifications.updated` (`:359`), `admin.settings.bot.updated` (`:482`), `admin.bot_key.regenerated` (`:511`), `campaign.archived` (`campaigns.py:797`), `campaign.restored` (`:821`), `campaign.member.removed` (`:1148`). Missing: campaign scoping, target identification, most destructive actions, any frontend, and pruning. ## Spec **Model extension** (migration): add nullable `campaign_id` (UUID, indexed), `target_type` (String, e.g. "campaign"/"user"/"session"/"member"), `target_id` (UUID/Text). Composite index `(campaign_id, created_at)`; `actor_id` already indexed. Keep the existing `event` string convention (`domain.action`), no enum table. **Coverage — wire these call sites** (one `log_event` line each; keep the list tight — destructive/administrative only, explicitly NOT a general activity feed): - campaign delete (hard delete, if/where exposed) — `campaign.deleted` - backup config changes — `admin.settings.backup.updated` - Whisper/LLM (AI) settings changes — `admin.settings.ai.updated` - platform-link admin recovery/unlink — `admin.platform_link.recovered` - account deletion — `user.deleted` (from the account-deletion issue in this milestone) - erasure/retention actions — `member.recordings_erased`, `retention.enforced` (from the sibling issues) - (already covered: role grants, bot settings, bot key, archive/restore, member remove) **Admin UI**: new Admin section "Audit log" — table over `GET /admin/audit-log` with filters (actor, event prefix, date range — extend the endpoint with `since`/`until` and `campaign_id` params) and pagination. Render `context` as an expandable JSON blob. **Retention/pruning**: keep entries 2 years (instance setting, range 1-2y); a Beat task (pattern: `cleanup_trashed_audio`, `tasks/reminder_tasks.py:1715`) deletes older rows monthly and logs a count — the pruning run itself is NOT audit-logged (avoid self-noise), just app-logged. ## Out of scope - A general user-activity feed (views, edits of ordinary content). - Log shipping / SIEM export (the table + API is the interface). - Tamper-proofing/signatures. ## Acceptance criteria - Each newly wired action produces exactly one entry with actor, event, target, campaign (where applicable), and useful context (before/after for settings — redact secrets: never log key material, mirroring the existing `admin.settings.bot.updated` practice). - Admin UI lists, filters (actor/event/date/campaign), and paginates. - Pruning removes only rows older than the configured window and is idempotent. - Existing 7 call sites keep working unchanged (backward-compatible migration — new columns nullable). ## References - `webapp/backend/app/models/audit_log.py` (existing model) - `webapp/backend/app/services/audit_service.py` (`log_event`, `get_recent_events`) - `webapp/backend/app/routers/users.py:267` (`admin_get_audit_log`), `:217/:359/:482/:511` (existing write sites) - `webapp/backend/app/routers/campaigns.py:797/:821/:1148` (existing write sites) - `webapp/backend/app/tasks/reminder_tasks.py:1715` (Beat-task pruning pattern) _Filed from the July 2026 full-project review._

claude-bot added the

backend

enhancement

labels

2026-07-14 19:54:49 +00:00

claude-bot added this to the v3.8.0 - Privacy & Data Lifecycle milestone

2026-07-14 19:54:50 +00:00

rbrooks referenced this issue from a commit

2026-07-18 06:23:42 +00:00

feat(backend): audit log campaign scoping, coverage, pruning (#120)

rbrooks referenced this issue from a commit

2026-07-18 06:23:42 +00:00

feat(frontend): admin audit-log viewer (#120)

claude-bot referenced this issue from a pull request that will close it,

2026-07-18 06:23:52 +00:00

feat: audit log campaign scoping, coverage, admin UI, pruning (#120) #200

claude-bot closed this issue

2026-07-18 06:57:57 +00:00

claude-bot referenced this issue from a commit

2026-07-18 06:57:57 +00:00

Merge pull request 'feat: audit log campaign scoping, coverage, admin UI, pruning (#120)' (#200) from feat/120-audit-log into main

claude-bot commented

2026-07-18 06:58:13 +00:00

Author

Contributor

Done — merged in PR #200 (backend b2633b8 + frontend + admin UI). CI green (first run was a flaky/slow backend job; verified locally twice at 541 pass incl. the exact CI command, re-run went green).

Shipped:

AuditLog gains nullable campaign_id (indexed), target_type, target_id + composite index (campaign_id, created_at); migration a1c2d3e4f5a6 (round-trips, backward-compatible).
log_event/get_recent_events gain campaign/target/since/until/offset; new count_events.
GET /admin/audit-log gains filters + X-Total-Count; new Audit log admin tab (filter by event/actor/campaign/date, expandable context JSON, pagination).
Coverage: campaign.deleted, admin.settings.backup.updated, + campaign/target backfilled on existing sites. (AI settings share the already-logged bot handler; no admin platform-link-recovery endpoint exists — both left alone.)
Pruning: audit_log_retention_years (default 2, clamp 1–2) + monthly prune_audit_log Beat task (idempotent, not self-audited).

Tests: backend 541 pass (+test_audit_log.py); frontend 276 (+13).

The three sibling v3.8.0 issues (#117/#118/#119) now have the campaign-scoped log_event to write user.deleted / member.recordings_erased / retention.enforced into. Closing.

Done — merged in PR #200 (backend `b2633b8` + frontend + admin UI). CI green (first run was a flaky/slow backend job; verified locally twice at 541 pass incl. the exact CI command, re-run went green). **Shipped:** - `AuditLog` gains nullable `campaign_id` (indexed), `target_type`, `target_id` + composite index `(campaign_id, created_at)`; migration `a1c2d3e4f5a6` (round-trips, backward-compatible). - `log_event`/`get_recent_events` gain campaign/target/since/until/offset; new `count_events`. - `GET /admin/audit-log` gains filters + `X-Total-Count`; new **Audit log** admin tab (filter by event/actor/campaign/date, expandable context JSON, pagination). - Coverage: `campaign.deleted`, `admin.settings.backup.updated`, + campaign/target backfilled on existing sites. (AI settings share the already-logged bot handler; no admin platform-link-recovery endpoint exists — both left alone.) - Pruning: `audit_log_retention_years` (default 2, clamp 1–2) + monthly `prune_audit_log` Beat task (idempotent, not self-audited). **Tests:** backend 541 pass (+`test_audit_log.py`); frontend 276 (+13). The three sibling v3.8.0 issues (#117/#118/#119) now have the campaign-scoped `log_event` to write `user.deleted` / `member.recordings_erased` / `retention.enforced` into. Closing.

claude-bot referenced this issue

2026-07-18 07:27:59 +00:00

feat: configurable audio & transcript retention (#119) #201

claude-bot referenced this issue

2026-07-18 08:04:20 +00:00

feat: per-player erasure from transcripts & recordings (#118) #202

rbrooks referenced this issue from a commit

2026-07-18 09:02:57 +00:00

chore(release): v3.8.0

claude-bot referenced this issue

2026-07-18 09:03:07 +00:00

chore(release): v3.8.0 — Privacy & Data Lifecycle #204