Capture radar snapshots for every alert, with an independent retention window

claude-bot commented

2026-07-27 20:06:41 +00:00

Contributor

Parent: #22.

ALERT_RADAR_SNAPSHOT_ENABLED defaults to False (app/config.py:180), so the radar-at-alert-time images built in #85 — explicitly to front-load Explorer history — are not captured on a default install.

Two problems beyond the default:

Retention is transitive, not owned. _cleanup_alert_radar_snapshots (app/services/retention.py:116-145) only deletes orphaned PNGs — files whose name no longer matches a live sent_alerts.radar_snapshot_path. Image lifetime is therefore whatever sent_alerts happens to be, and since sent_alerts prunes on cleared_at, a row that never gets cleared keeps its image forever.
One image per lifecycle. _radar_snapshot_filename (alert_processor.py:724-733) is deterministic ({lifecycle_id}__{location_id}.png), so a re-fire overwrites the previous capture. Fine for "conditions when it first fired"; insufficient for a loop.

Tasks

Default ALERT_RADAR_SNAPSHOT_ENABLED to true.
Give snapshots their own retention window derived from the unified horizon, not sent_alerts.cleared_at.
Decide capture cadence: first-fire only (today) vs periodic through the event. A radar loop needs more than one frame — coordinate with the radar-loop issue.
Route writes through the media storage abstraction rather than direct disk paths.
Measure disk growth per alert; post the numbers here.

Parent: #22. `ALERT_RADAR_SNAPSHOT_ENABLED` defaults to **False** (`app/config.py:180`), so the radar-at-alert-time images built in #85 — explicitly to front-load Explorer history — are not captured on a default install. Two problems beyond the default: 1. **Retention is transitive, not owned.** `_cleanup_alert_radar_snapshots` (`app/services/retention.py:116-145`) only deletes *orphaned* PNGs — files whose name no longer matches a live `sent_alerts.radar_snapshot_path`. Image lifetime is therefore whatever `sent_alerts` happens to be, and since `sent_alerts` prunes on `cleared_at`, a row that never gets cleared keeps its image forever. 2. **One image per lifecycle.** `_radar_snapshot_filename` (`alert_processor.py:724-733`) is deterministic (`{lifecycle_id}__{location_id}.png`), so a re-fire overwrites the previous capture. Fine for "conditions when it first fired"; insufficient for a loop. ## Tasks - [ ] Default `ALERT_RADAR_SNAPSHOT_ENABLED` to true. - [ ] Give snapshots their own retention window derived from the unified horizon, not `sent_alerts.cleared_at`. - [ ] Decide capture cadence: first-fire only (today) vs periodic through the event. A radar loop needs more than one frame — coordinate with the radar-loop issue. - [ ] Route writes through the media storage abstraction rather than direct disk paths. - [ ] Measure disk growth per alert; post the numbers here.

claude-bot added this to the v2.0.0 - Historical Weather Event Explorer milestone

2026-07-27 20:06:41 +00:00

claude-bot added the

v2.0.0

type:feature

labels

2026-07-27 20:06:41 +00:00

claude-bot referenced this issue

2026-07-27 20:09:21 +00:00

Historical Weather Event Explorer #22

claude-bot commented

2026-07-27 23:21:36 +00:00

Author

Contributor

Picking this up on branch feat/radar-frames.

Cadence decided: periodic frames with a per-event cap. Rationale is the same one that made #129 urgent — uncaptured history is unrecoverable, and #140 (radar loops) is far enough down the queue that shipping single-frame now would leave every event before it unreconstructable.

Rough sizing that informed the call, at ~120 KB/PNG and ~150 alerts/location/year:

Cadence	Per location/year	10 locations over 13 months
One frame per alert	~18 MB	~200 MB
Frames every 5 min	~165 MB	~1.8 GB

~10x, judged affordable on disk. The cap is the important guardrail: it bounds both disk and outbound load on NOAA's MapServer during an outbreak, when many alerts are active at once.

Implementation shape

New alert_radar_frames table (FK to the dashboard sent_alerts row, captured_at, path), plus migration 0031.
_radar_snapshot_filename currently returns a deterministic name per (lifecycle, location), so re-fires overwrite. Frames need a timestamped filename so they accumulate.
New scheduler job capturing a frame for each active dashboard alert when the newest frame is older than the interval and the event is under its frame cap.
SentAlert.radar_snapshot_path / radar_snapshot_at stay as the representative first frame, so /media/alert-radar/{id}.png and the public alert page keep working unchanged. Frames are purely additive.
New config: ALERT_RADAR_FRAME_INTERVAL_MINUTES, ALERT_RADAR_FRAME_MAX_PER_EVENT, RETENTION_ALERT_RADAR_FRAMES_DAYS; ALERT_RADAR_SNAPSHOT_ENABLED flips to true.
Retention gains an owned window for frames rather than the current transitive orphan-only sweep.

Deferred to #131: routing writes through the media storage abstraction. That issue moves every media call site at once, and this one will be among them — doing it here would mean writing the same code twice.

Picking this up on branch `feat/radar-frames`. **Cadence decided: periodic frames with a per-event cap.** Rationale is the same one that made #129 urgent — uncaptured history is unrecoverable, and #140 (radar loops) is far enough down the queue that shipping single-frame now would leave every event before it unreconstructable. Rough sizing that informed the call, at ~120 KB/PNG and ~150 alerts/location/year: | Cadence | Per location/year | 10 locations over 13 months | |---|---|---| | One frame per alert | ~18 MB | ~200 MB | | Frames every 5 min | ~165 MB | ~1.8 GB | ~10x, judged affordable on disk. The **cap** is the important guardrail: it bounds both disk and outbound load on NOAA's MapServer during an outbreak, when many alerts are active at once. ## Implementation shape - New `alert_radar_frames` table (FK to the dashboard `sent_alerts` row, `captured_at`, `path`), plus migration `0031`. - `_radar_snapshot_filename` currently returns a **deterministic** name per `(lifecycle, location)`, so re-fires overwrite. Frames need a timestamped filename so they accumulate. - New scheduler job capturing a frame for each active dashboard alert when the newest frame is older than the interval and the event is under its frame cap. - `SentAlert.radar_snapshot_path` / `radar_snapshot_at` stay as the representative first frame, so `/media/alert-radar/{id}.png` and the public alert page keep working unchanged. Frames are purely additive. - New config: `ALERT_RADAR_FRAME_INTERVAL_MINUTES`, `ALERT_RADAR_FRAME_MAX_PER_EVENT`, `RETENTION_ALERT_RADAR_FRAMES_DAYS`; `ALERT_RADAR_SNAPSHOT_ENABLED` flips to true. - Retention gains an owned window for frames rather than the current transitive orphan-only sweep. **Deferred to #131:** routing writes through the media storage abstraction. That issue moves every media call site at once, and this one will be among them — doing it here would mean writing the same code twice.

rbrooks referenced this issue from a commit

2026-07-27 23:49:50 +00:00

Capture periodic radar frames for active alerts (#130)

claude-bot referenced this issue from a pull request that will close it,

2026-07-27 23:50:15 +00:00

Capture periodic radar frames for active alerts (#130) #144

claude-bot closed this issue

2026-07-28 00:00:35 +00:00

claude-bot referenced this issue from a commit

2026-07-28 00:00:37 +00:00

Merge pull request 'Capture periodic radar frames for active alerts (#130)' (#144) from feat/radar-frames into main

claude-bot commented

2026-07-28 00:00:53 +00:00

Author

Contributor

Done in #144 (merged to main).

Shipped: alert_radar_frames table (migration 0031), a poll_alert_radar_frames scheduler job appending frames to each active dashboard alert on a 5-minute interval with a 24-frame per-event cap, ALERT_RADAR_SNAPSHOT_ENABLED flipped to true, an owned retention window for frames, and GET /media/alert-radar/frames/{frame_id}.png. SentAlert.radar_snapshot_path/_at are unchanged as the representative first frame, so the existing media route and public alert page are unaffected.

Worth knowing for #140: the first frame deliberately shares its filename with radar_snapshot_path — one file on disk, not two. That plus independent retention windows meant each retention sweep could delete a file the other still owned, in both directions. Both are now guarded and regression-tested. Anything that later moves these files (#131's storage abstraction, in particular) must preserve that mutual protection.

Verified locally in Docker against every CI gate: ruff clean, compileall OK, 791 SQLite-tier tests passed, alembic upgrade head clean through 0031, a 0031 downgrade → re-upgrade round trip clean on Postgres 16, 4 Postgres-tier tests passed. CI on the PR was green in 3m8s.

Open follow-ups:

Disk-growth measurement is still open — the ~165 MB/location/year estimate is arithmetic, not observation. Worth measuring alert_radar_frames row count and the on-disk size of {radar_cache_dir}/alert_snapshots after a real storm season, and feeding the number into #133 before the unified horizon is fixed.
The cap interacts with long-lived alerts. At 5-minute intervals a 24-frame cap covers ~2 hours. A multi-day flood warning will stop accruing frames well before it clears, so its loop will cover only the opening window. If that turns out to matter, the fix is probably a longer interval for low-severity long-duration events rather than a bigger cap.
Routing writes through the media storage abstraction is deferred to #131, as noted above.

Done in #144 (merged to `main`). **Shipped:** `alert_radar_frames` table (migration `0031`), a `poll_alert_radar_frames` scheduler job appending frames to each active dashboard alert on a 5-minute interval with a 24-frame per-event cap, `ALERT_RADAR_SNAPSHOT_ENABLED` flipped to true, an owned retention window for frames, and `GET /media/alert-radar/frames/{frame_id}.png`. `SentAlert.radar_snapshot_path`/`_at` are unchanged as the representative first frame, so the existing media route and public alert page are unaffected. **Worth knowing for #140:** the first frame deliberately shares its filename with `radar_snapshot_path` — one file on disk, not two. That plus independent retention windows meant each retention sweep could delete a file the other still owned, in both directions. Both are now guarded and regression-tested. Anything that later moves these files (#131's storage abstraction, in particular) must preserve that mutual protection. **Verified locally in Docker against every CI gate:** ruff clean, `compileall` OK, 791 SQLite-tier tests passed, `alembic upgrade head` clean through `0031`, a `0031` downgrade → re-upgrade round trip clean on Postgres 16, 4 Postgres-tier tests passed. CI on the PR was green in 3m8s. **Open follow-ups:** 1. **Disk-growth measurement is still open** — the ~165 MB/location/year estimate is arithmetic, not observation. Worth measuring `alert_radar_frames` row count and the on-disk size of `{radar_cache_dir}/alert_snapshots` after a real storm season, and feeding the number into #133 before the unified horizon is fixed. 2. **The cap interacts with long-lived alerts.** At 5-minute intervals a 24-frame cap covers ~2 hours. A multi-day flood warning will stop accruing frames well before it clears, so its loop will cover only the opening window. If that turns out to matter, the fix is probably a longer interval for low-severity long-duration events rather than a bigger cap. 3. Routing writes through the media storage abstraction is deferred to #131, as noted above.

claude-bot referenced this issue

2026-07-28 03:40:47 +00:00

CD deploys only to the dev host; the live instance is 6 days behind and manually managed #154

claude-bot referenced this issue

2026-07-28 04:54:17 +00:00

Media storage abstraction (disk backend now, object storage later) #131

rbrooks referenced this issue from a commit

2026-07-28 05:26:36 +00:00

Add a media storage abstraction (#131)

claude-bot referenced this issue

2026-07-28 05:27:02 +00:00

Add a media storage abstraction (#131) #155

claude-bot referenced this issue

2026-07-28 05:36:41 +00:00

Media storage abstraction (disk backend now, object storage later) #131

claude-bot referenced this issue

2026-07-28 14:26:25 +00:00

Lightning cluster archive #132

claude-bot referenced this issue

2026-07-28 14:34:37 +00:00

Unified 13-month history horizon across all history tables #133

claude-bot commented

2026-07-28 14:34:58 +00:00

Author

Contributor

Measured — the ~165 MB/location/year estimate is high by roughly 1.5–2.3×

Full measurement set is on #133 (comment); the radar-frame part, since this issue asked for it:

Measured in prod at 2026-07-28 14:30 UTC:

Input	Measured	Source
Frame PNG size	~7.1 kB	216 kB across 31 files in `radar_cache/alert_snapshots`
Alert events	1.82/day/location ≈ 666/year	839 dashboard records over 92 days, 5 locations
Frames per event	15.5 avg, cap 24	`alert_radar_frames`, n=2 events

At the measured average: 666 × 15.5 × 7.1 kB ≈ 73 MB/location/year
At the alert_radar_frame_max_per_event = 24 ceiling: ≈ 113 MB/location/year

Both land below the 165 MB/location/year estimated here, so the original figure was conservative in the right direction — but it is worth having the real number before #133 locks a horizon that multiplies it by 13 months.

Caveat, stated plainly: the frames-per-event average rests on n=2 events, because capture only ungated at 03:50 UTC today and the DB holds 31 frames total. The event rate is solid (839 records over 92 days); frames-per-event is not. Worth re-measuring after a real convective season before treating 73 MB as settled.

Also measured: alert_radar_frames costs 96 kB of database for 31 rows, and that is 80 kB index against 16 kB heap — at this row count it is all fixed index-page overhead. The PNG bytes on disk dominate the DB rows by orders of magnitude, so disk is the number that matters for this feature, not table size.

## Measured — the ~165 MB/location/year estimate is high by roughly 1.5–2.3× Full measurement set is on #133 ([comment](https://git.rhoving.com/rbrooks/WeatherBot/issues/133#issuecomment-18937)); the radar-frame part, since this issue asked for it: Measured in prod at 2026-07-28 14:30 UTC: | Input | Measured | Source | |---|---|---| | Frame PNG size | **~7.1 kB** | 216 kB across 31 files in `radar_cache/alert_snapshots` | | Alert events | **1.82/day/location ≈ 666/year** | 839 dashboard records over 92 days, 5 locations | | Frames per event | **15.5 avg**, cap 24 | `alert_radar_frames`, n=2 events | - At the measured average: 666 × 15.5 × 7.1 kB ≈ **73 MB/location/year** - At the `alert_radar_frame_max_per_event = 24` ceiling: ≈ **113 MB/location/year** Both land below the 165 MB/location/year estimated here, so the original figure was conservative in the right direction — but it is worth having the real number before #133 locks a horizon that multiplies it by 13 months. **Caveat, stated plainly:** the frames-per-event average rests on **n=2 events**, because capture only ungated at 03:50 UTC today and the DB holds 31 frames total. The event *rate* is solid (839 records over 92 days); frames-per-event is not. Worth re-measuring after a real convective season before treating 73 MB as settled. Also measured: `alert_radar_frames` costs 96 kB of database for 31 rows, and that is 80 kB index against 16 kB heap — at this row count it is all fixed index-page overhead. The PNG bytes on disk dominate the DB rows by orders of magnitude, so disk is the number that matters for this feature, not table size.

claude-bot referenced this issue

2026-07-28 15:38:48 +00:00

Unify retention onto one 13-month horizon (#133) #157

claude-bot referenced this issue

2026-07-28 15:39:34 +00:00

Measure capture-table storage after a convective season and revisit the retention windows #158

claude-bot referenced this issue

2026-08-01 05:20:38 +00:00

Explorer: derived radar loop windows for event replay #140

rbrooks referenced this issue from a commit

2026-08-01 05:31:55 +00:00

Add derived radar loop windows for event replay (#140)