Stop the SPC day match crossing days, which failed every SPC poll on production (#214) #215
No reviewers
Labels
No labels
area:ai
area:ci-cd
area:notifications
area:observability
area:public-pages
backlog
bug
duplicate
enhancement
help wanted
invalid
question
security
type:decision
type:feature
type:infra
type:maintenance
type:security
v1.0.1
v1.1.0
v1.2.0
v1.3.0
v2.0.0
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
rbrooks/WeatherBot!215
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/spc-day-match"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Fixes #214. Production hotfix for v2.0.0.
What broke
Since the v2.0.0 upgrade at 05:50Z, production's
poll_spc_jobhas raised on every run (UniqueViolationonuq_sent_alerts_dashboard_dedup), so SPC convective outlook notifications are not being sent. NWS warning alerts are unaffected.Why
_upgrade_checkselects a location's active SPC dashboard rows for one outlook day withnws_alert_id.contains(f"_{day}_"), an unescapedLIKE. InLIKE,_matches any character. Product ids are timestamps, so on production's real ids the Day 2 pattern also matches the Day 1 and Day 3 rows. A same-risk reissue then renamed the wrong day's row onto an id another row already held, and the flush failed the whole poll.The bug predates 2.0.0: production's rows show ids that moved between days. Why it fails on every cycle only since 2.0.0 is my inference: the query had no
ORDER BY, and 2.0.0's new indexes likely changed which row comes first.Change
contains(day_suffix, autoescape=True)in_upgrade_checkand inapp/api/alerts.py's per-day SPC history query, which had the same pattern._upgrade_checkskips rows whose parsed day differs from the outlook's day, and orders rows bysent_at.Verification
IntegrityErrorproduction logs; with the fix it passes, each row stays on its own day and nothing re-notifies.Not done here
Rows already moved between days on production are not repaired. Each day-id still maps to one active row, so continuations should settle as new products arrive; that gets checked on production after the hotfix.
🤖 Generated with Claude Code
After the 2.0.0 upgrade, poll_spc_job raised on every run with a UniqueViolation on uq_sent_alerts_dashboard_dedup, so SPC outlook notifications stopped. NWS warnings were unaffected. _upgrade_check found a location's active SPC rows for one outlook day with nws_alert_id.contains("_2_"), an unescaped LIKE. An underscore matches any character, and product ids are timestamps, so the Day 2 pattern also matched Day 1 and Day 3 ids. The first same-risk row was then "continued" by renaming it to the new product's id; when it belonged to another day and that day's row already held the id, the unique index rejected the flush and the whole poll failed. The match predates 2.0.0 (production rows show ids moved between days); the query had no ORDER BY, and 2.0.0's new indexes plausibly changed which row came first. - contains(..., autoescape=True), here and in the dashboard's per-day SPC history query, which had the same pattern. - _upgrade_check skips any row whose parsed day differs from the outlook's, and orders rows by sent_at so the choice is deterministic. The regression test seeds production's exact id shapes (one MRGL row per day, timestamp product ids) and reproduces the production IntegrityError without the fix. Existing tests used ids like "D1_P1", which never put a digit beside an underscore. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>