Harden SPC parsing — one malformed feature currently aborts the whole poll cycle #62

Closed
opened 2026-07-18 05:31:26 +00:00 by claude-bot · 1 comment
Contributor

Per-feature parsing (_parse_convective_feature and siblings,
app/services/spc.py:190-242) sits outside the fetch try/except; a
non-numeric dn raises through the unguarded asyncio.gather calls in
poll_spc/poll_spc_dashboard (spc_processor.py:92-111, 695-724) and kills
the entire cycle. Wrap per-feature parses (log + skip) and gather with
return_exceptions=True, handling per-source failures individually so one
bad product/day doesn't take out the rest. Record parse failures via the
existing freshness observability.

Acceptance criteria:

  • Malformed feature is skipped with a log; remaining features process
  • One failing source doesn't stop the other gathered sources
  • Test with a bad dn value

Filed from the 2026-07-17 codebase audit (docs/.internal/report-2026-07-17.md), finding F-23.

Per-feature parsing (`_parse_convective_feature` and siblings, app/services/spc.py:190-242) sits outside the fetch try/except; a non-numeric `dn` raises through the unguarded `asyncio.gather` calls in `poll_spc`/`poll_spc_dashboard` (spc_processor.py:92-111, 695-724) and kills the entire cycle. Wrap per-feature parses (log + skip) and gather with `return_exceptions=True`, handling per-source failures individually so one bad product/day doesn't take out the rest. Record parse failures via the existing freshness observability. **Acceptance criteria:** - [ ] Malformed feature is skipped with a log; remaining features process - [ ] One failing source doesn't stop the other gathered sources - [ ] Test with a bad `dn` value --- _Filed from the 2026-07-17 codebase audit (`docs/.internal/report-2026-07-17.md`), finding F-23._
Author
Contributor

Fixed in #92 (merged to main). Per-feature SPC parsing is now wrapped (log + skip the bad feature) and the poll_spc/poll_spc_dashboard gathers use return_exceptions=True with per-source handling, so one malformed feature (e.g. a non-numeric dn) or one failing source no longer aborts the whole cycle. Test covers a bad dn value skipped while good features still parse.

Fixed in #92 (merged to `main`). Per-feature SPC parsing is now wrapped (log + skip the bad feature) and the `poll_spc`/`poll_spc_dashboard` gathers use `return_exceptions=True` with per-source handling, so one malformed feature (e.g. a non-numeric `dn`) or one failing source no longer aborts the whole cycle. Test covers a bad `dn` value skipped while good features still parse.
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/WeatherBot#62
No description provided.