Emails silently skipped after the first one in each worker process (the pooled engine on a dead loop, #129's gap) #814

New issue

Closed

opened 2026-09-24 18:46:05 +00:00 by claude-bot · 0 comments

claude-bot commented

2026-09-24 18:46:05 +00:00

Collaborator

Found by J2's new no-Discord leg (lane 4, #730) on 2026-09-24. Fixed in the lane 4 journeys PR.

What happened

EmailNotificationBackend._load_config (app/notifications/email.py) read the SMTP settings through the pooled AsyncSessionLocal, on a temporary event loop.

The first call in a Celery worker process worked.
Every later call in that process failed with Future … attached to a different loop. That was logged only as a warning ("Failed to load SMTP config") and treated as "no SMTP configured", so the send was silently skipped.

What it hit

Every email sent from a worker:

session confirmations;
reminders;
calendar invites;
notification emails;
test emails.

For a group without Discord, email is the only channel, so whether they heard anything came down to which prefork worker took the task. On a long-lived worker, most of them didn't. It would have looked like flaky SMTP.

Why #129 didn't catch it

#129 fixed this exact class of bug in app/tasks/, and its guard test (test_task_session_hygiene.py) scans only app/tasks/. The email backend lives in app/notifications/, but is called from task code.

The fix

_load_config reads through task_session(), a NullPool engine per call, as the #129 note in reminder_tasks.py requires.
A regression test (test_email_backend_config.py) makes two calls in one process with the pooled engine patched to raise. It fails on the old code.
The guard now covers all of app/. Only database.py, main.py (the web process's startup) and cli.py (one-shot commands) may use the pooled engine; each lives on one event loop for its whole life. A sweep found no other offender.

🤖 Generated with Claude Code

**Found by J2's new no-Discord leg (lane 4, #730) on 2026-09-24. Fixed in the lane 4 journeys PR.** ## What happened `EmailNotificationBackend._load_config` (`app/notifications/email.py`) read the SMTP settings through the pooled `AsyncSessionLocal`, on a temporary event loop. - **The first call in a Celery worker process worked.** - **Every later call in that process** failed with `Future … attached to a different loop`. That was logged only as a warning ("Failed to load SMTP config") and treated as **"no SMTP configured"**, so **the send was silently skipped**. ## What it hit Every email sent from a worker: - session confirmations; - reminders; - calendar invites; - notification emails; - test emails. For a group without Discord, email is the only channel, so whether they heard anything came down to which prefork worker took the task. On a long-lived worker, most of them didn't. It would have looked like flaky SMTP. ## Why #129 didn't catch it #129 fixed this exact class of bug in `app/tasks/`, and its guard test (`test_task_session_hygiene.py`) scans only `app/tasks/`. The email backend lives in `app/notifications/`, but is called from task code. ## The fix - `_load_config` reads through `task_session()`, a NullPool engine per call, as the #129 note in `reminder_tasks.py` requires. - A regression test (`test_email_backend_config.py`) makes two calls in one process with the pooled engine patched to raise. It fails on the old code. - The guard now covers **all of `app/`**. Only `database.py`, `main.py` (the web process's startup) and `cli.py` (one-shot commands) may use the pooled engine; each lives on one event loop for its whole life. A sweep found no other offender. 🤖 Generated with [Claude Code](https://claude.com/claude-code)