Routes answer 200 before their commit: get_db commits after the response is sent #851

Closed
opened 2026-09-26 18:16:23 +00:00 by claude-bot · 1 comment
Collaborator

Found by lane 6's J3/J2 runs (#732, on build/lane6-j3-seed) and confirmed from a trace.

The bug. get_db commits when the request's dependencies close, and FastAPI 0.141 closes them after the response has been sent. Every route that leaves its commit to get_db therefore answers before its write is committed. There are about 400 Depends(get_db) uses.

Consequences:

  1. A client's next request can miss the write. The J2 vote grid pressed the same cell twice, 100 ms apart. The second press found no vote, inserted one and hit uq_votes_slot_user. Main's #847 made that particular insert race-safe, but the ordering problem remains. It may also explain #845, the stale status once after a start.
  2. A commit that fails after the 200 is lost silently. The client was told it succeeded.

Also: vote_service.notify_vote_cast doesn't catch a failed enqueue. The broker is set to fail fast because request-path enqueues are meant to be non-fatal, so a broker hiccup can fail the vote. lane 6's branch fixes this for votes only.

Fix:

  • Make commits happen before the response for every non-streaming route.
  • Check the streaming endpoints and any route that relies on post-response commit.
  • Add a regression test that asserts commit-before-response generally.
  • Make vote-path enqueues best-effort.

Refs #845, #844, #732.

Found by lane 6's J3/J2 runs (#732, on `build/lane6-j3-seed`) and confirmed from a trace. **The bug.** `get_db` commits when the request's dependencies close, and FastAPI 0.141 closes them **after the response has been sent**. Every route that leaves its commit to `get_db` therefore answers before its write is committed. There are about 400 `Depends(get_db)` uses. **Consequences:** 1. **A client's next request can miss the write.** The J2 vote grid pressed the same cell twice, 100 ms apart. The second press found no vote, inserted one and hit `uq_votes_slot_user`. Main's #847 made that particular insert race-safe, but the ordering problem remains. It may also explain #845, the stale status once after a start. 2. **A commit that fails after the 200 is lost silently.** The client was told it succeeded. **Also:** `vote_service.notify_vote_cast` doesn't catch a failed enqueue. The broker is set to fail fast because request-path enqueues are meant to be non-fatal, so a broker hiccup can fail the vote. lane 6's branch fixes this for votes only. **Fix:** - Make commits happen before the response for every non-streaming route. - Check the streaming endpoints and any route that relies on post-response commit. - Add a regression test that asserts commit-before-response generally. - Make vote-path enqueues best-effort. Refs #845, #844, #732.
Author
Collaborator

Fixed by #854, merged 2026-09-26.

The fix:

  • All 377 Depends(get_db) are now scope="function", so every route commits before the response is sent.
  • A commit that fails now returns 500 instead of being lost.
  • Votes commit before notifying, and their enqueue is best-effort.

The guard: tests/test_commit_before_response.py walks every route and fails if any get_db isn't function-scoped. It also probes vote PUT and DELETE, and a failing commit, from a second connection.

Corroborating evidence: the J5 phone failures seen on main and on #853 had the same shape. A tap's new state was overwritten by the next poll, which read stale data. J5 passed in #854's own CI run.

Fixed by #854, merged 2026-09-26. **The fix:** - All 377 `Depends(get_db)` are now `scope="function"`, so every route commits before the response is sent. - A commit that fails now returns 500 instead of being lost. - Votes commit before notifying, and their enqueue is best-effort. **The guard:** `tests/test_commit_before_response.py` walks every route and fails if any `get_db` isn't function-scoped. It also probes vote PUT and DELETE, and a failing commit, from a second connection. **Corroborating evidence:** the J5 phone failures seen on main and on #853 had the same shape. A tap's new state was overwritten by the next poll, which read stale data. J5 passed in #854's own CI run.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#851
No description provided.