SQLite configuration is not ready for a concurrent worker #87

Closed
opened 2026-07-28 06:00:30 +00:00 by claude-bot · 1 comment

Severity: MEDIUM (blocking for #2)

The problem

backend/app/db/session.py:13-27 sets only journal_mode=WAL and foreign_keys=ON. Probed
against the installed stack: pool is QueuePool, busy timeout is pysqlite's implicit 5000 ms, and
synchronous=FULL.

Three gaps once app/workers/ becomes a real process writing concurrently with the API:

  1. synchronous=FULL fsyncs the WAL on every commit. Every decision writes a decision row, a
    projection update, and an audit row; each ingest commits individually. NORMAL is the standard
    WAL pairing and loses no durability except on power failure.
  2. SQLAlchemy issues a deferred BEGIN. A read-then-write transaction — exactly the shape of
    _get_photo_or_404_conflict_check → write — that races a worker commit fails with
    SQLITE_BUSY_SNAPSHOT immediately. The busy handler cannot rescue a lock upgrade.
  3. The 5 s busy timeout is implicit, not chosen. A worker holding a write transaction longer
    than 5 s — say an AI handler that writes early then calls a slow API inside the transaction —
    turns reviewer saves into 500s.

Fix

  • Add PRAGMA synchronous=NORMAL and an explicit PRAGMA busy_timeout to the connect listener.
  • Use BEGIN IMMEDIATE for write transactions (via event.listens_for(engine, "begin")), at
    minimum in the worker's claim loop and the photo write paths.
  • Document that job handlers must never hold a write transaction across a network call.

Done when

  • Pragmas are set explicitly and asserted by a test
  • Write transactions begin immediately rather than deferring
  • A concurrent worker-plus-reviewer test produces no database is locked errors
  • The handler transaction rule is documented for #3 and #4

References

  • backend/app/db/session.py:13-27

Blocks: #2 (worker runtime). Related: #48.

## Severity: MEDIUM (blocking for #2) ## The problem `backend/app/db/session.py:13-27` sets only `journal_mode=WAL` and `foreign_keys=ON`. Probed against the installed stack: pool is `QueuePool`, busy timeout is pysqlite's implicit 5000 ms, and `synchronous=FULL`. Three gaps once `app/workers/` becomes a real process writing concurrently with the API: 1. **`synchronous=FULL` fsyncs the WAL on every commit.** Every decision writes a decision row, a projection update, and an audit row; each ingest commits individually. `NORMAL` is the standard WAL pairing and loses no durability except on power failure. 2. **SQLAlchemy issues a deferred `BEGIN`.** A read-then-write transaction — exactly the shape of `_get_photo_or_404` → `_conflict_check` → write — that races a worker commit fails with `SQLITE_BUSY_SNAPSHOT` **immediately**. The busy handler cannot rescue a lock upgrade. 3. **The 5 s busy timeout is implicit, not chosen.** A worker holding a write transaction longer than 5 s — say an AI handler that writes early then calls a slow API inside the transaction — turns reviewer saves into 500s. ## Fix - Add `PRAGMA synchronous=NORMAL` and an explicit `PRAGMA busy_timeout` to the connect listener. - Use `BEGIN IMMEDIATE` for write transactions (via `event.listens_for(engine, "begin")`), at minimum in the worker's claim loop and the photo write paths. - Document that job handlers must never hold a write transaction across a network call. ## Done when - [ ] Pragmas are set explicitly and asserted by a test - [ ] Write transactions begin immediately rather than deferring - [ ] A concurrent worker-plus-reviewer test produces no `database is locked` errors - [ ] The handler transaction rule is documented for #3 and #4 ## References - `backend/app/db/session.py:13-27` Blocks: #2 (worker runtime). Related: #48.
claude-bot added this to the v0.2.0 milestone 2026-07-28 06:00:30 +00:00
Author

Done in 4db9ad9.

One correction to the diagnosis. The issue says a read-then-write transaction racing a worker commit fails immediately with SQLITE_BUSY_SNAPSHOT. Probed against the installed stack, that is not what happens today — because the read is not in a transaction at all. pysqlite emits BEGIN before INSERT/UPDATE/DELETE and never before a SELECT, so after a SELECT through a Session the driver reports in_transaction = False. The actual defect was worse in a different way: two reads in one request could see two different databases, and the read behind a check-then-write was not covered by the write that followed. (review_version survived that only because #77 had already moved its check into the UPDATE itself.)

Fixing that is what makes the issue's diagnosis true. isolation_level = None plus our own BEGIN makes reads transactional — and then a deferred read-then-write does fail instantly with a stale snapshot, which no busy timeout can retry. So the two halves had to land together; doing only the pragmas would have been a no-op, and doing only isolation_level would have introduced the failure the issue describes. TestDeferredWritesAreTheHazard produces that failure directly, so the reason is an executable fact rather than a comment.

What landed

  • configure_sqlite() in app/db/session.py sets journal_mode=WAL, foreign_keys=ON, synchronous=NORMAL, an explicit busy_timeout, and installs the begin listener. Shared by the app engine, the test harness and any tooling, so tests cannot pass against transaction behaviour production does not have.
  • Write transactions BEGIN IMMEDIATE. Intent is decided by HTTP method in get_db, so a route cannot forget it; reads stay deferred, which in WAL neither waits for the write lock nor holds it. Session renewal is the only write in the codebase on a GET and asks explicitly via begin_write(). Workers and CLI use write_session().
  • Outside production, DML on a deferred transaction raises DeferredWriteError at the first statement instead of becoming a race that only appears under load. CIRCA_SQLITE_STRICT_WRITE_INTENT overrides.
  • Config: CIRCA_SQLITE_SYNCHRONOUS (a Literal, so a typo is a startup error rather than a string interpolated into a PRAGMA), CIRCA_SQLITE_BUSY_TIMEOUT_MS (10 s).

Done when

  • Pragmas set explicitly and asserted by a test
  • Write transactions begin immediately rather than deferring
  • A concurrent worker-plus-reviewer test produces no database is locked errors — three threads × 40 read-modify-writes, zero errors, and all 120 increments land, so nothing was lost to a race or to a retry papering over one
  • The handler transaction rule documented for #3 and #4 — in the app/db/session.py module docstring and the README: never hold a write transaction across a network call; do the slow work first, then open a short transaction to record the result

Measured, not assumed: a writer blocked by a 1.0 s holder waits 1.06 s and then succeeds, where the deferred form fails instantly. tests/test_sqlite_concurrency.py, 24 tests. Full suite 515 passed.

Two knock-on effects worth recording for #2.

  1. A write-intent transaction holds SQLite's write lock for its whole life, including the parts of a request that are not writing. For POST /api/ingest that spans the sandboxed image parse. It is bounded and the busy timeout covers it, but it is the reason the handler rule above is a rule and not a suggestion — and #86 (ingest blocks the event loop) is now also a lock-holding issue, not only a latency one.
  2. TestGenuineRace in test_review_version.py could no longer stage two overlapping open write transactions, because there is no longer any such thing. It now stages the shape that does still occur — an object outliving the transaction that loaded it, which is every ORM object handed to a route — and still proves the mapper-level guard from #77.
Done in 4db9ad9. **One correction to the diagnosis.** The issue says a read-then-write transaction racing a worker commit fails immediately with `SQLITE_BUSY_SNAPSHOT`. Probed against the installed stack, that is not what happens today — because the read is not in a transaction at all. pysqlite emits `BEGIN` before `INSERT`/`UPDATE`/`DELETE` and never before a `SELECT`, so after a `SELECT` through a `Session` the driver reports `in_transaction = False`. The actual defect was worse in a different way: two reads in one request could see two different databases, and the read behind a check-then-write was not covered by the write that followed. (`review_version` survived that only because #77 had already moved its check into the `UPDATE` itself.) Fixing that is what makes the issue's diagnosis true. `isolation_level = None` plus our own `BEGIN` makes reads transactional — and *then* a deferred read-then-write does fail instantly with a stale snapshot, which no busy timeout can retry. So the two halves had to land together; doing only the pragmas would have been a no-op, and doing only `isolation_level` would have introduced the failure the issue describes. `TestDeferredWritesAreTheHazard` produces that failure directly, so the reason is an executable fact rather than a comment. **What landed** - `configure_sqlite()` in `app/db/session.py` sets `journal_mode=WAL`, `foreign_keys=ON`, `synchronous=NORMAL`, an explicit `busy_timeout`, and installs the `begin` listener. Shared by the app engine, the test harness and any tooling, so tests cannot pass against transaction behaviour production does not have. - Write transactions `BEGIN IMMEDIATE`. Intent is decided by **HTTP method** in `get_db`, so a route cannot forget it; reads stay deferred, which in WAL neither waits for the write lock nor holds it. Session renewal is the only write in the codebase on a `GET` and asks explicitly via `begin_write()`. Workers and CLI use `write_session()`. - Outside production, DML on a deferred transaction raises `DeferredWriteError` at the first statement instead of becoming a race that only appears under load. `CIRCA_SQLITE_STRICT_WRITE_INTENT` overrides. - Config: `CIRCA_SQLITE_SYNCHRONOUS` (a `Literal`, so a typo is a startup error rather than a string interpolated into a PRAGMA), `CIRCA_SQLITE_BUSY_TIMEOUT_MS` (10 s). **Done when** - [x] Pragmas set explicitly and asserted by a test - [x] Write transactions begin immediately rather than deferring - [x] A concurrent worker-plus-reviewer test produces no `database is locked` errors — three threads × 40 read-modify-writes, zero errors, and all 120 increments land, so nothing was lost to a race *or* to a retry papering over one - [x] The handler transaction rule documented for #3 and #4 — in the `app/db/session.py` module docstring and the README: never hold a write transaction across a network call; do the slow work first, then open a short transaction to record the result **Measured**, not assumed: a writer blocked by a 1.0 s holder waits 1.06 s and then succeeds, where the deferred form fails instantly. `tests/test_sqlite_concurrency.py`, 24 tests. Full suite 515 passed. **Two knock-on effects worth recording for #2.** 1. A write-intent transaction holds SQLite's write lock for its whole life, including the parts of a request that are not writing. For `POST /api/ingest` that spans the sandboxed image parse. It is bounded and the busy timeout covers it, but it is the reason the handler rule above is a rule and not a suggestion — and #86 (ingest blocks the event loop) is now also a lock-holding issue, not only a latency one. 2. `TestGenuineRace` in `test_review_version.py` could no longer stage two overlapping open write transactions, because there is no longer any such thing. It now stages the shape that does still occur — an object outliving the transaction that loaded it, which is every ORM object handed to a route — and still proves the mapper-level guard from #77.
Sign in to join this conversation.
No description provided.