rbrooks/Circa

Watch

Fork 0

Ingest enqueues no jobs, so OCR and AI analysis never run on their own #144

New issue

Closed

opened 2026-08-05 16:37:59 +00:00 by claude-bot · 3 comments

claude-bot commented

2026-08-05 16:37:59 +00:00

Severity: MEDIUM — two implemented pipelines that nothing starts

Found while probing #105.

docs/circa-spec.md §8.3 lists the ingest steps. Two of them are not performed:

Run OCR on the back scan if present

Enqueue AI analysis, near-duplicate detection, and constraint propagation

backend/app/services/ingest.py contains no call to enqueue at all. Both handlers
exist and work — app/workers/handlers/ocr.py (#4) and app/workers/handlers/ai_analysis.py
(#3) are implemented, registered and tested. Nothing starts them.

Today the only way OCR runs is a reviewer pressing the button behind
POST /api/photos/{id}/evidence/ocr-rerun, one photograph at a time. So on a freshly
ingested archive, the back of every print — where a date is most often written down,
which is the entire reason backs are scanned at all — carries no evidence until somebody
manually asks for it on that specific photograph. The signal the scanning of backs exists
to capture is collected only on request.

This matters more after #105 than before it. Bulk ingest makes it realistic to put
thousands of scans in at once, and #105 pairs backs to their fronts automatically — so
the archive will now routinely hold a back scan that has never been read.

Why it was not fixed in #105

It is a separate gap with its own design questions, and #105 was already carrying a
pairing model, a fold, and an ingest ledger. Attaching a back to an existing photograph
in #105 deliberately matches what ingest does today — which is to enqueue nothing — so
that pairing did not quietly change job behaviour as a side effect.

The questions that need answering

Inline or enqueued? Derivatives are generated inline in ingest_photo, deliberately
(#93 — the worker is not guaranteed to be running, and a missing thumbnail is a visible
hole). OCR is a much heavier operation and the same argument does not obviously carry.
Does a bulk import enqueue thousands of jobs at once? #105's folder scan can present
5,000 files in one run. Whether that should produce 5,000 queued OCR jobs, or whether
bulk ingest should stage them differently, is a real decision.
What happens when a back is attached later? #105's attach and fold paths are the
natural place to enqueue OCR for a back that has just arrived, and currently they do
not — matching ingest.
Near-duplicate detection and constraint propagation (the rest of step 9) should be
scoped here or split out.

References

backend/app/services/ingest.py — no enqueue call anywhere
backend/app/workers/handlers/ocr.py (#4), backend/app/workers/handlers/ai_analysis.py (#3)
docs/circa-spec.md §8.3 steps 5 and 9
#105 — bulk ingest and pairing, which makes this more visible

## Severity: MEDIUM — two implemented pipelines that nothing starts Found while probing #105. `docs/circa-spec.md` §8.3 lists the ingest steps. Two of them are not performed: > 5. Run OCR on the back scan if present > 9. Enqueue AI analysis, near-duplicate detection, and constraint propagation **`backend/app/services/ingest.py` contains no call to `enqueue` at all.** Both handlers exist and work — `app/workers/handlers/ocr.py` (#4) and `app/workers/handlers/ai_analysis.py` (#3) are implemented, registered and tested. Nothing starts them. Today the only way OCR runs is a reviewer pressing the button behind `POST /api/photos/{id}/evidence/ocr-rerun`, one photograph at a time. So on a freshly ingested archive, the back of every print — *where a date is most often written down*, which is the entire reason backs are scanned at all — carries no evidence until somebody manually asks for it on that specific photograph. The signal the scanning of backs exists to capture is collected only on request. This matters more after #105 than before it. Bulk ingest makes it realistic to put thousands of scans in at once, and #105 pairs backs to their fronts automatically — so the archive will now routinely hold a back scan that has never been read. ## Why it was not fixed in #105 It is a separate gap with its own design questions, and #105 was already carrying a pairing model, a fold, and an ingest ledger. Attaching a back to an existing photograph in #105 deliberately matches what ingest does today — which is to enqueue nothing — so that pairing did not quietly change job behaviour as a side effect. ## The questions that need answering - **Inline or enqueued?** Derivatives are generated inline in `ingest_photo`, deliberately (#93 — the worker is not guaranteed to be running, and a missing thumbnail is a visible hole). OCR is a much heavier operation and the same argument does not obviously carry. - **Does a bulk import enqueue thousands of jobs at once?** #105's folder scan can present 5,000 files in one run. Whether that should produce 5,000 queued OCR jobs, or whether bulk ingest should stage them differently, is a real decision. - **What happens when a back is attached later?** #105's attach and fold paths are the natural place to enqueue OCR for a back that has just arrived, and currently they do not — matching ingest. - **Near-duplicate detection and constraint propagation** (the rest of step 9) should be scoped here or split out. ## References - `backend/app/services/ingest.py` — no `enqueue` call anywhere - `backend/app/workers/handlers/ocr.py` (#4), `backend/app/workers/handlers/ai_analysis.py` (#3) - `docs/circa-spec.md` §8.3 steps 5 and 9 - #105 — bulk ingest and pairing, which makes this more visible

claude-bot referenced this issue

2026-08-05 17:26:50 +00:00

Bulk ingest, front/back pairing, and album linkage #105

claude-bot commented

2026-08-05 18:38:47 +00:00

Author

Picking this up, after v0.3.0.

Probe confirms the body, and adds one thing that changes the design. app/services/ingest.py still contains no enqueue call of any kind; the only two callers in the whole application are enqueue_ai_rerun and enqueue_ocr_rerun in app/api/routes/jobs.py, both driven by a reviewer pressing a button on one photograph.

The two halves of §8.3 must not be treated the same way, because one of them spends money.

ocr is local. ocr_backend: "auto" uses tesseract when it is installed and refuses cleanly when it is not, so it costs CPU and nothing else.

ai_analysis is a paid API call, and the failure mode under bulk ingest is worse than "expensive":

docs/circa-spec.md §14 estimates ~$30 for a full pass over 10,000 photos, which is ~$0.003 a photograph.
The ceilings are ai_daily_budget_usd: 5.00 and ai_monthly_budget_usd: 30.00, checked before every call.
Exceeding one raises PermanentJobError (app/workers/handlers/ai_analysis.py:102-104). Permanent means it does not retry.

So auto-enqueueing AI at ingest on a 5,000-scan import would spend the daily ceiling after roughly 1,650 photographs and then permanently fail the remaining ~3,350 jobs, each of which would need a manual rerun to recover. Not deferred, not throttled — failed. And that is the good case; with the ceilings raised it would quietly spend the entire monthly budget as a side effect of importing a box of photographs, which is not a thing importing photographs should do.

The ceiling behaviour is right for what it was designed for — a reviewer pressing rerun. It is the wrong shape for an automatic per-ingest trigger, and that is a property of the trigger, not a bug in the ceiling.

So this issue is really two decisions, and only one of them is easy. OCR should run automatically: it is free, it is local, and reading the back of a print is the entire reason backs are scanned. What ingest should do about AI is a real question, and I am putting it to @rbrooks rather than reading §8.3 step 9 literally and spending his money on the strength of a spec line written before the cost model existed.

Also settled by the probe, and worth writing down: OCR should be enqueued rather than run inline. #93's argument for generating derivatives inline was that a missing thumbnail is a visible hole in the grid — a camera emoji where a photograph should be. A missing OCR result is not that; it is simply evidence that has not arrived yet, which the evidence panel already represents honestly. OCR is also far heavier than a thumbnail, and ingest already holds SQLite's write lock through a ~300 ms derivative per file.

Where the enqueue points are, after #105. A back scan can now reach a photograph three ways, and OCR should follow the back rather than the row: an ingest that carries a back, an attach (a back arriving for a front already in the archive), and an orphan back (a back kept because its front has not arrived). A fold needs no enqueue — the back was already there and was queued when it arrived, and re-queuing would read the same handwriting twice.

Picking this up, after v0.3.0. **Probe confirms the body, and adds one thing that changes the design.** `app/services/ingest.py` still contains no `enqueue` call of any kind; the only two callers in the whole application are `enqueue_ai_rerun` and `enqueue_ocr_rerun` in `app/api/routes/jobs.py`, both driven by a reviewer pressing a button on one photograph. **The two halves of §8.3 must not be treated the same way, because one of them spends money.** `ocr` is local. `ocr_backend: "auto"` uses tesseract when it is installed and refuses cleanly when it is not, so it costs CPU and nothing else. `ai_analysis` is a paid API call, and the failure mode under bulk ingest is worse than "expensive": - `docs/circa-spec.md` §14 estimates **~$30 for a full pass over 10,000 photos**, which is ~$0.003 a photograph. - The ceilings are `ai_daily_budget_usd: 5.00` and `ai_monthly_budget_usd: 30.00`, checked *before* every call. - **Exceeding one raises `PermanentJobError`** (`app/workers/handlers/ai_analysis.py:102-104`). Permanent means it does not retry. So auto-enqueueing AI at ingest on a 5,000-scan import would spend the daily ceiling after roughly 1,650 photographs and then **permanently fail the remaining ~3,350 jobs**, each of which would need a manual rerun to recover. Not deferred, not throttled — failed. And that is the *good* case; with the ceilings raised it would quietly spend the entire monthly budget as a side effect of importing a box of photographs, which is not a thing importing photographs should do. The ceiling behaviour is right for what it was designed for — a reviewer pressing rerun. It is the wrong shape for an automatic per-ingest trigger, and that is a property of the trigger, not a bug in the ceiling. **So this issue is really two decisions, and only one of them is easy.** OCR should run automatically: it is free, it is local, and reading the back of a print is the entire reason backs are scanned. What ingest should do about AI is a real question, and I am putting it to @rbrooks rather than reading §8.3 step 9 literally and spending his money on the strength of a spec line written before the cost model existed. Also settled by the probe, and worth writing down: **OCR should be enqueued rather than run inline.** #93's argument for generating derivatives inline was that a missing thumbnail is a visible hole in the grid — a camera emoji where a photograph should be. A missing OCR result is not that; it is simply evidence that has not arrived yet, which the evidence panel already represents honestly. OCR is also far heavier than a thumbnail, and ingest already holds SQLite's write lock through a ~300 ms derivative per file. **Where the enqueue points are, after #105.** A back scan can now reach a photograph three ways, and OCR should follow the back rather than the row: an ingest that carries a back, an *attach* (a back arriving for a front already in the archive), and an *orphan back* (a back kept because its front has not arrived). A **fold** needs no enqueue — the back was already there and was queued when it arrived, and re-queuing would read the same handwriting twice.

rbrooks referenced this issue from a commit

2026-08-05 19:30:00 +00:00

Read the back of a print without being asked to

rbrooks closed this issue

2026-08-05 19:30:00 +00:00

claude-bot commented

2026-08-05 19:30:40 +00:00

Author

Done in 4f07d67.

What landed

OCR follows the back scan, not the photo row. After #105 a back can reach a photograph three ways, and each queues one reading: an ingest carrying its own back, an attach onto a front already here, and an orphan back whose front has not turned up. A fourth falls out of the implementation and is right — an ambiguous back, refused a pairing because two fronts matched, is a row whose only image is a back scan, and the case where the archive is least sure what it holds is a poor one to gather no evidence on.

A fold queues nothing, and that is the case worth naming. The back was already there and was read when it arrived; a second reading would recognise the same handwriting again and append a row saying what the first already says.

That falls out of a guard derived from the variable that decided where the bytes went — key_back is not None and not is_duplicate — rather than a list of outcomes free to fall out of step the next time a path is added. Which is precisely how #105's dry run came to predict folds the real run would not perform, so it seemed worth not doing twice.

AI is not queued, per @rbrooks. Knowing deviation from §8.3 step 9, recorded in the code where someone holding the spec would otherwise "fix" it. The numbers are in my earlier comment; the short version is that the ceiling is right for what it was built for and it was the trigger that was wrong.

The enqueue is one definition rather than two. Attempt numbering, the idempotency key, the in-flight fast path and the race recovery lived inside _enqueue_rerun; ingest needed all of it. Now app/services/jobs.py, called by both.

One line had to change in the move and it is the one that mattered: the IntegrityError recovery is a savepoint, not db.rollback(). On the route that rollback discarded a transaction holding nothing but the failed insert. Inside ingest_photo it would have discarded the photograph, its evidence, its album and the ledger row recording that the file ever arrived — losing a scan because a job could not be queued for it. There is also a db.flush() before the savepoint opens, so whatever the caller has pending lands outside it; that detail was not in my spec and is the better call.

Verified rather than assumed

Run against a real worker, not only in tests, and with the queue drained between batches so the in-flight fast path could not mask the result:

ingest a front and its back → exactly one ocr job, against the front the back attached to, and no ai_analysis job;
worker claims it and runs it;
orphan back alone → one reading; drain; front arrives in a later run → folds, and queues nothing. One photo row, one job.

That last one matters more than it looks. Three of the new tests were vacuous on first write — they passed off enqueue_photo_job's in-flight fast path rather than off the guard, because a second enqueue in the same session adds no row. Rebuilding them around a drained queue surfaced a real behavioural fact rather than only a test one: without the key_back guard, a fold in a later batch would queue a second reading. The fast path only hides that within one session, and §8.2 makes later batches the normal case.

Fifteen mutations, each breaking exactly one behaviour, all caught — including one that adds the ai_analysis enqueue back per §8.3 step 9, which exists to pin the deviation, because a future reader with the spec in hand is exactly who would re-add it.

What I could not verify

Text recognition itself was never exercised live. Neither this machine nor the Linux dev server has tesseract installed, so get_ocr_backend() returns DisabledOcr on both. What I confirmed is the chain up to it — ingest queues, worker claims, handler runs — failing with the correct and actionable message:

Job … failed permanently: No OCR engine found: 'tesseract' is not on PATH.
Install tesseract-ocr on the host running the worker.

The recognition is #4's code and test_ocr_handler.py covers it against a stub. But "tesseract reads a real back scan and the evidence lands" remains unobserved on any machine, and it is worth someone running once on a host that has the package before trusting a bulk import to it.

Not done, deliberately

The rest of §8.3 step 9 — near-duplicate detection and constraint propagation — is untouched. Both are their own work with their own design questions, and neither is blocked by this. Say the word and I will file them; I have not, to avoid manufacturing issues nobody asked for.

Queue depth is the one cost I would flag. A 5,000-back import leaves 5,000 rows the Jobs page paginates through. That is a UI question rather than an ingest one, and it does not need solving before a first real import — but it will be visible on one.

Done in 4f07d67. ## What landed **OCR follows the back scan, not the photo row.** After #105 a back can reach a photograph three ways, and each queues one reading: an ingest carrying its own back, an *attach* onto a front already here, and an *orphan back* whose front has not turned up. A **fourth** falls out of the implementation and is right — an *ambiguous* back, refused a pairing because two fronts matched, is a row whose only image is a back scan, and the case where the archive is least sure what it holds is a poor one to gather no evidence on. A **fold** queues nothing, and that is the case worth naming. The back was already there and was read when it arrived; a second reading would recognise the same handwriting again and append a row saying what the first already says. That falls out of a guard derived from the variable that decided where the bytes went — `key_back is not None and not is_duplicate` — rather than a list of outcomes free to fall out of step the next time a path is added. Which is precisely how #105's dry run came to predict folds the real run would not perform, so it seemed worth not doing twice. **AI is not queued**, per @rbrooks. Knowing deviation from §8.3 step 9, recorded in the code where someone holding the spec would otherwise "fix" it. The numbers are in my earlier comment; the short version is that the ceiling is right for what it was built for and it was the *trigger* that was wrong. **The enqueue is one definition rather than two.** Attempt numbering, the idempotency key, the in-flight fast path and the race recovery lived inside `_enqueue_rerun`; ingest needed all of it. Now `app/services/jobs.py`, called by both. One line had to change in the move and it is the one that mattered: the `IntegrityError` recovery is a **savepoint**, not `db.rollback()`. On the route that rollback discarded a transaction holding nothing but the failed insert. Inside `ingest_photo` it would have discarded the photograph, its evidence, its album and the ledger row recording that the file ever arrived — losing a scan because a job could not be queued for it. There is also a `db.flush()` before the savepoint opens, so whatever the caller has pending lands *outside* it; that detail was not in my spec and is the better call. ## Verified rather than assumed Run against a **real worker**, not only in tests, and with the queue **drained between batches** so the in-flight fast path could not mask the result: - ingest a front and its back → exactly one `ocr` job, against the front the back attached to, and **no `ai_analysis` job**; - worker claims it and runs it; - orphan back alone → one reading; drain; front arrives in a later run → **folds, and queues nothing**. One photo row, one job. That last one matters more than it looks. Three of the new tests were **vacuous on first write** — they passed off `enqueue_photo_job`'s in-flight fast path rather than off the guard, because a second enqueue in the same session adds no row. Rebuilding them around a drained queue surfaced a real behavioural fact rather than only a test one: without the `key_back` guard, a fold *in a later batch* would queue a second reading. The fast path only hides that within one session, and §8.2 makes later batches the normal case. **Fifteen mutations, each breaking exactly one behaviour, all caught** — including one that adds the `ai_analysis` enqueue back per §8.3 step 9, which exists to pin the deviation, because a future reader with the spec in hand is exactly who would re-add it. ## What I could not verify **Text recognition itself was never exercised live.** Neither this machine nor the Linux dev server has tesseract installed, so `get_ocr_backend()` returns `DisabledOcr` on both. What I confirmed is the chain up to it — ingest queues, worker claims, handler runs — failing with the correct and actionable message: ``` Job … failed permanently: No OCR engine found: 'tesseract' is not on PATH. Install tesseract-ocr on the host running the worker. ``` The recognition is #4's code and `test_ocr_handler.py` covers it against a stub. But "tesseract reads a real back scan and the evidence lands" remains unobserved on any machine, and it is worth someone running once on a host that has the package before trusting a bulk import to it. ## Not done, deliberately The rest of §8.3 step 9 — **near-duplicate detection and constraint propagation** — is untouched. Both are their own work with their own design questions, and neither is blocked by this. Say the word and I will file them; I have not, to avoid manufacturing issues nobody asked for. **Queue depth** is the one cost I would flag. A 5,000-back import leaves 5,000 rows the Jobs page paginates through. That is a UI question rather than an ingest one, and it does not need solving before a first real import — but it will be visible on one.

claude-bot referenced this issue

2026-08-05 19:38:39 +00:00

Implement near-duplicate detection via perceptual hashing #16

claude-bot referenced this issue

2026-08-05 19:38:58 +00:00

Implement the group constraint propagation engine #19

claude-bot commented

2026-08-05 19:41:21 +00:00

Author

Correction to my previous comment: the gap is closed. Real text recognition now verified end to end.

I said recognition had never been exercised live because neither machine has tesseract. @rbrooks then noted Circa is not deployed anywhere at all — so there is no host to run it on and never was, which makes "run it once on a machine with the package" a recommendation with nowhere to land. So I built the machine instead: the e2e image is Ubuntu-based and its venv already holds the backend dependencies, so apt-get install tesseract-ocr on top of it gives a complete environment.

The whole chain, with tesseract 5.3.4 actually reading a print:

A front scan and a back scan carrying real rendered text — Easter 1962 and Aunt Mary's garden, the second deliberately included as a line with no date in it.

=== ingest ===
  redbk42_001.jpg              ingested         album redbk42 #1
  redbk42_001_b.jpg            paired           -> redbk42 #1

=== worker ===
  Claimed job 325765d2 (ocr, attempt 1)
  Job 325765d2 succeeded: Read 1 date(s) at 96% confidence: 1962.

=== what the archive now holds ===
  job ocr -> success
  evidence rows: 1
    source=ocr   reliability=low   1962-01-01..1962-12-31
      raw: "Easter 1962\nAunt Mary's garden"

Nobody asked for that. The scan was imported and the handwriting on the reverse became dated evidence on its own, which is the entire point of this issue.

Four things worth noting from it, none of which the test suite would have shown:

The date range is right. Easter 1962 became 1962-01-01..1962-12-31 — year precision, not a guessed Easter date. The parser did not invent a day it could not know.
reliability=low, correctly. Spec §10.1 rates a handwritten date Low because it may simply be wrong — written years later, from memory. Under projections.fold that means this does not promote the photograph to needs_review on its own, which is the intended reading: a machine's guess at someone's handwriting is not a reason to tell a reviewer the photograph is ready to date.
The full text is preserved on the row, including the line with no date in it. A reviewer can see the back said "Aunt Mary's garden" and stop wondering whether OCR ran — the distinction #4's handler docstring argues for, working as described.
One evidence row, not two. The ingest carried no filename date (redbk42_001.jpg), so there was nothing to duplicate — but it confirms the OCR row lands on the same photograph the back attached to rather than a row of its own.

So the only claim I could not stand behind is now observed. What remains genuinely unverified is scale: this was two files, and nothing here has ever ingested a real collection.

**Correction to my previous comment: the gap is closed. Real text recognition now verified end to end.** I said recognition had never been exercised live because neither machine has tesseract. @rbrooks then noted Circa is not deployed anywhere at all — so there is no host to run it on and never was, which makes "run it once on a machine with the package" a recommendation with nowhere to land. So I built the machine instead: the e2e image is Ubuntu-based and its venv already holds the backend dependencies, so `apt-get install tesseract-ocr` on top of it gives a complete environment. **The whole chain, with tesseract 5.3.4 actually reading a print:** A front scan and a back scan carrying real rendered text — `Easter 1962` and `Aunt Mary's garden`, the second deliberately included as a line with no date in it. ``` === ingest === redbk42_001.jpg ingested album redbk42 #1 redbk42_001_b.jpg paired -> redbk42 #1 === worker === Claimed job 325765d2 (ocr, attempt 1) Job 325765d2 succeeded: Read 1 date(s) at 96% confidence: 1962. === what the archive now holds === job ocr -> success evidence rows: 1 source=ocr reliability=low 1962-01-01..1962-12-31 raw: "Easter 1962\nAunt Mary's garden" ``` Nobody asked for that. The scan was imported and the handwriting on the reverse became dated evidence on its own, which is the entire point of this issue. Four things worth noting from it, none of which the test suite would have shown: - **The date range is right.** `Easter 1962` became `1962-01-01..1962-12-31` — year precision, not a guessed Easter date. The parser did not invent a day it could not know. - **`reliability=low`, correctly.** Spec §10.1 rates a handwritten date Low because it may simply be wrong — written years later, from memory. Under `projections.fold` that means this does **not** promote the photograph to `needs_review` on its own, which is the intended reading: a machine's guess at someone's handwriting is not a reason to tell a reviewer the photograph is ready to date. - **The full text is preserved on the row**, including the line with no date in it. A reviewer can see the back said "Aunt Mary's garden" and stop wondering whether OCR ran — the distinction #4's handler docstring argues for, working as described. - **One evidence row, not two.** The ingest carried no filename date (`redbk42_001.jpg`), so there was nothing to duplicate — but it confirms the OCR row lands on the same photograph the back attached to rather than a row of its own. So the only claim I could not stand behind is now observed. What remains genuinely unverified is scale: this was two files, and nothing here has ever ingested a real collection.

claude-bot referenced this issue

2026-08-05 21:58:28 +00:00

No upload UI: the browser cannot put a photograph into the archive #142

rbrooks referenced this issue from a commit

2026-08-06 00:50:43 +00:00

Build Circa into images something can actually deploy

claude-bot referenced this issue

2026-08-06 18:43:42 +00:00

Create the production deployment and packaging story #50