Implement the OCR job handler for back-of-photo date extraction #4

Closed
opened 2026-07-28 04:52:48 +00:00 by claude-bot · 3 comments

Context

POST /api/photos/{photo_id}/evidence/ocr-rerun enqueues a JobType.ocr job and
EvidenceSource.ocr exists, but nothing performs OCR. Backs of photos frequently
carry handwritten or stamped dates, which is the point of scanning them.

Scope

The ocr job handler: run OCR over the back scan, extract date-like strings,
normalize them into DateEvidence, and keep the raw text for reviewer inspection.

Implementation notes

  • OCR engine choice is open — Tesseract via pytesseract is the obvious default,
    but it is a system dependency that must be documented and installed in CI and on
    the dev server. Note the decision in the issue before implementing.
  • Store the full recognized text in raw_value even when no date is parsed; the
    reviewer panel should be able to show what the scanner actually read.
  • Date parsing must handle the messy real cases: MAR 87, 3/87, 1987, MARCH 1987,
    processing-lab date stamps, and ambiguous 3/4/87 day/month order. Ambiguity should
    widen the range rather than guess — emit date_low / date_high with the correct
    DatePrecision.
  • reliability should be low by default. The spec is explicit that handwritten dates
    may be wrong, so OCR evidence must not outrank EXIF or a reviewer decision.
  • Append-only, same as AI reruns: a second OCR run adds a row, never replaces one.
  • Photos with no back scan should fail fast with a clear reason.

Done when

  • An OCR rerun on a photo with a back scan produces evidence with raw text preserved
  • Recognized-but-unparseable text still yields an evidence row with raw_value set
  • Ambiguous dates produce a widened range rather than a guessed exact date
  • The OCR system dependency is documented in the README and installed in CI

References

  • backend/app/api/routes/jobs.py
  • backend/app/models/models.py (EvidenceSource.ocr, JobType.ocr)
  • docs/circa-spec.md (back-of-photo OCR, handwritten date trust)

Depends on: the background worker runtime.

## Context `POST /api/photos/{photo_id}/evidence/ocr-rerun` enqueues a `JobType.ocr` job and `EvidenceSource.ocr` exists, but nothing performs OCR. Backs of photos frequently carry handwritten or stamped dates, which is the point of scanning them. ## Scope The `ocr` job handler: run OCR over the back scan, extract date-like strings, normalize them into `DateEvidence`, and keep the raw text for reviewer inspection. ## Implementation notes - OCR engine choice is open — Tesseract via `pytesseract` is the obvious default, but it is a system dependency that must be documented and installed in CI and on the dev server. Note the decision in the issue before implementing. - Store the full recognized text in `raw_value` even when no date is parsed; the reviewer panel should be able to show what the scanner actually read. - Date parsing must handle the messy real cases: `MAR 87`, `3/87`, `1987`, `MARCH 1987`, processing-lab date stamps, and ambiguous `3/4/87` day/month order. Ambiguity should widen the range rather than guess — emit `date_low` / `date_high` with the correct `DatePrecision`. - `reliability` should be low by default. The spec is explicit that handwritten dates may be wrong, so OCR evidence must not outrank EXIF or a reviewer decision. - Append-only, same as AI reruns: a second OCR run adds a row, never replaces one. - Photos with no back scan should fail fast with a clear reason. ## Done when - [ ] An OCR rerun on a photo with a back scan produces evidence with raw text preserved - [ ] Recognized-but-unparseable text still yields an evidence row with `raw_value` set - [ ] Ambiguous dates produce a widened range rather than a guessed exact date - [ ] The OCR system dependency is documented in the README and installed in CI ## References - `backend/app/api/routes/jobs.py` - `backend/app/models/models.py` (`EvidenceSource.ocr`, `JobType.ocr`) - `docs/circa-spec.md` (back-of-photo OCR, handwritten date trust) Depends on: the background worker runtime.
claude-bot added this to the v0.2.0 milestone 2026-07-28 04:52:48 +00:00
Author

Amended by the audit of 2026-07-28.

This issue covers OCR of the back only. The orange film datestamps printed on the front of
late-80s and 90s prints are ubiquitous and rated Medium-High by the spec, and have no extraction
path anywhere — now tracked in #112 alongside the format-rule knowledge base.

Also applies here: auto quality-flagging on low OCR confidence (#119), and running image work
inside the worker under resource limits (#65).

**Amended by the audit of 2026-07-28.** This issue covers OCR of the **back** only. The orange film datestamps printed on the **front** of late-80s and 90s prints are ubiquitous and rated Medium-High by the spec, and have no extraction path anywhere — now tracked in #112 alongside the format-rule knowledge base. Also applies here: auto quality-flagging on low OCR confidence (#119), and running image work inside the worker under resource limits (#65).
Author

Picking this up now that #2 (worker runtime) and #87 (SQLite concurrency) are done. The issue asks for the engine decision to be noted before implementing, so:

Engine: Tesseract, invoked as a subprocess rather than through pytesseract.

Considered:

  • Tesseract — mature, offline, free, packaged everywhere (tesseract-ocr on Debian). Good on printed and stamped text, mediocre on block printing, poor on cursive. That limitation is real and is exactly why the spec rates a handwritten date on the back Low (§10.1) and why the raw text is preserved for the reviewer regardless of whether a date parses out of it.
  • EasyOCR / PaddleOCR — better on handwriting, but each pulls in PyTorch. Hundreds of megabytes and a GPU story, for a job that runs a handful of times per photo. Not proportionate.
  • A cloud OCR API — best accuracy on handwriting, and rejected: it would send scans of family photographs to a third party for a signal the spec already says to distrust. §16 and the local-processing requirement in the face-clustering scope note both point the other way.

Why the binary directly rather than pytesseract. pytesseract is a thin wrapper that shells out to the same binary, so it adds a dependency without removing the system one. Calling it ourselves means the timeout, the environment, and the failure modes are ours — and this codebase already has the pattern from #65's image sandbox, including launching with a minimal environment that withholds every CIRCA_* variable, since the process reading attacker-supplied bytes has no need for the signing key. The only thing pytesseract would have saved is TSV parsing for per-word confidence, which is a dozen lines.

It stays behind an OcrBackend interface with a null implementation, so a deployment without tesseract installed fails the job with "OCR is not configured" rather than crashing the worker, and tests get a deterministic fake.

Two things I am deliberately not doing here, for the record.

  1. All back OCR evidence is low, per §10.1 and this issue. Note that the spec's Medium-High row is for a printed/stamped date on the front (#112). Processing-lab stamps also appear on the back, and those are arguably better than Low — but telling a lab stamp from handwriting needs classification we cannot do from recognised text alone. The per-word confidence tesseract reports is a reasonable future proxy, so it is stored on the evidence row; acting on it is #119.
  2. Century for a two-digit year is resolved, not widened. Day/month ambiguity (3/4/87) genuinely is a coin flip and gets widened as the issue asks. A two-digit year is not: prints carrying date stamps are overwhelmingly 20th century, so 87 resolves to 1987 rather than producing a useless hundred-year range. The assumption is written into the evidence notes where the reviewer can see it, and the pivot is configurable.

Will comment again when it lands. Tesseract is not installed on the Windows dev machine, so the against-the-real-engine test will be verified on the Linux dev server, and CI gets the apt package.

Picking this up now that #2 (worker runtime) and #87 (SQLite concurrency) are done. The issue asks for the engine decision to be noted before implementing, so: **Engine: Tesseract, invoked as a subprocess rather than through `pytesseract`.** Considered: - **Tesseract** — mature, offline, free, packaged everywhere (`tesseract-ocr` on Debian). Good on printed and stamped text, mediocre on block printing, poor on cursive. That limitation is real and is exactly why the spec rates a handwritten date on the back **Low** (§10.1) and why the raw text is preserved for the reviewer regardless of whether a date parses out of it. - **EasyOCR / PaddleOCR** — better on handwriting, but each pulls in PyTorch. Hundreds of megabytes and a GPU story, for a job that runs a handful of times per photo. Not proportionate. - **A cloud OCR API** — best accuracy on handwriting, and rejected: it would send scans of family photographs to a third party for a signal the spec already says to distrust. §16 and the local-processing requirement in the face-clustering scope note both point the other way. **Why the binary directly rather than `pytesseract`.** `pytesseract` is a thin wrapper that shells out to the same binary, so it adds a dependency without removing the system one. Calling it ourselves means the timeout, the environment, and the failure modes are ours — and this codebase already has the pattern from #65's image sandbox, including launching with a minimal environment that withholds every `CIRCA_*` variable, since the process reading attacker-supplied bytes has no need for the signing key. The only thing `pytesseract` would have saved is TSV parsing for per-word confidence, which is a dozen lines. It stays behind an `OcrBackend` interface with a null implementation, so a deployment without tesseract installed fails the job with "OCR is not configured" rather than crashing the worker, and tests get a deterministic fake. **Two things I am deliberately not doing here, for the record.** 1. **All back OCR evidence is `low`,** per §10.1 and this issue. Note that the spec's **Medium-High** row is for a printed/stamped date on the *front* (#112). Processing-lab stamps also appear on the *back*, and those are arguably better than Low — but telling a lab stamp from handwriting needs classification we cannot do from recognised text alone. The per-word confidence tesseract reports is a reasonable future proxy, so it is stored on the evidence row; acting on it is #119. 2. **Century for a two-digit year is resolved, not widened.** Day/month ambiguity (`3/4/87`) genuinely is a coin flip and gets widened as the issue asks. A two-digit year is not: prints carrying date stamps are overwhelmingly 20th century, so `87` resolves to 1987 rather than producing a useless hundred-year range. The assumption is written into the evidence `notes` where the reviewer can see it, and the pivot is configurable. Will comment again when it lands. Tesseract is not installed on the Windows dev machine, so the against-the-real-engine test will be verified on the Linux dev server, and CI gets the apt package.
Author

Done in c467a28. CI green.

Done when

  • An OCR rerun on a photo with a back scan produces evidence with raw text preserved
  • Recognised-but-unparseable text still yields an evidence row with raw_value set
  • Ambiguous dates produce a widened range rather than a guessed exact date
  • The OCR system dependency is documented in the README and installed in CI

71 tests across test_date_text.py and test_ocr_handler.py.

Three outcomes, kept distinct. A run that reads a date, a run that reads text with no date in it, and a run that reads nothing at all are three different facts about the archive, and all three write evidence. Collapsing the last two into "no evidence" would tell a reviewer the back was blank when what actually happened was a bad scan — an absence the software never established, which is what #78 was about. The dateless rows carry raw_value and a note saying which of the two happened.

What the parser does with the forms in the issue. MAR 87, 3/87, 1987, MARCH 1987, MAR 12 87, 12th March 1987, and lab stamps among other words all parse. 3/4/87 comes back as 1987-03-04..1987-04-03 at year precision — precision is the granularity of the claim, not the width of the range, and two candidate days a month apart support no month-level claim. 13/4/87 is unambiguous (13 cannot be a month) and stays day-precise.

Two cases the issue did not name, both decided toward keeping information rather than discarding it:

  • 30 FEB 1987 is not a date, but it is almost always a misread digit with a legible month and year around it. It falls back to February 1987 with a note saying the day was dropped. Clamping to the 28th would invent a date nobody wrote; refusing the line throws away two thirds of a legible date.
  • Neg 32/1987 is not read as 1987. A four-digit number inside a separator-joined token is far more often a catalogue or frame reference, and reading it anyway is how a negative sleeve number becomes a confidently dated photograph.

Writing the tests found three bugs in my own parser, one of which mattered: \s in the regexes matched a newline, so Mar 87 on one line and 1962 on the next parsed as a single Mar 87 1962 — day 87 of March 1962 — which then fell back to the whole of March 1962 and lost 1987 entirely. Two real dates became one wrong one. Lines on the back of a photograph are separate statements and the patterns now say so.

Verified against the real engine. Tesseract is not installed on the Windows dev machine and sudo on the Linux dev server asks for a password (contrary to AGENTS.md, worth knowing), so this ran in a container on that host: tesseract 5.5.0, the full backend suite, 618 passed including the three real-engine tests that skip on Windows. CI now installs tesseract-ocr and tesseract-ocr-eng and runs them on every push.

Following on from here

  • #119 (auto-flag low-confidence scans) has what it needs: mean per-word confidence is on every OCR evidence row.
  • #112 (front datestamps) does not reuse this handler — the front stamp is rated Medium-High and is a different extraction problem — but date_text.py is engine-agnostic and is the parser it should use.
  • Back-of-photo lab stamps are recorded at low alongside handwriting, because telling a stamp from handwriting is not something recognised text supports. If that proves too pessimistic in practice, the confidence value is already stored and the change is one line.
Done in c467a28. CI green. **Done when** - [x] An OCR rerun on a photo with a back scan produces evidence with raw text preserved - [x] Recognised-but-unparseable text still yields an evidence row with `raw_value` set - [x] Ambiguous dates produce a widened range rather than a guessed exact date - [x] The OCR system dependency is documented in the README and installed in CI 71 tests across `test_date_text.py` and `test_ocr_handler.py`. **Three outcomes, kept distinct.** A run that reads a date, a run that reads text with no date in it, and a run that reads nothing at all are three different facts about the archive, and all three write evidence. Collapsing the last two into "no evidence" would tell a reviewer the back was blank when what actually happened was a bad scan — an absence the software never established, which is what #78 was about. The dateless rows carry `raw_value` and a note saying which of the two happened. **What the parser does with the forms in the issue.** `MAR 87`, `3/87`, `1987`, `MARCH 1987`, `MAR 12 87`, `12th March 1987`, and lab stamps among other words all parse. `3/4/87` comes back as 1987-03-04..1987-04-03 at **year** precision — precision is the granularity of the claim, not the width of the range, and two candidate days a month apart support no month-level claim. `13/4/87` is unambiguous (13 cannot be a month) and stays day-precise. Two cases the issue did not name, both decided toward keeping information rather than discarding it: - `30 FEB 1987` is not a date, but it is almost always a misread digit with a legible month and year around it. It falls back to February 1987 with a note saying the day was dropped. Clamping to the 28th would invent a date nobody wrote; refusing the line throws away two thirds of a legible date. - `Neg 32/1987` is **not** read as 1987. A four-digit number inside a separator-joined token is far more often a catalogue or frame reference, and reading it anyway is how a negative sleeve number becomes a confidently dated photograph. **Writing the tests found three bugs in my own parser**, one of which mattered: `\s` in the regexes matched a newline, so `Mar 87` on one line and `1962` on the next parsed as a single `Mar 87 1962` — day 87 of March 1962 — which then fell back to the whole of March 1962 and lost 1987 entirely. Two real dates became one wrong one. Lines on the back of a photograph are separate statements and the patterns now say so. **Verified against the real engine.** Tesseract is not installed on the Windows dev machine and `sudo` on the Linux dev server asks for a password (contrary to `AGENTS.md`, worth knowing), so this ran in a container on that host: tesseract 5.5.0, the full backend suite, **618 passed** including the three real-engine tests that skip on Windows. CI now installs `tesseract-ocr` and `tesseract-ocr-eng` and runs them on every push. **Following on from here** - #119 (auto-flag low-confidence scans) has what it needs: mean per-word confidence is on every OCR evidence row. - #112 (front datestamps) does not reuse this handler — the front stamp is rated Medium-High and is a different extraction problem — but `date_text.py` is engine-agnostic and is the parser it should use. - Back-of-photo lab stamps are recorded at `low` alongside handwriting, because telling a stamp from handwriting is not something recognised text supports. If that proves too pessimistic in practice, the confidence value is already stored and the change is one line.
Sign in to join this conversation.
No description provided.