Implement the OCR job handler for back-of-photo date extraction #4
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Context
POST /api/photos/{photo_id}/evidence/ocr-rerunenqueues aJobType.ocrjob andEvidenceSource.ocrexists, but nothing performs OCR. Backs of photos frequentlycarry handwritten or stamped dates, which is the point of scanning them.
Scope
The
ocrjob handler: run OCR over the back scan, extract date-like strings,normalize them into
DateEvidence, and keep the raw text for reviewer inspection.Implementation notes
pytesseractis the obvious default,but it is a system dependency that must be documented and installed in CI and on
the dev server. Note the decision in the issue before implementing.
raw_valueeven when no date is parsed; thereviewer panel should be able to show what the scanner actually read.
MAR 87,3/87,1987,MARCH 1987,processing-lab date stamps, and ambiguous
3/4/87day/month order. Ambiguity shouldwiden the range rather than guess — emit
date_low/date_highwith the correctDatePrecision.reliabilityshould be low by default. The spec is explicit that handwritten datesmay be wrong, so OCR evidence must not outrank EXIF or a reviewer decision.
Done when
raw_valuesetReferences
backend/app/api/routes/jobs.pybackend/app/models/models.py(EvidenceSource.ocr,JobType.ocr)docs/circa-spec.md(back-of-photo OCR, handwritten date trust)Depends on: the background worker runtime.
Amended by the audit of 2026-07-28.
This issue covers OCR of the back only. The orange film datestamps printed on the front of
late-80s and 90s prints are ubiquitous and rated Medium-High by the spec, and have no extraction
path anywhere — now tracked in #112 alongside the format-rule knowledge base.
Also applies here: auto quality-flagging on low OCR confidence (#119), and running image work
inside the worker under resource limits (#65).
Picking this up now that #2 (worker runtime) and #87 (SQLite concurrency) are done. The issue asks for the engine decision to be noted before implementing, so:
Engine: Tesseract, invoked as a subprocess rather than through
pytesseract.Considered:
tesseract-ocron Debian). Good on printed and stamped text, mediocre on block printing, poor on cursive. That limitation is real and is exactly why the spec rates a handwritten date on the back Low (§10.1) and why the raw text is preserved for the reviewer regardless of whether a date parses out of it.Why the binary directly rather than
pytesseract.pytesseractis a thin wrapper that shells out to the same binary, so it adds a dependency without removing the system one. Calling it ourselves means the timeout, the environment, and the failure modes are ours — and this codebase already has the pattern from #65's image sandbox, including launching with a minimal environment that withholds everyCIRCA_*variable, since the process reading attacker-supplied bytes has no need for the signing key. The only thingpytesseractwould have saved is TSV parsing for per-word confidence, which is a dozen lines.It stays behind an
OcrBackendinterface with a null implementation, so a deployment without tesseract installed fails the job with "OCR is not configured" rather than crashing the worker, and tests get a deterministic fake.Two things I am deliberately not doing here, for the record.
low, per §10.1 and this issue. Note that the spec's Medium-High row is for a printed/stamped date on the front (#112). Processing-lab stamps also appear on the back, and those are arguably better than Low — but telling a lab stamp from handwriting needs classification we cannot do from recognised text alone. The per-word confidence tesseract reports is a reasonable future proxy, so it is stored on the evidence row; acting on it is #119.3/4/87) genuinely is a coin flip and gets widened as the issue asks. A two-digit year is not: prints carrying date stamps are overwhelmingly 20th century, so87resolves to 1987 rather than producing a useless hundred-year range. The assumption is written into the evidencenoteswhere the reviewer can see it, and the pivot is configurable.Will comment again when it lands. Tesseract is not installed on the Windows dev machine, so the against-the-real-engine test will be verified on the Linux dev server, and CI gets the apt package.
Done in
c467a28. CI green.Done when
raw_valueset71 tests across
test_date_text.pyandtest_ocr_handler.py.Three outcomes, kept distinct. A run that reads a date, a run that reads text with no date in it, and a run that reads nothing at all are three different facts about the archive, and all three write evidence. Collapsing the last two into "no evidence" would tell a reviewer the back was blank when what actually happened was a bad scan — an absence the software never established, which is what #78 was about. The dateless rows carry
raw_valueand a note saying which of the two happened.What the parser does with the forms in the issue.
MAR 87,3/87,1987,MARCH 1987,MAR 12 87,12th March 1987, and lab stamps among other words all parse.3/4/87comes back as 1987-03-04..1987-04-03 at year precision — precision is the granularity of the claim, not the width of the range, and two candidate days a month apart support no month-level claim.13/4/87is unambiguous (13 cannot be a month) and stays day-precise.Two cases the issue did not name, both decided toward keeping information rather than discarding it:
30 FEB 1987is not a date, but it is almost always a misread digit with a legible month and year around it. It falls back to February 1987 with a note saying the day was dropped. Clamping to the 28th would invent a date nobody wrote; refusing the line throws away two thirds of a legible date.Neg 32/1987is not read as 1987. A four-digit number inside a separator-joined token is far more often a catalogue or frame reference, and reading it anyway is how a negative sleeve number becomes a confidently dated photograph.Writing the tests found three bugs in my own parser, one of which mattered:
\sin the regexes matched a newline, soMar 87on one line and1962on the next parsed as a singleMar 87 1962— day 87 of March 1962 — which then fell back to the whole of March 1962 and lost 1987 entirely. Two real dates became one wrong one. Lines on the back of a photograph are separate statements and the patterns now say so.Verified against the real engine. Tesseract is not installed on the Windows dev machine and
sudoon the Linux dev server asks for a password (contrary toAGENTS.md, worth knowing), so this ran in a container on that host: tesseract 5.5.0, the full backend suite, 618 passed including the three real-engine tests that skip on Windows. CI now installstesseract-ocrandtesseract-ocr-engand runs them on every push.Following on from here
date_text.pyis engine-agnostic and is the parser it should use.lowalongside handwriting, because telling a stamp from handwriting is not something recognised text supports. If that proves too pessimistic in practice, the confidence value is already stored and the change is one line.