EXIF extraction fails on every TIFF, and the obvious fix silently loses DateTimeOriginal #145

New issue

Closed

opened 2026-08-05 23:43:18 +00:00 by claude-bot · 0 comments

claude-bot commented

2026-08-05 23:43:18 +00:00

Severity: HIGH — an entire evidence source is missing for an archival scan format, silently

Found while building the container images for #50. Confirmed on the host as well, so it is not container-induced.

The bug

backend/app/services/image_sandbox.py:125 calls a private Pillow API:

raw = img._getexif()  # type: ignore[attr-defined]

_getexif exists on JpegImageFile and does not exist on TiffImageFile. Verified against the pinned Pillow 12.3.0:

TIFF   type=TiffImageFile    has _getexif=False   has getexif=True
JPEG   type=JpegImageFile    has _getexif=True    has getexif=True

So every TIFF ingested raises AttributeError inside the EXIF sandbox, and exif evidence (#74) — trust-weighted by tag, with the whole exif_trust machinery behind it — is never recorded for any of them.

TIFF is not an edge case here. It is in ALLOWED_EXTENSIONS and in ingest_batch.ACCEPTED_SUFFIXES, it is the archival format a flatbed scanner is usually configured to produce, and docs/circa-spec.md treats scanned prints as the whole subject of the application.

Why nothing caught it

Every EXIF test builds a JPEG. tests/test_exif_trust.py saves with format="JPEG" and there is no TIFF anywhere in the EXIF tests. So the suite exercises the one type that happens to have the private method, and the format most likely to arrive from a real scanner is untested.

The failure is also recorded but invisible. ingest_photo puts exif_status and exif_error in the audit event — #65 added exactly that so "EXIF could not be read" stays distinguishable from "there was none". The distinction is in the ledger and nothing surfaces it, so an operator sees no error and simply gets no EXIF evidence, forever.

app/services/exif_extractor.py:130 already carries a comment noting _getexif is private API.

The fix is NOT a one-line swap — this is the important half

img.getexif() is the public replacement, but it does not return the same thing. DateTimeOriginal (tag 36867) lives in the Exif sub-IFD at 0x8769. _getexif() merged that in; getexif() does not. Measured:

_getexif()                 has DateTimeOriginal: True
dict(getexif())            has DateTimeOriginal: False
getexif().get_ifd(0x8769)  has DateTimeOriginal: True

DateTimeOriginal is the tag this application cares about most — exif_trust rates it the strongest EXIF signal precisely because it means "when the shutter opened" rather than "when the file was written". So a naive _getexif() → getexif() swap would fix the loud TIFF failure and introduce a quiet wrong answer on every JPEG, which is strictly worse: an absent date becomes an absent date nobody can tell was ever there.

The fix must merge the top-level IFD with get_ifd(0x8769), and the tests must pin that it does.

Scope

Replace _getexif() with a public-API read that merges the Exif sub-IFD
Cover TIFF in the EXIF tests, not only JPEG — the format the archive is actually in
Pin that DateTimeOriginal survives the change, so the sub-IFD merge cannot be dropped later
Decide whether a failed EXIF read should be visible anywhere other than the audit ledger; it is currently recorded and unread, which is how this survived

Worth checking at the same time

Whether anything else in the image path reaches for a private Pillow attribute, since the same class of breakage would be equally silent.

Timing

This is worth doing before a real collection is ingested, not after. EXIF evidence is recorded at ingest, so every TIFF that goes in before the fix carries no exif evidence — and re-deriving it later means re-reading every original file rather than a database pass. app/cli/reassess_exif.py exists and may cover that, which is worth confirming as part of this.

References

backend/app/services/image_sandbox.py:125
backend/app/services/exif_extractor.py (the _getexif is-private comment)
backend/tests/test_exif_trust.py (JPEG only)
#74 (EXIF trust weighting), #65 (EXIF status recorded in the ledger), #50 (found while containerizing)

## Severity: HIGH — an entire evidence source is missing for an archival scan format, silently Found while building the container images for #50. Confirmed on the host as well, so it is not container-induced. ## The bug `backend/app/services/image_sandbox.py:125` calls a **private** Pillow API: ```python raw = img._getexif() # type: ignore[attr-defined] ``` `_getexif` exists on `JpegImageFile` and **does not exist on `TiffImageFile`**. Verified against the pinned Pillow 12.3.0: ``` TIFF type=TiffImageFile has _getexif=False has getexif=True JPEG type=JpegImageFile has _getexif=True has getexif=True ``` So **every TIFF ingested raises `AttributeError` inside the EXIF sandbox**, and `exif` evidence (#74) — trust-weighted by tag, with the whole `exif_trust` machinery behind it — is never recorded for any of them. TIFF is not an edge case here. It is in `ALLOWED_EXTENSIONS` and in `ingest_batch.ACCEPTED_SUFFIXES`, it is the archival format a flatbed scanner is usually configured to produce, and `docs/circa-spec.md` treats scanned prints as the whole subject of the application. ## Why nothing caught it **Every EXIF test builds a JPEG.** `tests/test_exif_trust.py` saves with `format="JPEG"` and there is no TIFF anywhere in the EXIF tests. So the suite exercises the one type that happens to have the private method, and the format most likely to arrive from a real scanner is untested. The failure is also *recorded but invisible*. `ingest_photo` puts `exif_status` and `exif_error` in the audit event — #65 added exactly that so "EXIF could not be read" stays distinguishable from "there was none". The distinction is in the ledger and nothing surfaces it, so an operator sees no error and simply gets no EXIF evidence, forever. `app/services/exif_extractor.py:130` already carries a comment noting `_getexif` is private API. ## The fix is NOT a one-line swap — this is the important half `img.getexif()` is the public replacement, but it **does not return the same thing**. `DateTimeOriginal` (tag 36867) lives in the Exif sub-IFD at 0x8769. `_getexif()` merged that in; `getexif()` does not. Measured: ``` _getexif() has DateTimeOriginal: True dict(getexif()) has DateTimeOriginal: False getexif().get_ifd(0x8769) has DateTimeOriginal: True ``` `DateTimeOriginal` is the tag this application cares about most — `exif_trust` rates it the strongest EXIF signal precisely because it means "when the shutter opened" rather than "when the file was written". So a naive `_getexif()` → `getexif()` swap would **fix the loud TIFF failure and introduce a quiet wrong answer on every JPEG**, which is strictly worse: an absent date becomes an absent date nobody can tell was ever there. The fix must merge the top-level IFD with `get_ifd(0x8769)`, and the tests must pin that it does. ## Scope - [ ] Replace `_getexif()` with a public-API read that merges the Exif sub-IFD - [ ] Cover **TIFF** in the EXIF tests, not only JPEG — the format the archive is actually in - [ ] Pin that `DateTimeOriginal` survives the change, so the sub-IFD merge cannot be dropped later - [ ] Decide whether a failed EXIF read should be visible anywhere other than the audit ledger; it is currently recorded and unread, which is how this survived ## Worth checking at the same time Whether anything else in the image path reaches for a private Pillow attribute, since the same class of breakage would be equally silent. ## Timing This is worth doing **before a real collection is ingested**, not after. EXIF evidence is recorded at ingest, so every TIFF that goes in before the fix carries no `exif` evidence — and re-deriving it later means re-reading every original file rather than a database pass. `app/cli/reassess_exif.py` exists and may cover that, which is worth confirming as part of this. ## References - `backend/app/services/image_sandbox.py:125` - `backend/app/services/exif_extractor.py` (the `_getexif` is-private comment) - `backend/tests/test_exif_trust.py` (JPEG only) - #74 (EXIF trust weighting), #65 (EXIF status recorded in the ledger), #50 (found while containerizing)

claude-bot referenced this issue

2026-08-06 18:43:42 +00:00

Create the production deployment and packaging story #50

claude-bot referenced this issue

2026-08-07 04:32:01 +00:00

A bulk import never says how many scans it could not read EXIF from #148