rbrooks/Circa

Watch

Fork 0

Scanner EXIF dates are recorded as hard, day-precision evidence #74

New issue

Closed

opened 2026-07-28 06:00:23 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-07-28 06:00:23 +00:00

Severity: HIGH - this is writing wrong data today

Found independently by two audit agents and re-verified directly.

The bug

backend/app/services/exif_extractor.py:58 tries date tags in order:

for tag_name in ("DateTimeOriginal", "DateTimeDigitized", "DateTime"):

For a scanned print, DateTimeDigitized and DateTime are the scan date, not the capture
date. backend/app/services/ingest.py:117-130 then writes whatever it found as:

reliability=EvidenceReliability.hard,
precision=DatePrecision.day,

and promotes the photo to needs_review. There is no plausibility check against the collection's
date range.

So a scanner that stamps EXIF mints confident, day-precise, maximum-reliability evidence
saying a 1975 print is from 2025.

Two things that make it harder to undo

Provenance is discarded. ExifResult never records which tag matched, and raw_value
holds only the date string. After ingest you cannot distinguish a genuine capture date from a
scanner timestamp — so this is not cleanly repairable without re-reading every original file.
Camera make/model extraction is dead code with a bug. exif_extractor.py:71-76 builds the
tag id with next((v for k, v in ExifTags.TAGS.items() if ExifTags.TAGS[k] == "Make"), None),
which yields the name string "Make", then looks it up in an integer-keyed dict. Always
None. It is marked "future use" and intended to feed format-rule evidence — it would have
failed silently whenever that was built. It also performs two pointless O(n) scans per photo.

Why this is urgent

hard evidence is exactly what the constraint propagation engine (#19) will trust and spread to
neighbouring photos. This poisons the mechanism the product is built on. Fix before ingesting
more.

It also contradicts the spec's own trust table (docs/circa-spec.md §10.1), which rates EXIF
from a digital camera as High, and reserves hard for format rules.

Scope

The collection is mostly scanned prints, but will include some genuinely digital photos from
more recent years — so EXIF evidence must stay, and be made scan-aware rather than disabled.

Record which tag the date came from, so provenance survives.
Trust DateTimeOriginal at high reliability. Treat bare DateTime/DateTimeDigitized as
probable scan dates: record them at low with an explanatory note, and do not promote the
photo on their basis alone.
Add a plausibility gate: a date outside the configured collection range, or within about a year
of ingest time, is almost certainly a scan date — record at low with a note rather than
promoting.
Fix the make/model tag lookup (use ExifTags.TAGS correctly, or the numeric ids directly);
make/model is a useful format-rule signal later.
Consider a CIRCA_EXIF_TRUST setting (full / scan_aware / off) so behaviour can be tuned
per collection without a code change.

Done when

Evidence records which EXIF tag produced the date
DateTimeOriginal and the digitization tags are treated differently
Implausible dates are recorded at low reliability with a note, not promoted
Make/model extraction actually returns values
Tests cover: real capture date, scanner timestamp, missing EXIF, corrupt EXIF, and a date
outside the collection range
A one-off command can re-evaluate existing EXIF evidence once provenance is recorded

References

backend/app/services/exif_extractor.py:39-78
backend/app/services/ingest.py:114-130
docs/circa-spec.md §10.1

Blocks: #19 (constraint propagation) should not run on poisoned evidence.

## Severity: HIGH - this is writing wrong data today Found independently by two audit agents and re-verified directly. ## The bug `backend/app/services/exif_extractor.py:58` tries date tags in order: ```python for tag_name in ("DateTimeOriginal", "DateTimeDigitized", "DateTime"): ``` For a **scanned print**, `DateTimeDigitized` and `DateTime` are the *scan* date, not the capture date. `backend/app/services/ingest.py:117-130` then writes whatever it found as: ```python reliability=EvidenceReliability.hard, precision=DatePrecision.day, ``` and promotes the photo to `needs_review`. There is no plausibility check against the collection's date range. So a scanner that stamps EXIF mints confident, day-precise, **maximum-reliability** evidence saying a 1975 print is from 2025. ## Two things that make it harder to undo - **Provenance is discarded.** `ExifResult` never records *which* tag matched, and `raw_value` holds only the date string. After ingest you cannot distinguish a genuine capture date from a scanner timestamp — so this is not cleanly repairable without re-reading every original file. - **Camera make/model extraction is dead code with a bug.** `exif_extractor.py:71-76` builds the tag id with `next((v for k, v in ExifTags.TAGS.items() if ExifTags.TAGS[k] == "Make"), None)`, which yields the *name string* `"Make"`, then looks it up in an integer-keyed dict. Always `None`. It is marked "future use" and intended to feed format-rule evidence — it would have failed silently whenever that was built. It also performs two pointless O(n) scans per photo. ## Why this is urgent `hard` evidence is exactly what the constraint propagation engine (#19) will trust and spread to neighbouring photos. This poisons the mechanism the product is built on. Fix before ingesting more. It also contradicts the spec's own trust table (`docs/circa-spec.md` §10.1), which rates EXIF from a digital camera as **High**, and reserves `hard` for format rules. ## Scope The collection is mostly scanned prints, but **will include some genuinely digital photos from more recent years** — so EXIF evidence must stay, and be made scan-aware rather than disabled. - Record which tag the date came from, so provenance survives. - Trust `DateTimeOriginal` at `high` reliability. Treat bare `DateTime`/`DateTimeDigitized` as probable scan dates: record them at `low` with an explanatory note, and do not promote the photo on their basis alone. - Add a plausibility gate: a date outside the configured collection range, or within about a year of ingest time, is almost certainly a scan date — record at `low` with a note rather than promoting. - Fix the make/model tag lookup (use `ExifTags.TAGS` correctly, or the numeric ids directly); make/model is a useful format-rule signal later. - Consider a `CIRCA_EXIF_TRUST` setting (`full` / `scan_aware` / `off`) so behaviour can be tuned per collection without a code change. ## Done when - [ ] Evidence records which EXIF tag produced the date - [ ] `DateTimeOriginal` and the digitization tags are treated differently - [ ] Implausible dates are recorded at low reliability with a note, not promoted - [ ] Make/model extraction actually returns values - [ ] Tests cover: real capture date, scanner timestamp, missing EXIF, corrupt EXIF, and a date outside the collection range - [ ] A one-off command can re-evaluate existing EXIF evidence once provenance is recorded ## References - `backend/app/services/exif_extractor.py:39-78` - `backend/app/services/ingest.py:114-130` - `docs/circa-spec.md` §10.1 Blocks: #19 (constraint propagation) should not run on poisoned evidence.

claude-bot added this to the v0.2.0 milestone

2026-07-28 06:00:23 +00:00

claude-bot added the

bug

backend

labels

2026-07-28 06:00:23 +00:00

claude-bot referenced this issue

2026-07-28 06:07:52 +00:00

Implement the group constraint propagation engine #19

rbrooks referenced this issue from a commit

2026-08-01 05:31:06 +00:00

Stop recording scanner timestamps as capture dates

rbrooks closed this issue

2026-08-01 05:31:06 +00:00

claude-bot commented

2026-08-01 05:52:53 +00:00

Author

Done in be78124. All six "done when" items covered, 36 tests in backend/tests/test_exif_trust.py.

The trust model. DateTimeOriginal → high and promotes. Digitisation tags → low, no promotion. A DateTimeOriginal that fails a plausibility check — in the future, before 1826, or within a year of now — is also low, because some scanners write that tag too and a "capture" date from last month on a box of prints is a scan. high rather than hard per §10.1, which reserves hard for format rules.

One decision beyond what the issue asked for, which I'd like you to sanity-check. A distrusted date is recorded as an upper bound — date_high set, date_low left open — rather than as a low-reliability point estimate. The issue said "record them at low with an explanatory note", and I did that too, but recording date_low == date_high == 2025-03-01 still asserts the photograph was taken on the day it was digitised. That's the false claim this issue exists to stop; lowering the reliability just makes it quieter. "This photograph was not taken after the scan" is true, and it's genuinely useful to #19. If you'd rather have the point estimate, it's a one-line change in _upper_bound.

Provenance is what makes this repairable. date_evidence.source_detail (migration 006) stores the tag name. Without it a wrongly-trusted row is indistinguishable from a correct one and the only repair is re-reading every original file; with it, reassessment is a database operation. python -m app.cli.reassess_exif [--dry-run] does that, sharing the exact assess() function ingest uses so the two cannot drift, and corrects by superseding rather than editing — the archive should record that it once believed something wrong, which is the whole point of having a ledger. Rows predating source_detail are reported and skipped rather than guessed at. It's idempotent; a second run corrects nothing.

Given there's no data yet, the command is insurance rather than a migration you need to run.

The make/model bug was worse than dead code. next((v for k, v in ExifTags.TAGS.items() if ExifTags.TAGS[k] == "Make"), None) yields the name "Make", which was then used as a key in an integer-keyed dict — so it was always None, and it did two pointless O(n) scans per photo to get there. Marked "future use", it would have failed silently the day format-rule evidence was built on it. Now reads tags 271/272 directly and strips the null padding EXIF strings routinely carry.

CIRCA_EXIF_TRUST is implemented as suggested (scan_aware default / full / off), with CIRCA_EXIF_EARLIEST_PLAUSIBLE_YEAR and CIRCA_EXIF_SCAN_WINDOW_DAYS for the plausibility gate.

This unblocks #19 — constraint propagation will no longer be fed poisoned hard evidence.

Done in be78124. All six "done when" items covered, 36 tests in `backend/tests/test_exif_trust.py`. **The trust model.** `DateTimeOriginal` → `high` and promotes. Digitisation tags → `low`, no promotion. A `DateTimeOriginal` that fails a plausibility check — in the future, before 1826, or within a year of now — is also `low`, because some scanners write that tag too and a "capture" date from last month on a box of prints is a scan. `high` rather than `hard` per §10.1, which reserves `hard` for format rules. **One decision beyond what the issue asked for, which I'd like you to sanity-check.** A distrusted date is recorded as an **upper bound** — `date_high` set, `date_low` left open — rather than as a low-reliability point estimate. The issue said "record them at `low` with an explanatory note", and I did that too, but recording `date_low == date_high == 2025-03-01` still asserts the photograph was taken on the day it was digitised. That's the false claim this issue exists to stop; lowering the reliability just makes it quieter. "This photograph was not taken after the scan" is true, and it's genuinely useful to #19. If you'd rather have the point estimate, it's a one-line change in `_upper_bound`. **Provenance is what makes this repairable.** `date_evidence.source_detail` (migration `006`) stores the tag name. Without it a wrongly-trusted row is indistinguishable from a correct one and the only repair is re-reading every original file; with it, reassessment is a database operation. `python -m app.cli.reassess_exif [--dry-run]` does that, sharing the exact `assess()` function ingest uses so the two cannot drift, and corrects by **superseding rather than editing** — the archive should record that it once believed something wrong, which is the whole point of having a ledger. Rows predating `source_detail` are reported and skipped rather than guessed at. It's idempotent; a second run corrects nothing. Given there's no data yet, the command is insurance rather than a migration you need to run. **The make/model bug was worse than dead code.** `next((v for k, v in ExifTags.TAGS.items() if ExifTags.TAGS[k] == "Make"), None)` yields the *name* `"Make"`, which was then used as a key in an integer-keyed dict — so it was always `None`, and it did two pointless O(n) scans per photo to get there. Marked "future use", it would have failed silently the day format-rule evidence was built on it. Now reads tags 271/272 directly and strips the null padding EXIF strings routinely carry. `CIRCA_EXIF_TRUST` is implemented as suggested (`scan_aware` default / `full` / `off`), with `CIRCA_EXIF_EARLIEST_PLAUSIBLE_YEAR` and `CIRCA_EXIF_SCAN_WINDOW_DAYS` for the plausibility gate. This unblocks #19 — constraint propagation will no longer be fed poisoned `hard` evidence.

rbrooks referenced this issue from a commit

2026-08-01 06:29:33 +00:00

Make photo projections derivable from history, and prove it

claude-bot referenced this issue

2026-08-01 06:29:48 +00:00

No projection rebuild path; add one as the test oracle #79

rbrooks referenced this issue from a commit

2026-08-01 21:22:32 +00:00

Read the back of the photograph

rbrooks referenced this issue from a commit

2026-08-02 06:24:23 +00:00

Stop the filename parser inventing dates