Generate thumbnails and display derivatives at ingest #93

New issue

Closed

opened 2026-07-28 06:03:02 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-07-28 06:03:02 +00:00

Severity: HIGH - blocks the browser being usable at all

The problem

The only media the backend can serve is the original scan. There is no thumbnail endpoint, no
derivative generation, and no resize code anywhere (grep confirms no Pillow resize call).

Consequences today:

The photo grid renders a literal camera emoji placeholder
(frontend/src/pages/PhotoBrowserPage.tsx:29), and .photo-tile-img in styles.css:398 is
dead CSS. Browsing a photo collection currently means reading filenames.
The review workspace loads the full-resolution original for every view, per side.
image/tiff is accepted at ingest, and browsers cannot render TIFF in <img> at all — so
derivatives are a display-correctness requirement, not merely a performance optimization.

The moment real images are added to the grid without derivatives, 100 tiles × multi-megabyte
originals is catastrophic.

Scope

Generate bounded-size derivatives at ingest with Pillow (already a dependency): a review-size
JPEG or WebP (around 2048px) and a thumbnail (around 400px).
Store under derived storage keys alongside the original; never modify the original.
Add /media/thumb and serve the review-size derivative in the workspace, with the original one
click away.
Backfill derivatives for already-ingested photos — a job type fits naturally here.
Handle formats that cannot be decoded gracefully, recording the failure rather than breaking the
tile.
Set explicit width/height and decoding="async" on the img elements to avoid layout thrash.

Done when

Ingest produces thumbnail and review-size derivatives
The grid renders real photographs
The workspace loads a derivative, not the original, with the original available on demand
TIFF scans display correctly in the browser
Existing photos can be backfilled

References

frontend/src/pages/PhotoBrowserPage.tsx:29; styles.css:398
backend/app/api/routes/photos.py:168-191
backend/app/services/ingest.py

Related: #48 scheduled this at v0.5.0, which is far too late. Pair with the media caching issue.

## Severity: HIGH - blocks the browser being usable at all ## The problem The only media the backend can serve is the original scan. There is no thumbnail endpoint, no derivative generation, and no resize code anywhere (grep confirms no Pillow resize call). Consequences today: - The photo grid renders a literal camera emoji placeholder (`frontend/src/pages/PhotoBrowserPage.tsx:29`), and `.photo-tile-img` in `styles.css:398` is dead CSS. **Browsing a photo collection currently means reading filenames.** - The review workspace loads the full-resolution original for every view, per side. - `image/tiff` is accepted at ingest, and browsers **cannot render TIFF in `<img>` at all** — so derivatives are a display-correctness requirement, not merely a performance optimization. The moment real images are added to the grid without derivatives, 100 tiles × multi-megabyte originals is catastrophic. ## Scope - Generate bounded-size derivatives at ingest with Pillow (already a dependency): a review-size JPEG or WebP (around 2048px) and a thumbnail (around 400px). - Store under derived storage keys alongside the original; never modify the original. - Add `/media/thumb` and serve the review-size derivative in the workspace, with the original one click away. - Backfill derivatives for already-ingested photos — a job type fits naturally here. - Handle formats that cannot be decoded gracefully, recording the failure rather than breaking the tile. - Set explicit `width`/`height` and `decoding="async"` on the img elements to avoid layout thrash. ## Done when - [ ] Ingest produces thumbnail and review-size derivatives - [ ] The grid renders real photographs - [ ] The workspace loads a derivative, not the original, with the original available on demand - [ ] TIFF scans display correctly in the browser - [ ] Existing photos can be backfilled ## References - `frontend/src/pages/PhotoBrowserPage.tsx:29`; `styles.css:398` - `backend/app/api/routes/photos.py:168-191` - `backend/app/services/ingest.py` Related: #48 scheduled this at v0.5.0, which is far too late. Pair with the media caching issue.

claude-bot added this to the v0.3.0 milestone

2026-07-28 06:03:02 +00:00

claude-bot added the

backend

enhancement

labels

2026-07-28 06:03:02 +00:00

claude-bot referenced this issue

2026-07-28 06:07:50 +00:00

Performance optimization and progress tracking at collection scale #48

rbrooks referenced this issue from a commit

2026-08-03 19:15:35 +00:00

Show photographs in the photo browser

rbrooks closed this issue

2026-08-03 19:15:35 +00:00

claude-bot commented

2026-08-03 19:16:08 +00:00

Author

Done in ae6dc43. Migration 011, app/services/derivatives.py, a derivatives job type, a backfill CLI, and 24 backend + 5 component tests.

Most of the machinery already existed

#3 built a sandboxed _downscale for shrinking images before an AI upload. So this is mostly reuse — with one addition worth naming: a derivatives operation that produces both sizes from a single decode.

Measured: two separate calls cost ~480 ms, one call ~300 ms. The decode dominates and each sandbox launch is ~125 ms on top, so asking twice pays for both twice. Each size is resampled from the full-resolution source rather than by shrinking the review image — chaining compounds resampling error, and the thumbnail is the image a reviewer scans a grid of and can least afford softness in.

Verified on a real 25.9 MB TIFF: 463 ms → a 924-byte thumbnail and a 16 KB review JPEG.

Inline at ingest, not a job

The issue asks for both ("generate at ingest", "backfill … a job type fits naturally"), and the split matters:

Ingest builds them inline. A job would keep uploads fast, but the worker is a separate process a deployment may not have started — and a photograph with no thumbnail is a camera emoji in the grid, which is the exact bug being fixed. ~300 ms on a request that is already threadpooled (#86) and already launches two subprocesses for verification and EXIF.

The derivatives job handles backfill and retry, where the work is genuinely deferred. python -m app.cli.backfill_derivatives enqueues rather than working: 300 ms a scan across a collection of thousands is a job queue's problem, and a long-running CLI whose failure loses its place is not the answer. It is idempotent — a photo with a job already waiting is skipped — and --retry-failed is opt-in, because sweeping up scans known not to decode would fill the queue with work that cannot succeed.

Serving

/media/thumb — the grid. No variant parameter: a grid that could ask for originals is the catastrophe the endpoint prevents.
/media/front?variant=review — the workspace.
/media/front — still the untouched original.

That last default is deliberate and is the decision I'd most want a second opinion on. For an archive the safe answer to a bare request for a photograph's front is the archival bytes; the optimisation is opt-in. A caller who wants the real thing and forgets a parameter must not silently receive a re-encoded JPEG. The workspace keeps a View original link so the archive is always one click away.

Everything falls back to the original when a derivative is absent — which is what lets the backfill be a background job rather than a migration everyone waits for.

Two properties the tests hold

The original is never touched. A derivative is an additional object at a key derived from the original's content-addressed key, so the two cannot drift and a second run writes nothing.

A scan that will not decode is recorded, not raised. The bytes are still worth keeping and the photograph is still worth a row. derivatives_status and derivatives_error carry the fact, has_thumbnail reaches the client, and the tile shows a warning rather than a broken image that would look like a load in progress (#78).

Frontend

Real <img> tiles with intrinsic width/height and decoding="async". Without the dimensions a hundred-tile grid reflows as each image lands, moving the tile out from under the reviewer's cursor between deciding to click and clicking — a new failure mode that a grid of emoji did not have.

Done when

Ingest produces thumbnail and review-size derivatives
The grid renders real photographs
The workspace loads a derivative, with the original available on demand
TIFF scans display correctly in the browser
Existing photos can be backfilled

1135 backend, 64 component, 7 e2e — the reviewer journey now asserts the grid tile has naturalWidth, so "the tile is a photograph" is checked in a real browser rather than inferred from the markup.

One knock-on: #8's route matrix failed the moment /media/thumb appeared, exactly as designed — a new endpoint cannot ship without someone declaring who may reach it.

Done in ae6dc43. Migration `011`, `app/services/derivatives.py`, a `derivatives` job type, a backfill CLI, and 24 backend + 5 component tests. ## Most of the machinery already existed #3 built a sandboxed `_downscale` for shrinking images before an AI upload. So this is mostly reuse — with one addition worth naming: a `derivatives` operation that produces **both sizes from a single decode**. Measured: two separate calls cost ~480 ms, one call ~300 ms. The decode dominates and each sandbox launch is ~125 ms on top, so asking twice pays for both twice. Each size is resampled from the full-resolution source rather than by shrinking the review image — chaining compounds resampling error, and the thumbnail is the image a reviewer scans a grid of and can least afford softness in. Verified on a real 25.9 MB TIFF: 463 ms → a 924-byte thumbnail and a 16 KB review JPEG. ## Inline at ingest, not a job The issue asks for both ("generate at ingest", "backfill … a job type fits naturally"), and the split matters: **Ingest builds them inline.** A job would keep uploads fast, but the worker is a separate process a deployment may not have started — and a photograph with no thumbnail is a camera emoji in the grid, which is the exact bug being fixed. ~300 ms on a request that is already threadpooled (#86) and already launches two subprocesses for verification and EXIF. **The `derivatives` job handles backfill and retry**, where the work is genuinely deferred. `python -m app.cli.backfill_derivatives` **enqueues rather than working**: 300 ms a scan across a collection of thousands is a job queue's problem, and a long-running CLI whose failure loses its place is not the answer. It is idempotent — a photo with a job already waiting is skipped — and `--retry-failed` is opt-in, because sweeping up scans known not to decode would fill the queue with work that cannot succeed. ## Serving - `/media/thumb` — the grid. No `variant` parameter: a grid that *could* ask for originals is the catastrophe the endpoint prevents. - `/media/front?variant=review` — the workspace. - `/media/front` — **still the untouched original.** That last default is deliberate and is the decision I'd most want a second opinion on. For an archive the safe answer to a bare request for a photograph's front is the archival bytes; the optimisation is opt-in. A caller who wants the real thing and forgets a parameter must not silently receive a re-encoded JPEG. The workspace keeps a **View original** link so the archive is always one click away. Everything falls back to the original when a derivative is absent — which is what lets the backfill be a background job rather than a migration everyone waits for. ## Two properties the tests hold **The original is never touched.** A derivative is an additional object at a key derived from the original's content-addressed key, so the two cannot drift and a second run writes nothing. **A scan that will not decode is recorded, not raised.** The bytes are still worth keeping and the photograph is still worth a row. `derivatives_status` and `derivatives_error` carry the fact, `has_thumbnail` reaches the client, and the tile shows a warning rather than a broken image that would look like a load in progress (#78). ## Frontend Real `<img>` tiles with intrinsic `width`/`height` and `decoding="async"`. Without the dimensions a hundred-tile grid reflows as each image lands, moving the tile out from under the reviewer's cursor between deciding to click and clicking — a new failure mode that a grid of emoji did not have. ## Done when - [x] Ingest produces thumbnail and review-size derivatives - [x] The grid renders real photographs - [x] The workspace loads a derivative, with the original available on demand - [x] TIFF scans display correctly in the browser - [x] Existing photos can be backfilled **1135 backend, 64 component, 7 e2e** — the reviewer journey now asserts the grid tile has `naturalWidth`, so "the tile is a photograph" is checked in a real browser rather than inferred from the markup. One knock-on: `#8`'s route matrix failed the moment `/media/thumb` appeared, exactly as designed — a new endpoint cannot ship without someone declaring who may reach it.

claude-bot referenced this issue

2026-08-05 16:37:59 +00:00

Ingest enqueues no jobs, so OCR and AI analysis never run on their own #144

claude-bot referenced this issue

2026-08-05 18:38:47 +00:00

Ingest enqueues no jobs, so OCR and AI analysis never run on their own #144