rbrooks/Circa

Watch

Fork 0

Local face clustering to accelerate person tagging #128

New issue

Open

opened 2026-07-28 16:02:33 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-07-28 16:02:33 +00:00

Decision

The audit rejected face work as an explicit spec non-goal. The user has since approved face
clustering — with a specific, experience-driven requirement about cluster hygiene, below.

Clustering here means grouping visually similar faces without identifying anyone. No names are
inferred, no identities are matched against a database. A human still assigns every name. This is a
tagging accelerator, not a recognition system.

Why

Manual person tagging (#110) is the highest-value evidence source available, but tagging thousands
of photos one at a time is the bottleneck that would stop it being used. Clustering turns "tag
every photo individually" into "confirm this group of 40 faces is Uncle Pete."

The hard requirement: cluster hygiene

Direct from the user, based on using tools that got this wrong:

"It should be easy to remove faces from the clusters. I've used photo tagging tools with face
clustering where it was very hard to remove them before tagging."

This is the make-or-break feature, not a nice-to-have. Clustering is always imperfect —
especially on scanned prints, where faces are small, grainy, off-angle, and half a century apart in
age for the same person. A cluster that cannot be easily corrected is worse than no cluster,
because the reviewer either propagates a wrong tag to dozens of photos or abandons the tool.

Required operations, all available before a name is committed and after:

Remove a single face from a cluster, in one click, from the cluster grid
Multi-select and bulk-remove several faces at once
Split a cluster into two
Merge two clusters
Move a face directly to another named cluster
Undo any of the above
Removed faces return to the unassigned pool rather than vanishing

Design the cluster review grid so removal is the primary interaction, not buried in a context
menu. Assume roughly one in ten faces is misgrouped and make that cheap to fix.

The covenant still holds

Clustering produces no evidence and no tags on its own. It proposes groups; a human names them.
Naming a cluster creates the same photo_person tags a manual tag would, with the same
attribution — no separate "auto-tagged" tier.
A reviewer can always tag a photo directly without touching clusters.

Privacy and architecture

All processing local. Detection and embedding run on the machine or dev server. Face crops
and embeddings must never be sent to a cloud model.
Face embeddings are biometric data. Store them alongside the collection, cover them in backup
(#117), and document what they are.
The whole subsystem is optional and disableable, with a path to delete all embeddings and clusters
without touching photos or tags.
Keep the boundary clean enough that local recognition could be added later — the user is
reconsidering that non-goal if it can run entirely locally. Do not design clustering in a way that
makes that a rewrite, but do not build it now either.

Technical sketch

Face detection plus embedding, run as a worker job type over ingested photos. Candidate stacks:
InsightFace, or dlib/face_recognition. Choose for offline operation and CPU viability, and record
the decision.
Store per-face: photo id, bounding box, embedding, quality score, cluster id.
Cluster embeddings with a density-based or agglomerative method and a tunable distance threshold.
Prefer conservative clustering — many small pure clusters beat few large mixed ones, because
merging is cheap and unpicking is expensive.
Re-clustering must never silently discard human corrections: once a face has been moved or removed
by a person, that decision is sticky across re-runs.
Show the face crop, not the whole photo, in the cluster grid, with the full photo one click away.

Spec amendment required

docs/circa-spec.md lists "facial recognition or person identification" as a non-goal. Amend it to
state that local face clustering is in scope as a manual-tagging accelerator, that no automated
identification occurs, and that all processing is local. Note the open question about local
recognition rather than leaving the document contradicting the code.

Done when

Faces are detected and clustered locally, as a worker job
A reviewer can remove, bulk-remove, split, merge, move, and undo — all in one or two clicks
Human corrections survive re-clustering
Naming a cluster produces ordinary person tags with ordinary attribution
Nothing is ever auto-tagged by similarity alone
No face data leaves the machine
Embeddings can be deleted wholesale without affecting photos or tags
The spec non-goal is amended

References

docs/circa-spec.md (non-goals section)

Depends on: #110 (person registry), #2 (worker runtime). Feeds: #110's apparent-age dating.

## Decision The audit rejected face work as an explicit spec non-goal. **The user has since approved face clustering** — with a specific, experience-driven requirement about cluster hygiene, below. Clustering here means grouping visually similar faces **without identifying anyone**. No names are inferred, no identities are matched against a database. A human still assigns every name. This is a tagging accelerator, not a recognition system. ## Why Manual person tagging (#110) is the highest-value evidence source available, but tagging thousands of photos one at a time is the bottleneck that would stop it being used. Clustering turns "tag every photo individually" into "confirm this group of 40 faces is Uncle Pete." ## The hard requirement: cluster hygiene Direct from the user, based on using tools that got this wrong: > "It should be easy to remove faces from the clusters. I've used photo tagging tools with face > clustering where it was very hard to remove them before tagging." This is the make-or-break feature, not a nice-to-have. Clustering is *always* imperfect — especially on scanned prints, where faces are small, grainy, off-angle, and half a century apart in age for the same person. A cluster that cannot be easily corrected is worse than no cluster, because the reviewer either propagates a wrong tag to dozens of photos or abandons the tool. Required operations, all available **before** a name is committed and **after**: - Remove a single face from a cluster, in one click, from the cluster grid - Multi-select and bulk-remove several faces at once - Split a cluster into two - Merge two clusters - Move a face directly to another named cluster - Undo any of the above - Removed faces return to the unassigned pool rather than vanishing Design the cluster review grid so removal is the *primary* interaction, not buried in a context menu. Assume roughly one in ten faces is misgrouped and make that cheap to fix. ## The covenant still holds - Clustering produces **no evidence and no tags on its own.** It proposes groups; a human names them. - Naming a cluster creates the same `photo_person` tags a manual tag would, with the same attribution — no separate "auto-tagged" tier. - A reviewer can always tag a photo directly without touching clusters. ## Privacy and architecture - **All processing local.** Detection and embedding run on the machine or dev server. Face crops and embeddings must never be sent to a cloud model. - Face embeddings are biometric data. Store them alongside the collection, cover them in backup (#117), and document what they are. - The whole subsystem is optional and disableable, with a path to delete all embeddings and clusters without touching photos or tags. - Keep the boundary clean enough that local **recognition** could be added later — the user is reconsidering that non-goal if it can run entirely locally. Do not design clustering in a way that makes that a rewrite, but do not build it now either. ## Technical sketch - Face detection plus embedding, run as a worker job type over ingested photos. Candidate stacks: InsightFace, or dlib/face_recognition. Choose for offline operation and CPU viability, and record the decision. - Store per-face: photo id, bounding box, embedding, quality score, cluster id. - Cluster embeddings with a density-based or agglomerative method and a tunable distance threshold. Prefer conservative clustering — many small pure clusters beat few large mixed ones, because merging is cheap and unpicking is expensive. - Re-clustering must never silently discard human corrections: once a face has been moved or removed by a person, that decision is sticky across re-runs. - Show the face crop, not the whole photo, in the cluster grid, with the full photo one click away. ## Spec amendment required `docs/circa-spec.md` lists "facial recognition or person identification" as a non-goal. Amend it to state that **local face clustering is in scope as a manual-tagging accelerator**, that no automated identification occurs, and that all processing is local. Note the open question about local recognition rather than leaving the document contradicting the code. ## Done when - [ ] Faces are detected and clustered locally, as a worker job - [ ] A reviewer can remove, bulk-remove, split, merge, move, and undo — all in one or two clicks - [ ] Human corrections survive re-clustering - [ ] Naming a cluster produces ordinary person tags with ordinary attribution - [ ] Nothing is ever auto-tagged by similarity alone - [ ] No face data leaves the machine - [ ] Embeddings can be deleted wholesale without affecting photos or tags - [ ] The spec non-goal is amended ## References - `docs/circa-spec.md` (non-goals section) Depends on: #110 (person registry), #2 (worker runtime). Feeds: #110's apparent-age dating.

claude-bot added this to the v0.4.0 milestone

2026-07-28 16:02:33 +00:00

claude-bot added the

frontend

backend

enhancement

labels

2026-07-28 16:02:33 +00:00

claude-bot referenced this issue

2026-07-28 16:02:35 +00:00

Person registry with birth years and apparent-age dating #110

rbrooks referenced this issue from a commit

2026-07-28 16:18:35 +00:00

Amend spec: face clustering in scope, Circa archive view standalone

claude-bot commented

2026-07-28 16:19:02 +00:00

Author

Spec amendment done in 66d5e77 — the "Done when" item for it can be considered closed.

docs/circa-spec.md §2 now distinguishes face clustering (in scope) from face recognition (out of
scope), and records:

clustering proposes groups only — no names, no evidence, no tags on its own
clusters must be cheaply correctable, with remove/split/merge/undo as first-class operations
available before and after naming, because clustering on scanned prints is unavoidably imperfect
all processing is local; crops and embeddings never go to a cloud model
embeddings are biometric data — backed up, and deletable wholesale without affecting photos or tags
the subsystem is optional and disableable
local offline recognition is recorded as an open question, explicitly not in the roadmap, with
the note that clustering should not be architected in a way that makes it a rewrite

The genealogy non-goal was also annotated as possibly reconsidered much later, per the same
conversation, so that thought is not lost.

**Spec amendment done** in 66d5e77 — the "Done when" item for it can be considered closed. `docs/circa-spec.md` §2 now distinguishes face clustering (in scope) from face recognition (out of scope), and records: - clustering proposes groups only — no names, no evidence, no tags on its own - clusters must be cheaply correctable, with remove/split/merge/undo as first-class operations available before and after naming, because clustering on scanned prints is unavoidably imperfect - all processing is local; crops and embeddings never go to a cloud model - embeddings are biometric data — backed up, and deletable wholesale without affecting photos or tags - the subsystem is optional and disableable - local offline recognition is recorded as an **open question**, explicitly not in the roadmap, with the note that clustering should not be architected in a way that makes it a rewrite The genealogy non-goal was also annotated as possibly reconsidered much later, per the same conversation, so that thought is not lost.