Implement a local AI backend (ollama/llava) #42

New issue

Open

opened 2026-07-28 04:59:53 +00:00 by claude-bot · 0 comments

claude-bot commented

2026-07-28 04:59:53 +00:00

Context

The Phase 4 acceptance criterion states a local-only deployment can disable cloud AI
entirely. Family photographs are private, and sending every one of them to a third-party
API is a choice some deployments will not want to make.

Scope

A local AI backend (ollama with a vision model such as llava) behind the existing
AIBackend interface.

Implementation notes

Implement against the interface defined with the Claude backend in v0.2.0. If that
interface leaked provider-specific assumptions, this is where it shows.
Configurable endpoint and model name, since ollama is typically self-hosted.
Local models are markedly weaker at this task than frontier vision models. Reflect that
in the evidence reliability assigned to their output, and be explicit in the UI about
which backend produced a given piece of evidence â€” a reviewer weighing AI evidence needs
to know its provenance.
The same structured-output validation as the cloud backend applies. Local models are
more likely to return malformed JSON, so validation must be strict and failures must
be retryable.
Local inference is slow; ensure the worker's timeout and retry settings are configurable
and do not treat a slow local run as a failure.
Cost tracking is not applicable locally, but the run should still record timing so the
dashboard has something meaningful to show.
A configuration that disables cloud AI entirely, with no code path able to reach an
external API â€” verified by test, not by inspection.

Done when

A local backend produces date evidence through the existing interface
Backend selection is configuration-only
Evidence records which backend produced it, with appropriate reliability
Cloud AI can be fully disabled, proven by test
Malformed local model output is rejected and retried

References

The AI backend interface from #3
docs/circa-spec.md Phase 4 acceptance criteria

Depends on: #3 (AI analysis job handler).

## Context The Phase 4 acceptance criterion states a local-only deployment can disable cloud AI entirely. Family photographs are private, and sending every one of them to a third-party API is a choice some deployments will not want to make. ## Scope A local AI backend (ollama with a vision model such as llava) behind the existing `AIBackend` interface. ## Implementation notes - Implement against the interface defined with the Claude backend in v0.2.0. If that interface leaked provider-specific assumptions, this is where it shows. - Configurable endpoint and model name, since ollama is typically self-hosted. - Local models are markedly weaker at this task than frontier vision models. Reflect that in the evidence `reliability` assigned to their output, and be explicit in the UI about which backend produced a given piece of evidence â€” a reviewer weighing AI evidence needs to know its provenance. - The same structured-output validation as the cloud backend applies. Local models are *more* likely to return malformed JSON, so validation must be strict and failures must be retryable. - Local inference is slow; ensure the worker's timeout and retry settings are configurable and do not treat a slow local run as a failure. - Cost tracking is not applicable locally, but the run should still record timing so the dashboard has something meaningful to show. - A configuration that disables cloud AI entirely, with no code path able to reach an external API â€” verified by test, not by inspection. ## Done when - [ ] A local backend produces date evidence through the existing interface - [ ] Backend selection is configuration-only - [ ] Evidence records which backend produced it, with appropriate reliability - [ ] Cloud AI can be fully disabled, proven by test - [ ] Malformed local model output is rejected and retried ## References - The AI backend interface from #3 - `docs/circa-spec.md` Phase 4 acceptance criteria Depends on: #3 (AI analysis job handler).

claude-bot added this to the v0.6.0 milestone

2026-07-28 04:59:53 +00:00

claude-bot added the

backend

enhancement

labels

2026-07-28 04:59:53 +00:00