[Frontend] Explain provider capability gaps instead of degrading silently #358
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Found in the August 2026 session lifecycle review (#319).
Why
Providers differ in ways that materially change output quality: some offer no word timestamps, some cannot enforce a JSON schema, some have a context window too small for single-window summarisation. If those gaps are invisible, a user experiences them as "Quest Board is bad" rather than "this provider cannot do that" — and the most likely person hitting them is a self-hoster who picked a model without knowing the implications.
Proposed fix
Expose a capability-flags endpoint from the provider abstraction and use it in Admin → Bot Settings to state plainly what the configured combination can and cannot do. For example: "This ASR provider does not return word timestamps — quote citations and timeline precision will be approximate", or "This model's context window requires the transcript to be processed in 6 chunks".
Carry the important ones through to the session UI as a small provenance note, so a GM reading a summary produced under a degraded configuration knows that. This is the same provenance principle as marking pre-timeline-fix sessions.
Acceptance criteria
Landed in PR #492 (merged 2026-09-05, CI green; 1744 backend / 479 frontend tests).
capability_service.derive_capability_gaps— pure, worst-first, seven gap codes (llm_schema_not_enforcedhigh;llm_window_needs_chunkingmedium with N from the summariser's own budgeting math;llm_no_prompt_token_reporting;llm_front_truncates, escalated when nothing can notice;llm_provider_guessed;asr_no_word_timestamps;asr_no_vocabulary).GET /api/admin/ai/capabilities; the Test LLM / Test Whisper responses now carryprovider,provider_pinnedandgaps, on the failure path too.a5b6c7d8e9fa:summarisation_runs.capability_gaps(nullable JSONB). Runs are markedschema_degradedfrom the declaration before any output is parsed — which closes the criterion #351 left partial.Left for later: canary status (#359) is not yet on the bot-settings response; the constant is exported for the panel.