[Backend] The prose summary path and the admin "test LLM" button ignore the explicit AI provider and still sniff the URL #488
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Found while grounding #484, on
mainat233dc71— i.e. this is in v4.1.1, which is the release that introduced the setting.What is wrong
#351 added an explicit provider select in Admin → Bot Settings and
app/providers/registry.resolve_llm_provider, and the changelog says the provider now decides schema format, truncation detection and the default window. That is true forgenerate_structured_textand for/ask(answer_question). It is not true for the two paths below, which accept the setting and ignore it:_dispatch_prose(audio_service.py:2058) — the session-summary prose path, which carries the largest prompts in the product. It takesllm_provider: str | Nonein its signature and never reads it; the body dispatches on"anthropic.com" in endpoint_url,"openai.com" in endpoint_url,_is_ollama_url(endpoint_url), else llama.cpp (lines 2081 / 2096 / 2111), and hands the four_summarise_*transports the loose URL. Its comment says "Same resolve-then-preflight asgenerate_structured_text", which is what it was meant to do.test_llm(audio_service.py:4785) — what the admin "test connection" button calls. Same four-way URL sniff, and its docstring still says "Uses the same provider detection logic as summarise()".Who it bites
Exactly the operator the setting exists for: a gateway or reverse proxy on their own domain, who sets the provider to
anthropic/openai/ollama. Structured generation and/askroute correctly; session summaries go through the llama.cpp transport (wrong request shape, wrong truncation detection, 32k window), and the test button reports success against the wrong transport — so it looks configured, works for some features and not others, which is the "partial adoption is worse than none" failure thegenerate_structured_textdocstring warns about.With the setting left on "Detect from the URL" nothing is wrong, which is why the suite did not catch it.
Fix
Being done inside the #484 PR as its own commit, since #484 rewrites both functions to take
LLMConfig: resolve the provider once via the registry and dispatch onprovider.capabilities.name, the wayanswer_questionalready does. One test per path that setsprovider="anthropic"on a non-anthropic URL and asserts the Anthropic transport is the one called.[Backend] The prose summary path and the admin "test LLM" button ignore the explicit AI provider and still sniff the URL</title> <parameter name="body">Found while grounding #484, onto [Backend] The prose summary path and the admin "test LLM" button ignore the explicit AI provider and still sniff the URLmainat233dc71— i.e. this is in v4.1.1, which is the release …Fixed in PR #489, commit
1b0e273, merged 2026-09-05.Both
_dispatch_proseandtest_llmnow resolve the provider once throughregistry.llm_provider_for(target)and dispatch onprovider.capabilities.name, the same wayanswer_questionandgenerate_structured_textalready did — so the explicit setting wins everywhere or nowhere._dispatch_prosealso takes its window from the resolved provider, so a pinned Anthropic/OpenAI target on a custom domain gets that provider's window rather than the llama.cpp default.Two tests (
test_the_prose_summary_honours_an_explicit_provider,test_the_admin_connection_test_honours_an_explicit_provider) setprovider="anthropic"on a non-anthropic gateway URL and assert the Anthropic transport is the one awaited and the llama.cpp one is not. Both were verified red against the pre-fix file before being verified green.Changelog entry under Fixed for the next release. Not yet deployed anywhere: prod is on v4.1.0 (does not have the provider select at all), dev is on v4.1.1 (has the select, and this bug).