rbrooks/Quest-Board

Watch

Fork 0

[Backend] The prose summary path and the admin "test LLM" button ignore the explicit AI provider and still sniff the URL #488

New issue

Closed

opened 2026-09-05 00:17:15 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-09-05 00:17:15 +00:00

Collaborator

Found while grounding #484, on main at 233dc71 — i.e. this is in v4.1.1, which is the release that introduced the setting.

What is wrong

#351 added an explicit provider select in Admin → Bot Settings and app/providers/registry.resolve_llm_provider, and the changelog says the provider now decides schema format, truncation detection and the default window. That is true for generate_structured_text and for /ask (answer_question). It is not true for the two paths below, which accept the setting and ignore it:

_dispatch_prose (audio_service.py:2058) — the session-summary prose path, which carries the largest prompts in the product. It takes llm_provider: str | None in its signature and never reads it; the body dispatches on "anthropic.com" in endpoint_url, "openai.com" in endpoint_url, _is_ollama_url(endpoint_url), else llama.cpp (lines 2081 / 2096 / 2111), and hands the four _summarise_* transports the loose URL. Its comment says "Same resolve-then-preflight as generate_structured_text", which is what it was meant to do.
test_llm (audio_service.py:4785) — what the admin "test connection" button calls. Same four-way URL sniff, and its docstring still says "Uses the same provider detection logic as summarise()".

Who it bites

Exactly the operator the setting exists for: a gateway or reverse proxy on their own domain, who sets the provider to anthropic/openai/ollama. Structured generation and /ask route correctly; session summaries go through the llama.cpp transport (wrong request shape, wrong truncation detection, 32k window), and the test button reports success against the wrong transport — so it looks configured, works for some features and not others, which is the "partial adoption is worse than none" failure the generate_structured_text docstring warns about.

With the setting left on "Detect from the URL" nothing is wrong, which is why the suite did not catch it.

Fix

Being done inside the #484 PR as its own commit, since #484 rewrites both functions to take LLMConfig: resolve the provider once via the registry and dispatch on provider.capabilities.name, the way answer_question already does. One test per path that sets provider="anthropic" on a non-anthropic URL and asserts the Anthropic transport is the one called.

Found while grounding #484, on `main` at `233dc71` — i.e. this is in v4.1.1, which is the release that introduced the setting. ## What is wrong #351 added an explicit provider select in Admin → Bot Settings and `app/providers/registry.resolve_llm_provider`, and the changelog says the provider now decides schema format, truncation detection and the default window. That is true for `generate_structured_text` and for `/ask` (`answer_question`). It is **not** true for the two paths below, which accept the setting and ignore it: - **`_dispatch_prose`** ([audio_service.py:2058](webapp/backend/app/services/audio_service.py#L2058)) — the session-summary prose path, which carries the largest prompts in the product. It takes `llm_provider: str | None` in its signature and never reads it; the body dispatches on `"anthropic.com" in endpoint_url`, `"openai.com" in endpoint_url`, `_is_ollama_url(endpoint_url)`, else llama.cpp (lines 2081 / 2096 / 2111), and hands the four `_summarise_*` transports the loose URL. Its comment says "Same resolve-then-preflight as `generate_structured_text`", which is what it was meant to do. - **`test_llm`** ([audio_service.py:4785](webapp/backend/app/services/audio_service.py#L4785)) — what the admin "test connection" button calls. Same four-way URL sniff, and its docstring still says "Uses the same provider detection logic as summarise()". ## Who it bites Exactly the operator the setting exists for: a gateway or reverse proxy on their own domain, who sets the provider to `anthropic`/`openai`/`ollama`. Structured generation and `/ask` route correctly; **session summaries go through the llama.cpp transport** (wrong request shape, wrong truncation detection, 32k window), and the test button reports success against the wrong transport — so it looks configured, works for some features and not others, which is the "partial adoption is worse than none" failure the `generate_structured_text` docstring warns about. With the setting left on "Detect from the URL" nothing is wrong, which is why the suite did not catch it. ## Fix Being done inside the #484 PR as its own commit, since #484 rewrites both functions to take `LLMConfig`: resolve the provider once via the registry and dispatch on `provider.capabilities.name`, the way `answer_question` already does. One test per path that sets `provider="anthropic"` on a non-anthropic URL and asserts the Anthropic transport is the one called.

claude-bot changed title from [Backend] The prose summary path and the admin "test LLM" button ignore the explicit AI provider and still sniff the URL</title> <parameter name="body">Found while grounding #484, on main at 233dc71 — i.e. this is in v4.1.1, which is the release … to [Backend] The prose summary path and the admin "test LLM" button ignore the explicit AI provider and still sniff the URL

2026-09-05 00:18:41 +00:00

claude-bot added this to the v4.2.0 - AI Provider Abstraction & BYO-AI milestone

2026-09-05 00:18:41 +00:00

claude-bot added the

backend

bug

labels

2026-09-05 00:18:55 +00:00

claude-bot referenced this issue from a pull request that will close it,

2026-09-05 01:00:32 +00:00

refactor(llm): pass LLMConfig through the generation paths as one object (#484), and honour the explicit provider on the prose path (#488) #489

rbrooks referenced this issue from a commit

2026-09-05 01:00:56 +00:00

docs(changelog): cite the bug issue (#488), not the refactor, for the provider fix

claude-bot closed this issue

2026-09-05 01:09:35 +00:00

claude-bot referenced this issue from a commit

2026-09-05 01:09:36 +00:00

Merge PR #489: pass LLMConfig through the generation paths as one object (#484); honour the explicit provider on the prose path (#488)

claude-bot referenced this issue

2026-09-05 01:10:16 +00:00

[Backend] Pass LLMConfig through the generation paths instead of destructuring it into loose parameters #484

claude-bot commented

2026-09-05 01:10:21 +00:00

Author

Collaborator

Fixed in PR #489, commit 1b0e273, merged 2026-09-05.

Both _dispatch_prose and test_llm now resolve the provider once through registry.llm_provider_for(target) and dispatch on provider.capabilities.name, the same way answer_question and generate_structured_text already did — so the explicit setting wins everywhere or nowhere. _dispatch_prose also takes its window from the resolved provider, so a pinned Anthropic/OpenAI target on a custom domain gets that provider's window rather than the llama.cpp default.

Two tests (test_the_prose_summary_honours_an_explicit_provider, test_the_admin_connection_test_honours_an_explicit_provider) set provider="anthropic" on a non-anthropic gateway URL and assert the Anthropic transport is the one awaited and the llama.cpp one is not. Both were verified red against the pre-fix file before being verified green.

Changelog entry under Fixed for the next release. Not yet deployed anywhere: prod is on v4.1.0 (does not have the provider select at all), dev is on v4.1.1 (has the select, and this bug).

Fixed in **PR #489**, commit `1b0e273`, merged 2026-09-05. Both `_dispatch_prose` and `test_llm` now resolve the provider once through `registry.llm_provider_for(target)` and dispatch on `provider.capabilities.name`, the same way `answer_question` and `generate_structured_text` already did — so the explicit setting wins everywhere or nowhere. `_dispatch_prose` also takes its window from the resolved provider, so a pinned Anthropic/OpenAI target on a custom domain gets that provider's window rather than the llama.cpp default. Two tests (`test_the_prose_summary_honours_an_explicit_provider`, `test_the_admin_connection_test_honours_an_explicit_provider`) set `provider="anthropic"` on a non-anthropic gateway URL and assert the Anthropic transport is the one awaited and the llama.cpp one is not. Both were verified red against the pre-fix file before being verified green. Changelog entry under **Fixed** for the next release. Not yet deployed anywhere: prod is on v4.1.0 (does not have the provider select at all), dev is on v4.1.1 (has the select, and this bug).

rbrooks referenced this issue from a commit

2026-09-05 01:17:05 +00:00

chore(release): v4.1.2

claude-bot referenced this issue

2026-09-05 01:17:20 +00:00

chore(release): v4.1.2 #490

rbrooks referenced this issue from a commit

2026-09-05 03:40:34 +00:00

feat(admin): an ASR provider select, and a Test button that speaks its dialect

rbrooks referenced this issue from a commit

2026-09-05 05:15:16 +00:00

feat(api): a GM's campaign AI settings, and the admin switch above them (#128)