feat: per-campaign AI providers — a campaign brings its own ASR and LLM, behind an instance switch (#128) #501

Merged

claude-bot merged 9 commits from feat/128-campaign-byo-ai into main 2026-09-05 05:24:07 +00:00

Conversation 0 Commits 9 Files changed 29 +3001 -139

claude-bot commented 2026-09-05 05:15:42 +00:00

Contributor

Copy link

Closes #128 (the per-campaign BYO-AI half; the adapter half landed in #496, the usage counters in #494).

One instance, several groups, each bringing their own API key and paying for their own transcription and summarisation — with the instance's providers as the fallback and an admin switch above it all.

What

• Migration d8e9fa0b1c2d (after c7d8e9fa0b1c, one head, up/down/up verified twice): campaign_ai_settings — per campaign, each side's endpoint, key (encrypted with the existing AES-256-GCM/HKDF scheme via new crypto.encrypt_secret/decrypt_secret), model, provider, window; updated_by.
• Instance switch allow_campaign_ai_overrides (app_settings, default off) on the bot-settings API and as one checkbox in Admin → Bot Settings.
• Resolution with fallback, one resolver per side (get_llm_config(db, campaign_id=None), get_whisper_config(db, campaign_id=None)), documented next to the four not-configured policies: no campaign → instance; no row or blank endpoint for that side → instance; switch off → instance, row untouched; otherwise the campaign's, each side independently. An override that is in effect never silently falls back: a campaign URL rejected by policy is a clear error naming the campaign's settings, and an undecryptable campaign key becomes an empty credential with a warn-once error — substituting the admin's provider would send a group's audio somewhere they did not choose.
• Every origin with a campaign in scope passes it — process_audio (resolved before the capability gaps, slots and usage row), the session-scoped tasks, the @llm_task decorator (so all four converted tasks and any future one get it for free), planning tasks, the Workbench generators, /ask, lore and beat services, describe_*. Admin test buttons, /bot/settings at bot startup, and the monthly canary stay instance-level, each with a comment. An AST guard test walks the origin sites so a call site written next month cannot forget the campaign.
• API: GET|PUT|DELETE /api/campaigns/{id}/ai-settings and POST …/test-llm|test-asr (GM; 403 with a message naming the admin when the switch is off); URLs through the same SSRF guard as instance settings; providers validated against the registries; responses carry *_api_key_set and *_using_instance_default, never a key; blank endpoint returns that side to the instance default; keys preserve on blank and clear on an explicit flag. Test buttons refuse (400) when that side has no override rather than probing the instance's endpoint for a non-admin. GET /api/settings/instance (any authenticated user) gates the frontend.
• Frontend: CampaignAiSettings on the campaign settings page for GMs when the switch is on — LLM and ASR blocks, "using instance default" indicators, test and clear; key never rendered.
• Docs: OPERATIONS "Per-campaign AI providers (bring your own)" — the switch, what a GM can set, that campaign keys share the instance secret's key derivation, that audio leaves the server when a campaign points at a managed provider, and that turning the switch off disables every override at once.

One behaviour change to note: /ask resolves the LLM after the campaign is known, so a guild with no summaries and no LLM now gets the "no summaries yet" 200 rather than a 503; same response schema, no bot contract change.

Rebased over #352 and #353/#354 (changelog and OPERATIONS conflicts, both sides kept; the rebase surfaced and fixed a vacuous assertion of this branch's own — a coroutine compared to None).

Verification

• Backend 2185 passed, 13 skipped; frontend 503 passed; ruff and eslint clean (one pre-existing warning); alembic heads → d8e9fa0b1c2d; ciphertext-at-rest, SSRF, precedence, masking and GM/player/other-GM authorisation tests; version sync OK.

🤖 Generated with Claude Code

Closes #128 (the per-campaign BYO-AI half; the adapter half landed in #496, the usage counters in #494). One instance, several groups, each bringing their own API key and paying for their own transcription and summarisation — with the instance's providers as the fallback and an admin switch above it all. ## What - **Migration `d8e9fa0b1c2d`** (after `c7d8e9fa0b1c`, one head, up/down/up verified twice): `campaign_ai_settings` — per campaign, each side's endpoint, key (encrypted with the existing AES-256-GCM/HKDF scheme via new `crypto.encrypt_secret`/`decrypt_secret`), model, provider, window; `updated_by`. - **Instance switch** `allow_campaign_ai_overrides` (app_settings, default off) on the bot-settings API and as one checkbox in Admin → Bot Settings. - **Resolution with fallback**, one resolver per side (`get_llm_config(db, campaign_id=None)`, `get_whisper_config(db, campaign_id=None)`), documented next to the four not-configured policies: no campaign → instance; no row or blank endpoint for that side → instance; switch off → instance, row untouched; otherwise the campaign's, each side independently. **An override that is in effect never silently falls back**: a campaign URL rejected by policy is a clear error naming the campaign's settings, and an undecryptable campaign key becomes an empty credential with a warn-once error — substituting the admin's provider would send a group's audio somewhere they did not choose. - **Every origin with a campaign in scope passes it** — `process_audio` (resolved before the capability gaps, slots and usage row), the session-scoped tasks, the `@llm_task` decorator (so all four converted tasks and any future one get it for free), planning tasks, the Workbench generators, `/ask`, lore and beat services, `describe_*`. Admin test buttons, `/bot/settings` at bot startup, and the monthly canary stay instance-level, each with a comment. An AST guard test walks the origin sites so a call site written next month cannot forget the campaign. - **API**: `GET|PUT|DELETE /api/campaigns/{id}/ai-settings` and `POST …/test-llm|test-asr` (GM; 403 with a message naming the admin when the switch is off); URLs through the same SSRF guard as instance settings; providers validated against the registries; responses carry `*_api_key_set` and `*_using_instance_default`, never a key; blank endpoint returns that side to the instance default; keys preserve on blank and clear on an explicit flag. Test buttons refuse (400) when that side has no override rather than probing the instance's endpoint for a non-admin. `GET /api/settings/instance` (any authenticated user) gates the frontend. - **Frontend**: `CampaignAiSettings` on the campaign settings page for GMs when the switch is on — LLM and ASR blocks, "using instance default" indicators, test and clear; key never rendered. - **Docs**: OPERATIONS "Per-campaign AI providers (bring your own)" — the switch, what a GM can set, that campaign keys share the instance secret's key derivation, that audio leaves the server when a campaign points at a managed provider, and that turning the switch off disables every override at once. One behaviour change to note: `/ask` resolves the LLM after the campaign is known, so a guild with no summaries **and** no LLM now gets the "no summaries yet" 200 rather than a 503; same response schema, no bot contract change. Rebased over #352 and #353/#354 (changelog and OPERATIONS conflicts, both sides kept; the rebase surfaced and fixed a vacuous assertion of this branch's own — a coroutine compared to `None`). ## Verification - Backend **2185 passed, 13 skipped**; frontend **503 passed**; ruff and eslint clean (one pre-existing warning); `alembic heads` → `d8e9fa0b1c2d`; ciphertext-at-rest, SSRF, precedence, masking and GM/player/other-GM authorisation tests; version sync OK. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 9 commits 2026-09-05 05:15:42 +00:00

db: campaign_ai_settings, where a group's own AI providers live (#128) ... e3f8df5de2

Transcription and the LLM have been one app_settings row each since the
beginning — right for the single-group self-host this started as, wrong for the
case it has grown into: one box, several groups who share nothing but a host,
each with their own API key and their own bill.

The table is an *override* set, not a configuration. Every column is nullable
and a campaign with no row — which is every campaign that exists — resolves to
the instance default. campaign_id is the primary key as well as the foreign key,
because a campaign has one override set or none and a surrogate id would only
make two rows for one campaign representable. ON DELETE CASCADE on the campaign;
SET NULL on updated_by_id, because a group's providers must not vanish when the
GM who typed them in deletes their account.

The two key columns hold ciphertext from the new crypto.encrypt_secret, which
wraps the existing encrypt_setting rather than reimplementing it: one key
derivation, one cipher, one storage format. A campaign's key is protected
exactly as well as the instance's, and SECRET_KEY relates them in the only way
an operator should ever have to reason about.

An unreadable key deliberately does not fall back to the instance's — see
campaign_ai_service.decode_api_key. Quietly rerouting a group's transcripts to
the admin's provider is a privacy decision made on someone else's behalf; a
request with no credential fails visibly and is theirs to fix.

Additive: new table, no backfill, nothing existing touched. Verified up, down,
up against Postgres 16.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(settings): per-campaign AI configuration with instance fallback (#128) ... a87d849d64

get_llm_config, require_llm_config and get_whisper_config now take an optional
campaign_id. There is exactly one resolver per side and the campaign-vs-instance
decision lives only there, because a second function that could answer "which
endpoint" is how one condition grew eight behaviours in #287 — one layer down.

The precedence, per side and independently for each: no campaign in scope, no
row (or a blank endpoint on this side), or the admin switch off, all mean the
instance default; otherwise the campaign's own, with the window resolved by the
same _KNOWN_CONTEXT_TOKENS rule and a blank provider sniffed from the URL
exactly as the instance's is. The two sides are separate, so "my own LLM key,
the instance's Whisper" is a legal and ordinary configuration.

The rule that matters more than it looks: an override that is in effect never
falls back. A campaign endpoint rejected by URL policy resolves to "not
configured" and require_llm_config says so in words a GM can act on, rather than
silently substituting the admin's endpoint and sending a group's session audio
somewhere they did not choose.

The campaign row is looked up before the instance switch is read, so an instance
where nobody uses the feature pays one primary-key miss and never touches the
settings row at all.

allow_campaign_ai_overrides is off unless an admin turns it on: an override lets
a non-admin cause this instance to make outbound requests to a host of their
choosing, which belongs to whoever runs the box. Turning it off makes every
override inert without deleting anything.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(tasks): resolve the campaign's providers at every origin (#128) ... 56ee9091a6

Since #484 the config travels as one target object, so making a group's own
providers real is purely a change at the origin — every site that resolves a
config now says which campaign the work belongs to.

process_audio resolves both sides against session.campaign_id before anything
downstream reads them, so the provider slots, the capability gaps #358 seeds
from them and the usage row #357 writes all describe the providers the run
actually used, with no change to any of them.

The three lore chunk tasks get the campaign from their session id through a new
session_service.get_campaign_id — one scalar select for the one immutable
column they need, rather than loading a row two of them already load. The
workbench and draft paths take it from the result row and the wiki entry. In
llm_task the campaign is derived from the decorator's own subject/obj rather
than asked of each body, so every task converted to it resolves correctly
without doing anything and a body cannot forget to.

/ask now resolves after the campaign is known instead of before. One visible
consequence: a guild with no summaries and no LLM gets "no session summaries are
available yet" rather than "LLM endpoint not configured" — the more accurate of
the two, since there is nothing to answer from either way. The response shape is
untouched, so no bot contract bump.

The admin test buttons, the bot's startup settings read and the monthly provider
canary stay instance-level, each with a comment saying why: they answer
questions about the deployment, and routing them through a campaign's provider
would report on an endpoint the admin cannot fix, or spend a group's key on a
schedule nobody asked for.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(api): a GM's campaign AI settings, and the admin switch above them (#128) ... 509cfe914f

GET/PUT/DELETE /api/campaigns/{id}/ai-settings, plus test-llm and test-asr,
all GM-only and all 403 while allow_campaign_ai_overrides is off — including the
read, so a GM cannot make this instance call an arbitrary host by finding the
route. The switch itself rides on the existing admin bot-settings GET/PUT, and
is written only when the field is present, like the VAD trio: an older client
must not disable every campaign's override by omission.

A key goes in and never comes out. The response carries llm_api_key_set /
asr_api_key_set and nothing else about it — no ciphertext, no prefix, no length
— and the audit entry records that the settings changed, not what to.

Endpoint URLs, models, providers and the window are replaced by what is sent,
deliberately unlike the admin endpoint where blank preserves: here the endpoint
URL *is* the on/off switch for its side, so blank has to mean "back to the
instance default" or an override could be turned on and never off. The two keys
keep the familiar convention — blank preserves, clear_llm_key / clear_asr_key
delete — because a form that cannot show you your key cannot ask you to retype
it on every save.

Every URL goes through normalize_service_url with the same allowed internal
hosts and the same private-range rule an admin's does. That guard is what makes
handing this form to a non-admin defensible at all. Provider names are validated
against the adapter registry, and the ASR model rule is imported from the admin
router rather than restated — two copies of "which model does this gateway
serve" is how they stop agreeing.

The Test buttons probe the campaign's own endpoint or refuse. Falling through to
the instance's would put its latency and, via a transport error string, its URL
in front of a non-admin, and would make one button mean two different things
depending on state. The ASR probe moves to ai_probe_service so the admin's
button and a GM's make the identical call — a Test button that drifts from the
path it claims to test is exactly the bug #488 and #128's ASR half both were.

GET /api/settings/instance is new and deliberately tiny: the capabilities any
signed-in user may know about, so the campaign page can decide whether to render
the panel instead of interpreting a 403 that equally means "you are not the GM".

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(frontend): a campaign's own AI providers, and the admin switch (#128) ... f925285c49

CampaignAiSettings sits on the campaign settings page for GMs and renders
nothing at all — not a disabled form, not an explanation — when the instance has
the feature off. Whether it exists is read from GET /api/settings/instance
rather than inferred from a 403 on the settings route: that 403 is equally the
answer for "you are not the GM", and a panel appearing or not should not depend
on telling two refusals apart.

The key field is always empty and says "a key is set" beside it, because the API
returns nothing else about it. Leaving it blank keeps what is stored; removing
one is an explicit Clear, since "I did not retype my key" and "delete my key"
must not be the same gesture. Each side carries its own "using instance default"
badge and its own Test button, and the Test button only appears for a side this
campaign actually overrides — one combined indicator would be false about half
of an ordinary "my LLM, your Whisper" configuration.

Admin → Bot Settings gets the one checkbox, with the two consequences an
operator needs before ticking it: a group pointing at a hosted service is
uploading their session audio under their contract rather than yours, and
turning it back off makes every override inert without deleting any of them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

test: precedence, secrecy, and a guard for the call sites nobody has written yet (#128) ... 6b9e1d952a

Three things here would not be noticed if they broke, so each is asserted
against the thing itself rather than through a helper equally capable of hiding
the problem: the key is read out of the raw column and searched for in the raw
response body, the toggle-off case asserts the row still *exists* as well as
being inert, and the two sides are checked separately so an implementation where
ASR follows LLM cannot pass.

The authorisation case that matters is a GM of another campaign — authenticated,
and a GM, elsewhere — built by giving a second campaign to another user rather
than by standing up a second authenticated client.

test_campaign_ai_origin_sites is the stopgap for next month. The way this
feature breaks is that somebody adds a call site, writes get_llm_config(db)
because that is what every example in the file says, and a group's
summarisation quietly goes back on the admin's key: nothing fails, the summary
is fine, the bill is just wrong on someone else's card. An omitted argument is
invisible to review, so it is checked mechanically in the spirit of #484's
signature guard — parse the modules that resolve AI configs and require
campaign_id= wherever a campaign is plainly in scope. The escape hatch is an
allowlist carrying a reason per entry, not a magic comment, and a second test
fails if an allowlisted site stops existing, so the list cannot become a place
exemptions hide. Verified by breaking a real call site and watching it fail.

app/routers/admin.py is watched despite having no resolver calls today: it is
where AI-facing admin endpoints go, and the next one should be policed on the
day it is written.

CampaignDetail's users.js mock gains fetchInstanceSettings, resolved with the
feature off — the instance default, which keeps that file about everything else.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

docs: what an operator is agreeing to when they let campaigns bring their own AI (#128) ... e9afec0270

A new OPERATIONS section covering the switch, what a GM can set on each side,
and the four things worth knowing before enabling it: campaign keys are
encrypted with the same key as the instance's own secrets, session audio leaves
this server when a campaign points at a managed provider, turning the switch off
disables every override at once without deleting anything, and an override that
is in effect fails visibly rather than borrowing the instance's endpoint.

The audio point is stated plainly because it is the one an operator needs before
the fact: if the instance has a data-handling promise to its groups, this switch
is what lets them opt out of it — for themselves, deliberately.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

db: chain campaign_ai_settings behind the transcript-words migration (#128) ... f4a8235c08

d8e9fa0b1c2d was written against b6c7d8e9fa0b because #352's c7d8e9fa0b1c was
not on main yet. It is now, and two revisions naming the same parent is two
heads — which alembic will not upgrade past and which nobody notices until a
deploy stops halfway.

Nothing else changes: the two are unrelated and additive, the order between them
does not matter, and `upgrade head → downgrade -1 → upgrade head` is verified
again on the new chain with `alembic heads` showing exactly one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

test: await the gap derivation, which became a coroutine under us (#128, #352) ... ebd5b49e52

_resolve_capability_gaps went async in #352, because the ASR side is now
discovered from the endpoint rather than declared. The rebase merged cleanly and
the assertion kept passing — on a coroutine object, which is never None, so it
was asserting nothing at all and would have gone on passing against an
implementation that raised the moment it ran.

Caught by pytest's "coroutine was never awaited" warning rather than by a
failure, which is the whole hazard: this is the one shape of test rot that looks
identical to a green test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

claude-bot scheduled this pull request to auto merge when all checks succeed 2026-09-05 05:15:48 +00:00

claude-bot merged commit bfe54fa927 into main 2026-09-05 05:24:07 +00:00

claude-bot deleted branch feat/128-campaign-byo-ai 2026-09-05 05:24:08 +00:00

claude-bot referenced this pull request from a commit 2026-09-05 05:24:09 +00:00

Merge PR #501: per-campaign AI providers behind an instance switch (#128)

claude-bot referenced this pull request 2026-09-05 05:25:03 +00:00

[Backlog] AI provider abstraction and per-campaign BYO-AI #128

rbrooks referenced this pull request from a commit 2026-09-05 05:41:26 +00:00

db: chain the usage-identity columns behind #501's campaign settings (#502)

claude-bot referenced this pull request 2026-09-05 05:41:44 +00:00

fix(usage): record the resolved provider, price managed ASR by host and model, and let live conformance reach the managed adapter (#502) #503

claude-bot referenced this pull request 2026-09-05 06:04:34 +00:00

chore(release): v4.2.0 — AI Provider Abstraction & BYO-AI #504

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

backend

Issue related to the backend

  

bot

Issue related to the bot

  

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

frontend

Issue related to the frontend

  

help wanted

Need some help

  

invalid

Something is wrong

  

question

More information is needed

  

recording

Issue related to the recording system

  

security

Security hardening / vulnerability work

  

wontfix

This won't be fixed

No labels

backend

bot

bug

duplicate

enhancement

frontend

help wanted

invalid

question

recording

security

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

rbrooks/Quest-Board!501

No description provided.