[GM Workbench] Generalized workbench generation endpoint + tool registry

claude-bot commented

2026-07-15 22:01:04 +00:00

Collaborator

Motivation / Context

Today's generation pattern is one endpoint per tool: POST /sessions/{session_id}/name-options/generate (webapp/backend/app/routers/sessions.py:235-264) and GET /{campaign_id}/planning/names/{category} (webapp/backend/app/routers/campaigns.py:3093-3144). That does not scale to the ~12-tool GM Workbench catalog (docs/.internal/gm-planning-expansion-2026-07-15.md §3). Name generation also carries an awkward wart worth fixing while generalizing: the custom-prompt path requires picking an active session (sessions.py:235, CampaignPlanning.jsx:545-548) purely because canonical-name selection lives on Session — an odd constraint for what is conceptually a campaign-level tool.

This is issue 2 of 3 foundational items. It turns the one-off name-generation plumbing into a registry-driven surface every subsequent tool (descriptions, rumors, loot, tables, improv NPCs, prep sheets, …) registers into instead of getting its own bespoke endpoint.

Approach

Tool registry. New webapp/backend/app/services/generation_service.py with a GENERATOR_TOOLS registry: a dict of tool definitions keyed by tool_id, each carrying system prompt, input schema, context builders, output schema, json_mode flag, prefetchable flag, and sync_allowed flag (per §5.1 of the report).
Endpoints, campaign-scoped, GM-only via the existing require_gm dependency:
- POST /api/campaigns/{campaign_id}/workbench/{tool_id}/generate — validates params against the tool's input schema, dispatches to sync or Celery execution per the tool's sync_allowed flag.
- The planning prefetch endpoint (campaigns.py:3074-3090) generalizes to any tool flagged prefetchable, reusing the existing Redis lock + TTL pattern from planning_tasks.py:53-94 verbatim (lock key per campaign+tool).
Sync vs. Celery split (§5.1): fast JSON tools (names, rumors, tables — small outputs) run inline exactly as the cache-miss path does today (campaigns.py:3119-3139). Prose-heavy tools (backstory, prep sheet, shop sheet) go through a Celery task modeled on generate_lore_entry_draft (webapp/backend/app/tasks/reminder_tasks.py:2018-2048), writing status into a pending → ready/failed record — the same lifecycle the draft pipeline already uses, and the record this issue's status writes into is the GenerationResult model landing in #138 (issue 3). Rule of thumb from the report: anything that can exceed a few seconds on a local Ollama box must be async, since the hosted product cannot hold HTTP requests open on GPU-server latency.
Migrate name generation as the first registered tool. Move _NAME_CATEGORY_DESCRIPTION_HINTS (audio_service.py:163-218), _NAME_GENERATOR_SYSTEM_PROMPT (audio_service.py:220-228), and generate_name_options (audio_service.py:759-826) into a registry entry. Drop the session-required constraint on the custom-prompt path — the new endpoint is campaign-scoped, so a session link becomes optional metadata rather than a gate. Saving a chosen name to a LoreEntry still works as it does today (sessions.py:283-296).
Keep old endpoints as thin wrappers during migration — POST /sessions/{session_id}/name-options/generate and GET /{campaign_id}/planning/names/{category} continue to work, internally calling into the registry, so no frontend call sites break before #153 (Workbench UX reorganization) migrates them.
Context builders (§5.3), standardizing today's ad-hoc context assembly (campaign_name + (f" ({game_system})"), audio_service.py:780): campaign_context(campaign), lore_context(db, campaign_id, *, entry_ids | category, limit), threads_context(db, campaign_id), recent_sessions_context(db, campaign_id, n). Each tool declares which builders it needs.
Built on llm_service.py from issue #134.

Dependencies

#134 (Extract LLM core into llm_service) — the registry's generation calls route through generate_structured_text in llm_service.py.
Feeds #138 (Generation scratchpad/history) — the async status lifecycle described here writes into the GenerationResult model that issue defines; the two are typically implemented together or in tight sequence.
Every Phase 1+ tool issue (#141 description, #144 rumors, #147 random tables, #149 loot, #153 UX reorg, #155 improv NPCs, #158 prep sheet) registers into this endpoint rather than adding new routes.

Out of scope

The GenerationResult scratchpad/history model and its endpoints — that's issue #138.
Any individual generation tool beyond names (descriptions, rumors, loot, tables, etc.) — those are separate Phase 1/2 issues.
Frontend UX reorganization (tool palette, unified GeneratorPanel) — issue #153.
Per-campaign LLM config / bring-your-own-key — get_llm_config (settings_service.py:182-201) stays instance-global for now; the registry should take a resolved LLMConfig per-request so this swap is localized later, but implementing tenant-level config is out of scope here.
Per-campaign generation quotas / metering — noted as a hosted-service concern (§6) but not built in this issue.

Acceptance criteria

POST /api/campaigns/{campaign_id}/workbench/{tool_id}/generate exists, is GM-only, validates against the tool's schema, and dispatches sync or async per the tool's flags.
Name generation is fully registered as a tool (tool_id="names" or similar) and produces identical output to today's flow.
Custom-prompt name generation no longer requires selecting an active session; a session link is optional.
Old endpoints (sessions.py:235, campaigns.py:3093, campaigns.py:3074 prefetch) continue to work unmodified for existing frontend callers.
Context builders (campaign_context, lore_context, threads_context, recent_sessions_context) exist and are used by the names tool for at least campaign_context.
Prefetch generalizes to any tool flagged prefetchable, reusing the existing Redis lock/TTL pattern.

## Motivation / Context Today's generation pattern is one endpoint per tool: `POST /sessions/{session_id}/name-options/generate` (`webapp/backend/app/routers/sessions.py:235-264`) and `GET /{campaign_id}/planning/names/{category}` (`webapp/backend/app/routers/campaigns.py:3093-3144`). That does not scale to the ~12-tool **GM Workbench** catalog (`docs/.internal/gm-planning-expansion-2026-07-15.md` §3). Name generation also carries an awkward wart worth fixing while generalizing: the custom-prompt path requires picking an active session (`sessions.py:235`, `CampaignPlanning.jsx:545-548`) purely because canonical-name selection lives on `Session` — an odd constraint for what is conceptually a campaign-level tool. This is issue 2 of 3 foundational items. It turns the one-off name-generation plumbing into a registry-driven surface every subsequent tool (descriptions, rumors, loot, tables, improv NPCs, prep sheets, …) registers into instead of getting its own bespoke endpoint. ## Approach - **Tool registry.** New `webapp/backend/app/services/generation_service.py` with a `GENERATOR_TOOLS` registry: a dict of tool definitions keyed by `tool_id`, each carrying system prompt, input schema, context builders, output schema, `json_mode` flag, `prefetchable` flag, and `sync_allowed` flag (per §5.1 of the report). - **Endpoints**, campaign-scoped, GM-only via the existing `require_gm` dependency: - `POST /api/campaigns/{campaign_id}/workbench/{tool_id}/generate` — validates params against the tool's input schema, dispatches to sync or Celery execution per the tool's `sync_allowed` flag. - The planning prefetch endpoint (`campaigns.py:3074-3090`) generalizes to any tool flagged `prefetchable`, reusing the existing Redis lock + TTL pattern from `planning_tasks.py:53-94` verbatim (lock key per campaign+tool). - **Sync vs. Celery split** (§5.1): fast JSON tools (names, rumors, tables — small outputs) run inline exactly as the cache-miss path does today (`campaigns.py:3119-3139`). Prose-heavy tools (backstory, prep sheet, shop sheet) go through a Celery task modeled on `generate_lore_entry_draft` (`webapp/backend/app/tasks/reminder_tasks.py:2018-2048`), writing status into a `pending → ready/failed` record — the same lifecycle the draft pipeline already uses, and the record this issue's status writes into is the `GenerationResult` model landing in **#138** (issue 3). Rule of thumb from the report: anything that can exceed a few seconds on a local Ollama box must be async, since the hosted product cannot hold HTTP requests open on GPU-server latency. - **Migrate name generation as the first registered tool.** Move `_NAME_CATEGORY_DESCRIPTION_HINTS` (`audio_service.py:163-218`), `_NAME_GENERATOR_SYSTEM_PROMPT` (`audio_service.py:220-228`), and `generate_name_options` (`audio_service.py:759-826`) into a registry entry. Drop the session-required constraint on the custom-prompt path — the new endpoint is campaign-scoped, so a session link becomes optional metadata rather than a gate. Saving a chosen name to a `LoreEntry` still works as it does today (`sessions.py:283-296`). - **Keep old endpoints as thin wrappers during migration** — `POST /sessions/{session_id}/name-options/generate` and `GET /{campaign_id}/planning/names/{category}` continue to work, internally calling into the registry, so no frontend call sites break before **#153** (Workbench UX reorganization) migrates them. - **Context builders** (§5.3), standardizing today's ad-hoc context assembly (`campaign_name + (f" ({game_system})")`, `audio_service.py:780`): `campaign_context(campaign)`, `lore_context(db, campaign_id, *, entry_ids | category, limit)`, `threads_context(db, campaign_id)`, `recent_sessions_context(db, campaign_id, n)`. Each tool declares which builders it needs. - Built on `llm_service.py` from issue #134. ## Dependencies - **#134** (Extract LLM core into `llm_service`) — the registry's generation calls route through `generate_structured_text` in `llm_service.py`. - Feeds **#138** (Generation scratchpad/history) — the async status lifecycle described here writes into the `GenerationResult` model that issue defines; the two are typically implemented together or in tight sequence. - Every Phase 1+ tool issue (**#141** description, **#144** rumors, **#147** random tables, **#149** loot, **#153** UX reorg, **#155** improv NPCs, **#158** prep sheet) registers into this endpoint rather than adding new routes. ## Out of scope - The `GenerationResult` scratchpad/history model and its endpoints — that's issue **#138**. - Any individual generation tool beyond names (descriptions, rumors, loot, tables, etc.) — those are separate Phase 1/2 issues. - Frontend UX reorganization (tool palette, unified `GeneratorPanel`) — issue **#153**. - Per-campaign LLM config / bring-your-own-key — `get_llm_config` (`settings_service.py:182-201`) stays instance-global for now; the registry should take a resolved `LLMConfig` per-request so this swap is localized later, but implementing tenant-level config is out of scope here. - Per-campaign generation quotas / metering — noted as a hosted-service concern (§6) but not built in this issue. ## Acceptance criteria - `POST /api/campaigns/{campaign_id}/workbench/{tool_id}/generate` exists, is GM-only, validates against the tool's schema, and dispatches sync or async per the tool's flags. - Name generation is fully registered as a tool (`tool_id="names"` or similar) and produces identical output to today's flow. - Custom-prompt name generation no longer requires selecting an active session; a session link is optional. - Old endpoints (`sessions.py:235`, `campaigns.py:3093`, `campaigns.py:3074` prefetch) continue to work unmodified for existing frontend callers. - Context builders (`campaign_context`, `lore_context`, `threads_context`, `recent_sessions_context`) exist and are used by the names tool for at least `campaign_context`. - Prefetch generalizes to any tool flagged `prefetchable`, reusing the existing Redis lock/TTL pattern.

claude-bot added this to the v3.10.0 - GM Workbench milestone

2026-07-15 22:01:10 +00:00

claude-bot added the

backend

enhancement

labels

Rows
Columns

[GM Workbench] Generalized workbench generation endpoint + tool registry #136

Motivation / Context

Approach

Dependencies

Out of scope

Acceptance criteria