[GM Workbench] Extract LLM core from audio_service into llm_service #134

Closed
opened 2026-07-15 22:00:35 +00:00 by claude-bot · 2 comments
Contributor

Motivation / Context

audio_service.py currently contains transcription and the entire structured-LLM stack: provider adapters (generate_structured_text at webapp/backend/app/services/audio_service.py:1158-1221 plus the four _structured_* backends), a dozen module-level system-prompt constants (:41-228), and domain functions for lore, names, storyline, and Q&A. The file is "audio" in name only.

This is issue 1 of 3 foundational items behind the GM Workbench vision: turning GM Planning from one generator (names) into ~12 campaign-grounded generation tools (descriptions, backstories, loot, rumors, encounter seeds, improv NPCs, prep sheets, etc. — see the full catalog in docs/.internal/gm-planning-expansion-2026-07-15.md §3). Before piling ~12 more prompts and provider-routing call sites into audio_service.py, split the reusable transport layer out. Every new Workbench tool sits on generate_structured_text; it needs a clean home.

Approach

Purely mechanical move, no behavior change:

  • New webapp/backend/app/services/llm_service.py — move generate_structured_text (audio_service.py:1158-1221), the four _structured_* provider-adapter backends (Anthropic/OpenAI/Ollama/llama.cpp-OpenAI-compatible), and extract_json_object (tolerant JSON parsing). Pure transport: provider routing by URL, system_prompt, json_mode, max_tokens=None support for local reasoning models, configurable timeout.
  • Leave audio_service.py owning transcription + summarisation of audio sessions, plus its existing prompt constants and domain functions (lore extraction, names, storyline, Q&A) for now — those get progressively pulled into generation_service.py in issue 2 as tools are registered, not all at once here.
  • Add re-export shims in audio_service.py (from .llm_service import generate_structured_text, extract_json_object) so the ~10 existing call sites keep working without edits in this PR: reminder_tasks.py:1985/1997/2227/2494, lore_service.py:746, and others found via grep for generate_structured_text/extract_json_object.
  • No new endpoints, no schema changes, no prompt changes. This is scaffolding only.

Dependencies

None — this is the first foundational item. Issue 2 (generalized workbench endpoint + tool registry) builds directly on llm_service.py existing as a stable import target.

Out of scope

  • Any new generation tool or prompt.
  • Moving the lore/name/storyline/Q&A domain functions out of audio_service.py — that happens incrementally as tools are registered in generation_service.py (issue 2) and later tool issues, not in this mechanical split.
  • Removing the shims / updating call sites to import directly from llm_service — leave the shims in place indefinitely unless a later cleanup pass wants to do that separately.

Acceptance criteria

  • llm_service.py exists and owns generate_structured_text, all four provider adapters, and extract_json_object.
  • All existing call sites (name generation, lore extraction/dedup/match, TL;DR summaries, relationship proposals, storyline chapters, journal entries, Q&A) continue to work unchanged via re-export shims in audio_service.py.
  • No behavioral difference in any existing generation flow (name generator, lore drafts, journal entries, etc.) — verified by existing test suite passing unmodified.
  • No new endpoints or routes added.
## Motivation / Context `audio_service.py` currently contains transcription *and* the entire structured-LLM stack: provider adapters (`generate_structured_text` at `webapp/backend/app/services/audio_service.py:1158-1221` plus the four `_structured_*` backends), a dozen module-level system-prompt constants (`:41-228`), and domain functions for lore, names, storyline, and Q&A. The file is "audio" in name only. This is issue 1 of 3 foundational items behind the **GM Workbench** vision: turning GM Planning from one generator (names) into ~12 campaign-grounded generation tools (descriptions, backstories, loot, rumors, encounter seeds, improv NPCs, prep sheets, etc. — see the full catalog in `docs/.internal/gm-planning-expansion-2026-07-15.md` §3). Before piling ~12 more prompts and provider-routing call sites into `audio_service.py`, split the reusable transport layer out. Every new Workbench tool sits on `generate_structured_text`; it needs a clean home. ## Approach Purely mechanical move, no behavior change: - New `webapp/backend/app/services/llm_service.py` — move `generate_structured_text` (`audio_service.py:1158-1221`), the four `_structured_*` provider-adapter backends (Anthropic/OpenAI/Ollama/llama.cpp-OpenAI-compatible), and `extract_json_object` (tolerant JSON parsing). Pure transport: provider routing by URL, `system_prompt`, `json_mode`, `max_tokens=None` support for local reasoning models, configurable timeout. - Leave `audio_service.py` owning transcription + summarisation of audio sessions, plus its existing prompt constants and domain functions (lore extraction, names, storyline, Q&A) for now — those get progressively pulled into `generation_service.py` in issue 2 as tools are registered, not all at once here. - Add re-export shims in `audio_service.py` (`from .llm_service import generate_structured_text, extract_json_object`) so the ~10 existing call sites keep working without edits in this PR: `reminder_tasks.py:1985/1997/2227/2494`, `lore_service.py:746`, and others found via grep for `generate_structured_text`/`extract_json_object`. - No new endpoints, no schema changes, no prompt changes. This is scaffolding only. ## Dependencies None — this is the first foundational item. Issue 2 (generalized workbench endpoint + tool registry) builds directly on `llm_service.py` existing as a stable import target. ## Out of scope - Any new generation tool or prompt. - Moving the lore/name/storyline/Q&A domain functions out of `audio_service.py` — that happens incrementally as tools are registered in `generation_service.py` (issue 2) and later tool issues, not in this mechanical split. - Removing the shims / updating call sites to import directly from `llm_service` — leave the shims in place indefinitely unless a later cleanup pass wants to do that separately. ## Acceptance criteria - `llm_service.py` exists and owns `generate_structured_text`, all four provider adapters, and `extract_json_object`. - All existing call sites (name generation, lore extraction/dedup/match, TL;DR summaries, relationship proposals, storyline chapters, journal entries, Q&A) continue to work unchanged via re-export shims in `audio_service.py`. - No behavioral difference in any existing generation flow (name generator, lore drafts, journal entries, etc.) — verified by existing test suite passing unmodified. - No new endpoints or routes added.
Author
Contributor

Picking this up as the first foundational item of the v3.10.0 GM Workbench pillar, on feat/134-llm-service-extraction → PR onto the integration branch feat/v3.10-gm-workbench (per-issue PRs onto the integration branch, then one reviewed merge to main, same workflow as v3.9.0).

Purely mechanical: move the LLM transport layer (generate_structured_text, the four _structured_* provider adapters, extract_json_object) out of audio_service.py into a new llm_service.py, with re-export shims in audio_service.py so all existing call sites keep working unchanged. No behaviour change; verified by the existing suite passing unmodified. (Note: the issue's line numbers predate v3.9.0 — audio_service.py grew a lot with the stat-block work — so current locations will be found by grep.)

Picking this up as the first foundational item of the v3.10.0 GM Workbench pillar, on `feat/134-llm-service-extraction` → PR onto the integration branch `feat/v3.10-gm-workbench` (per-issue PRs onto the integration branch, then one reviewed merge to `main`, same workflow as v3.9.0). Purely mechanical: move the LLM transport layer (`generate_structured_text`, the four `_structured_*` provider adapters, `extract_json_object`) out of `audio_service.py` into a new `llm_service.py`, with re-export shims in `audio_service.py` so all existing call sites keep working unchanged. No behaviour change; verified by the existing suite passing unmodified. (Note: the issue's line numbers predate v3.9.0 — `audio_service.py` grew a lot with the stat-block work — so current locations will be found by grep.)
Author
Contributor

Done and verified — merged into the integration branch via PR #216.

Verification (Docker, py3.12): 659 backend tests pass, unmodified (no behaviour change; the LLM-mocking suites — convert_stats, highlights, statblock_schema_targeting, system_aware_prompts, session_title_suggestions — all still resolve generate_structured_text through the re-export shim). ruff check + ruff format --check clean (0.4.4).

Delivered: new services/llm_service.py owns the LLM transport (generate_structured_text, the four _structured_* provider adapters, extract_json_object, and _LLM_TIMEOUT/_is_ollama_url/_LORE_PROPOSAL_SYSTEM_PROMPT); audio_service re-exports them so every call site is unchanged; no circular import (audio_service.generate_structured_text is llm_service.generate_structured_text). Domain functions stay in audio_service.

Closing; ships to main with the v3.10.0 release. Next: #136 (generalized workbench endpoint + tool registry) and #138 (GenerationResult scratchpad), which build directly on this.

Done and verified — merged into the integration branch via PR #216. **Verification (Docker, py3.12):** **659 backend tests pass, unmodified** (no behaviour change; the LLM-mocking suites — convert_stats, highlights, statblock_schema_targeting, system_aware_prompts, session_title_suggestions — all still resolve `generate_structured_text` through the re-export shim). `ruff check` + `ruff format --check` clean (0.4.4). **Delivered:** new `services/llm_service.py` owns the LLM transport (`generate_structured_text`, the four `_structured_*` provider adapters, `extract_json_object`, and `_LLM_TIMEOUT`/`_is_ollama_url`/`_LORE_PROPOSAL_SYSTEM_PROMPT`); `audio_service` re-exports them so every call site is unchanged; no circular import (`audio_service.generate_structured_text is llm_service.generate_structured_text`). Domain functions stay in `audio_service`. Closing; ships to `main` with the v3.10.0 release. Next: #136 (generalized workbench endpoint + tool registry) and #138 (GenerationResult scratchpad), which build directly on this.
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#134
No description provided.