[GM Workbench] Extract LLM core from audio_service into llm_service

claude-bot commented

2026-07-15 22:00:35 +00:00

Collaborator

Motivation / Context

audio_service.py currently contains transcription and the entire structured-LLM stack: provider adapters (generate_structured_text at webapp/backend/app/services/audio_service.py:1158-1221 plus the four _structured_* backends), a dozen module-level system-prompt constants (:41-228), and domain functions for lore, names, storyline, and Q&A. The file is "audio" in name only.

This is issue 1 of 3 foundational items behind the GM Workbench vision: turning GM Planning from one generator (names) into ~12 campaign-grounded generation tools (descriptions, backstories, loot, rumors, encounter seeds, improv NPCs, prep sheets, etc. — see the full catalog in docs/.internal/gm-planning-expansion-2026-07-15.md §3). Before piling ~12 more prompts and provider-routing call sites into audio_service.py, split the reusable transport layer out. Every new Workbench tool sits on generate_structured_text; it needs a clean home.

Approach

Purely mechanical move, no behavior change:

New webapp/backend/app/services/llm_service.py — move generate_structured_text (audio_service.py:1158-1221), the four _structured_* provider-adapter backends (Anthropic/OpenAI/Ollama/llama.cpp-OpenAI-compatible), and extract_json_object (tolerant JSON parsing). Pure transport: provider routing by URL, system_prompt, json_mode, max_tokens=None support for local reasoning models, configurable timeout.
Leave audio_service.py owning transcription + summarisation of audio sessions, plus its existing prompt constants and domain functions (lore extraction, names, storyline, Q&A) for now — those get progressively pulled into generation_service.py in issue 2 as tools are registered, not all at once here.
Add re-export shims in audio_service.py (from .llm_service import generate_structured_text, extract_json_object) so the ~10 existing call sites keep working without edits in this PR: reminder_tasks.py:1985/1997/2227/2494, lore_service.py:746, and others found via grep for generate_structured_text/extract_json_object.
No new endpoints, no schema changes, no prompt changes. This is scaffolding only.

Dependencies

None — this is the first foundational item. Issue 2 (generalized workbench endpoint + tool registry) builds directly on llm_service.py existing as a stable import target.

Out of scope

Any new generation tool or prompt.
Moving the lore/name/storyline/Q&A domain functions out of audio_service.py — that happens incrementally as tools are registered in generation_service.py (issue 2) and later tool issues, not in this mechanical split.
Removing the shims / updating call sites to import directly from llm_service — leave the shims in place indefinitely unless a later cleanup pass wants to do that separately.

Acceptance criteria

llm_service.py exists and owns generate_structured_text, all four provider adapters, and extract_json_object.
All existing call sites (name generation, lore extraction/dedup/match, TL;DR summaries, relationship proposals, storyline chapters, journal entries, Q&A) continue to work unchanged via re-export shims in audio_service.py.
No behavioral difference in any existing generation flow (name generator, lore drafts, journal entries, etc.) — verified by existing test suite passing unmodified.
No new endpoints or routes added.

## Motivation / Context `audio_service.py` currently contains transcription *and* the entire structured-LLM stack: provider adapters (`generate_structured_text` at `webapp/backend/app/services/audio_service.py:1158-1221` plus the four `_structured_*` backends), a dozen module-level system-prompt constants (`:41-228`), and domain functions for lore, names, storyline, and Q&A. The file is "audio" in name only. This is issue 1 of 3 foundational items behind the **GM Workbench** vision: turning GM Planning from one generator (names) into ~12 campaign-grounded generation tools (descriptions, backstories, loot, rumors, encounter seeds, improv NPCs, prep sheets, etc. — see the full catalog in `docs/.internal/gm-planning-expansion-2026-07-15.md` §3). Before piling ~12 more prompts and provider-routing call sites into `audio_service.py`, split the reusable transport layer out. Every new Workbench tool sits on `generate_structured_text`; it needs a clean home. ## Approach Purely mechanical move, no behavior change: - New `webapp/backend/app/services/llm_service.py` — move `generate_structured_text` (`audio_service.py:1158-1221`), the four `_structured_*` provider-adapter backends (Anthropic/OpenAI/Ollama/llama.cpp-OpenAI-compatible), and `extract_json_object` (tolerant JSON parsing). Pure transport: provider routing by URL, `system_prompt`, `json_mode`, `max_tokens=None` support for local reasoning models, configurable timeout. - Leave `audio_service.py` owning transcription + summarisation of audio sessions, plus its existing prompt constants and domain functions (lore extraction, names, storyline, Q&A) for now — those get progressively pulled into `generation_service.py` in issue 2 as tools are registered, not all at once here. - Add re-export shims in `audio_service.py` (`from .llm_service import generate_structured_text, extract_json_object`) so the ~10 existing call sites keep working without edits in this PR: `reminder_tasks.py:1985/1997/2227/2494`, `lore_service.py:746`, and others found via grep for `generate_structured_text`/`extract_json_object`. - No new endpoints, no schema changes, no prompt changes. This is scaffolding only. ## Dependencies None — this is the first foundational item. Issue 2 (generalized workbench endpoint + tool registry) builds directly on `llm_service.py` existing as a stable import target. ## Out of scope - Any new generation tool or prompt. - Moving the lore/name/storyline/Q&A domain functions out of `audio_service.py` — that happens incrementally as tools are registered in `generation_service.py` (issue 2) and later tool issues, not in this mechanical split. - Removing the shims / updating call sites to import directly from `llm_service` — leave the shims in place indefinitely unless a later cleanup pass wants to do that separately. ## Acceptance criteria - `llm_service.py` exists and owns `generate_structured_text`, all four provider adapters, and `extract_json_object`. - All existing call sites (name generation, lore extraction/dedup/match, TL;DR summaries, relationship proposals, storyline chapters, journal entries, Q&A) continue to work unchanged via re-export shims in `audio_service.py`. - No behavioral difference in any existing generation flow (name generator, lore drafts, journal entries, etc.) — verified by existing test suite passing unmodified. - No new endpoints or routes added.

claude-bot added this to the v3.10.0 - GM Workbench milestone

2026-07-15 22:00:38 +00:00

claude-bot added the

backend

enhancement

labels

Rows
Columns

[GM Workbench] Extract LLM core from audio_service into llm_service #134

Motivation / Context

Approach

Dependencies

Out of scope

Acceptance criteria