rbrooks/Quest-Board
Watch
1
0
Fork
You've already forked Quest-Board
 0
Code Issues 77 Pull requests 2 Projects Releases 42 Packages Wiki Activity Actions

[Game Systems] Opt-in conversion wizard: free-form to system with LLM-assisted stat backfill #145

New issue
Closed
opened 2026-07-15 22:02:48 +00:00 by claude-bot · 2 comments
claude-bot commented 2026-07-15 22:02:48 +00:00
Contributor
Copy link

Motivation/Context

Registry linkage (#133) and system-aware prompts (#137) work immediately for new
choices, but existing campaigns have years of free-form game_system text and loose
sidebar_fields rows describing NPC/creature/PC stats informally ("HP: 22", "AC" / "15", "Class" / "Rogue 3"). No automatic backfill was done in the storage migrations
(#133, #139) deliberately — free text is too dirty to convert silently, and a wrong
silent link would change LLM prompts and future Foundry behaviour (per the report,
§4.1). This issue is the last mile: a GM-driven, fully reversible, per-campaign wizard
that performs the actual migration, with the LLM doing the tedious mapping work and the
GM approving every change through the existing review rail.

Approach

GM-triggered from campaign settings, two independently skippable steps:

Step 1 — Link the system. Pick from the registry (already possible via #135's
selector, or the alias-match nudge). Writes game_system_id. Effect is immediate but
shallow: display strings, prompt hints (#137), and (later) Foundry adapter selection
now come from the registry. Nothing touches lore entries yet.

Step 2 — Backfill structured stats (optional, LLM-assisted, GM-reviewed). For each
npc/creature/player_character entry (opt-in per entry, or a "review all" queue):

  1. A Celery task (same worker pattern as run_lore_entry_draft_generation,
    webapp/backend/app/tasks/reminder_tasks.py:~2079) sends the entry body, existing
    loose sidebar_fields rows, and the target schema (from #139) to the LLM via
    generate_structured_text(json_mode=True), instructed to map, not invent: emit
    values only for stats actually present in the source material, plus a
    confidence/source note per field.
  2. The result enters the existing draft review rail (LoreEntryDraft,
    draft_service.py, review/iterate/approve endpoints
    webapp/backend/app/routers/campaigns.py:~2212-2389) as a new draft mode
    (convert_stats) — same UX as rephrase/merge and #130's modes: old vs. new,
    inline edit, iterate with feedback, approve or discard.
  3. On approval: write the stats envelope (per #139); never delete or rewrite
    the original sidebar_fields rows or body prose     — append-only, same philosophy as
    #130. A later manual cleanup of now-redundant sidebar rows is the GM's call; the
    review UI offers per-row "hide this sidebar row (superseded by stats)" checkboxes.
  4. Every approval snapshots a LoreEntryVersion (existing behaviour) — per-entry
    rollback is the existing version-restore path, no new rollback mechanism needed.

Rollback/unlink guarantee (§4.4 of the report): setting game_system_id = NULL
restores the exact pre-link display text (the free-text column was never touched).
Existing stats envelopes become dormant rather than breaking — they self-describe
{system, schema_version} so they render read-only with a "not the campaign's current
system" badge (per #140's edge-state handling) rather than erroring; the GM may delete
them per-entry. Re-linking the same system reactivates them.

Frontend: the wizard UI (step 1/step 2 flow, per-entry or "review all" queue,
draft review reusing #140's/existing draft-review components) lives alongside the
campaign settings UI from #135.

Dependencies

  • #139 (Versioned stat schemas + stats storage + validation service) — the wizard
    writes into the stats envelope this issue defines and validates against it.
  • #140 (Stat-block editor + infobox projection) — provides the read-only/dormant-state
    rendering this issue's rollback guarantee relies on, and the review surface pattern
    the convert_stats draft mode follows.

Out of scope

  • The registry, FK, and resolution helper (#133) and the selector/nudge UI (#135) —
    step 1 of this wizard consumes them, doesn't reimplement them.
  • Schema-targeted generation for new stat blocks (#142/issue 6) — this issue is about
    backfilling existing free-form data, not generating new content from scratch.
  • Automatic/silent backfill — every field written by this flow goes through GM
    approval; nothing is ever auto-applied.
  • Bulk "convert every campaign" tooling — this is a per-campaign, GM-initiated flow.

Acceptance criteria

  • GM can link a campaign to a registry system from the wizard (step 1) without
    affecting any lore entry.
  • GM can opt individual npc/creature/player_character entries (or "review all") into
    stat backfill (step 2); each produces a convert_stats draft via the Celery task
    pattern used by existing draft generation.
  • The LLM mapping prompt only emits values for stats actually present in source
    material (body + sidebar_fields), with per-field confidence/source metadata surfaced
    in the review UI.
  • Draft review for convert_stats uses the existing review rail (old vs. new, inline
    edit, iterate, approve/discard) — no new review UI pattern invented.
  • Approval writes the stats envelope and snapshots a LoreEntryVersion; the original
    sidebar_fields rows and body prose are never deleted or rewritten by this flow.
  • Setting game_system_id = NULL on a converted campaign restores the original
    free-text display exactly, and existing stats envelopes become read-only/dormant
    (not deleted, not erroring) until re-linked.
  • Declining or skipping either step leaves the campaign fully functional in its
    pre-wizard state.
## Motivation/Context Registry linkage (#133) and system-aware prompts (#137) work immediately for new choices, but existing campaigns have years of free-form `game_system` text and loose `sidebar_fields` rows describing NPC/creature/PC stats informally (`"HP: 22"`, `"AC" / "15"`, `"Class" / "Rogue 3"`). No automatic backfill was done in the storage migrations (#133, #139) deliberately — free text is too dirty to convert silently, and a wrong silent link would change LLM prompts and future Foundry behaviour (per the report, §4.1). This issue is the last mile: a GM-driven, fully reversible, per-campaign wizard that performs the actual migration, with the LLM doing the tedious mapping work and the GM approving every change through the existing review rail. ## Approach GM-triggered from campaign settings, two independently skippable steps: **Step 1 — Link the system.** Pick from the registry (already possible via #135's selector, or the alias-match nudge). Writes `game_system_id`. Effect is immediate but shallow: display strings, prompt hints (#137), and (later) Foundry adapter selection now come from the registry. Nothing touches lore entries yet. **Step 2 — Backfill structured stats (optional, LLM-assisted, GM-reviewed).** For each `npc`/`creature`/`player_character` entry (opt-in per entry, or a "review all" queue): 1. A Celery task (same worker pattern as `run_lore_entry_draft_generation`, `webapp/backend/app/tasks/reminder_tasks.py:~2079`) sends the entry `body`, existing loose `sidebar_fields` rows, and the target schema (from #139) to the LLM via `generate_structured_text(json_mode=True)`, instructed to **map, not invent**: emit `values` only for stats actually present in the source material, plus a `confidence`/`source` note per field. 2. The result enters the **existing draft review rail** (`LoreEntryDraft`, `draft_service.py`, review/iterate/approve endpoints `webapp/backend/app/routers/campaigns.py:~2212-2389`) as a new draft mode (`convert_stats`) — same UX as `rephrase`/`merge` and #130's modes: old vs. new, inline edit, iterate with feedback, approve or discard. 3. **On approval:** write the `stats` envelope (per #139); **never delete or rewrite the original `sidebar_fields` rows or body prose** — append-only, same philosophy as #130. A later manual cleanup of now-redundant sidebar rows is the GM's call; the review UI offers per-row "hide this sidebar row (superseded by stats)" checkboxes. 4. Every approval snapshots a `LoreEntryVersion` (existing behaviour) — per-entry rollback is the existing version-restore path, no new rollback mechanism needed. **Rollback/unlink guarantee (§4.4 of the report):** setting `game_system_id = NULL` restores the exact pre-link display text (the free-text column was never touched). Existing `stats` envelopes become dormant rather than breaking — they self-describe `{system, schema_version}` so they render read-only with a "not the campaign's current system" badge (per #140's edge-state handling) rather than erroring; the GM may delete them per-entry. Re-linking the same system reactivates them. **Frontend:** the wizard UI (step 1/step 2 flow, per-entry or "review all" queue, draft review reusing #140's/existing draft-review components) lives alongside the campaign settings UI from #135. ## Dependencies - #139 (Versioned stat schemas + `stats` storage + validation service) — the wizard writes into the `stats` envelope this issue defines and validates against it. - #140 (Stat-block editor + infobox projection) — provides the read-only/dormant-state rendering this issue's rollback guarantee relies on, and the review surface pattern the `convert_stats` draft mode follows. ## Out of scope - The registry, FK, and resolution helper (#133) and the selector/nudge UI (#135) — step 1 of this wizard consumes them, doesn't reimplement them. - Schema-targeted generation for *new* stat blocks (#142/issue 6) — this issue is about backfilling *existing* free-form data, not generating new content from scratch. - Automatic/silent backfill — every field written by this flow goes through GM approval; nothing is ever auto-applied. - Bulk "convert every campaign" tooling — this is a per-campaign, GM-initiated flow. ## Acceptance criteria - GM can link a campaign to a registry system from the wizard (step 1) without affecting any lore entry. - GM can opt individual npc/creature/player_character entries (or "review all") into stat backfill (step 2); each produces a `convert_stats` draft via the Celery task pattern used by existing draft generation. - The LLM mapping prompt only emits values for stats actually present in source material (body + sidebar_fields), with per-field confidence/source metadata surfaced in the review UI. - Draft review for `convert_stats` uses the existing review rail (old vs. new, inline edit, iterate, approve/discard) — no new review UI pattern invented. - Approval writes the `stats` envelope and snapshots a `LoreEntryVersion`; the original `sidebar_fields` rows and `body` prose are never deleted or rewritten by this flow. - Setting `game_system_id = NULL` on a converted campaign restores the original free-text display exactly, and existing `stats` envelopes become read-only/dormant (not deleted, not erroring) until re-linked. - Declining or skipping either step leaves the campaign fully functional in its pre-wizard state.
claude-bot commented 2026-07-18 17:58:58 +00:00
Author
Contributor
Copy link

Picking this up as the final issue of the v3.9.0 pillar, on feat/145-conversion-wizard (PR to target the integration branch). Doing it in two passes on one branch — backend now, then the frontend wizard/review UI — merged as a single PR.

Backend approach: a new convert_stats draft mode that LLM-maps an entry's existing free-form data (body + loose sidebar_fields) into schema-validated structured stats (map, not invent; per-field confidence/source surfaced in review), entering the existing draft-review rail. Much of the plumbing is already in place from #142 — the lore_entry_drafts.current_stats column and the approval path that writes a re-validated envelope to entry.stats. This flow stages current_stats while leaving current_sidebar_fields null and current_body unchanged, so approval is append-only — the original sidebar rows and prose are never deleted or rewritten (per the issue's §4.1/rollback guarantees). Adds a "review all" queue endpoint for eligible npc/creature entries. Step 1 (linking a system) is already handled by #135; unlink/dormant-stats rollback by #133/#140. New enum value → migration via Alembic's autocommit block. No bot API change.

Picking this up as the final issue of the v3.9.0 pillar, on `feat/145-conversion-wizard` (PR to target the integration branch). Doing it in two passes on one branch — **backend now**, then the frontend wizard/review UI — merged as a single PR. Backend approach: a new `convert_stats` draft mode that LLM-**maps** an entry's existing free-form data (body + loose `sidebar_fields`) into schema-validated structured `stats` (map, not invent; per-field confidence/source surfaced in review), entering the existing draft-review rail. Much of the plumbing is already in place from #142 — the `lore_entry_drafts.current_stats` column and the approval path that writes a re-validated envelope to `entry.stats`. This flow stages `current_stats` while leaving `current_sidebar_fields` null and `current_body` unchanged, so approval is **append-only** — the original sidebar rows and prose are never deleted or rewritten (per the issue's §4.1/rollback guarantees). Adds a "review all" queue endpoint for eligible npc/creature entries. Step 1 (linking a system) is already handled by #135; unlink/dormant-stats rollback by #133/#140. New enum value → migration via Alembic's autocommit block. No bot API change.
claude-bot commented 2026-07-18 18:51:06 +00:00
Author
Contributor
Copy link

Done and verified — merged into the integration branch via PR #211 (backend + frontend, two commits).

Verification (Docker):

  • Backend: 654 tests pass (py3.12; new test_convert_stats.py), no regressions. Migration e9f0a1b2c3d4 up/down/up clean (enum value + current_stats_notes column).
  • Frontend: 338 tests pass (32 files; new wizard + convert-review suites) + production build succeeds (node:20-alpine).

Delivered: convert_stats draft mode that LLM-maps existing body + loose sidebar_fields into schema-validated stats (map-not-invent, per-field confidence/source, retry-once-then-fail), reusing #142's staging + append-only approval (original prose/sidebar rows never rewritten); a GM-only eligibility queue endpoint; the CampaignDetail wizard card + the WikiDraftReview convert_stats branch (prose read-only, mapped stats with schema labels + confidence · source).

Latent #139 fix included: LoreEntry/LoreEntryVersion.stats now use none_as_null=True so a Python None stores as SQL NULL (not the JSON scalar 'null') — surfaced by the queue's IS NULL filter; also fixes #140's "delete stat block" round-trip.

Closing. This completes the v3.9.0 Game-Aware Systems pillar (7/7 issues). Next: the release (version bump + changelog) and the integration → main PR.

Done and verified — merged into the integration branch via PR #211 (backend + frontend, two commits). **Verification (Docker):** - Backend: **654 tests pass** (py3.12; new `test_convert_stats.py`), no regressions. Migration `e9f0a1b2c3d4` up/down/up clean (enum value + `current_stats_notes` column). - Frontend: **338 tests pass** (32 files; new wizard + convert-review suites) + production build succeeds (`node:20-alpine`). **Delivered:** `convert_stats` draft mode that LLM-maps existing body + loose sidebar_fields into schema-validated `stats` (map-not-invent, per-field confidence/source, retry-once-then-fail), reusing #142's staging + append-only approval (original prose/sidebar rows never rewritten); a GM-only eligibility queue endpoint; the CampaignDetail wizard card + the WikiDraftReview `convert_stats` branch (prose read-only, mapped stats with schema labels + `confidence · source`). **Latent #139 fix included:** `LoreEntry`/`LoreEntryVersion.stats` now use `none_as_null=True` so a Python `None` stores as SQL `NULL` (not the JSON scalar `'null'`) — surfaced by the queue's `IS NULL` filter; also fixes #140's "delete stat block" round-trip. Closing. **This completes the v3.9.0 Game-Aware Systems pillar (7/7 issues).** Next: the release (version bump + changelog) and the integration → `main` PR.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
ui/table-notes-label
renovate/npm-vitest-vulnerability
chore/87-align-python-3.12
v4.3.0
v4.2.3
v4.2.2
v4.2.1
v4.2.0
v4.1.2
v4.1.1
v4.1.0
v4.0.1
v4.0.0
v3.11.4
v3.11.3
v3.11.2
v3.11.1
v3.11.0
v3.10.0
v3.9.1
v3.9.0
v3.8.0
v3.7.0
v3.6.1
v3.6.0
v3.5.0
v3.4.1
v3.4.0
v3.3.0
v3.2.0
v3.1.0
v3.0.0
v2.1.0
v2.0.0
v1.1.0
v1.0.0
v0.12.0
v0.11.0
v0.10.0
v0.9.1
v0.3.3
v0.9.0
v0.8.0
v0.3.2
v0.3.1
v0.3.0
v0.7.0
v0.6.0
v0.5.0
v0.4.0
v0.2.1
v0.2.0
v0.1.1
v0.1.0
Labels
Clear labels   
backend
Issue related to the backend

  
bot
Issue related to the bot

  
bug
Something is not working

  
duplicate
This issue or pull request already exists

  
enhancement
New feature

  
frontend
Issue related to the frontend

  
help wanted
Need some help

  
invalid
Something is wrong

  
question
More information is needed

  
recording
Issue related to the recording system

  
security
Security hardening / vulnerability work

  
wontfix
This won't be fixed

No labels
backend
bot
bug
duplicate
enhancement
frontend
help wanted
invalid
question
recording
security
wontfix
Milestone
Clear milestone
No items
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#145
No description provided.