Epic: The Shelf Listens — transcript-driven auto-surface #237

New issue

Open

opened 2026-07-21 21:03:38 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-07-21 21:03:38 +00:00

Contributor

What — the moat

The transcript-driven layer on top of the Session Reference Shelf (v3.11, manual spine). Quest Board uniquely already has the bot capturing + transcribing the live table plus an entity-rich wiki graph — so the shelf can hear the game and surface the right reference before the GM reaches for it. All three brainstorm models (Sonnet / Opus / Fable) independently named this the single boldest bet and the un-clonable differentiator.

Capabilities

Auto-surface on mention — live transcript → entity match against the wiki index → the matching card floats into a fixed "just mentioned" tray (peripheral, non-modal, one-tap accept). The headline.
Graph ghost-neighbours — surfacing/pinning an entry pre-fetches its one-hop wiki-graph neighbours (faction, home, linked threads, associates) as faded cards. (Fable)
Open-loop catcher — the transcript extracts unresolved threads live ("you promised the blacksmith a favour") into a dangling-threads card that seeds next session's prep. (Opus)
Continuity guard — flag contradictions vs. canon, but batched into the end-of-session sweep, NOT surfaced live (all three models flagged live fact-checking as harmful to the performer).
Hands-free voice spotlight ("show me the harbormaster") — stretch; needs a wake-word + tiny fast ASR, a different path from entity auto-surface.

Hard dependency

The current pipeline transcribes post-session only. This layer needs near-real-time transcription — see the dedicated spike (Spike: live/streaming transcription — options & viability): multiple options, scored on accuracy AND speed, before committing.

Guardrails (consensus)

Entity match is a cheap fuzzy lookup, never an LLM call in the hot loop.
Proactive output is silent, non-modal, confidence-gated, instantly dismissible — never audio, and only ever adds to a tray (never moves the GM's cards).
The manual shelf (v3.11) must remain fully functional with the LLM / GPU / ASR server down — the listening layer is additive, never a dependency of the core.

🤖 Generated with Claude Code

## What — the moat The **transcript-driven layer** on top of the Session Reference Shelf (v3.11, manual spine). Quest Board uniquely already has the bot **capturing + transcribing the live table** plus an entity-rich wiki graph — so the shelf can *hear the game* and surface the right reference before the GM reaches for it. All three brainstorm models (Sonnet / Opus / Fable) **independently named this the single boldest bet** and the un-clonable differentiator. ## Capabilities - [ ] **Auto-surface on mention** — live transcript → entity match against the wiki index → the matching card floats into a fixed "just mentioned" tray (peripheral, non-modal, one-tap accept). *The headline.* - [ ] **Graph ghost-neighbours** — surfacing/pinning an entry pre-fetches its one-hop wiki-graph neighbours (faction, home, linked threads, associates) as faded cards. *(Fable)* - [ ] **Open-loop catcher** — the transcript extracts unresolved threads live ("you promised the blacksmith a favour") into a dangling-threads card that seeds next session's prep. *(Opus)* - [ ] **Continuity guard** — flag contradictions vs. canon, but **batched into the end-of-session sweep, NOT surfaced live** (all three models flagged live fact-checking as harmful to the performer). - [ ] **Hands-free voice spotlight** ("show me the harbormaster") — **stretch**; needs a wake-word + tiny fast ASR, a different path from entity auto-surface. ## Hard dependency - The current pipeline transcribes **post-session only**. This layer needs near-real-time transcription — see the dedicated spike *(Spike: live/streaming transcription — options & viability)*: multiple options, scored on **accuracy AND speed**, before committing. ## Guardrails (consensus) - Entity match is a **cheap fuzzy lookup, never an LLM call** in the hot loop. - Proactive output is **silent, non-modal, confidence-gated, instantly dismissible — never audio**, and only ever *adds* to a tray (never moves the GM's cards). - The manual shelf (v3.11) must remain **fully functional with the LLM / GPU / ASR server down** — the listening layer is additive, never a dependency of the core. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added this to the (deleted) milestone

2026-07-21 21:03:57 +00:00

claude-bot referenced this issue

2026-07-27 18:48:26 +00:00

Epic: Session Reference Shelf — manual spine #235

claude-bot commented

2026-07-27 22:52:44 +00:00

Author

Contributor

Deferred to the backlog (2026-07-27)

Moving this and the #238 spike to Backlog — Speculative / Long-Term, and retiring the v3.12.0 milestone. The v3 pillar (collaborative GM/player tooling) closes at v3.11.0; the next versioned milestone is v4.0.0, which opens the FoundryVTT pillar.

This is a deferral, not a cancellation — the reasoning, so picking it back up doesn't start cold:

1. The spike needs its own session. #238 carries that instruction in its own body. It's measurement work — five transcription approaches scored on latency, WER, entity precision/recall, and GPU contention, on the real GPU box with representative TTRPG audio. Half-measured numbers would be worse than none, because this milestone's whole architecture gets committed on them.

2. There's an unmet prerequisite. The dev bot logs Discord bot disabled because DISCORD_BOT_TOKEN is not configured — discovered during the v3.11.0 deploy. This milestone is entirely a bot-audio feature, so right now the capture path can only be exercised in production. A dev Discord app + test guild should land before serious work starts here.

3. The surface it builds on is unproven in play. v3.11's shelf is deployed but has never been used in a real session. This layer auto-surfaces cards into that shelf; if a session shows the glance faces or scene model want reshaping, that changes what auto-surface should do. Better to build on a proven surface than an assumed one.

Preserved verbatim from the retired v3.12.0 milestone description, since it holds the strategic framing:

P3. The shelf listens: the transcript-driven layer on top of the Session Reference Shelf (v3.11). The live per-speaker audio feed auto-surfaces the right wiki card when an entity is named, catches open loops, and pulls in graph neighbours, turning the recording pipeline from a post-session archive into a live co-GM. Depends on new live/streaming transcription infra (dedicated spike required: multiple options evaluated on accuracy vs speed). The un-clonable moat.

Also carried forward from the v3.11 epic (#235) — two things this layer should reuse when it resumes:

Auto-surfaced cards belong in a separate tray, not merged into the board. The shelf's "never auto-move the GM's cards" guardrail means anything proactive only adds. list_shelf is the natural insertion point, but the output must not disturb board layout.
_REVEAL_DISPATCH in shelf_service.py is the pattern for anything fanning out to multiple destinations — a name-to-dispatcher registry, so a new surface is one entry rather than a new branch.

To resume: recreate a versioned milestone (v3.12.0 if the v3 line reopens, otherwise renumber), pull both issues back onto it, and start with #238.

🤖 Generated with Claude Code

### Deferred to the backlog (2026-07-27) Moving this and the #238 spike to **Backlog — Speculative / Long-Term**, and retiring the v3.12.0 milestone. The v3 pillar (collaborative GM/player tooling) closes at **v3.11.0**; the next versioned milestone is v4.0.0, which opens the FoundryVTT pillar. This is a deferral, not a cancellation — the reasoning, so picking it back up doesn't start cold: **1. The spike needs its own session.** [#238](https://git.rhoving.com/rbrooks/Quest-Board/issues/238) carries that instruction in its own body. It's measurement work — five transcription approaches scored on latency, WER, entity precision/recall, and GPU contention, on the real GPU box with representative TTRPG audio. Half-measured numbers would be worse than none, because this milestone's whole architecture gets committed on them. **2. There's an unmet prerequisite.** The dev bot logs `Discord bot disabled because DISCORD_BOT_TOKEN is not configured` — discovered during the v3.11.0 deploy. This milestone is *entirely* a bot-audio feature, so right now the capture path can only be exercised in production. A dev Discord app + test guild should land before serious work starts here. **3. The surface it builds on is unproven in play.** v3.11's shelf is deployed but has never been used in a real session. This layer auto-surfaces cards *into* that shelf; if a session shows the glance faces or scene model want reshaping, that changes what auto-surface should do. Better to build on a proven surface than an assumed one. --- **Preserved verbatim from the retired v3.12.0 milestone description**, since it holds the strategic framing: > P3. The shelf listens: the transcript-driven layer on top of the Session Reference Shelf (v3.11). The live per-speaker audio feed auto-surfaces the right wiki card when an entity is named, catches open loops, and pulls in graph neighbours, turning the recording pipeline from a post-session archive into a live co-GM. Depends on new live/streaming transcription infra (dedicated spike required: multiple options evaluated on accuracy vs speed). The un-clonable moat. Also carried forward from the v3.11 epic ([#235](https://git.rhoving.com/rbrooks/Quest-Board/issues/235)) — two things this layer should reuse when it resumes: - **Auto-surfaced cards belong in a separate tray, not merged into the board.** The shelf's "never auto-move the GM's cards" guardrail means anything proactive only *adds*. `list_shelf` is the natural insertion point, but the output must not disturb board layout. - **`_REVEAL_DISPATCH` in `shelf_service.py` is the pattern for anything fanning out to multiple destinations** — a name-to-dispatcher registry, so a new surface is one entry rather than a new branch. To resume: recreate a versioned milestone (v3.12.0 if the v3 line reopens, otherwise renumber), pull both issues back onto it, and start with #238. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot modified the milestone from (deleted) to Backlog - Speculative / Long-Term

2026-07-27 22:52:53 +00:00

claude-bot referenced this issue

2026-07-27 23:14:21 +00:00

docs: sweep for v3.11.0 and correct stale guidance #270