[Bot] Make the bot self-diagnosing: /help, guild-join welcome, and real error strings instead of a generic "try again" #373

Closed
opened 2026-08-25 20:42:13 +00:00 by claude-bot · 2 comments
Contributor

Impact: HIGH

Found in the August 2026 session lifecycle review (#319).

What the user experiences

A customer invites the bot to their server. There is no /help, no welcome message, no presence string, and no in-Discord discovery surface of any kind — every interactive surface is an emoji reaction, and commands sync globally so they can take up to an hour to even appear, with nothing telling the user that. Once commands do appear, /next, /quests, and /history give the exact same reply for "no campaign is bound to this Discord server" as for "a campaign is bound but nothing is scheduled" — so a customer who never wired up the campaign-to-server link gets told, forever, "No upcoming sessions are scheduled for this server," with nothing naming the actual missing configuration. And whenever the backend does return something diagnosable — like "LLM endpoint not configured. Set it in Admin → Bot Settings" — the bot discards it and shows a generic "Could not reach Quest Board right now. Please try again shortly.", presenting a permanent misconfiguration as a transient outage.

Evidence

  • No /help command, no on_guild_join handler, no presence/activity string, and no discord.ui views/buttons/modals anywhere in bot/ (grep confirms zero results).
  • bot/questboard_bot/main.py:133await self.tree.sync() with no guild=, meaning up to an hour before commands appear after invite, with nothing telling the user that.
  • bot/questboard_bot/cogs/sessions.py:72, :130, :261/next, /quests, /history collapse "no campaign bound to this guild" and "nothing scheduled" into one message.
  • bot/questboard_bot/cogs/recording.py:62-65 — the /record start unauthorised message never mentions /link, even though GET /bot/guilds/{id}/gms (webapp/backend/app/routers/bot.py:617-624) only returns GMs with a verified link, making this the most likely failure mode for a new GM.
  • Grep for detail in bot/questboard_bot/ finds only responses the bot itself emits — api_client calls raise_for_status() (e.g. :345) and cogs catch bare Exception, so every 4xx/5xx becomes "Could not reach Quest Board right now. Please try again shortly." (bot/questboard_bot/cogs/sessions.py:65,123,322,422), including the LLM-not-configured detail from webapp/backend/app/routers/bot.py:1139-1143.
  • /ask's helpful "no recordings yet" branch (sessions.py:431-436) is dead code, because the backend always returns a non-empty placeholder string (routers/bot.py:1157-1161).

Why it matters for a hosted product

A customer invites the bot, types /next, and is told nothing is scheduled — forever — with no way to learn that the actual problem is an unlinked Discord server, an unlinked GM account, or a backend setting the owner never touched. Every one of these becomes a "the bot doesn't work" support message that a better error string would have prevented.

Proposed fix

Add /help and an on_guild_join welcome post explaining the campaign-linking step. Move command sync to guild-scoped where practical, or at minimum post a "commands may take up to an hour to appear" note on join. Distinguish "no campaign bound to this guild" from "nothing scheduled" in /next, /quests, /history, naming the missing config in the former case. Have api_client parse and re-raise the backend's detail string so cogs can show it verbatim instead of the generic fallback (the audit's P19, P20).

Acceptance criteria

  • A /help command exists and lists all available commands with a one-line description each.
  • on_guild_join posts a welcome message naming the campaign-linking step.
  • /next, /quests, /history give visibly different messages for "no campaign bound to this guild" vs. "nothing scheduled".
  • The /record start unauthorised message names /link as a possible fix.
  • api_client surfaces the backend's detail string to the cog layer, and at least the LLM-not-configured and campaign-not-linked cases show it instead of the generic "try again" message.
**Impact: HIGH** Found in the August 2026 session lifecycle review (#319). ## What the user experiences A customer invites the bot to their server. There is no `/help`, no welcome message, no presence string, and no in-Discord discovery surface of any kind — every interactive surface is an emoji reaction, and commands sync globally so they can take up to an hour to even appear, with nothing telling the user that. Once commands do appear, `/next`, `/quests`, and `/history` give the exact same reply for "no campaign is bound to this Discord server" as for "a campaign is bound but nothing is scheduled" — so a customer who never wired up the campaign-to-server link gets told, forever, "No upcoming sessions are scheduled for this server," with nothing naming the actual missing configuration. And whenever the backend does return something diagnosable — like *"LLM endpoint not configured. Set it in Admin → Bot Settings"* — the bot discards it and shows a generic **"Could not reach Quest Board right now. Please try again shortly."**, presenting a permanent misconfiguration as a transient outage. ## Evidence - No `/help` command, no `on_guild_join` handler, no presence/activity string, and no `discord.ui` views/buttons/modals anywhere in `bot/` (grep confirms zero results). - `bot/questboard_bot/main.py:133` — `await self.tree.sync()` with no `guild=`, meaning up to an hour before commands appear after invite, with nothing telling the user that. - `bot/questboard_bot/cogs/sessions.py:72`, `:130`, `:261` — `/next`, `/quests`, `/history` collapse "no campaign bound to this guild" and "nothing scheduled" into one message. - `bot/questboard_bot/cogs/recording.py:62-65` — the `/record start` unauthorised message never mentions `/link`, even though `GET /bot/guilds/{id}/gms` (`webapp/backend/app/routers/bot.py:617-624`) only returns GMs with a *verified* link, making this the most likely failure mode for a new GM. - Grep for `detail` in `bot/questboard_bot/` finds only responses the bot itself emits — `api_client` calls `raise_for_status()` (e.g. `:345`) and cogs catch bare `Exception`, so every 4xx/5xx becomes **"Could not reach Quest Board right now. Please try again shortly."** (`bot/questboard_bot/cogs/sessions.py:65,123,322,422`), including the LLM-not-configured detail from `webapp/backend/app/routers/bot.py:1139-1143`. - `/ask`'s helpful "no recordings yet" branch (`sessions.py:431-436`) is dead code, because the backend always returns a non-empty placeholder string (`routers/bot.py:1157-1161`). ## Why it matters for a hosted product A customer invites the bot, types `/next`, and is told nothing is scheduled — forever — with no way to learn that the actual problem is an unlinked Discord server, an unlinked GM account, or a backend setting the owner never touched. Every one of these becomes a "the bot doesn't work" support message that a better error string would have prevented. ## Proposed fix Add `/help` and an `on_guild_join` welcome post explaining the campaign-linking step. Move command sync to guild-scoped where practical, or at minimum post a "commands may take up to an hour to appear" note on join. Distinguish "no campaign bound to this guild" from "nothing scheduled" in `/next`, `/quests`, `/history`, naming the missing config in the former case. Have `api_client` parse and re-raise the backend's `detail` string so cogs can show it verbatim instead of the generic fallback (the audit's P19, P20). ## Acceptance criteria - [ ] A `/help` command exists and lists all available commands with a one-line description each. - [ ] `on_guild_join` posts a welcome message naming the campaign-linking step. - [ ] `/next`, `/quests`, `/history` give visibly different messages for "no campaign bound to this guild" vs. "nothing scheduled". - [ ] The `/record start` unauthorised message names `/link` as a possible fix. - [ ] `api_client` surfaces the backend's `detail` string to the cog layer, and at least the LLM-not-configured and campaign-not-linked cases show it instead of the generic "try again" message.
Author
Contributor

Picking this up as v4.3.0 phase 5, lane B (#514), followed by #393 and #392 on the same branch. Decisions: api_client raises a typed error carrying the backend's detail, and cogs show it verbatim for 4xx; /help, a guild-join welcome naming the linking step and the command-sync delay, and a presence string; "no campaign bound" and "nothing scheduled" become different sentences; the dead /ask branch gets an explicit signal from the backend instead of a placeholder string.

Picking this up as v4.3.0 phase 5, lane B (#514), followed by #393 and #392 on the same branch. Decisions: `api_client` raises a typed error carrying the backend's `detail`, and cogs show it verbatim for 4xx; `/help`, a guild-join welcome naming the linking step and the command-sync delay, and a presence string; "no campaign bound" and "nothing scheduled" become different sentences; the dead `/ask` branch gets an explicit signal from the backend instead of a placeholder string.
Author
Contributor

Done in PR #521 (merged); ships with v4.3.0.

api_client now raises QuestBoardApiError(status, detail) from the response body, subclassing httpx.HTTPStatusError so the startup handshake and existing handlers are untouched. user_message_for() shows the backend's own sentence for a 4xx or a 503, and the generic "try again shortly" for a 500 or a network failure. That is a small departure from "4xx only": the acceptance case here, "LLM endpoint not configured. Set it in Admin → Bot Settings.", is a 503, and on this backend a 503 always names an unconfigured or unavailable dependency in actionable prose, while a 500's detail is FastAPI boilerplate. Both branches are tested.

/next, /quests, /history could not distinguish the two states at all: every guild endpoint reports an unlinked guild and an idle one identically. Added GET /api/bot/guilds/{id}/campaigns (additive; empty list rather than 404, because an unconfigured server is ordinary). It is consulted only on the empty path, and a failed check keeps the old wording rather than telling a working server it is misconfigured. The message names the Discord Server ID and quotes the guild's own.

/ask's dead branch: the backend now sets reason: "no_transcripts" (additive) and keeps the prose, so older bots are unaffected; making answer null would have crashed a pre-#373 bot on .strip().

/record start's refusal now says Quest Board only recognises a GM whose Discord account is linked, and names /link.

New Help cog: /help lists all commands (a test asserts the list covers every registered command); on_guild_join posts the three setup steps, link a campaign (guild ID pasted in), everyone runs /link, then /help, and warns that Discord takes up to an hour to roll out global slash commands; the presence reads "Watching Quest Board · /help" and is set in the constructor so it survives reconnects. Sync stays global.

Also folded in the prod finding from the v4.2.3 deploy: command_prefix is now commands.when_mentioned, the only prefix discord.py exempts from "Privileged message content intent is missing", with a test so the noise cannot return.

No contract bump (all additive). Bot 398 tests pass, backend 2378, frontend 514, ruff and version-sync clean.

Done in PR #521 (merged); ships with v4.3.0. **`api_client` now raises `QuestBoardApiError(status, detail)`** from the response body, subclassing `httpx.HTTPStatusError` so the startup handshake and existing handlers are untouched. `user_message_for()` shows the backend's own sentence for a **4xx or a 503**, and the generic "try again shortly" for a 500 or a network failure. That is a small departure from "4xx only": the acceptance case here, "LLM endpoint not configured. Set it in Admin → Bot Settings.", is a 503, and on this backend a 503 always names an unconfigured or unavailable dependency in actionable prose, while a 500's detail is FastAPI boilerplate. Both branches are tested. **`/next`, `/quests`, `/history`** could not distinguish the two states at all: every guild endpoint reports an unlinked guild and an idle one identically. Added `GET /api/bot/guilds/{id}/campaigns` (additive; empty list rather than 404, because an unconfigured server is ordinary). It is consulted only on the empty path, and a failed check keeps the old wording rather than telling a working server it is misconfigured. The message names the Discord Server ID and quotes the guild's own. **`/ask`'s dead branch:** the backend now sets `reason: "no_transcripts"` (additive) and keeps the prose, so older bots are unaffected; making `answer` null would have crashed a pre-#373 bot on `.strip()`. **`/record start`'s refusal** now says Quest Board only recognises a GM whose Discord account is linked, and names `/link`. **New `Help` cog:** `/help` lists all commands (a test asserts the list covers every registered command); `on_guild_join` posts the three setup steps, link a campaign (guild ID pasted in), everyone runs `/link`, then `/help`, and warns that Discord takes up to an hour to roll out global slash commands; the presence reads "Watching Quest Board · /help" and is set in the constructor so it survives reconnects. Sync stays global. Also folded in the prod finding from the v4.2.3 deploy: `command_prefix` is now `commands.when_mentioned`, the only prefix discord.py exempts from "Privileged message content intent is missing", with a test so the noise cannot return. No contract bump (all additive). Bot 398 tests pass, backend 2378, frontend 514, ruff and version-sync clean.
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#373
No description provided.