[Bot] Stop announcing failed transcriptions as "Session Summary Ready" #372

Closed
opened 2026-08-25 20:42:12 +00:00 by claude-bot · 2 comments
Contributor

Impact: CRITICAL

Found in the August 2026 session lifecycle review (#319).

What the user experiences

When a summary fails to generate, the Discord channel announces "📜 Session Summary Ready — Untitled Session" with body "No summary was generated." and seeds it with 👍/⚠️/ feedback reactions — inviting the GM to rate the accuracy of a summary that was never produced. The web app correctly shows "Processing failed" on the same session at the same time; only the Discord side lies about it.

Evidence

  • webapp/backend/app/tasks/reminder_tasks.py:2367-2373 sends the notification with extra={"error": ...} on the failure path.
  • bot/questboard_bot/cogs/notifications.py:675-697 (_handle_session_summarised) never reads extra["error"] — it always builds the success embed with title or "Untitled Session" and summary or "No summary was generated.", then seeds the accuracy-feedback reactions unconditionally.

Why it matters for a hosted product

This is the most misleading user-visible string anywhere in the product, on its flagship pipeline. A GM who trusts the Discord message believes the session was successfully summarised when it wasn't, and the feedback reactions actively solicit a rating for a nonexistent artifact.

Proposed fix

_handle_session_summarised must branch on extra.get("error"): when present, post a distinct failure embed (naming the session if a title is available) with a next action — e.g. "summary generation failed; a GM can retry from the session page" — and skip seeding the feedback reactions. This is the audit's P17.

Acceptance criteria

  • _handle_session_summarised checks extra["error"] and posts a failure-specific embed when it is set.
  • The failure embed does not claim the summary is ready and does not seed 👍/⚠️/ reactions.
  • The failure embed names the session (falling back gracefully, not to "Untitled Session", when no title exists) and gives a next action.
  • A test covers both the success and failure branches of _handle_session_summarised.
**Impact: CRITICAL** Found in the August 2026 session lifecycle review (#319). ## What the user experiences When a summary fails to generate, the Discord channel announces **"📜 Session Summary Ready — Untitled Session"** with body **"No summary was generated."** and seeds it with 👍/⚠️/❌ feedback reactions — inviting the GM to rate the accuracy of a summary that was never produced. The web app correctly shows "Processing failed" on the same session at the same time; only the Discord side lies about it. ## Evidence - `webapp/backend/app/tasks/reminder_tasks.py:2367-2373` sends the notification with `extra={"error": ...}` on the failure path. - `bot/questboard_bot/cogs/notifications.py:675-697` (`_handle_session_summarised`) never reads `extra["error"]` — it always builds the success embed with `title or "Untitled Session"` and `summary or "No summary was generated."`, then seeds the accuracy-feedback reactions unconditionally. ## Why it matters for a hosted product This is the most misleading user-visible string anywhere in the product, on its flagship pipeline. A GM who trusts the Discord message believes the session was successfully summarised when it wasn't, and the feedback reactions actively solicit a rating for a nonexistent artifact. ## Proposed fix `_handle_session_summarised` must branch on `extra.get("error")`: when present, post a distinct failure embed (naming the session if a title is available) with a next action — e.g. "summary generation failed; a GM can retry from the session page" — and skip seeding the feedback reactions. This is the audit's P17. ## Acceptance criteria - [ ] `_handle_session_summarised` checks `extra["error"]` and posts a failure-specific embed when it is set. - [ ] The failure embed does not claim the summary is ready and does not seed 👍/⚠️/❌ reactions. - [ ] The failure embed names the session (falling back gracefully, not to "Untitled Session", when no title exists) and gives a next action. - [ ] A test covers both the success and failure branches of `_handle_session_summarised`.
Author
Contributor

Picking this up as part of v4.3.0 phase 1 (#514), shipping early as v4.2.3. Lane: bot, together with #391 and #394 on one branch since they share cogs/notifications.py. The failure embed will name the session, give the retry path, and seed no feedback reactions; the raw error stays in the log.

Picking this up as part of v4.3.0 phase 1 (#514), shipping early as v4.2.3. Lane: bot, together with #391 and #394 on one branch since they share `cogs/notifications.py`. The failure embed will name the session, give the retry path, and seed no feedback reactions; the raw error stays in the log.
rbrooks referenced this issue from a commit 2026-09-06 02:00:15 +00:00
Author
Contributor

Fixed in PR #517 (merged), shipping in v4.2.3.

The backend fires session_summarised for both outcomes, with a summary or with an error and no summary, and the bot only knew the first shape. _handle_session_summarised now branches on extra["error"] before it reads summary, and a failure gets its own message: a red "⚠️ Summary Failed — " embed, a body saying the recording was kept so it can be run again, and a "What now?" field pointing at the retry on the session page (with the deep link from #391). No feedback reactions, and no transcript_feedback message mapping either, since a stray 👍 must not attach a vote to a summary that does not exist. The raw error is logged, never posted: it is an internal exception string and the channel is the whole party.

Naming the session needed the backend too: the failure payload was literally {"error": ...}, so the bot had nothing but "Untitled Session" to print. Both failure call sites in reminder_tasks.py now send the title (raw, so an empty title stays empty and the bot can fall through rather than printing a placeholder), the confirmed time, and the campaign name where it is already in scope. The bot falls back title → campaign + date → date → "your latest session".

One regression the full backend suite caught: test_provider_concurrency's fake session row is a partial SimpleNamespace with no title. The stub was completed with title=None, confirmed_time=None rather than making production code defensive; every real Session has both columns, and None/None is exactly the case the fallback chain exists for.

Additive payload fields; BOT_CONTRACT_VERSION unchanged. Six new bot tests cover both branches. Not observable on dev (no Discord token there); the next failed summary on prod will show the new embed.

Fixed in PR #517 (merged), shipping in v4.2.3. The backend fires `session_summarised` for both outcomes, with a `summary` or with an `error` and no summary, and the bot only knew the first shape. `_handle_session_summarised` now branches on `extra["error"]` before it reads `summary`, and a failure gets its own message: a red "⚠️ Summary Failed — <session>" embed, a body saying the recording was kept so it can be run again, and a "What now?" field pointing at the retry on the session page (with the deep link from #391). No feedback reactions, and no `transcript_feedback` message mapping either, since a stray 👍 must not attach a vote to a summary that does not exist. The raw error is logged, never posted: it is an internal exception string and the channel is the whole party. Naming the session needed the backend too: the failure payload was literally `{"error": ...}`, so the bot had nothing but "Untitled Session" to print. Both failure call sites in `reminder_tasks.py` now send the title (raw, so an empty title stays empty and the bot can fall through rather than printing a placeholder), the confirmed time, and the campaign name where it is already in scope. The bot falls back title → campaign + date → date → "your latest session". One regression the full backend suite caught: `test_provider_concurrency`'s fake session row is a partial `SimpleNamespace` with no `title`. The stub was completed with `title=None, confirmed_time=None` rather than making production code defensive; every real `Session` has both columns, and `None`/`None` is exactly the case the fallback chain exists for. Additive payload fields; `BOT_CONTRACT_VERSION` unchanged. Six new bot tests cover both branches. Not observable on dev (no Discord token there); the next failed summary on prod will show the new embed.
Sign in to join this conversation.
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
rbrooks/Quest-Board#372
No description provided.