Loading…
Reference in a new issue
No description provided.
Delete branch "fix/572-summary-pipeline"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #566, #567, #572, #575. One commit per issue.
#566 — beats reach compose in time order.
beat_service.in_time_orderis the one sort key;render_beats_for_composeandrender_summary_anchorssort at the door andgather_validated_beatssorts after every merge, so neither the pipeline nor the eval harness can hand the composer a per-pass-stacked list while telling it the list is chronological.#567 — the critic no longer reads sentence openers as names. A sentence's first word is skipped unless a known name vouches for it or the same word appears capitalised mid-sentence elsewhere; a small opener stoplist covers the nine words seen in production. Accepted false negative, documented: a name used exactly once and only at a sentence start.
#572 — a window the summariser cannot read is retried, recorded and reported. Provider exceptions are logged with their type and repr instead of a bare
%s(an httpx timeout stringifies to nothing), an empty answer is distinguished from a raise, each window gets one retry within a pass, windows that errored and never produced beats land insummarisation_runs.windows_failed(migration7c8d9e0f1a2b) and on the run API, and the summary gains one plain sentence naming the stretch that could not be summarised, placed before the "What's next?" hook. A quiet window that legitimately returns no events is not counted as lost; a run that falls back to prose clears the list, since prose re-reads the whole transcript.#575 — the highlights call is windowed. It was
extract_highlights, handed the whole compacted transcript (31k tokens against a 23k budget on the replay). It now splits with the same arithmetic as beat extraction, merges round-robin so quotes span the session, caps quotes and moments per session, survives one window failing, and samples rather than sending anyway when a single line exceeds the budget.Rebased over #582 (changelog only). Full backend suite 3,076 passed; migration upgraded, downgraded and re-upgraded on a real database; the stub eval harness scores unchanged; formatted with CI's ruff.
🤖 Generated with Claude Code
On the replayed session, window 4 of 5 — the payoff — failed in every pass and the composed summary covered none of that stretch. The only trace was Beat extraction failed for window 4/5 (pass 1): once per pass, with nothing after the colon, because the log line ended in `%s` of an exception whose message is the empty string. Three changes, all in the direction of saying what happened: - `_extract_one_pass` logs `type(exc).__name__` and `repr(exc)`, and says "the extractor answered with no beats" in different words from "the call raised". It now returns per-window outcomes alongside the raw beats, so the caller can tell a stretch that was lost from one that was read and found quiet. - Each window gets a second call within the same pass before it is given up on (`_WINDOW_ATTEMPTS = 2`), counted in the log. The failures are intermittent per window; a window that fails twice in a row is a different problem. - `gather_validated_beats` collects the windows that never returned anything — errored at least once and produced beats never — onto the record as `windows_failed` and into an out-list for the caller without a record (the #349 harness passes none). `_summarise_from_beats` turns those indices into the windows' own ranges and appends one plain sentence to the composed summary, before the "What's next?" hook where one can be found: "Part of the session (from HH:MM to HH:MM) could not be summarised because the summariser returned nothing for it; the transcript is complete." A window that returned an empty list is *not* counted as failed. That is a legitimate answer for a quiet stretch of a session, and telling a GM their session is missing when it is not is the false alarm that stops the true one being believed. `summarisation_runs.windows_failed` (JSONB, nullable) carries it to the run history via migration `7c8d9e0f1a2b`, exposed on the run responses. NULL is "not measured" — a run that fell back to prose has no windows to lose — and `[]` is the stronger claim that every window was read, the convention `hallucination_drops` set. The head pin moves from `tests/test_name_snapping.py` to `tests/test_failed_windows.py`. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>llamacpp prompt is ~31348 tokens against a 22937-token budget (32768-token window at 70%) — it will be sent anyway `extract_highlights` was the last caller that read a whole compacted transcript in one prompt: `process_audio` hands it `transcript` directly, and it built one prompt around it whatever its size. Everything else that reads a transcript splits it first (`extraction_windows` for beats, `split_transcript_windows` for prose, `sample_evenly_over_segments` for lore). A backend that front-truncates then answers from the tail, without the system prompt, and returns a thin quote board with nothing to explain it. - The transcript is split with `split_transcript_windows` against the same budget arithmetic the rest of the module uses, minus the header, the highlights system prompt and room for the response, and each window is asked separately and sequentially (a local slot count is small; `_summarise_chunked` made the same call for the same reason). - `max_quotes` is the *session's* cap, applied on the merge, and quotes are taken round-robin across the windows rather than in order — taking the first N takes them all from the opening of the session, which is head-truncation arriving by a different route (#340). "Moments" gain a session cap of `max(3, max_quotes)` for the same reason: two or three per call was two or three per session while there was one call. - A window whose call fails is logged and skipped rather than losing the whole board; total failure still returns `[]` and never raises. - The one case windowing cannot fix — a single line longer than the whole budget — is sampled with `sample_evenly` and says so, rather than being sent anyway. That is the "send anyway" default made into a decision at the one layer that can make it. A transcript that already fitted takes exactly one call, with the prompt it had before. Also carries the `ruff format` reflow of one signature added by the #567 commit. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>