[Backend] Per-session cost and usage telemetry #357
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Found in the August 2026 session lifecycle review (#319).
Why
A hosted tier cannot be priced or operated without knowing what a session actually costs, and a self-hoster benefits from the same numbers in GPU-minutes rather than dollars. Today nothing records either.
The review's cost model puts a typical session around $0.95 on the recommended managed stack, dominated by ASR — but that is a projection. Real per-session numbers are what turn a pricing guess into a pricing decision, and they are what will tell you when the break-even against owned hardware has actually arrived.
Proposed fix
Record per session: audio minutes submitted before and after the VAD pre-pass (the single biggest cost lever, worth roughly a 72% reduction), ASR provider and cost, LLM input/output/cached tokens per pass and cost, total wall-clock processing time, and retry count.
Expose it in the admin area as per-session and per-campaign rollups. Make the cost fields provider-supplied where available and estimated from published rates otherwise, and mark which is which — an estimate presented as a measurement is worse than no number.
Acceptance criteria
Landed in PR #494 (merged 2026-09-05, CI green; 1892 backend / 484 frontend tests).
b6c7d8e9fa0b:session_usage, one row per processing run — audio seconds submitted and transcribed (the VAD lever), ASR/LLM provider and model, LLM calls / prompt / completion / cached tokens, phase wall times, retry count, estimated costs withcost_is_estimateandpricing_as_of, andstatusso a failed run is costed too.ContextVarscope opened byprocess_audioand recorded insidellm_service._check_usage— the one funnel — with no threaded parameter. The four_summarise_*prose transports had bypassed that funnel, so the session summary itself would have gone uncounted; they now route through it with truncation behaviour unchanged.Usage.cached_tokensread from Anthropic and OpenAI — the cache-hint half #351 deferred here.app/services/ai_pricing.pylist rates as of 2026-05-01 (rendered on the panel so staleness is visible); self-hosted rows carry GPU/CPU minutes andnullcosts, never0.GET /api/admin/ai/usage(sessions, per-campaign rollups with VAD reduction, per-month line) and per-campaign detail;AdminAiUsagepanel under the capability panel, every dollar figure badged "estimated".Not done: no managed-ASR rate exists to populate yet (
ASR_HOURLY_USDis an empty hook); the per-campaign endpoint has no UI beyond the aggregate. Worth a follow-up:process_audio's body has no committed test — the instrumentation was proven with a throwaway eager run, not a test in the suite.