perf(backend): Redis pooling, SSE session release, N+1 fixes, composite index (#96) #173

Merged

claude-bot merged 1 commit from feat/96-hotpath-perf into main

2026-07-16 02:04:38 +00:00

claude-bot commented

2026-07-16 02:01:33 +00:00

Contributor

Summary

Four verified, independent hot-path inefficiencies, batched.

(a) Redis connection pooling

Per-call aioredis.from_url(...) meant a connect+teardown on every authenticated request (auth/session.py) and per-publish (bot_pubsub.py). Now module-level ConnectionPool.from_url(...) built once (DB 0); helpers get clients off the pool. from_url now appears only in the one-time pool constructions (1 async pool in session.py; sync + async pools in bot_pubsub.py). Pool-backed clients don't tear down the shared pool on aclose().

(b) SSE no longer holds a pooled DB session

session_record_stream read guild_id up front and await db.close() before returning the StreamingResponse — the generator closes over the plain string, so long-lived streams no longer pin a pooled connection. create_async_engine given explicit pool_size=10, max_overflow=20 (commented) instead of the exhaustion-prone 5+10 default.

(c) N+1 → grouped aggregates (all O(1))

Vote counts in bot_session_timeslots and _auto_close_voting_async: one select(Vote.time_slot_id, Vote.availability, func.count()).group_by(...) instead of N per-slot queries; scores/tie-breaks unchanged.
list_milestones: one NPCAppearance … IN (npc_ids) fetch bucketed in Python (the reappearance synthesis needs the individual rows, so IN-fetch not count) — 1+N → 2 queries; "NPC reappeared" output identical.

(d) Composite index

New migration e5f6g7h8i9j0 (down_revision d4e5f6g7h8i9, the current head) adds ix_sessions_status_confirmed_time on sessions (status, confirmed_time) — the poll_session_reminders minute-loop and the bot upcoming-session endpoint filter on these. Matching Index(...) added to the Session model. Upgrade + downgrade both implemented.

Verification

Backend full suite 395 passed; migration applies up and down cleanly (verified d4e5f6g7h8i9 → e5f6g7h8i9j0 → downgrade → re-upgrade); confirmed the new revision chains from main's single head (no split heads). ruff 0.4.4 clean.

Note: the dev test host has no live Redis (auth tests inject the user directly), so the pooled client's end-to-end round-trip isn't exercised there; the pool construction is import-safe (lazy) and the suite is green.

Closes #96

🤖 Generated with Claude Code

## Summary Four verified, independent hot-path inefficiencies, batched. ## (a) Redis connection pooling Per-call `aioredis.from_url(...)` meant a connect+teardown on **every authenticated request** (`auth/session.py`) and per-publish (`bot_pubsub.py`). Now module-level `ConnectionPool.from_url(...)` built once (DB 0); helpers get clients off the pool. `from_url` now appears only in the one-time pool constructions (1 async pool in `session.py`; sync + async pools in `bot_pubsub.py`). Pool-backed clients don't tear down the shared pool on `aclose()`. ## (b) SSE no longer holds a pooled DB session `session_record_stream` read `guild_id` up front and `await db.close()` **before** returning the `StreamingResponse` — the generator closes over the plain string, so long-lived streams no longer pin a pooled connection. `create_async_engine` given explicit `pool_size=10, max_overflow=20` (commented) instead of the exhaustion-prone 5+10 default. ## (c) N+1 → grouped aggregates (all O(1)) - Vote counts in `bot_session_timeslots` and `_auto_close_voting_async`: one `select(Vote.time_slot_id, Vote.availability, func.count()).group_by(...)` instead of N per-slot queries; scores/tie-breaks unchanged. - `list_milestones`: one `NPCAppearance … IN (npc_ids)` fetch bucketed in Python (the reappearance synthesis needs the individual rows, so IN-fetch not count) — 1+N → 2 queries; "NPC reappeared" output identical. ## (d) Composite index New migration `e5f6g7h8i9j0` (down_revision `d4e5f6g7h8i9`, the current head) adds `ix_sessions_status_confirmed_time` on `sessions (status, confirmed_time)` — the `poll_session_reminders` minute-loop and the bot upcoming-session endpoint filter on these. Matching `Index(...)` added to the `Session` model. Upgrade + downgrade both implemented. ## Verification Backend full suite **395 passed**; migration applies **up and down** cleanly (verified `d4e5f6g7h8i9 → e5f6g7h8i9j0 → downgrade → re-upgrade`); confirmed the new revision chains from main's single head (no split heads). ruff 0.4.4 clean. Note: the dev test host has no live Redis (auth tests inject the user directly), so the pooled client's end-to-end round-trip isn't exercised there; the pool construction is import-safe (lazy) and the suite is green. Closes #96 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit

2026-07-16 02:01:33 +00:00

perf(backend): pool Redis, release SSE DB session, drop N+1 loops, index sessions

CI / Docker image build (pull_request) Successful in 17s

Details

CI / Backend lint (ruff) (pull_request) Successful in 38s

Details

CI / Frontend tests, audit, and build (pull_request) Successful in 1m21s

Details

CI / Bot tests and audit (pull_request) Successful in 2m16s

Details

CI / Backend migration, tests, and audit (pull_request) Successful in 3m30s

Details

20d1d6882c

Four independent hot-path fixes (#96):

(a) Redis connection pooling — auth/session.py and services/bot_pubsub.py now
    build module-level pools once at import (async pool for sessions; sync +
    async pools for bot events / recording status) and check clients out of
    them, instead of a connect+teardown from_url() on every request/publish.

(b) SSE no longer pins a pooled DB session — session_record_stream resolves the
    guild_id up front and releases the request-scoped connection before entering
    the long-lived stream loop. Bound the engine pool (pool_size=10,
    max_overflow=20) so a burst of streams can't exhaust it.

(c) N+1 → grouped aggregates — bot timeslots, _auto_close_voting_async, and
    list_milestones replace per-slot / per-NPC query loops with a single
    IN + group_by (votes) / IN-ordered fetch (NPC appearances), assembled in
    Python. Query count is now O(1) instead of O(n).

(d) Composite index sessions(status, confirmed_time) via new migration
    e5f6g7h8i9j0 + matching Index in the Session model __table_args__, for the
    reminder minute-loop and bot upcoming-session filters.

Tests: updated bot_pubsub redis mocks to the pooled call sites; added a
multi-slot/multi-vote aggregation assertion. Full backend suite green (395),
migration applies up and down cleanly.

Closes #96

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

claude-bot scheduled this pull request to auto merge when all checks succeed

2026-07-16 02:01:48 +00:00

claude-bot merged commit cd765b6f73 into main

2026-07-16 02:04:38 +00:00

claude-bot deleted branch feat/96-hotpath-perf

2026-07-16 02:04:38 +00:00

claude-bot referenced this pull request from a commit

2026-07-16 02:04:39 +00:00

Merge pull request 'perf(backend): Redis pooling, SSE session release, N+1 fixes, composite index (#96)' (#173) from feat/96-hotpath-perf into main