media-processor and federation-sync queues never trim completed jobs (Redis growth) #100

New issue

Closed

opened 2026-07-15 19:51:32 +00:00 by claude-bot · 1 comment

claude-bot commented

2026-07-15 19:51:32 +00:00

Contributor

Severity: Medium · Confidence: High · Effort: XS · Category: ops

Evidence

api/src/services/mediaProcessorWorker.ts:19 - new Queue(QUEUE_NAME, { connection }) with no removeOnComplete/Fail.
api/src/services/federationSyncWorker.ts:27 - same. Contrast import/reanalyze/bulk/trash/ap-delivery/update-check queues, which set retention.

Problem
Every media upload and every federation sync leaves a completed job in Redis forever; two queues don't set job retention.

Impact
Steady unbounded Redis growth on a media-heavy or federated instance → eventual OOM/eviction affecting sessions and all queues.

Fix
Add defaultJobOptions: { removeOnComplete: 100, removeOnFail: 100 } to both queues.

Acceptance criteria

Both queues retain a bounded number of completed/failed jobs.

Filed from the 2026-07-15 codebase audit. Full report: docs/.internal/report-2026-07-15.md (gitignored).

**Severity:** Medium · **Confidence:** High · **Effort:** XS · Category: ops **Evidence** - `api/src/services/mediaProcessorWorker.ts:19` - `new Queue(QUEUE_NAME, { connection })` with no `removeOnComplete/Fail`. - `api/src/services/federationSyncWorker.ts:27` - same. Contrast import/reanalyze/bulk/trash/ap-delivery/update-check queues, which set retention. **Problem** Every media upload and every federation sync leaves a completed job in Redis forever; two queues don't set job retention. **Impact** Steady unbounded Redis growth on a media-heavy or federated instance → eventual OOM/eviction affecting sessions and all queues. **Fix** Add `defaultJobOptions: { removeOnComplete: 100, removeOnFail: 100 }` to both queues. **Acceptance criteria** - [ ] Both queues retain a bounded number of completed/failed jobs. --- _Filed from the 2026-07-15 codebase audit. Full report: `docs/.internal/report-2026-07-15.md` (gitignored)._

claude-bot added this to the v7.2.0 - Reliability, data integrity & docs milestone

2026-07-15 19:51:32 +00:00

claude-bot added the

bug

label

2026-07-15 19:51:32 +00:00

rbrooks referenced this issue from a commit

2026-07-17 02:36:46 +00:00

fix(v7.2): wave 2 — ops hygiene (#95, #100, #101, #102)

claude-bot commented

2026-07-17 02:53:56 +00:00

Author

Contributor

Fixed in c1ed843 (v7.2.0 wave 2).

Both queues now set retention, matching the other seven:

defaultJobOptions: { attempts: 1, removeOnComplete: 100, removeOnFail: 100 }

— mediaProcessorWorker.ts (processorQueue) and federationSyncWorker.ts (syncQueue). Added attempts: 1 alongside, since every other queue declares it explicitly and these two were relying on the BullMQ default.

Acceptance criteria:

Both queues retain a bounded number of completed/failed jobs.

CI green (242/242).

Fixed in `c1ed843` (v7.2.0 wave 2). Both queues now set retention, matching the other seven: ```ts defaultJobOptions: { attempts: 1, removeOnComplete: 100, removeOnFail: 100 } ``` — `mediaProcessorWorker.ts` (`processorQueue`) and `federationSyncWorker.ts` (`syncQueue`). Added `attempts: 1` alongside, since every other queue declares it explicitly and these two were relying on the BullMQ default. **Acceptance criteria:** - [x] Both queues retain a bounded number of completed/failed jobs. CI green (242/242).

claude-bot closed this issue

2026-07-17 02:54:18 +00:00