Take a scheme's colours from a photo (#17) #131

Merged

claude-bot merged 1 commit from feat/17-palette into main 2026-09-05 01:57:31 +00:00

Conversation 0 Commits 1 Files changed 4 +468 -1

claude-bot commented 2026-09-05 00:50:17 +00:00

Contributor

Copy link

Closes #17.

Pick an image in the scheme editor and its dominant colours fill the segment slots. Nothing is written until Save, so the result is an ordinary editable scheme on the ordinary path.

Two deviations from the issue's sketch — flagging both

Extraction is client-side, where the sketch proposed a Pillow endpoint

Server-side extraction means a self-hosted box parsing arbitrary uploaded images with several megabytes of native code. That's a meaningful addition to the attack surface #60 spent a release shrinking, on the one code path that consumes attacker-chosen bytes.

The browser already decodes JPEG, PNG, WebP and HEIC in a sandbox hardened far beyond anything we would add. Doing it there means: no new backend dependency, no upload endpoint, no size caps or temp files to get wrong, and the photo never leaves the machine it's on — which matters for a self-hosted app where someone might drop in a family photo.

Every acceptance criterion is met either way — image → palette, preview before save, a standard editable scheme. Only the mechanism differs. Say the word if you'd rather have the endpoint and I'll add it.

Median cut rather than k-means

The issue asks for "deterministic, offline color sourcing", and k-means is not deterministic: it depends on how its centroids are seeded, so the same photo can give two different palettes on two runs. A palette you can't reproduce isn't worth saving. Median cut has no random component at all.

The split point was found by a failing test, not reasoned in advance

Textbook median cut splits at the median index. My first test — equal parts red, green and blue, asking for three colours — failed with a bucket 120 units from any colour in the image.

Sorting by red puts 200 non-red pixels before 100 red ones, so the median lands inside the non-red run: the first bucket averages green with half the blues and returns a colour present nowhere in the image. A flag or a logo comes back as mud.

Cutting at the widest gap along the widest channel finds the boundary between clusters instead. The tie-break matters as much: in a photo the values are near-continuous and every gap is equal, so preferring the gap nearest the middle restores median cut's balance exactly where balance is the right instinct — rather than peeling off one outlier pixel at a time.

Smaller decisions

• Pixels are sampled with a stride across the whole image, not read from the front. A photo is laid out row by row, so the first N pixels describe the sky and call it the palette. There's a test for this.
• Transparent pixels are skipped. Averaging them drags the result toward whatever sits in the transparent region's RGB channels — usually black, never something the user can see.
• Fewer colours are returned than asked for when the image genuinely has fewer. Padding a two-tone logo out to five would hand the user three colours their image doesn't contain.
• Five colours are extracted but only three fit WLED's slots. The extras stay on screen so someone who wants the fourth can set it by hand rather than re-uploading.

Verification

14 new vitest cases asserting properties rather than fixture values — that the colours are ones actually present, that the dominant one comes first, that the same input gives the same output — because median cut always returns something, and eyeballing swatches won't tell you whether they came from the photo or from the order the buckets split in.

Frontend in node:22: tsc -b clean, eslint clean, 95 vitest passing. No backend change at all.

🤖 Generated with Claude Code

Closes #17. Pick an image in the scheme editor and its dominant colours fill the segment slots. Nothing is written until **Save**, so the result is an ordinary editable scheme on the ordinary path. ## Two deviations from the issue's sketch — flagging both ### Extraction is client-side, where the sketch proposed a Pillow endpoint Server-side extraction means a self-hosted box parsing **arbitrary uploaded images with several megabytes of native code**. That's a meaningful addition to the attack surface #60 spent a release shrinking, on the one code path that consumes attacker-chosen bytes. The browser already decodes JPEG, PNG, WebP and HEIC in a sandbox hardened far beyond anything we would add. Doing it there means: no new backend dependency, no upload endpoint, no size caps or temp files to get wrong, and **the photo never leaves the machine it's on** — which matters for a self-hosted app where someone might drop in a family photo. Every acceptance criterion is met either way — image → palette, preview before save, a standard editable scheme. Only the mechanism differs. Say the word if you'd rather have the endpoint and I'll add it. ### Median cut rather than k-means The issue asks for *"deterministic, offline color sourcing"*, and k-means is not deterministic: it depends on how its centroids are seeded, so the same photo can give two different palettes on two runs. A palette you can't reproduce isn't worth saving. Median cut has no random component at all. ## The split point was found by a failing test, not reasoned in advance Textbook median cut splits at the median **index**. My first test — equal parts red, green and blue, asking for three colours — failed with a bucket 120 units from any colour in the image. Sorting by red puts 200 non-red pixels before 100 red ones, so the median lands *inside* the non-red run: the first bucket averages green with half the blues and returns a colour present nowhere in the image. A flag or a logo comes back as mud. Cutting at the **widest gap** along the widest channel finds the boundary *between* clusters instead. The tie-break matters as much: in a photo the values are near-continuous and every gap is equal, so preferring the gap nearest the middle restores median cut's balance exactly where balance is the right instinct — rather than peeling off one outlier pixel at a time. ## Smaller decisions - **Pixels are sampled with a stride across the whole image**, not read from the front. A photo is laid out row by row, so the first N pixels describe the sky and call it the palette. There's a test for this. - **Transparent pixels are skipped.** Averaging them drags the result toward whatever sits in the transparent region's RGB channels — usually black, never something the user can see. - **Fewer colours are returned than asked for when the image genuinely has fewer.** Padding a two-tone logo out to five would hand the user three colours their image doesn't contain. - Five colours are extracted but only three fit WLED's slots. The extras stay on screen so someone who wants the fourth can set it by hand rather than re-uploading. ## Verification 14 new vitest cases asserting properties rather than fixture values — that the colours are ones actually present, that the dominant one comes first, that the same input gives the same output — because median cut always returns *something*, and eyeballing swatches won't tell you whether they came from the photo or from the order the buckets split in. Frontend in `node:22`: `tsc -b` clean, `eslint` clean, **95 vitest passing**. No backend change at all. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit 2026-09-05 00:50:17 +00:00

Take a scheme's colours from a photo (#17) ... cd5ee38cee

Pick an image in the scheme editor and its dominant colours fill the
segment slots. Nothing is written until Save, so the result is an
ordinary editable scheme on the ordinary path.

Two deviations from the issue's sketch, both deliberate.

Extraction is client-side, where the sketch proposed a Pillow endpoint.
Server-side means a self-hosted box parsing arbitrary uploaded images
with several megabytes of native code, which is a meaningful addition to
the attack surface #60 spent a release shrinking. The browser already
decodes JPEG, PNG, WebP and HEIC in a sandbox hardened far beyond
anything we would add. Doing it there means no new backend dependency, no
upload endpoint, no size caps or temp files to get wrong, and the photo
never leaves the machine it is on. Every acceptance criterion -- image to
palette, preview before save, a standard editable scheme -- is met either
way; only the mechanism differs.

Median cut rather than k-means, because the issue asks for deterministic
colour sourcing and k-means is not deterministic: it depends on how the
centroids are seeded, so the same photo can give two different palettes.
Median cut has no random component.

The split point is the widest gap along the widest channel, not the
median index, and this was found by a failing test rather than reasoned
in advance. Given equal parts red, green and blue, sorting by red puts
200 non-red pixels before 100 red ones and the median lands inside the
non-red run -- so the first bucket averages green with half the blues and
returns a colour present nowhere in the image. A flag or a logo comes
back as mud. Cutting at the widest gap finds the boundary between
clusters instead. The tie-break matters as much: in a photo the values
are near-continuous and every gap is equal, so preferring the gap nearest
the middle restores median cut's balance exactly where balance is the
right instinct, rather than peeling off one outlier pixel at a time.

Pixels are sampled with a stride across the whole image rather than read
from the front: a photo is laid out row by row, so the first N pixels
describe the sky and call it the palette. Transparent pixels are skipped,
since averaging them drags the result toward whatever sits in the
transparent region's RGB channels.

Fewer colours are returned than asked for when the image genuinely has
fewer. Padding a two-tone logo out to five would hand the user three
colours their image does not contain.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

rbrooks force-pushed feat/17-palette from cd5ee38cee to e1d252c9f2 2026-09-05 01:10:32 +00:00 Compare

rbrooks force-pushed feat/17-palette from e1d252c9f2 to f25f4d7b76 2026-09-05 01:41:44 +00:00 Compare

claude-bot merged commit 001bf5fc0f into main 2026-09-05 01:57:31 +00:00

claude-bot deleted branch feat/17-palette 2026-09-05 01:57:32 +00:00

claude-bot referenced this pull request from a commit 2026-09-05 01:57:33 +00:00

Take a scheme's colours from a photo (#17) (#131)

claude-bot referenced this pull request 2026-09-05 01:57:44 +00:00

Palette-from-image scheme generator #17

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/ai

AI suggestion pipeline

  

area/backend

FastAPI / Python backend

  

area/frontend

React / Vite frontend

  

area/infra

Docker, deployment, and infrastructure

  

area/scheduler

Scheduler and time envelope

  

area/wled

WLED controller integration

  

good-first-issue

Good for newcomers

  

priority/high

Blocks release or high impact

  

priority/low

Nice to have

  

priority/medium

Should be done soon

  

type/bug

Something isn't working

  

type/chore

Maintenance, release, or housekeeping

  

type/ci-cd

CI/CD pipeline, build, or release automation

  

type/docs

Documentation

  

type/feature

New capability or enhancement

  

type/qa

Testing, validation, or quality assurance

  

v1.0.0

Target release: v1.0.0

  

v1.1.0

Target release: v1.1.0

  

v1.2.0

Target release: v1.2.0

  

v2.0.0

Target release: v2.0.0

No labels

area/ai

area/backend

area/frontend

area/infra

area/scheduler

area/wled

good-first-issue

priority/high

priority/low

priority/medium

type/bug

type/chore

type/ci-cd

type/docs

type/feature

type/qa

v1.0.0

v1.1.0

v1.2.0

v2.0.0

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

rbrooks/Iris-WLED!131

No description provided.