A picture's metadata is removed before the picture is opened (#1087) #1088
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/1087-exif-open-bomb"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #1087. Not to be merged before the independent review reports.
Summary
Pillow parses a file's metadata when it opens it, before any pixel check can run, and the cost is the file's to choose. The worst file found is 70 kB.
image_service._sanitise(raw)walks the container and removes every metadata segment before Pillow sees the bytes._open_sanitisedis the one place inapp/that opens upload or import bytes; an AST test holds that.APPnandCOMsegment is dropped; the scan onward is copied in one piece.EXIF,XMPandICCPare dropped and the header flags naming them cleared.Measured
512 MB container, peak over the state before the open:
Honest files still accepted, upright, with no metadata out: a 4032×3024 phone JPEG with a maker note and GPS; an MPO/HDR photo with a gain-map frame (first frame kept); a PNG with 500 kB of XMP, a 200 kB ICC profile and
eXIf.Through nginx on a warm backend: the MPF and EXIF files to the picture, attachment and map routes were accepted as the small harmless pictures inside them. The backend stayed at its warm 457 MiB; an honest 12 MP photo took it to 496 to 500 MiB.
oom_kill 0and no restarts.Not closed here
FF 00takes 2.6 s to profile at 8 MB and 17.6 s as a 75 MB map; a GIF of millions of one-byte sub-blocks takes about 2 s, and the new strip adds a second walk of about the same. All are linear in the file's size and flat in memory.Changelog
changelog.d/1087-exif-open-bomb.security.mdTest plan
tests/wiki/test_exif_open_bomb.py, 30 tests, with the orientation and pixel-bound tests:329 passed.tests/wikiwith the damaged-image and session-hygiene tests:962 passed.tests/export: the first run had2 failed, both a truncated PNG no longer reaching the decode; fixed, and both files re-run:85 passed.7672 passed, 14 skipped, 107 deselected. It ran before the merge of main, which added nothing to the image code.tracemalloc, which counts Python allocations only. The figures in the table are from separate runs.🤖 Generated with Claude Code
Independent review of
6a302c47: not to be merged yet. One blocking defect, two regressions for honest files, four smaller points. Fixes are in progress on this branch.FFC8,FFF0toFFFDandFF00, and resumes scanning inside what the stripper copied as one opaque segment. An EXIF block placed there survives: after the strip,Image.openstill reportedexif, so the parse this PR exists to prevent still ran. Fix: an allowlist that mirrors Pillow's marker table, with a test over all 256 marker bytes so a Pillow upgrade cannot reopen it.APP14removes what libjpeg uses to tell YCCK from CMYK, and Photoshop writes CMYK JPEGs as YCCK. Fix: keep anAPP14that is the fixed 12-byte Adobe segment.Image.open(...).Found sound
Not checked by the review: the test suites, and memory or timing by measurement. Part of the review was cut short, so items 4 and 6 are from reading.
All seven review findings fixed in
ba1c694b, each with a test.FFC8,FFF0toFFFD, anything absent from its table) is refused;FF00is dropped. A test over all 256 marker bytes holds the table to Pillow's, and another puts an EXIF block behind each marker and expects a refusal or noexifAPP14is kept; any otherAPP14is dropped.APP0(JFIF) is kept too. Pixels are identical before and after for a YCCK and an RGB-with-Adobe JPEGIDAT,fdAT, the WebP image chunks) is not counted towards the piece cap. A 26 MB PNG of 97,000 small chunks and a 1,000-frame GIF are acceptedopenandImageFile.Parserthrough each module's imports, counts opens per function, and scansapp/anddemo/, with every exempt site listedMeasured after the fixes
Ran at
ba1c694b300 passed. The orientation and pixel-bound tests:299 passed.tests/wikiwith the damaged-image and session-hygiene tests:1232 passed.tests/export:462 passed.7942 passed, 14 skipped, 107 deselected.FFF0): 422, backend flat at 452 MiB;oom_kill 0, no restarts.Not tested
One residual point, by reading:
APP0segments are kept whole, so their total is bounded by the file's size, not by a fixed limit.