feat(backend): 5etools and Dungeon Master's Vault (.orcbrew) adapters for content packs (#553) #560
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/553-fivetools-orcbrew-adapters"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Phase 1 of #553, lane 3 of the content-packs spec. Stacked on #559 (the Foundry adapter); merge that first and this PR's diff shrinks to its own two commits.
What it reads. 5etools homebrew JSON (
_meta+ per-kind arrays: monster, spell, item, and the rest) and Dungeon Master's Vault.orcbrew(EDN). Both map to the shared kinds, render 5etools{@tag ...}markup and orcbrew's nested maps to Markdown through the shared sanitiser, and record no licence the file does not declare.edn-format==0.8.0is the one new dependency, import-guarded so a missing package becomes anAdapterErrorrather than a crash.Pipeline change. Adapter warnings now reach the importer:
import_service.normalisedrains the record iterator and then mergesmeta.extra["warnings"]into the outcome through the existing_warn, so the cap and count behave like every other warning. Three tests cover it, including an end-to-end.orcbrewwhose armour class is"as tough as a door"and imports anyway with a warning.Bug fixed on the way.
.orcbrewfiles saved on Windows (CRLF) were rejected as "not EDN" becauseedn_format's lexer refuses\r.read()normalises line endings first; regression test included.Also. The formats-endpoint test is now an invariant over the registry rather than a list of names, so later adapters need not touch it;
test_reference_detect.pygains a case per adapter so all four are pinned in one place; docs (CONTENT-PACKS.md,API.md, changelog) describe the Foundry reader too, since #559 carried no docs.Tests: 204 across the adapter, detection, import and search suites; 53 more in export, task, URL-fetch, migration and admin. Fixtures are invented end to end (Hollow Fen, Thornhollow Reaches), nothing from 5etools' data or a real
.orcbrew.🤖 Generated with Claude Code
Lane 2 of the content-packs work: read what a Foundry user actually has on disk — a folder or a zip of document JSON, one document per file, or a single JSON array of the same documents — and turn it into NormalizedRecords. `adapters/base.py` is the spec's §2 contract verbatim (lane 1 writes the same bytes, so the two merge cleanly). `adapters/foundry.py` owns the container, the identity and the prose; `adapters/foundry_maps/{pf2e,dnd5e}.py` own the meaning of the `system` block, and `generic.py` catches every other system as a text record with the raw block kept in `extra`. Three decisions worth the reader's time: - **Two passes, bounded memory.** PackMeta must carry the game system and the pack's licence, which are facts about the whole pack, while records are yielded lazily. So `read` scans once to vote on the system and count licences, discarding each document as it goes, then returns a generator that walks them again. A zip is read entry by entry and an entry whose declared size exceeds the per-file ceiling is refused before it is decompressed. - **Slugs come from the name, not the file.** A Foundry `_id` is a random 16-character handle and the file stem only exists for the folder and zip shapes. Name-derived slugs are the only rule that gives identical records whether a pack arrives as a folder, a zip or one array — asserted by the tests, and the property that makes re-import replace rather than duplicate. - **Stats are mapped faithfully or not at all.** Values are keyed by the real `app/game_systems/*/schema_v1.json` field keys and coerced to the schema's types; anything unreadable is dropped rather than rounded or guessed. The tests validate both creature fixtures against the live active schema, so a map that drifts from the schema fails here instead of importing silently with `stats = None`. `_html_to_md` is a local nh3 + markdownify pass so this lane does not depend on lane 1's file existing; swap it for `app.reference.sanitize` at merge. Every fixture is invented — creatures, rule text, numbers and book titles are written for the tests, per the fixtures policy in CLAUDE.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>`questboard.sniff` claimed any `.json` whose head started with `[`, and it is asked first. A Foundry export — an array of documents, one of the shapes the Foundry adapter exists to read — was therefore claimed by the native reader and imported as five empty `other` records: no body, no stats, no licence, no error, nothing in the warnings. Open5e's per-page array will land the same way when that lane arrives. The bare-array branch now looks at the array's first element. Three answers, because the evidence comes in three strengths: - The element arrived whole: decide on its keys. Ours has `kind` and `name`; a Foundry document has `_id`; an Open5e page has a name and no kind. - The element is truncated — the usual case, since `sniff` is given 512 bytes and a stat block is longer than that. Its keys cannot be enumerated, so a missing `kind` proves nothing. Only a positive signal decides: `_id` without `kind` is someone else's document, and anything else stays with the native reader, which is what keeps `[{"name": "Truncated"` answering "not valid JSON" rather than "could not tell what format". - An empty array is nobody's format in particular, so it stays here too and the importer is told it contains no records. `DETECT_ORDER` is unchanged. The Foundry adapter now also judges an extension- less filename on shape, as the native one already did, so a Foundry array fetched from a URL with no extension is detected rather than refused. One existing expectation moved: `[{"level": 3}]` has neither kind nor name, so nothing detects it and the answer is now "Could not tell what format". The reader's own "record 1 has no name" is still covered, by a test that declares the format instead of relying on detection. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>Lane 3 of the content packs work: two importers for formats people already have homebrew in, both producing the normalised record shape from the spec. `fivetools.py` reads a 5etools homebrew JSON file — a `_meta` block plus one array per content type. Two parts carry the weight. `render_tags` turns 5etools' inline markup (`{@hit 7}`, `{@damage 2d8+4}`, `{@dc 14}`, `{@condition grappled}`) into plain text rather than dropping it, because the numbers in those tags are usually the only place the number appears; unknown tags keep their text argument, so a homebrew author's invention degrades to readable prose instead of vanishing. `entries_to_markdown` flattens the recursive `entries` structure — nests, lists, tables, insets — into Markdown. Monsters map onto the real dnd5e stat schema (AC, HP, speed, CR, the six scores, size, creature type); everything the schema has no room for, `senses` and the rest, is kept in `extra` for export. `orcbrew.py` reads a Dungeon Master's Vault / OrcPub2 export, which is EDN. The format has no schema and no version marker, so the only things that fail the file are "not EDN" and "not a map"; an unknown content type, an entity that is not a map, or `:armor-class "as tough as a door"` each cost one field and one warning, never the import. A pack that loads 340 of its 341 monsters with a warning is a better answer for the person who uploaded it than a rejection. `:hit-points` prefers `:mean` and computes the dice average only when the export omits it; `:challenge 1/4` survives as the string "1/4" and the number 0.25. The pack's identity comes from the modal `:option-pack` across its entities, because there is no header to read it from. `base.py` is the spec's section 2 code, byte for byte, so the foundation lane's copy merges as a no-op. `_text.py` is a stopgap HTML/Markdown cleaner for these two adapters only; it is replaced by `app.reference.sanitize` when lane 1 lands. Neither `__init__.py` is created here — the registry lines are the foundation lane's to add. Fixtures are invented end to end, per the fixtures policy: invented sources, authors, creatures, spells, items and numbers, and the EDN was written by hand. Nothing is taken from a published book, from 5etools' data, or from a real `.orcbrew`. 64 tests, covering: sniffing in both directions (neither adapter claims the other's file, and neither claims a `.qbpack`); every array and content type mapping to its kind; the monster stat maps validating against the real dnd5e schema through `validate_stats_values`; the tag renderer; and malformed input raising `AdapterError` with a message a person can act on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>