feat(backend): rightsholder notices and takedown across every content-pack scope (#555) #561

Merged

claude-bot merged 1 commit from feat/555-content-notices into main 2026-09-08 07:43:50 +00:00

Conversation 0 Commits 1 Files changed 12 +2328 -26

claude-bot commented 2026-09-08 07:27:56 +00:00

Contributor

Copy link

Closes #555. Stacked on #560 (which stacks on #559); merge those first and this PR's diff shrinks to its own commit.

What it adds. A content_notices table (migration 2d3e4f5a6b7c, raw SQL, no FKs so the record of why content went survives the content) and admin endpoints to record a notice, review it, and act on it in one place: disable/enable or remove a pack loaded for the whole instance or in any campaign, or remove individual records inside either. Disabling is immediate, reversible, and takes the content out of every search on the next query; removal is final. A batch of actions is all-or-nothing (SAVEPOINT). One operator-set contact for notices is readable by any signed-in user so a licences page can print it.

Telling the importer. There is no per-user notification path in the app (the notification backends are session-scheduling shaped and the attention feed is campaign-scoped), so every action that leaves a pack standing writes status_detail on the pack naming the notice and quoting the affirmation version that importer accepted, and every action lands in the audit log with the notice id, importer and affirmation version. Removing a pack outright leaves no row to carry the message; docs/OPERATIONS.md recommends disable-then-remove for that reason.

Quest Board decides nothing. No automatic takedown, no status moved by an action, no counter-notice reinstating anything, no repeat-infringer logic. The operator's steps are in docs/OPERATIONS.md → "Handling a rightsholder notice"; the endpoints are in docs/API.md.

Tests: 27 new in test_content_notices.py; full backend suite passed before the rebase (2661), and the notices, migration, import and admin suites after it (95).

🤖 Generated with Claude Code

Closes #555. Stacked on #560 (which stacks on #559); merge those first and this PR's diff shrinks to its own commit. **What it adds.** A `content_notices` table (migration `2d3e4f5a6b7c`, raw SQL, no FKs so the record of *why* content went survives the content) and admin endpoints to record a notice, review it, and act on it in one place: disable/enable or remove a pack loaded for the whole instance or in *any* campaign, or remove individual records inside either. Disabling is immediate, reversible, and takes the content out of every search on the next query; removal is final. A batch of actions is all-or-nothing (SAVEPOINT). One operator-set contact for notices is readable by any signed-in user so a licences page can print it. **Telling the importer.** There is no per-user notification path in the app (the notification backends are session-scheduling shaped and the attention feed is campaign-scoped), so every action that leaves a pack standing writes `status_detail` on the pack naming the notice and quoting the affirmation version *that importer* accepted, and every action lands in the audit log with the notice id, importer and affirmation version. Removing a pack outright leaves no row to carry the message; `docs/OPERATIONS.md` recommends disable-then-remove for that reason. **Quest Board decides nothing.** No automatic takedown, no status moved by an action, no counter-notice reinstating anything, no repeat-infringer logic. The operator's steps are in `docs/OPERATIONS.md` → "Handling a rightsholder notice"; the endpoints are in `docs/API.md`. Tests: 27 new in `test_content_notices.py`; full backend suite passed before the rebase (2661), and the notices, migration, import and admin suites after it (95). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 7 commits 2026-09-08 07:27:57 +00:00

feat(backend): Foundry VTT document adapter for content packs (#553) ... cf47e3f619

Lane 2 of the content-packs work: read what a Foundry user actually has on
disk — a folder or a zip of document JSON, one document per file, or a single
JSON array of the same documents — and turn it into NormalizedRecords.

`adapters/base.py` is the spec's §2 contract verbatim (lane 1 writes the same
bytes, so the two merge cleanly). `adapters/foundry.py` owns the container,
the identity and the prose; `adapters/foundry_maps/{pf2e,dnd5e}.py` own the
meaning of the `system` block, and `generic.py` catches every other system as
a text record with the raw block kept in `extra`.

Three decisions worth the reader's time:

- **Two passes, bounded memory.** PackMeta must carry the game system and the
  pack's licence, which are facts about the whole pack, while records are
  yielded lazily. So `read` scans once to vote on the system and count
  licences, discarding each document as it goes, then returns a generator that
  walks them again. A zip is read entry by entry and an entry whose declared
  size exceeds the per-file ceiling is refused before it is decompressed.

- **Slugs come from the name, not the file.** A Foundry `_id` is a random
  16-character handle and the file stem only exists for the folder and zip
  shapes. Name-derived slugs are the only rule that gives identical records
  whether a pack arrives as a folder, a zip or one array — asserted by the
  tests, and the property that makes re-import replace rather than duplicate.

- **Stats are mapped faithfully or not at all.** Values are keyed by the real
  `app/game_systems/*/schema_v1.json` field keys and coerced to the schema's
  types; anything unreadable is dropped rather than rounded or guessed. The
  tests validate both creature fixtures against the live active schema, so a
  map that drifts from the schema fails here instead of importing silently
  with `stats = None`.

`_html_to_md` is a local nh3 + markdownify pass so this lane does not depend
on lane 1's file existing; swap it for `app.reference.sanitize` at merge.

Every fixture is invented — creatures, rule text, numbers and book titles are
written for the tests, per the fixtures policy in CLAUDE.md.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

fix(backend): read a pf2e actor's licence from system.details (#553) ... d7dd813240

The pf2e system does not keep `publication` in one place. Items carry it at
`system.publication`, which is what the adapter read; actors carry it at
`system.details.publication`, which it did not. Every creature in a Monster
Core-style pack therefore imported with `licence = None` — the one field a
content pack cannot afford to lose, since the export path decides what may be
shared on from exactly that value.

`foundry_maps.pf2e.licence_block` now looks in both places, plus the
pre-remaster `source` spelling of each for older exports, and the generic
fallback map looks in the same places for the same reason: a document from an
unrecognised system may still declare a licence. `_detect_system` counts
`details.publication` as pf2e evidence too, so a bare actor with no other
pf2e-shaped field is still recognised.

Fixture `cairnlight-warden.json` is an invented pf2e creature that declares
ORC only under `details`, with tests for the licence, the system evidence and
the stat mapping.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(backend): register the Foundry adapter and adopt the shared sanitiser (#553) ... 552b2f4c88

Joins lane 2 to the foundation now that it exists.

- Registered: `foundry.adapter` is a module-level instance, imported and
  `register()`ed in `adapters/__init__.py` under the Registrations banner.
  `detect` therefore routes a Foundry zip here rather than to the native
  reader asked before it, which is asserted.
- Sanitising: the local nh3 + markdownify pass is gone in favour of
  `app.reference.sanitize.sanitize_markdown` — not `to_markdown`, which is
  markdownify alone with no allowlist in front of it. The Foundry-specific
  enricher pass still runs first, on the raw HTML, because markdownify
  escapes the brackets it matches on. Summaries use `sanitize_summary` for
  the cap but still take only the opening paragraph, so a creature's summary
  is its description rather than the first 200 characters of its strikes.
- Slugs come from `app.reference.slugs.slugify`, as that module asks every
  adapter to do, so the same creature from two formats lands on one slug.
- Warnings: `PackMeta.extra["warnings"]` is a list the record generator holds
  and appends to, for the things this adapter would otherwise swallow — an
  unidentifiable system, a pack that mixes systems, a hint that contradicts
  the documents, a document with no name, a journal page that is not an
  object. Bounded at 50, well under the pipeline's own cap.
- `foundry_maps/__init__.py` added now that the package sits inside regular
  packages rather than being reached as a namespace package.

`test_the_formats_endpoint_lists_the_registered_adapters` now expects
questboard *and* foundry, in DETECT_ORDER: registering a format is exactly
what changes that endpoint's answer.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

fix(backend): a bare JSON array is only native if it holds native records (#553) ... 9deacde6f2

`questboard.sniff` claimed any `.json` whose head started with `[`, and it is
asked first. A Foundry export — an array of documents, one of the shapes the
Foundry adapter exists to read — was therefore claimed by the native reader
and imported as five empty `other` records: no body, no stats, no licence, no
error, nothing in the warnings. Open5e's per-page array will land the same way
when that lane arrives.

The bare-array branch now looks at the array's first element. Three answers,
because the evidence comes in three strengths:

- The element arrived whole: decide on its keys. Ours has `kind` and `name`;
  a Foundry document has `_id`; an Open5e page has a name and no kind.
- The element is truncated — the usual case, since `sniff` is given 512 bytes
  and a stat block is longer than that. Its keys cannot be enumerated, so a
  missing `kind` proves nothing. Only a positive signal decides: `_id` without
  `kind` is someone else's document, and anything else stays with the native
  reader, which is what keeps `[{"name": "Truncated"` answering "not valid
  JSON" rather than "could not tell what format".
- An empty array is nobody's format in particular, so it stays here too and
  the importer is told it contains no records.

`DETECT_ORDER` is unchanged. The Foundry adapter now also judges an extension-
less filename on shape, as the native one already did, so a Foundry array
fetched from a URL with no extension is detected rather than refused.

One existing expectation moved: `[{"level": 3}]` has neither kind nor name, so
nothing detects it and the answer is now "Could not tell what format". The
reader's own "record 1 has no name" is still covered, by a test that declares
the format instead of relying on detection.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(backend): 5etools and Dungeon Master's Vault (.orcbrew) adapters for content packs (#553) ... 88d0dc86b9

Lane 3 of the content packs work: two importers for formats people already
have homebrew in, both producing the normalised record shape from the spec.

`fivetools.py` reads a 5etools homebrew JSON file — a `_meta` block plus one
array per content type. Two parts carry the weight. `render_tags` turns
5etools' inline markup (`{@hit 7}`, `{@damage 2d8+4}`, `{@dc 14}`,
`{@condition grappled}`) into plain text rather than dropping it, because the
numbers in those tags are usually the only place the number appears; unknown
tags keep their text argument, so a homebrew author's invention degrades to
readable prose instead of vanishing. `entries_to_markdown` flattens the
recursive `entries` structure — nests, lists, tables, insets — into Markdown.
Monsters map onto the real dnd5e stat schema (AC, HP, speed, CR, the six
scores, size, creature type); everything the schema has no room for, `senses`
and the rest, is kept in `extra` for export.

`orcbrew.py` reads a Dungeon Master's Vault / OrcPub2 export, which is EDN.
The format has no schema and no version marker, so the only things that fail
the file are "not EDN" and "not a map"; an unknown content type, an entity
that is not a map, or `:armor-class "as tough as a door"` each cost one field
and one warning, never the import. A pack that loads 340 of its 341 monsters
with a warning is a better answer for the person who uploaded it than a
rejection. `:hit-points` prefers `:mean` and computes the dice average only
when the export omits it; `:challenge 1/4` survives as the string "1/4" and
the number 0.25. The pack's identity comes from the modal `:option-pack`
across its entities, because there is no header to read it from.

`base.py` is the spec's section 2 code, byte for byte, so the foundation
lane's copy merges as a no-op. `_text.py` is a stopgap HTML/Markdown cleaner
for these two adapters only; it is replaced by `app.reference.sanitize` when
lane 1 lands. Neither `__init__.py` is created here — the registry lines are
the foundation lane's to add.

Fixtures are invented end to end, per the fixtures policy: invented sources,
authors, creatures, spells, items and numbers, and the EDN was written by
hand. Nothing is taken from a published book, from 5etools' data, or from a
real `.orcbrew`.

64 tests, covering: sniffing in both directions (neither adapter claims the
other's file, and neither claims a `.qbpack`); every array and content type
mapping to its kind; the monster stat maps validating against the real dnd5e
schema through `validate_stats_values`; the tag renderer; and malformed input
raising `AdapterError` with a message a person can act on.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(backend): register the 5etools and .orcbrew adapters with the import pipeline (#553) ... 6530eaaacc

Wires lane 3's two readers into the registry and the pipeline, alongside the
native and Foundry readers that landed first.

**Registered.** `fivetools` and `orcbrew` join `questboard` and `foundry` in
`adapters/__init__`, each exposing a module-level `adapter` instance the way
the others do. One import and one `register` per line, in `DETECT_ORDER`, so
the file reads the way detection runs — though the order of the lines decides
nothing, since `detect` and `list_formats` both walk `DETECT_ORDER`. That is
what lets the remaining lane add its line without reasoning about the lines
around it.

The formats-endpoint test is rewritten against the registry instead of against
a list of names, so no further lane has to edit it, and so what it checks is
the wiring rather than today's roster: every registered adapter is served (an
adapter registered under an id `DETECT_ORDER` does not mention would otherwise
vanish from the endpoint in silence), in `DETECT_ORDER` and not import order,
with the name and extensions its adapter declares. The one thing still spelled
out is that the native reader comes first, because `detect` takes the first yes
and whichever adapter leads decides what the rest ever see.

`tests/test_reference_detect.py` gains a case per lane-3 format, so all four
adapters are pinned in the one file that owns the boundary question.

**One sanitiser, not two.** `adapters/_text.py` — the stopgap cleaner lane 3
carried while the foundation was unmerged — is deleted, and both adapters use
`app.reference.sanitize` and `app.reference.slugs`. `sanitize_markdown` rather
than `to_markdown`: the latter is only the markdownify half, so on its own it
would run a converter over unsanitised HTML, which is the hole `nh3.clean`
exists to close, and would mangle the plain prose that most of a 5etools entry
or an `.orcbrew` description actually is. Each adapter keeps three-line
`_markdown`/`_plain`/`_summary` wrappers so a caller never has to think about
coercing a keyword or an int, and so the 200-character cap that comes with
`sanitize_summary` is documented where it bites.

Fragments are cleaned as they are read rather than the assembled body being
cleaned at the end, so what the pipeline receives is already markup-free and
its own sanitising pass is a no-op instead of an HTML converter running over
Markdown the adapter wrote.

**Adapter warnings now reach the importer.** `read()` returns only a meta and
an iterator, so an adapter reports what it could not map by appending to
`meta.extra["warnings"]` — the same list object the record iterator appends to
as it is consumed. `normalise` folds that list into the outcome's warnings
after the walk, de-duplicated and under `MAX_WARNINGS`. Without this the
pipeline could only ever warn about what it saw after the fact: a bad kind, a
duplicate slug. Only the adapter knows that a monster's armour class was the
words "as tough as a door", and a GM who is never told will spend the session
wondering where the AC went.

**Fixed: `.orcbrew` files with CRLF failed as "not EDN".** A carriage return is
legal EDN whitespace but `edn_format`'s lexer rejects it outright, so a file
opened and saved on Windows — which is most of them — was refused for a reason
that had nothing to do with its contents. Line endings are normalised before
the parse, with a regression test. The bug surfaced because git checked the
fixture out with CRLF, which is exactly how a self-hoster would have met it.

Docs: `docs/CONTENT-PACKS.md` gains a section per non-native format — what maps
to what, how each pack is named, where each licence is read from and why
neither 5e homebrew format records one the file does not declare —
`docs/API.md`'s formats example lists all four, and the changelog describes the
Foundry, 5etools and `.orcbrew` readers for the people who will use them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

feat(backend): rightsholder notices and takedown across every content-pack scope (#555) ... e5c9ef2915

A pack is a unit of provenance; this is what an operator does when someone
claims one of them is theirs. An admin records the notice — reporter,
contact, claim, targets, when it arrived, notes — and acts on it across
everything on the instance: an instance pack, a pack in any campaign, or
individual records in either. Disable is immediate and reversible; removal
is final. A batch is applied in a SAVEPOINT, so it is all-or-nothing.

The importer is told on the pack itself: status_detail names the notice and
quotes the affirmation version stored on their own pack row, which is the
wording they actually agreed to rather than today's. There is no general
per-user notification path in the app — the notification backends are
session-scheduling shaped — so that in-app message plus an audit row naming
the importer is the whole of the telling, and it is why disabling before
removing is the recommended order: removal leaves no row to carry a message.

Every action writes an audit row carrying the notice id, the pack's scope
and key, who imported it and which affirmation version they made, so the log
answers "why did this go?" after the pack no longer exists. targets and
actions_taken hold ids as JSON rather than foreign keys for the same reason:
a takedown must not delete its own justification.

The software decides nothing. Nothing disables itself, no notice moves its
own status, no counter-notice reinstates anything, there is no
repeat-infringer logic, and no status transition is validated — applying
actions deliberately does not set a notice to "actioned", because having
acted and considering the matter answered are different facts. docs/
OPERATIONS.md says what the software does and what the operator decides.

New migration 2d3e4f5a6b7c adds content_notices (raw SQL, TEXT + CHECK, one
head). One operator-set notice contact lives in app_settings behind a typed
accessor and is readable by any signed-in user for a licences page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

claude-bot scheduled this pull request to auto merge when all checks succeed 2026-09-08 07:38:49 +00:00

claude-bot merged commit 073690dcc0 into main 2026-09-08 07:43:50 +00:00

claude-bot deleted branch feat/555-content-notices 2026-09-08 07:43:50 +00:00

claude-bot referenced this pull request from a commit 2026-09-08 07:43:50 +00:00

Merge PR #561: feat(backend): rightsholder notices and takedown across every content-pack scope (#555)

claude-bot referenced this pull request 2026-09-08 07:44:25 +00:00

[Hosting] Rightsholder notices and takedown: report, review, remove, and keep the record #555

claude-bot referenced this pull request 2026-09-08 07:50:22 +00:00

feat(frontend): Content packs screen for admins and GMs, with the import affirmation (#553) #563

claude-bot referenced this pull request 2026-09-08 08:38:18 +00:00

[Feature] Rules and bestiary lookup at the table and in Prep #553

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

backend

Issue related to the backend

  

bot

Issue related to the bot

  

bug

Something is not working

  

duplicate

This issue or pull request already exists

  

enhancement

New feature

  

frontend

Issue related to the frontend

  

help wanted

Need some help

  

invalid

Something is wrong

  

question

More information is needed

  

recording

Issue related to the recording system

  

security

Security hardening / vulnerability work

  

wontfix

This won't be fixed

No labels

backend

bot

bug

duplicate

enhancement

frontend

help wanted

invalid

question

recording

security

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

rbrooks/Quest-Board!561

No description provided.