[Foundry] Decide & document connection topology: self-hosted relay vs direct URL, and the SSRF/URL-policy exemption #143
Closed
opened 2026-07-15 22:02:32 +00:00 by claude-bot
·
8 comments
No Branch/Tag specified
main
feat/1328-fill-it-in
feat/1328-fill-frontend
feat/1328-fill-backend
feat/9-foundry-push
fix/9-foundry-live
feat/9-foundry-push-frontend
renovate/celery-5.x
feat/5-foundry-connect
feat/9-foundry-push-backend
feat/9-foundry-push-module
feat/5-foundry-frontend
feat/5-foundry-backend
feat/5-foundry-module
release/v4.8.0-rc1
docs/d341-convert-draft
feat/958-update-to-version
feat/958-convert-sheet
feat/958-pathbuilder-import-frontend
fix/1360-heading-order
feat/958-pathbuilder-import-backend
feat/11-weekly-digest
feat/1341-auto-update
docs/d332-superseded
chore/1345-names-cleanup
feat/1342-mentioned-pages
feat/1328-recipes
fix/1346-pgdg-key
perf/1247-shared-recording-stream
chore/132-gitleaks-allowlist
release/v4.7.1
spike/foundry-module
release/v4.7.1-rc1
docs/decisions-d332-d337
fix/bot-certifi
chore/release-4.7.0
chore/release-4.7.0-rc4
chore/deps-4.7.0
chore/release-4.7.0-rc3
fix/1280-quote-must-say-it
fix/1257-names-row-squeeze
chore/release-4.7.0-rc2
fix/1245-pg-upgrade-count
chore/release-4.7.0-rc1
feat/1257-names-in-prep
feat/1257-be
feat/1257-fe
feat/1257-rating-fixes
design/1328-fill-it-in
feat/1257-name-splits
fix/1326-redis-503
feat/1280-fact-suggestions
feat/1280-be
feat/1280-fe
design/1280-fact-suggestions
feat/1279-infobox
feat/1279-be
feat/1279-fe
feat/1321-shelf-copies
design/1279-infobox
feat/1243-table-find
feat/1257-who-rated
feat/1243-prep-drawer
feat/1243-sheet-pickers
design/1243-browse-in-place
feat/1315-hotwords-default
feat/1242-system-filters
feat/1257-more-name-styles
feat/506-leg3-hotwords
design/1242-system-filters
fix/1302-packs-polish
feat/1241-rules-library
design/1241-rules-library
feat/1245-postgres-18
integ/4.6.2
fix/1263-map-ui-2
fix/1263-map-server
fix/1263-map-ui-1
fix/1261-editor-link-menu
fix/1229-axe-4.14-name-mismatch
renovate/axllent-mailpit-1.x
renovate/eslint-monorepo
renovate/authlib-1.x
chore/release-4.6.0
chore/release-4.6.0-rc2
fix/460-rc2-screens
fix/460-rc2-server
fix/release-prerelease-exit
chore/release-4.6.0-rc1
feat/125-campaign-templates
feat/126-calendar
feat/1181-pictures-and-files
feat/1190-event-people
integ/4.6.0
feat/1191-line-of-the-night
feat/1200-starter-packs
chore/bot-multidict-and-playwright-group
fix/1202-this-week-answered-count
fix/122-124-leftovers
fix/1178-get-routes-that-write
fix/1203-journeys-check-names
design/125-campaign-templates
feat/126-calendar-engine
design/126-in-world-calendar
fix/1189-cited-stamp-in-range
fix/1189-v4-looser-check
fix/1189-richer-beats
fix/1113-export-bind-lists
fix/1154-release-notes-by-file
chore/release-4.5.0
chore/release-4.5.0-rc12
fix/1135-wiki-markup-as-text
fix/demo-usage-cost-in-window
chore/release-4.5.0-rc11
fix/1121-sheet-consistency
fix/1121-sheet-5e-terms-and-card-hp
copy/1122-busiest-pages
fix/1121-relevant-choices
fix/1121-wiki-create-npc-save-sheet-links
fix/1029-menu-stays-on-screen
docs/prototype-transcript-bin-wording
fix/1023-editor-save-row-back-scroll
fix/1020-search-exact-first
fix/1020-table-prep
fix/1020-shell-wordmark-footer-bell
fix/1020-wiki-article-party
fix/872-admin-kept-until-phone-tables
feat/l13-milestone-level-waiting
fix/1000-1001-bell-older-setup-context
fix/999-pronouns-in-prompts
fix/l19-other-class-feats
fix/1010-transcript-window-ordinary-confirm
docs/d254-d269-decisions
fix/702-member-search
fix/680-transcript-bin
fix/993-unidentified-prompts
fix/841-redis-timeouts
fix/990-binned-not-exported
fix/830-export-prep-shelf
fix/deps-audit-2026-10-01
fix/877-title-reads
fix/798-audio-received-at
build/lane11-17-j7
build/lane11-16-levelup
build/lane11-14-seams
build/lane11-14-seed
build/lane11-15-start
build/lane11-13-old-block-out
fix/980-review-r3
build/lane11-12-block
build/lane11-8-packs
fix/demo-seed-canonical-names
build/lane11-11-hot-half
docs/l11-decisions
build/lane11-7-convert
build/lane11-10-editing
fix/npm-audit-brace-expansion
build/lane11-9-sheet-page
fix/bot-pyjwt
build/lane11-6-content
fix/lane11-rank-names
build/lane11-2b-language
build/lane11-4-js-evaluator
build/lane11-3-server
build/lane11-5a-pf2e
build/lane11-5b-5e
feat/design-admin
build/lane11-2-evaluator
build/lane11-1-sheet-pieces
build/lane11-0-drawing
fix/943-quest-objectives-fk
fix/prepsheet-shelf-flake
fix/945-leftovers
build/demo-gm-copy
fix/901-email-landings
build/lane10-8-journeys
build/demo-seed-lanes-8-10
fix/887-gm-own-confirmation
build/lane9-5-j1-admin
fix/918-audience-click
fix/useasync-overtaken
ci/939-critical-path
test/937-backend-suite-speed
chore/claude-orchestration
fix/924-xp-gm-only
fix/session-stale-poll
docs/decisions-d239-d242
fix/930-pypdf
build/lane8-journeys
build/925-quest-editor
fix/894-private-page-pictures
build/lane8-p4-drafts
build/lane8-p7-map
build/lane8-p6-handouts-rules
fix/wiki-fixture-names
build/lane10-d227-test-usage
fix/rsvp-horizon-test-tuesday
fix/media-volume-owner
build/lane8-p3-editor
fix/912-unidentified-rename
build/lane8-p5-party-quests
build/lane8-handout-takeback
build/lane10-pickers
build/lane8-p2-article
build/lane8-6-map
build/lane10-7b-email-pages
build/lane8-5-handouts
build/lane10-7a-test-the-bot-pages
build/lane8-p1-pages
build/lane8-4-quests
build/lane8-2b-feeds
build/lane8-3-parties
build/lane8-2a-links
build/lane10-5-commands
build/lane9-6-seed
build/lane10-4b-reactions
build/lane9-4d-pages
build/lane9-4c-pages
build/lane10-4a-posts
build/lane10-3-email
fix/891-web-fonts
build/lane9-4b-pages
build/lane10-6-test-the-bot
build/lane10-2-notify-out
build/lane8-1-wiki-text
build/lane9-4a-frame
design/lane10-drawings
fix/j6-budget
fix/j5-phone-flake
fix/869-test-leaks
fix/843-record-start-followups
build/lane7-all
build/lane6-all
build/lane9-b2-server
chore/changelog-fragments
docs/decisions-d218
build/lane9-a-owner-overview
fix/rule-card-level-label
fix/834-retention-semantics
fix/o3c-record-without-announcement
docs/decisions-d179
docs/decisions-d131-d170
fix/851-commit-before-response
build/lane6-j3-seed
build/lane9-b1-d67-identity
fix/841-blocking-loop
build/lane9-c-backups-audit
fix/vote-race-and-start-commit
fix/recording-announcement
fix/proposal-read-leak
fix/last-admin-guard
build/lane6-p4c-planning
build/lane6-p4b-margin
build/lane6-p4a-sheet
build/lane5-j4-seed
build/lane6-base
build/lane7-pages-talk
build/lane5-pages
build/lane7-pages-safety
build/lane7-p3-safety
build/lane7-p2-notify
build/lane6-p3-planning
build/lane6-p2-history
build/lane6-p1b-items
build/lane7-p1-threads
build/lane5-p6-faces
build/lane6-p1a-sheet
build/lane5-p5-find
build/lane5-p4a-board
build/lane5-p4b-scenes
build/lane5-p3-live
build/lane5-base
build/lane5-p2-cards
docs/lane5-decisions
build/review-r2-wide
fix/review-r1
build/lane5-p1-scenes
build/demo-seed
build/lane4-journeys-j5
build/lane4-page-review-b
build/lane4-page-review-a
build/lane4-page-read
build/lane4-journeys
build/lane4-page-sess2
build/lane4-page-sess1
build/lane4-notes
build/lane4-discord-on-approval
build/lane4-scheduling
build/lane4-versions
build/760-session-number
build/lane4-decisions
build/797-erase-no-regenerate
build/test-race-audit
build/lane3-completions
build/794-dark-only
build/796-titles-scroll
build/791-retention-anchor
build/a11y-sweep
build/shell-followups
build/retention-preview
build/753-backup-readback
docs/d91-d93
build/lane3-j1
build/ci-harness-paths-selftest
fix/useauth-test-race
build/ci-harness-paths
build/777-ai-settings-read
build/lane3-settings-c
build/lane3-settings-b
build/lane3-j6
build/lane3-sessionform
build/lane3-settings-a
build/lane3-members
build/lane3-thisweek
build/lane3-story
chore/87-align-python-3.12
v4.8.0-rc1
v4.7.1
v4.7.1-rc1
v4.7.0
v4.7.0-rc4
v4.7.0-rc3
v4.7.0-rc2
v4.7.0-rc1
v4.6.2
v4.6.2-rc4
v4.6.2-rc3
v4.6.2-rc2
v4.6.2-rc1
v4.6.1
v4.6.1-rc1
v4.6.0
v4.6.0-rc2
v4.6.0-rc1
v4.5.3
v4.5.3-rc1
v4.5.2
v4.5.2-rc1
v4.5.1
v4.5.1-rc2
v4.5.1-rc1
v4.5.0
v4.5.0-rc12
v4.5.0-rc11
v4.5.0-rc10
v4.5.0-rc9
v4.5.0-rc8
v4.5.0-rc7
v4.5.0-rc6
v4.5.0-rc5
v4.5.0-rc4
v4.5.0-rc3
v4.5.0-rc2
v4.5.0-rc1
v4.4.1
v4.4.0
v4.3.0
v4.2.3
v4.2.2
v4.2.1
v4.2.0
v4.1.2
v4.1.1
v4.1.0
v4.0.1
v4.0.0
v3.11.4
v3.11.3
v3.11.2
v3.11.1
v3.11.0
v3.10.0
v3.9.1
v3.9.0
v3.8.0
v3.7.0
v3.6.1
v3.6.0
v3.5.0
v3.4.1
v3.4.0
v3.3.0
v3.2.0
v3.1.0
v3.0.0
v2.1.0
v2.0.0
v1.1.0
v1.0.0
v0.12.0
v0.11.0
v0.10.0
v0.9.1
v0.3.3
v0.9.0
v0.8.0
v0.3.2
v0.3.1
v0.3.0
v0.7.0
v0.6.0
v0.5.0
v0.4.0
v0.2.1
v0.2.0
v0.1.1
v0.1.0
Labels
Clear labels
backend
Issue related to the backend
bot
Issue related to the bot
bug
Something is not working
duplicate
This issue or pull request already exists
enhancement
New feature
frontend
Issue related to the frontend
help wanted
Need some help
invalid
Something is wrong
question
More information is needed
recording
Issue related to the recording system
security
Security hardening / vulnerability work
wontfix
This won't be fixed
No labels
backend
bot
bug
duplicate
enhancement
frontend
help wanted
invalid
question
recording
security
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
rbrooks/Quest-Board#143
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Motivation/Context
Live NPC/token push (see the FoundryVTT live-push investigation, 2026-07-15) needs a hard architectural decision before any of #5's connection settings work can be considered final: how does the Quest Board backend actually reach a GM's Foundry instance, given that Foundry is frequently
localhost:30000on a GM's PC, or on a home LAN/VPN, never reliably reachable by direct inbound HTTP from the backend (#24 already calls this out explicitly)? The investigation evaluated three options: (A) direct backend→Foundry REST module HTTP call, (B) an outbound WebSocket relay — the ThreeHatsfoundryvtt-rest-apimodule already named in #24 is architected exactly this way, with the Foundry-side module dialing out to a relay server and the backend talking REST to the relay — and (C) browser-mediated push (rejected: HTTPS→HTTP mixed-content blocking, no server-side retries/idempotency, tab-dependent).This also surfaces a real code-level collision: every existing admin-configured service URL is normalized through
normalize_service_url()inwebapp/backend/app/services/url_policy_service.py, called fromsettings_service.py(e.g. lines 167–171, 187–192) with private-IP targets blocked unlesssettings.allow_private_service_urlsis set. A per-campaign, GM-settable Foundry/relay URL pointing at a LAN address is by design for this feature — so #5's connection-settings shape cannot be finalized until this issue resolves whether/how the Foundry URL is exempted from that SSRF guard.Approach
Recommended architecture per the investigation: Option B (relay) as the primary mechanism, with Option A (direct URL) as a degenerate case of the same code path —
foundry/client.py(per #24) just speaks to "a base URL + API key"; whether that URL is a self-hosted relay or a directly reachable Foundry endpoint is a per-campaign configuration detail, not an architectural fork.For the SSRF collision, the investigation's recommended resolution (the safer of the two options considered) is:
app_settingslike any other admin-configured service endpoint (no SSRF exposure — it's an admin, not GM, setting).normalize_service_urlwith its own allowlist policy) should be documented as rejected, with the reasoning, if this issue confirms the admin-relay approach.Concrete deliverables for this issue:
foundryvtt-rest-apimodule's relay mode against the actual production Foundry instance and core version (per #24's explicit sequencing warning — this must happen before adapter code in #7 is written).Dependencies
Out of scope
npc_to_actor, etc.) — tracked separately in #7/#9.Acceptance criteria
foundryvtt-rest-apimodule's relay mode has been validated end-to-end against the real production Foundry instance (tested module + core version pair recorded).normalize_service_urlis specified.foundry_linksdocument-mapping table for idempotent push #146sidebar_fields, #130) intonpc_to_actor#148Owner, 27 Oct 2026: give this serious consideration when v5.0.0 starts, and evaluate every option before choosing. The owner's first idea: our own Foundry module (plugin) that runs in the GM's world and connects back to Quest Board. That would be an outbound connection from Foundry, so it works behind home NAT with no inbound port, a third-party relay or a GM-entered URL.
The evaluation should at least cover:
foundryvtt-rest-apimodule and its relay, self-hosted or theirs.Criteria:
localhost:30000behind NAT.Moved ahead of the hosted service (Foundry is now v5.0.0 and v5.1.0; hosted is v6.0.0), so the outcome must already suit hosted. #4 (install the REST module) depends on the choice made here.
How Quest Board connects to a Foundry world (#143)
Design note, 2026-11-06, for the owner's decision before v5.0.0 starts. Researched from primary sources where cited; the Foundry 14.368 release (16 Sep 2026) was re-checked against foundryvtt.com.
The problem
Quest Board needs to read from and write to a GM's Foundry world: actors, journals, scenes, and live events during play. The world often runs at
localhost:30000on the GM's PC or on a home network, so Quest Board's server usually cannot reach it, and a hosted Quest Board never can.Facts that shape every option
minimum,verifiedandmaximumcore versions in its manifest (compatibility data).module.jsonplus a zip download) (module development). A listing on foundryvtt.com is optional. Unverified: the listing's current submission rules.wss://to any server.fetch()to another origin needs that server to send CORS headers. Community modules that call outside APIs fail exactly here (Kenku issue, DiscordConnect issue).httpsFoundry page cannot open plainws://(mixed content). Anhttp://localhostpage can openwss://.wss://is allowed.The options
A. Direct: the Quest Board backend calls a URL on the Foundry side
How it works: the GM enters a URL, and the backend calls a REST endpoint served by a module or proxy next to Foundry.
normalize_service_url()exists to block (webapp/backend/app/services/url_policy_service.py, which rejects private-IP targets unlessallow_private_ip_targets).B. ThreeHats
foundryvtt-rest-apimodule plus its relayHow it works: the module, in the GM's browser, dials out over WebSocket to a relay, and Quest Board calls the relay's REST API with an
x-api-key(module, relay)./sessionendpoint for "Headless Foundry sessions via Puppeteer" (relay). Community docs describe an auto-start of a headless session when the world is offline. Unverified against the official docs. It implies storing a Foundry login somewhere.execute-jspermission (GM by default) (module docs). A leaked key is close to full control of the world.D. Our own Quest Board Foundry module that dials out to Quest Board
How it works: the GM installs "Quest Board" from a manifest URL. In campaign settings, Quest Board shows a pairing code. The GM types it into the module, which receives a campaign-scoped token. While a GM has the world open, the module holds a
wss://connection to Quest Board's own backend.E. Other routes considered
Scored against the owner's criteria
Key: ✔ good · ~ workable with caveats · ✘ poor
execute-jsShared limitation of B and D: nothing happens while no GM has the world open. For v5.0 that's acceptable if pushes queue in Quest Board and the module drains the queue when a GM connects. Member sync (#8) becomes "sync when the GM opens the world", not "scheduled".
Recommendation
D: our own module, connecting out to Quest Board, with a pairing code.
Before committing, a short spike on the owner's Foundry instance (its core version is not yet known; record it) (this replaces #4's "install the REST module"):
wss://connection to a dev Quest Board.This proves the browser and CSP assumptions marked unverified above. Reading the ThreeHats module's source first is worthwhile; forking it is allowed by its MIT licence.
What it changes in the v5.0 issues
BOT_API_KEY(#24) and never logged.foundry/client.pyspeaks to a socket instead of HTTP.updateActorevents. Not on a timer.normalize_service_url()is needed. #143's option of an admin-set relay URL becomes moot, and a per-campaign URL is never stored.Risks
verified, and test on each Foundry release candidate.connect-srcon Quest Board's own pages doesn't affect the module, because the module runs on Foundry's page.Decisions for the owner
Decided (owner, 2026-11-06, Q233–Q237; recorded as D333–D337):
Next: rewrite #4 as the test, and update #5, #6, #7 and #8 to match the "What it changes" section above when v5.0 starts.
The test Foundry instance for D337 is live (2026-11-06)
This is the instance the owner said they'd build. Infra is tracked in Rhoving/iac-repo#874 (PR #875). Everything below was checked on the running server today.
Versions (D337: "record its core version")
The core version matches prod on purpose, and the iac role keeps it pinned. A core bump on one should move the other with it. pf2e on the test instance is one minor ahead of prod; keep that in mind if a pf2e mapping behaves differently on prod.
Where it is
foundry-dev(imagefelddy/foundryvtt:14.369.1, pinned by digest), host port18130. Data is in/opt/foundry-dev/data, owned by uid/gid 1226 (foundry-dev).*.lan.ryanbrooks.dev(that host doesn't use the Pi-holes). Usecurl --resolve foundry-dev.lan.ryanbrooks.dev:443:192.168.1.30 …, orhttp://10.1.1.14:18130directly from the LAN.Facts for the test and the module
https://, so the module must usewss://.ws://is blocked as mixed content. Dev Quest Board (questboard-dev.rhoving.com) is HTTPS, sowss://questboard-dev.rhoving.com/...is the target. Its nginx must pass the upgrade headers. The edge Caddy passes WebSockets as-is./auth,/joinand/game, both straight from Foundry and through int-proxy. There's no CSP header and no<meta>CSP; the only security header isX-Frame-Options: DENY. That settles the design note's "Unverified: whether Foundry v14 sends a CSP" for 14.369: nothing in Foundry blocks an outboundwss://from a module. Caveat: checked with no world running, so the test should still confirm the connection from inside a launched world.Access-Control-Allow-Origin: *. That covers requests to Foundry, not the module's requests from the page. Afetch()from the module to Quest Board still needs Quest Board's CORS headers forhttps://foundry-dev.lan.ryanbrooks.dev. WebSocket handshakes aren't subject to CORS, though the backend should checkOriginitself.questboard-dev.rhoving.com, both 200. So installing from a manifest URL served by dev Quest Board (D334) will work.hotReload: true, so CSS/HTML/lang changes apply without a refresh) and minification off (readable stack traces). Proxy settings:hostname: foundry-dev.lan.ryanbrooks.dev,proxySSL: true,proxyPort: 443, no route prefix.Installing a test module
/opt/foundry-dev/data/Data/modules/<id>/on docker-test, thensudo chown -R 1226:1226it, and return to Setup (or reload the world).Still needed before the test
Operating it
ssh docker-test 'sudo docker logs -f foundry-dev'.sudo systemctl restart foundry-dev.foundry-dev-ansible/(README there).Test world created (owner, 2026-11-06): The Amberwake Accord
the-amberwake-accord. On docker-test it's at/opt/foundry-dev/data/Data/worlds/the-amberwake-accord/.This clears the "no world yet" item in the comment above. Still open: a second GM user in this world, for testing first-holder plus handoff.
Users in The Amberwake Accord (owner, 2026-11-06): the owner created two GM accounts. Read from the world's user data on the server (names and roles only):
Ryan: GM (role 4)Test-GM: GM (role 4)Player2: Player (role 1)This clears the last setup item. The test can cover:
Ryanin one browser, then asTest-GMin another (a private window works). Only one should hold the link. When the first closes, the other should take over.Player2. The module should not connect, or should refuse to pair.Foundry's default "Gamemaster" user from world creation also showed up in the user data. It may be a stale record from before a rename. If it's still in the world's User Management, remove it or keep it in mind as a third GM.
The D337 test passed (2026-11-07, #4). All seven checks passed on Foundry 14.369 with pf2e 8.7.0. Two findings shape the real module: the pairing UI must be registered for GMs only, because
restricteddoesn't hide client-scope settings, and it needs a lasting connection indicator. Details on #4. The connection design is confirmed.Decided and proven; closing. The owner chose option D, our own module connecting out, as D333–D337 (2026-11-06). The #4 test passed 7/7 on Foundry 14.369 (2026-11-07). The build contract is
docs/design/foundry-connection.mdonfeat/5-foundry-connect: pairing over the socket, per-GM tokens (stored as hashes, never logged), one holder per campaign across workers through Redis DB 0, and the module served by each server (D334). The build continues in #5 and #6 (slice 1, Connect), then #7, #8 and #9.