The seven journeys as end-to-end tests: the Playwright harness #726
Closed
opened 2026-09-22 00:12:18 +00:00 by claude-bot
·
5 comments
No Branch/Tag specified
main
feat/1328-fill-it-in
feat/1328-fill-frontend
feat/1328-fill-backend
feat/9-foundry-push
fix/9-foundry-live
feat/9-foundry-push-frontend
renovate/celery-5.x
feat/5-foundry-connect
feat/9-foundry-push-backend
feat/9-foundry-push-module
feat/5-foundry-frontend
feat/5-foundry-backend
feat/5-foundry-module
release/v4.8.0-rc1
docs/d341-convert-draft
feat/958-update-to-version
feat/958-convert-sheet
feat/958-pathbuilder-import-frontend
fix/1360-heading-order
feat/958-pathbuilder-import-backend
feat/11-weekly-digest
feat/1341-auto-update
docs/d332-superseded
chore/1345-names-cleanup
feat/1342-mentioned-pages
feat/1328-recipes
fix/1346-pgdg-key
perf/1247-shared-recording-stream
chore/132-gitleaks-allowlist
release/v4.7.1
spike/foundry-module
release/v4.7.1-rc1
docs/decisions-d332-d337
fix/bot-certifi
chore/release-4.7.0
chore/release-4.7.0-rc4
chore/deps-4.7.0
chore/release-4.7.0-rc3
fix/1280-quote-must-say-it
fix/1257-names-row-squeeze
chore/release-4.7.0-rc2
fix/1245-pg-upgrade-count
chore/release-4.7.0-rc1
feat/1257-names-in-prep
feat/1257-be
feat/1257-fe
feat/1257-rating-fixes
design/1328-fill-it-in
feat/1257-name-splits
fix/1326-redis-503
feat/1280-fact-suggestions
feat/1280-be
feat/1280-fe
design/1280-fact-suggestions
feat/1279-infobox
feat/1279-be
feat/1279-fe
feat/1321-shelf-copies
design/1279-infobox
feat/1243-table-find
feat/1257-who-rated
feat/1243-prep-drawer
feat/1243-sheet-pickers
design/1243-browse-in-place
feat/1315-hotwords-default
feat/1242-system-filters
feat/1257-more-name-styles
feat/506-leg3-hotwords
design/1242-system-filters
fix/1302-packs-polish
feat/1241-rules-library
design/1241-rules-library
feat/1245-postgres-18
integ/4.6.2
fix/1263-map-ui-2
fix/1263-map-server
fix/1263-map-ui-1
fix/1261-editor-link-menu
fix/1229-axe-4.14-name-mismatch
renovate/axllent-mailpit-1.x
renovate/eslint-monorepo
renovate/authlib-1.x
chore/release-4.6.0
chore/release-4.6.0-rc2
fix/460-rc2-screens
fix/460-rc2-server
fix/release-prerelease-exit
chore/release-4.6.0-rc1
feat/125-campaign-templates
feat/126-calendar
feat/1181-pictures-and-files
feat/1190-event-people
integ/4.6.0
feat/1191-line-of-the-night
feat/1200-starter-packs
chore/bot-multidict-and-playwright-group
fix/1202-this-week-answered-count
fix/122-124-leftovers
fix/1178-get-routes-that-write
fix/1203-journeys-check-names
design/125-campaign-templates
feat/126-calendar-engine
design/126-in-world-calendar
fix/1189-cited-stamp-in-range
fix/1189-v4-looser-check
fix/1189-richer-beats
fix/1113-export-bind-lists
fix/1154-release-notes-by-file
chore/release-4.5.0
chore/release-4.5.0-rc12
fix/1135-wiki-markup-as-text
fix/demo-usage-cost-in-window
chore/release-4.5.0-rc11
fix/1121-sheet-consistency
fix/1121-sheet-5e-terms-and-card-hp
copy/1122-busiest-pages
fix/1121-relevant-choices
fix/1121-wiki-create-npc-save-sheet-links
fix/1029-menu-stays-on-screen
docs/prototype-transcript-bin-wording
fix/1023-editor-save-row-back-scroll
fix/1020-search-exact-first
fix/1020-table-prep
fix/1020-shell-wordmark-footer-bell
fix/1020-wiki-article-party
fix/872-admin-kept-until-phone-tables
feat/l13-milestone-level-waiting
fix/1000-1001-bell-older-setup-context
fix/999-pronouns-in-prompts
fix/l19-other-class-feats
fix/1010-transcript-window-ordinary-confirm
docs/d254-d269-decisions
fix/702-member-search
fix/680-transcript-bin
fix/993-unidentified-prompts
fix/841-redis-timeouts
fix/990-binned-not-exported
fix/830-export-prep-shelf
fix/deps-audit-2026-10-01
fix/877-title-reads
fix/798-audio-received-at
build/lane11-17-j7
build/lane11-16-levelup
build/lane11-14-seams
build/lane11-14-seed
build/lane11-15-start
build/lane11-13-old-block-out
fix/980-review-r3
build/lane11-12-block
build/lane11-8-packs
fix/demo-seed-canonical-names
build/lane11-11-hot-half
docs/l11-decisions
build/lane11-7-convert
build/lane11-10-editing
fix/npm-audit-brace-expansion
build/lane11-9-sheet-page
fix/bot-pyjwt
build/lane11-6-content
fix/lane11-rank-names
build/lane11-2b-language
build/lane11-4-js-evaluator
build/lane11-3-server
build/lane11-5a-pf2e
build/lane11-5b-5e
feat/design-admin
build/lane11-2-evaluator
build/lane11-1-sheet-pieces
build/lane11-0-drawing
fix/943-quest-objectives-fk
fix/prepsheet-shelf-flake
fix/945-leftovers
build/demo-gm-copy
fix/901-email-landings
build/lane10-8-journeys
build/demo-seed-lanes-8-10
fix/887-gm-own-confirmation
build/lane9-5-j1-admin
fix/918-audience-click
fix/useasync-overtaken
ci/939-critical-path
test/937-backend-suite-speed
chore/claude-orchestration
fix/924-xp-gm-only
fix/session-stale-poll
docs/decisions-d239-d242
fix/930-pypdf
build/lane8-journeys
build/925-quest-editor
fix/894-private-page-pictures
build/lane8-p4-drafts
build/lane8-p7-map
build/lane8-p6-handouts-rules
fix/wiki-fixture-names
build/lane10-d227-test-usage
fix/rsvp-horizon-test-tuesday
fix/media-volume-owner
build/lane8-p3-editor
fix/912-unidentified-rename
build/lane8-p5-party-quests
build/lane8-handout-takeback
build/lane10-pickers
build/lane8-p2-article
build/lane8-6-map
build/lane10-7b-email-pages
build/lane8-5-handouts
build/lane10-7a-test-the-bot-pages
build/lane8-p1-pages
build/lane8-4-quests
build/lane8-2b-feeds
build/lane8-3-parties
build/lane8-2a-links
build/lane10-5-commands
build/lane9-6-seed
build/lane10-4b-reactions
build/lane9-4d-pages
build/lane9-4c-pages
build/lane10-4a-posts
build/lane10-3-email
fix/891-web-fonts
build/lane9-4b-pages
build/lane10-6-test-the-bot
build/lane10-2-notify-out
build/lane8-1-wiki-text
build/lane9-4a-frame
design/lane10-drawings
fix/j6-budget
fix/j5-phone-flake
fix/869-test-leaks
fix/843-record-start-followups
build/lane7-all
build/lane6-all
build/lane9-b2-server
chore/changelog-fragments
docs/decisions-d218
build/lane9-a-owner-overview
fix/rule-card-level-label
fix/834-retention-semantics
fix/o3c-record-without-announcement
docs/decisions-d179
docs/decisions-d131-d170
fix/851-commit-before-response
build/lane6-j3-seed
build/lane9-b1-d67-identity
fix/841-blocking-loop
build/lane9-c-backups-audit
fix/vote-race-and-start-commit
fix/recording-announcement
fix/proposal-read-leak
fix/last-admin-guard
build/lane6-p4c-planning
build/lane6-p4b-margin
build/lane6-p4a-sheet
build/lane5-j4-seed
build/lane6-base
build/lane7-pages-talk
build/lane5-pages
build/lane7-pages-safety
build/lane7-p3-safety
build/lane7-p2-notify
build/lane6-p3-planning
build/lane6-p2-history
build/lane6-p1b-items
build/lane7-p1-threads
build/lane5-p6-faces
build/lane6-p1a-sheet
build/lane5-p5-find
build/lane5-p4a-board
build/lane5-p4b-scenes
build/lane5-p3-live
build/lane5-base
build/lane5-p2-cards
docs/lane5-decisions
build/review-r2-wide
fix/review-r1
build/lane5-p1-scenes
build/demo-seed
build/lane4-journeys-j5
build/lane4-page-review-b
build/lane4-page-review-a
build/lane4-page-read
build/lane4-journeys
build/lane4-page-sess2
build/lane4-page-sess1
build/lane4-notes
build/lane4-discord-on-approval
build/lane4-scheduling
build/lane4-versions
build/760-session-number
build/lane4-decisions
build/797-erase-no-regenerate
build/test-race-audit
build/lane3-completions
build/794-dark-only
build/796-titles-scroll
build/791-retention-anchor
build/a11y-sweep
build/shell-followups
build/retention-preview
build/753-backup-readback
docs/d91-d93
build/lane3-j1
build/ci-harness-paths-selftest
fix/useauth-test-race
build/ci-harness-paths
build/777-ai-settings-read
build/lane3-settings-c
build/lane3-settings-b
build/lane3-j6
build/lane3-sessionform
build/lane3-settings-a
build/lane3-members
build/lane3-thisweek
build/lane3-story
chore/87-align-python-3.12
v4.8.0-rc1
v4.7.1
v4.7.1-rc1
v4.7.0
v4.7.0-rc4
v4.7.0-rc3
v4.7.0-rc2
v4.7.0-rc1
v4.6.2
v4.6.2-rc4
v4.6.2-rc3
v4.6.2-rc2
v4.6.2-rc1
v4.6.1
v4.6.1-rc1
v4.6.0
v4.6.0-rc2
v4.6.0-rc1
v4.5.3
v4.5.3-rc1
v4.5.2
v4.5.2-rc1
v4.5.1
v4.5.1-rc2
v4.5.1-rc1
v4.5.0
v4.5.0-rc12
v4.5.0-rc11
v4.5.0-rc10
v4.5.0-rc9
v4.5.0-rc8
v4.5.0-rc7
v4.5.0-rc6
v4.5.0-rc5
v4.5.0-rc4
v4.5.0-rc3
v4.5.0-rc2
v4.5.0-rc1
v4.4.1
v4.4.0
v4.3.0
v4.2.3
v4.2.2
v4.2.1
v4.2.0
v4.1.2
v4.1.1
v4.1.0
v4.0.1
v4.0.0
v3.11.4
v3.11.3
v3.11.2
v3.11.1
v3.11.0
v3.10.0
v3.9.1
v3.9.0
v3.8.0
v3.7.0
v3.6.1
v3.6.0
v3.5.0
v3.4.1
v3.4.0
v3.3.0
v3.2.0
v3.1.0
v3.0.0
v2.1.0
v2.0.0
v1.1.0
v1.0.0
v0.12.0
v0.11.0
v0.10.0
v0.9.1
v0.3.3
v0.9.0
v0.8.0
v0.3.2
v0.3.1
v0.3.0
v0.7.0
v0.6.0
v0.5.0
v0.4.0
v0.2.1
v0.2.0
v0.1.1
v0.1.0
Labels
Clear labels
backend
Issue related to the backend
bot
Issue related to the bot
bug
Something is not working
duplicate
This issue or pull request already exists
enhancement
New feature
frontend
Issue related to the frontend
help wanted
Need some help
invalid
Something is wrong
question
More information is needed
recording
Issue related to the recording system
security
Security hardening / vulnerability work
wontfix
This won't be fixed
No labels
backend
bot
bug
duplicate
enhancement
frontend
help wanted
invalid
question
recording
security
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
rbrooks/Quest-Board#726
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Part of the v4.5.0 build plan (
docs/design/v4.5.0-build-plan.md§4; umbrella #366). The brief's seven journeys (§2) are the acceptance bar of every build lane. This issue is the harness they run in; each lane then adds or extends the specs that cross its surfaces.What it is
A Playwright suite under
webapp/frontend/e2e/, one spec per journey (j1-first-campaign.spec.ts…j7-a-character.spec.ts), written against roles and the brief's vocabulary (getByRole('button', { name: 'Send my votes' })), never against CSS classes. The fiction is the prototype's (Dana, Priya, The Amberwake Accord, Session 14), so a test asserts what the mockup shows.The stack it runs against
docker-compose.e2e.ymloverlay. The bot container is not started.providers/stub.py) and the eval harness's stub LLM enabled for the worker by an environment setting this stack alone sets. Foundry and Stripe are not exercised.Three projects per journey
Desktop viewport; an emulated phone; keyboard-only (Tab, Enter, Space, no pointer). That is the brief's "passes on desktop and on a phone" and #367's "every core journey completable by keyboard alone" as executable checks.
@axe-core/playwrightscans every page a journey visits and fails on a WCAG 2.2 AA violation.In CI
One job,
journeys, after the image build; traces, videos and screenshots kept as artefacts on failure. The same suite runs locally against the Docker loop withnpx playwright test.Acceptance
webapp/frontend/e2e/README.mdjourneysjob inci.ymlwith artefacts on failure; a red journey blocks merge like any other jobStarting the harness (2026-09-22), in parallel with lane 1's server change (#727). First PR: the Compose overlay, the fake OIDC issuer, the mail catcher, the fake bot server, the seed script, the Playwright configuration with its three projects and the axe scan, the
journeysCI job with artefacts on failure, and one smoke journey (sign in through the fake issuer, land on Home) so the pipe is proven end to end. J1 and J2 follow once the new shell is in the app, since the specs are written against its roles and names.The harness is built and runs green locally, end to end, in one command (
make journeys/scripts/journeys.sh): the stack fromdocker-compose.e2e.yml, the fake OIDC issuer, Mailpit, a fake bot, a fake AI server, the seed (Dana runs The Amberwake Accord, Priya plays in it, Session 13 completed, Session 14 a week ahead), then Playwright in the three projects with the axe scan. Nine smoke tests pass; the seed also proves that product mail reaches the catcher and that asession_confirmedevent reaches the fake bot off the Redis stream.Departures from this issue's text, all deliberate:
.tsfiles would have looked typed and checked nothing. Renaming later is mechanical.providers/stub.pysays in its own docstring why the stub ASR is kept out of the registry (an operator-selectable provider that invents text is a foot-gun), and there is no stub LLM at the provider interface at all. The endpoint URL is already configuration, so the harness runs a small server that speaks the bundled WhisperX and llama.cpp shapes and the real adapters run against it. Strictly more coverage, and zero product code changed./login's guide links;/dashboard's New Campaign button, role pills and countdown link). They are listed per page, per node, with the lane that removes them (#727, #728); anything else fails, including a new violation of the same rule on the same page.QB_E2E_STRICT_A11Y=1ignores the baseline so a lane can ask whether its page is clean.Housekeeping the harness needed:
.gitattributesforcing LF on shell scripts (a CRLFinit.shstops Postgres on a Windows checkout),PLAYWRIGHT_SKIP_BROWSER_DOWNLOADin the frontend images and CI job, ruff overe2e/, a gitleaks allowlist for the obviously fake.env.e2e, exact pins for@playwright/test,@axe-core/playwrightand the Playwright image, which move together.One thing that cannot be verified from here: whether the Forgejo runner's bind mounts reach the workspace (the
synthetic-sessionjob already works around docker-in-docker for the same reason). The script probes it right afterupand fails with a message naming the remedy; if the first CI run dies there, the fix is to bakee2e/into the fake and seed images as that job does.Found on the way: #743 (campaign create silently drops eight declared fields). Next: rebase onto the shell (#742) once it merges, rewrite J0 against the new frame (the account menu and the Contents instead of the old nav bar), unify the fiction on the prototype's "Dana Whitlock", then the PR.
#746 is merged on green CI, with the
journeysjob green on the runner twice over. What is on main:docker-compose.e2e.ymlwith no host bind mount (a backend test refuses one), the fake OIDC issuer, Mailpit, the fake bot that also reads the Redis stream, the fake AI server, the seed in the prototype's fiction, Playwright in three projects with the enforced keyboard guard and the WCAG 2.2 AA axe gate with its owned baseline,scripts/journeys.sh/make journeys, thejourneysCI job with artefacts on failure (upload-artifact@v3; v4 is not supported on Forgejo), andwebapp/frontend/e2e/README.md.Acceptance so far: the overlay, fakes, seed and README are done; the CI job blocks a PR like any other; each lane issue names its journeys (the build plan's table). Still open here: J1 and J2 green in all three projects, which arrive with lane 1's Home PR (in progress), so this stays open until then.
On its first day the gate found #745 (fixed) and the seed found #743 (fixed).
With #749 merged, J1 (a first-time GM sees the two ways in; Dana starts a campaign from nothing and lands on Home with it first) and J2 (Priya is asked whether she is in, on Home and in the bell) are green in all three projects on the runner, and
/dashboardand/campaigns/newpass the strict axe gate;/loginis the only tolerated entry left, owned by lane 2. That completes this issue's acceptance: the harness with its fakes, seed and README; J1 and J2 introduced with lane 1; thejourneysjob blocking merges; each later lane naming its journeys. Later lanes extend the specs from here. Closing.J6 has landed in #770, and it surfaced two things about the harness itself
Neither is urgent; both are the kind of thing that costs a day when somebody meets it cold.
The suite is now visibly paying the sign-in limiter
J6 costs three sign-ins (one per project), taking the suite to six per project, eighteen per run, against a limit of 10/minute per IP.
signInAswaits the 429 out, so every run is green and nothing is broken. But one test per run now takes ~23s instead of ~3s — and which test moves between runs:[phone] J1in one,[phone] J2in the next. That wandering is the signature of a rate-limit window being waited out, not of a slow page, and it is exactly the shape of thing somebody will otherwise spend an afternoon profiling.J3, J4 and J5 will make it two or three tests. Worth deciding before then whether the e2e stack should raise the limit for its own OIDC fake, or whether the suite should share a signed-in state between projects.
The axe gate only ever sees light mode
Chromium's default
prefers-color-schemeis light, so every scan in the suite is a light-mode scan. The brief (§6, constraint 6) makes both themes first-class on every page.This is not theoretical: #770 fixed a real 1.4.3 contrast failure whose dark half is computed, not measured, because nothing in the suite can measure it. A
colorScheme: "dark"pass, or a fourth project, would close it — at the cost of a third or a quarter more run time, which is why it is a decision for whoever owns this issue rather than something to slip into a lane's PR.Also
identities.jsoncarried the GM's retired surname ("sub": "e2e-dana-okafor") — in the file this issue's README calls "the one place their names are written down". Fixed in #770.a11y-baseline.jscannot describe a campaign-scoped page at all — filed as #771. It matters to this issue because from lane 3 onward, "tolerate this one rule on this one page" is not expressible; the only key that matches a UUID address is"*", which quietens the rule everywhere.openContents()andgoVia(name, url), documented in the README's fixture table. The Contents is a rail on a desktop and a sheet behind a button on a phone; every lane from here walks it, and one spelling beats three.🤖 Generated with Claude Code