The journeys harness: the app stack in Docker, a fake issuer, bot and AI, Playwright in three projects with an axe gate (#726) #746
Loading…
Reference in a new issue
No description provided.
Delete branch "build/journeys-harness"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The journeys harness (#726): the brief's seven journeys as end-to-end tests, run against the real app stack in Docker, in three projects (desktop, an emulated phone, keyboard-only) with an axe scan on every page. One command locally (
make journeys) and one CI job (journeys, after the image build, artefacts on failure).The stack (
docker-compose.e2e.yml, an overlay on the root compose; the bot and beat are never started): the app; a fake OIDC issuer so the real login and callback code runs and each test picks a seeded identity, with no test-only login in the product; Mailpit as the mail catcher; a fake bot that accepts what the backend posts and also consumes the Redis stream, because that is how a campaign with a guild is told things; a fake AI server speaking the bundled WhisperX and llama.cpp shapes, so the real provider adapters run against it. Zero product code changed..env.e2eis committed with obviously fake values.The seed creates the prototype's fiction through the API as the signed-in identities: Dana Whitlock runs The Amberwake Accord (with a Discord guild), Priya Raghunathan plays in it, Session 13 is completed, Session 14 is confirmed a week ahead; it proves on the way that product mail reaches the catcher and a
session_confirmedevent reaches the fake bot. Each test creates what it mutates under a unique name.Playwright (
webapp/frontend/e2e/): JavaScript, since the repo has no TypeScript toolchain and Playwright would strip types without checking them; three projects;signInAs(),openAccount()andaxe()fixtures. The keyboard project is enforced, not requested: any pointer call throws, and J0 asserts the guard itself. The axe gate is WCAG 2.2 AA and it fails the run, with an owned baseline (a11y-baseline.js) listing the pre-redesign pages' known violations per page, per element, with the lane that removes them; anything else fails, and the message carries axe's own sentence per element.QB_E2E_STRICT_A11Y=1ignores the baseline so a lane can ask whether its page is clean yet.J0, the smoke journey, is written against the new shell (#742): sign in as Dana, land on Home, the campaign in the Contents rail and on the page, the account menu opened with Enter on every project (picked by what is visible, since the phone and desktop controls are different elements), the bell by name, the axe scan open and closed. J1 and J2 are stubbed with their steps as a TODO for lane 1's Home PR to fill in. Nine tests, green in all three projects.
What the gate found on its first day (both filed as #745, tolerated as
BUGentries until fixed): lane 0's account menu puts a radio group insiderole="menu", and draws the person's own name at 1.14:1 because the panel inherits the dark chrome's ink. Also #743 (campaign create silently drops eight declared fields).Repo housekeeping the harness needed:
.gitattributesforcing LF on shell scripts (a CRLFinit.shstops Postgres on a Windows checkout);PLAYWRIGHT_SKIP_BROWSER_DOWNLOADin the frontend images and the CI frontend job; vitest'sincludenarrowed tosrc/so it stops collecting Playwright specs; the Playwright output directories ignored by ESLint; ruff overe2e/; a gitleaks allowlist for.env.e2e; exact pins for@playwright/test,@axe-core/playwrightand the Playwright image, which move together.The one thing this PR's CI run will answer: whether the Forgejo runner's bind mounts reach the workspace (the
synthetic-sessionjob already works around docker-in-docker for the same reason). The script probes it right afterupand fails with a message naming the remedy; if it does, the follow-up is to bakee2e/into the fake and seed images as that job does.Frontend: lint clean, 1,156 tests, build green.
docs:webapp/frontend/e2e/README.md.🤖 Generated with Claude Code
First CI run: every other job green; the
journeysjob failed atdocker compose upwithquestboard-e2e-db-1 exited (126), before the bind-mount probe could run. That is the predicted docker-in-docker case: the runner is a container using the host's daemon, so the checkout's bind-mount sources do not exist on the host and Postgres's entrypoint finds an empty directory whereinit.shshould be. The fix in progress makes the e2e stack bind-mount-free, the way thesynthetic-sessionjob already is: the db init script, the fakes, the seed and the specs baked into their images, with the results copied out for the artefact step.a152a49bd8d47fa958f5Pushed the rework: the e2e stack mounts nothing from the host. There were nine bind mounts across eight services in the effective config (the db's init script,
./backupson three product services, thee2e/tree on the four harness services, and the frontend on the runner), so the probe would only have moved the failure. Three Dockerfile stages replace them, following thesynthetic-sessionjob's precedent:e2e-db(Postgres with the init script copied in),e2e-fakes(the backend image pluse2e/, for the three fakes and the seed) ande2e-playwright(the pinned Playwright image withnpm ciand the specs baked in);./backupsis a named volume. Artefacts come out bydocker cpfrom the runner container before teardown, proven on a deliberately failing run. A new backend test fails ifdocker-compose.e2e.ymlever grows a host path, or if a service the base file mounts into is not restated withvolumes: !override. The local edit loop survives as an opt-indocker-compose.e2e.dev.ymlunderQB_E2E_DEV=1, which CI never loads.Also rebased onto #744, so the bell has real data now; that added no violation. The two baseline entries for #745 stay until #747 merges.
Second run: the
journeysjob is green on the runner. The stack came up with no host mount, the seed ran, the nine smoke tests passed in all three projects, and the artefact path works. The remaining jobs are the ordinary ones. Merging on green.#745 has merged since this branch's last rebase, so the two
BUGbaseline entries are now stale; they come out with lane 1's Home PR, which removes the/dashboardentries as well when Home replaces the dashboard, and the gate will prove both.d47fa958f563259060a3Housekeeping: the first merge attempt answered "try again later" while Forgejo was recomputing mergeability after #747 landed, and a chained cleanup step deleted the head branch before that was noticed, which auto-closed the PR. The branch is restored, the PR reopened, and the real cause of the non-mergeable state, a conflict in
AppFrame.test.jsxbetween #747's new identity-block assertion and this branch's surname rename, is resolved by rebasing onto main (the assertion keeps #747's structure with the prototype's "Dana Whitlock"). CI is running on the rebased head; the merge waits for it, with the cleanup as a separate step this time.Third run's two failures, both read from the log and fixed: the smoke test looked for Dana's name inside the
role="menu"element, which #747 had just moved the identity block out of (the phone project passed because the sheet is different markup), so the account helper now returns the popover's box that holds both the head and the menu panel; andupload-artifact@v4is not supported on Forgejo, so the artefact steps use v3. Nine of nine green locally in CI mode; CI is running again on the new head. The bind-mount-free stack itself worked on the runner, and the artefacts were found where thedocker cpput them.