End-to-end test matrix for the Lakekeeper web consoles. One command spins up a full Lakekeeper stack, serves the app, drives real user journeys in a browser, and produces a visual pass/fail dashboard — across both apps, every auth mode, and multiple browsers.

• console — the open-source console
• console-plus — the premium console (adds Cedar authorization)

New here? Read Quick start, run just test-one console authn, then open the dashboard. The rest of this doc explains the matrix, the modes, and the gotchas.

Every cell is a full stack brought up from scratch, the app served, and the journeys run end to end:

…and each combo runs on chromium (full suite), plus firefox (full parity) and webkit (smoke) as a third dimension. Cedar is premium-only, so it only appears for console-plus.

The modes (this is the whole point — Lakekeeper behaves very differently):

Each spec is a realistic user flow, not a unit test (see specs/):

Component unit tests (Vitest, in the component repos) run once per matrix and show up on the dashboard too — see Unit tests.

• podman with podman compose (this engine drives podman, not docker — see Why podman). Verify: podman compose version.
• Node 20+ and just (brew install just).
• The sibling app + component repos checked out next to this one:

  <workspace>/
    console-e2e/        ← you are here
    console/            ← OSS app
    console-plus/       ← premium app
    console-components/ ← shared component library
    lakekeeper/         ← for the Keycloak realm + (optional) policies
  

cp .env.example .env                  # non-secret config (paths, ports, images)
cp .env.secret.example .env.secret    # secrets (Cedar license, cloud creds)

Edit both. .env and .env.secret are git-ignored — never commit them. See Configuration.

just test-setup        # npm install + playwright browser download (chromium)
npx playwright install firefox webkit   # only if you want the cross-browser dims

just test-one console authn     # one combo (~3 min) — best first run
just test-matrix                # the whole matrix (~35 min) — the release gate
just test-dashboard             # open the visual matrix in a browser

That's it. just --list shows every recipe.

All recipes are namespaced test-:

• just test-dashboard → DASHBOARD.html: every test × every app·mode·browser with the latest pass/fail. Frozen header + frozen first column, scrollable. While a run is active it shows an in-progress banner with the live test name. Each run resets the matrix to just that run's combos (every just test-* clears the previous run's results at start); past runs stay browsable via the archived-run dropdown at the top of the page.
• just test-report → the merged Playwright report: traces, video, and a screenshot of every step (e.g. the CORS test's :3001 result vs :3002 error box).
• just test-catalog → TEST-CATALOG.md/.html: what test cases exist, ordered by journey — a planning view that needs no run.
• History: every full run is archived under history/<timestamp>__<scope> and kept until you delete it (just test-history, test-history-keep).

The normal matrix serves each app from a local npm dev server (testing your working-tree source). just test-matrix-docker instead tests the console that ships inside a pushed lakekeeper-plus image — the embedded UI it serves at :8181/ui, same origin as its API. This validates the release artifact end-to-end, not local source.

• Image: LK_IMAGE_DOCKER in .env (pinned tag; arm64 for Apple-Silicon podman).
• No npm server: SERVED_UI=1 points Playwright's baseURL at :8181; the pre-built image is configured via LAKEKEEPER__UI__* (no VITE_*). Needs the license key.
• Silo only, fully local (no AWS): cloud backends are disabled; the local Silo warehouse vends short-lived creds via STS so the in-browser LoQE write/read round-trip works. S3_LOCAL_DEEP=1 turns on the deep flows (LoQE + access-control) for it; wildcard bucket CORS makes the :3002 CORS-negative test moot.
• Split-horizon: the warehouse endpoint is the host LAN IP (run.mjs auto-detects it) so it resolves from both the lakekeeper container and the host browser.

just test-matrix-docker      # all modes against the pushed image
just test-docker authn       # one mode

The auto-detected LAN-IP endpoint is verified on macOS/Windows (VM-based runtimes). On native Linux / CI the container→host-IP hairpin may need an explicit S3_LOCAL_ENDPOINT override.

Two files, both git-ignored:

.env — non-secret: image tags, host paths to the sibling repos, APP_PORT, test usernames. Key values:

• APP_PORT=3001 — run the apps on :3001. The AWS demo bucket's CORS allows that origin, so the in-browser LoQE Iceberg reads/writes pass. A different port breaks storage CORS (the storage/cors test proves :3002 is blocked).
• CEDAR_POLICY_DIR=./cedar — a self-contained policy dir shipped in this repo. The Cedar access-control test mutates policies.cedar live; don't point this at a real policy dir.

.env.secret — secrets: the Cedar license key (premium image), and cloud storage credentials (AWS/R2/ADLS/…). Storage backends are skip-if-absent: a backend only runs when its creds are present. ⚠️ Use throwaway, least-privilege keys — never a prod key.

The warehouse + LoQE journeys run per enabled backend (see specs/_data/storage-backends.ts):

• Silo — a local S3 (a maintained MinIO fork), started by the stack; always on.

LK_BIN=/path/to/lakekeeper-plus runs the catalog as a native binary instead of the compose container:

LK_BIN=~/Biz/lakekeeper-enterprise/target/release/lakekeeper-plus \
  SERVED_UI=1 just test-docker authn

Why it exists: a macOS build cannot go into a Linux image, so testing an embedded-UI change would otherwise mean a cross-compile first. SERVED_UI only talks to the backend over HTTP, so a native binary serves the embedded console just as well.

The mode files address services by compose name (postgres:5432, openfga:8081, host.docker.internal), which mean nothing on the host, so run.mjs rewrites each to its published host port — which is why postgres and openfga publish ports at all (on non-default numbers, to coexist with a dev stack). Reachable from both the browser and the lakekeeper container via the host LAN IP (auto-detected by run.mjs to solve the split-horizon problem). Create+verify only (no deep browser flows by default).

• AWS S3 — enabled when AWS_* creds are set; runs the deep flows (open detail, LoQE table create/read). Requires an STS-enabled warehouse + a bucket CORS that allows http://localhost:3001 — see LoQE & CORS.
• R2 / ADLS / OneLake / GCS — enabled when their creds are set; otherwise skipped.

Component-repo Vitest tests (e.g. console-components/src/plugins/authToken.test.ts, which guards the token-hydration fix) are browser-independent, so run.mjs runs them once per matrix and shows the result on the dashboard. Run standalone with just test-unit. Disable in a matrix run with NO_UNIT=1.

just test-coverage collects V8 code coverage during the chromium combos (coverage is chromium-only) and maps it back to source via sourcemaps, producing a report per combo under coverage/. It builds console-components with sourcemaps first so coverage maps to real source. (Component-library mapping additionally needs the source-alias — see troubleshooting.) E2E coverage is broad/integration-level; for line coverage of the library use the Vitest unit tests.

• run.mjs — the orchestrator. For each app × mode: compose down -v → bring up infra → migrate → serve lakekeeper → wait for health → run Playwright → (cross-browser passes) → archive + rebuild the dashboard → tear down.
• docker-compose.yml — Postgres, Keycloak, OpenFGA, Silo (+ bucket-init), and Lakekeeper. The Lakekeeper image and modes/<mode>.env are swapped per combo.
• modes/<mode>.env — the per-mode knobs: backend LAKEKEEPER__* (container) + VITE_* (build-time app flags, e.g. VITE_ENABLE_AUTHENTICATION).
• playwright.config.ts — dynamic: reads APP/TEST_MODE/BROWSER, loads that mode's VITE_* into the dev-server env, tags-filters specs by mode, and launches the app on APP_PORT.
• specs/ — the journeys + shared helpers (_utils/, _fixtures/, _data/).

docker on this setup is a shell alias to podman (invisible to Node's spawn), and a standalone docker-compose binary targets the wrong socket. run.mjs drives podman compose so it wires podman's own socket. Override with COMPOSE_BIN / COMPOSE_SUBCMD for a real docker install.

The in-browser DuckDB-WASM engine reads/writes Iceberg data straight from the browser to S3. That only works when (a) the app origin is :3001 and the bucket's CORS allows it, and (b) for AWS, the warehouse is STS-enabled (plain access-key creds write to the bucket root and 404). The catalog API is CORS-*, so the tree loads on any origin — but data reads need bucket CORS. The storage/cors test demonstrates :3001 works / :3002 is blocked.

Local Silo must be reachable from both the browser (host) and the lakekeeper container, which see different hostnames. run.mjs auto-detects the host LAN IP and uses it as the single endpoint so the SigV4 signature matches on both sides.

If the app flashes a "Lakekeeper Unreachable / Check status" page, lakekeeper is usually not down — it's a token-hydration race (an early request goes out before the token is ready). The helpers recover from it (_utils/app.ts). It traces back to a real console-components bug worth fixing (the request interceptor should not attach Bearer undefined).

run.mjs frees :3001/:3002 and waits for release before each combo (--strictPort means a half-released port would otherwise fail the server with "address already in use"). If a run dies mid-combo, just test-down cleans up.

LoQE is pinned to DuckDB-WASM 1.4.x; 1.5.x regressed Iceberg writes.

Match the scope to the change:

• While coding — the relevant unit test (vitest) or one spec via just test-ui.
• Before a PR — just test-one console authn (~3 min) + the unit tests.
• Before a release — just test-matrix (the gate). Ideally in CI so it doesn't block your machine.

run.mjs                 orchestrator (the matrix loop)
docker-compose.yml      the Lakekeeper stack
modes/*.env             per-mode backend + VITE_* config
cedar/policies.cedar    self-contained Cedar base policy (test mutates it)
playwright.config.ts    dynamic per app/mode/browser
dashboard.mjs           builds DASHBOARD.html
catalog.mjs             builds TEST-CATALOG
reporters/current.mjs   live "now-running test" marker for the dashboard
specs/
  _data/                storage backends
  _fixtures/            auth + coverage fixtures
  _utils/               login, warehouse, loqe, permissions, cedar, app helpers
  <area>/*.spec.ts      the journeys

A run clears every stored result first, so the dashboard never shows "IN PROGRESS" beside columns from an earlier run. Pass --keep-results (or KEEP_RESULTS=1) to preserve combos the run will not touch — the case where the npm matrix and the docker matrix are meant to sit side by side.