From 9dbe0c26897b39675eff4e21208b3cd549df2499 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 8 Oct 2026 15:14:02 +0000 Subject: [PATCH] fix(gates): surface status, realpath source root, JOURNEY_DIR token (#233 #234 #235) #233: new test_surface_hit() in gate-check.sh is the one test behind check_stage_6 and the Surface column, so the harness pair (docs/verification/report.html + docs/report.json) no longer reads MISSING while the gate passes. #234: source-sufficiency report title uses realpath basename, so "." / trailing slash no longer yield an empty or "." title. #235: artifact-manifest.tsv accepts a ${JOURNEY_DIR} token; artifact-check.sh resolves it via project-bin/_common.sh (env > toolkit.env > ~/.mxcli-toolkit.env > journeys/), relative or absolute. Ad-hoc checked: default, relative env, absolute env, toolkit.env file. No golden-input capture applies (no parser added). Checks: bash -n; check-portability, check-no-client-data (leak guard), check-scripts, check-pr-discipline; fixtures test-source-sufficiency (51/0), test-source-sufficiency-gate (7/0), test-bug03-gates (31/0), test-wrong-verdicts (20/0). Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01VJgWP5vEoAsNsJYqCDMGNw --- CHANGELOG.md | 1 + bin/gate-check.sh | 32 ++++++++++++++++++++------------ bin/lib/artifact-check.sh | 28 +++++++++++++++++++++++++--- bin/lib/artifact-manifest.tsv | 6 +++++- bin/source-sufficiency.sh | 2 +- 5 files changed, 52 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index df3d67a9..2e93f621 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- fix(gates): **Stage 6 Surface agrees with the gate, source-sufficiency title resolves `.`, journeys artifact row honours `JOURNEY_DIR`.** `test_surface_hit()` is now the one test behind both check_stage_6 and the Surface column (#233); the report title takes the realpath basename (#234); the manifest accepts a `${JOURNEY_DIR}` token resolved like verify-module.sh (#235). Field run 2026-10-08, existing app — MendixMau - perf(routing): **CLAUDE.local.md routing carries triggers only.** The `baseline` view (the copy loaded into every session of every project) now cuts each row's "when" cell at its first " — " and replaces the three-paragraph preamble with two lines pointing at `ROUTING.md` for the full text; `readme-baseline` (README, runbook) is unchanged. Fresh scaffold: CLAUDE.local.md 17.5 KB → 11.7 KB (−33%). Field run 2026-10-07 (existing app, one main session per run, 2 runs per side, an entity + 2 microflows + 2 pages): 10/10 steps applied each side, new mx errors 5 vs 5 (the same CE2421 page-binding slip in both), cost $1.39 vs $1.38; the slim lanes opened the MDL pre-flight skill as often or more. Also drops two stray merge-artifact lines from this section. — MendixMau - perf(ui-loop): **fewer mxbuilds, no fixed sleeps, legible screenshots.** Field run 2026-10-07 (a requirements-driven build): `exec.sh` was 22% of tool time at two mxbuilds per exec, journeys and page waits slept a fixed time, and screenshots were 11.5% of re-read tokens. `exec.sh` now skips the pre-flight baseline mxbuild when the model is byte-identical to its last *clean* pass: the verification stamp gains an `errors:` field (`model-stamp.sh check --clean` accepts only `errors: 0`, so a delta-gate pass over pre-existing errors never counts; `SKIP_BASELINE=0` is unchanged). New `tests/e2e/settle.js` waits for the page's own state (no visible Mendix progress/loading indicator, DOM size stable) instead of a fixed time; `journey-runner.js` uses it after each action (an explicit `settleMs` still wins) and `page-audit.js` per page. Probed in a real browser: on a page that loads in 1.8 s the old 1200 ms sleep and networkidle+600 both captured it half-loaded, settle waited for it; on a fast page settle took 775 ms vs 1200. `SETTLE_MODE=fixed` restores the old waits for an A/B run. `page-audit.js` also saves a viewport shot `page-audit-.top.png` beside the full-page one, and `ui-loop.md` says to look at viewport shots: a tall full-page shot reaches the model at ~1024 px high, text unreadable. `shrink-image-read.sh` now works off macOS (ImageMagick or Pillow, python when jq is absent; it was a silent no-op without `sips`). — MendixMau - perf(ui-loop): **a stale app reloads instead of restarting, journeys stop sleeping, worst pages are looked at first.** Field run 2026-10-07, existing app. `test-stack-up.sh` records which model the running app was built from (`.claude/loop/served-model`); when the model has changed since, it hot-reloads with `mxcli docker reload` (adding `--skip-check` when the exec.sh gate already passed this model) and restarts only if that fails: 93 s instead of 179 s. `ui-loop.md` and the script header carry the measured table and point at `mxcli run --local --watch` as the fast loop (~18 s per page change). `journey-runner.js` waits on page state after a combobox action instead of fixed pauses, and a combobox step now opens the widget it names: it used to open the page's first combobox, so on a form with three every step but the first failed. `page-audit.js` clicks the first copy of a nav item a user could actually click, not just the first match: with the sidebar collapsed, every in-nav page faulted while the same item sat in the top menu bar. It also writes `page-audit-look-order.txt`, all pages worst first (`module-review.md` §4a: order, not selection, every page is still looked at). `design-audit.js` drops its fixed 1.2 s wait. Same 7-step journey on the same app: 56 s with `SETTLE_MODE=fixed`, 24 s with settle, 11/11 pass both ways, 3 runs each. — MendixMau diff --git a/bin/gate-check.sh b/bin/gate-check.sh index c1772943..d4705782 100755 --- a/bin/gate-check.sh +++ b/bin/gate-check.sh @@ -1315,21 +1315,26 @@ EOF fi } -check_stage_6() { - local f test_ok="" review_ok="" +# The one test for "is there a test surface?" — check_stage_6 and the dashboard's Surface +# column both call it, so the gate and the column cannot disagree about one file (#233). +# Accepts test-report.html (root or reports/, project or analysis base) OR the harness pair +# verify-module.sh's render rung produces: docs/verification/report.html rendered FROM +# docs/report.json. Both halves must be present — accepting one without the other would accept +# a surface whose machine half is gone. (Before this, test-report.html was the only accepted +# name and nothing in the toolkit generated it, so Stage 6 was structurally unpassable — +# improvement-plan Finding 14 / P9.) +test_surface_hit() { + local f for f in "$PROJECT_DIR/test-report.html" "$PROJECT_DIR"/reports/test-report.html \ "$ANALYSIS_BASE/test-report.html" "$ANALYSIS_BASE"/reports/test-report.html; do - [ -s "$f" ] && test_ok=1 + [ -s "$f" ] && return 0 done - # The harness's own surface also satisfies the test half: verify-module.sh's final render - # rung produces docs/verification/report.html from docs/report.json (report-render.js / - # report-normalize.js). Both halves must be present — the HTML is rendered FROM the JSON, - # and accepting one without the other would accept a surface whose machine half is gone. - # Before this, test-report.html was the only accepted name and nothing in the toolkit - # generated it, so Stage 6 was structurally unpassable (improvement-plan Finding 14 / P9). - if [ -s "$PROJECT_DIR/docs/verification/report.html" ] && [ -s "$PROJECT_DIR/docs/report.json" ]; then - test_ok=1 - fi + [ -s "$PROJECT_DIR/docs/verification/report.html" ] && [ -s "$PROJECT_DIR/docs/report.json" ] +} + +check_stage_6() { + local test_ok="" review_ok="" + test_surface_hit && test_ok=1 # module-review.md report — any non-empty dated report under a ui-reviews/ dir if [ -n "$(find_artifact -path '*/ui-reviews/ui-review-*.html' -size +0c)" ]; then review_ok=1 @@ -1706,6 +1711,9 @@ stage_surface_status() { fi for pat in $pats; do hit=0 + # test-report.html is answered by the same function check_stage_6 uses (#233), so the + # harness report pair also discharges it. + if [ "$pat" = "test-report.html" ] && test_surface_hit; then continue; fi # SURFACE_BASES, not just the project root. §2's Surface cells are inconsistent about # paths: some carry one ("architecture/blueprint.html", "docs/report.json") and some name # the file bare ("design-system.html", "build-plan.html", "test-report.html"). A bare cell diff --git a/bin/lib/artifact-check.sh b/bin/lib/artifact-check.sh index e2f5ee6a..7da2fa3e 100755 --- a/bin/lib/artifact-check.sh +++ b/bin/lib/artifact-check.sh @@ -189,9 +189,29 @@ _art_newer() { } # resolve_artifact() already serves. + +# Resolve the ${JOURNEY_DIR} manifest token (#235) with verify-module.sh's own precedence +# (env > /.claude/toolkit.env > ~/.mxcli-toolkit.env > /journeys) by sourcing +# project-bin/_common.sh in a subshell with PROJECT_ROOT pinned. Prints a path relative to the +# root when it lies under it, otherwise as given (absolute); falls back to "journeys". +_art_journey_dir() { + local root="$1" common="$_ART_LIB_DIR/../../project-bin/_common.sh" jd="" + if [ -f "$common" ]; then + jd="$( export PROJECT_ROOT="$root"; . "$common" >/dev/null 2>&1; printf '%s' "${JOURNEY_DIR:-}" )" || jd="" + else + jd="${JOURNEY_DIR:-}" + fi + jd="${jd:-journeys}"; jd="${jd%/}" + case "$jd" in "$root"/*) jd="${jd#"$root"/}" ;; esac + printf '%s' "$jd" +} + _art_find() { - local root="$1" paths="$2" pat base f rest + local root="$1" paths="$2" pat base f rest jd ART_HIT=""; ART_EMPTY="" + case "$paths" in + *'${JOURNEY_DIR}'*) jd="$(_art_journey_dir "$root")"; paths="${paths//\$\{JOURNEY_DIR\}/$jd}" ;; + esac rest="$paths" while [ -n "$rest" ]; do pat="${rest%%|*}" @@ -207,8 +227,10 @@ _art_find() { # defect, same day, as the one fixed in obligation-check.sh's `_ob_find`, where it DID # change the verdict. local newest_hit="" - for base in "$root" "$root"/analysis/*; do - [ -d "$base" ] || continue + local bases=("$root" "$root"/analysis/*) + case "$pat" in /*) bases=("") ;; esac # absolute (a JOURNEY_DIR outside the root) + for base in "${bases[@]}"; do + [ -z "$base" ] || [ -d "$base" ] || continue # base quoted, pattern not: the base must survive spaces, the pattern must glob. for f in "$base"/$pat; do [ -e "$f" ] || continue diff --git a/bin/lib/artifact-manifest.tsv b/bin/lib/artifact-manifest.tsv index b830de02..93c912ab 100644 --- a/bin/lib/artifact-manifest.tsv +++ b/bin/lib/artifact-manifest.tsv @@ -36,6 +36,10 @@ # ANALYSIS_BASE serves). Alternatives exist where real projects legitimately # differ (the three ledger shapes of skills/coverage-ledger.md §"Where the # ledger lives"), never as a hedge against not knowing the convention. +# A path may start with the token ${JOURNEY_DIR}: artifact-check.sh resolves it the +# way verify-module.sh does (env > .claude/toolkit.env > ~/.mxcli-toolkit.env > +# journeys/), so a project keeping journeys elsewhere is judged where its journey +# rung actually reads them (#235). # 4 producer the skill step or script that emits it — the thing a PENDING row tells you # to run # 5 consumers the citations that earn the row its place (see above) @@ -92,7 +96,7 @@ build-plan-json 4 architecture/build-plan.json project-bin/build-plan-status.sh module-brief 4 architecture/modules/*/module-brief.md|architecture/modules/*-brief.md skills/module-brief.md — ba-agent writes the first at Stage 4, the rest just-in-time bin/gate-check.sh:1121 (build-ready fails without one), project-bin/build-plan-status.sh:109 migration,requirements,existing-app-change gate coverage-ledger 4 architecture/modules/*/coverage-ledger.md|architecture/modules/*-coverage-ledger.md|architecture/coverage-ledger.md skills/coverage-ledger.md procedure at Stage 4 (BRD leaves × build-plan claims); the three shapes are §"Where the ledger lives", first canonical project-bin/conformance-check.sh:85-93, project-bin/coverage-preflight.sh:134-137, project-bin/verify-module.sh:299-300, project-bin/review-module.sh:289, conversion-runbook.md:487 (Stage-4 produces row) — the eleven-consumers-no-producer artifact of improvement-plan Finding 12 migration,requirements report ui-reviews 5 design/ui-reviews/ui-review-*.html skills/module-review.md pass (review-agent), one report per module close-out bin/gate-check.sh:984 (check_stage_6), bin/lib/obligations.tsv:47 (the look row — per-module coverage and denominators live THERE; this row only asks whether the stage produced any report at all) migration,requirements,greenfield,existing-app-change gate -journeys 5 journeys/*.journey.json journey authoring per skills/testing-shape.md / skills/journey-proof.md (worked example: project-tests/e2e/example.journey.json) project-bin/verify-module.sh:376,404 (the journeys rung), project-tests/e2e/journey-runner.js:5 migration,requirements,greenfield,existing-app-change report +journeys 5 ${JOURNEY_DIR}/*.journey.json journey authoring per skills/testing-shape.md / skills/journey-proof.md (worked example: project-tests/e2e/example.journey.json) project-bin/verify-module.sh:376,404 (the journeys rung), project-tests/e2e/journey-runner.js:5 migration,requirements,greenfield,existing-app-change report page-scope 5 .claude/loop/page-scope.json project-bin/page-scope.sh (the phantom of improvement-plan Finding 15 until 078c7d8 shipped it) project-tests/e2e/design-audit.js:55, project-tests/e2e/page-audit.js:66, project-tests/e2e/report-normalize.js:112 — without it the LOOK instruments degrade to --static-only migration,requirements,greenfield,existing-app-change report lint-last 5 .claude/loop/lint-last.json project-bin/lint-gate.sh (written on every run that reaches a verdict, not only --update-baseline), run as part of skills/module-review.md stage 2 GATE ("bin/exec.sh; gate-agent confirms 0 mxbuild errors, lint clean" — same step named in skills/iterative-build-loop.md's "Gate: BUILD") bin/status.sh:75-81,129 (the OVERDUE line's lint readout — without it status.sh reports "lint gate never recorded here") migration,requirements,greenfield,existing-app-change report improvement-register 5 docs/improvement-register.md skills/improvement-register.md:39 (append-only, one row per finding, written during build/verify passes) project-bin/build-plan-status.sh:69,148 (the proven-vs-built cross-reference), skills/close-the-loop.md migration,requirements,greenfield,existing-app-change report diff --git a/bin/source-sufficiency.sh b/bin/source-sufficiency.sh index cb50d41e..016b2218 100755 --- a/bin/source-sufficiency.sh +++ b/bin/source-sufficiency.sh @@ -622,7 +622,7 @@ if html_out: src_list = ''.join(f'
  • {esc(os.path.basename(s))}
    • ' for s in sources) \ or '
  • no source files named
    • ' - page = f"""Source sufficiency — {esc(os.path.basename(os.environ.get('PROJECT_DIR','')))} + page = f"""Source sufficiency — {esc(os.path.basename(os.path.realpath(os.environ.get('PROJECT_DIR',''))))}