diff --git a/CHANGELOG.md b/CHANGELOG.md index df3d67a9..2f1aff99 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are. ## Unreleased +- new(lint): **the lint gate keeps a ledger, so a rule's history is readable.** `project-bin/lint-gate.sh` appends one row per rule per run to `.claude/loop/lint-ledger.tsv` (ts, verdict, rule, count, severity, blind — plus a `_run` row with the total), and new `project-bin/lint-trend.sh` prints per rule: runs fired in, first/last count with dates, best/worst, blind runs, and up/down/flat (`--rule ` for one rule's series, `--runs` for the run rows). Until now the only per-rule history was `lint-baseline.json` in git, which records the count that was accepted, not what each run saw — so "did the rule we shipped catch anything, and did it go down" had no answer. This is the measurement the lint rollout plan's "promote after two clean projects" step depends on. Probed on the gate's own parser with a crafted lint result (3 findings, one blind rule): ledger rows and trend output as expected; `lint-last.json` and the ratchet verdict unchanged. Field run on a real model still owed (laptop). — MendixMau - perf(routing): **CLAUDE.local.md routing carries triggers only.** The `baseline` view (the copy loaded into every session of every project) now cuts each row's "when" cell at its first " — " and replaces the three-paragraph preamble with two lines pointing at `ROUTING.md` for the full text; `readme-baseline` (README, runbook) is unchanged. Fresh scaffold: CLAUDE.local.md 17.5 KB → 11.7 KB (−33%). Field run 2026-10-07 (existing app, one main session per run, 2 runs per side, an entity + 2 microflows + 2 pages): 10/10 steps applied each side, new mx errors 5 vs 5 (the same CE2421 page-binding slip in both), cost $1.39 vs $1.38; the slim lanes opened the MDL pre-flight skill as often or more. Also drops two stray merge-artifact lines from this section. — MendixMau - perf(ui-loop): **fewer mxbuilds, no fixed sleeps, legible screenshots.** Field run 2026-10-07 (a requirements-driven build): `exec.sh` was 22% of tool time at two mxbuilds per exec, journeys and page waits slept a fixed time, and screenshots were 11.5% of re-read tokens. `exec.sh` now skips the pre-flight baseline mxbuild when the model is byte-identical to its last *clean* pass: the verification stamp gains an `errors:` field (`model-stamp.sh check --clean` accepts only `errors: 0`, so a delta-gate pass over pre-existing errors never counts; `SKIP_BASELINE=0` is unchanged). New `tests/e2e/settle.js` waits for the page's own state (no visible Mendix progress/loading indicator, DOM size stable) instead of a fixed time; `journey-runner.js` uses it after each action (an explicit `settleMs` still wins) and `page-audit.js` per page. Probed in a real browser: on a page that loads in 1.8 s the old 1200 ms sleep and networkidle+600 both captured it half-loaded, settle waited for it; on a fast page settle took 775 ms vs 1200. `SETTLE_MODE=fixed` restores the old waits for an A/B run. `page-audit.js` also saves a viewport shot `page-audit-.top.png` beside the full-page one, and `ui-loop.md` says to look at viewport shots: a tall full-page shot reaches the model at ~1024 px high, text unreadable. `shrink-image-read.sh` now works off macOS (ImageMagick or Pillow, python when jq is absent; it was a silent no-op without `sips`). — MendixMau - perf(ui-loop): **a stale app reloads instead of restarting, journeys stop sleeping, worst pages are looked at first.** Field run 2026-10-07, existing app. `test-stack-up.sh` records which model the running app was built from (`.claude/loop/served-model`); when the model has changed since, it hot-reloads with `mxcli docker reload` (adding `--skip-check` when the exec.sh gate already passed this model) and restarts only if that fails: 93 s instead of 179 s. `ui-loop.md` and the script header carry the measured table and point at `mxcli run --local --watch` as the fast loop (~18 s per page change). `journey-runner.js` waits on page state after a combobox action instead of fixed pauses, and a combobox step now opens the widget it names: it used to open the page's first combobox, so on a form with three every step but the first failed. `page-audit.js` clicks the first copy of a nav item a user could actually click, not just the first match: with the sidebar collapsed, every in-nav page faulted while the same item sat in the top menu bar. It also writes `page-audit-look-order.txt`, all pages worst first (`module-review.md` §4a: order, not selection, every page is still looked at). `design-audit.js` drops its fixed 1.2 s wait. Same 7-step journey on the same app: 56 s with `SETTLE_MODE=fixed`, 24 s with settle, 11/11 pass both ways, 3 runs each. — MendixMau diff --git a/bin/lib/install-manifest.sh b/bin/lib/install-manifest.sh index 4bb29c67..3b831d7d 100644 --- a/bin/lib/install-manifest.sh +++ b/bin/lib/install-manifest.sh @@ -110,7 +110,7 @@ MXTK_AGENTS_STAGE_BUILD="mdl-agent.md gate-agent.md test-agent.md review-agent.m # own sibling, and the PostToolUse(Read) hook install-claude-permissions.sh writes runs # `bash bin/look-ledger.sh seen`. Two unattended builds built every page and opened no # screenshot of one; the Stage 5 gate now joins what this script records. -MXTK_PROJECT_BIN="assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh session-check.sh constants-audit.sh context-pack.sh" +MXTK_PROJECT_BIN="assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh session-check.sh constants-audit.sh context-pack.sh" # Files in project-bin/ that are deliberately NOT installed into projects. The reverse check # below flags anything named by NEITHER list, so a new file in project-bin/ has to be either diff --git a/project-bin/lint-gate.sh b/project-bin/lint-gate.sh index a2e7069c..dca90311 100755 --- a/project-bin/lint-gate.sh +++ b/project-bin/lint-gate.sh @@ -18,7 +18,8 @@ # # Exit codes: 0 clean or unchanged · 1 a rule increased · 2 could not run # -# PRODUCES .claude/loop/lint-last.json on every run that actually gets as far as computing +# PRODUCES .claude/loop/lint-ledger.tsv (append-only, one row per rule per run; read with +# project-bin/lint-trend.sh) and .claude/loop/lint-last.json on every run that actually gets as far as computing # counts (timestamp, verdict, total, per-rule counts) — the dir is created if absent. This is # NOT docs/BUILD-LOG.md (the exec table Studio's parseExecRows / bin/status.sh / # project-bin/done-drift-check.sh parse) and must never become another writer of it; it is a @@ -155,6 +156,39 @@ def write_last(verdict, viol_list, counts_now): open(last_path, "a").write("\n") except OSError as e: print("lint-gate: could not write %s: %s" % (last_path, e), file=sys.stderr) + write_ledger(verdict, viol_list, counts_now) + +# LEDGER. lint-last.json is overwritten every run, so it can answer "what did lint say last +# time" and nothing else. The ledger is append-only: one row per rule per run, so a rule's +# count can be read ACROSS runs — did the thing we shipped ever fire, and did its count go +# down once people saw it. That is the question the rollout plan in process/lint-backlog.md +# asks before promoting a warning to an error ("two clean projects"), and until this file +# existed nobody could answer it without a git archaeology of lint-baseline.json. +# +# .claude/loop/lint-ledger.tsv ts verdict rule count severity blind +# +# `blind` is 1 when the rule emitted a `_rule` finding that run (it inspected nothing), so a +# zero next to a 1 is never read as clean. Rules with zero findings are NOT written: a rule's +# absence on a date means "ran, found nothing" only if the run row (rule `_run`) is there — +# which it always is, carrying the total. Read it with project-bin/lint-trend.sh. +def write_ledger(verdict, viol_list, counts_now): + try: + loop_dir = os.path.join(os.path.dirname(base_path), "loop") + os.makedirs(loop_dir, exist_ok=True) + ledger = os.path.join(loop_dir, "lint-ledger.tsv") + ts = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") + blind_rules = {v["ruleId"] for v in viol_list if v.get("module") == "_rule"} + sev_now = {v["ruleId"]: v.get("severity", "") for v in viol_list} + new_file = not os.path.exists(ledger) + with open(ledger, "a") as f: + if new_file: + f.write("ts\tverdict\trule\tcount\tseverity\tblind\n") + f.write("%s\t%s\t_run\t%d\t\t%d\n" % (ts, verdict, len(viol_list), 1 if blind_rules else 0)) + for r in sorted(counts_now): + f.write("%s\t%s\t%s\t%d\t%s\t%d\n" % ( + ts, verdict, r, counts_now[r], sev_now.get(r, ""), 1 if r in blind_rules else 0)) + except OSError as e: + print("lint-gate: could not append %s: %s" % (ledger, e), file=sys.stderr) # mxcli prints a plaintext progress banner before the JSON; slice from the first brace. raw = open(out_path).read() diff --git a/project-bin/lint-trend.sh b/project-bin/lint-trend.sh new file mode 100755 index 00000000..fb22fb67 --- /dev/null +++ b/project-bin/lint-trend.sh @@ -0,0 +1,109 @@ +#!/usr/bin/env bash +# lint-trend.sh — did a lint rule ever fire here, and is its count going down? +# +# Reads the append-only ledger project-bin/lint-gate.sh writes on every run +# (.claude/loop/lint-ledger.tsv: ts, verdict, rule, count, severity, blind) and prints one +# line per rule: runs it fired in, first and last count with dates, best and worst, and how +# many of those runs it was blind (a `_rule` self-check finding — it inspected nothing). +# +# This is the evidence the rollout plan (process/lint-backlog.md) asks for before a warning +# becomes an error: a rule that fired and whose count fell is doing its job; a rule that +# never fired is either clean here or wrong, and the blind column says which. Without this +# file the only trace of a rule's history was lint-baseline.json in git, which records the +# count that was ACCEPTED, not what each run saw. +# +# Usage: +# project-bin/lint-trend.sh # one line per rule, worst first +# project-bin/lint-trend.sh --rule UX001 # every run's count for one rule, oldest first +# project-bin/lint-trend.sh --runs # the run rows only (ts, verdict, total, blind) +# +# Exit 0 with output; 2 when there is no ledger yet (the gate has never run here). Read-only. +# Keep bash-3.2 compatible. + +set -uo pipefail +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +LEDGER="$ROOT/.claude/loop/lint-ledger.tsv" + +if [ -f "$(dirname "${BASH_SOURCE[0]}")/_common.sh" ]; then + . "$(dirname "${BASH_SOURCE[0]}")/_common.sh" +fi +if ! type require_py >/dev/null 2>&1; then + require_py() { + _c="" + for _c in python3 python py; do # portability-ok: this IS the interpreter probe + case "$(command -v "$_c" 2>/dev/null)" in *[Ww]indows[Aa]pps*) continue ;; esac + if "$_c" -c 'import sys; sys.exit(0 if sys.version_info[0] == 3 else 1)' >/dev/null 2>&1; then + PY="$_c"; export PY; return 0 + fi + done + echo "lint-trend: Python 3 is required and was not found (tried python3, python, py)." >&2 # portability-ok: names in a diagnostic + exit 2 + } +fi +require_py + +MODE="rules"; RULE="" +while [ $# -gt 0 ]; do + case "$1" in + --rule) MODE="one"; RULE="${2:-}"; [ -n "$RULE" ] || { echo "lint-trend: --rule needs an id" >&2; exit 2; }; shift ;; + --runs) MODE="runs" ;; + -h|--help) sed -n '2,22p' "$0"; exit 0 ;; + *) echo "lint-trend: unknown argument $1" >&2; exit 2 ;; + esac + shift +done + +if [ ! -f "$LEDGER" ]; then + echo "lint-trend: no ledger at $LEDGER — the lint gate has not run here yet (project-bin/lint-gate.sh writes it)." >&2 + exit 2 +fi + +LEDGER="$LEDGER" MODE="$MODE" RULE="$RULE" "$PY" - <<'PY' +import csv, os, sys +rows = [] +with open(os.environ["LEDGER"]) as f: + for r in csv.DictReader(f, delimiter="\t"): + try: + r["count"] = int(r["count"]); r["blind"] = int(r.get("blind") or 0) + except ValueError: + continue + rows.append(r) +mode, want = os.environ["MODE"], os.environ["RULE"] + +if mode == "runs": + print("%-20s %-18s %6s %s" % ("ts", "verdict", "total", "blind")) + for r in rows: + if r["rule"] == "_run": + print("%-20s %-18s %6d %s" % (r["ts"], r["verdict"], r["count"], "yes" if r["blind"] else "")) + sys.exit(0) + +runs = [r for r in rows if r["rule"] == "_run"] +if mode == "one": + hits = [r for r in rows if r["rule"] == want] + if not hits: + print("%s: never fired in %d run(s) here." % (want, len(runs))) + sys.exit(0) + print("%-20s %6s %-8s %s" % ("ts", "count", "severity", "blind")) + for r in hits: + print("%-20s %6d %-8s %s" % (r["ts"], r["count"], r["severity"], "yes" if r["blind"] else "")) + sys.exit(0) + +by = {} +for r in rows: + if r["rule"] == "_run": + continue + by.setdefault(r["rule"], []).append(r) +if not by: + print("No rule has fired in %d run(s) here." % len(runs)) + sys.exit(0) +print("%d run(s), first %s, last %s" % (len(runs), runs[0]["ts"][:10] if runs else "?", runs[-1]["ts"][:10] if runs else "?")) +print("%-10s %-8s %5s %-18s %-18s %5s %5s %s" % ("rule", "severity", "runs", "first", "last", "best", "worst", "blind")) +for rule, hs in sorted(by.items(), key=lambda kv: -kv[1][-1]["count"]): + counts = [h["count"] for h in hs] + trend = "" if len(hs) < 2 else (" down" if counts[-1] < counts[0] else (" UP" if counts[-1] > counts[0] else " flat")) + blind = sum(h["blind"] for h in hs) + print("%-10s %-8s %5d %-18s %-18s %5d %5d %s%s" % ( + rule, hs[-1]["severity"], len(hs), + "%d @ %s" % (counts[0], hs[0]["ts"][:10]), "%d @ %s" % (counts[-1], hs[-1]["ts"][:10]), + min(counts), max(counts), ("%d BLIND" % blind) if blind else "", trend)) +PY diff --git a/skills/lint-that-actually-runs.md b/skills/lint-that-actually-runs.md index ed2c7195..ab304477 100644 --- a/skills/lint-that-actually-runs.md +++ b/skills/lint-that-actually-runs.md @@ -111,6 +111,16 @@ all the same order of magnitude, and no project produced a single real error-sev the ratchet is the *only* mechanism available. `project-bin/lint-gate.sh` implements it (`--update-baseline` to accept current counts). +**The ratchet remembers one number per rule; the ledger remembers every run.** Every gate run +appends one row per rule to `.claude/loop/lint-ledger.tsv` (timestamp, verdict, rule, count, +severity, blind), and `project-bin/lint-trend.sh` reads it back: runs a rule fired in, first and +last count with dates, best/worst, and how many runs it was blind. That is the answer to "did +the rule we shipped ever catch anything here, and did the count go down once people saw it" — +the evidence `process/lint-backlog.md` asks for before a warning becomes an error. A rule that +never appears in the ledger is clean here *or* wrong; `--rule ` and the blind column say +which. Before this file existed the only history was `lint-baseline.json` in git, which records +the count that was *accepted*, not what each run saw. + ## Two false-pass bugs already designed out of `lint-gate.sh` — keep them out Both were found by testing the gate, not by reading it: