Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Sections dated before 2026-09-19 predate the cycle and stay as they are.

## Unreleased

- new(lint): **the lint gate keeps a ledger, so a rule's history is readable.** `project-bin/lint-gate.sh` appends one row per rule per run to `.claude/loop/lint-ledger.tsv` (ts, verdict, rule, count, severity, blind — plus a `_run` row with the total), and new `project-bin/lint-trend.sh` prints per rule: runs fired in, first/last count with dates, best/worst, blind runs, and up/down/flat (`--rule <id>` for one rule's series, `--runs` for the run rows). Until now the only per-rule history was `lint-baseline.json` in git, which records the count that was accepted, not what each run saw — so "did the rule we shipped catch anything, and did it go down" had no answer. This is the measurement the lint rollout plan's "promote after two clean projects" step depends on. Probed on the gate's own parser with a crafted lint result (3 findings, one blind rule): ledger rows and trend output as expected; `lint-last.json` and the ratchet verdict unchanged. Field run on a real model still owed (laptop). — MendixMau
- perf(routing): **CLAUDE.local.md routing carries triggers only.** The `baseline` view (the copy loaded into every session of every project) now cuts each row's "when" cell at its first " — " and replaces the three-paragraph preamble with two lines pointing at `ROUTING.md` for the full text; `readme-baseline` (README, runbook) is unchanged. Fresh scaffold: CLAUDE.local.md 17.5 KB → 11.7 KB (−33%). Field run 2026-10-07 (existing app, one main session per run, 2 runs per side, an entity + 2 microflows + 2 pages): 10/10 steps applied each side, new mx errors 5 vs 5 (the same CE2421 page-binding slip in both), cost $1.39 vs $1.38; the slim lanes opened the MDL pre-flight skill as often or more. Also drops two stray merge-artifact lines from this section. — MendixMau
- perf(ui-loop): **fewer mxbuilds, no fixed sleeps, legible screenshots.** Field run 2026-10-07 (a requirements-driven build): `exec.sh` was 22% of tool time at two mxbuilds per exec, journeys and page waits slept a fixed time, and screenshots were 11.5% of re-read tokens. `exec.sh` now skips the pre-flight baseline mxbuild when the model is byte-identical to its last *clean* pass: the verification stamp gains an `errors:` field (`model-stamp.sh check --clean` accepts only `errors: 0`, so a delta-gate pass over pre-existing errors never counts; `SKIP_BASELINE=0` is unchanged). New `tests/e2e/settle.js` waits for the page's own state (no visible Mendix progress/loading indicator, DOM size stable) instead of a fixed time; `journey-runner.js` uses it after each action (an explicit `settleMs` still wins) and `page-audit.js` per page. Probed in a real browser: on a page that loads in 1.8 s the old 1200 ms sleep and networkidle+600 both captured it half-loaded, settle waited for it; on a fast page settle took 775 ms vs 1200. `SETTLE_MODE=fixed` restores the old waits for an A/B run. `page-audit.js` also saves a viewport shot `page-audit-<Page>.top.png` beside the full-page one, and `ui-loop.md` says to look at viewport shots: a tall full-page shot reaches the model at ~1024 px high, text unreadable. `shrink-image-read.sh` now works off macOS (ImageMagick or Pillow, python when jq is absent; it was a silent no-op without `sips`). — MendixMau
- perf(ui-loop): **a stale app reloads instead of restarting, journeys stop sleeping, worst pages are looked at first.** Field run 2026-10-07, existing app. `test-stack-up.sh` records which model the running app was built from (`.claude/loop/served-model`); when the model has changed since, it hot-reloads with `mxcli docker reload` (adding `--skip-check` when the exec.sh gate already passed this model) and restarts only if that fails: 93 s instead of 179 s. `ui-loop.md` and the script header carry the measured table and point at `mxcli run --local --watch` as the fast loop (~18 s per page change). `journey-runner.js` waits on page state after a combobox action instead of fixed pauses, and a combobox step now opens the widget it names: it used to open the page's first combobox, so on a form with three every step but the first failed. `page-audit.js` clicks the first copy of a nav item a user could actually click, not just the first match: with the sidebar collapsed, every in-nav page faulted while the same item sat in the top menu bar. It also writes `page-audit-look-order.txt`, all pages worst first (`module-review.md` §4a: order, not selection, every page is still looked at). `design-audit.js` drops its fixed 1.2 s wait. Same 7-step journey on the same app: 56 s with `SETTLE_MODE=fixed`, 24 s with settle, 11/11 pass both ways, 3 runs each. — MendixMau
Expand Down
2 changes: 1 addition & 1 deletion bin/lib/install-manifest.sh
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ MXTK_AGENTS_STAGE_BUILD="mdl-agent.md gate-agent.md test-agent.md review-agent.m
# own sibling, and the PostToolUse(Read) hook install-claude-permissions.sh writes runs
# `bash bin/look-ledger.sh seen`. Two unattended builds built every page and opened no
# screenshot of one; the Stage 5 gate now joins what this script records.
MXTK_PROJECT_BIN="assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh session-check.sh constants-audit.sh context-pack.sh"
MXTK_PROJECT_BIN="assemble-prototype.js prototype-route.js check-prototype-links.js wf-add-path-terminators.py wf-set-call-captions.py mxunit_bson.py _common.sh app-facts.sh _claims.sh snapshot-mpr.sh restore-mpr.sh exec.sh look-ledger.sh save-sp.sh restart-sp.sh check-sp-health.sh verify-module.sh test-stack-up.sh fixture-manifest.sh check-root-clean.sh lint-gate.sh lint-trend.sh close-task.sh conformance-check.sh coverage-preflight.sh graph-sweep.sh review-module.sh coherence-cadence.sh build-plan-status.sh done-drift-check.sh page-scope.sh render-improvement-register.sh check-design-portability.sh check-design-reaches-app.sh check-page-shell.sh page-fidelity.js model-stamp.sh verify-model.sh install-project-hooks.sh session-check.sh constants-audit.sh context-pack.sh"

# Files in project-bin/ that are deliberately NOT installed into projects. The reverse check
# below flags anything named by NEITHER list, so a new file in project-bin/ has to be either
Expand Down
36 changes: 35 additions & 1 deletion project-bin/lint-gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@
#
# Exit codes: 0 clean or unchanged · 1 a rule increased · 2 could not run
#
# PRODUCES .claude/loop/lint-last.json on every run that actually gets as far as computing
# PRODUCES .claude/loop/lint-ledger.tsv (append-only, one row per rule per run; read with
# project-bin/lint-trend.sh) and .claude/loop/lint-last.json on every run that actually gets as far as computing
# counts (timestamp, verdict, total, per-rule counts) — the dir is created if absent. This is
# NOT docs/BUILD-LOG.md (the exec table Studio's parseExecRows / bin/status.sh /
# project-bin/done-drift-check.sh parse) and must never become another writer of it; it is a
Expand Down Expand Up @@ -155,6 +156,39 @@ def write_last(verdict, viol_list, counts_now):
open(last_path, "a").write("\n")
except OSError as e:
print("lint-gate: could not write %s: %s" % (last_path, e), file=sys.stderr)
write_ledger(verdict, viol_list, counts_now)

# LEDGER. lint-last.json is overwritten every run, so it can answer "what did lint say last
# time" and nothing else. The ledger is append-only: one row per rule per run, so a rule's
# count can be read ACROSS runs — did the thing we shipped ever fire, and did its count go
# down once people saw it. That is the question the rollout plan in process/lint-backlog.md
# asks before promoting a warning to an error ("two clean projects"), and until this file
# existed nobody could answer it without a git archaeology of lint-baseline.json.
#
# .claude/loop/lint-ledger.tsv ts verdict rule count severity blind
#
# `blind` is 1 when the rule emitted a `_rule` finding that run (it inspected nothing), so a
# zero next to a 1 is never read as clean. Rules with zero findings are NOT written: a rule's
# absence on a date means "ran, found nothing" only if the run row (rule `_run`) is there —
# which it always is, carrying the total. Read it with project-bin/lint-trend.sh.
def write_ledger(verdict, viol_list, counts_now):
try:
loop_dir = os.path.join(os.path.dirname(base_path), "loop")
os.makedirs(loop_dir, exist_ok=True)
ledger = os.path.join(loop_dir, "lint-ledger.tsv")
ts = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
blind_rules = {v["ruleId"] for v in viol_list if v.get("module") == "_rule"}
sev_now = {v["ruleId"]: v.get("severity", "") for v in viol_list}
new_file = not os.path.exists(ledger)
with open(ledger, "a") as f:
if new_file:
f.write("ts\tverdict\trule\tcount\tseverity\tblind\n")
f.write("%s\t%s\t_run\t%d\t\t%d\n" % (ts, verdict, len(viol_list), 1 if blind_rules else 0))
for r in sorted(counts_now):
f.write("%s\t%s\t%s\t%d\t%s\t%d\n" % (
ts, verdict, r, counts_now[r], sev_now.get(r, ""), 1 if r in blind_rules else 0))
except OSError as e:
print("lint-gate: could not append %s: %s" % (ledger, e), file=sys.stderr)

# mxcli prints a plaintext progress banner before the JSON; slice from the first brace.
raw = open(out_path).read()
Expand Down
109 changes: 109 additions & 0 deletions project-bin/lint-trend.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
#!/usr/bin/env bash
# lint-trend.sh — did a lint rule ever fire here, and is its count going down?
#
# Reads the append-only ledger project-bin/lint-gate.sh writes on every run
# (.claude/loop/lint-ledger.tsv: ts, verdict, rule, count, severity, blind) and prints one
# line per rule: runs it fired in, first and last count with dates, best and worst, and how
# many of those runs it was blind (a `_rule` self-check finding — it inspected nothing).
#
# This is the evidence the rollout plan (process/lint-backlog.md) asks for before a warning
# becomes an error: a rule that fired and whose count fell is doing its job; a rule that
# never fired is either clean here or wrong, and the blind column says which. Without this
# file the only trace of a rule's history was lint-baseline.json in git, which records the
# count that was ACCEPTED, not what each run saw.
#
# Usage:
# project-bin/lint-trend.sh # one line per rule, worst first
# project-bin/lint-trend.sh --rule UX001 # every run's count for one rule, oldest first
# project-bin/lint-trend.sh --runs # the run rows only (ts, verdict, total, blind)
#
# Exit 0 with output; 2 when there is no ledger yet (the gate has never run here). Read-only.
# Keep bash-3.2 compatible.

set -uo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
LEDGER="$ROOT/.claude/loop/lint-ledger.tsv"

if [ -f "$(dirname "${BASH_SOURCE[0]}")/_common.sh" ]; then
. "$(dirname "${BASH_SOURCE[0]}")/_common.sh"
fi
if ! type require_py >/dev/null 2>&1; then
require_py() {
_c=""
for _c in python3 python py; do # portability-ok: this IS the interpreter probe
case "$(command -v "$_c" 2>/dev/null)" in *[Ww]indows[Aa]pps*) continue ;; esac
if "$_c" -c 'import sys; sys.exit(0 if sys.version_info[0] == 3 else 1)' >/dev/null 2>&1; then
PY="$_c"; export PY; return 0
fi
done
echo "lint-trend: Python 3 is required and was not found (tried python3, python, py)." >&2 # portability-ok: names in a diagnostic
exit 2
}
fi
require_py

MODE="rules"; RULE=""
while [ $# -gt 0 ]; do
case "$1" in
--rule) MODE="one"; RULE="${2:-}"; [ -n "$RULE" ] || { echo "lint-trend: --rule needs an id" >&2; exit 2; }; shift ;;
--runs) MODE="runs" ;;
-h|--help) sed -n '2,22p' "$0"; exit 0 ;;
*) echo "lint-trend: unknown argument $1" >&2; exit 2 ;;
esac
shift
done

if [ ! -f "$LEDGER" ]; then
echo "lint-trend: no ledger at $LEDGER — the lint gate has not run here yet (project-bin/lint-gate.sh writes it)." >&2
exit 2
fi

LEDGER="$LEDGER" MODE="$MODE" RULE="$RULE" "$PY" - <<'PY'
import csv, os, sys
rows = []
with open(os.environ["LEDGER"]) as f:
for r in csv.DictReader(f, delimiter="\t"):
try:
r["count"] = int(r["count"]); r["blind"] = int(r.get("blind") or 0)
except ValueError:
continue
rows.append(r)
mode, want = os.environ["MODE"], os.environ["RULE"]

if mode == "runs":
print("%-20s %-18s %6s %s" % ("ts", "verdict", "total", "blind"))
for r in rows:
if r["rule"] == "_run":
print("%-20s %-18s %6d %s" % (r["ts"], r["verdict"], r["count"], "yes" if r["blind"] else ""))
sys.exit(0)

runs = [r for r in rows if r["rule"] == "_run"]
if mode == "one":
hits = [r for r in rows if r["rule"] == want]
if not hits:
print("%s: never fired in %d run(s) here." % (want, len(runs)))
sys.exit(0)
print("%-20s %6s %-8s %s" % ("ts", "count", "severity", "blind"))
for r in hits:
print("%-20s %6d %-8s %s" % (r["ts"], r["count"], r["severity"], "yes" if r["blind"] else ""))
sys.exit(0)

by = {}
for r in rows:
if r["rule"] == "_run":
continue
by.setdefault(r["rule"], []).append(r)
if not by:
print("No rule has fired in %d run(s) here." % len(runs))
sys.exit(0)
print("%d run(s), first %s, last %s" % (len(runs), runs[0]["ts"][:10] if runs else "?", runs[-1]["ts"][:10] if runs else "?"))
print("%-10s %-8s %5s %-18s %-18s %5s %5s %s" % ("rule", "severity", "runs", "first", "last", "best", "worst", "blind"))
for rule, hs in sorted(by.items(), key=lambda kv: -kv[1][-1]["count"]):
counts = [h["count"] for h in hs]
trend = "" if len(hs) < 2 else (" down" if counts[-1] < counts[0] else (" UP" if counts[-1] > counts[0] else " flat"))
blind = sum(h["blind"] for h in hs)
print("%-10s %-8s %5d %-18s %-18s %5d %5d %s%s" % (
rule, hs[-1]["severity"], len(hs),
"%d @ %s" % (counts[0], hs[0]["ts"][:10]), "%d @ %s" % (counts[-1], hs[-1]["ts"][:10]),
min(counts), max(counts), ("%d BLIND" % blind) if blind else "", trend))
PY
10 changes: 10 additions & 0 deletions skills/lint-that-actually-runs.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,16 @@ all the same order of magnitude, and no project produced a single real error-sev
the ratchet is the *only* mechanism available. `project-bin/lint-gate.sh` implements it
(`--update-baseline` to accept current counts).

**The ratchet remembers one number per rule; the ledger remembers every run.** Every gate run
appends one row per rule to `.claude/loop/lint-ledger.tsv` (timestamp, verdict, rule, count,
severity, blind), and `project-bin/lint-trend.sh` reads it back: runs a rule fired in, first and
last count with dates, best/worst, and how many runs it was blind. That is the answer to "did
the rule we shipped ever catch anything here, and did the count go down once people saw it" —
the evidence `process/lint-backlog.md` asks for before a warning becomes an error. A rule that
never appears in the ledger is clean here *or* wrong; `--rule <id>` and the blind column say
which. Before this file existed the only history was `lint-baseline.json` in git, which records
the count that was *accepted*, not what each run saw.

## Two false-pass bugs already designed out of `lint-gate.sh` — keep them out

Both were found by testing the gate, not by reading it:
Expand Down
Loading