[ci-scan-feedback] ci-scan: add Hard Rule 10 for explicit early exit on no scannable build #7648

Triggering signals

• Tally misreporting on skip-only runs (rubric: occurrence-honest): ci-scan emits | 0 | 0 | 0 | 0 | instead of | 0 | 0 | 0 | 1 | when Step 1 yields a no-scannable-build conclusion, because the Step 7 tally is never reached or the skipped count is not reflected. The skip reason appears in the noop prose but not in the structured pipe table that ci-scan-feedback extracts. (run #86, run #87)
• Unnecessary token spend on skip-only runs: 87 consecutive ci-scan runs (all since 2026-06-08T13:53:53Z) have hit stale build window (>14d). Runs that should stop immediately at Step 1 instead proceed to Steps 2–7, consuming 2.2–2.4M+ effective tokens on AzDO timeline fetches, task log downloads, and Helix queries that yield no useful output. Correct early-exit runs consume ~250K ET.
• 12 consecutive fallback issues ([ci-scan-feedback] ci-scan: add Hard Rule 10 for early exit on no scannable build #7627, [ci-scan-feedback] ci-scan: add Hard Rule 10 for early exit on no scannable build #7630, [ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build #7636, [ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build #7637, [ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build #7639, [ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build #7640, [ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build #7641, [ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build #7642, [ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build #7643, [ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build #7644, [ci-scan-feedback] ci-scan: add Hard Rule 10 to force early exit on no scannable build #7645, [ci-scan-feedback] ci-scan: add Hard Rule 10 for explicit early exit on no scannable build #7646): each prior feedback tick proposed this exact edit via create_pull_request; repeated git push failures caused the patch to be filed as a GitHub Issue instead. The ambiguous and stop clause in Step 1 is the root cause — it lacks the specificity of a Hard Rule and agents continue past it.

Proposed edits

• .github/workflows/ci-scan.agent.md lines 80–92 (Hard Rules + Step 1 trailing sentence):
  1. Add Hard Rule 10 (after Rule 9): names the exact forbidden operations (AzDO timeline fetch, task log download, Helix query), mandates appending the coverage entry, printing the Step 7 tally as | 0 | 0 | 0 | 1 |, and calling noop — no ambiguity about what "stop" means.
  2. Update Step 1 trailing sentence: replace the inline restatement of skip reasons with a single reference to "apply Hard Rule 10 immediately" so the constraint is stated once, authoritatively.

Expected behavior change

On any run where Step 1 yields a selection-time skip (stale build window (>14d), no follow-up build yet, defer to next run, or no failed build in 7d), the scanner will write the coverage entry, print | total-signatures | issues-filed | reused-existing | skipped-with-reason | followed by | 0 | 0 | 0 | 1 |, call noop, and stop immediately — without fetching any AzDO timeline, downloading any task log, or querying any Helix work item. The tally will correctly reflect 1 skipped-with-reason on every skip-only run.

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

Generated by CI Failure Scanner - Feedback (machinelearning) · ● 4.4M · ◷

To create a pull request with the changes:

# Download the artifact from the workflow run
gh run download 28419478972 -n agent -D /tmp/agent-28419478972

# Create a new branch
git checkout -b ci-scan-feedback/hard-rule-10-2026-06-30-e31ecb6e02ca2136

# Apply the patch (--3way handles cross-repo patches where files may already exist)
git am --3way /tmp/agent-28419478972/aw-ci-scan-feedback-hard-rule-10-2026-06-30.patch

# Push the branch to origin
git push origin ci-scan-feedback/hard-rule-10-2026-06-30-e31ecb6e02ca2136

# Create the pull request
gh pr create --title '[ci-scan-feedback] ci-scan: add Hard Rule 10 for explicit early exit on no scannable build' --base main --head ci-scan-feedback/hard-rule-10-2026-06-30-e31ecb6e02ca2136 --repo dotnet/machinelearning

From a85b09d4da22c31803ca79cdb6e1bc6906fba72e Mon Sep 17 00:00:00 2001
From: "github-actions[bot]" <github-actions[bot]@users.noreply.github.com>
Date: Tue, 30 Jun 2026 04:13:36 +0000
Subject: [PATCH] ci-scan: add Hard Rule 10 for explicit early exit on no
 scannable build

83+ consecutive ci-scan runs have skipped via Step 1 prose ('stale build
window (>14d)') after consuming 2.2-2.4M+ effective tokens on AzDO
timeline fetches, log downloads, and Helix queries that serve no purpose
once no scannable build exists. The 'and stop' sentence in Step 1 was
insufficient to prevent continuation; agents kept fetching the AzDO
timeline and walking Steps 2-7.

Add Hard Rule 10 that elevates the no-scannable-build exit to the same
level as the issue-cap and label rules. When Step 1 determines no
scannable build exists, the scanner must:
- append the coverage entry
- print | 0 | 0 | 0 | 1 | as the Step 7 tally (honest: 1 skipped)
- call noop and stop immediately

Update Step 1's trailing sentence to reference Hard Rule 10 directly
so the constraint is stated once, authoritatively.

Triggering signal: 12 prior ci-scan-feedback runs proposed this exact
edit; git push kept failing. Issues #7627, #7630, #7636, #7637, #7639,
#7640, #7641, #7642, #7643, #7644, #7645, #7646 are misfiled fallbacks.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---
 .github/workflows/ci-scan.agent.md | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.github/workflows/ci-scan.agent.md b/.github/workflows/ci-scan.agent.md
index 0937c5f..258fd61 100644
--- a/.github/workflows/ci-scan.agent.md
+++ b/.github/workflows/ci-scan.agent.md
@@ -78,6 +78,7 @@ These invariants are not delegated to the shared file. Honor them even if a shar
 7. **All state under `/tmp/gh-aw/agent/`;** each bash call is a fresh subshell.
 8. **AzDO REST is anonymous;** stay on `https://dev.azure.com/dnceng-public/public/_apis/build/...`. Follow every rule in [Environment constraints](shared/
... (truncated)