AB#3757505 Improve vulnerability triage fixability gates and reporting, Fixes AB#3757505

[Cesar Acosta (cacosta33)]

Summary

This PR expands the vuln-triage-reporter skill and related Android Auth guidance so security triage distinguishes whether a finding is actionable, already fixed, covered upstream, or not fixable by a client-side change. It also strengthens the evidence, environment, reporting, and safety gates used by the workflow.

What changed

• Added explicit verdicts for Not-Fixable (By-Design) and Fixed-Since-Filed, including shipped-release exposure checks.
• Split bundled vulnerability reports into independently evaluated sub-claims.
• Added existing-work tracking for branches and commits carrying an unshipped fix.
• Hardened preflight checks for shipping refs, release branches, nested repositories, required tooling, and repository state.
• Added vetted challenger prompts, protocol and intake references, remediation guidance, report templates, and output verification.
• Added scripts for finding discovery, finding creation, transcription, linting, rollups, status reports, research pages, agent specs, and safety checks.
• Updated the related codebase-researcher, incident-investigator, kusto-analyst, and repository instructions to align with the new workflow.
• Addressed PR review feedback by tightening private-workspace guidance, password handling, slug validation, remote validation, manifest parsing, finding discovery, suggested-fix transcription, and generated-artifact ignores.

Work items

• AB#3757505

Validation

• python -m py_compile across .github/skills/vuln-triage-reporter/scripts/*.py
• python .github/skills/vuln-triage-reporter/scripts/safety_check.py .github/skills/vuln-triage-reporter --all-tracked
• Temp-workspace smoke test for new_finding.py force cleanup, slug rejection, and corrupt-manifest fail-closed behavior
• No secrets, access tokens, customer data, incident identifiers, or environment-specific credentials are included in this PR.

Review notes

This PR changes workflow guidance and tooling; it does not modify product authentication or runtime behavior.

[github-actions]

❌ Work item link check failed. Description does not contain AB#{ID}.

Click here to Learn more.

[github-actions]

✅ Work item link check complete. Description contains link AB#3654306 to an Azure Boards work item.

[Copilot]

🟡 Changes recommended

Unresolved critical and moderate findings remain in intake, safety, repository-gating, and reporting workflows.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR expands vulnerability-triage workflows with fixability verdicts, evidence gates, reporting automation, and public-repository safety checks. It does not change runtime authentication behavior.

Changes:

• Adds tooling for intake, discovery, transcription, linting, reporting, rollups, preflight, and output verification.
• Adds guidance and templates for research, remediation, protocols, and reporting.
• Updates related skills, repository instructions, and ignore rules.

File summaries

File	Reviewed change
.kotlin/errors/errors-1782956770091.log	Adds a generated Kotlin compiler diagnostic artifact.
.gitignore	Updates repository ignore rules.
.github/skills/vuln-triage-reporter/scripts/verify_outputs.py	Verifies generated shift outputs.
.github/skills/vuln-triage-reporter/scripts/transcribe_finding.py	Transcribes saved findings into reports.
.github/skills/vuln-triage-reporter/scripts/shift.py	Manages shift windows and manifests.
.github/skills/vuln-triage-reporter/scripts/scaffold_itd.py	Scaffolds ITD investigations.
.github/skills/vuln-triage-reporter/scripts/safety_check.py	Checks public-repository safety.
.github/skills/vuln-triage-reporter/scripts/rollup.py	Produces aggregate triage summaries.
.github/skills/vuln-triage-reporter/scripts/rebuild_shift.py	Rebuilds shift artifacts.
.github/skills/vuln-triage-reporter/scripts/preflight.py	Validates repository and environment state.
.github/skills/vuln-triage-reporter/scripts/new_finding.py	Creates finding reports.
.github/skills/vuln-triage-reporter/scripts/lint_finding.py	Validates report structure.
.github/skills/vuln-triage-reporter/scripts/discover_findings.py	Discovers and inventories findings.
.github/skills/vuln-triage-reporter/scripts/build_agent_spec.py	Generates agent specifications.
.github/skills/vuln-triage-reporter/references/status-report-template.md	Defines status-report formatting.
.github/skills/vuln-triage-reporter/references/research-discipline.md	Defines research and claim-ledger guidance.
.github/skills/vuln-triage-reporter/references/report-template.md	Defines the finding report schema.
.github/skills/vuln-triage-reporter/references/remediation-spec.md	Defines remediation requirements.
.github/skills/vuln-triage-reporter/references/remediation-execution.md	Documents remediation execution.
.github/skills/vuln-triage-reporter/references/protocol-constraints.md	Documents protocol constraints.
.github/skills/vuln-triage-reporter/references/msrc-bundle-intake.md	Documents MSRC bundle intake.
.github/skills/vuln-triage-reporter/references/itd-intake.md	Documents ITD intake.
.github/skills/vuln-triage-reporter/references/intake-interview.md	Defines intake questions.
.github/skills/vuln-triage-reporter/references/glossary.md	Defines report terminology.
.github/skills/vuln-triage-reporter/references/flight-verification.md	Defines flight-verification requirements.
.github/skills/vuln-triage-reporter/references/challenger-prompt.md	Defines challenger prompts.
.github/skills/vuln-triage-reporter/references/agent-spec-template.md	Defines agent-spec formatting.
.github/skills/vuln-triage-reporter/.gitignore	Ignores local Python artifacts.
.github/skills/update-skills/SKILL.md	Updates skill-maintenance guidance.
.github/skills/kusto-analyst/SKILL.md	Updates telemetry-analysis guidance.
.github/skills/incident-investigator/SKILL.md	Updates incident-investigation guidance.
.github/skills/codebase-researcher/SKILL.md	Updates codebase-research guidance.
.github/copilot-instructions.md	Updates repository-wide instructions.

Review details

Suppressed comments (22)

.github/skills/incident-investigator/SKILL.md:109

• The new zero-row guard correctly tells investigators to use correlation_id_v2, but the bundled reference queries this workflow points to still filter and project the legacy correlation_id. Following those snippets produces the exact false-negative zero-row result this guard warns about; update the linked queries and field table in the same change.

When a `android_spans` query for an incident's correlation IDs returns zero rows, do **not** conclude "the broker was never invoked" from that alone. Zero rows is ambiguous — it could mean the query is wrong (e.g., querying `correlation_id` instead of `correlation_id_v2`), or coverage is genuinely absent. Resolve the ambiguity with **two additional queries** before relying on the result:

1. **Reference healthy trace** — pull one known-good recent example of the same operation to confirm what the row *should* look like:
   ```kql
   android_spans
   | where EventInfo_Time >= ago(30m)
   | where calling_package_name == '<same-package>'
   | where span_name == '<same-span>'
   | where span_status == 'OK'
   | where isnotempty(correlation_id_v2)

.github/skills/vuln-triage-reporter/references/remediation-execution.md:90

• This guidance explicitly permits an ADO work-item link in a public commit/PR body, but the public-repository policy for this project forbids exposing AB# identifiers in PR descriptions or code comments. The URL itself carries that identifier and can couple public history to a sensitive finding; keep the real WI/IcM linkage in the private execution tracker and make the public-safe rule consistent throughout this section.

| **Branch name** | `cesaracosta/itd-635851-intent-allowlist-bypass` | `cesaracosta/webview-intent-validation` (neutral feature name) |
| **Commit title/body** | "Fix MSRC intent-scheme bypass / CWE-939", IcM/MSRC numbers, "vulnerability/exploit" | Generic one-liner ("Add flighted validation in WebView intent handling") + **only** a corp-gated ADO work-item link |
| **Code comments** | "the intent:// scheme lets the page embed a component that redirects to an *attacker-chosen* activity" (teaches the attack) | State what the code *enforces*: "activity resolution is driven solely by the validated package" |
| **Test fixtures / names** | `TEST_INTENT_SPOOFED_PACKAGE`, `com.attacker.evil/.EvilActivity`, "decoy substrings satisfy the gate" | `TEST_INTENT_WITH_NON_ALLOWLISTED_PACKAGE`, `com.example.unrelatedapp/.SampleActivity`, neutral comments |

.github/skills/vuln-triage-reporter/references/remediation-execution.md:169

• The broker destination here is stale: the preflight gate and current routing guidance use msft.ghe.com/security/ad-accounts-for-android, not identity-authnz-teams/ad-accounts-for-android. Following this table can open remediation against a retired or incorrect organization.

| **broker** / broker4j | **GitHub Enterprise** (`identity-authnz-teams/ad-accounts-for-android`, GHE) | **EMU** (Enterprise Managed User) | Use the **EMU** identity for this org. |

.github/skills/vuln-triage-reporter/references/status-report-template.md:10

• This newly added status-report guidance promises that --auto-token/--token will fall back to live ADO state, but the referenced build_status_report.py currently sends ****** rather than the supplied bearer token in its Authorization header. Following this workflow therefore produces unauthenticated ADO reads and stale/default statuses until the script uses the token.

> Generate it with [`scripts/build_status_report.py`](../scripts/build_status_report.py), which reads
> `classifications.csv`, auto-discovers a persisted `work-item-map.json` (IcM → AB#) beside it, reads the
> **execution tracker** (`EXECUTION-TRACKER.md`) for real remediation status, and (with `--auto-token`)
> falls back to live ADO work-item state. Output is a self-contained HTML table that pastes cleanly into

.github/skills/vuln-triage-reporter/scripts/build_agent_spec.py:213

• clean() deliberately removes parenthesized text, but the disposition values use the parentheses to distinguish Already-Covered, Fixed-Since-Filed, and Not-Fixable. The generated agent spec therefore collapses all of those to Won't-Fix/Not-Fixable, losing the Gate 0 outcome promised by its schema.

        assignment = clean(field(md, "Disposition")) or clean(field(md, "Assignment"))

.github/skills/vuln-triage-reporter/scripts/build_status_report.py:305

• The token passed into fetch_ado is never used: every request sends the literal ****** as its Authorization header. Consequently --token and --auto-token always receive an unauthorized response and the report silently falls back to Not started/missing Updated dates instead of live ADO state.

    req.add_header("Authorization", f"Bearer {token}")

.github/skills/vuln-triage-reporter/scripts/lint_finding.py:122

• The claim-ledger gate accepts any row with three cells even though its own error text and references/research-discipline.md require claim, channel, evidence, Pass 2 result, and status. A row containing only ID/claim/channel therefore passes without evidence or a verdict state, undermining the new adversarial-evidence gate.

            if len(r) < 3:
                issues.append((REQ, f"claim {cid}: row is missing columns "
                                    f"(need ID | claim | channel | evidence | pass2 | status)"))
                continue

.github/skills/vuln-triage-reporter/scripts/preflight.py:193

• The fetch and remote-ref commands' return codes are ignored. If the fetch fails or origin/<branch> does not exist, behind_n becomes -1 and age_days remains unset, which reaches the final PASS branch and declares a stale/unverified checkout current. A freshness gate must fail when it cannot refresh or compare the ref.

        run(["git", "-C", str(mod), "fetch", "origin", "--prune"])
        rc, branch, _ = run(["git", "-C", str(mod), "rev-parse", "--abbrev-ref", "HEAD"])
        if rc != 0:
            continue
        rc, behind, _ = run(
            ["git", "-C", str(mod), "rev-list", "--count", f"HEAD..origin/{branch}"]
        )
        behind_n = int(behind) if behind.isdigit() else -1
        rc, iso, _ = run(
            ["git", "-C", str(mod), "log", "-1", "--format=%cI", f"origin/{branch}"]
        )

.github/skills/vuln-triage-reporter/scripts/preflight.py:313

• This check has the same false-pass path for ESTS: a failed fetch, missing branch, or failed rev-list produces behind_n == -1, which falls through to ESTS present and current. That can let the workflow answer server-side questions from an unrefreshed checkout.

    run(["git", "-C", str(ests), "fetch", "origin", "--prune"])
    rc, branch, _ = run(["git", "-C", str(ests), "rev-parse", "--abbrev-ref", "HEAD"])
    rc, behind, _ = run(
        ["git", "-C", str(ests), "rev-list", "--count", f"HEAD..origin/{branch}"]
    )
    behind_n = int(behind) if behind.isdigit() else -1

.github/skills/vuln-triage-reporter/scripts/preflight.py:352

• The app freshness check also ignores the fetch/comparison status and treats behind_n == -1 as current. A network failure or absent origin/<branch> can therefore pass the very check intended to protect the shipped-release exposure decision.

    run(["git", "-C", str(app), "fetch", "origin", "--prune"])
    rc, branch, _ = run(["git", "-C", str(app), "rev-parse", "--abbrev-ref", "HEAD"])
    if rc != 0 or not branch:
        return
    rc, behind, _ = run(
        ["git", "-C", str(app), "rev-list", "--count", f"HEAD..origin/{branch}"]
    )
    behind_n = int(behind) if behind.isdigit() else -1

.github/skills/vuln-triage-reporter/scripts/preflight.py:403

• --skip-fetch is documented as skipping network freshness checks, but check_ests() is called unconditionally and performs its own git fetch. Offline callers still hit the network and can fail or hang despite opting out; pass the flag through the ESTS check so only local validation remains when requested.

    if not args.skip_fetch:
        check_freshness(root, rep)
        check_app_freshness(root, rep)
    check_ests(ests, rep)

.github/skills/vuln-triage-reporter/scripts/preflight.py:123

• any(p.iterdir()) treats a directory containing only a submodule .git marker as populated, so the source tree can still be absent while this hard gate reports PASS. Verify actual working-tree content (for example with git ls-files or an expected source path) instead of mere directory entries.

        populated = p.is_dir() and any(p.iterdir()) if p.is_dir() else False
        if populated:
            rep.add("PASS", f"submodule populated: {rel}")

.github/skills/vuln-triage-reporter/scripts/preflight.py:261

• Missing or unresolvable broker/common silently returns even though this path is declared as a required nested checkout and broker/settings.gradle builds against it. The run can proceed without evidence for the code actually built; emit a FAIL when the nested path, parent gitlink, or checked-out HEAD cannot be resolved.

        if not (pdir / ".git").exists() or not ndir.exists():
            continue
        rc, out, _ = run(["git", "-C", str(pdir), "ls-tree", "HEAD", nested])
        if rc != 0 or not out:
            continue

.github/skills/vuln-triage-reporter/scripts/preflight.py:237

• A remote whose newest commit is older than the frozen-mirror threshold is recorded as WARN, but Report.failed ignores warnings and main() exits 0 when there are no FAIL items. The preflight can therefore authorize investigation against exactly the frozen remote snapshot this gate is meant to reject; this condition should be blocking or require explicit override.

        elif age_days is not None and age_days > STALE_DAYS * 10:
            # Up to date with a remote whose newest commit is ancient => probably a frozen mirror.
            rep.add(
                "WARN",
                f"{module}: up to date, but the remote itself looks FROZEN",

.github/skills/vuln-triage-reporter/scripts/rebuild_shift.py:89

• The research pages are generated before the agent specs, but build_research_pages.py only adds the dispatch link when the matching .agent.md already exists. On a fresh run the HTML therefore permanently omits the agent-spec link, despite the documented generation order and the --agent-dir argument. Generate the specs first, then build the research pages.

    rc |= run("build_research_pages.py", glob_arg, "--out", research_dir,
              "--index", "--agent-dir", "../agent-specs")
    rc |= run("build_agent_spec.py", glob_arg, "--out", specs_dir)

.github/skills/vuln-triage-reporter/scripts/rollup.py:175

• Unlike the earlier total, this sum does not catch malformed eng_days values. A non-numeric value in any kept row raises ValueError and aborts the roll-up, so one bad estimate prevents all reporting.
  .github/skills/vuln-triage-reporter/scripts/scaffold_itd.py:28
• The default root omits the shift slug (<workspace>/msrc/<window>/). A documented invocation without --root therefore writes ITD READMEs outside the shift folder, while rebuild_shift.py and the report gates only process the shift's artifact tree. Make the default shift-aware or require the caller to provide the shift-specific root.
  .github/skills/vuln-triage-reporter/scripts/shift.py:51
• Supplying only --start or only --end silently ignores the supplied value and computes the current shift instead. That can put findings in the wrong window without an error; reject partial explicit windows (and validate their shape) rather than falling back to today.
  .github/skills/vuln-triage-reporter/scripts/transcribe_finding.py:126
• EXCLUDE_SECTIONS is never consulted and the extracted rows carry no heading ancestry, so parse_code_locations scans every table, including source-to-sink or exploit sections the module promises to omit. A table-backed PoC or trace can therefore be copied into the output; section context must be preserved and filtered before emitting rows.
  .github/skills/vuln-triage-reporter/scripts/transcribe_finding.py:156
• The wrapper parser only receives TextExtractor.parts, which records headings and table rows but not the documented fd-kv elements. For normal saved portal wrappers, wrapper-only Source, Severity, and Finding ID values are therefore silently lost.
  .github/skills/vuln-triage-reporter/scripts/verify_outputs.py:176
• If build_research_pages.py cannot be imported, this closing gate only emits a warning and returns success for the parser check. That allows a run to pass the required-artifact gate without validating the metadata that drives the HTML tiles, which defeats the purpose of this check; an unavailable parser should be a required failure.
  .github/skills/vuln-triage-reporter/scripts/verify_outputs.py:161
• The structure gate never validates the MSRC/ITD source tag. A report with a missing or malformed [MSRC]/[ITD] title can satisfy every REQUIRED_META field, while build_master_report.extract() defaults that title to ITD; the verifier can therefore publish the MSRC→ITD mislabel this gate claims to prevent.

• Files reviewed: 35/38 changed files
• Comments generated: 14
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

❌ Invalid work item number: AB#3757505

##. Work item number must be a valid integer.

Click here to learn more.