Skip to content

fix(engine): pass Copilot prompt using attached --prompt= form - #2187

Merged
jamesadevine merged 3 commits into
mainfrom
copilot/fix-engine-pass-copilot-prompt
Sep 22, 2026
Merged

jamesadevine merged 3 commits into
mainfrom
copilot/fix-engine-pass-copilot-prompt

Conversation

Copilot AI commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Copilot CLI 1.0.84-5+ treats a separate --prompt value beginning with --- as another option, breaking generated pipelines whose runtime-import prompt starts with Markdown frontmatter.

  • Generate Copilot invocations with the attached flag form for Agent and Detection:
    /tmp/awf-tools/copilot --prompt="$(cat /tmp/awf-tools/agent-prompt.md)"
  • Add compiler regression coverage for Agent and Detection prompt wiring.
  • Update the AWF SafeOutputs runner fixture to match generated command shape.

Test plan

  • cargo test --test compiler_tests test_compiled_agent_job_copilot_invocation_uses_safeoutputs_mcp_config
  • cargo test --test compiler_tests test_1es_compiled_output_is_valid_yaml
  • bash -n tests/awf-copilot-safeoutputs/run.sh
  • Manual parse check with pinned Copilot CLI and a prompt beginning with ---

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix engine to pass Copilot prompt using attached --prompt= form fix(engine): pass Copilot prompt using attached --prompt= form Sep 17, 2026
Copilot AI requested a review from jamesadevine September 17, 2026 19:48
@jamesadevine
jamesadevine marked this pull request as ready for review September 17, 2026 19:51
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

✅ Rust Code Quality Reviewer completed the Rust code quality review.

🦀 Rust code quality review by Rust Code Quality Reviewer

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

✅ Compiler Contract Reviewer completed the compiler contract review.

Compiler-contract review of PR #2187 (copilot --prompt attached-flag fix) complete. This is a minimal, well-contained fix: engine.rs::copilot_invocation, the AWF fixture (tests/awf-copilot-safeoutputs/run.sh), and tests/compiler_tests.rs were updated in lockstep, with new regression assertions covering both Agent and Detection jobs. No front-matter grammar, safe-output schema, typed IR, codegen (types.gen.ts/fact-catalog.gen.json), or documentation contracts are affected by this change, and no drift was found (compiled .lock.yml workflows, docs/engine.md prose, and BLOCKED_ARG_PREFIXES are all consistent with the new --prompt= form). Verified locally: the added test passes and ENFORCE_BASH_LINT=1 shell-script lint suite passes. No inline findings to post.

🏗️ Compiler contract review by Compiler Contract Reviewer

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

✅ PR Security Reviewer completed the security review.

🔒 Security review by PR Security Reviewer

@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

✅ Test Quality Sentinel completed the test quality analysis.

🧪 Test quality analysis by Test Quality Sentinel

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security review: this diff only changes the Copilot CLI prompt flag from the separated form (--prompt "X") to the attached form (--prompt="X") to accommodate Copilot CLI 1.0.84-5+ argument parsing. The shell quoting/escaping of the interpolated prompt_path is unchanged, no new injection surface is introduced, no tokens or credentials are affected, and no validation logic is touched. Security-neutral change — no findings.

🔒 Security review by PR Security Reviewer · auto · 15.7 AIC · ⌖ 1.7 AIC · ⊞ 10.2K
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Rust review: no issues found

The fix cleanly switches the Copilot CLI invocation to the attached --prompt="..." form to avoid the ----prefixed value being misparsed as another flag, updates the doc comment to match, and adds regression assertions for both the Agent and Detection jobs plus the 1ES compiled-output check. Verified cargo test --test compiler_tests passes for the touched tests.

Notes
  • The rust-critic sub-agent could not run (backing model was inaccessible: 400 model "gpt-5.4-mini" is not accessible via the /chat/completions endpoint), so this review reflects my own pass only.
  • No error-handling, correctness, or maintainability concerns on the changed lines in src/engine.rs or tests/compiler_tests.rs.

🦀 Rust code quality review by Rust Code Quality Reviewer · auto · 28 AIC · ⌖ 1.74 AIC · ⊞ 10.1K
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small, well-targeted fix. The new assertions correctly pin the --prompt= shape and guard against the old broken form, but as noted inline, coverage stops at string-matching the exact literal — the underlying regression (a ----prefixed prompt value being mis-parsed as an option) is never actually exercised in the test suite; only manually per the PR description. Not blocking, since the fix itself is correct and low-risk, but worth strengthening before this class of bug can recur silently.

🧪 Test quality analysis by Test Quality Sentinel · auto · 49.8 AIC · ⌖ 1.67 AIC · ⊞ 9.8K
Comment /review to run again

Comment thread tests/compiler_tests.rs
"agent job should pass compiler-emitted MCP config to Copilot CLI: {agent}"
);
assert!(
!agent.contains("--prompt \"$(cat "),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No test actually reproduces the reported regression

This assertion (and the twin at line 1911) only pins the literal generated string; nothing in this PR exercises the actual failure mode from #2186 — a prompt value that begins with --- being mis-parsed as another CLI option. The PR description even says the real verification was "Manual parse check with pinned Copilot CLI and a prompt beginning with ---", which never lands in CI.

A future refactor of copilot_invocation (e.g. reintroducing a space, or changing the quoting/escaping) could still reintroduce the bug while this string-equality assertion stays green if the refactor happens to produce a different-but-still-broken shape.

💡 Suggested strengthening

Add a case where the compiled prompt content itself starts with --- (e.g. a markdown body with YAML-frontmatter-like content) and assert the emitted invocation still has exactly one --prompt= token — ideally by shell-tokenizing the ENGINE_RUN-equivalent string (e.g. via shlex/shell-words in a small helper) and asserting the prompt argument round-trips as a single token, rather than only substring-matching the exact literal.

@jamesadevine

Copy link
Copy Markdown
Collaborator

@copilot add the suggested test coverage

Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>

Copilot AI commented Sep 17, 2026

Copy link
Copy Markdown
Contributor Author

@copilot add the suggested test coverage

Added the requested regression coverage in b3a05cf: it uses a runtime-import fixture whose prompt begins with ---, asserts Agent and Detection use --prompt=, and exercises the attached form against the pinned Copilot CLI when available.

@jamesadevine
jamesadevine merged commit ee4a242 into main Sep 22, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(engine): pass Copilot prompt using attached --prompt= form

2 participants