Repository navigation
fix(engine): pass Copilot prompt using attached --prompt= form - #2187
Conversation
|
Azure Pipelines: 2 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
|
Azure Pipelines: 2 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
|
✅ Rust Code Quality Reviewer completed the Rust code quality review.
|
|
✅ Compiler Contract Reviewer completed the compiler contract review. Compiler-contract review of PR #2187 (copilot --prompt attached-flag fix) complete. This is a minimal, well-contained fix: engine.rs::copilot_invocation, the AWF fixture (tests/awf-copilot-safeoutputs/run.sh), and tests/compiler_tests.rs were updated in lockstep, with new regression assertions covering both Agent and Detection jobs. No front-matter grammar, safe-output schema, typed IR, codegen (types.gen.ts/fact-catalog.gen.json), or documentation contracts are affected by this change, and no drift was found (compiled .lock.yml workflows, docs/engine.md prose, and BLOCKED_ARG_PREFIXES are all consistent with the new --prompt= form). Verified locally: the added test passes and ENFORCE_BASH_LINT=1 shell-script lint suite passes. No inline findings to post.
|
|
✅ PR Security Reviewer completed the security review.
|
|
✅ Test Quality Sentinel completed the test quality analysis.
|
There was a problem hiding this comment.
Security review: this diff only changes the Copilot CLI prompt flag from the separated form (--prompt "X") to the attached form (--prompt="X") to accommodate Copilot CLI 1.0.84-5+ argument parsing. The shell quoting/escaping of the interpolated prompt_path is unchanged, no new injection surface is introduced, no tokens or credentials are affected, and no validation logic is touched. Security-neutral change — no findings.
🔒 Security review by PR Security Reviewer · auto · 15.7 AIC · ⌖ 1.7 AIC · ⊞ 10.2K
Comment /review to run again
There was a problem hiding this comment.
✅ Rust review: no issues found
The fix cleanly switches the Copilot CLI invocation to the attached --prompt="..." form to avoid the ----prefixed value being misparsed as another flag, updates the doc comment to match, and adds regression assertions for both the Agent and Detection jobs plus the 1ES compiled-output check. Verified cargo test --test compiler_tests passes for the touched tests.
Notes
- The
rust-criticsub-agent could not run (backing model was inaccessible:400 model "gpt-5.4-mini" is not accessible via the /chat/completions endpoint), so this review reflects my own pass only. - No error-handling, correctness, or maintainability concerns on the changed lines in
src/engine.rsortests/compiler_tests.rs.
🦀 Rust code quality review by Rust Code Quality Reviewer · auto · 28 AIC · ⌖ 1.74 AIC · ⊞ 10.1K
Comment /review to run again
There was a problem hiding this comment.
Small, well-targeted fix. The new assertions correctly pin the --prompt= shape and guard against the old broken form, but as noted inline, coverage stops at string-matching the exact literal — the underlying regression (a ----prefixed prompt value being mis-parsed as an option) is never actually exercised in the test suite; only manually per the PR description. Not blocking, since the fix itself is correct and low-risk, but worth strengthening before this class of bug can recur silently.
🧪 Test quality analysis by Test Quality Sentinel · auto · 49.8 AIC · ⌖ 1.67 AIC · ⊞ 9.8K
Comment /review to run again
| "agent job should pass compiler-emitted MCP config to Copilot CLI: {agent}" | ||
| ); | ||
| assert!( | ||
| !agent.contains("--prompt \"$(cat "), |
There was a problem hiding this comment.
No test actually reproduces the reported regression
This assertion (and the twin at line 1911) only pins the literal generated string; nothing in this PR exercises the actual failure mode from #2186 — a prompt value that begins with --- being mis-parsed as another CLI option. The PR description even says the real verification was "Manual parse check with pinned Copilot CLI and a prompt beginning with ---", which never lands in CI.
A future refactor of copilot_invocation (e.g. reintroducing a space, or changing the quoting/escaping) could still reintroduce the bug while this string-equality assertion stays green if the refactor happens to produce a different-but-still-broken shape.
💡 Suggested strengthening
Add a case where the compiled prompt content itself starts with --- (e.g. a markdown body with YAML-frontmatter-like content) and assert the emitted invocation still has exactly one --prompt= token — ideally by shell-tokenizing the ENGINE_RUN-equivalent string (e.g. via shlex/shell-words in a small helper) and asserting the prompt argument round-trips as a single token, rather than only substring-matching the exact literal.
|
@copilot add the suggested test coverage |
Co-authored-by: jamesadevine <4742697+jamesadevine@users.noreply.github.com>
Added the requested regression coverage in |
Summary
Copilot CLI
1.0.84-5+treats a separate--promptvalue beginning with---as another option, breaking generated pipelines whose runtime-import prompt starts with Markdown frontmatter./tmp/awf-tools/copilot --prompt="$(cat /tmp/awf-tools/agent-prompt.md)"Test plan
cargo test --test compiler_tests test_compiled_agent_job_copilot_invocation_uses_safeoutputs_mcp_configcargo test --test compiler_tests test_1es_compiled_output_is_valid_yamlbash -n tests/awf-copilot-safeoutputs/run.sh---