feat(autoland): add an x step that runs a command and stops the land if it fails

CHANGELOG.md

@@ -1,5 +1,14 @@
 # Top of tree
 
+* `autoland` plans have a new step, `x <command>`, which runs a command once
+  the PRs before it have landed and stops the land unless it exits 0 — for
+  automating what would otherwise be a manual `c` step, such as provisioning.
+  The command runs without a shell (use `x bash -c '...'` for pipes), from
+  the top of the checkout, with the landed SHA, the landed PRs, the branch,
+  the remote, and the target in `AUTOLAND_*` environment variables.
+  `--resume` runs a failed or interrupted command again, so commands should
+  be idempotent. The VS Code extension (0.3.0) highlights and formats `x`
+  steps.
 * `autoland` now merges each run of consecutive `l` steps (with no `w` or `c`
   step between them) as a GitHub stack, using GitHub's native [stacked
   PRs](https://docs.github.com/en/pull-requests/get-started/about-stacked-prs):

README.md

@@ -575,10 +575,12 @@ Options:
   autoland reaches that step (`Confirm "QA sign-off complete" is complete —
   ready to proceed?`). A bare `c` just prompts `Ready to proceed?`. When
   `autoland.default_workflow` is configured, the pre-filled plan already ends
-  with a `w <default_workflow>` step, which you can edit or delete.
+  with a `w <default_workflow>` step, which you can edit or delete. Command
+  steps (`x <command>` — run a command, and stop unless it succeeds) can go
+  there too; see [Running commands](#running-commands).
 - `--plan-file PATH`: Load the landing plan from a file instead of editing it
   interactively. The file uses the exact same format as the `-i` editor (`l` /
-  `w <workflow>` / `c [condition]` lines, with `#` comments and blank lines
+  `w <workflow>` / `c [condition]` / `x <command>` lines, with `#` comments and blank lines
   ignored — see [Example plan](#example-plan) below), so you can save a plan
   from `-i`, or write one by hand, and reuse it for repeatable/scripted runs.
   Plans conventionally use the `.autoland-plan` suffix.
@@ -605,10 +607,10 @@ Options:
   `state_file`, `state_file_exists`, `lock_file`, `pid`, `last_saved` (ISO
   8601), `abort_reason`, `resume_command`, `plan` (`total` / `done` /
   `remaining` counts and a `steps` list), and `other_runs`. Each step has
-  `number`, `type` (`land` / `workflow` / `confirm`), `outcome` (`done` /
-  `active` / `pending` / `failed`), `status`, `is_next` and `detail`, plus
-  `pr_number` / `pr_url` / `title`, `workflow`, or `condition` depending on
-  its type. If the state file can't be read, stdout stays empty and the error
+  `number`, `type` (`land` / `workflow` / `confirm` / `exec`), `outcome`
+  (`done` / `active` / `pending` / `failed`), `status`, `is_next` and
+  `detail`, plus `pr_number` / `pr_url` / `title`, `workflow`, `condition`, or
+  `command` depending on its type. If the state file can't be read, stdout stays empty and the error
   goes to stderr with a nonzero exit.
 - `--state-file PATH`: Override the checkpoint path (default:
   `~/.stack-pr/autoland/<branch>.json`).
@@ -655,7 +657,7 @@ default_workflow = deploy.yaml
 ##### Merging as a stack
 
 By default, `autoland` lands each run of two or more consecutive `l` steps (no
-`w` or `c` step between them) with GitHub's native [stacked
+`w`, `c`, or `x` step between them) with GitHub's native [stacked
 PRs](https://docs.github.com/en/pull-requests/get-started/about-stacked-prs)
 instead of one PR at a time:
 
@@ -673,8 +675,8 @@ instead of one PR at a time:
    every PR.
 
 So a plan of four bare `l` steps waits on the merge queue once instead of four
-times. A `w` or `c` step ends a run, since it has to see the PRs below it land
-first; `--dry-run` lists the runs a plan would merge as stacks.
+times. A `w`, `c`, or `x` step ends a run, since it has to see the PRs below it
+land first; `--dry-run` lists the runs a plan would merge as stacks.
 
 To keep PRs from merging together, put a `c` step between them in the plan,
 which splits the run there (`autoland` asks you to confirm before continuing).
@@ -707,8 +709,9 @@ installed (`pipx install 'stack-pr[rich]'` or add the `rich` extra); otherwise
 
 With `-i`, `autoland` opens a plan in `$EDITOR`. Each non-comment line is one
 step, run top to bottom: `l <pr>` lands that PR, `w <workflow>` waits for a
-named GitHub Actions workflow to finish with the landed code, and
-`c [condition]` pauses for manual confirmation (the condition is optional). For
+named GitHub Actions workflow to finish with the landed code,
+`c [condition]` pauses for manual confirmation (the condition is optional), and
+`x <command>` runs a command (see [Running commands](#running-commands)). For
 a three-PR stack, a plan that lands the bottom PR, waits for a deploy, gets
 manual sign-off, then lands the rest looks like:
 
@@ -720,6 +723,8 @@ manual sign-off, then lands the rest looks like:
 # c [condition] = pause for manual confirmation; the optional
 #                 condition names what to verify before proceeding
 #                 (e.g. 'c QA sign-off complete')
+# x <command>   = run a command, and stop unless it exits 0; it runs
+#                 without a shell (use 'x bash -c ...' for pipes)
 #
 # Lines starting with # are comments and are ignored.
 # Blank lines are ignored.
@@ -750,12 +755,53 @@ repeatable or scripted landings. Any path works, but `.autoland-plan` is the
 conventional suffix: it is what `-i` writes, and what editors key off to
 highlight the plan syntax (see [Editor support](#editor-support)).
 
+##### Running commands
+
+An `x <command>` step runs a command once the PRs before it have landed, and
+stops the land unless it exits 0 — so a manual step that would otherwise be a
+`c` ("provision the new tables, then confirm") can be automated:
+
+```
+l 101                         # Add the widgets table
+x ./scripts/provision.sh --env staging
+l 102                         # Start writing widgets
+x bash -c 'curl -fsS "$HEALTH_URL" | grep -q ok'
+l 103
+```
+
+- The command is split into words the way a POSIX shell would (quotes and
+  backslashes work), but it runs **without a shell**: no pipes, redirection,
+  globs, or `$VARIABLE` expansion. For those, run a shell yourself, as in
+  `x bash -c '...'`.
+- As with every step, ` #` (a space, then `#`) starts a comment, even inside
+  quotes, so a command can't contain one.
+- It runs from the top of the checkout being landed (the temporary worktree,
+  with `--branch`), with its output going straight to the terminal, and with
+  `autoland`'s own environment plus:
+  - `AUTOLAND_LANDED_SHA`: the commit of the code landed so far — the merge
+    commit of the last PR this run landed, or else the head of the target
+    branch (empty if it can't be found);
+  - `AUTOLAND_LANDED_PRS`: the PRs the plan has landed before this step,
+    space-separated in plan order, including any that landed in an earlier
+    run;
+  - `AUTOLAND_BRANCH`, `AUTOLAND_REMOTE`, `AUTOLAND_TARGET`: the branch whose
+    stack is landing, and the remote and branch it lands into.
+- A nonzero exit status, a command killed by a signal, or a command that can't
+  be started (no such program, not executable) stops the land; `--status`
+  shows why. There is no timeout.
+- `--resume` skips a command that succeeded and **runs a failed or interrupted
+  one again**, so commands should be safe to re-run (idempotent). Ctrl+C
+  while one runs stops it along with `autoland`. As with `w` and `c` steps, an
+  `x` step before an already-landed PR is assumed done, and `--replan` keeps a
+  finished one when the new plan has the same command after the same landed
+  PRs.
+
 ##### Re-using a plan across a partially-landed stack
 
 Because each `l` step names the PR it lands, one plan file stays valid as the
 stack lands piece by piece. A PR that has already merged is no longer part of
-the stack, so `autoland` skips its `l` step — along with any `w` and `c` steps
-that come before it, since those must have run before that PR could merge:
+the stack, so `autoland` skips its `l` step — along with any `w`, `c`, and `x`
+steps that come before it, since those must have run before that PR could merge:
 
 ```
 l 1                           # PR #1, already landed upstream
@@ -796,9 +842,9 @@ need when only the code changed. Either way it:
    the run continues in your terminal.
 2. Rediscovers the stack, since the code may have changed, and parses the new
    plan against it. Landed PRs are skipped as usual.
-3. Carries over each `w` and `c` step the previous run completed, provided the
-   new plan has the same step (same workflow, or same condition text) after the
-   same set of landed PRs. A step means "once these PRs have landed, this
+3. Carries over each `w`, `c`, and `x` step the previous run completed,
+   provided the new plan has the same step (same workflow, condition text, or
+   command) after the same set of landed PRs. A step means "once these PRs have landed, this
    holds", so its result still stands. Adding, removing, or reordering steps
    keeps that credit; editing a step's text, or landing a different set of PRs
    before it, makes it run again.
@@ -812,8 +858,8 @@ autoland, so you can check what a replan would do before committing to it.
 ##### Editor support
 
 [`editors/vscode`](editors/vscode) is a VS Code extension that highlights plan
-files: the `l` / `w` / `c` keywords, PR numbers and URLs, workflow names,
-confirmation conditions, and `#` comments — plus an error scope for steps
+files: the `l` / `w` / `c` / `x` keywords, PR numbers and URLs, workflow
+names, confirmation conditions, commands, and `#` comments — plus an error scope for steps
 `autoland` would reject, so a typo shows up before you run the plan. It also
 formats them: **Format Document** aligns trailing comments at column 30 (or
 past the longest `l`/`w` step in the block, when one runs that far) and tidies

editors/vscode/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## 0.3.0
+
+* Highlights `x <command>` steps, which run a command and stop the land
+  unless it exits 0: the `x` keyword, and the command as a string. A bare `x`
+  highlights as an error, since the parser rejects it. The formatter
+  collapses the gap after `x` like any other step, and, as for `c`, an `x`
+  step never sets the comment column.
+
 ## 0.2.0
 
 * The extension is now a formatter for the language: **Format Document**

editors/vscode/README.md

@@ -1,7 +1,7 @@
 # stack-pr Autoland Plan — VS Code support
 
 Syntax highlighting and formatting for `stack-pr autoland` landing plans: the
-`l` / `w` / `c` step files you edit with `autoland -i` or pass to
+`l` / `w` / `c` / `x` step files you edit with `autoland -i` or pass to
 `autoland --plan-file`.
 
 See [`examples/example.autoland-plan`](examples/example.autoland-plan) for a
@@ -11,25 +11,27 @@ plan exercising every step type.
 
 | Plan syntax | Scope |
 | --- | --- |
-| `l`, `w`, `c` step keywords | `keyword.control.<step>.autoland-plan` |
+| `l`, `w`, `c`, `x` step keywords | `keyword.control.<step>.autoland-plan` |
 | `l 123` — the PR number | `constant.numeric.pr-number.autoland-plan` |
 | `l https://…/pull/123` — the PR URL | `markup.underline.link.autoland-plan` |
 | `w deploy.yaml` — the workflow name | `entity.name.function.workflow.autoland-plan` |
 | `c QA sign-off complete` — the condition | `string.unquoted.condition.autoland-plan` |
+| `x ./provision.sh staging` — the command | `string.unquoted.command.autoland-plan` |
 | `#` comments, whole-line and trailing | `comment.line.number-sign.autoland-plan` |
 | a step the parser would reject | `invalid.illegal.*.autoland-plan` |
 
 The `invalid` scopes mirror `parse_plan` in `stack_pr/autoland/plan.py`, so a
 plan that highlights as an error is a plan `autoland` will refuse to run: an `l`
 with something that is neither a PR number nor a PR URL, a `w` with no
-workflow, or a line that is not a step at all.
+workflow, an `x` with no command, or a line that is not a step at all.
 
 Two details worth knowing, both inherited from the parser:
 
 - A `#` only starts a comment when whitespace precedes it. `l #123` is a bare
   `l` followed by a comment (which is why `l #123` does not pin PR 123), while
   `c ship it#now` keeps the `#` inside the condition.
-- A workflow name runs to the end of the step, spaces included.
+- A workflow name runs to the end of the step, spaces included, and so does
+  an `x` command — so a ` #` ends the command even inside quotes.
 
 ## Formatting
 
@@ -45,14 +47,14 @@ does — the formatted text parses into exactly the same steps:
 - **A long `l` or `w` step widens its block**: if one reaches into column 30,
   that block's comments move to four spaces past the longest such step, so the
   block still lines up internally without affecting the rest of the file.
-- **`c` steps never set the column.** A confirm condition is free text that
-  routinely runs long, and letting it decide would drag every comment in the
-  block off to the right. Their comments are still aligned with everything
-  else; a condition already past the column keeps a single space before its
+- **`c` and `x` steps never set the column.** A confirm condition is free
+  text and a command line can be anything, and both routinely run long;
+  letting them decide would drag every comment in the block off to the right.
+  Their comments are still aligned with everything else; a condition already past the column keeps a single space before its
   `#`, since losing that space would fold the comment into the condition.
 - **The gap after a step keyword is collapsed** to a single space, and steps are
-  unindented. The argument itself is left alone: a condition is free text, and a
-  workflow name may contain spaces.
+  unindented. The argument itself is left alone: a condition is free text, a
+  workflow name may contain spaces, and a command's spacing may be quoted.
 - **Whole-line comments keep their own indentation**, since the generated plan
   header is hand-aligned. They do not break up a block.
 - **Trailing whitespace goes**, and the file ends with exactly one newline.
@@ -88,7 +90,7 @@ From this directory:
 ```bash
 # Package and install into VS Code.
 npx --yes @vscode/vsce package
-code --install-extension autoland-plan-0.2.0.vsix
+code --install-extension autoland-plan-0.3.0.vsix
 ```
 
 Or, for development, symlink it into your extensions directory and reload:

editors/vscode/examples/example.autoland-plan

@@ -5,13 +5,16 @@
 # c [condition] = pause for manual confirmation; the optional
 #                 condition names what to verify before proceeding
 #                 (e.g. 'c QA sign-off complete')
+# x <command>   = run a command, and stop unless it exits 0; it runs
+#                 without a shell (use 'x bash -c ...' for pipes)
 #
 # Lines starting with # are comments and are ignored.
 # Blank lines are ignored.
 #
 l 101                         # Add /widgets API endpoint
 w deploy.yaml                 # wait for the deploy of PR #101 to finish
 c QA sign-off complete        # pause until QA has signed off
+x ./scripts/provision.sh staging # stop unless it succeeds
 l 102                         # Wire up the widgets UI
 
 # A bare 'l' lands whatever PR is next in the stack, and a PR can also be

editors/vscode/examples/generated.autoland-plan

@@ -5,12 +5,15 @@
 # c [<condition>] = pause for manual confirmation; the optional
 #                   condition names what to verify before proceeding
 #                   (e.g. 'c QA sign-off complete')
+# x <command>     = run a command, and stop unless it exits 0; it runs
+#                   without a shell (use 'x bash -c ...' for pipes)
 #
 # Lines starting with # are comments and are ignored.
 # Blank lines are ignored.
 #
 l 101                         # Add /widgets API endpoint
 w deploy.yaml
 c QA sign-off complete
+x ./scripts/provision.sh --env staging
 l 1024                        # Wire up the widgets UI
 l 103

editors/vscode/package.json

@@ -1,8 +1,8 @@
 {
   "name": "autoland-plan",
   "displayName": "stack-pr Autoland Plan",
-  "description": "Syntax highlighting and formatting for stack-pr autoland plan files (l / w / c steps).",
-  "version": "0.2.0",
+  "description": "Syntax highlighting and formatting for stack-pr autoland plan files (l / w / c / x steps).",
+  "version": "0.3.0",
   "publisher": "micahjsmith",
   "license": "MIT",
   "repository": {

editors/vscode/src/format.js

@@ -14,9 +14,10 @@ const COMMENT_COLUMN = 30;
 // comments move this far past the longest one.
 const COMMENT_GAP = 4;
 
-// 'l', 'w' or 'c', optionally followed by an argument. The argument keeps its
-// own spacing: a condition is free text and a workflow name may contain spaces.
-const STEP = /^([lwc])[ \t]+(.*)$/;
+// 'l', 'w', 'c' or 'x', optionally followed by an argument. The argument keeps
+// its own spacing: a condition is free text, a workflow name may contain
+// spaces, and a command's spacing may sit inside its quotes.
+const STEP = /^([lwcx])[ \t]+(.*)$/;
 // A step with no argument at all: a bare 'l' or 'c'.
 const BARE_STEP = /^[lwc]$/;
 
@@ -58,9 +59,9 @@ function parseLine(raw) {
 /**
  * Comment column for one block.
  *
- * Only 'l' and 'w' steps are measured. A 'c' condition is free text that
- * routinely runs past any sensible column, and letting it set one would drag
- * every comment in the block off to the right.
+ * Only 'l' and 'w' steps are measured. A 'c' condition is free text, and an
+ * 'x' command line, that routinely runs past any sensible column, and letting
+ * either set one would drag every comment in the block off to the right.
  */
 function commentColumn(block) {
   const widths = block

editors/vscode/syntaxes/autoland-plan.tmLanguage.json

@@ -30,6 +30,12 @@
     {
       "include": "#confirm-bare"
     },
+    {
+      "include": "#exec"
+    },
+    {
+      "include": "#exec-missing"
+    },
     {
       "include": "#unknown-step"
     },
@@ -145,6 +151,27 @@
         }
       }
     },
+    "exec": {
+      "comment": "x <command> — a command line, run without a shell. Like every step it ends at the first ' #', quotes or not, so it cannot start with '#'.",
+      "match": "^[ \\t]*(x)[ \\t]+([^ \\t#](?:(?! +#).)*)",
+      "captures": {
+        "1": {
+          "name": "keyword.control.exec.autoland-plan"
+        },
+        "2": {
+          "name": "string.unquoted.command.autoland-plan"
+        }
+      }
+    },
+    "exec-missing": {
+      "comment": "'x' requires a command.",
+      "match": "^[ \\t]*(x)(?=[ \\t]*$| +#)",
+      "captures": {
+        "1": {
+          "name": "invalid.illegal.missing-command.autoland-plan"
+        }
+      }
+    },
     "unknown-step": {
       "comment": "Anything else makes the parser fail with 'unrecognized step'.",
       "match": "^[ \\t]*(\\S(?:(?! +#).)*)",

editors/vscode/tests/format.test.js

@@ -33,6 +33,19 @@ test("a c step never sets the column", () => {
   );
 });
 
+test("an x step never sets the column", () => {
+  // Like a condition, a command line routinely runs long.
+  const src = ["x ./scripts/provision.sh --env staging --verbose # provision", "l 101 # land it"].join("\n");
+  assert.strictEqual(
+    formatPlan(src),
+    ["x ./scripts/provision.sh --env staging --verbose # provision", "l 101                         # land it", ""].join("\n")
+  );
+});
+
+test("collapses the space after x but not inside its command", () => {
+  assert.strictEqual(formatPlan("x    bash -c 'echo  hi' # say hi\n"), "x bash -c 'echo  hi'          # say hi\n");
+});
+
 test("a c step past the column keeps a space before its comment", () => {
   // Without the space the '#' would become part of the condition, which would
   // change what the plan does.