Skip to content

build: cache every task that build:all runs - #492

Draft
wan9chi wants to merge 8 commits into
mainfrom
feat/remote-cache-build-all
Draft

wan9chi wants to merge 8 commits into
mainfrom
feat/remote-cache-build-all

Conversation

@wan9chi

@wan9chi wan9chi commented Oct 19, 2026 •

Copy link
Copy Markdown
Member

Motivation

vp run build:all (38 package and playground builds) runs every build from scratch on every CI run and every local run. This caches those builds, locally and in CI's remote cache, by running vp run --cache build:all. Inputs and outputs are tracked automatically, so a hit restores dist/, .void/, .output/ and so on.

Important

Depends on the vite-plus preview build from voidzero-dev/vite-plus#3113, which bumps vite-task to its feat/rebuildable-cache-rules branch. Those rules cache tasks that write files they read, and restore outputs that a hit can rebuild. Don't merge before a vite-plus release includes them. Drop the commit that switches to the preview build then.

What's configured

  • Every build stays a package.json script, as on main. --cache turns caching on for package.json scripts as well as vite.config tasks, and nested vp run calls without a cache flag inherit it, so no package needs a vite.config task to be cached. Each step of an && chain is cached on its own. Plain vp run build:all, and every other vp run, still caches no scripts, so dev and friends are unaffected.
  • vp run --cache build:all is used where build:all runs: the root test:all task and the CI checks job. build:all itself is unchanged.
  • The root build script builds packages in dependency order instead of --parallel: vp run --filter './packages/**' --filter '!./packages/md' build && vp run --filter ./packages/md build. With --parallel, a build could read a dependency's outputs while they were still being restored or rebuilt. @void/md builds last because it and void depend on each other.
  • @void/platform's build script is vp pack && node scripts/build-runtime.mjs && vp run build:stamp, so a new commit reruns only a small uncached step:
    • scripts/build-runtime.mjs (admin client, five Workers, migrations) writes dist/runtime/manifest.json with sourceRevision: null and is cached like the other steps.
    • build:stamp, the only vite.config task, runs the new scripts/stamp-source-revision.mjs with cache: false. It sets sourceRevision as before, from VOID_PLATFORM_SOURCE_REVISION or else git (with -dirty for a dirty tree), validates the manifest again with parsePlatformRuntimeManifest and writes it back. An uncached task gets the full environment, so the variable needs no declaration.
    • The runtime build no longer fingerprints VOID_PLATFORM_SOURCE_REVISION or Git state. Nothing else reads the manifest at build time, and its SHA-256 is computed by the CLI from the installed file, so the stamp doesn't invalidate other tasks.
    • @void/isr is a new dev dependency of @void/platform: the dispatch Worker bundles @void/isr/runtime from isr's build output, and the dependency makes the root build run isr first. isr depends only on @void/edge and @void/deploy-core, so this adds no cycle.
  • playground/astro's script is NODE_USE_SYSTEM_CA=1 astro build. Miniflare downloads Request.cf while prerendering; trusting the system CA store makes that work behind a TLS-intercepting proxy whose CA is installed there, without forwarding a per-machine NODE_EXTRA_CA_CERTS path.

CI (second commit), for reviewers

  • The checks job reads the endpoint from the VP_REMOTE_CACHE_URL repository variable, which is now set. Pushes and pull requests from this repository upload (read-write); fork pull requests, which get no OIDC token, only download (read). Without the variable, remote caching is off.
  • The cache server accepts pull request uploads only for namespaces that opt in, and void's opt-in is still pending (a cache server PR, then a production deploy). Until then this PR's uploads are rejected and its runs show no remote hits; after it, a second run of this PR should restore from the first. Pull request uploads land in the namespace main reads, so same-repo PR code can publish entries main later restores; Scope pull request uploads to the pull request vite-plus-remote-cache-cloudflare#16 tracks scoping them to the pull request.
  • Uploads authenticate with a GitHub Actions OIDC token. That needs new id-token: write permissions on the checks job and on publish.yml's ci: caller; a reusable workflow can't request more than its caller grants. Please check these.
  • The build job no longer sets VOID_PLATFORM_SOURCE_REVISION; the uncached stamp step runs git itself. Set at job level, the variable reached every task, and the playgrounds that fingerprint the whole environment (react-router, tanstack-start, astro, sveltekit) missed on every commit because its value changes with each one. The release workflows (publish.yml, prepublish-test.yml) still set it, as on main.

Validation (macOS, preview build)

run hits misses
task cache and build outputs cleared, vp run --cache build:all 0/40 all no previous cache entry, none refused (26 s)
again 39/40 runtime step: 'dist' added in 'packages/void' (listing issue below)
after an empty commit 39/40 runtime step: 'packages/platform/dist/runtime/manifest.json' modified (stamped-manifest issue below); the manifest records the new revision
build outputs removed again, cache kept (like a fresh CI checkout) 40/40 –
vp run build:all without --cache – all 41 cache disabled (24 s)

The stamp runs every time. node test/check-playground-build-artifacts.mjs passes.

Known issues

  • Git worktrees (no longer affects void): in a linked worktree .git is a file, so a .git/HEAD input fails with Not a directory (os error 20). The failure isn't reported as an error: vp run build / build:core skipped the platform runtime and void builds and still exited 0. void no longer has .git inputs, and vp run build:core in a linked worktree now runs all 9 tasks. The silent skip remains a vite-task bug.
  • No hits across checkout paths: vite-plus passes the checkout's absolute vite-plus bin path to vp pack / vp build, so each of those tasks misses with args changed at another path. CI checks out at the same path every run, so CI isn't affected. Path-embedding playgrounds (astro, nuxt, …) can't cross paths anyway.
  • Platform runtime step misses on directory listings: it bundles sibling packages' sources and resolves through their directories, without depending on those packages. When their dist, node_modules/.vite-temp, the dashboard's generated .void/tsconfig.json or node_modules/.pnpm-workspace-state-v1.json appear later, the step misses once (about 5 s).
  • Platform runtime step misses after it reruns over a stamped manifest: when scripts/build-runtime.mjs reruns while an earlier dist/runtime is still there, removing the old folder counts as reading the old manifest.json, so the entry records the manifest as the step left it, with sourceRevision: null. The stamp then changes it, and each later run misses with 'packages/platform/dist/runtime/manifest.json' modified and records the same again. Locally this starts when the runtime step reruns while vp pack hits, for example after editing a platform Worker, and ends after a run without dist/runtime. In CI, prepare runs build:core uncached before build:all, which always leaves a stamped dist/runtime, so once uploads are accepted the runtime step would miss on every run. Avoiding it needs either the stamp to write the revision outside the runtime step's outputs, or vite-task to not treat a path a task deleted and recreated as one it read.

🤖 Generated with Claude Code

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

🚀 Deploying Preview to Cloudflare 🚀

Preview URL: https://feat-remote-cache-build-all-void-cloud.voidzero-docs.workers.dev (commit d9e4832)

This URL reflects your latest Preview deployment

Preview Deployments by commit

Status Deployment URL Commit Updated (UTC) See this deployment's details
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://8f1a2bf2-void-cloud.voidzero-docs.workers.dev d9e4832 2026-10-19T07:46:29.139Z Visit the dashboard ↗
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://35a2cdcc-void-cloud.voidzero-docs.workers.dev f48ca6a 2026-10-19T07:26:30.305Z Visit the dashboard ↗
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://77cf7817-void-cloud.voidzero-docs.workers.dev 9a4e504 2026-10-19T07:16:41.851Z Visit the dashboard ↗
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://fe212a3b-void-cloud.voidzero-docs.workers.dev 252c3e6 2026-10-19T07:14:26.089Z Visit the dashboard ↗
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://a6a9737d-void-cloud.voidzero-docs.workers.dev e9206e0 2026-10-19T06:53:38.590Z Visit the dashboard ↗
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://f0787d7a-void-cloud.voidzero-docs.workers.dev a2fc5b4 2026-10-19T06:48:25.879Z Visit the dashboard ↗
  • Build: Success ✅
  • Deployment: Success ✅

View logs ↗
https://0717f50b-void-cloud.voidzero-docs.workers.dev 57dcba9 2026-10-19T05:29:54.998Z Visit the dashboard ↗

wan9chi and others added 3 commits October 19, 2026 14:43
Move each package's and playground's `build` script into `run.tasks.build`
in its vite.config.ts, so `vp run` caches it, locally and in the remote
cache, with automatically tracked inputs and outputs.

- Build packages in dependency order with a root `build` task instead of
  `--parallel`, so no build reads a dependency's outputs while they are
  rebuilt or restored. @void/md builds last: it and void depend on each
  other.
- @void/platform: run `vp pack` as its own `build:pack` task. The runtime
  build tracks VOID_PLATFORM_SOURCE_REVISION and Git's HEAD and refs,
  which its manifest records, and waits for @void/isr, whose build output
  the dispatch Worker bundles.
- playground/astro: pass NODE_EXTRA_CA_CERTS to the build without
  fingerprinting it, for Miniflare's `Request.cf` download behind a
  TLS-intercepting proxy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read the remote cache endpoint from the VP_REMOTE_CACHE_URL repository
variable. Pushes upload results, authenticated with a GitHub Actions OIDC
token; pull requests only download them. Without the variable the remote
cache stays off.

Set VOID_PLATFORM_SOURCE_REVISION to the commit, so the cached platform
runtime build records it without running `git`, whose reads of the whole
checkout would otherwise become inputs of the task.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ules

Install vite-plus, its core, and prompts at 0.0.0-commit.80a3cbc3 from
the Vite+ registry bridge: the preview of voidzero-dev/vite-plus#3113,
which bumps vite-task to the feat/rebuildable-cache-rules branch. That
vite-task caches tasks that write files they read and restores outputs a
hit can rebuild, which the build tasks need.

Drop this commit once a vite-plus release includes those rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wan9chi and others added 3 commits October 19, 2026 14:52
Pull requests from this repository now upload their task results too, so a
pull request's later runs reuse its earlier ones and the cache can be seen
working before merging. The cache server accepts them only once void's
namespace opts in to pull request uploads (voidzero-dev/vite-plus-remote-cache-cloudflare#16
tracks scoping them to the pull request). Fork pull requests get no OIDC
token and stay read-only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runtime build recorded the source revision in its manifest, so it
fingerprinted VOID_PLATFORM_SOURCE_REVISION and Git's HEAD and refs and
reran on every commit. It now writes the manifest without a revision, and
an uncached `build` step records the revision afterwards, so a new commit
reruns only that step.

- `build:runtime` runs scripts/build-runtime.mjs after `build:pack` and
  `@void/isr#build`, with default caching.
- `build` runs scripts/stamp-source-revision.mjs with `cache: false`. It
  resolves the revision as before, from VOID_PLATFORM_SOURCE_REVISION or
  `git`, and validates the manifest again. An uncached task gets the full
  environment, so CI's VOID_PLATFORM_SOURCE_REVISION still reaches it.
- Drop the `.git/HEAD` and `.git/refs/**` inputs. In a linked Git
  worktree, where `.git` is a file, they made the cache lookup fail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It was set so the cached runtime build would record the commit without
running `git`. The revision is now stamped by an uncached step, which can
run `git` itself. Set at job level, the variable reached every task, and
the playgrounds whose builds fingerprint the whole environment
(react-router, tanstack-start, astro, sveltekit) missed on every commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Prerendering starts Miniflare, which downloads Cloudflare's `Request.cf`
data. Instead of forwarding NODE_EXTRA_CA_CERTS as an untracked env, the
task runs with NODE_USE_SYSTEM_CA=1, so a TLS-intercepting proxy whose CA
is in the system store works without per-machine paths, and the task's
environment is the same everywhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wan9chi added a commit to voidzero-dev/vite-plus-remote-cache-cloudflare that referenced this pull request Oct 19, 2026
…ith void (#17)

## Motivation

[voidzero-dev/void#492](voidzero-dev/void#492)
caches every task `build:all` runs, and its PR CI should show remote
cache reuse before it merges. void#492 already makes same-repository PR
runs read-write, but the production cache only accepts uploads from
default-branch `push` jobs, so those PR uploads get `403 write_policy`.
The decision is to let void's same-repository PR CI upload to its
namespace
(`https://voidzero-remote-cache.voidzero-docs.workers.dev/projects/void`)
for now. Every other namespace keeps the main-branch-push-only policy.

## Design

- **Opt-in per namespace, off by default.** Migration
`0002_pull_requests.sql` adds `scopes.pull_requests` (0/1, default 0)
and recreates the `changed_policy` trigger with that column, so turning
the opt-in on or off bumps `policy_version` and stops uploads admitted
under the old policy.
- **Write policy (`src/auth.ts`).** `push` tokens work as before. For an
opted-in namespace, a `pull_request` token passes the same audience,
`repository_id`, `repository_owner_id`, and `repository_visibility:
public` checks, and also needs:
  - `ref` matching `refs/pull/<number>/merge`
  - `ref_type` `branch`
- `base_ref` equal to the registered default branch's short name
(`main`, not `refs/heads/main`)

These are the values GitHub signs for a PR's merge-branch run: see the
[`ref`
description](https://docs.github.com/en/actions/reference/workflows-and-actions/variables)
("for pull request events except `pull_request_target` that were not
merged, it is `refs/pull/<pr_number>/merge`"), `GITHUB_BASE_REF` ("for
example, `main`"), the [OIDC claim
reference](https://docs.github.com/en/actions/reference/security/oidc),
and the decoded `pull_request` token in
[github/actions-oidc-debugger](https://github.com/github/actions-oidc-debugger)
(`"ref": "refs/pull/27/merge"`, `"ref_type": "branch"`, `"base_ref":
"main"`). `pull_request_target`, `workflow_run`, tags, PRs to other
branches, the `closed` run of a merged PR (whose `ref` is the base
branch), and workflows in fork repositories are still denied.
- **Configuration.** A `.github/production-repositories.jsonc` entry can
now be `{ "repository": "owner/repo", "pullRequests": true }` as well as
`"owner/repo"`, which means `pullRequests: false`. Only `void` uses the
object form. `pnpm ci:deploy production` sets every namespace's opt-in
from the list on each run, the same way it already handles
`enabled`/`writes_enabled`. Self-hosters use `pnpm operator policy
--pull-requests on|off`, or `setup --pull-requests on` when creating a
namespace. `status` already prints the full row.
- **Staging** keeps `pull_requests = 0`, so PR staging runs still check
that PR tokens are rejected.
- **Logging.** Verified writes now also log `event_name` and `head_ref`.
`base_ref` always equals the default branch, and the PR number is
already in `workflow_ref`.

## Risk

PR entries share the namespace with default-branch entries. Code from
any same-repository PR, including Renovate dependency branches, runs
before review and can publish an entry that a later default-branch build
restores. That's why the opt-in is off by default and only on for void.
The README, self-hosting guide, e2e plan, and RFC describe it as interim
and link the long-term fix, #16 (scope pull request uploads to the pull
request).

The server can't tell fork PRs from same-repository PRs: fork PR runs
carry the base repository's IDs. It relies on GitHub not granting
`id-token: write` to fork PR workflows. GitHub only does that when "Send
write tokens to workflows from pull requests" is enabled, a setting it
offers for private repositories only. The docs state this dependency.

## After merge

Production changes only after a maintainer runs **Actions → Remote cache
production** on `main`. That run applies migration 0002 and enables PR
uploads for `void`. Its log should print `Enabled pull request uploads
for namespace void`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
… run --cache

Move every `build` back into package.json scripts, as on main, and turn
caching on where build:all runs (the `test:all` task and CI's build job)
with `vp run --cache build:all`. `--cache` caches package.json scripts
too, and reaches the nested `vp run` calls, so no package needs a
vite.config task to be cached. Each step of an `&&` chain is cached on
its own.

- Root `build` script: build packages in dependency order instead of
  `--parallel`, @void/md last, since it and void depend on each other.
- @void/platform: `vp pack && node scripts/build-runtime.mjs && vp run
  build:stamp`. `build:stamp`, the only vite.config task left, records
  the source revision with `cache: false`, so a new commit reruns only
  that step.
- @void/platform: add @void/isr as a dev dependency. The runtime build
  bundles `@void/isr/runtime` from isr's build output, and the
  dependency makes the root build run isr first.
- playground/astro: `NODE_USE_SYSTEM_CA=1 astro build`, for Miniflare's
  `Request.cf` download behind a TLS-intercepting proxy.
- Delete the vite.config.ts files that only held the build task.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant