Repository navigation
watch: no burst throttle for checkout storms; a rebase runs per-file git subprocesses, state writes and topology recomputes, plus repeated graph rebuilds #201
Description
Raised by Fazal after reading watchProcess (main.go:28) and asking whether newWatchProcess throttles inotify event batches during a large git rebase checkout or triggers continuous AST rebuilds. Answer from the code on main: partly. Writes are debounced and the graph rebuild is coalesced, but there is no batch-level throttle on a checkout storm, and several per-file costs run synchronously inside it.
What is already throttled
- Pure WRITE events get a per-path trailing-edge debounce, 100 ms window (
eventLoop,watch/events.go:156; injectable per daemon since fix(mcp,watch,hooks): no confident zeros for unindexed files, surface state write errors, Codex post-edit always emits JSON #200). CREATE, REMOVE and RENAME are never debounced (debounceAction,events.go:447). .git/is never watched:addWatchDirsskips every dot-directory (watch/daemon.go:624), so git's object and index churn produces no events.- Gitignored paths and non-source extensions are dropped before any work (
handleEvent,events.go:516; extension check atevents.go:274). - The dependency-graph rebuild is coalesced:
refreshDependencies(daemon.go:409) hands one snapshot to a single worker, further invalidations only setdependencyPending, andhandleDependencyGraphResultschedules exactly one trailing rebuild. At most one in-flight plus one catch-up build per quiet period. Repos overlimits.LargeRepoFileCount(5,000) skip the graph entirely.
What is not throttled
A checkout writes each file through a temp file and a rename, so every touched source file arrives as CREATE and RENAME events, which bypass the debounce. For each one, handleEvent does the following synchronously on the event loop:
| per changed file | where |
|---|---|
os.Stat plus a full read to count lines |
events.go:597 |
a git diff --quiet -- <file> subprocess on CREATE/WRITE, racing git's own index.lock |
isFileDirty, events.go:900 |
a synchronous state.json write for every configured file |
events.go:686 |
a full computeTopology (re-parses build manifests) on every source create/remove/rename |
events.go:693, daemon.go:228 |
a rebuild kick with no settle window, so a rebase whose checkout steps each outlast one graph build gets one full BuildFileGraph per step |
refreshDependencies, daemon.go:409 |
So a rebase touching N files costs N git subprocesses, N state writes, N topology recomputes and several graph rebuilds. Not continuous in the unbounded sense, but not bounded by the burst either.
Proposed fix
- A settle window on
refreshDependencies(500 ms to 2 s after the last invalidation) instead of kicking the worker on the first one; put thestate.jsonwrite andcomputeTopologybehind the same window. The control-event path already does this for.gitignoreedits withcontrolRefreshWindow(150 ms,events.go:50), so the pattern exists. - Checkout detection: while
.git/rebase-merge,.git/rebase-apply,.git/MERGE_HEADor.git/index.lockexists, defer rebuilds and dirty checks, then do one rebuild when they disappear. - One
git status --porcelainper burst instead ofgit diff --quietper file.
Exit test
A fixture repo with 500 source files and a 20-commit branch; git rebase it onto a moved base while the daemon runs. Count, over the rebase: git subprocesses spawned by the daemon (target: at most one per settle window), state.json writes (at most one per settle window), computeTopology calls (at most one per settle window), and BuildFileGraph runs (exactly one after the rebase completes, plus at most one mid-rebase). The daemon's final state must equal a fresh scan of the post-rebase tree.
Related: #151, #155, #185 (daemon lifecycle), #186 (guard rails). Milestone M3 in the October roadmap.
Activity
fix(watch): coalesce checkout storms into one batch; prove a second s…
test(watch): burst detector takes its clock from the caller; storm te…