Skip to content

watch: no burst throttle for checkout storms; a rebase runs per-file git subprocesses, state writes and topology recomputes, plus repeated graph rebuilds #201

Closed

Description

@JordanCoin

Raised by Fazal after reading watchProcess (main.go:28) and asking whether newWatchProcess throttles inotify event batches during a large git rebase checkout or triggers continuous AST rebuilds. Answer from the code on main: partly. Writes are debounced and the graph rebuild is coalesced, but there is no batch-level throttle on a checkout storm, and several per-file costs run synchronously inside it.

What is already throttled

  • Pure WRITE events get a per-path trailing-edge debounce, 100 ms window (eventLoop, watch/events.go:156; injectable per daemon since fix(mcp,watch,hooks): no confident zeros for unindexed files, surface state write errors, Codex post-edit always emits JSON #200). CREATE, REMOVE and RENAME are never debounced (debounceAction, events.go:447).
  • .git/ is never watched: addWatchDirs skips every dot-directory (watch/daemon.go:624), so git's object and index churn produces no events.
  • Gitignored paths and non-source extensions are dropped before any work (handleEvent, events.go:516; extension check at events.go:274).
  • The dependency-graph rebuild is coalesced: refreshDependencies (daemon.go:409) hands one snapshot to a single worker, further invalidations only set dependencyPending, and handleDependencyGraphResult schedules exactly one trailing rebuild. At most one in-flight plus one catch-up build per quiet period. Repos over limits.LargeRepoFileCount (5,000) skip the graph entirely.

What is not throttled

A checkout writes each file through a temp file and a rename, so every touched source file arrives as CREATE and RENAME events, which bypass the debounce. For each one, handleEvent does the following synchronously on the event loop:

per changed file where
os.Stat plus a full read to count lines events.go:597
a git diff --quiet -- <file> subprocess on CREATE/WRITE, racing git's own index.lock isFileDirty, events.go:900
a synchronous state.json write for every configured file events.go:686
a full computeTopology (re-parses build manifests) on every source create/remove/rename events.go:693, daemon.go:228
a rebuild kick with no settle window, so a rebase whose checkout steps each outlast one graph build gets one full BuildFileGraph per step refreshDependencies, daemon.go:409

So a rebase touching N files costs N git subprocesses, N state writes, N topology recomputes and several graph rebuilds. Not continuous in the unbounded sense, but not bounded by the burst either.

Proposed fix

  1. A settle window on refreshDependencies (500 ms to 2 s after the last invalidation) instead of kicking the worker on the first one; put the state.json write and computeTopology behind the same window. The control-event path already does this for .gitignore edits with controlRefreshWindow (150 ms, events.go:50), so the pattern exists.
  2. Checkout detection: while .git/rebase-merge, .git/rebase-apply, .git/MERGE_HEAD or .git/index.lock exists, defer rebuilds and dirty checks, then do one rebuild when they disappear.
  3. One git status --porcelain per burst instead of git diff --quiet per file.

Exit test

A fixture repo with 500 source files and a 20-commit branch; git rebase it onto a moved base while the daemon runs. Count, over the rebase: git subprocesses spawned by the daemon (target: at most one per settle window), state.json writes (at most one per settle window), computeTopology calls (at most one per settle window), and BuildFileGraph runs (exactly one after the rebase completes, plus at most one mid-rebase). The daemon's final state must equal a fresh scan of the post-rebase tree.

Related: #151, #155, #185 (daemon lifecycle), #186 (guard rails). Milestone M3 in the October roadmap.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions