Skip to content

fix(hooks): spool captures for a detached drainer where the resident worker cannot run (#659) - #663

Merged
cdeust merged 6 commits into
mainfrom
fix/capture-windows-659
Oct 7, 2026
Merged

cdeust merged 6 commits into
mainfrom
fix/capture-windows-659

Conversation

@cdeust

@cdeust cdeust commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Fixes #659 for the capture path (auto-capture stored nothing on Windows). The other half of #659's text, check_setup answering ready: true while every capture fails, is not covered here: it is tracked by #660.

Root cause

Since 17370a6 every capture goes through a resident worker that needs AF_UNIX, fcntl, geteuid and Popen(pass_fds). None exist on Windows; capture_peer.supported() raised and dispatch() reported capture_skipped for every event.

The first version of this PR stored in the hook process. Measured 2026-10-07 (macOS, isolated SQLite, warm): sentence_transformers import 3.61 s, whole capture 4.4 s warm and 10.5 s cold, against the hook's timeout: 10. That puts the cost the worker exists to amortize (ADR-0486/0508) back on the synchronous path, so the host would abort the hook instead of the capture being refused.

Fix (ADR-1094)

Where capture_peer.is_supported() is false, the hook never loads the handler:

  • capture_dispatch.dispatch validates the payload, writes it unchanged and atomically to <CLAUDE_DIR>/.capture-worker/spool/ (infrastructure/capture_spool.py) and starts python -m mcp_server.hooks.capture_drain without waiting. Hook body: 0.06 s measured.
  • popen_options(platform, stream) is a pure function: Windows DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP, close_fds, no pass_fds, no start_new_session; other platforms start_new_session. stdout/stderr go to drain.err.
  • capture_drain: wire_composition_root() ([audit] 17 core/ modules import os or pathlib (core must be zero-I/O) #560), close_shared_store_on_exit (fix(hooks): consolidate_background never exits after a successful cycle — unclosed psycopg pool spins a core at 98% CPU indefinitely #398), non-blocking kernel lock (msvcrt.locking / flock), stores every pending file through the same capture_store.store the worker awaits (one handler load per burst), deletes after storing (at-least-once, the write gate absorbs a replay), releases, rescans once. A refused or failing file becomes *.rejected and a capture_skipped report.
  • POSIX is unchanged. resident_worker_available() and store_in_process are removed.
  • ci.yml: the three capture test files join the Windows leg (ADR-1091 convention), where is_supported() is false natively, the msvcrt lock is real and the detached child runs with the real flags.
  • docs/provenance/capture-worker-design.md: false "no Windows runner" claim removed; spool described.

Completion Ledger

Item State
Capture path stores on a platform without the worker (simulated locally, real on the Windows leg) done, tests below
Hook no longer loads the handler on that path done (test: _load_remember raises)
Windows CI leg runs the capture tests done, ci.yml
Windows runner result not run locally; the leg is the signal
Real Windows end to end (Git Bash launch, cold latency, host tree cleanup) not verified; signal: reporter's telemetry.jsonl with remember samples and no capture_skipped, spool empty
check_setup "ready" while capture is broken not in this PR, #660
Spool on POSIX not done, outside scope (changes the ADR-0486/0508 path)

Tests

test_capture_unsupported_platform.py (hook path, popen_options, capability), test_capture_drain.py (real handler, held lock, reject, handler failure, rescan, drainer outliving its parent), test_capture_spool.py. The hook behaviour is red on origin/main (nothing spooled, capture_skipped) and on baa8ece (handler loaded in the hook).

🤖 Generated with Claude Code

https://claude.ai/code/session_01LFAg4i6CQpZrNXb8cHH8ax

cdeust and others added 3 commits October 6, 2026 21:52
…t run (#659)

The worker needs AF_UNIX, fcntl, geteuid and Popen(pass_fds): none exist on
Windows, so every capture raised OSError and was reported as capture_skipped.
A capability test (capture_peer.is_supported) now selects the path before any
I/O; unsupported platforms run capture_store.remember, the same function the
worker awaits, once per hook process.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LFAg4i6CQpZrNXb8cHH8ax
…worker cannot run (#659)

The first fix stored in the hook process on Windows, which puts the
sentence_transformers import (3.61 s measured, 4.4 s warm and 10.5 s cold for
the whole capture) on the synchronous path of a hook with timeout 10. The hook
now validates the payload, writes it atomically to .capture-worker/spool and
starts capture_drain detached (DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP on
Windows) without waiting. The drainer takes a non-blocking kernel lock, stores
every pending file through capture_store.store with one handler load per
burst, deletes after storing and rescans once after releasing the lock. A
refused or failing file is renamed *.rejected and reported as capture_skipped.
POSIX keeps the resident worker. The three capture test files join the Windows
CI leg. ADR-1094.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LFAg4i6CQpZrNXb8cHH8ax
@cdeust cdeust changed the title fix(hooks): store captures in-process where the resident worker cannot run (#659) fix(hooks): spool captures for a detached drainer where the resident worker cannot run (#659) Oct 7, 2026
cdeust and others added 2 commits October 7, 2026 14:10
@cdeust

cdeust commented Oct 7, 2026

Copy link
Copy Markdown
Owner Author

Revue du contrôleur (Opus 5.5), quatrième passage
Head reviewed: 1aadf34

PR #663 (#659), 4e passage : APPROVE

Les deux bloquants de 940a6d3 sont corrigés.

Bloquants précédents, corrigés

  1. Âge du spool. Le calcul est maintenant now - min(mtime) sur tous les fichiers en attente (capture_spool.py:110-119), et FileNotFoundError est sauté. Le test fixe cinq mtimes par os.utime (700/100/900/300/500) sans rapport avec les noms et attend 900.0. Le drainer trie seulement pour le déterminisme (capture_drain.py:60). Comme le dit ADR-1094, l'ordre n'est pas un contrat. Seul effet d'ordre : la concaténation de merge_contents (cosmétique).
  2. Mode du log. fchmod agit sur le descripteur écrit, dans open_rotating_log. Rotation réelle (umask 022, 0o644 à MAX_LOG_BYTES) : A inside: log 0o600 fd 0o600 size 0 / A after: .1 0o644 lock 0o600 / B after: log 0o600 .1 0o600 / C fresh: log 0o600 lock 0o600. Le .1 en 0o644 hérite d'un contenu déjà lisible. O_NOFOLLOW est couvert par lstat (log_rotation.py:28-36).

CI

Tout est vert, CI Green compris. Windows (run 37625174379) : 213 passed, 6 skipped. Le log -q ne contient pas de noms. Par position contre un --collect-only local (219 = 219) : unsupported 14/14, drain 11/11, spool 11 passés et 3 sautés (208/210/211 = les trois tests de mode win32). oldest_pending_age est passé (position 214).

Non bloquant (issues pour le lead)

  • path.unlink() hors try (capture_drain.py:72) : trace dans le log, passage interrompu, puis télémétrie stall. L'au-moins-une-fois tient.
  • wait_for n'annule pas un blocage synchrone. La télémétrie stall en est la garde, et l'ADR le dit.
  • manifest.json : CONFLICT (content) avec fix(mcp): publish domain-or-memory_ids constraint in recall_hierarchical schema (#661) #668 (OPEN). Le second à fusionner relance check_project_wiki.py --write.
  • tests_py/hooks/ : 1 exécution sur 4 donne 1 failed, 680 passed, avec l'identité perdue. Non reproduit en 2 reruns -rf. Les trois fichiers capture passent 15 fois sur 15 sous charge.
  • test_entry.py (non touché) laisse des mcp-server orphelins avec PPID 1. J'ai arrêté les miens. PID 60737 (autre session) laissé.

Preuves

  • ruff : All checks passed!. Craftsmanship --base origin/main : OK. Doc claims : OK. Wiki : mirrors match.
  • pyright 1.1.414 : 0 errors. Avec Windows : seulement geteuid (capture_peer.py:53), déjà sur main.
  • hooks (SQLite explicite) : 681 passed, 23 skipped, sauts PG-only.
  • Suite complète (SQLite explicite, PG-only sautés) : 9026 passed, 259 skipped, exit=0.
  • PG de l'owner : 60965 puis 60971. : 6 post_tool_capture de mes commandes (hôte), aucune issue des tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LFAg4i6CQpZrNXb8cHH8ax
@cdeust

cdeust commented Oct 7, 2026

Copy link
Copy Markdown
Owner Author

ZETETIC-REVIEW: APPROVE
0c663b9

Tête fusionnée: 0c663b9 = 1aadf34 (approuvée par le contrôleur Opus, 4e passage, détail ci-dessous) + merge de origin/main 306fafe (#668). Vérifié par le lead: le diff de la PR contre sa base est identique avant et après la fusion hors wiki/manifest.json (15 fichiers, 1239 insertions, 32 suppressions, mêmes fichiers); le manifeste garde ADR-1093 (#668) et ADR-1094 (cette PR). CI: 24 vérifications réussies, 4 ignorées, Windows compris, sur 0c663b9. Non vérifié: tout le natif Windows hors ce que le job CI exécute.

PR #663 (#659), 4e passage : APPROVE

Les deux bloquants de 940a6d3 sont corrigés.

Bloquants précédents, corrigés

  1. Âge du spool. Le calcul est maintenant now - min(mtime) sur tous les fichiers en attente (capture_spool.py:110-119), et FileNotFoundError est sauté. Le test fixe cinq mtimes par os.utime (700/100/900/300/500) sans rapport avec les noms et attend 900.0. Le drainer trie seulement pour le déterminisme (capture_drain.py:60). Comme le dit ADR-1094, l'ordre n'est pas un contrat. Seul effet d'ordre : la concaténation de merge_contents (cosmétique).
  2. Mode du log. fchmod agit sur le descripteur écrit, dans open_rotating_log. Rotation réelle (umask 022, 0o644 à MAX_LOG_BYTES) : A inside: log 0o600 fd 0o600 size 0 / A after: .1 0o644 lock 0o600 / B after: log 0o600 .1 0o600 / C fresh: log 0o600 lock 0o600. Le .1 en 0o644 hérite d'un contenu déjà lisible. O_NOFOLLOW est couvert par lstat (log_rotation.py:28-36).

CI

Tout est vert, CI Green compris. Windows (run 37625174379) : 213 passed, 6 skipped. Le log -q ne contient pas de noms. Par position contre un --collect-only local (219 = 219) : unsupported 14/14, drain 11/11, spool 11 passés et 3 sautés (208/210/211 = les trois tests de mode win32). oldest_pending_age est passé (position 214).

Non bloquant (issues pour le lead)

  • path.unlink() hors try (capture_drain.py:72) : trace dans le log, passage interrompu, puis télémétrie stall. L'au-moins-une-fois tient.
  • wait_for n'annule pas un blocage synchrone. La télémétrie stall en est la garde, et l'ADR le dit.
  • manifest.json : CONFLICT (content) avec fix(mcp): publish domain-or-memory_ids constraint in recall_hierarchical schema (#661) #668 (OPEN). Le second à fusionner relance check_project_wiki.py --write.
  • tests_py/hooks/ : 1 exécution sur 4 donne 1 failed, 680 passed, avec l'identité perdue. Non reproduit en 2 reruns -rf. Les trois fichiers capture passent 15 fois sur 15 sous charge.
  • test_entry.py (non touché) laisse des mcp-server orphelins avec PPID 1. J'ai arrêté les miens. PID 60737 (autre session) laissé.

Preuves

  • ruff : All checks passed!. Craftsmanship --base origin/main : OK. Doc claims : OK. Wiki : mirrors match.
  • pyright 1.1.414 : 0 errors. Avec Windows : seulement geteuid (capture_peer.py:53), déjà sur main.
  • hooks (SQLite explicite) : 681 passed, 23 skipped, sauts PG-only.
  • Suite complète (SQLite explicite, PG-only sautés) : 9026 passed, 259 skipped, exit=0.
  • PG de l'owner : 60965 puis 60971. : 6 post_tool_capture de mes commandes (hôte), aucune issue des tests.

@cdeust
cdeust merged commit d574219 into main Oct 7, 2026
28 checks passed
@cdeust
cdeust deleted the fix/capture-windows-659 branch October 7, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Windows: auto-capture has stored nothing since 4.x: the resident capture worker refuses every platform but Linux/macOS, with no fallback

1 participant