From c36b5683ada13c2fd31d2d5c7d405462783d261c Mon Sep 17 00:00:00 2001 From: arzafran Date: Tue, 15 Sep 2026 17:19:31 -0300 Subject: [PATCH 1/2] docs: record the Rust core spike, concept inventory, and t3code review --- docs/plans/rust-core-concepts.md | 434 +++++++++++++++++++++++++ docs/plans/rust-core-spike.md | 157 +++++++++ docs/plans/t3code-inventory.md | 541 +++++++++++++++++++++++++++++++ 3 files changed, 1132 insertions(+) create mode 100644 docs/plans/rust-core-concepts.md create mode 100644 docs/plans/rust-core-spike.md create mode 100644 docs/plans/t3code-inventory.md diff --git a/docs/plans/rust-core-concepts.md b/docs/plans/rust-core-concepts.md new file mode 100644 index 00000000..70c4af7c --- /dev/null +++ b/docs/plans/rust-core-concepts.md @@ -0,0 +1,434 @@ +# Programa concept inventory + +What a replacement core (any toolkit, any terminal engine) has to reproduce. Companion to `rust-core-spike.md`. Generated 2026-09-15 from a read-only pass over the repo. + +Read-only research pass over `/Users/frz/Developer/@darkroom/programa`. Each section names the +owning source files and the external contract (CLI/socket/MCP/settings/shortcut names) so a +Rust/cross-platform core spec can enumerate what has to be reimplemented. + +## 1. Object model + +Nesting: `window` (native macOS window) -> `workspace` (sidebar entry, often called "tab" in the +UI) -> `pane` (a split region from vendor/bonsplit) -> `surface` (a tab within a pane: terminal or +browser). `panel` is the internal implementation term for what the public API calls `surface` +(`docs/agent-browser-port-spec.md:30-40`). + +- **Window**: `Sources/AppDelegate.swift`, `Sources/WindowAccessor.swift`, `Sources/MainWindowHostingView.swift`, `Sources/WindowChrome.swift`, `Sources/WindowSwizzles.swift`, `Sources/TerminalController+Window.swift`. +- **Workspace**: `Sources/Workspace.swift` (`final class Workspace: Identifiable, ObservableObject`, `let id: UUID`, `Sources/Workspace.swift:14-15`), plus `Workspace+Bonsplit.swift`, `Workspace+FocusGeometry.swift`, `Workspace+Layout.swift`, `Workspace+Persistence.swift`, `Workspace+SidebarTelemetry.swift`, `Workspace+Surfaces.swift`, `Workspace+Theme.swift`. Owned by `Sources/TabManager.swift` (`class TabManager: ObservableObject`, `@Published var tabs: [Workspace]`, `Sources/TabManager.swift:593,624`) — the manager's own vocabulary is "tabs" even though the public/UI concept is "workspace." Sidebar rendering: `Sources/VerticalTabsSidebar.swift`, `Sources/TabItemView.swift`, `Sources/WorkspaceSidebarModels.swift`. +- **Pane / split tree**: `vendor/bonsplit` (in-tree, MIT, not a git submodule — edit/commit like normal source per `CLAUDE.md`). Public types: `vendor/bonsplit/Sources/Bonsplit/Public/Types/PaneID.swift` (`struct PaneID: Hashable, Codable, Sendable` wrapping a `UUID`), `TabID.swift` (wraps a `UUID`, internal `id` — `Tab` here is bonsplit's own generic split-tree leaf, not `Workspace`), `SplitOrientation.swift`, `NavigationDirection.swift`, `LayoutSnapshot.swift`, `TabContextAction.swift`. Controller: `vendor/bonsplit/Sources/Bonsplit/Public/BonsplitController.swift`, `BonsplitConfiguration.swift`, `BonsplitDelegate.swift`, `BonsplitView.swift`. App-side glue: `Sources/TerminalController+Pane.swift`, `Sources/TabManager+Splits.swift`. +- **Surface**: `Sources/TerminalSurface.swift` wraps a Ghostty `ghostty_surface_new` C surface tied synchronously (main actor) to a specific `NSView*` (`docs/plans/detached-sessions.md:94-99`). Panel protocol (`Panel`, `@MainActor`, `ObservableObject`, `Identifiable`) with `PanelType` enum (`.terminal`, `.browser`, `.markdown`, and a planned `.review`) at `Sources/Panels/Panel.swift:7`. `Sources/Panels/TerminalPanel.swift`, `BrowserPanel.swift`, `MarkdownPanel.swift`, `ReviewPanel.swift` (see §5). +- **Sidebar**: `Sources/VerticalTabsSidebar.swift`, `Sources/ContentView+SidebarResizer.swift`, `Sources/WorkspaceSidebarModels.swift`. + +**IDs and refs.** Internally everything is a `UUID`. The external (socket/CLI/MCP) contract never +exposes raw UUIDs as the primary handle; it issues short ordinal "handle refs" of the form +`window:1`, `workspace:1`, `pane:1`, `surface:1` (`enum V2HandleKind: window, workspace, pane, +surface`, `Sources/TerminalController.swift:177-182`, ref store `V2HandleRefStore` at +`Sources/TerminalController.swift:184+`). There is also a `tab:` ref alias that is literally the +`surface:` ref with the prefix swapped (`v2TabRef`, `Sources/TerminalController.swift:2396-2400`) +— kept for the historical v1 "panel"/"tab" vocabulary in tmux-compat commands. Refs are assigned +lazily/on first issuance and are stable for the life of the running app (not persisted across +restart — a fresh launch restarts ordinals at 1). + +**Focus and selection.** `TabManager.selectedTabId: UUID?` tracks the selected workspace +(`Sources/TabManager.swift:642`). Per-workspace pane/surface focus is tracked inside `Workspace` +(`Workspace+FocusGeometry.swift`) and surfaced over the socket via `surface.focus`/`pane.focus`. +Socket/CLI focus is gated: only methods in `focusIntentV2Methods` may move macOS app focus or +raise the window (`docs/mcp-server.md:107-115`; enforced in `Sources/TerminalController.swift`, +consulted via `v2FocusAllowed()`), regardless of what a client asks for — see §3's focus policy +and §5's `focus_` MCP tool naming. + +## 2. Sessions + +Two independent mechanisms, deliberately not merged (`docs/plans/snapshot-restore.md:87-94`): + +**Layout/state snapshot** (`docs/plans/snapshot-restore.md`, `Sources/SessionPersistence.swift`, +`Sources/TabManager+SessionPersistence.swift`, `Sources/Workspace+Layout.swift`) — captures +window/workspace/pane geometry, cwd, titles, and scrollback-as-text, and replays it on next +launch into **brand-new** shell processes (not a live reconnect). Live file: +`~/Library/Application Support/programa/session-.json`. History archive (added by the +snapshot-restore feature): copied (never moved) into `session-history/-.json` +once per launch, pruned to 10 newest, deduped by byte-identical content. Records a +`cleanShutdown: Bool?` flag (`true` only from orderly-quit paths). Socket: `snapshot.list`, +`snapshot.restore`. CLI: `programa snapshot list [--json]`, `programa snapshot restore +[|latest]`. + +**Process survival / detached sessions** (`docs/plans/detached-sessions.md`, shipped 0.3.0, +2026-07-24; `Sources/SessionEscrow.swift`, `session_escrow_shim.c`/`.h`) — "escrow the dup, don't +move the custody." At PTY creation the app `dup()`s the Ghostty PTY master fd and sends it once +via `SCM_RIGHTS` to a small detached "holder" process (the same app binary launched in a hidden +mode, `SessionEscrowHolder.runIfRequested()`, spawned with `posix_spawn` + `POSIX_SPAWN_SETSID` + +`POSIX_SPAWN_CLOEXEC_DEFAULT` so it survives app death and doesn't accidentally inherit every +other surface's fd). If the app quits or crashes, the escrowed dup keeps the PTY pair alive (child +never gets SIGHUP); the holder detects app death (kqueue `EVFILT_PROC` + heartbeat) and drains +PTY output into an append-only WAL (`SessionWALStore`/`SessionWALPaths`) so the child never blocks +on a full PTY buffer. On relaunch the app retrieves the fd back over `SCM_RIGHTS` (token-gated), +replays the WAL tail through Ghostty's own VT parser, and re-applies size/`SIGWINCH` +(tmux-style idempotent attach). Ghostty-fork dependency: read-only child PID / PTY path / PTY +master-fd accessors, and surface revival through an existing fd+pid without Ghostty taking +ownership or signaling that process (`docs/ghostty-fork.md` §10, `96316fc50`/`bccfc8333`; see §8 +of this inventory). Note a caution flagged in memory: `[[scm-rights-sender-close-race]]` — never +close the sender's fd right after `sendmsg`. + +**What survives an app restart today:** with escrow, the live child process and PTY (detached +sessions). Without/before escrow completes for a given surface, only layout/cwd/scrollback text +survives (snapshot). Neither mechanism persists env vars, command line, or focus state +(`v2-api-migration.md`/`layout.save` explicitly excludes "command/env/focus, which have no live +'what's running' signal"). + +## 3. Socket API v2 (`docs/v2-api-migration.md`, `tests_v2/`) + +JSON-RPC-shaped, one JSON object per line, `{"id","method","params"}` -> `{"id","ok","result"}` / +`{"id","ok":false,"error":{"code","message"}}`. `auth.login` is a connection preamble, not a +regular method (password-mode socket access, see `automation.socketPassword` in §7). v1 +line-protocol was removed 2026-07-08; a non-JSON line now gets `v1_removed`. + +Full v2 method list by area (authoritative source: `Sources/V2CommandCatalog.swift`): + +- **System**: `system.ping`, `system.identify`, `system.capabilities`, `rpc` (raw passthrough — CLI `programa rpc`). +- **Window**: `window.list`, `window.current`, `window.focus` (focus-intent), `window.create`, `window.close`. +- **Workspace**: `workspace.list`, `workspace.create`, `workspace.select` (focus-intent), `workspace.current`, `workspace.close`, `workspace.move_to_window`, `workspace.next`/`workspace.previous`/`workspace.last` (focus-intent), `workspace.rename`. +- **Worktree** (`docs/plans/worktree-and-layouts.md`): `worktree.create`, `worktree.open` (focus-intent, opt-in `focus`), `worktree.remove`, `worktree.list`. Params/errors detailed in `docs/mcp-server.md`/`v2-api-migration.md:458-511`. +- **Layout**: `layout.save`, `layout.apply`, `layout.list`. +- **Snapshot**: `snapshot.list`, `snapshot.restore` (see §2). +- **Agent detection**: `agent.detection.list`, `agent.detection.classify` (see §5). +- **Surface / split**: `surface.list`, `surface.focus` (focus-intent), `surface.split`, `surface.create`, `surface.close`, `surface.drag_to_split`, `surface.refresh`, `surface.health`, `surface.trigger_flash`, `surface.wait` (event-driven one-shot wait — see below), `surface.read_text`, `surface.send_text`, `surface.send_key`. +- **Surface telemetry** (`surface.report_tty`, `surface.ports_kick`, `surface.report_pwd`, `surface.report_shell_state`, `surface.report_git_branch`, `surface.clear_git_branch`, `surface.report_pr`, `surface.clear_pr`, `surface.report_ports`, `surface.clear_ports`) — hot-path threading policy from `CLAUDE.md`: parse/validate/dedupe off-main, only the minimal model mutation hops to `DispatchQueue.main.async`. +- **Pane**: `pane.list`, `pane.focus` (focus-intent), `pane.surfaces`, `pane.create`, `pane.last` (focus-intent, MCP-exposed). +- **Sidebar metadata** (workspace-scoped): `workspace.set_status`/`clear_status`/`list_status`, `workspace.log`/`clear_log`/`list_log`, `workspace.set_progress`/`clear_progress`, `workspace.sidebar_state`, `workspace.clear_agent_pid`/`set_agent_pid`, `workspace.report_meta_block`/`clear_meta_block`/`list_meta_blocks`, `workspace.reset_sidebar`. +- **Notification**: `notification.create`, `notification.create_for_surface`, `notification.create_for_target`, `notification.list`, `notification.clear`. +- **App**: `app.focus_override.set`, `app.simulate_active`, `app.reload_config`, `app.browsers` (lists installed/running browsers + system default; see `Sources/Panels/BrowserAvailability.swift`). +- **Browser**: `browser.open_split` (focus-intent varies), `browser.navigate`, `browser.back`, `browser.forward`, `browser.reload`, `browser.url.get`, `browser.focus_webview` (focus-intent), `browser.is_webview_focused`, `browser.focus` (focus-intent, element-level), `browser.tab.switch` (focus-intent). Full agent-browser-shaped surface (`browser.snapshot`, `.click`, `.fill`, `.screenshot`, `.console.list`, `.tab.new/close`, etc.) documented in `docs/aside-browser.md` and `docs/agent-browser-port-spec.md`; Playwright-shaped network/viewport/raw-input methods return `not_supported` (no CDP under `WKWebView`). +- **Review** (`review.*`, docs/plans/diff-review-panel.md — see §5): `review.open` (focus-intent, opt-in), `review.refresh`, `review.comment.add`, `review.comment.remove`, `review.comment.list`, `review.send_comments`. +- **Markdown**: `markdown.open` (app-chrome, not MCP-exposed). +- **Subscriptions**: `subscribe` (`classes`: `agent_state`|`output`|`workspace_lifecycle`; `surface_ids` required for `output`), `unsubscribe`. Push frames use a bare `{"event": ...}` shape, not the request/response envelope; 256-event drop-oldest queue per subscription with a `{"event":"dropped","count"}` marker frame. +- **Debug/test-only** (`debug.*`, DEBUG builds, not MCP-exposed): `debug.shortcut.set`/`simulate`, `debug.type`, `debug.app.activate`, `debug.terminal.is_focused`/`read_text`/`render_stats`, `debug.layout`, `debug.bonsplit_underflow.count/reset`, `debug.empty_panel.count/reset`, `debug.notification.focus`, `debug.flash.count/reset`, `debug.panel_snapshot.snapshot/reset`, `debug.window.screenshot`. + +**`surface.wait`** (#166): server-owned, event-driven, one request/response wait on `pattern` +(regex vs current screen+scrollback, polled ~100ms), `exit` (child process exit, fully +event-driven off `GHOSTTY_ACTION_SHOW_CHILD_EXITED`), or `agent_state` (idle/working/blocked/ +any_change, event-driven off the single main-thread mutation point for +`Workspace.panelAgentStates`). No-missed-events guarantee via a synchronous main-thread hop +(`v2MainSync`) that checks-then-registers atomically. CLI: `programa wait-surface`. + +**`agent.prompt`** (#166): submit a prompt (`surface.send_text`'s path) + wait for the agent to go +idle, phased: send+register atomically, wait `working_grace_ms` for a `working` transition +(non-fatal if not observed), then wait the remaining `timeout_ms` for `idle`. CLI: `programa +prompt-agent`. + +**Threading/focus policy** (from root `CLAUDE.md`, cross-referenced throughout +`v2-api-migration.md`): telemetry hot-path commands must not use `DispatchQueue.main.sync`; only +`focusIntentV2Methods` may mutate in-app focus/window activation; everything else must preserve +the user's current focus while still applying data/model mutations. + +## 4. CLI and MCP server + +**CLI** (`CLI/programa.swift`, dispatcher `CLI/CLICommandDispatcher.swift`, custom argument +parser — not Swift ArgumentParser). Socket path resolution: `PROGRAMA_SOCKET_PATH` env var, else a +DEBUG-only hint file at `/tmp/programa-last-socket-path`, else `/tmp/programa-debug.sock` +(DEBUG) / `/tmp/programa.sock` (release). Full top-level command name list, in the order declared +in `commandDescriptors()` (`CLI/programa.swift`, one `CommandDescriptor` per name group): +`welcome`, `shortcuts`, `feedback`, `themes`, `claude-teams`, `omo`, `omx`, `omc`, `codex`, +`claude`, `opencode`, `aside`, `ping`, `version`, `capabilities`, `rpc`, `identify`, +`list-windows`, `current-window`, `new-window`, `focus-window`, `close-window`, +`move-workspace-to-window`, `reorder-workspace`, `workspace-action`, `worktree`, +`agent-detection`, `race`, `layout`, `snapshot`, `list-workspaces`, `new-workspace`, `new-split`, +`list-panes`, `list-pane-surfaces`, `focus-pane`, `new-pane`, `new-surface`, `close-surface`, +`move-surface`, `reorder-surface`, `tab-action`, `rename-tab`, `drag-surface-to-split`, +`refresh-surfaces`, `reload-config`, `surface-health`, `debug-terminals`, `trigger-flash`, +`list-panels`, `focus-panel`, `close-workspace`, `select-workspace`, `rename-workspace` (alias +`rename-window`), `current-workspace`, `read-screen`, `wait-surface`, `prompt-agent`, +`watch-events`, `send`, `send-key`, `send-panel`, `send-key-panel`, `notify`, +`list-notifications`, `clear-notifications`, `set-status`, `clear-status`, `list-status`, +`set-progress`, `clear-progress`, `log`, `clear-log`, `list-log`, `sidebar-state`, +`set-app-focus`, `simulate-app-active`, `__tmux-compat` (family of tmux-compatibility commands, +`CLI/CLI+TmuxCompat.swift`), `markdown`, `review`, `recap`, `browser` (with legacy flat aliases +`open-browser`, `navigate`, `browser-back`, `browser-forward`, `browser-reload`, `get-url`, +`focus-webview`, `is-webview-focused`), `help`. Command families with their own subcommand sets: +`CLI/CLI+Aside.swift`, `CLI/CLI+Browser.swift` (49-verb agent-browser-shaped surface — see +`docs/agent-browser-port-spec.md`), `CLI/CLI+Review.swift`, `CLI/CLI+Markdown.swift`, +`CLI/CLI+Recap.swift`, `CLI/CLI+Themes.swift`, `CLI/CLI+AgentWrappers.swift` (`claude`, `codex`, +`opencode` install/uninstall-integration), `CLI/CLI+Hooks.swift`/`CLI/CLI+HookCommands.swift`. +Notifications CLI subset is also documented standalone in `docs/notifications.md`: `programa +notify --title [--subtitle][--body][--tab][--panel]`, `list-notifications`, +`clear-notifications`, `set-status `, `clear-status `, `ping`. + +**MCP server** (`docs/mcp-server.md`, binary `programa-mcp`, `CLI-MCP/programa-mcp.swift`, +`CLI-MCP/MCPServer+Capabilities.swift`, `CLI-MCP/MCPSocketBridge.swift`, +`CLI-MCP/ToolCatalog.swift`, `CLI-MCP/ResourceCatalog.swift`, `CLI-MCP/MCPErrorMapping.swift`). +Separate binary embedded at `Contents/Resources/bin/programa-mcp` inside the app bundle; talks to +the running app over the same v2 socket. 187 tools, one per exposed socket method with `.` -> +`_` (`surface.read_text` -> `surface_read_text`, `review.comment.add` -> `review_comment_add`). +Thirteen tools carry an explicit `focus_` prefix because they are allowed to move macOS focus: +`focus_window`, `focus_workspace_select`, `focus_workspace_next`, `focus_workspace_previous`, +`focus_workspace_last`, `focus_surface`, `focus_pane`, `focus_pane_last`, `focus_review_open`, +`focus_worktree_open`, `focus_browser_webview`, `focus_browser_element`, +`focus_browser_tab_switch` (`worktree_create` is a deliberate exception: it drops the underlying +method's `focus` param entirely). Not exposed: `debug.*` (DEBUG-only, UI-test hooks) and app-chrome +methods (`auth.login`, `settings.open`, `feedback.*`, `markdown.open`, `app.*`). Resources: +`programa://tree` (full window/workspace/pane/surface tree) and +`programa://surface/{surface_id}/text` (optionally `?lines=n`, capped at 10,000 lines). Auth: set +`PROGRAMA_SOCKET_PATH` to target a specific instance, `PROGRAMA_SOCKET_PASSWORD` if +password-mode socket access is enabled (`automation.socketPassword`). + +## 5. Agent integration + +**Agent detection** (`docs/agent-detection-manifests.md`, `docs/plans/screen-manifest-detection.md`, +`Sources/AgentManifest.swift`, `Sources/AgentManifestLoader.swift`, +`Sources/AgentScreenDetectionEngine.swift`, `Sources/AgentActivityState.swift`, +`Sources/AgentSupervision.swift`, `Sources/AgentRPCDispatcher.swift`). Two tiers: (1) lifecycle +hooks installed for agents that support them (Claude Code, Codex, OpenCode — always wins over +inference), (2) a screen-manifest fallback that regex-matches the visible terminal screen against +a declarative JSON manifest per agent to infer `working`/`blocked`/`idle`/`done` (mapped to the +3-value wire enum, `done` folds into `idle`). Bundled manifests: +`Resources/AgentDetection/.json` for the seven ids `claude-code`, `codex`, `gemini-cli`, +`opencode`, `copilot-cli`, `cursor-agent`, `aider`. User overrides (full replace, no merge): +`~/.config/programa/agent-detection/.json`. Schema v1: `agent`, `display_name`, +`recognize.process_names`/`screen_patterns`, `states[].bucket/priority/anchor_last_n_lines/ +patterns/confidence/source_notes`. CLI: `programa agent-detection list|scaffold|test`. Socket: +`agent.detection.list`, `agent.detection.classify`. + +**Notifications** (`docs/notifications.md`, `Sources/NotificationsPage.swift`, +`Sources/TerminalNotificationStore.swift`, `Sources/AgentOverviewWindow.swift`). Notification +panel + macOS system notifications. Env vars set in every child shell: +`PROGRAMA_SOCKET_PATH`, `PROGRAMA_TAB_ID`, `PROGRAMA_PANEL_ID`, `PROGRAMA_DEFAULT_BROWSER`, +`PROGRAMA_DEFAULT_BROWSER_BUNDLE_ID`. Settings: `notifications.showInMenuBar`, `.sound`, +`.command`, `.longCommandThresholdSeconds` (see §7). Integration recipes documented for Claude +Code hooks, GitHub Copilot CLI hooks (`~/.copilot/config.json` or `.github/hooks/notify.json`), +OpenAI Codex (`~/.codex/config.toml` `notify` array), and an OpenCode plugin +(`.opencode/plugins/programa-notify.js`). + +**Attention/status/progress metadata**: workspace-scoped sidebar metadata methods (`workspace. +set_status/log/set_progress/sidebar_state` family, §3) mutate a `Tab`/`Workspace` (not a specific +surface). `Workspace.panelAgentStates: [UUID: AgentActivityState]` +(`Workspace.swift:111`) is the single source of truth for per-surface agent activity, mutated only +via `Workspace+SidebarTelemetry.swift`'s `updatePanelAgentState`/`clearPanelAgentState` +(`:163`/`:180`), which fan out to `AgentStateWaitRegistry` (backs `surface.wait`/`agent.prompt`) +and `SocketEventBroadcaster` (backs `subscribe`). + +**Provider usage display**: `Sources/ClaudeQuotaMonitor.swift` reads +`~/.claude/tmp/rate-limits.json` to show Claude Code 5h/7d rate-limit headroom in the sidebar +footer; gated by `sidebarAppearance.showClaudeQuota` (§7). + +**Diff review panel** (`docs/plans/diff-review-panel.md`, status "proposed" per the plan doc +but the `review.*` socket family and `Sources/Panels/ReviewPanel.swift` / +`ReviewPanelView.swift` / `Sources/ReviewComment.swift` / `ReviewCommentSerializer.swift` / +`ReviewDiffParser.swift` / `ReviewDiffProber.swift` exist in-tree — cross-check `PanelType` for +current `.review` case status before assuming it's fully live). Shows a terminal surface's +worktree git diff (`"uncommitted"` vs `HEAD`, or `"branch"` vs merge-base) beside the pane, with +line comments serialized as `path:start-end — text` and sent back into the source terminal's +input. Read-only w.r.t. git — never mutates worktree/index/branches. CLI: `programa review +open|refresh|comment|send`. Socket family listed in §3. + +## 6. Browser panel + +**Two distinct browser surfaces** (`docs/aside-browser.md`): Programa's embedded panel +(`WKWebView`-based, per-workspace profile, socket-driven `browser.*`) for local +previews/smoke-tests/DOM inspection that an agent reads back without leaving the pane; and Aside +(`aside.com`, Chromium-based, external app) for logged-in/private-session work, registered as an +MCP server for Claude Code/Codex via `programa aside install-mcp`. + +Embedded browser source: `Sources/Panels/BrowserPanel.swift` + companions +(`+Automation.swift`, `+DeveloperTools.swift`, `+Focus.swift`, `+Navigation.swift`, `+Theme.swift`, +`+WorkspaceLifecycle.swift`), `BrowserPanelSupport.swift`, `BrowserPanelView.swift`, +`BrowserPanelWebDelegates.swift`, `BrowserAvailability.swift` (backs `app.browsers`), +`BrowserHistoryStore.swift`, `BrowserProfileStore.swift`, `BrowserSettings.swift`, +`BrowserToolbarViews.swift`, `BrowserUserProxySettings.swift`, `BrowserWebDialogPresenter.swift`, +`Omnibar.swift`/`OmnibarSuggestionsView.swift`/`OmnibarTextField.swift`, `InspectorDock.swift`, +`DesignMode.swift`. Depends on WebKit (`WKWebView`) — there is no Chrome DevTools Protocol +underneath, so Playwright-shaped tools (`browser_viewport_set`, `browser_network_route`, +`browser_input_mouse`, etc.) return `not_supported` deliberately rather than failing as unknown +tools (`docs/mcp-server.md:135-141`). `docs/agent-browser-port-spec.md` is a historical porting-gap +tracker against `vercel-labs/agent-browser`'s CLI/protocol surface (its "keep v1 working" framing +is stale — see the doc's own 2026-07-08 historical note — but its "Concepts (Canonical Terms)" +section §30-40 is the accurate current terminology, and its counted command/flag/protocol-action +inventory is useful as an upper bound on what a full port would need). + +## 7. Configuration + +**`settings.json`** (`docs/settings-json.md`, `~/.config/programa/settings.json`, schema +`Resources/settings.schema.json`, JSONC with `//` comments, reloads on file change, a +file-set key wins over the Settings UI until removed). Top-level sections and every key: +- `app`: `appearance`, `terminalTheme`, `terminalOpacity`, `terminalBlur`, `terminalFont`, + `newWorkspacePlacement`, `minimalMode`, `preferredEditor`, `reorderOnNotification`, + `warnBeforeQuit`, `commandPaletteSearchesAllSurfaces`. +- `notifications`: `showInMenuBar`, `sound`, `command`, `longCommandThresholdSeconds`. +- `workspaceColors`: `indicatorStyle`, `selectionColor`, `notificationBadgeColor`, `colors`. +- `sidebarAppearance`: `matchTerminalBackground`, `tintColor`, `lightModeTintColor`, + `darkModeTintColor`, `tintOpacity`, `showClaudeQuota`. +- `automation`: `socketControlMode` (`off|cmuxOnly|automation|password|allowAll|openAccess| + fullOpenAccess|notifications|full`), `socketPassword`, `claudeCodeIntegration`, + `openBrowserWithAgentSplits`, `claudeBinaryPath`, `portBase`, `portRange`. +- `customCommands`: `trustedDirectories`. +- `browser`: `defaultSearchEngine`, `showSearchSuggestions`, `theme`, + `openTerminalLinksInProgramaBrowser`, `interceptTerminalOpenCommandInProgramaBrowser`, + `hostsToOpenInEmbeddedBrowser`, `urlsToAlwaysOpenExternally`, `externalBrowser`, + `insecureHttpHostsAllowedInEmbeddedBrowser`, `proxy`. +- `worktrees`: `directory` (default `~/.programa/worktrees`). +- `shortcuts`: `showModifierHoldHints`, `bindings` (keyed by Programa action id; string or array + for a chord). Special-cased action id in the doc: `openAgentOverview` (unbound by default). +Swift source of truth: `Sources/ProgramaSettingsFileStore.swift`, +`Sources/KeyboardShortcutSettings.swift`. + +**`programa.json`** (`docs/programa-json.md`, `Sources/ProgramaConfig.swift`, +`Sources/ProgramaConfigExecutor.swift`). Command-palette entries, walked up from the focused +workspace's cwd plus a global `~/.config/programa/programa.json` fallback (legacy names +`cmux.json`/`~/.config/cmux/cmux.json` still read). Two entry kinds: `commands` (either a +`workspace` command — opens/recreates a named workspace with a saved layout — or a `command` +command — types+submits shell text) and `recipes` (fills a prompt template and types it into the +focused terminal **without** auto-submitting, so a cloned repo can't fire attacker-chosen text +straight at an agent). Untrusted by default: every command/recipe is confirmed until the user +trusts the source directory (`customCommands.trustedDirectories`). Parameter substitution: +`{{name}}` placeholders, unresolved ones left literal, values not shell-quoted (the confirmation +dialog is the control, not escaping). + +**Keyboard shortcuts** (`docs/keyboard-shortcuts.md`, `Sources/KeyboardShortcutSettings.swift`, +`Sources/WorkspaceShortcutMapper.swift`). Every shortcut editable in Settings and via +`shortcuts.bindings`. Full default table spans workspaces, surfaces, split panes, browser, +notifications, find, terminal, window, and review; several actions ship intentionally unbound +(`openAgentOverview`, review-panel-open, git-worktree/layout commands — CLI/palette only "by +design, not an oversight"). + +**Terminal themes** (`docs/terminal-themes.md`). Reads Ghostty's bundled theme catalog plus user +theme directories (`GHOSTTY_RESOURCES_DIR`, `XDG_DATA_DIRS`); ships two Programa-specific themes, +**Min Light** and **Min Dark** (Min Theme VS Code extension style). CLI: `programa themes +list|set|clear`. Managed block written into `~/Library/Application Support/ +com.darkroom.programa/config.ghostty`, field-by-field (never captures a field it doesn't +explicitly manage). Mirrors the four `app.terminalTheme/terminalOpacity/terminalBlur/ +terminalFont` settings.json keys. + +**Localization**: `Resources/Localizable.xcstrings`, source language `en`, translated languages +present: `en`, `ja` (English and Japanese — matches `CLAUDE.md`'s "currently English and +Japanese"). Every user-facing string must use `String(localized:defaultValue:)` per project +convention — no bare literals. + +## 8. Ghostty fork delta (`docs/ghostty-fork.md`) + +Fork head as of this inventory: `bccfc8333` on Darkroom Engineering's `ghostty` fork `main`, +reconciled with upstream `ghostty-org/ghostty` `main` at `c8634f3fce1` (2026-08-21), Zig 0.16.0. +Every item below is something a non-Ghostty (e.g. a Rust-native terminal-emulation) core would +have to reimplement or find an equivalent for: + +1. **macOS display-link restart on display change** (`src/renderer/generic.zig`) — prevents a + stuck-vsync state after a `CGDisplay` ID change. +2. **Resize stale-frame mitigation** (`pkg/macos/animation.zig`, `src/Surface.zig`, + `src/apprt/embedded.zig`, `src/renderer/Metal.zig`, `src/renderer/generic.zig`, + `src/renderer/metal/IOSurfaceLayer.zig`) — replays the last frame with correct anchoring during + a live resize to avoid transient blank/scaled frames. +3. **OSC 99 (kitty) notification parser** (`src/terminal/osc.zig`, + `src/terminal/osc/parsers/kitty_notification.zig`). +4. **Programa theme-picker helper hooks** (`build.zig`, `src/cli/list_themes.zig`, + `src/main_ghostty.zig`) — a `zig build cli-helper` step and env-var-driven live-preview mode + for `+list-themes` that writes Programa's managed theme override and posts a reload + notification. +5. **DECRPM mode 2031 color-scheme reporting fixes** (`src/Surface.zig`, + `src/termio/stream_handler.zig`). +6. **Re-exported selection C API** (`include/ghostty.h`, `src/Surface.zig`, + `src/apprt/embedded.zig`) — `ghostty_surface_select_cursor_cell`, + `ghostty_surface_clear_selection`, restored after upstream removed them; backs keyboard copy + mode. +7. **`macos-background-from-layer` config flag** (`src/config/Config.zig`, + `src/renderer/generic.zig`) — lets the host app supply terminal background via + `CALayer.backgroundColor` instead of a Metal fill, avoiding alpha double-stacking during + resizes. +8. **Occluded-surface frame-generation throttle** (`src/renderer/Thread.zig`, current shape + `08bac45e9`) — `updateFrame` runs unthrottled while visible, at most once per 250ms while + occluded, instead of a hard skip (a prior hard-skip attempt, `c25020f99`, deadlocked + `ghostty_surface_read_text` on CI's permanently-occluded virtual display because + `scrollbar_dirty` is only cleared inside `drawFrame`, which is also gated off while invisible). +9. **Offscreen renderer-realization API** (`include/ghostty.h`, `src/apprt/embedded.zig`, + `src/renderer/Thread.zig`, `src/renderer/message.zig`) — `ghostty_surface_set_renderer_realized` + lets the embedder release an occluded surface's Metal swap chain/IOSurfaces while keeping + PTY/terminal state/scrollback alive, non-blocking mailbox push with an enqueue-result return. +10. **Session introspection and revival APIs** (`include/ghostty.h`, `src/Surface.zig`, + `src/apprt/embedded.zig`, `src/termio/Exec.zig`) — read-only child PID / PTY path / PTY + master-fd accessors, and surface revival through an existing fd+pid without Ghostty taking + ownership/signaling — this is what §2's escrow-based detached sessions depend on. Also: the + `ghostty_surface_set_pty_tee_cb` callback (runs pre-VT-parse) backs the session WAL and + superseded an older Programa-only output-tap export (intentionally not restored). + +Also upstreamed (no longer fork-only): cursor-click-to-move honoring OSC 133. Dropped as +superseded: several zsh prompt-redraw patches (upstream's newer prompt-marking made them +redundant after the 2026-03-30 rebase); an older initial-focus-seeding/DECSET 1004 patch +(replaced by post-create focus synchronization, which the current fork preserves as items 6/10's +neighbor behavior — surfaces start Ghostty-default-focused, host focus callback reports real +transitions, enabling DECSET 1004 immediately reports current state). + +## 9. Rendering/perf design decisions worth keeping + +- **Renderer realization + idle reclaim**: item 9 above — release Metal swap chain/IOSurfaces for + occluded surfaces while keeping PTY/terminal state alive; `Sources/RendererRealization.swift` + is the app-side driver. +- **Occluded-render throttle**: item 8 above, 250ms/4Hz cap instead of a hard skip, because a hard + skip starves state machines (like the scrollbar dirty/clear split) that live partly inside the + throttled call. +- **Portal layering contract** (`CLAUDE.md` pitfalls): `SurfaceSearchOverlay` must mount from + `GhosttySurfaceScrollView` (`Sources/GhosttyTerminalView.swift`, the AppKit portal layer), not + from SwiftUI panel containers (`Sources/Panels/TerminalPanelView.swift`) — portal-hosted + terminal views can sit above SwiftUI during split/workspace churn. Portal registry: + `Sources/HostedViewPortalRegistry.swift`, `Sources/TerminalWindowPortal.swift`, + `Sources/TerminalWindowPortalRegistry.swift`, `Sources/WindowPaneChromePortal.swift`, + `vendor/bonsplit/Sources/Bonsplit/Public/BonsplitPaneChromePortalBridge.swift`. +- **Typing-latency-sensitive paths** (`CLAUDE.md`, must-not-regress list): no app-level display + link or manual `ghostty_surface_draw` loop (rely on Ghostty's own wakeup/renderer); + `WindowTerminalHostView.hitTest()` (`Sources/WindowTerminalHostView.swift`) takes a keyboard-event + fast path that must stay free of added work; `NSWindow.programa_sendEvent` + (`Sources/WindowSwizzles.swift`) computes its cached hit-view context only for pointer-down + events; `TabItemView` (`Sources/ContentView.swift`) relies on `Equatable` + `.equatable()` to + skip SwiftUI body re-evaluation during typing — no new `@EnvironmentObject`/`@ObservedObject`/ + `@Binding` without updating `==`; `TerminalSurface.forceRefresh()` + (`Sources/GhosttyTerminalView.swift`) runs on every keystroke and must stay allocation/IO-free. +- **Socket command threading policy** (`CLAUDE.md`): telemetry hot-path commands + (`surface.report_*`, `surface.ports_kick`, status/progress/log metadata) must not use + `DispatchQueue.main.sync`; parse/validate/dedupe off-main, minimal main-thread mutation only. + +## 10. Release/update + +Single auto-ship lane (`CLAUDE.md`, `.github/workflows/release.yml`): every commit on `main` that +passes `CI` is built, signed, notarized, and published as the latest GitHub release, triggered by +`workflow_run` on `CI` success. No nightly/beta channel — fix-forward on `main`. Auto-ship builds +get a monotonic build number from the CI run ID plus a version string with the patch component +replaced by the run number (e.g. `0.4.213`), injected into `Info.plist` at build time (never +committed). Published to a single reused `rolling` GitHub release (title = effective version, +marked latest, overwritten each ship). Each ship also promotes one sealed +`rolling-candidate-` prerelease into that ship's permanent `vX.Y.Z`-independent archive +tag, then prunes older promoted candidates to the two newest (rollback window). Milestone +major/minor bumps remain manual (`scripts/bump-version.sh`), optionally tagged `vX.Y.Z` (also +built by the same workflow on tag push). Diagnostics log: +`~/Library/Logs/Programa/diagnostics.log`, always-on since a prior PR (per team memory +`[[release-diagnostics-log]]` — ask for it first on release bug reports); `programa-update.log` +for update-flow-specific bugs. (Sparkle is the updater: `Sources/Update/UpdateController.swift` drives `SPUUpdater` with a custom delegate and UI in `Sources/Update/`; the feed points at the GitHub `rolling` release described in CLAUDE.md.) + +## 11. Things removed on purpose (`docs/removed/*.md`) + +Reductive pass of 2026-09-02, base commit `903027ccef`. Every entry names the commit to restore +from (`git checkout 903027ccef -- `) and a "what we learned" section (not reproduced here +— read the individual file before re-adding). + +- **`applescript.md`** — AppleScript support (`Sources/AppleScriptSupport.swift`, + `Resources/programa.sdef`). +- **`browser-data-import.md`** — browser data import wizard. +- **`browser-developer-tools.md`** — **not actually removed**; scoped for the same pass but the + implementer stopped and reported back instead of guessing. The hosted inspector dock is still + live (`Sources/Panels/InspectorDock.swift`). +- **`browser-extensions.md`** — browser extension support + (`BrowserExtensionManager.swift`, `BrowserExtensionAdapters.swift`). +- **`browser-react-grab.md`** — React Grab (`Sources/Panels/ReactGrab.swift`). +- **`custom-notification-sounds.md`** — custom notification sound files. +- **`inline-vscode.md`** — inline VS Code / `serve-web` integration + (`Sources/VSCodeIntegration.swift`). +- **`mobile-bridge-and-ios.md`** — Mobile Bridge and an iOS companion app + (`Sources/MobileBridge`, `ios/`, `vendor/CmuxIrohTransport`, `vendor/CMUXMobileCore`, iOS + TestFlight CI workflows). +- **`ssh-remote-workspaces.md`** — SSH remote workspaces: the largest removal, a full remote + daemon/session/proxy stack (`Sources/Workspace+Remote.swift` and ~15 sibling files, + `CLI/CLI+SSH.swift`, a `daemon/` directory, `docs/remote-daemon-spec.md`, ~15 `tests_v2/ + test_ssh_remote_*.py` files). Notable because `docs/plans/detached-sessions.md` explicitly + models its local escrow design on this removed feature's `session.*` naming/resize semantics — + the removed remote daemon is still a live design reference even though its code is gone. + +Core kept per the removal pass's own summary (`docs/removed/README.md`): the Ghostty terminal, +workspaces and splits, the sidebar, agent status detection and hooks, notifications, the browser +panel and its automation API, the diff review panel, worktrees and race, layouts, the markdown +recap panel, the CLI, socket API, and MCP server, updates, session persistence and escrow, the +Claude quota footer, and the local tmux-compat CLI — i.e. everything covered in §1-§10 above is +the deliberately-retained surface a cross-platform core spec should target. diff --git a/docs/plans/rust-core-spike.md b/docs/plans/rust-core-spike.md new file mode 100644 index 00000000..632e4ed4 --- /dev/null +++ b/docs/plans/rust-core-spike.md @@ -0,0 +1,157 @@ +# Rust core spike: decide with numbers, not a rewrite + +Status: spike complete, decision recorded (2026-09-15) + +## Question + +Two questions came up together and have different answers: + +1. Programa feels slower than it should. Where does the time go, and how much of it is the Swift/SwiftUI shell versus the Ghostty core? +2. A Windows version is wanted. Ghostty (libghostty) has no Windows support and no roadmap, so any Windows build needs a different terminal core regardless of UI toolkit. + +The candidate for both is a Rust core on gpui-kit (Longbridge's framework on Zed's GPUI, Apache-2.0, macOS/Windows/Linux, no terminal widget) plus alacritty_terminal (the VT engine Zed's terminal uses). + +## Decision rule + +The Rust core becomes the long-term base only if the spike beats the current app on the same Mac on all of: + +See "Spike results" for the filled table. + +If the spike loses on latency or memory, Windows waits and the macOS app keeps its narrow perf fixes. If it wins, GPUI is the Windows and Linux client toolkit. Either way the macOS app is not rewritten; see "Reframe" below for why the client toolkit is now the smaller question. + +## Workstreams (parallel) + +1. Perf attribution and easy fixes on the Swift app. Branch and results: see "Perf findings". +2. Spike at `~/Developer/@darkroom/programa-spike`: tabs, splits, PTY, measurements. +3. Concept inventory: what a replacement core has to reproduce, see "Concepts a new core must carry". + +## Perf findings + +Measured 2026-09-15 on a tagged Debug build (`perf-attrib`) with `sample`, `ps -M`, and the CPU occlusion harness. Debug numbers overstate production. + +| Scenario | Result | Attribution | +|---|---|---| +| Idle, 3 panes | app 0.95 % CPU, total 2.15 % | over 99.9 % of samples in kernel wait; 12 of about 275k samples in SwiftUI AttributeGraph; no Programa or Ghostty frames above noise | +| 4 hidden busy panes, 20 s | app 6.9 % CPU, total 20.5 % | consistent with the occluded-render throttle; remaining Swift cost is ANSI parsing and string splitting, no single hotspot | +| Churn, 20 cycles new workspace / split / split / close | dominated by `posix_spawn`, `stat`, `open`, `rename` | per-workspace git metadata and port-scan probes, spread across `GitMetadataProber`, `PortScanner`, `TerminalThemeStore`, `GhosttyConfig`; no single Programa function above noise | +| Keystroke path, 312 real key events via System Events | `app.sendEvent` elapsed p50 0.45 ms, p95 1.11 ms; prelude 0.10, shortcut check 0.26, dispatch 0.71 ms average | no main-thread turn crossed the 3 ms logging threshold; sample during the burst was over 99.9 % kernel wait. The 24 to 60 ms event-to-log delay is System Events queueing a burst far faster than a human, not app cost. Note: socket `send-key` bypasses NSEvent (`Sources/TerminalSurface.swift:2505`), so the CLI cannot measure this path | + +Conclusions so far: + +- The shell is not burning CPU at idle or with hidden output. The "slow" feeling, if it is real, has to be on the keystroke path or in churn, both of which are latency, not throughput. +- The one recurring main-thread cost at idle is session autosave (every ~8 s, 1 to 5 ms), already debounced and fingerprint-skipped. +- Git and port probes on workspace creation are already scheduled off-main with a generation token and cancelled on close (`Sources/TabManager.swift:1378-1439`, `Sources/PortScanner.swift:111-176`). The churn cost is one `git branch` and `git status` per new workspace, by design. +- A `new-workspace` CLI call timed out once right after a relaunch and did not reproduce in five tries. Cold-start artifact. + +No fixes were applied in either pass because no hotspot existed. Conclusion: the shell is not the source of a perf problem on any measured path. If the app still feels slow, the next measurements are frame pacing during resize and split drags (the resize stale-frame mitigation lives in the Ghostty fork) and first-window startup time, neither of which was in scope here. + +## Spike results + +Repo: `~/Developer/@darkroom/programa-spike` (5 commits, `cargo build --release` clean). Pinned: `gpui-kit 0.6.1` (which pulls `gpui-pre 0.3.5`, Longbridge's private republish of a Zed gpui snapshot; the `gpui` crate on crates.io is unrelated), `alacritty_terminal 0.26.0`. + +Works: PTY-backed shells, canvas grid render with 256 and truecolor, cursor, keyboard including Ctrl and Alt-as-Meta, wheel scrollback, tabs with Cmd+T/W/1-9, recursively nested splits with Cmd+D and Cmd+Shift+D, click focus, reflow on resize. Screenshot in the spike repo at `docs/screenshot.png`. Cut: drag-resize of splits, dock panels, terminfo, IME, native menu, accessibility, any socket API. + +| Metric | Programa (tagged Debug, today) | Spike (release) | Caveat | +|---|---|---|---| +| Keystroke cost | `sendEvent` elapsed p95 1.11 ms, real NSEvents | PTY write to wakeup p95 0.27 ms | not the same path: the spike number excludes OS key delivery and the compositor frame; both say the layer measured is not the bottleneck | +| Memory, 4 tabs idle | not measured today; production known to hold 3 IOSurface frames (about 23 MB each at 2x) per realized renderer | 63 MB footprint, 82 MB RSS; 84 MB with 8 PTYs | | +| CPU, 4 hidden busy panes, 20 s | app 6.9 % (Debug) | 1.25 % | the spike never repaints an inactive tab, Programa throttles to about 4 Hz; not apples to apples | +| Idle CPU, 20 s | 0.95 % | 0.85 % | equal within noise | +| Startup to first frame | not measured | 171 to 206 ms warm, 437 ms cold | | +| Binary | | 8.2 MB stripped | | +| Windows | n/a | dependency graph resolves for `x86_64-pc-windows-msvc` (505 crates); cross-compile from macOS stops at gpui's build script needing `llvm-rc`; a native Windows runner with MSVC should pass but this is inferred, not verified | alacritty_terminal ships ConPTY | + +Rough edges recorded by the spike: published crates strip examples, so real signatures had to be read from the cargo registry source; documentation summaries were wrong often enough to not compile from. One real bug: the PTY event channel is multi-consumer, so a second consumer starves the first. Rule for the real build: one task per PTY event channel. + +Effort estimate from the spike author, labelled est.: tab and split polish 1 to 2 weeks, a socket and CLI API 3 to 4 weeks, terminal-protocol tail (terminfo, IME, ligatures, images, accessibility) 2 to 3 months. The rendering core is not the long pole; the control API and protocol completeness are. + +## Decision + +- **No rewrite of the macOS app.** Every measured path in the Swift shell is clean; there is no perf case for it, and the concept inventory shows what a rewrite would have to reproduce. +- **GPUI is viable as the Windows and Linux client toolkit.** Rendering, memory, and startup are fine, the dependency graph resolves for Windows, and the cut features are all tractable. The unverified item is a real Windows CI build; the first action on that track is a `windows-latest` GitHub Actions job on the spike repo. +- **The VT engine for non-macOS clients should be `libghostty-vt`, not alacritty_terminal**, so every client shares the fork's parser. Swap it in the spike before building further. +- **The core-plus-clients shape is the plan.** Steps 1 and 2 (contract schema, one seam in the app) start now and are worth doing regardless. Step 3 (Rust core) is sized after those land. Org mode waits for a product decision. + +## Concepts a new core must carry + +Full inventory with every command, tool, key, and file path: `rust-core-concepts.md`. The parts that decide effort: + +- **Object model**: window > workspace > pane (bonsplit split tree) > surface (terminal or browser). Public handles are ordinal refs (`workspace:1`, `surface:1`); internally everything is UUIDs and the names are inconsistent (`TabManager.tabs` holds workspaces, bonsplit's `Tab` is a split leaf, the API's `surface` is the internal `panel`). A new core should fix the naming once. +- **Sessions are two mechanisms, deliberately separate**: a layout plus scrollback-text snapshot that replays into new processes, and process-survival escrow that hands PTY fds to a detached holder over `SCM_RIGHTS` and revives them through the VT parser. The escrow half depends on fork-only Ghostty APIs (fork item 10) and has no Windows equivalent as designed; ConPTY has no fd to hand off. +- **Socket API v2, CLI, MCP**: about 36 socket commands across window/workspace/pane/surface/browser/notification areas, the dev CLI mirrors them, and the MCP server exposes a subset as tools. This is the contract agents and tests depend on; it is toolkit independent and should be the first thing a new core implements, so `tests_v2/` can run against it unchanged. +- **Agent integration**: agent detection manifests, OSC 99 notifications, attention/status/progress metadata, provider usage display, diff review panel. All shell-side, all portable. +- **Browser panel**: WebKit only. No GPUI equivalent today. +- **Configuration**: `~/.config/programa/settings.json`, `programa.json`, shortcut registry, terminal themes, English and Japanese localization. Portable as data. +- **Ghostty fork delta, 10 patches**: three are load-bearing for perf (resize stale-frame mitigation, occluded-render throttle at 250 ms, offscreen renderer realization that releases the swap chain while keeping PTY and scrollback), one for detached sessions (revival API plus the pre-parse PTY tee callback that feeds the session WAL), and the rest are OSC/DECRPM/selection C-API details. Each needs an alacritty_terminal counterpart; alacritty_terminal has no renderer, so the swap-chain items become the spike's own rendering code. +- **Release**: auto-ship on green CI to a rolling GitHub release, Sparkle in-app updates, always-on diagnostics log. A Windows build needs its own signing, notarization equivalent, and updater. + +## Reframe: a core that runs anywhere, and clients that only draw + +Added 2026-09-15 after the question "should every Programa terminal be a dumb terminal running elsewhere, with an org mode and a personal mode?" + +### What this is not + +It is not the SSH remote workspaces feature removed on 2026-09-02 (`docs/removed/ssh-remote-workspaces.md`). That design bolted a remote branch onto a local app: 17,000 lines of Swift plus an 8,600-line Go daemon, four hand-written socket clients that drifted apart, and remote conditionals threaded through workspaces, persistence, sidebar, browser, and drag-and-drop. The removal note's own conclusion is the design rule here: make the RPC contract the product boundary, generate every client from it, and keep remote behind one seam instead of branching the model. + +### The shape + +Two processes, one contract. + +- **Core** (`programad`, one per user, local or hosted): owns PTYs and child processes, the session WAL and snapshots, escrow and revival, agent detection and attention state, git and port probes, settings, and the socket API v2 plus the MCP server. It has no UI. It runs on the developer's Mac, on a Linux box they own, or on org infrastructure. +- **Client** (the macOS app today, a GPUI app for Windows and Linux later, a web or phone client if wanted): draws terminals, tabs, splits, sidebar, and the browser panel, and routes input. It holds no session state that the core does not also hold, so any client can attach to any core and get the same workspaces. + +The terminal stays "dumb" in the ssh sense: the client keeps the VT state (Ghostty on macOS), the core keeps the PTY. Bytes flow raw in both directions. Reattach replays the WAL tail through the client's own parser, which is what detached sessions already do. + +### Two attach modes, one client + +| | Personal | Organization | +|---|---|---| +| Where the core runs | localhost, spawned by the app; or a machine you own over SSH | org-hosted, one core per user or per workspace | +| Identity | none needed | org login (OIDC), device pairing | +| Transport | Unix socket; PTY fds handed to the client over `SCM_RIGHTS`, so the local keystroke path has no extra hop | mTLS or WebSocket, bytes over the network | +| "Brain" | the user's own settings, skills, MCP servers, provider accounts | shared skills and memory, org MCP servers, managed provider credentials, permission policies, audit | +| Windows and Linux clients | need a local core built for that OS (ConPTY, no fd handoff) | work immediately, the PTYs live on Linux | + +The org column is a product and business decision (hosting cost, who pays, what "org brain" contains). The personal column is pure engineering and is the same code path with a different transport. + +### Why this answers both original questions + +- **Performance.** Everything the churn sample blamed (git probes, port scans, spawn, autosave) leaves the UI process. The UI process becomes render plus input. Local latency is unchanged because the client still holds a PTY fd directly, exactly as escrow revival works today. +- **Windows.** The first Windows product is a client only. It does not need ConPTY, escrow, or a Windows session story; it needs a terminal renderer and the generated RPC client. That is what the GPUI spike measures. A local Windows core can come later. +- **The rewrite question dissolves.** Nothing is thrown away. The macOS app keeps shipping and becomes the first client. The core is new code, in Rust, sized by the socket API v2 surface that already exists and is already covered by `tests_v2/`. + +### What already exists toward this + +- Socket API v2 with ordinal handles, a command catalog (`Sources/V2CommandCatalog.swift`), and a Python test suite that only speaks the socket. This is the contract; it needs to become a schema that generates clients. +- The escrow holder (`Sources/SessionEscrow.swift`, `Sources/SessionWALStore.swift`) already is a second process that holds PTY fds and a WAL across app death. It is the seed of the core. +- The Ghostty fork already revives a surface from an existing fd and child pid (fork item 10). That is the local attach path. +- The MCP server and CLI already talk to the app only through the socket. + +### Sequence + +1. **Contract first.** Turn the v2 command catalog into a machine-readable schema (JSON Schema or similar) and generate the Swift CLI, the MCP bridge, and the Python test client from it. Any drift becomes a build error. Small, independent of everything else, pays off even if nothing below happens. +2. **One seam in the app.** Route every core-owned concern in the Swift app through one interface, so the app can talk to an in-process core or an out-of-process one without `if isRemote` branches. This is the refactor the removal note asked for. +3. **Core in Rust, local first.** Grow the escrow holder into `programad`: PTY ownership, WAL, snapshots, agent detection, probes, socket API v2. The macOS app attaches over the Unix socket and receives PTY fds. `tests_v2/` runs unchanged against the new core. Ship this as the default local mode with no user-visible change. +4. **Remote transport.** Same core on a Linux box you own, mTLS, reattach from any client. This is the removed SSH feature rebuilt as one seam. +5. **Windows and Linux client** on GPUI, against a remote or local core. The spike in this document decides the toolkit. +6. **Org mode.** Login, hosted cores, shared brain. Product decision; the engineering above makes it possible without another rewrite. + +Steps 1 and 2 are weeks and are worth doing regardless. Step 3 is the real investment, on the order of months, and is the thing to size after the spike numbers land. + +### Watch list + +- **t3code** (`github.com/pingdotgg/t3code`, MIT). Full inventory with file citations: `t3code-inventory.md`. What matters for us: + - Same shape as the reframe above, shipping today: a Node server owns providers, PTYs, git, files, and an event log; Electron, web, iOS, and Android are thin RPC clients sharing one client runtime. The desktop app has a `localEnvironmentEnabled` toggle, so pure thin-client mode against a remote server already exists. + - Contract-first: every RPC method is declared once in `packages/contracts/src/rpc.ts` (Effect RPC over WebSocket) with a required auth scope. This is step 1 of our sequence, done. + - Remote pairing without a proxy: "T3 Connect" uses a Clerk-backed relay only to broker a one-time bootstrap credential; app traffic then goes direct to the environment over DPoP-bound tokens. The relay never sees session tokens. This is the personal-mode-over-internet design to copy. No organization concept exists anywhere in the repo. + - Agent output is structured events, not PTY scraping: `session.*`, `thread.*`, `turn.*`, `item.*`, `content.delta`, `request.opened/resolved`, `user-input.requested/resolved`. Providers: Claude via the official Agent SDK in-process, Codex via its app-server protocol, Cursor via ACP, OpenCode via its SDK. Permission modes are just event handling on the same stream. + - Shell panels use node-pty on the server (ConPTY on Windows) and render client-side with `libghostty-vt`, Ghostty's VT parser as a C ABI, compiled to WASM for web and native for Android. + - Features worth porting, ranked by the inventory: structured provider events instead of terminal scraping for agent turns; capability-flag negotiation between client and server instead of version lock; the DPoP relay bootstrap for pairing; hidden-git-ref checkpoints per turn; running native telemetry as an isolated child. Avoid: their five-way Linux screenshot backends, a five-provider PR API matrix, and the dual-backend WSL design on Windows. + +## Known losses with a GPUI core + +- GPUI draws everything itself: no native macOS menus, sheets, glass, or accessibility tree for free. +- No webview: the browser panel does not carry over. +- The Ghostty fork delta splits in two. Parser-side patches (OSC 99 notifications, DECRPM 2031, selection API) carry over if the client uses `libghostty-vt`, which our fork already builds with explicit Windows and WASM targets (`ghostty/build.zig:131-165`, `ghostty/src/lib_vt.zig`). Renderer-side patches (occluded-render throttle, offscreen realization, resize stale-frame mitigation) have to be reimplemented in whatever draws the grid on Windows and Linux, because libghostty's Metal renderer does not exist there. +- The running spike uses alacritty_terminal as its VT engine. A follow-up should swap in `libghostty-vt` over FFI so every client shares one parser; the spike's GPUI rendering and memory numbers stay valid either way. +- gpui tracks Zed's internal API; gpui-kit pins a matching crate set to absorb breakage, so upgrades happen on gpui-kit's cadence. diff --git a/docs/plans/t3code-inventory.md b/docs/plans/t3code-inventory.md new file mode 100644 index 00000000..427be1b5 --- /dev/null +++ b/docs/plans/t3code-inventory.md @@ -0,0 +1,541 @@ +# t3code inventory + +Read-only pass over a shallow clone of github.com/pingdotgg/t3code (MIT) on 2026-09-15. Companion to `rust-core-spike.md`; kept as a reference for the core-plus-clients design and for features worth porting. + + +T3 Code is "an agent harness control surface": iOS/Android app, web app +(app.t3.codes), and Electron desktop app, all controlling Claude Code, Codex, +Cursor, Grok Build, OpenCode, and Antigravity running on a machine you own. +Stack: Effect (effect-ts) end to end, Vite+ (`vp`) monorepo tooling, SQLite +persistence, Clerk for cloud identity. This is the closest public reference +to the "thin client / server owns sessions" model Programa is evaluating. + +## 1. Architecture: client/server split, transport, auth, bundling + +**Split.** The server (`apps/server`, Node 22.16+/23.11+/24.10+) owns every +stateful thing: provider processes, PTYs, git, project files, durable event +log, SQLite. Clients (web `apps/web`, desktop renderer `apps/desktop`, +mobile `apps/mobile`) are pure RPC consumers and hold no filesystem or +provider state of their own — stated explicitly in +`docs/internals/overview.md:3-7`: "A remote client must never substitute its +own filesystem, provider credentials, or machine state for the environment's." +A running server instance is called an "environment" and keeps a stable ID +across restarts (`apps/server/src/environment/ServerEnvironment.ts`). + +**Shared client logic.** `packages/client-runtime/src/` is a platform-agnostic +package (connection retry, RPC session, auth token refresh, cached +projections) that all three clients import; platform code only supplies +storage/credentials/lifecycle hooks (`docs/internals/connection-runtime.md`). +Key files: `packages/client-runtime/src/connection/supervisor.ts` (single +retry owner, exponential backoff, distinguishes offline/foreground/background +transitions), `.../connection/registry.ts` (per-environment connection scope), +`.../rpc/session.ts` (waits for initial server config before "ready"), +`.../rpc/client.ts` (resolves RPC calls against current session, subscriptions +survive reconnect, mutations are not auto-replayed), `.../state/threads.ts` +(subscription lifetime separated from a 5-minute idle cache). + +**Transport & schema.** Effect RPC (`effect/unstable/rpc`) over WebSocket + +HTTP, not tRPC. The entire RPC surface (request/response schemas, ~1500 +lines) lives in one file: `packages/contracts/src/rpc.ts`, built from +per-domain schema modules in `packages/contracts/src/` (`orchestration.ts`, +`git.ts`, `terminal.ts`, `provider.ts`, `auth.ts`, `filesystem.ts`, +`review.ts`, `relay.ts`, `worktreeSetup.ts`, `resourceTelemetry.ts`, etc — full +list of ~50 files in that dir). Everything is `effect/Schema`-typed end to +end, client and server share the exact same TS types from `@t3tools/contracts`. +Subscriptions are scoped (e.g. per-thread), so a client viewing one thread +doesn't pay for every thread's history (`overview.md:17-18`). + +**Auth (device→server).** The server issues its own sessions; a separate +relay/cloud identity layer (Clerk) is a distinct trust boundary +(`docs/internals/environment-auth.md`). Mechanisms: (a) pairing — delegates a +scoped grant, cannot be widened by exchanging a bootstrap credential; (b) +bearer + DPoP (Demonstrating Proof-of-Possession) tokens — short-lived +WebSocket "tickets" obtained via authenticated HTTP so long-lived tokens never +sit in socket URLs; (c) browser cookie sessions. Every RPC method declares a +required scope, checked in `apps/server/src/auth/RpcAuthorization.ts`; a +successful socket handshake grants no extra authority. Desktop keeps one +reusable local bearer/bootstrap token across restarts +(`apps/server/src/persistence/AuthSessions.ts`). A dev-only +`T3CODE_DEV_AUTH_TOKEN` gives shared admin access across worktrees on one +host, ignored by desktop/production builds. + +**Phone ↔ laptop over the internet: three real transports, no built-in relay +proxying of app traffic.** +1. **Direct/LAN pairing** — client and server on the same network, endpoint + advertised by the server; the connecting device is the only one that can + prove a route actually works (`remote.md:22-25`), so there's no blind + trust of advertised addresses. +2. **Tailscale** — just supplies a reachable endpoint; not a distinct + environment type, auth still goes through the normal environment auth + path (`remote.md:42-44`). +3. **SSH tunnel** — desktop main process can spawn `ssh` to forward a port + and/or launch a remote server; renderer only ever talks to the local + forwarded endpoint (`packages/ssh/src/tunnel.ts`). Desktop main owns + the SSH process lifecycle because it needs to handle interactive auth + prompts (`packages/ssh/src/tunnel.ts`, `apps/desktop/src/ssh/`). +4. **T3 Connect (cloud relay)** — `docs/internals/t3-connect.md`. Clerk + provides cloud identity; a Cloudflare Worker relay + (`infra/relay/src/environments/EnvironmentConnector.ts`, + `ManagedEndpointProvider.ts`) manages environment *links* and mints + managed tunnel hostnames, but **application traffic (HTTP/WebSocket) goes + directly client→environment tunnel hostname; the relay Worker does not + proxy it** ("the relay Worker does not proxy their HTTP or WebSocket + sessions", `t3-connect.md:6`). The relay's only job in the data path is a + one-time bootstrap: it asks the environment to mint a credential bound to + the client's DPoP key; the relay never sees the resulting session token + (`t3-connect.md:13-16`). So it's "relay for pairing/discovery, direct/ + tunnel for data" — not a full reverse-proxy relay. SSH device-authorization + grant handles headless/CLI pairing without a local browser listener + (`t3-connect.md:81-86`). + +**Electron bundling/startup.** `apps/desktop/src/main.ts` is the Electron +main entry. The server ships *inside* the desktop app and is started as a +child process, managed per-instance by +`apps/desktop/src/backend/DesktopBackendManager.ts` (factory, +`makeBackendInstance(spec)`) and pooled by `DesktopBackendPool.ts` — the pool +can run more than one backend at once (primary + a Windows WSL backend via +`apps/desktop/src/wsl/DesktopWslBackend.ts`). Desktop-specific server config +resolution is `DesktopBackendConfiguration.ts`; local auth for the +desktop↔bundled-server pair is `DesktopLocalEnvironmentAuth.ts`; LAN exposure +toggle is `DesktopServerExposure.ts`. The desktop *renderer* is not served by +the bundled server — a custom `t3code://` scheme serves the bundled web +client from disk (Vite dev server in development); only API/RPC traffic goes +over HTTP/WS to the environment (`remote.md:69-71`). A desktop setting, +`localEnvironmentEnabled` (`apps/desktop/src/settings/DesktopAppSettings.ts`), +lets desktop run with **no** bundled/local server at all — pure thin client +mode, connecting only to saved remote/paired/relay environments +(`remote.md:61-71`). This is effectively the exact "thin client" mode +Programa is evaluating, already shipped as a toggle rather than a rewrite. + +**"Remote-ready" today, concretely:** any client (web, desktop, mobile) can +attach to any environment (local machine, LAN, Tailscale, SSH-tunneled +remote, or cloud-relay-linked) using the same RPC/auth stack, with capability +negotiation (`overview.md:21-38` — environments advertise flags like +`threadPullRequests`; older/newer client-server pairs degrade gracefully +instead of assuming a coordinated release). A remote server can outlive +several client releases (`remote.md:55-59`). + +## 2. Terminal rendering + +Two different mechanisms depending on what's being shown: + +- **Agent output (the actual conversation with Claude/Codex/etc.) is + structured events, not PTY bytes.** See §3's event list — there is no + terminal emulator involved in rendering agent turns/tool calls; it's a + typed event stream rendered as native UI components. +- **A literal shell/terminal panel** (for running arbitrary commands, seeing + raw agent CLI passthrough, etc.) **is PTY-backed and does use a terminal + emulator.** Server side: `node-pty` via + `apps/server/src/terminal/NodePtyAdapter.ts`, spawning through + `apps/server/src/terminal/Manager.ts`, which owns PTY lifecycle, session + retention, and incremental output persistence + (`docs/internals/terminal-runtime.md`). On Windows this goes through + ConPTY; `NodePtyAdapter.ts:162-165` has a comment noting "the ConPTY path + leaves the environment untouched" and compensates by injecting `TERM` when + absent. Client rendering uses **`libghostty-vt`**, a C ABI terminal engine + (Ghostty's VT parser/renderer core) shared between Android and web — + `native/libghostty-vt/`, pinned by `native/libghostty-vt/VERSION`, consumed + by the web renderer at `apps/web/src/terminal/ghostty/core.ts` as + WebAssembly (one WASM instance per browser tab, each terminal owns/frees + its own handle) and natively on Android. React does not touch terminal + frames directly — platform adapters own drawing/input + (`terminal-runtime.md:30-34`). Server-side history is capped (5,000 lines / + 8 MiB per terminal; client buffer cap 512 KiB) and strips terminal + query/response escape sequences from replayed history so restoring + scrollback can't provoke junk replies at the live prompt + (`terminal-runtime.md:40-44`). + +**Structured provider event types** (from +`packages/contracts/src/providerRuntime.ts:204-232`, the `ProviderRuntimeEvent` +discriminated union — this is the full canonical list): + +``` +session.started +session.configured +session.state.changed +session.exited +thread.started +thread.state.changed +thread.metadata.updated +thread.token-usage.updated +thread.realtime.started +thread.realtime.item-added +thread.realtime.audio.delta +thread.realtime.error +thread.realtime.closed +turn.started +turn.completed +turn.aborted +turn.plan.updated +turn.proposed.delta +turn.proposed.completed +turn.diff.updated +item.started +item.updated +item.completed +content.delta +request.opened +request.resolved +user-input.requested +user-input.resolved +task.started +``` +(plus more `task.*` entries — list truncates at the read window; grep +`packages/contracts/src/providerRuntime.ts` for the full `Type` block if you +need the tail). Item/request typing is further split by +`CanonicalItemType` and `CanonicalRequestType` (lines 116-149 of the same +file) — worth reading directly if Programa builds an analogous typed-event +model, since it already encodes tool-lifecycle items, plans, diffs, and +realtime (voice) audio deltas as first-class event kinds. + +## 3. Agent harness integration (one paragraph per provider) + +All five adapters live in `apps/server/src/provider/Layers/*Adapter.ts` +(pattern: `Layers/XAdapter.ts` is the live Effect implementation, +`Services/XAdapter.ts` is the service interface/contract). All normalize into +the same `ProviderRuntimeEvent` stream above via the +`ProviderAdapter` boundary (`apps/server/src/provider/Services/ProviderAdapter.ts`, +referenced from `docs/internals/providers.md:5`) — provider-specific logic is +supposed to stay entirely inside the adapter, never leak into orchestration +or clients. + +- **Claude** — `apps/server/src/provider/Layers/ClaudeAdapter.ts`. Uses the + **official `@anthropic-ai/claude-agent-sdk`** package directly (`query`, + `forkSession`, `getSessionMessages`, `CanUseTool`, `PermissionMode`, + `SDKMessage` types imported straight from the SDK) — this is an in-process + SDK integration, not a CLI subprocess wrapping JSON output. +- **Codex** — `apps/server/src/provider/Layers/CodexAdapter.ts`, backed by an + in-repo package `packages/effect-codex-app-server` (own `schema.ts`, + `errors.ts`) that wraps Codex's app-server protocol as a typed Effect + service; the adapter spawns/talks to it via + `effect/unstable/process/ChildProcessSpawner`. So: CLI subprocess, but with + a structured/typed IPC protocol (Codex's own JSON-RPC-like app-server + mode), not raw stdout scraping. Notably, Codex "async questions" arrive as + notifications with no pending RPC response — answered by sending a new + user message rather than a reply (`providers.md:81-85`), a protocol quirk + worth knowing if Programa ever integrates Codex directly. +- **Cursor** — `apps/server/src/provider/Layers/CursorAdapter.ts`, doc + comment: *"Cursor CLI (`agent acp`) via ACP."* This is the **Agent Client + Protocol** (ACP, the emerging cross-editor agent protocol) — the adapter + uses an in-repo `effect-acp` package (`packages/effect-acp/src`, + `schema.ts`/`errors.ts`) to speak ACP to the `agent acp` CLI subprocess. +- **OpenCode** — `apps/server/src/provider/Layers/OpenCodeAdapter.ts`, uses + the official `@opencode-ai/sdk/v2` client (`OpencodeClient`, `Part`, + `PermissionRequest`, `QuestionRequest` types). Per `providers.md:14-19`, + T3 runs **one OpenCode server per thread** (directory-scoped MCP vs. + thread-scoped T3 MCP connection — sharing a server across threads in one + directory would let threads swap each other's MCP connection). Idle + instance-owned helper servers close after a timeout + (`apps/server/src/provider/OpenCodeServerOwner.ts`). Persistent per-directory + approval grants exist; automatic full-access replies use a one-shot + `once` grant so they can't silently widen permissions on an externally + shared server (`providers.md:20-23`). +- **Antigravity** (Google) — `apps/server/src/provider/Layers/AntigravityAdapter.ts` + + `AntigravityProvider.ts`. No CLI login; uses native sign-in + (`AntigravityAuth.ts`) with per-instance file-based credential storage + (forced, because macOS Keychain entries would otherwise be shared across + instances) and an installer with immutable, leased releases + (`AntigravityInstallation.ts`). Antigravity can capture workspace + checkpoints but **cannot roll back its own conversation state**, so revert + is rejected up front rather than silently desyncing filesystem and + conversation (`providers.md:92-95`). +- **Grok Build** — `apps/server/src/provider/Layers/GrokAdapter.ts` / + `GrokProvider.ts`, present in the README's provider list; adapter avoids + triggering auth/session-creation as a side effect of health probes + (`providers.md:40-42`). + +**Permission modes → approval flow** +(`docs/user/permission-modes.md`, `apps/server/src/auth`/orchestration): +four user-facing modes — **Supervised** (approve every command/file change), +**Auto-accept edits** (edits auto-approved, other actions still gated), +**Auto** (defers to the provider's own automatic-review; only Claude, Codex, +Cursor implement that — OpenCode and Antigravity fall back to asking), **Full +access** (no prompts, though Antigravity can still push native approval +requests even here). Default mode is set per-environment +(Settings → General) and per-project override; new threads take the +environment default, not whatever mode you were last viewing. Mechanically, +approvals are just `request.opened` / `request.resolved` / +`user-input.requested` / `user-input.resolved` events in the same +`ProviderRuntimeEvent` stream — the UI renders a pending request inline in +the conversation and a resolve command flows back through the same +orchestration command path as any other user action (`overview.md`'s +event-sourced command/decider/projector model, `providers.md` "Protocol +traps" section for edge cases like Codex's notification-only async +questions). + +## 4. Feature list (grouped, one line + owning directory each) + +**Sessions / threads** +- Threads = one durable conversation entity with a provider, event-sourced — + `apps/server/src/orchestration/` (engine, decider, projector — see + `overview.md` "Durable intent and side effects"). +- Thread search/snapshot/full-diff RPCs — `packages/contracts/src/orchestration.ts` + (`OrchestrationSearchThreadsInput`, `OrchestrationGetFullThreadDiffInput`, etc). +- Client-side thread cache/subscription lifecycle — + `packages/client-runtime/src/state/threads.ts`. +- Thread sidebar UI doc — `docs/user/thread-sidebar.md`. + +**Projects / workspaces** +- Project entity + settings — `packages/contracts/src/project.ts`, + `apps/server/src/project/`. +- Project-scoped agent session import/scan (importing existing CLI sessions + into T3) — `apps/server/src/project/AgentSessionScanner.ts`, + `packages/contracts/src/agentSessions.ts`. +- Project setup script runner — `apps/server/src/project/ProjectSetupScriptRunner.ts`. +- Project cloning — `apps/server/src/projectClone.ts` (server contracts side), + `packages/contracts/src/projectClone.ts`... actually contract is + `apps/server` side per grep; project file format — + `packages/contracts/src/t3ProjectFile.ts`. + +**Git worktrees** +- Worktree create/remove/setup-stream RPCs — + `packages/contracts/src/worktreeSetup.ts`, `packages/contracts/src/git.ts` + (`VcsCreateWorktreeInput/Result`, `VcsRemoveWorktreeInput`). +- Server implementation — `apps/server/src/git/GitManager.ts`, + `GitWorkflowService.ts`. + +**Diff review / checkpoints** +- Turn/thread diff RPCs — `OrchestrationGetTurnDiffInput`, + `OrchestrationGetFullThreadDiffInput` in `packages/contracts/src/orchestration.ts`. +- Review diff preview/file-contents RPCs — `packages/contracts/src/review.ts`, + server side `apps/server/src/review/`. +- **Checkpointing**: hidden Git refs capture workspace state per-turn without + polluting the user's branch history — `apps/server/src/checkpointing/CheckpointStore.ts`, + `CheckpointDiffQuery.ts`, `Diffs.ts` (see `overview.md` "Turn completion and + checkpoints" — revert coordinates workspace + provider conversation state, + and providers that can't roll back their own conversation must reject + revert before touching files). + +**Source control** +- Multi-host PR/MR support: GitHub, GitLab, Bitbucket, Azure DevOps, Forgejo + — `apps/server/src/pullRequest/{GitHubPullRequestProvider,GitLabPullRequestProvider, + BitbucketPullRequestProvider,AzureDevOpsPullRequestProvider,ForgejoPullRequestProvider}.ts`, + each with a CLI-backed and JSON/API-backed variant (e.g. + `GitHubPullRequestCli.ts` vs `gitHubPullRequestJson.ts`). + Multi-link/stacked-PR support negotiated via environment capability flags + `threadPullRequests` / `threadPullRequestLinking` (`overview.md:21-38`). +- Generic VCS status/ref RPCs — `packages/contracts/src/vcs.ts` + (`VcsStatusInput/Result`, `VcsListRefsInput`, `VcsSwitchRefInput`, + `VcsPullInput`). +- Forgejo CLI wrapper — `apps/server/src/sourceControl/ForgejoCli.ts`. +- Docs: `docs/user/source-control.md`. + +**Notifications** +- Desktop dock/taskbar badge — `apps/desktop/src/ipc/methods/notificationBadge.ts`. +- Mobile push: permission handling, payload schema, deep-link navigation from + a push, response consumption — all under + `apps/mobile/src/features/agent-awareness/` (`notificationPermissions.ts`, + `notificationPayload.ts`, `notificationNavigation.ts`, + `notificationResponseConsumer.ts`). +- Sound cues — `apps/web/src/assets/notification-{completion,input}.mp3`. +- Settings UI — `apps/web/src/components/settings/NotificationSettings.tsx`. +- Docs: `docs/user/mobile-notifications.md`, + `docs/operations/android-notifications.md`. + +**Multi-account** +- Provider account/instance model (a "driver kind" = integration type, an + "instance" = one account/config, so two accounts on the same driver never + share session/catalog state) — `apps/server/src/provider/Services/ProviderInstanceRegistry.ts`, + `ProviderInstanceRegistryMutator.ts`, doc: `providers.md:8-11`. +- Per-provider multi-account user docs: `docs/user/providers-codex.md`, + `docs/user/providers-claude.md`. + +**Settings** +- Client-local vs. environment/project-owned settings split, explicitly + documented as a design rule (`overview.md:44-51` "Settings ownership") — + contracts in `packages/contracts/src/settings.ts`, + `packages/contracts/src/keybindings.ts`. +- Desktop-specific app settings — `apps/desktop/src/settings/DesktopAppSettings.ts`, + `DesktopClientSettings.ts`, `DesktopSavedEnvironments.ts`. + +**Keyboard shortcuts** +- Contract — `packages/contracts/src/keybindings.ts`. +- User doc — `docs/user/keybindings.md`, `docs/user/keyboard-focus.md`. + +**Mobile-specific** +- Native modules/plugins — `apps/mobile/modules/`, `apps/mobile/plugins/`. +- Voice input — `docs/internals/voice-input.md`. +- Push notification stack — see Notifications above. +- Composer context references / attachments — `docs/internals/composer-context-references.md`, + `docs/user/question-attachments.md`. +- Mobile navigation architecture — `docs/internals/mobile-navigation.md`. + +**Browser panel (Programa-relevant, T3 has an equivalent)** +- Desktop embeds a live preview/browser panel with element-pick and + Playwright-driven automation — `apps/desktop/src/preview/` + (`Manager.ts`, `PlaywrightInjectedRuntime.ts`, `PickPreload.ts`, + `Annotation.css`/`AnnotationKeyboard.ts`, `BrowserSession.ts`, + `FaviconCapture.ts`). Browser import/profile handling — + `packages/contracts/src/browserImport.ts`, `browserProfile.ts`. + +## 5. Windows/Linux specifics + +- **ConPTY** — `apps/server/src/terminal/NodePtyAdapter.ts:162-165`, Windows + path via node-pty's ConPTY backend; comment notes ConPTY "leaves the + environment untouched" so the adapter injects `TERM` manually when absent. +- **Windows foreground/focus handling** uses native FFI (`ffi-rs`) loaded + lazily, isolated from Electron main startup — + `apps/desktop/src/electron/WindowsForeground.ts`, + `WindowsForegroundFocusThread.ts`, `WindowsForegroundFocusWorker.ts` + (`overview.md:105-110`: native modules never load on the main-process + startup path; `ffi-rs` loads lazily just for a few Win32 calls). +- **Windows Subsystem for Linux (WSL) backend** — Desktop can run a *second* + bundled server instance inside WSL alongside the native Windows primary: + `apps/desktop/src/wsl/DesktopWslBackend.ts`, `DesktopWslEnvironment.ts`, + `DesktopWslServerTree.ts`, `wslPathParsing.ts`. Windows packages currently + ship only a Windows resource-monitor binary, so WSL-backend process + telemetry is unavailable even though the Electron power feed still works + (`resource-telemetry.md:47-49`). +- **Linux desktop-entry / portal identity** — must be set *before* Chromium + init via `DesktopPreReadyPlatform.layer`, because Chromium caches the first + desktop-entry registration including failures + (`overview.md:94-103`); AppImage updates can invalidate the entry's `Exec` + path, refreshed just before portal registration. Handler: + `apps/desktop/src/app/DesktopLinuxUrlHandler.ts`, + `apps/desktop/src/app/DesktopPreReadyPlatform.ts`. +- **Linux screenshot capture per-compositor**: separate implementations for + GNOME (`apps/desktop/src/snapShot/GnomeCaptureSetup.ts`, + `gnomeCaptureBundle.ts`, plus a native `apps/desktop/gnome-extension/`), + KDE (`KdeSnapShot.ts`, native crate `native/kde-snap-shot/`), Hyprland + (`HyprlandSnapShot.ts`, native crate `native/hyprland-snap-shot/`), Niri + (`NiriSnapShot.ts`), and a generic xdg-desktop-portal path + (`PortalCaptureShortcut.ts`, `LinuxSnapShot.ts` dbus-based). +- **Linux secret storage** — `apps/desktop/src/linuxSecretStorage.ts` + (libsecret-style, distinct from macOS Keychain / Windows DPAPI paths used + elsewhere). +- **macOS-specific**: accessibility permission flow + (`apps/desktop/src/permissions/MacPermission*.ts`, + `mac-permission-preload.ts`), macOS window lookup shells out to + `osascript` rather than a native addon (`overview.md:109`), macOS-specific + screenshot capture and modifier-pair global shortcut + (`snapShot/MacSnapShot.ts`, `MacModifierPairShortcutProcess.ts`). +- **Native snapshot workers run out-of-process** — `@crowecawcaw/xa11y` + (accessibility tree access for screenshot annotation) only runs inside + forked Node child processes (`SnapShotAccessibilityWorker.ts`, + `RegionSnapShotWorker.ts`) or a worker thread, never in Electron main, so a + crash there can't take the app down (`overview.md:105-110`). +- **Packaging**: found only Arch Linux AUR PKGBUILDs in-repo — + `packaging/aur/t3code-bin/PKGBUILD` (stable) and + `packaging/aur/t3code-nightly-bin/PKGBUILD` (nightly). README documents + `winget install T3Tools.T3Code` for Windows and `brew install --cask + t3-code` for macOS, but no `electron-builder` config or winget manifest is + checked into this shallow clone — likely generated by CI + (`.github/workflows/release.yml`, `release-desktop.yml`) or lives in a + separate winget-pkgs/homebrew-cask submission repo, not in this monorepo. + Could not find an `electron-builder.yml`/`.json` in the tree at all in + this shallow clone — desktop build config may be inline in + `apps/desktop/package.json` or generated; didn't locate it definitively. + +## 6. Team/org/cloud + +- **T3 Connect** (`docs/internals/t3-connect.md`) is the only multi-user/cloud + surface found: Clerk-based cloud identity + a relay + (`infra/relay/`, Cloudflare Worker judging by `Worker` terminology and + `wrangler`-style migrations dir `infra/relay/migrations/`) that links a + cloud user account to one or more owned environments and brokers pairing + credentials. This is **account-to-own-machine linking**, not + organization/team multi-tenancy — no "org" concept, shared project, or + hosted multi-tenant server surfaced in docs or contracts searched. +- No README/AGENTS.md/issue-tracker mention of "organization" as a product + concept found in this clone (repo doesn't ship a GitHub Issues export; + only the code+docs are available offline). If Programa specifically needs + "log in as org," t3code's relay model (client-owns-credentials, + server-mints-scoped-session, relay-never-sees-session-token) is a + reasonable pattern to borrow but there's no org-scoping precedent to copy + directly — you'd be extending their per-user link model, not reusing an + existing org layer. +- Relay observability/ops docs exist (`docs/operations/relay-observability.md`, + `docs/operations/connect-setup.md`) confirming it's a real production + service, not a stub. + +## 7. Rust components (Resource Monitor) + +- `native/resource-monitor/` — standalone Rust binary (`Cargo.toml`: + `t3-resource-monitor`, edition 2024), single dependency of substance: + `sysinfo = "0.39.3"` (+ serde/serde_json for the wire format), built with + `panic = "abort"`, `lto = "thin"`, `codegen-units = 1`, `strip = true` + (small, crash-isolated binary, not a library). +- **Why Rust, explicitly stated** (`docs/internals/resource-telemetry.md:3-8`): + "Keeping native collection outside Node isolates collector crashes and + avoids a Node/Electron addon ABI matrix." I.e., not performance — it's + process isolation (a crash in the monitor can't take the Electron/Node + process down) and avoiding native Node addon rebuilds across Electron/Node + ABI versions. It's driven as a child process over a private protocol, + same protocol for desktop and CLI/headless server + (`apps/server/src/resourceTelemetry/ResourceMonitorBinary.ts`). +- **What it does**: continuous or on-demand snapshots of process CPU/memory/IO + and host power, with independently bounded history (age, snapshot count, + process-row count, retained bytes — a naive count cap doesn't bound memory + because command lines vary in size). Linux per-thread (`/proc//task/`) + enumeration is explicitly disabled because it makes sampling itself + expensive. Windows process I/O counters include more than disk traffic + (a documented measurement trap, `resource-telemetry.md:39-40`). Electron + main separately supplies host power + Electron-process metrics over a + private inherited pipe, independent of the renderer/RPC connection, so + power telemetry survives even with no client connected. + +## 8. Judgment: what Programa should port, and what to avoid + +**Port, ranked by user value:** + +1. **Structured provider-event model instead of (or alongside) raw PTY + passthrough for agent turns.** T3's `session.*/turn.*/item.*/request.*` + event taxonomy (`packages/contracts/src/providerRuntime.ts`) is the + single biggest architectural lever here — it's what makes diff review, + checkpoints, approval UI, notifications, and multi-client sync all + possible without scraping terminal text. Programa already renders agent + output through PTY bytes; a typed event layer (even sourced by parsing + Claude Code's/Codex's own structured hooks/JSON output rather than a raw + VT stream) would unlock diff review and reliable notification triggers + that don't depend on regex-matching terminal text. +2. **Environment capability negotiation instead of version-pinned + client/server assumptions** (`overview.md:21-38`). If Programa moves to + a client/server split, this pattern (advertise flags, clients branch on + flags not versions, servers keep emitting legacy fields) avoids forcing + synchronized client/server releases — directly relevant since Programa + ships via auto-rolling CI releases already and would otherwise need to + coordinate app-store review lag against server releases. +3. **DPoP-bound bootstrap credentials for remote pairing** + (`t3-connect.md`, `environment-auth.md`) — the relay-mints, + client-redeems-with-proof-key, relay-never-sees-session-token pattern is + a solid, already-battle-tested design for "log in on phone, reach your + laptop" without trusting the relay with actual session authority. Worth + copying almost exactly if Programa builds a phone/companion client. +4. **Checkpointing via hidden Git refs per turn** (`CheckpointStore.ts`, + `overview.md` "Turn completion and checkpoints") — cheap, git-native + undo/revert per agent turn without commit pollution; directly reusable + for a "revert this agent turn" feature, and pairs naturally with + Programa's existing git-heavy workflows. +5. **Out-of-process, isolated Rust telemetry/native-capability children** + (`overview.md:105-110`, resource monitor) — the general policy ("new + native capability goes in a child with a deadline, not an `import` in + main") is a good rule to adopt verbatim for any future native addon in + Programa's Swift/AppKit + Ghostty stack, especially cross-platform Rust + utilities if Programa goes to Windows/Linux. + +**Avoid, with reasons:** + +1. **Per-compositor Linux screenshot capture sprawl** (GNOME extension + + KDE/Hyprland/Niri native crates + generic portal path, `apps/desktop/src/snapShot/*`). + This is a lot of platform-specific surface area (5+ separate + implementations) for one feature; it's a maintenance tax that only pays + off once you're already committed to a broad Linux desktop-environment + matrix. Programa should not replicate this breadth unless/until Linux + support is a firm commitment, and even then should scope down to the + generic xdg-desktop-portal path first. +2. **Five-way pull-request provider matrix maintained in-house** + (GitHub/GitLab/Bitbucket/Azure DevOps/Forgejo, each with both a CLI- and + API-backed implementation, `apps/server/src/pullRequest/`). Real + maintenance surface (10 provider files, ongoing API drift risk) for + modest incremental value beyond GitHub, which is almost certainly what + matters most for Programa's actual usage; start with GitHub only and + resist widening until there's clear demand. +3. **WSL-as-a-second-backend architecture on Windows** + (`apps/desktop/src/wsl/DesktopWslBackend.ts`, WSL server tree, separate + telemetry gap already documented as a known hole). Running two + full backend server instances (native Windows + WSL) from one Electron + shell is architecturally heavy and, per their own docs, has an + unresolved telemetry gap. If Programa targets Windows, prefer picking + one execution environment (native Windows via ConPTY, matching their own + NodePtyAdapter approach) rather than shipping a dual-backend model from + day one. From 423bb60e640b0bd8555ca019edc0c0343a6c64ed Mon Sep 17 00:00:00 2001 From: arzafran Date: Wed, 16 Sep 2026 11:40:02 -0300 Subject: [PATCH 2/2] docs: reconcile the Rust core plan with the Windows frontend on main --- docs/plans/rust-core-spike.md | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/docs/plans/rust-core-spike.md b/docs/plans/rust-core-spike.md index 632e4ed4..50e16b6a 100644 --- a/docs/plans/rust-core-spike.md +++ b/docs/plans/rust-core-spike.md @@ -1,6 +1,6 @@ # Rust core spike: decide with numbers, not a rewrite -Status: spike complete, decision recorded (2026-09-15) +Status: spike complete, decision recorded (2026-09-15); reconciled with the Windows frontend that landed on main (2026-09-16) ## Question @@ -148,7 +148,20 @@ Steps 1 and 2 are weeks and are worth doing regardless. Step 3 is the real inves - Shell panels use node-pty on the server (ConPTY on Windows) and render client-side with `libghostty-vt`, Ghostty's VT parser as a C ABI, compiled to WASM for web and native for Android. - Features worth porting, ranked by the inventory: structured provider events instead of terminal scraping for agent turns; capability-flag negotiation between client and server instead of version lock; the DPoP relay bootstrap for pairing; hidden-git-ref checkpoints per turn; running native telemetry as an isolated child. Avoid: their five-way Linux screenshot backends, a five-provider PR API matrix, and the dual-backend WSL design on Windows. -## Known losses with a GPUI core +## What landed on main on 2026-09-16, and how the plan changes + +Commits `ac23cdf900` through `d4c0a7bcd8` added a native Windows frontend and a shared core in-tree, in parallel with the workstreams below. Reconciliation: + +| Piece | What main has | What this plan proposed | Resolution | +|---|---|---|---| +| Windows UI | WinUI 3 in C# on .NET 10 (`windows/`), native tabs, splits, settings, en and ja resources, CI-built unsigned EXE | GPUI client (spike) | WinUI is the Windows product. The GPUI spike stays as a measured reference and a possible Linux client; no further investment unless Linux is wanted. | +| Portable state | `core/crates/programa-domain` and `programa-ffi`: workspace, pane, surface, layout, selection, tab order as a C ABI library (`core/ABI.md`), in-process on both platforms; macOS adapter `Sources/SharedWorkspaceCore.swift` seeds one pane and projects reorders | a headless daemon owning PTYs, sessions, and the socket API | Complementary, not competing. The in-tree core is the model layer; `programad` (`~/Developer/@darkroom/programa-core`) is the process layer: PTY ownership, WAL, attach and detach, fd handoff, the remote transport. Next step is to make `programad` link `programa-domain` so there is one state model, and to move `programa-core` into `core/crates/programad` in this repo. | +| Terminal engine on Windows | `core/crates/programa-terminal`: vendored, patched `alacritty_terminal` 0.26.0 over ConPTY (patch: final PTY output loss under snapshot lock contention, with a regression test) | `libghostty-vt` for every non-macOS client | alacritty_terminal is shipping and patched; keep it. `libghostty-vt` remains an option if parser parity with the macOS fork (OSC 99, DECRPM 2031) becomes a requirement; the spike proves it builds and links. | +| Contract, seam, agent events (this session's PRs #338, #339, #340) | not present | steps 1 to 3 of the sequence | Unchanged. The seam (`ProgramaCore` in #339) and `SharedWorkspaceCore` are two adapters on the same side of the boundary and should merge into one once workspace lifecycle moves into the shared core. | + +Sequence, updated: contract (#338) and seam (#339) land first; agent events (#340) next; then `programad` joins `core/` and takes PTY ownership on macOS behind the seam; then the remote transport per `programa-core/docs/remote-transport.md`; org mode remains a product decision. + +## Known losses with a GPUI core (now moot for Windows, kept for a Linux client) - GPUI draws everything itself: no native macOS menus, sheets, glass, or accessibility tree for free. - No webview: the browser panel does not carry over.