Repository navigation
feat(record): run recording groundwork (bundle, HTTP cursor, redaction, viewer, step times) - #470
Merged
Merged
Conversation
Adds the internal/record package foundations for run recording. The cursor reads probe.http_log without skipping requests that are still in flight, detects app restarts and counts exchanges the agent's ring evicted. The bundle writer lays out manifest, timeline, exchanges, widget trees and screenshots with validated names and atomic 0600 writes. Nothing is wired into the runner or the CLI yet. Claude-Session: https://claude.ai/code/session_01QGnLBdCp6x94qKGY5kAxHK
Step events carry the wall-clock time they were emitted, so traffic from outside the agent can be placed in a step. The redaction helpers remove credential headers, query values and body values, including numeric card numbers and PINs, and keep everything else unchanged. Both pieces are internal; nothing is wired into the CLI yet. Claude-Session: https://claude.ai/code/session_01QGnLBdCp6x94qKGY5kAxHK
Renders a recording bundle as one HTML page that opens from disk with no server. The manifest and timeline are inlined with script-safe escaping, every bundle value is inserted as text, and a content security policy blocks network requests. Claude-Session: https://claude.ai/code/session_01QGnLBdCp6x94qKGY5kAxHK
The viewer cannot read files once it is opened from disk, so a page built from the timeline alone would show exchange IDs and no widget trees. WriteViewer now inlines the exchanges and trees a timeline names, and refuses unsafe IDs and missing files. Checked in a browser against a sample bundle: steps, screenshots, redacted bodies, widget trees and failures all render, with no console errors. Claude-Session: https://claude.ai/code/session_01QGnLBdCp6x94qKGY5kAxHK
Member
Author
|
Browser check of the offline viewer, against a generated sample bundle served from loopback, at desktop width in light mode:
Not verified: phone width and dark mode. The browser tool could not resize the window, and the viewer's Content-Security-Policy blocks framing, so the page cannot be checked inside an iframe. Found and fixed in c0f4b11: a page built from the timeline alone showed exchange IDs and no widget trees. Follow-up: an empty request body renders as a blank block rather than a label. 🤖 Generated with Claude Code |
6 of 8 tasks
The disclosure guard flags complete secret-shaped tokens in added lines. The three Stripe and webhook fixtures are split across string pieces, as internal/hosted/scrub.go already does for its webhook pattern. The tests still exercise the same rules. Claude-Session: https://claude.ai/code/session_01QGnLBdCp6x94qKGY5kAxHK
…undations # Conflicts: # CHANGELOG.md
pbertsch
added a commit
that referenced
this pull request
Oct 17, 2026
… test ## Summary - Adds `probe test --record` and `--record-dir`. Each test attempt gets a bundle under `<reports_folder>/recordings/<run>/<device>/`: its steps with source file and line, screenshots before and after each step, a widget tree after each step, the app's `dart:io` HTTP exchanges with credentials redacted, and an offline HTML viewer. - Adds a blocking per-step recorder hook to the runner, separate from `OnStep`. It is nil by default, and with no recorder configured the runner's behaviour is unchanged. - Adds `internal/record.Recorder`, which writes one bundle per test attempt. Exchange files are numbered per bundle, so an app restart cannot overwrite an earlier exchange. - Recording is an observer. Capture failures become warnings and never change a result or the exit code. Nothing is uploaded. - Docs: the CLI reference, the Probe Cloud preview note, the README, the reports page, the landing feature card, the MCP `run_tests` flags, and the CHANGELOG. Builds on #469 (design) and #470 (bundle writer, HTTP cursor, redaction, viewer, step start times). Both are merged. ## Not recorded yet (stated in the docs) - Native SDK and WebView traffic. Only the app's `dart:io` HTTP is captured. - The element each step acted on. - Screenshot redaction rules. Screenshots are not redacted. - Widget trees are captured after steps, not before them. - The manifest's `platform` and `os_version` fields are empty. The CLI does not pass them yet. ## Review notes - The recorder numbers exchange files per bundle. Using the agent's sequence number would overwrite files after an app restart. A regression test covers this. - Composite runs: the composite runner is rebuilt when recording is on, with the same timeout and verbosity as `buildCompositeRunner`, plus the recorder factory. `buildCompositeRunner` registers no recipes, so nothing else is lost. - Local runs do not set a device name in the run metadata, so the recording directory falls back to the `--device` serial. - `internal/cli/test.go` was run through gofmt, so some untouched alignment lines changed too. Those hunks are whitespace only. - `probe record` (the interaction recorder) is a different command from `probe test --record`. The docs name both. ## Test plan - [x] `go test ./...` passes on the root module - [x] `go build ./...` passes - [x] `gofmt -l` is clean for the changed files, and `go vet` is clean for `internal/cli`, `internal/mcp`, `internal/record` and `internal/runner` - [x] New tests: bundle writing, redaction, retries and name collisions, device errors, the app-restart regression, runner hook ordering and error isolation, and the CLI event conversion and device slug - [x] `probe test --record` run on an iOS simulator and an Android emulator with the project's end-to-end fixture app (5 tests on each). The fixture makes no HTTP calls, so no `http/` folder is written - [ ] Open a bundle in a browser at phone width and in dark mode. Not done yet - [x] CI passes, including the disclosure guard - [ ] Review 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
internal/record:StepEvent.At, the wall-clock time each step event was emitted.probe test --recordcome in a follow-up PR.internal/redact, which this PR does not change.One existing expectation changed:
redact_test.goused to expect"token": 123to stay unredacted. It now expects"<redacted>", because numeric tokens, PINs and card numbers are credentials.Test plan
go test ./...passes on the root modulego build ./...passesgofmt -lis clean forinternal/recordand the two changed runner filesgo vet ./internal/record/ ./internal/runner/is cleaninternal/record/viewer/index.htmlin a browser against a real bundle: check the layout at phone width and in dark mode, and the screenshot pathsgofmt -l internal/runnerlists seven files that were already unformatted onmain(ai_text_test.go,executor_ai_test.go,http_steps.go,http_steps_test.go,reporter.go,runner.go,runner_test.go). This PR does not touch them.🤖 Generated with Claude Code