Skip to content

Retain the state root a commit computed until the graph changes - #796

Merged
flyingrobots merged 9 commits into
mainfrom
perf/retain-committed-state-root
Oct 13, 2026
Merged

flyingrobots merged 9 commits into
mainfrom
perf/retain-committed-state-root

Conversation

@flyingrobots

Copy link
Copy Markdown
Owner

What changes

After an executable-operation Action commit, the next reading of the frontier's state root hashed the whole reachable state again, although the commit had just computed it. The next tick paid for that when it resolved its frontier basis.

  • WorldlineState retains the root its commit computed, bound to the root key it was computed under. state_root() returns it without hashing.
  • Any mutable access to the graph drops it. To make that hold by construction, the graph field is now private to its module: mutation goes through warp_state_mut(), which drops the retained root, or apply_patch_journaled(), which drops it only when the patch applied. About twenty call sites change from field access to the accessor.
  • The engine's commit guard rebuilds a state through WorldlineState::from_parts instead of a struct literal; a state built that way has no retained root.
  • A rollback marker carries the retained root of the graph it restores, so an abandoned tick restores it exactly.
  • Debug builds assert that a retained root equals the hashed root. That check covers the whole test suite; it also means debug builds do not get the saving.

Only the Action commit paths retain a root. The native-rule engine path does not (its states are rebuilt through from_parts), so it behaves as before.

Evidence

Guarded Docker worker (Linux VM), release build, compare-and-set workload, no WAL, 40 ticks, one run each, tick p50:

Graph size Actions per tick main (after #795) this PR
1,000 nodes 1 48 µs 43 µs
1,000 nodes 32 1.21 ms 1.22 ms
100,000 nodes 1 1.03 ms 0.54 ms
100,000 nodes 8 1.32 ms 0.83 ms
100,000 nodes 32 2.28 ms 1.82 ms

For reference, the 100,000-node rows were 5.03 ms, 8.13 ms and 18.09 ms before #793.

Tests

  • compiler_cas_tick_hashes_the_state_once_after_the_first_commit: after a first commit, each further Action tick performs exactly one whole-state root computation on the frontier state. RED/GREEN in the guarded worker: without retention the assertion fails with left: 3, right: 1; with this PR it passes.
  • a_retained_state_root_is_dropped_by_any_mutable_graph_access and a_retained_state_root_is_bound_to_the_root_key.
  • compiler_cas_tick_wal_failure_restores_exact_runtime_and_provenance (existing) compares the runtime's debug rendering, which includes the retained root, before and after an abandoned tick.

Guarded worker on this source: compare-and-set 8 (with host_test) and 6 (without); warp-core lib 721 and 751; default-feature integration tests; pipeline 42; host loop 50; the other workspace crates' tests; compare-and-set, pipeline and host loop again in release mode; fmt; strict Clippy on the library and the compare-and-set test target in both feature sets.

What remains

One whole-state hash per Action tick, about 0.5 ms at 100,000 nodes. Removing it needs an incremental state root, which changes what the root commits to and is a design decision recorded on #782.

Documentation impact

CHANGELOG.md updated. The design page's statement that each admitted tick computes a state root over the whole reachable state remains true.

Refs #782.

🤖 Generated with Claude Code

flyingrobots and others added 2 commits October 12, 2026 22:18
After an executable-operation Action commit, the next reading of the frontier
state root hashed the whole reachable state again, although the commit had
just computed it. The next tick paid for that when it resolved its frontier
basis.

WorldlineState now retains the root its commit computed, bound to the root key
it was computed under. state_root returns it without hashing. Any mutable
access to the graph drops it.

To make that hold by construction, the graph field is private to its module.
Mutation goes through warp_state_mut, which drops the retained root, or
apply_patch_journaled, which drops it only when the patch applied. The engine
rebuilds a state through from_parts instead of a struct literal. A rollback
marker carries the retained root of the graph it restores. Debug builds assert
that a retained root equals the hashed root.

Refs #782.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T05:53:34.522394Z d8c53b4 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 42 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: flyingrobots/echo/.coderabbit.yaml
  • Review profile: ASSERTIVE
  • Plan: Advanced
  • Run ID: 1b2a9127-7e4a-4c0a-811e-ea5894f0da8d
📥 Commits

Reviewing files that changed from the base of the PR and between 7651b52 and e796733.

📒 Files selected for processing (10)
  • CHANGELOG.md
  • crates/warp-core/src/coordinator.rs
  • crates/warp-core/src/echo_operation.rs
  • crates/warp-core/src/echo_operation/observed.rs
  • crates/warp-core/src/engine_impl.rs
  • crates/warp-core/src/trusted_runtime_host.rs
  • crates/warp-core/src/worldline.rs
  • crates/warp-core/src/worldline_state.rs
  • crates/warp-core/tests/edict_projected_cas_tests.rs
  • docs/architecture/sliced-readings-and-copy-on-write-strands.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 200cf2edb4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/warp-core/src/worldline_state.rs
flyingrobots and others added 2 commits October 12, 2026 22:31
Address review: the comment still said every call hashes the whole reachable
state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Integrate origin/main through f86acbd (#791). The only textual conflict was
CHANGELOG.md; both entries are retained.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@flyingrobots

Copy link
Copy Markdown
Owner Author

Independent agy review at 200cf2ed: REQUEST CHANGES for one documentation defect, now fixed.

The review proved the central claim: it enumerated every write of the graph, the root key and the retained root inside worldline_state.rs, every warp_state_mut call site, the rollback and journal-restore paths, and the engine guard's two rebuilds field by field, and found the retained root sound. It also judged the three tests non-vacuous.

  • Finding 1 (the reason for the verdict): the rustdoc on WorldlineState::state_root still said every call hashes the whole reachable state. Fixed in the latest commits, also raised by Codex.
  • Finding 2: benchmark figures are single VM runs. True and stated in the description.

Since then main (#791) was merged in; the only conflict was CHANGELOG.md. Revalidated in the guarded worker: warp-core lib 721 and 751; default-feature integration tests; compare-and-set 8; pipeline 42; host loop 57; WAL hardening 136; fmt; strict Clippy.

flyingrobots and others added 2 commits October 12, 2026 22:37
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The rustdoc warnings gate rejects the link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 401203c7c0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/architecture/sliced-readings-and-copy-on-write-strands.md

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d8c53b41e2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/warp-core/src/worldline_state.rs
It counts the hashes every build performs and excludes the hash by which a
debug build checks a retained root, so the count is the same in both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@flyingrobots
flyingrobots merged commit 3056a9d into main Oct 13, 2026
42 checks passed
@flyingrobots
flyingrobots deleted the perf/retain-committed-state-root branch October 13, 2026 06:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant