Repository navigation
Add C# comparison and align benchmark reports and CI - #17
Conversation
Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #16
Working session summaryPR #17 is implemented, pushed, and ready for review. Added C# benchmarks, consistent reports and provenance, backend equivalence tests, isolated CI, and template cleanup. Tests also uncovered and fixed two Rust correctness bugs. All local checks and all nine CI jobs pass. Working tree is clean. Full-size README results will publish after merge. This summary was automatically extracted from the AI working session output. |
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost estimation:
📊 Context and tokens usage:
Total: (388.1K + 7.3M cached) input tokens, 68.1K output tokens, $4.028427 cost 🤖 Models used:
📎 Log file uploaded as Gist (5934KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
This reverts commit 2a34244.
🎉 Auto-mergedThis pull request has been automatically merged by hive-mind.
Auto-merged by hive-mind with --auto-merge flag |
Adds the C# comparison and makes both languages report all seven operations against SpacetimeDB and four Doublets layouts. Results now show three significant digits with units and
≈ samewhen medians differ by less than 5% or their ±1 standard deviation ranges overlap. For example, the historical Query All comparison changes from21580 (1.0x faster)to21.6 µs (≈ same).Fixes #16.
create_bydiscarded source/target values, and Split updates classified self-references after detaching them from the index. Both are corrected. Query datasets now use unique pairs referencing actual background IDs, and query results are consumed to preserve the measured work.Validation completed locally:
cargo clippy --locked --all-targets -- -D warnings.Reproduction and regression coverage:
create_preserves_endpointsfails before the adapter correction;experiments/reproduce-split-index.shruns a bounded differential reproduction of the Split index error;rust/tests/same_behavior.rsand the C# same-behavior tests compare actual rows, normalized identities and repeated benchmark datasets. Report tests cover the original Query All noise case, deviation overlap, unit boundaries, independent backend artifacts, provenance and mismatched sizes.The README explicitly identifies existing Rust numbers as historical and does not invent missing CPU information. C# measurements and both updated language reports are generated by PR CI as review artifacts; the first full default-branch run publishes current tables and charts. PR validation uses N=10/B=30, while full runs use N=1000/B=3000.
CI on
79b4726: all nine jobs passed, including Rust tests on Linux/macOS, live C# tests, all four isolated benchmark jobs and report generation. Reviewed the benchmark-report artifact: both complete 7-operation/5-backend tables, four valid charts, raw measurements and recorded metadata.C# PR validation report (N=10, B=30)
Generated 2026-10-06 23:46 UTC by GitHub Actions run 37548080915 — 10 benchmarked links, 30 background links. SpacetimeDB.ClientSDK 2.10.1; Platform.Data.Doublets 0.18.1; SpacetimeDB module 2.10.1; SpacetimeDB server/CLI 2.10.1; CPU — spacetimedb: AMD EPYC 7763 64-Core Processor; doublets: AMD EPYC 7763 64-Core Processor.
Changes
.github/workflows/release.yml.github/workflows/rust-benchmark.yml.gitignore.pre-commit-config.yamlCONTRIBUTING.mdCargo.lockCargo.tomlREADME.mdchangelog.d/20261006_comparison_parity.mdcsharp/SpacetimeDBVSDoublets.Tests/HarnessTests.cscsharp/SpacetimeDBVSDoublets.Tests/SameBehaviorTests.cscsharp/SpacetimeDBVSDoublets.Tests/SpacetimeDBVSDoublets.Tests.csprojcsharp/SpacetimeDBVSDoublets.Tests/packages.lock.jsoncsharp/SpacetimeDBVSDoublets.slnxcsharp/SpacetimeDBVSDoublets/Benchmarks.cscsharp/SpacetimeDBVSDoublets/DoubletsLinks.cscsharp/SpacetimeDBVSDoublets/Harness.cscsharp/SpacetimeDBVSDoublets/ILinks.cscsharp/SpacetimeDBVSDoublets/ModuleBindings/Reducers/CreateLink.g.cscsharp/SpacetimeDBVSDoublets/ModuleBindings/Reducers/DeleteAllLinks.g.cscsharp/SpacetimeDBVSDoublets/ModuleBindings/Reducers/DeleteLink.g.cscsharp/SpacetimeDBVSDoublets/ModuleBindings/Reducers/UpdateLink.g.cscsharp/SpacetimeDBVSDoublets/ModuleBindings/SpacetimeDBClient.g.cscsharp/SpacetimeDBVSDoublets/ModuleBindings/Tables/Links.g.cscsharp/SpacetimeDBVSDoublets/ModuleBindings/Types/Link.g.cscsharp/SpacetimeDBVSDoublets/Program.cscsharp/SpacetimeDBVSDoublets/SpacetimeDBVSDoublets.csprojcsharp/SpacetimeDBVSDoublets/SpacetimeLinks.cscsharp/SpacetimeDBVSDoublets/packages.lock.jsoncsharp/global.jsonexamples/basic_usage.rsexperiments/reproduce-split-index.shrust/Cargo.tomlrust/benches/bench.rsrust/doublets-patched/PATCHES.mdrust/doublets-patched/doublets/src/mem/split/store.rsrust/out.pyrust/results.mdrust/spacetime-module/Cargo.tomlrust/src/benched/doublets_benched.rsrust/src/benched/mod.rsrust/src/benched/spacetimedb_benched.rsrust/src/doublets_impl.rsrust/src/lib.rsrust/src/workload.rsrust/test_out.pyrust/tests/same_behavior.rsscripts/benchmark-metadata.pyscripts/install-spacetimedb.shscripts/spacetimedb-version