Repository navigation
Add C# benchmarks and synchronize reports, correctness tests and CI - #43
Conversation
Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #42
Working session summaryImplemented issue #42 in PR #43, now ready for review. Added C# benchmarks, corrected report comparisons, generated tables and charts with provenance, added parity tests, isolated CI jobs, and removed dormant C++ code. All 19 tests, 96 local benchmark measurements, and final-commit CI passed. The branch includes current This summary was automatically extracted from the AI working session output. |
🤖 Solution Draft LogThis log file contains the complete execution trace of the AI solution draft process. 💰 Cost estimation:
📊 Context and tokens usage:
Total: (235.1K + 4.9M cached) input tokens, 48.7K output tokens, $2.651629 cost 🤖 Models used:
📎 Log file uploaded as Gist (3836KB)Now working session is ended, feel free to review and add any feedback on the solution draft. |
🎉 Auto-mergedThis pull request has been automatically merged by hive-mind.
Auto-merged by hive-mind with --auto-merge flag |
Fixes #42.
Rust and C# now benchmark the same eight point-link operations across four Doublets stores and PostgreSQL with and without an explicit transaction. Reports use the fastest PostgreSQL implementation as the baseline and distinguish
× faster,× slower, and≈ same(medians within 5% or overlapping ± one standard deviation ranges).Changes:
results:start/results:endmarkers. Every table records PostgreSQL, driver and Doublets versions, CPU, UTC date and an Actions run link. Preserve linear and logarithmic charts for both languages and every size.rust/out.pyas a compatibility entry point to the shared report script.17.6; add workflow path filters, concurrency and job timeouts. PRs use 100 background / 10 active links; default-branch runs also measure 1,000 / 100. PR reports are artifacts; default-branch reports are committed automatically.POSTGRES_CONNECTIONand C#'s fromNPGSQL_CONNECTION. Document both connection formats, sizes, measured boundaries and local commands. Reject invalid benchmark sizes.The new correctness tests also exposed existing Rust adapter defects: identity enumeration used
%1instead of$1, one count query joinedANDto the next column without a space, and missing-link lookups indexed empty result sets. These are fixed. SQL clients now use mutexes instead of the unsafe shared-mutableExclusivewrapper. Delete benchmarks previously deletedN + 1links, including one background point; both languages now delete exactly theNprepared points.Reproduction and verification:
python3 experiments/reproduce_report_ratio.pypreviously failed with200 (0.5x faster)for Doublets taking twice the PostgreSQL time; it now prints2× slower. This reproducer needs no plotting dependencies.python3 -m unittest discover -s scripts -v: 10 tests cover direction, the noise boundary, overlapping deviations, fastest-baseline selection, parsing, incomplete results, mismatched provenance and marker replacement.cargo fmt --manifest-path rust/Cargo.toml -- --checkandcargo clippy --locked --manifest-path rust/Cargo.toml --all-targets --all-features -- -D warningspass.cargo test --locked --manifest-path rust/Cargo.toml -- --include-ignored: both integration tests pass, exercising all six stores, all query shapes, update/restore, delete counts, absent links and repeated fork cleanup. A local run used a custom database and host to exercise the connection override.dotnet test csharp/PostgreSQLVSDoublets.slnx -c Release --no-build --blame-hang-timeout 2m: 7 passed, none skipped with PostgreSQL configured. Tests exercise all six stores, every benchmark operation, cleanup, sampling, statistics and output formatting.actionlint .github/workflows/rust.ymlpasses.experiments/run_benchmarks.shcompleted all 96 measurements and generated both languages’ reports and all four charts locally. See the README for setup and complete commands.Measurement scope: preparation and cleanup are excluded from both harnesses. PostgreSQL transaction begin/commit is outside the timed operations, matching the existing Rust workload; memory-mapped Doublets stores are not explicitly flushed to disk. Indexed read iterations query every background point, while writes use the active count. The parity tests cover this point-link benchmark workload. Local PostgreSQL validation uses version 16; CI uses the pinned 17.6 image.
The isolated CI run passed all test, benchmark and report jobs on commit
2396e66. The README tables and charts below use this run's PostgreSQL 17.6 measurements. Separate runner CPUs are identified in each table's provenance; these ratios apply to this workload and those machines.Final-commit CI also passed all eight jobs on
0142cb9, including both languages’ PostgreSQL/Doublets measurements and report generation. CodeFactor passed. The branch includes currentmainand the working tree is clean.Changes
.github/workflows/cpp.yml.github/workflows/rust.yml.gitignoreDocs/bench_csharp_100.pngDocs/bench_csharp_log_scale_100.pngDocs/bench_rust_100.pngDocs/bench_rust_log_scale_100.pngREADME.mdcpp/Benchmarks/AllBenchmarks.cppcpp/Benchmarks/Create.cppcpp/Benchmarks/Delete.cppcpp/Benchmarks/Each/All.cppcpp/Benchmarks/Each/Concrete.cppcpp/Benchmarks/Each/Identity.cppcpp/Benchmarks/Each/Incoming.cppcpp/Benchmarks/Each/Outgoing.cppcpp/Benchmarks/SetupTeardown.cppcpp/Benchmarks/Update.cppcpp/CMakeLists.txtcpp/LinksPSQL/Client.hcpp/LinksPSQL/ISQL.hcpp/LinksPSQL/PSQLBase.hcpp/LinksPSQL/Transaction.hcpp/conanfile.txtcpp/out.pycsharp/PostgreSQLVSDoublets.Tests/PostgreSQLVSDoublets.Tests.csprojcsharp/PostgreSQLVSDoublets.Tests/SameBehaviorTests.cscsharp/PostgreSQLVSDoublets.Tests/packages.lock.jsoncsharp/PostgreSQLVSDoublets.slnxcsharp/PostgreSQLVSDoublets/Benchmarks.cscsharp/PostgreSQLVSDoublets/DoubletsLinks.cscsharp/PostgreSQLVSDoublets/Harness.cscsharp/PostgreSQLVSDoublets/IBenchedLinks.cscsharp/PostgreSQLVSDoublets/PostgreSQLVSDoublets.csprojcsharp/PostgreSQLVSDoublets/PostgresLinks.cscsharp/PostgreSQLVSDoublets/Program.cscsharp/PostgreSQLVSDoublets/packages.lock.jsoncsharp/global.jsonexperiments/reproduce_report_ratio.pyexperiments/run_benchmarks.shrust/benches/bench.rsrust/benches/benchmarks/doublets/create.rsrust/benches/benchmarks/doublets/delete.rsrust/benches/benchmarks/doublets/each/all.rsrust/benches/benchmarks/doublets/each/concrete.rsrust/benches/benchmarks/doublets/each/identity.rsrust/benches/benchmarks/doublets/each/incoming.rsrust/benches/benchmarks/doublets/each/outgoing.rsrust/benches/benchmarks/doublets/update.rsrust/benches/benchmarks/psql/create.rs