Skip to content

ci: add memtrack walltime benchmarks to CI - #537

Open
not-matthias wants to merge 3 commits into
mainfrom
cod-3093-memtrack-data-format-improvements
Open

not-matthias wants to merge 3 commits into
mainfrom
cod-3093-memtrack-data-format-improvements

Conversation

@not-matthias

Copy link
Copy Markdown
Member

Adds a memtrack-benchmarks CI job that runs codspeed-memtrack track against a few representative workloads (ls, dd, tar), each with RSS-only and RSS+rmap variants, to catch regressions in memtrack's own overhead.

Runs both walltime (execution time) and memory (memtrack's own peak RSS/allocations while tracking each workload) modes against the same crates/memtrack/codspeed.yml config.

@codspeed

codspeed Bot commented Sep 16, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 17 untouched benchmarks
🆕 24 new benchmarks

Performance Changes

Mode Benchmark BASE HEAD Efficiency
🆕 WallTime memtrack track dd N/A 1.3 s N/A
🆕 WallTime memtrack track ls N/A 1.1 s N/A
🆕 WallTime memtrack track tar N/A 9.6 s N/A
🆕 WallTime memtrack track dd (with physical) N/A 1.5 s N/A
🆕 WallTime memtrack track ls (with physical) N/A 1.3 s N/A
🆕 WallTime memtrack track tar (with physical) N/A 9.7 s N/A
🆕 Memory memtrack track dd N/A 51 MB N/A
🆕 Memory memtrack track ls N/A 50.9 MB N/A
🆕 Memory memtrack track tar N/A 50.6 MB N/A
🆕 Memory memtrack track dd (with physical) N/A 51.1 MB N/A
🆕 Memory memtrack track ls (with physical) N/A 51 MB N/A
🆕 Memory memtrack track tar (with physical) N/A 50.9 MB N/A
🆕 WallTime encode_events_via_batched_channel N/A 151.7 ms N/A
🆕 WallTime encode_events_via_channel N/A 296.9 ms N/A
🆕 WallTime encode_events_worker_scaling[1] N/A 640.3 ms N/A
🆕 WallTime encode_events_worker_scaling[2] N/A 362.5 ms N/A
🆕 WallTime encode_events_worker_scaling[4] N/A 215.3 ms N/A
🆕 WallTime encode_events_worker_scaling[8] N/A 140.9 ms N/A
🆕 Simulation encode_events_via_batched_channel N/A 1.3 s N/A
🆕 Simulation encode_events_via_channel N/A 1.4 s N/A
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Comparing cod-3093-memtrack-data-format-improvements (829da2f) with main (f198706)

Open in CodSpeed

@not-matthias
not-matthias force-pushed the cod-3093-memtrack-data-format-improvements branch 2 times, most recently from 596e932 to 5f67554 Compare September 16, 2026 10:29
@not-matthias
not-matthias changed the base branch from main to cod-3492-memtrack-classic-tracepoints-cannot-attach-inside-the September 16, 2026 10:29
Base automatically changed from cod-3492-memtrack-classic-tracepoints-cannot-attach-inside-the to main September 16, 2026 10:49
@not-matthias
not-matthias force-pushed the cod-3093-memtrack-data-format-improvements branch 2 times, most recently from b7adccd to 3efb21e Compare September 16, 2026 14:49
@not-matthias
not-matthias marked this pull request as ready for review September 16, 2026 14:51
@greptile-apps

greptile-apps Bot commented Sep 16, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

The PR appears safe to merge with no outstanding correctness or repository-rule issues.

Summary

This PR adds CI coverage for memtrack overhead and optimizes its event pipeline by sending events in batches.

  • Runs RSS-only and physical-memory variants of representative ls, dd, and tar workloads in walltime and memory modes.
  • Uses the dedicated codspeed-macro runner for walltime measurements and pins both memtrack and the CodSpeed runner to the tested revision.
  • Batches ring-buffer events before channel delivery and updates downstream consumers accordingly.
  • Adds regression tests for partial-batch flushing during timeout, explicit drain, and shutdown.
  • Adds channel and worker-scaling benchmarks for the event encoder.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart LR
    PR[PR revision] --> Matrix{Benchmark mode}
    Matrix -->|walltime| Macro[codspeed-macro]
    Matrix -->|memory| Ubuntu[ubuntu-latest]
    Macro --> Setup[Install and configure memtrack]
    Ubuntu --> Setup
    Setup --> Workloads[Run ls, dd, and tar variants]
    Workloads --> Poller[eBPF ring-buffer poller]
    Poller --> Batch[Batch events]
    Batch --> Encode[Encode memtrack artifacts]
    Encode --> Results[CodSpeed benchmark results]
Loading

Reviews (2) · Last reviewed commit: "test(memtrack): bench the channel and wo..."

Comment thread .github/workflows/ci.yml Outdated
Comment thread crates/memtrack/src/ebpf/poller.rs
The poll thread sent one item per channel message, and std's mpsc allocates a
31-slot block per 31 messages, so a run that captured 704k events also
allocated 22.7k blocks purely to hand them over.

The callback now fills a shared 1024-item buffer and sends it whole. Partial
batches are flushed after every poll, after the drain-path consume before its
ack, and on the shutdown consume, so `drain()` keeps promising that all
pending entries sit in the channel once it returns.

Consumers take `Vec<T>` and flatten: the encode pipeline keeps its
`IntoIterator<Item = MemtrackEvent>` contract unchanged.
The encoder benches fed `encode_events` from a Vec, while production feeds it
an mpsc receiver, so the channel the events actually arrive through was absent
from the benchmarks. Adds a per-event and a batched channel bench that differ
only in batching, plus a worker sweep bounded by the core count, since asking
for more workers than pinned cores measures oversubscription rather than
scaling.
@not-matthias
not-matthias force-pushed the cod-3093-memtrack-data-format-improvements branch from 21ce920 to 829da2f Compare September 16, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant