Add Accumulator: a striped long counter primitive as an alternative to LongAdder (perf toolbox)

[dougqh]

What Does This Do

Adds Accumulator, an enum-keyed, striped counter primitive that closes LongAdder.sumThenReset()'s documented non-atomic reset hazard: increments landing between sum and zero are silently and permanently lost with LongAdder; Accumulator.accumulateAndReset combines and resets each stripe under a single atomic getAndSet per counter, so nothing can land in the gap.

This has grown from a bare primitive into a small toolkit, in response to review and to trial a real caller (see below):

• Accumulator — lock-free AtomicLongArray-per-stripe striping, one inc/add write path, accumulateAndReset() (destructive drain) and sum() (non-destructive peek), typed Counts<E> views (get, plus, from). Lives in products/metrics/metrics-api (moved from internal-api, next to Counter), not internal-api — see AccumulatorVsCounterBenchmark's javadoc for the rule of thumb on when to reach for this over the existing Counter.
• Accumulator.RunningTotal — an opt-in, lock-guarded composition of a periodic destructive drain() (for reporting) and a non-destructive live() (for a diagnostic read, e.g. summary()) that share one lock, so a live read can never observe a torn state straddling a concurrent drain. Most callers don't need this (a drain-only reporter, or a live-only diagnostic, needs no coordination); it's provided as a separate type for the one recurring shape that does.
• StatsDCounterKey/StatsDCountReporter — the statsd-reporting glue: an enum constant carries its own metric name and tags, and StatsDCountReporter owns an Accumulator plus the periodic drain-and-report cycle, retrying whatever a client exception fails to deliver on the next flush rather than discarding it.
• Stripe count is oversized to ~2x available cores (floor 4) rather than exactly core count, to reduce stripe-collision probability under contention (see Accumulator.stripeCount() javadoc).
• JMH benchmarks: AccumulatorBenchmark (vs. LongAdder, a CHM-of-AtomicLong anti-pattern, and an unstriped AtomicLongArray, at low/high contention) and AccumulatorVsCounterBenchmark (a decision-support comparison against metrics-api's existing Counter, establishing when each is the right call).
• JOL footprint test (AccumulatorFootprintTest) compares retained bytes against N LongAdders fresh vs. under real contention.

Row-wide atomicity was dropped, deliberately, in the move to lock-free striping. The previous synchronized-stripe design let several counters be updated together atomically; the current AtomicLongArray design only guarantees atomicity per-counter — two different counters written by the same call are no longer guaranteed to be observed together by a concurrent drain. No caller in this stack (or the real one wired in below) depends on that cross-counter invariant, and the lock-free design wins on both raw throughput and multi-counter "bulk update" call sites despite giving it up (shared-stripe cache locality plus no lock beats per-object LongAdder fan-out plus a monitor). See the class javadoc for the full reasoning.

There is now a real caller: #12383 migrates TracerHealthMetrics (dd-trace-core) fully onto this stack — Accumulator + RunningTotal + StatsDCounterKey/StatsDCountReporter — replacing ~49 hand-tracked LongAdder fields and the previousCounts/countIndex diffing ceremony Flush used to need. That PR is stacked on this one and is what settled the design questions below with real numbers instead of argument.

Motivation

The realistic alternative to this primitive is N separate LongAdder fields, one per counter, each independently diffed against a hand-tracked previous value. That has problems this stack fixes:

• Correctness: LongAdder#sumThenReset() is documented as not atomic against concurrent updates — an increment landing on a cell after it's summed but before it's zeroed is silently and permanently lost. Accumulator.accumulateAndReset() closes that gap per-counter.
• Ergonomics: one enum declares the schema; one Accumulator/StatsDCountReporter replaces N independent LongAdder fields plus hand-rolled diffing against a previousCounts array.
• A live diagnostic read that doesn't race the periodic drain: RunningTotal gives summary()-style callers a value that's always live and never inconsistent with a concurrent Flush, without either call blocking the other beyond one short critical section.
• Reporting resilience: StatsDCountReporter retries whatever a statsd.count() exception fails to deliver on the next flush, instead of the drained delta being silently discarded for that interval.

Additional Notes

• Accumulator's fixed up-front cost ends up lighter than contended LongAdders once measured under the load it will actually see in production — see the JOL footprint test.
• Test plan:
  • ./gradlew :products:metrics:metrics-api:test — unit, footprint, and StatsDCountReporter tests pass
  • ./gradlew :products:metrics:metrics-api:jmhJar builds; JMH benchmarks (AccumulatorBenchmark, AccumulatorVsCounterBenchmark) run manually with real numbers captured in javadoc
  • ./gradlew :products:metrics:metrics-api:spotlessCheck clean
  • /techdebt and /perf-review run over branch changes — no findings
  • Real-caller trial (Migrate TracerHealthMetrics onto the Accumulator primitive (perf toolbox adoption) #12383, TracerHealthMetrics) exercised against the existing HealthMetricsTest/MetricsReliabilityTest regression suites with zero test edits needed

Contributor Checklist

• Format the title according to the contribution guidelines
• Assign the type: and (comp: or inst:) labels in addition to any other useful labels (none set yet)
• Avoid using close, fix, or any linking keywords when referencing an issue
• Update the CODEOWNERS file on source file addition, migration, or deletion
• Update public documentation with any new configuration flags or behaviors
• Once approved, use merge queue to merge the PR

Jira ticket: APMLP-1779

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.67 s	14.67 s	[-0.8%; +0.8%] (no difference)
startup:insecure-bank:tracing:Agent	13.65 s	13.67 s	[-1.0%; +0.7%] (no difference)
startup:petclinic:appsec:Agent	17.71 s	17.62 s	[-0.3%; +1.3%] (no difference)
startup:petclinic:iast:Agent	17.58 s	17.63 s	[-0.8%; +0.3%] (no difference)
startup:petclinic:profiling:Agent	17.39 s	17.15 s	[+0.3%; +2.5%] (maybe worse)
startup:petclinic:sca:Agent	17.52 s	17.53 s	[-0.9%; +0.7%] (no difference)
startup:petclinic:tracing:Agent	16.60 s	16.75 s	[-1.9%; +0.2%] (no difference)

Commit: 4be76af1 · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[datadog-datadog-prod-us1]

The new counter code has no confirmed defect. Its concurrency test can fail because it does not wait for the drain task, and its map benchmark does not measure the stated allocation path.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit fc2f55c · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fc2f55cd81

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[mcculls]

IMHO the benchmarks here should exercise the expected scenario of many threads calling increment, but only one thread periodically summing up the count. Currently they exercise many threads each calling increment and then the same incrementing threads all immediately summing up the count, which is not how it would be used in practice.

[amarziali]

Automated review — request changes

The per-stripe synchronization protocol appears sound when all access goes through the provided operations: writers and combine/reset use the same stripe monitor, preventing the LongAdder.sumThenReset() loss window.

I found two blocking issues:

1. The raw long[][] API does not bind storage to its enum schema. A different enum can be passed to inc/add and silently update the wrong ordinal. Exposing the arrays also makes the synchronization protocol conventional rather than enforceable.
2. The concurrent drain test does not wait for its drainer to finish before asserting. It can fail against a correct implementation and can leave an active infinite task after a timeout.

The performance evidence also needs revision before its conclusions can support this abstraction:

• The drain benchmark creates many concurrent drainers instead of modeling many writers and one rare reporter.
• The footprint experiment expands unlocked LongAdder instances rather than measuring the correctness-equivalent locked alternative.
• The stripe-collision explanation contains incorrect math and assumes a distribution not produced by threadId() & mask.
• The CHM measurement is pre-warmed and does not exercise the claimed allocation-under-lock path.

There is also an unmeasured deployment risk on JDK 21–23: virtual threads contending on these monitors during a drain may pin carrier threads. This should either be constrained in the contract or evaluated with a virtual-thread workload.

I recommend encapsulating the storage in an enum-bound owning type, repairing the drainer lifecycle test, and reshaping the measurements around the intended production topology before merging. Since there is no production caller in this PR, migrating one intended caller would also help validate the API and workload assumptions.

This was an automated, read-only review of head 89d980c0f829c6df51d22131aa35b760c529fdb4.

[datadog-datadog-prod-us1]

RunningTotal removes existing values before the first report. Counts also exposes shared key state, and the size test can fail on hosts with many processors.

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit 8e6e624 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

[dougqh]

@amarziali Following up here since this is where the actual change landed: I moved Accumulator from internal-api into metrics-api (this PR), right next to Counter. This addresses the module-layering concern you raised on #12383 -- StatsDCountReporter's Accumulator.Counts overload no longer reaches into internal-api from metrics-api, it's a same-module reference now.

I'm not entirely sure how we originally decided what belongs in metrics-api vs. internal-api, but this placement feels consistent with Counter already living there.

Making that move also made it clear that Counter is the fairer comparison point for Accumulator than the synthetic alternatives (LongAdder, CHM, etc.) already in AccumulatorBenchmark -- Counter's real implementation (StatsDCounter) is the thing Accumulator actually displaces on a hot path. So I added AccumulatorVsCounterBenchmark alongside it.

[dougqh]

Added two small things here that came out of the compensating-retry work on the stacked TracerHealthMetrics migration (#12383):

• @ThreadSafe on Accumulator and RunningTotal — both were already safe for concurrent use, just not documented as such.
• Counts.from(fromIndex) — zeroes every entry before the given index, so a caller that only partially delivered a drained batch downstream (e.g. a statsd client throwing partway through a report loop) can represent "what's left to retry" without re-counting the part that already made it. StatsDCountReporter.flush() uses this to retry undelivered counters on the next flush cycle instead of dropping them.

Both covered by new tests in AccumulatorTest.

[mcculls]

LGTM

[dd-octo-sts]

/merge

[gh-worker-devflow-routing-ef8351]

View all feedbacks in Devflow UI.

2026-09-22 13:37:32 UTC ℹ️ Start processing command /merge

---

2026-09-22 13:37:36 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 1h (p90).

---

2026-09-22 14:55:02 UTC ℹ️ MergeQueue: This merge request was merged