feat(events): summarize and buffer events on the recording thread (tier 1)

[abelonogov-ld]

Requirements

• I have added test coverage for new or changed functionality
• I have followed the repository's pull request submission guidelines
• I have validated my changes against all supported platform versions

Related issues

Fixes #281. DefaultEventProcessor.close() waited with no limit for its final flush, which on a slow network blocked the calling thread long enough to trigger an ANR. DirectEventProcessor.close() waits at most two seconds for that delivery and then lets it finish in the background.

Tier 1 of the Event Durability spec, "Tier 1 — the buffer". Tier 2 is stacked on this branch as #403; tier 3 will follow.

Describe the solution you've provided

DirectEventProcessor takes over from java-sdk-internal's DefaultEventProcessor in ComponentsImpl, and OutboundEventBuffer replaces the bounded queue that used to sit between the calling thread and the summarizer.

That queue is the problem this tier removes. An evaluation handed its event to an ArrayBlockingQueue and a dispatcher thread summarized it on the far side, so a burst of evaluations of untracked flags could fill the queue and displace the track and identify events an application actually asked to send. Summarizing on the recording thread means an evaluation of an untracked flag costs a counter increment and can never displace anything, because it never occupies a slot in the first place.

OutboundEventBuffer holds in full only the events that have to be sent one by one. It reuses java-sdk-internal's EventOutputFormatter and summarizer implementations rather than reimplementing them, so there stays exactly one definition of what an event looks like on the wire.

Two smaller changes ride along, both prerequisites rather than additions:

• java-sdk-internal moves to 1.12.0, which makes the summarizer and formatter types public so the buffer can use them from its own package.
• Diagnostic events are no longer sent while the SDK is offline or the application is backgrounded.

Describe alternatives you've considered

Reuse DefaultEventProcessor and raise its queue capacity. This moves the threshold without removing it. The displacement is a property of having a bounded handoff between the caller and the summarizer at all, and a render loop re-evaluating a tracked flag will reach any capacity you pick.

Keep the buffer inside java-sdk-internal's package to reach its package-private types. This worked, but it relied on a split package, which breaks Java modules and needed a Javadoc exclusion to build at all. Making the types public upstream was the durable fix, and is why the dependency bump is here.

Additional context

Naming. DirectEventProcessor and OutboundEventBuffer rather than an Android prefix: that prefix in this package is reserved for adapters over Android OS APIs, such as AndroidPlatformState and AndroidTaskExecutor, and neither of these classes touches one. "Direct" names the contrast with DefaultEventProcessor — nothing sits between the caller and the summarizer. DefaultEventProcessor was unavailable as a name because java-sdk-internal already exports it and this package imports from it.

Tests. DirectEventProcessorTest, EventProcessorBufferingTest and EventProcessorPrivacyTest share an EventProcessorTestBase, adding 22 tests across buffering, capacity, private-attribute redaction and diagnostics. The full unit suite is 752 tests, all passing locally.

Platform validation. The requirements box above is left unchecked deliberately. Behaviour is covered by the unit suite, but performance was measured on one physical device rather than across the supported API range.

Example app. Gains an "Eval+track+kill" button that evaluates a flag, tracks an event, flushes, and then kills the process five seconds later. That is the reproduction for the loss this work addresses, and it still loses the events on this tier — tier 3 is what makes them survive. Its LDClient.init call also moves off the deprecated three-argument overload.

---

Note

Overview
Replaces the Android SDK’s analytics pipeline wrapper around java-sdk-internal’s DefaultEventProcessor with DirectEventProcessor and OutboundEventBuffer, so flag evaluations are summarized on the recording thread and only full-fidelity events consume buffer capacity. This removes the old bounded inbox queue that could drop summaries or crowd out track/identify during evaluation bursts.

ComponentsImpl now builds DirectEventProcessor with a dedicated diagnostics posting thread, a 2s close budget for final delivery (then continues in the background), and stricter shutdown/offline rules documented on EventProcessor. EventProcessorBuilder.capacity clamps values below 1 to 1. launchdarklyJavaSdkInternal is bumped to 1.12.0 so summarizer/formatter types can be reused from the new buffer.

The example app switches to blocking LDClient.init(..., INIT_WAIT_SECONDS) instead of awaiting a Future. A new internal test-app module (dexcount kept there) adds an Eval+track+kill harness for unsent-event scenarios; example drops dexcount. .gitignore now ignores local.properties without a leading slash.

Tests: large new suite (DirectEventProcessorTest, buffering/privacy/flags tests, shared EventProcessorTestBase) covering atomic record/flush, capacity, diagnostics, and redaction through the new path.

^{Reviewed by Cursor Bugbot for commit 15d3ffa. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Stale Bugbot comment from a previous run.

[tanderson-ld]

This went through a multi-perspective review (general / security / adversarial), with the java-sdk-internal 1.12.0 sources as the comparison baseline. The design fundamentals held up well under adversarial scrutiny — summarizer locking, capacity enforcement under concurrency, deadlock-freedom, diagnostics state transitions, and wire-format/privacy parity were all attacked and survived — but two proven error-path regressions versus DefaultEventProcessor need fixing before this merges, and a third cluster ties into an open field issue. Inline comments mark the specific spots; summary of the findings below.

Blocking

1. A single unserializable event permanently wedges all analytics delivery. OutboundEventBuffer.drain() restores the summarizer and never clears events when serialization fails, so a deterministic failure re-fails every subsequent flush — periodic, manual, and close()'s final delivery — while the buffer saturates and silently drops all new identify/track events and the restored summaries grow without bound. Reachable from the public API today: trackMetric(..., Double.NaN) (no finiteness check on the path; Gson's strict writer rejects it) and boolVariation(null, ...) from Java (null key poisons the summarizer itself). DefaultEventProcessor cleared the outbox before serializing: it lost one batch and recovered on the next flush. Every failure this catch block can see is deterministic (a BAOS-backed writer can't fail transiently), so the restore can only ever produce a permanent wedge — suggest discard-with-loud-log on serialization failure, or per-event quarantine. A failing regression test is available (ProofPoisonedBufferRecoveryTest).

2. An Error during a scheduled run silently and permanently stops all flushing. guarded() catches Exception, not Throwable; an Error escaping a periodic run (realistic: StackOverflowError from deeply nested LDValue data, OutOfMemoryError growing the payload stream) makes the executor cancel the task with nothing logged, and enableOrDisableTask returns the dead-but-non-null future forever, so no state transition revives it. DefaultEventProcessor installed an uncaught-exception handler that logged loudly and shut down in an orderly way; there's no counterpart here. Suggest catch (Throwable) in guarded() plus replacing a future whose isDone() is true. Failing regression test available (ProofFlushTaskSurvivesErrorTest).

3. close() blocking and teardown — this is #281's failure mode, inherited and extended. delivery.get() is unbounded, and because all sends now share the single scheduler thread, close's delivery can wait FIFO behind an in-flight send and a queued diagnostic POST: ~3 failed sends ≈ 2 minutes at default timeouts, versus ~82 s for the old close (which never waited on diagnostics). Issue #281 is exactly this ANR shape on the current release line (main thread parked in postMessageAndWait under LDClient.close()). Separately, scheduler.shutdown() → eventSender.close() doesn't quiesce the executor: shutdown() lets queued tasks run, so a delivery enqueued by a racing setOffline(false) (its submit doesn't re-check closed), or a still-running delivery abandoned by an interrupted wait, executes against the closed sender. Suggested as one change: bounded get(timeout) with a logged give-up, shutdownNow() + bounded awaitTermination() before eventSender.close(), and a closed re-check before the setOffline catch-up submit. (Note: timed get everywhere — untimed get on a task discarded by shutdownNow() hangs forever.) With tier 3 durability this wait stops being load-bearing entirely; bounding it now would let this work close out #281 rather than carry it forward.

Strongly recommended

4. Serialization runs inside the monitor every evaluation contends on. drain() holds the buffer lock across the full JSON encode (logger.warn on the drop path is under it too), so recording threads — often the main thread — block behind a flush's encoding: measured ~22 ms cold for 100 events, and the work scales with accumulated summary contexts, not capacity. Fix carefully: the wide lock is what makes the failure path atomic today (restoreTo replaces rather than merges), so snapshot-under-lock/encode-outside needs a merging restore upstream first.

5. capacity no longer bounds event-subsystem memory. The per-context summarizer is unbounded (one entry per distinct context, full attributes retained) and grows from construction while initiallyOffline. Pre-existing in degree, but the inbox backpressure that used to shed bursts is gone, and finding 1 turns "bounded by outage duration" into "bounded by process lifetime." Cap it (counting overflow into droppedEventCount) or state explicitly that a later tier owns it.

6. An evaluation's summary/full/debug writes are three separate lock acquisitions. A flush can interleave between them (summary in payload N, feature event in N+1), and a close() interleaving can deliver the summary while silently stranding the feature event. One buffer.record(event, requireFullEvent, debugEvent) call restores the atomicity the old single-dispatcher design had, and cuts hot-path lock traffic 3×.

7. A diagnostics period can be destroyed in exactly the way the comment says it can't. sendDiagnosticStats checks suspension, then destructively reads (getAndClearDroppedCount, createEventAndReset), then sendDiagnosticEvent re-checks suspension and can drop the built event — an offline/background transition in that window discards the period's stats and skews dataSinceDate. Drop the inner re-check for the stats path or re-inject on suppression.

8. capacity(0) now silently drops every full-fidelity event. The >= 0 ? capacity : 1 normalization is copied from EventsConfiguration (whose javadoc says zero-or-negative → 1), but on main new ArrayBlockingQueue<>(0) crashed init loudly, masking the off-by-one; that mask is gone with the inbox. Use > 0, and consider a builder-level floor.

9. Test gaps sit in precisely the new risk areas. diagnosticInitEventIsSentWhenComingOnline's only-one-init assertion never exercises the guard (the fixture makes setInBackground(false) a no-op transition); diagnosticInitEventIsNotSentWhileOffline waits for the init to be delivered before going offline; no test flushes concurrently with recording (the one race this design introduces, and the regression test a future lock-narrowing needs); and the injectable constructor is never used, leaving 401/mustShutDown, dropped-count-to-diagnostics accounting, and periodic stats uncovered. The two Proof* tests above are written as regression tests (they pass once 1 and 2 are fixed) and are available to include.

Worth stating what's good here too: privacy redaction is byte-for-byte parity with genuinely thorough new tests; capacity accounting, sampling, debug-event TTL, and wire ordering all verified equivalent; close() fixes a scheduler-thread leak the old wiring had on every init/close cycle; and blockingFlush() now actually matches its documented semantics.

[tanderson-ld]

I don't think this should be committed. This example isn't meant to be about edge cases we are expected to handle but more about public API usage.

[abelonogov-ld]

Moving into separate test-app

[tanderson-ld]

In the event processor on main, the flushing thread that handled events was separate from what handled diagnostics. Now it is possible for a slow diagnostics task to delay the event flushing task. This shouldn't be possible as diagnostics is a secondary concern to event reporting throughput. May just be worth mirroring the separation on main to maintain parity.

[abelonogov-ld]

Diagnostic Events now use independent executor as in Java Server

[tanderson-ld]

It looks like only closed actually needs to be an atomic boolean? Doesn't hurt / cost too much for them to be, just pointing it out.

[abelonogov-ld]

lastKnownPastTime will matter in tier-3 other 2 fixed

[tanderson-ld]

close() can block ~50% longer than the wait that's already generating field ANRs (#281) — suggest bounding it

close() waits on delivery.get() with no timeout (DirectEventProcessor.java:201). Since the single-threaded scheduler runs every send FIFO, that wait covers, worst case: the currently-running send (cancel(false) doesn't stop a body already executing — and that body can now be a diagnostic POST, not just analytics), any queued diagnostic-init send from a recent online/foreground transition, and then the final delivery itself. At default config each failed send costs ~41 s (2 attempts × (10 s connect + ~10 s read) + 1 s retry in DefaultEventSender), so the stacked worst case is ~2 minutes, against ~82 s for the code this replaces — DefaultEventProcessor.close() waited only on analytics flush workers (diagnostics ran on sharedExecutor and were never on the close path), and its 1-slot payload queue capped the backlog at two sends. Even the mildest single-timeout case (~10 s) exceeds the ANR threshold when close() runs on the main thread.

To be clear about the half this PR gets right: the old close could also return instantly under inbox pressure, silently dropping the final flush — this PR's "the caller is entitled to assume the events made it out" semantics and interruptible wait are genuine improvements. The problem is only that the wait is unbounded and its worst case grew.

This matters beyond theory: #281 (open) is this exact failure on the current release line — main thread parked in postMessageAndWait's semaphore under LDClient.close(), at scale, on Android 14 devices with half-open networks. This PR inherits that failure mode and lengthens the chain by moving diagnostic sends onto the same thread as the close-path delivery.

Suggestions, separable:

1. delivery.get(timeout, unit) with a logged give-up — a few seconds is a defensible budget for a last-chance delivery from a lifecycle callback.
2. Keep diagnostic sends from queueing ahead of close's delivery (or cap diagnostics at a single attempt, no retry — they're best-effort telemetry).

[abelonogov-ld]

Both are done

[tanderson-ld]

I don't think shutdown guarantees other race conditions haven't scheduled work (e.g. a simultaneous thread in setOffline).

[abelonogov-ld]

It is become different after diagnostic separation work

[tanderson-ld]

Don't you want to be willing to disable tasks independent of if the instance is closed?

[abelonogov-ld]

Good point. Cancelling no longer depends on the closed check: updateScheduledTasks now uses closed only to decide whether to start work, and close() cancels by calling it under stateLock instead of doing its own cancel calls.

[tanderson-ld]

Having closed checks throughout many functions of a class that can run in a multithreaded environment is usually an indication of code smell. The closed checks may prevent you from doing work that was not desired, but they don't protect against two threads getting past those closed checks, so you end up needing proper locking anyways. There appear to be a few of these race conditions in this code.

Closed checks can work nicely in a single threaded region or single isolate language though.

[abelonogov-ld]

Agreed, and I cut them down. Two entry-point checks in setOffline and setInBackground were redundant and are gone

[tanderson-ld]

serialization happening while holding class lock via due to synchronized.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 38337ab. Configure here.}

[tanderson-ld]

I am just now realizing the context hash done by !countedContexts.contains(event.getContext()) and by summarizeEvent for equality is not insignificant when compared to serialization of events. This PR is the first time LDContext hashing is being done on the hotpath.

Based on my local benchmarking, for the canonical single-context Android app, hashing overtakes serialization at 5.3 evaluations per 30-second window — one eval every ~5.6 seconds. Any app that evaluates a flag even once per screen render is thousands of times past the crossover where context equality hashing dominates CPU.

Have you investigated / measured this component of CPU usage?

[tanderson-ld]

Should read false if this event was not counted