Repository navigation
fix(android): read volume samples in native byte order - #466
darrenabridge wants to merge 1 commit into
Conversation
useTrackVolume's Android VolumeProcessor read 16-bit PCM from WebRTC's AudioTrackSink buffer with getShort() without setting a byte order. The buffer is a JNI direct buffer, which Java treats as big-endian, while the samples are little-endian, so every sample was byte-swapped and any non-trivial audio read as roughly -5 dBFS. Read through a duplicate() view in native order, which also leaves the sink's buffer position and order untouched. Co-authored-by: Cursor <cursoragent@cursor.com>
|
|
🦋 Changeset detectedLatest commit: 81d6d0e The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Devin Review found 1 potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| 2 -> audioData.getShort().toLong() | ||
| 4 -> audioData.getInt().toLong() | ||
| 1 -> samples.get().toLong() | ||
| 2 -> samples.getShort().toLong() |
There was a problem hiding this comment.
🟡 Quiet audio leaves track volume stale
When corrected samples produce zero, onVolumeCalculated emits a reading that useTrackVolume ignores. The track keeps its previous nonzero volume during near-silent frames.
Learn more
The processor emits normalized volume values for each audio buffer. The useTrackVolume listener updates state only when event.volume is truthy, so it discards zero. Correct native-order decoding can turn a low-level frame that previously decoded as nonzero into a zero RMS reading. The listener then preserves an earlier, louder reading.
Example: A 480-frame buffer containing one PCM16 sample of value 1 and otherwise zeros produces a zero RMS after native-order decoding. Before this change, swapping that sample to 256 produced a nonzero reading. If the last displayed volume was 0.5, the new zero event leaves it at 0.5.
Recommended fix: In useTrackVolume, check the event ID and tag independently of the numeric volume; accept zero as a valid reading. Add a transition test from nonzero to zero.
Was this helpful? React with 👍 or 👎 to provide feedback.
|
Hi @darrenabridge, can you sign the CLA? |
3 participants
Problem
On Android,
useTrackVolumereports roughly the same high level (~0.55, about −5 dBFS) for almost any audio, including quiet background noise. Only near-silent frames read low.BaseVolumeProcessor.onDatareads 16-bit PCM from theAudioTrackSinkbuffer withaudioData.getShort()but never sets a byte order. WebRTC hands the sink a JNI direct buffer (NewDirectByteBuffer), which Java treats as big-endian by default. The samples are little-endian, so every sample is byte-swapped. For anything above a couple of LSBs, the swapped values are effectively random across the 16-bit range, and the RMS of uniformly random 16-bit samples is 1/√3 ≈ 0.577 (−4.8 dBFS).The Android SDK already sets the order explicitly when it reads WebRTC audio buffers, for example
buffer.order(ByteOrder.nativeOrder())inMixerAudioBufferCallback.kt.Evidence
I logged the raw
useTrackVolumevalue for 20 seconds on a Samsung Galaxy S23 (SM-S911U1) in a normal room, on@livekit/react-native2.9.8, with no special audio settings. That gave 1,646 readings (converted to dBFS):More than half the readings sit in a single 3 dB band, and none are louder than −3.3 dBFS. Real audio doesn't pile up against a wall like that; byte-swapped samples do.
Fix
Read the samples through
audioData.duplicate().order(ByteOrder.nativeOrder()).duplicate()doesn't copy the audio: it gives the processor its own position and byte order on the same memory, so the sink's buffer is left untouched. That's also whymark()andreset()are no longer needed. The 8-bit case is unaffected, and the RMS and normalization math is unchanged.Scope
VolumeProcessor. iOS'sVolumeAudioRendererworks on float PCM viaAVAudioPCMBufferand isn't affected.FFTAudioAnalyzer, used byuseMultibandTrackVolume, also callsinputBuffer.getShort()on the sink buffer without setting an order, so it may have the same issue. I left it out to keep this change small.Testing
ci/androidlocally.Changeset included (
patch).