Repository navigation
[kv] Keep KV snapshots readable by FRocksDB 6.20.3 for uniform keys - #4598
Merged
platinumhamburg merged 1 commit intoOct 11, 2026
Merged
Conversation
RocksJava 11.8.1 defaults uniformCvThreshold to 0.2, so RocksDB sets an is_uniform bit in the footer of index blocks with evenly spaced keys. FRocksDB 6.20.3 reads that bit as part of the restart count and fails the scan with "bad block contents". Pin the threshold to -1 next to the existing format_version and checksum pins. Extend the FRocksDB compatibility test with a snapshot of sequential big-endian keys, scanned in full by the FRocksDB reader. Closes apache#4597.
platinumhamburg
approved these changes
Oct 11, 2026
platinumhamburg
left a comment
platinumhamburg
left a comment
Contributor
There was a problem hiding this comment.
LGTM, thanks @morazow!
Contributor
Author
|
Thanks @platinumhamburg for review! Could you please merge once the CI is okay 🤝 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: close #4597
#4133 pins
format_version=5and thekCRC32cchecksum so that FRocksDB 6.20.3 clients can read KV snapshots. RocksJava 11.8.1 also defaultsBlockBasedTableConfig.uniformCvThresholdto0.2. With that default, RocksDB sets anis_uniformbit (bit 29) in the footer of index blocks whose keys are evenly spaced, such as sequential integer primary keys. FRocksDB 6.20.3 reads that bit as part of the restart count and fails the scan withbad block contents.Brief change log
RocksDBResourceContainer: setuniformCvThreshold(-1)next to the existing FRocksDB compatibility pins, so the bit is never written. Fluss does not configure an index search type, so the threshold has no effect on reads.RocksIncrementalSnapshotTest: addtestSnapshotWithUniformKeysCanBeReadByFrocksDB, which snapshots 10,000 sequential big-endianlongkeys and has FRocksDB scan every row. The existing 2-key test cannot catch this: an index block needs at least 3 restart keys before RocksDB checks the spacing. The snapshot, download and reader-process steps move into a helper shared by both tests.FrocksDBSnapshotReader: add a--count <rows>mode that does a full iterator scan, like the client'sSnapshotFilesReader.Tests
RocksIncrementalSnapshotTest#testSnapshotWithUniformKeysCanBeReadByFrocksDBfails withRocksDBException: bad block contentswithout the fix and passes with it.RocksIncrementalSnapshotTestandRocksDBResourceContainerTestpass.API and Format
No API change. Snapshot files no longer carry the
is_uniformindex block flag.Documentation
None.
Generative AI disclosure: Yes, Claude Code (Claude Opus 5.5).
🤖 Generated with Claude Code