Repository navigation
wierd & dangerous responses to compress() #199
Description
Activity
I understand this pain, but snappy-java has an API for knowing the max compressed length (this can be larger than the input data size!):
https://oss.sonatype.org/service/local/repositories/releases/archive/org/xerial/snappy/snappy-java/1.1.7/snappy-java-1.1.7-javadoc.jar/!/org/xerial/snappy/Snappy.html#maxCompressedLength-int-And also
byte[] Snappy.compress(byte[] input)is a safe-API, which internally calls maxCompressedLength method, allocates the necessary amount of buffer, and truncate the resulting buffer. But this incurs additional memory copies.I'd like to keep some methods unchanged for optimum performance. Adding some warning is good to have as yo suggested.
Comments in JavaDocs is a good idea since these popup in most IDEs. The important thing I guess is to point out that mis-use causes JVM death or data loss where that is a risk.
Fixed in #739 (merged to main), which adds Java-side validation before every native call. Passing an undersized output buffer or an out-of-range offset/length to compress() now throws IllegalArgumentException instead of crashing the JVM. The fix will ship in the next release (1.1.10.9).
It seems snappyjava lib is not doing bounds checks, when I load the library and run compress() with incorrect size I can crash the JVM, testing this with Windows on IBM JVM. I've not tested other libraries.
Also its possible that compress() does not reduce the size of the data, when that happens we don't get an Exception thrown. The method returns a value greater than the destination byte[] length and crops the data. This is kind of acceptable in the world of C++ but not what a Java dev would expect. This "feature" should probably be documented in the methods that might not work.