Conversation
Contributor
Author
|
Closing for now. The checker change is correct for an exact renamed reader. But the decoder applies aliases twice (FastReaderBuilder/GenericDatumReader, then Resolver.resolve), so a swapped schema read by any other reader, e.g. one that adds a field, silently reverses the swap. Making the checker accept swaps would let those schemas register. The decoder would need additional fixing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of the change
When a reader swaps or rotates field names using aliases (for example
baliaseda, andaaliasedb),SchemaCompatibilitythrowsAvroRuntimeException("Reader record field b ... matches multiple fields in writer record schema"). The decoder reads the same schemas correctly. This fixes AVRO-4355.lookupWriterFieldcollects the writer field with the reader field's own name, plus each writer field named by one of its aliases, and throws when it finds more than one. The decoder instead goes throughSchema.applyAliases, which renames the writer'satoband itsbtoa, so each reader field reads the writer field its alias names. Record name swaps already work in both, because the checker compares type names at each location.The record-fields check now uses a new package-private
lookupWriterField(writer, reader, readerField), which can see the whole reader record. If the writer field with the reader field's own name is claimed by another reader field's alias, the reader field takes the writer field its own alias names. That is exactly whenapplyAliasesrenames that writer field away.Every other case falls through to the existing
lookupWriterField(Schema, Field), whose behavior is unchanged:Verifying this change
This change added tests to
TestSchemaCompatibility:GenericDatumReadergives the same pairing.TYPE_MISMATCH.The swap and rotation tests fail without the change. All
TestSchemaCompatibility*tests pass (82).Documentation