Skip to content

AVRO-3834: [python] Rescale decimals to the schema scale when encoding - #4020

Open
rioyu123 wants to merge 1 commit into
apache:mainfrom
rioyu123:fix/AVRO-3834-python-decimal-scale
Open

rioyu123 wants to merge 1 commit into
apache:mainfrom
rioyu123:fix/AVRO-3834-python-decimal-scale

Conversation

@rioyu123

@rioyu123 rioyu123 commented Oct 4, 2026

Copy link
Copy Markdown

What is the purpose of the change

Fixes AVRO-3834, reported by Steve Stagg.

The Python decimal encoder uses the value's digits without adjusting for the schema's scale. For example, Decimal("3.1") with scale: 2 is written as unscaled 31 and reads back as 0.31. Both bytes and fixed decimals are affected.

This change adjusts the unscaled integer by 10 ** (scale + exponent) before encoding it. Integer arithmetic keeps the result independent of the caller's Decimal precision. Values with too many fractional digits are still rejected as before.

This is limited to scale adjustment. It does not add writer-side precision validation or address the separate existing negative-zero encoding issue.

Verifying this change

This change added tests and can be verified as follows:

  • Added 22 cases checking both the encoded integer and a DatumWriter/DatumReader round trip, across bytes/fixed, positive/negative values, whole numbers, scientific notation, and scales 0, 2 and 4. The encoder runs under a low-precision Decimal context. These regressions fail on the original encoder.
  • Existing excessive-fraction rejection tests remain unchanged and pass.
  • On Linux with CPython 3.12: Python SDK suite completed with 611 tests, 2 skipped; whole-SDK ruff format --check, ruff check, and mypy avro/ passed.

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

AI disclosure: implementation and tests were prepared with OpenAI Codex; Claude Opus 5.5 reviewed the patch and drafted this description, which Codex edited. Review so far is by AI agents, not a human reviewer.

Preserve numeric values when the Decimal exponent differs from the schema scale for bytes and fixed encodings.

Generated-by: OpenAI Codex
@github-actions github-actions Bot added the Python label Oct 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant