Skip to content

[python][ray] Rebase exact-range self-merge updates - #9682

Draft
XiaoHongbo-Hope wants to merge 1 commit into
apache:masterfrom
XiaoHongbo-Hope:codex/fix-partitioned-self-merge-rebase
Draft

[python][ray] Rebase exact-range self-merge updates#9682
XiaoHongbo-Hope wants to merge 1 commit into
apache:masterfrom
XiaoHongbo-Hope:codex/fix-partitioned-self-merge-rebase

Conversation

@XiaoHongbo-Hope

Copy link
Copy Markdown
Contributor

Purpose

Fix self-merge retries when a concurrent compaction replaces an anchor file but preserves the same (partition, bucket, first_row_id, row_count) range. The old fast path found no files requiring physical rewrite and returned without advancing check_from_snapshot, so the subsequent commit still failed against the compacted-away anchor.

Changes

  • Advance staged update messages to the latest checked snapshot even when no file contents need rewriting.
  • Run the existing conditional VARIANT migration regression with content_key as an actual partition key.
  • Extend the test compaction helper to preserve row-id ranges per partition and bucket.

Tests

  • python -m unittest pypaimon.tests.ray_row_id_conflict_rewriter_test pypaimon.tests.ray_data_evolution_merge_into_test (140 passed)
  • flake8, py_compile, and git diff --check passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant