Skip to content

fix: support REST branches with distinct catalog identities - #1109

Merged
JingsongLi merged 1 commit into
apache:mainfrom
shyjsarah:fix/rest-branch-distinct-identity
Oct 12, 2026
Merged

JingsongLi merged 1 commit into
apache:mainfrom
shyjsarah:fix/rest-branch-distinct-identity

Conversation

@shyjsarah

Copy link
Copy Markdown
Contributor

Purpose

REST catalogs may represent a branch as a distinct catalog entity with its own UUID. Table::copy_with_branch currently compares the target branch UUID with the loaded handle UUID, which rejects that valid model even when both resolve to the same table location.

Changes

  • Verify that the source REST handle still resolves to the UUID it was loaded with before switching branches.
  • Resolve the source identity and target branch concurrently.
  • Keep the existing location check so a branch resolving to another table location is still rejected.
  • Cover distinct branch identities and stale source branch handles in the REST catalog integration tests.

Safety and compatibility

Catalogs that reuse the base table UUID for branch responses continue to work. A stale source handle is still rejected by UUID, and a target at another location is still rejected.

The REST response does not expose an explicit parent-table identity, so this keeps the existing table-location check as the branch membership boundary while moving UUID freshness validation to the source handle.

Verification

  • cargo fmt --all -- --check
  • cargo check -p paimon --all-targets --features fulltext,vortex
  • cargo clippy -p paimon --all-targets --features fulltext,vortex -- -D warnings
  • cargo test -p paimon --test rest_catalog_test test_branch_write_copy_loads_catalog_schema_with_distinct_identity -- --exact --test-threads=1
  • cargo test -p paimon --test rest_catalog_test test_branch_write_copy_rejects_replaced_source_identity_or_location -- --exact --test-threads=1

@QuakeWang QuakeWang left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requirement fit: SUPPORTED. Implementation: FINDINGS at 7d6690a. Supporting branches with independent catalog UUIDs has real end-to-end value. The source freshness check needs to cover the target load as well, however, to preserve the existing rejection of a concurrently replaced table.

Validation: all 119 original REST tests pass, and an additional actual filesystem/Parquet/tag-born branch/HTTP probe correctly reads dev and main with distinct UUIDs and rejects an already-replaced source. The automatic merge with main 87fc22e is clean. A separate HTTP recreation probe reproduces the inline issue in 16/16 attempts; the same probe admits 0 replacement identities with the baseline switch implementation. A temporary post-target source UUID/path check rejects all 16 replacements while all 121 REST tests and probes pass. No other actionable finding established.

Comment on lines +906 to +909
let (current, mut table) = tokio::try_join!(
env.current_table_identity_checked(),
env.get_table(&identifier)
)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Recheck the source identity after loading the target

These concurrent GETs can observe different catalog generations. For an external table re-registered at the same path, the source request can return the loaded v1 UUID, then the target request return v2 after re-registration. Both location checks pass, and the returned Table/RESTEnv silently adopt the replacement UUID/schema. This even reproduces with a loaded main handle calling copy_with_branch("main"): using the real HTTP mock's set_table_uuid_after_calls(..., "replacement-v2", 1), all 16 attempts return v2 from a handle loaded as v1. The baseline switch never adopts v2 in the same probe. Stable external paths are a supported catalog registration model, so matching paths do not establish a shared generation. Please bracket the target resolution with another source UUID/path check, or provide an equivalent identity association; a temporary post-target check fixes the probe and preserves the distinct-branch behavior. Add a regression for re-registration during the switch, in addition to the already-stale-source test.

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit 819d5a5 into apache:main Oct 12, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants