Repository navigation
fix: support REST branches with distinct catalog identities - #1109
Conversation
JingsongLi
left a comment
There was a problem hiding this comment.
Requirement fit: SUPPORTED. Implementation: FINDINGS at 7d6690a. Supporting branches with independent catalog UUIDs has real end-to-end value. The source freshness check needs to cover the target load as well, however, to preserve the existing rejection of a concurrently replaced table.
Validation: all 119 original REST tests pass, and an additional actual filesystem/Parquet/tag-born branch/HTTP probe correctly reads dev and main with distinct UUIDs and rejects an already-replaced source. The automatic merge with main 87fc22e is clean. A separate HTTP recreation probe reproduces the inline issue in 16/16 attempts; the same probe admits 0 replacement identities with the baseline switch implementation. A temporary post-target source UUID/path check rejects all 16 replacements while all 121 REST tests and probes pass. No other actionable finding established.
| let (current, mut table) = tokio::try_join!( | ||
| env.current_table_identity_checked(), | ||
| env.get_table(&identifier) | ||
| )?; |
There was a problem hiding this comment.
[P2] Recheck the source identity after loading the target
These concurrent GETs can observe different catalog generations. For an external table re-registered at the same path, the source request can return the loaded v1 UUID, then the target request return v2 after re-registration. Both location checks pass, and the returned Table/RESTEnv silently adopt the replacement UUID/schema. This even reproduces with a loaded main handle calling copy_with_branch("main"): using the real HTTP mock's set_table_uuid_after_calls(..., "replacement-v2", 1), all 16 attempts return v2 from a handle loaded as v1. The baseline switch never adopts v2 in the same probe. Stable external paths are a supported catalog registration model, so matching paths do not establish a shared generation. Please bracket the target resolution with another source UUID/path check, or provide an equivalent identity association; a temporary post-target check fixes the probe and preserves the distinct-branch behavior. Add a regression for re-registration during the switch, in addition to the already-stale-source test.
Purpose
REST catalogs may represent a branch as a distinct catalog entity with its own UUID.
Table::copy_with_branchcurrently compares the target branch UUID with the loaded handle UUID, which rejects that valid model even when both resolve to the same table location.Changes
Safety and compatibility
Catalogs that reuse the base table UUID for branch responses continue to work. A stale source handle is still rejected by UUID, and a target at another location is still rejected.
The REST response does not expose an explicit parent-table identity, so this keeps the existing table-location check as the branch membership boundary while moving UUID freshness validation to the source handle.
Verification
cargo fmt --all -- --checkcargo check -p paimon --all-targets --features fulltext,vortexcargo clippy -p paimon --all-targets --features fulltext,vortex -- -D warningscargo test -p paimon --test rest_catalog_test test_branch_write_copy_loads_catalog_schema_with_distinct_identity -- --exact --test-threads=1cargo test -p paimon --test rest_catalog_test test_branch_write_copy_rejects_replaced_source_identity_or_location -- --exact --test-threads=1