Skip to content

fix(clickhouse): re-raise non-NOT_IMPLEMENTED errors from EXCHANGE TABLES - #6105

Open
wolfgang-aura wants to merge 3 commits into
SQLMesh:mainfrom
wolfgang-aura:fix/clickhouse-exchange-tables-reraise
Open

wolfgang-aura wants to merge 3 commits into
SQLMesh:mainfrom
wolfgang-aura:fix/clickhouse-exchange-tables-reraise

Conversation

@wolfgang-aura

Copy link
Copy Markdown

Description

Fixes #6087.

ClickhouseEngineAdapter._exchange_tables caught every DatabaseError from EXCHANGE TABLES and only acted on NOT_IMPLEMENTED. Any other error, such as ACCESS_DENIED, was dropped. _insert_overwrite_by_condition then dropped the temp table in its finally and reported success while the target kept its old data.

The except block now re-raises unless the error contains NOT_IMPLEMENTED, as proposed in the issue. The non-atomic rename fallback moves out one indent level; its statements are the same, and test_exchange_tables covers that path.

Test Plan

Following the test plan in the issue thread, in tests/core/engine_adapter/test_clickhouse.py:

  • test_exchange_tables_reraises_other_errors: execute raises an ACCESS_DENIED DatabaseError. The test asserts it propagates and that only the EXCHANGE TABLES call ran (no RENAME, no throwaway-table DROP).
  • test_insert_overwrite_by_condition_replace_exchange_error_propagates: on the non-partitioned replace path, the exchange raises. The test asserts the error leaves _insert_overwrite_by_condition and the temp table is still dropped by the finally.
  • test_exchange_tables (the NOT_IMPLEMENTED fallback) is not edited and passes in the same run.

Both new tests fail on unpatched main (2 failed, 34 passed) and pass with the fix (36 passed). ruff check and ruff format --check pass on both files. mypy reports nothing new on them compared with main. make fast-test was not run.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

Written with Claude (claude-sonnet-5-5) and reviewed by a second Claude instance under the Mailman harness.

…BLES

_exchange_tables caught every DatabaseError and acted only on
NOT_IMPLEMENTED, so any other failure was dropped and a FULL model
reported success over stale data.

Fixes SQLMesh#6087

Signed-off-by: wolfgang-aura <169568318+wolfgang-aura@users.noreply.github.com>
@mday-io
mday-io self-requested a review October 2, 2026 02:42
@mday-io mday-io self-assigned this Oct 2, 2026

@mday-io mday-io left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The production fix looks correct: unexpected exchange errors now propagate, while the existing NOT_IMPLEMENTED fallback is preserved.

One nonblocking test simplification: the overwrite-level regression already exercises the real _exchange_tables implementation and catches the swallowed error, plus verifies cleanup. Could we consolidate around that test instead of repeating the same failure at the helper level?

Keep the error-propagation assertion, assert no rename occurs, and verify that the only table dropped across the entire execution trace is the temporary table. Please leave the existing NOT_IMPLEMENTED fallback test intact.

Additionally, please confirm that you ran make style and make fast-test locally

…e-level test

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@wolfgang-aura

Copy link
Copy Markdown
Author

Thanks. Pushed a test-only change on top of your merge commit. The helper-level _exchange_tables test is gone, and the overwrite-level test now also asserts that the only DROP across the whole trace is the temporary table. The NOT_IMPLEMENTED fallback test is untouched.

I ran tests/core/engine_adapter/test_clickhouse.py locally (35 passed) and ruff format/check on the file. I did not run make style or make fast-test in full, since this Windows host can't build the full dev environment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ClickhouseEngineAdapter._exchange_tables silently swallows non-NOT_IMPLEMENTED swap failures → FULL model reports success over stale data

2 participants