Skip to content

fix(table_diff): resolve key columns against the table schema - #6070

Open
Anant-gif wants to merge 1 commit into
SQLMesh:mainfrom
Anant-gif:fix/table_diff_case_key_error
Open

Anant-gif wants to merge 1 commit into
SQLMesh:mainfrom
Anant-gif:fix/table_diff_case_key_error

Conversation

@Anant-gif

@Anant-gif Anant-gif commented Sep 14, 2026

Copy link
Copy Markdown

Description

Fixes #6067

Change

Rather than making each individual dict lookup case insensitive, this change reconciles the
casing once, at the point where the key columns are parsed. A new _resolve_column_name helper
maps each normalized key back to the column name reported by the engine for its table:

  • exact match on the schema first, so case sensitive engines that can hold both KEY and key
    keep working
  • otherwise a case insensitive match
  • if nothing matches, the normalized name is returned unchanged so a genuinely missing column
    still fails loudly as before

The source and the target are resolved against their own schema, since they can disagree on casing. Both the list form of on and the expression form are covered.

Everything downstream then uses a single spelling that matches the engine, so no other call site needs to be aware of casing. This mirrors SchemaDiff._original_column_name, (which is opt-in) which already maps lowercased column names back to the engine-reported spelling for the schema diff , the key column resolution is always on since a key that does not resolve makes the row diff impossible to run.

Test Plan

Added three tests in tests/core/test_table_diff.py, all of which fail on main and pass with this change:

  • test_data_diff_non_lowercase_key_columns: key columns stored as KEY1 / Key2, supplied in
    both matching and non matching casing, plus the single key column path which fails in a
    different place from the multi key path
  • test_data_diff_key_columns_with_differing_case_between_source_and_target: source and target
    store the same key columns with different casing, so each side must be resolved against its own
    schema
  • test_data_diff_non_lowercase_key_columns_in_on_condition: same scenario with an on
    expression instead of a list, covering the second branch of key_columns

Also ran this against a live BigQuery project with mixed-case identifiers to confirm the CLI flags resolve without errors.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

Signed-off-by: Anant <75747269+Anant-gif@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

table_diff: KeyError when -o key columns are not lower case on Bigquery

1 participant