Skip to content

fix: reject transform renames that collide column names - #879

Closed
harsh-thakkar7 wants to merge 1 commit into
simonw:mainfrom
harsh-thakkar7:fix/transform-reject-duplicate-rename-targets
Closed

harsh-thakkar7 wants to merge 1 commit into
simonw:mainfrom
harsh-thakkar7:fix/transform-reject-duplicate-rename-targets

Conversation

@harsh-thakkar7

@harsh-thakkar7 harsh-thakkar7 commented Sep 24, 2026 •

Copy link
Copy Markdown

Table.transform(rename={"b": "a"}) on a table that already has both a and b silently destroyed column b and all of its data:

>>> from sqlite_utils import Database
>>> db = Database(memory=True)
>>> db["t"].insert({"a": 1, "b": 2})
>>> db["t"].transform(rename={"b": "a"})
>>> list(db["t"].rows)
[{'a': 1}]   # 'b' column and its value are gone

The rebuilt table was constructed from dict(new_column_pairs) in transform_sql(), so two columns mapped to the same target name collapsed to one — the earlier entry (with its data copy) was silently dropped from the CREATE TABLE and from the INSERT INTO ... SELECT copy.

Fix

After computing the transformed column list, detect target names that would appear more than once and raise TransformError before any SQL executes, leaving the table and its data untouched:

Cannot transform table 't': renaming or retyping columns would result in duplicate column name 'a'. No changes have been applied to this table.

A legitimate name swap (rename={"a": "b", "b": "a"}) and reusing a dropped column's name (e.g. drop=["a"], rename={"b": "a"}) are both still allowed.

Tests

Added to tests/test_transform.py:

  • test_transform_rename_cannot_lose_data_by_colliding_columns — renaming b→a raises TransformError and the original rows and schema are untouched (this failed before the fix by silently dropping the column).
  • test_transform_rename_two_columns_to_same_name_errors — two columns renamed to one target name also raise.
  • test_transform_can_swap_two_column_names — a straight a↔b swap still works.
pytest -q tests/test_transform.py   # 115 passed
pytest -q                           # 1500 passed, 16 skipped

📚 Documentation preview 📚: https://sqlite-utils--879.org.readthedocs.build/en/879/

Renaming a column onto the name of another existing column silently
collapsed the two into one and discarded the source column's entire
dataset, because the new CREATE TABLE column list was built with
dict(new_column_pairs) and the earlier entry was overwritten. Detect
duplicate target names and raise TransformError before any SQL is
executed, leaving the table untouched.
@harsh-thakkar7

Copy link
Copy Markdown
Author

Closing this — I don't think this change is mature enough to land right now. Thanks for the project!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant