Repository navigation
Conversation
Group consecutive records by supplied columns while preserving batch atomicity and schema inference. Fixes simonw#873.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #873.
upsert_all()can erase existing values when records supply different fields: updating one row's age and another row's name writesNULLinto the omitted fields. This also happens across batches because the column list accumulates. Group consecutive records by their supplied fields so each upsert updates only those fields, preserving record order and explicitNonevalues.Keep the original batch atomic and infer new column types from the whole batch when
alter=True. Ordinary inserts, replacement inserts, and positional input keep their existing behavior. Adjacent records with matching fields still share a bulk statement.Added regression coverage for both upsert implementations, multiple batch sizes, iterator input, trigger-visible update order, explicit nulls and defaults, rollback on invalid keys, hash IDs, compound keys, conversions, and mixed-type schema inference.
Validation on Windows, Python 3.13.2 / SQLite 3.47.1:
--sqlite-autocommit: 86 passed.git diff --checkpassed. The reported failure was reproduced on unpatched main before implementing the fix.Implemented and tested with OpenAI Codex.
📚 Documentation preview 📚: https://sqlite-utils--874.org.readthedocs.build/en/874/