Fail meaningfully on metadata a column cannot hold

[Shubham Padkonde (Shubham-Padkonde)]

Description

Adding metadata whose value is a nested dict or a sequence has failed with a raw sqlite3 binding error since #1444 was filed. On main at f6b9dd6:

new_data_set("ds", metadata={"a": {"b": 1}})

RuntimeError: Rolling back due to unhandled exception
  caused by: sqlite3.ProgrammingError: Error binding parameter 1: type 'dict' is not supported

which names neither the offending tag nor what to do instead. Measured across value types:

metadata value	before
{"b": 1}	sqlite3.ProgrammingError: ... type 'dict' is not supported
[1, 2]	... type 'list' is not supported
(1, 2)	... type 'tuple' is not supported
{1, 2}	... type 'set' is not supported

In the issue thread the ask was specifically to "fail in a meaningful way if you try to add nested fields as metadata", with the json-serialization workaround being the recommendation for the underlying use case. That is what this does — it does not try to pack/unpack sequences automatically.

validate_dynamic_column_data already rejects invalid tags and None values with clear messages, so this extends it with the same shape of check. After:

RuntimeError: Rolling back due to unhandled exception
  caused by: TypeError: Tag a has value of type dict. That is not a valid metadata value:
  a column stores a single SQLite value, so a nested dict or a sequence has to be
  serialized first, for example with json.dumps.

The error keeps the established house style: validation raises inside the transaction and the message reaches the caller through __cause__, which is how the existing tag and None checks behave and what error_caused_by in the test suite reads.

Why not a list of accepted types

QCoDeS registers sqlite3 adapters for NumPy types, so an allow-list would reject values that store correctly today. I checked what actually round-trips before the change:

value	stored
np.int64(3)	yes, as int
np.float64(2.5)	yes, as float
np.bool_(True)	yes, as bytes
np.str_("s")	yes, as str
np.array([1, 2])	yes, as NPY bytes

So the check asks SQLite itself whether the value binds, with a bare SELECT ? against a throwaway in-memory connection. Anything with a registered adapter — including all of the above — is still accepted, and nothing is written anywhere by the probe. All five still store after the change.

Related Issue(s)

Fixes #1444

Testing

Extended test_metadata in tests/dataset/test_dataset_basic.py, next to the existing bad-tag and None assertions, covering dict, list, tuple and set, plus an assertion that np.int64 metadata still round-trips.

Without the change the new assertion fails:

>           assert error_caused_by(e3, nested_value_msg)
E           AssertionError: assert False
E            +  where False = error_caused_by(<ExceptionInfo RuntimeError('Rolling back due to unhandled exception')>,
                 'Tag tag has value of type dict. That is not a valid metadata value')
FAILED tests/dataset/test_dataset_basic.py::test_metadata

With it:

tests/dataset/test_dataset_basic.py tests/dataset/test_sqlite_base.py
tests/dataset/test_sqlite_connection.py tests/dataset/test_dataset_loading.py
147 passed in 15.81s

ruff check and ruff format --check are clean on both touched files.

I could not run mypy on the change: it exits with an INTERNAL ERROR inside src/qcodes/dataset/data_set_protocol.py on this machine (mypy 2.3.1). That reproduces on clean main with my changes stashed, so it is not caused by this PR, but it does mean the type checking here has only been validated by CI rather than locally.

Backwards-compatibility

A value that previously raised sqlite3.ProgrammingError now raises TypeError, both wrapped in the same rollback RuntimeError. Nothing that stored successfully before is rejected now — that is what the NumPy table above is there to show.

Code catching the inner sqlite3.ProgrammingError specifically would need to catch TypeError instead, though that seems an unlikely thing to have depended on given the error was the bug being reported.

Documentation

The docstring of validate_dynamic_column_data now lists what it raises and why. The surrounding docstrings already said "None is not a valid value"; the new sentence extends that to nested values.

---

Disclosure: this change was written by Claude Code (Claude Opus 5) working as my agent, at my direction. The error messages and test output quoted above come from real runs in my local environment; I am accountable for the content of this PR and happy to iterate on review feedback.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.00%. Comparing base (faeb61d) to head (9c70ad6).
⚠️ Report is 31 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8518   +/-   ##
=======================================
  Coverage   71.99%   72.00%           
=======================================
  Files         305      305           
  Lines       32015    32025   +10     
=======================================
+ Hits        23050    23060   +10     
  Misses       8965     8965           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Shubham Padkonde (Shubham-Padkonde)]

Investigated both merge-queue removals. They are recorded as failed_checks, despite the PR-head checks passing.

The queue fails during dependency installation: pip install -c requirements.txt .[docs] reports ResolutionImpossible for the cf-xarray==0.11.3 constraint. The first queue attempt also fails installation before pytest/mypy; the subsequent missing-import and command-not-found errors follow that failed installation.

Evidence:

• Second queue attempt, docs installation: https://github.com/microsoft/Qcodes/actions/runs/36403319407/job/108866256757
• First queue attempt, test installation: https://github.com/microsoft/Qcodes/actions/runs/36402401611/job/108863103804
• The same constraint/install failure occurs on main at c4f9d16: https://github.com/microsoft/Qcodes/actions/runs/36402917042/job/108864769110

The PR does not change the dependency constraints. PyPI currently lists 0.11.3 as available and not yanked; I have not established why these runner installations cannot resolve it. Could the shared dependency-installation issue be checked before re-enqueuing? No code or workflow checks have been bypassed.

Investigation prepared with Codex assistance.

[Jens Hedegaard Nielsen (jenshnielsen)]

Shubham Padkonde (@Shubham-Padkonde) This is a pypi outage