fix(core)!: the JSON-column gate refuses $startsWith, $endsWith, $icontains and $like / $ilike as it refuses the equality family - #21165
Conversation
…er than membership $startsWith, $endsWith, $icontains and the staged $like / $ilike join JSON_COLUMN_INCOMPATIBLE_OPERATORS: on a JSON-stored column each matched the serialization as text (SQLite), failed at query time (PostgreSQL), or counted nothing (the per-aggregation filter). $contains / $notContains stay out: they are the membership pair. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
… dialect Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…ly on a multi-valued field, as where does Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
… unmoved on a JSON column Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…rs the text family other than membership Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…on-column-text-operators
…on-column-text-operators
…e refusal set's docblock Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 2 package(s): 4 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
What this run could not see
Coarse fallback — 34 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 8d095cbbfc917937b0c0e69e56687325c957778d && git checkout 8d095cbbfc917937b0c0e69e56687325c957778d
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin d34aa58a2affc87ded426dc6a326edb03534cb62 143f4ccd8f85d2fac1d5a56b3f8df070a766e160 && git checkout -B drift-repro d34aa58a2affc87ded426dc6a326edb03534cb62 && git merge --no-ff 143f4ccd8f85d2fac1d5a56b3f8df070a766e160
node scripts/docs-audit/affected-docs.mjs --json d34aa58a2affc87ded426dc6a326edb03534cb62
|
…on-column-text-operators
…ership A field the object declares multi-valued is stored as a JSON array, where every operator but the membership pair is refused. The search expander emitted $in on a label match and $icontains otherwise, so one such field in the resolved set failed the whole search. A label term now becomes one $contains per matched option value, and any other term $contains of the term; scalar fields are unchanged. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…lause beside the membership one Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…on-column-text-operators
…on-column-text-operators
Contract reviewServed-tier: PR #21165 (card #21009), head merging ① Derived judgments
② Semver level
③ Boundary flags
Implemented-by: VERDICT: PASS Generated by Claude Code |
Fixes #21009
Clause-②: no (narrowing)
Patch round 1 executes the seat answer on #21009 (5930243637):
$searchexpander matches a multi-valued field by membership, in this PR.not-required (no-migration-prescription).Head
143f4ccd8fmergesmainatd34aa58a2.What changes
@objectstack/core.JSON_COLUMN_INCOMPATIBLE_OPERATORSis the one setdriver-sql'swhereand objectql's per-aggregationfilterboth read since #21097. It gains the text operators other than the membership pair:$startsWith,$endsWith,$icontains;$like/$ilike, whichdriver-sqlanswers ahead ofFILTER_OPERATORS.On a JSON-stored column each now gets the
400the equality family already gets there:INVALID_FILTER, with the same withheld message, byte for byte;Nothing else on that gate moves:
$contains/$notContains(membership),$exists,$nulland$emptyanswer as before.sql-driver.ts,having-filter.ts,remote-transport.tsanddriver-memoryare untouched.@objectstack/objectql. The search expander (search-filter.ts,fieldClausesForTerm) matches a field the object declares multi-valued (isMultiValueField) by membership:$containsper matched option value, replacing the$inthat is refused there;tags, a multi-valued lookup), becomes$containsof the term.The declaration is read from the field map the engine already passes in: each entry is the object's whole field definition,
multipleincluded. No spec type moves. Scalar fields keep their clauses.The visible cost: to hit a multi-valued field, a term must now equal one of its members or match one of its option labels. SQLite used to match substrings of the serialized array as well, so
woodfound a row taggedredwood; it no longer does.Measured, before (
origin/main7a606a9a3) and afterThe
where/ per-aggregation filterMeasured through
POST /api/v1/data/:object/queryon SQLite and a private PostgreSQL 16.14. The fixture has six rows:ownersis a multi-value lookup (d1holdsu1, u2;d3holdsu3, u1;d5holds onlyu10).ownerswhere, beforewhere, beforefilter, before$startsWith: '['DATABASE_ERROR$startsWith: 'u1'u1$endsWith: ']'$icontains: 'U1'd1,d3,d5(d5holds onlyu10)$like/$iliked1,d3,d5where400 (this refusal); per-aggregation unchangedtagsfield gave the same results.jsonfield was already refused all seven text operators at the engine's declared-type door, which still answers first.titlecontrol answered the same rows before and after.$searchThe search was measured through the same route, with
search, on two objects:examples/app-todo'stodo_task.tags(select,multiple: true, in the auto-default set);tagsfield.main, SQLite / PostgreSQL (measured)Important,quick)$in) / 400meeting)red)n1andn2/ 500n1(member) andn2(scalar title)quick_win); note, a member (redwood)mainzebra); note, a substring of a member (wood)mainNo term answers 400 or 500 any more. The scalar controls (a
selectlabel, a text fold) are unchanged.H2, H3, H4
What a caller reads (H2). For
$startsWithonowners, the REST body is cut at the envelope's 500 characters:Per the seat answer, it stays byte-identical here, and #21067 owns the rewrite.
Turso remote (H3).
RemoteTransport.buildWhereSQLcompiles its own filters and has no JSON-column gate at all, for the equality family included. This PR leaves it alone, and it is reported for filing.driver-memory (H4). It answers each text operator per element. It is unchanged here; once #21066's shape gate reads this set, it refuses them too. The seat answer orders this PR ahead of PR #21159.
Pins
corejson-column-operator-refusal.test.ts:driver-sqlsql-driver-21009-json-column-text-operator-refusal.test.ts(new) is a dialect-cell suite: SQLite always, PostgreSQL and MySQL where provisioned, and the Temporal Conformance job provisions both. On a multi-value lookup and atagscolumn, each of the five gets:INVALID_FILTER/400throughfindandcount;The same file pins the scalar control (exact rows) and membership still answering.
driver-sqlsql-driver-json-column-operator-refusal.test.ts: the text family moves from the keep-working list to the refused list, on every face.driver-sqlsql-driver-17590-…andsql-driver-17343-…held the text family "unmoved" or "compiling" on a JSON column. They now pin the refusal on all three compilers, with the scalar column unmoved.objectqlengine-aggregate-filter-json-column-refusal.test.ts: the text family on the per-aggregation filter and its per-row floor; a structured-JSON field still meets the declared-type door first.objectqlsearch-filter.test.ts: membership clauses for a multi-valuedselect(label, partial label, no label), fortagsand for a multi-valued lookup, with the scalarselectand text controls.restdata-search-multi-valued-membership.test.ts(new) runs the table above throughPOST /api/v1/data/:object/querywithsearch, on SQLite and PostgreSQL, with MySQL where provisioned.restaggregation-filter-json-column-refusal.test.ts: the text family on both faces, with the per-aggregation body equal to thewheretwin's.The dogfood
search-conformance.ledger.tssummary now names membership for a multi-valued field. That half's HTTP proof is the REST file above, because no showcase object carries one in its search set.Reverse verification
Both fixes were committed before each ablation. Each restore leg proved the file's blob equal to HEAD and an empty
git diff HEAD.The core set. The ablation deleted the five new members (blob
8799778ctocbf406f9), rebuilt, and the preflight found the members--absent.The expander. The membership branch was disabled (
&& term === 'ablated-21009', blob44a09d96to61401d11), objectql rebuilt, and the preflight found the marker present.--absent)search-filter.test.tsINVALID_FILTERBoth moved in the expected direction: the pins turned red.
Tests and gates (head
143f4ccd8f)showcase-search.dogfood.test.ts) and the search-conformance ledgerThe full suites at
76d2fd5e8(mainbafb8c949merged; the last merge brought onlysql-driver.ts's sequence region anddriver-tursointo these packages) were:localdispatch-gates --commandswas derived at143f4ccd8fwith no paths. It named 70 commands, and 69 exited 0.check:dual-build-cjs-loadsexited 3 (PREREQUISITE NOT MET, a whole-workspace build): NOT MEASURED.--ranreconciled 70 derived, 69 run, 1 NOT MEASURED, 0 UNRUN. The derivation was stale by onemaincommit, a production-dependency bump (f3b16fc2f) that changespackage.jsononly.7a606a9a3) and after (143f4ccd8f)..tsfiles. The proof has three parts:eslint --print-config;--format jsonreports 12 files, 0 errors and 0 warnings;eslint.config.mjsnever enables type-aware linting, so no untouched file's verdict can move.@objectstack/coreand@objectstack/objectql, bothminor, BREAKING. It states the search cost. Its ADR-0087 disposition isnot-required (no-migration-prescription).Acceptance notes
SqlDriver.isNonTextColumn's docblock says "a text operator is legal against a JSON column". That now holds for the membership pair only. Carrier: none; it is outside this claim's surface.filter-text-operator-declared-type-refusednamesmultiselect/checkboxes/tagsand lookup ids as fields that must keep answering exactly as before. That over-claims once this lands. The seat records it as a spec-lane wording finding, filed at landing.Generated by Claude Code