fix(driver-turso)!: the remote filter compiler refuses the JSON-column family and answers $contains by membership (#21178) - #21208
Conversation
… gate and answers $contains by membership
RemoteTransport.buildWhereSQL now refuses every operator in @objectstack/core's
JSON_COLUMN_INCOMPATIBLE_OPERATORS on a column the driver stores as JSON text,
with the shared jsonColumnOperatorRefusalText sentence, and answers
$contains / $notContains there through jsonMembershipPredicate('sqlite'), the
negated form NULL-safe. The JSON-column population is the driver's own
jsonFields registry, handed down through a new optional
RemoteTransport.setJsonColumnResolver that TursoDriver wires to the inherited
SqlDriver.isJsonColumn, beside setDeclaredValueShapeResolver.
A local/remote parity suite holds both faces to one answer over the whole
shared set, the membership pair, the bare and null equality spellings and a
scalar text-field control.
Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp
Co-authored-by: Claude <noreply@anthropic.com>
…-refusal seam table; add the changeset The seam enumeration requires every refusal method that goes through the withheld seam to have a row: jsonColumnOperator gets one per position (operator map, bare value, bare null), on a half-2 transport told that exactly one column is JSON-stored. The class is read from jsonColumnOperatorRefusalText, never spelled in the test. The bare-null position of buildWhereSQL now asks the gate too, as the local face's bare-value positions do. Claude-Session: https://claude.ai/code/session_017xfMoEjKUuSh2xYB8sCozp Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift Check8 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 7 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 7213aa27b0367f41d157a772dad1e41ddc8a7626 && git checkout 7213aa27b0367f41d157a772dad1e41ddc8a7626
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 0d421041d14cfc716e85e9bdd1da3d57d4eb3aa1 c67a136e94b876497ea177a6b49cd12fd3b3eefe && git checkout -B drift-repro 0d421041d14cfc716e85e9bdd1da3d57d4eb3aa1 && git merge --no-ff c67a136e94b876497ea177a6b49cd12fd3b3eefe
node scripts/docs-audit/affected-docs.mjs --json 0d421041d14cfc716e85e9bdd1da3d57d4eb3aa1 |
Contract reviewServed-tier: Inputs read: card #21178 (body and all five comments: triage 5933993910, claim 5934703166, os-dev-report 5934971162, claim amendment 5935032674, os-dev-report 5935977677); PR #21208 body, file list (5 files, +553/-0) and the net diff; the check-runs on the head. Verified against ① Derived judgmentsAccept-set changes, all on the REMOTE face of
Public-surface changes:
Pins:
Check-runs on ② Semver level
Clause-②: yes (narrowing) Read as ③ Boundary flags
Implemented-by: VERDICT: PASS Generated by Claude Code |
Fixes #21178
Clause-②: yes (narrowing)
What changes
RemoteTransport.buildWhereSQL(@objectstack/driver-turso, the filter compiler every remote-modeTursoDriverread and filtered write uses) now applies the JSON-column half of the filter contract exactly as the local face (SqlDriver, which local and replica mode inherit) does, from the same shared home in@objectstack/core:JSON_COLUMN_INCOMPATIBLE_OPERATORSis refused withINVALID_FILTER/ 400 before any statement runs: in an operator map (at the top of the per-operator loop, ahead of every arm), in the bare{ field: value }spelling, and in the bare{ field: null }spelling. The message and the withheld diagnostic arejsonColumnOperatorRefusalText's output, byte for byte the local face's, through this transport's existing withheld-refusal seam ([A of #7929] a spec-declared provenance mark set at both read-scope merge boundaries, so the driver can restore the author-facing cross-field diagnostic without re-disclosing policy #8220 provenance, diagnostic sink).$contains/$notContainson such a column answer membership throughjsonMembershipPredicate('sqlite', ...)(libSQL is SQLite); the negated form sits insidenullSafeNegative, so a row with no value satisfies$notContains. A scalar string column keeps the substring test.driver-turso; no remote-only dialect.The widening (for the contract review)
The transport keeps no schema, so it learns "is this a JSON-stored column" the way it learns every other declared fact, through an injected resolver. This adds ONE optional public method on the exported
RemoteTransportclass, and one type:RemoteTransport.setJsonColumnResolver(resolver: JsonColumnResolver): voidtype JsonColumnResolver: a function taking(object: string, field: string)and returningboolean, exported fromremote-transport.tsand NOT re-exported fromindex.ts, like its siblingsNonTextColumnResolverandDeclaredValueShapeResolver; it reaches the published.d.tsonly as the method's parameter type.TursoDriver's constructor wires it to the inheritedSqlDriver.isJsonColumn, besidesetDeclaredValueShapeResolver(constructor wiring only inturso-driver.ts; theupsertregions are untouched). In remote moderegisterRemoteFieldMetadatacallsregisterExternalObject, which fills the samejsonFieldsregistry the local face's gate and membership reading ask, so both faces read one population. ARemoteTransportdriven standalone without the resolver treats no column as JSON-stored and compiles as before. This is the fourth sibling ofsetFilterColumnSql,setNonTextColumnResolverandsetDeclaredValueShapeResolver; the last of those shipped as "New optional API" in commitfb386074's changeset. The seat's answer to the fork is on the card (claim amendment 5935032674, option A, open to the maintainer's veto).Why (measured at base
0b12b9ea, on the libsql SQLite stub harness)Over a
multiple: truelookup holding["u1","u2"]r1,["u2"]r2,["u3","u1"]r3,["u10"]r4 and null r5, the remote face answered, while the local face answered the right-hand column:$contains: 'u1'$notContains: 'u1'$nin: ['u1'],$ne: 'u1'INVALID_FILTER/ 400$eq,$in, bare equalityINVALID_FILTER/ 400$lt/$lte'u1'INVALID_FILTER/ 400$startsWith: '[',$endsWith: ']'INVALID_FILTER/ 400$icontains: 'u1'INVALID_FILTER/ 400{ owners: null }INVALID_FILTER/ 400jsonfield,$contains: 'u1'Pins
turso-local-remote-json-column-parity.test.ts(new): one fixture on BOTH faces; every case asserts remote equals local AND the canonical answer. It iterates the refused set fromJSON_COLUMN_INCOMPATIBLE_OPERATORSas it stands (27 members at base), assertingcode+status+ equality withjsonColumnOperatorRefusalText(...).message(never the literal words); the bare infix spellings in an operator map stay refused on both faces. Also: bare equality,nullequality ({f: null},$eq: null,$ne: null), the gate under$and/$or/$not, the population (atagsfield and ajsonfield),count(); membershipu1vs["u10"],u10,$notContainscomplement with the NULL row,tags, thejsonobject, bind alignment beside sibling predicates,count(); the scalar text-field control ($contains,$nin,$eq, bare and null equality,$startsWithunchanged);$null/$existsstill answered.remote-transport-compile-refusal-seam.test.ts: the enumeration requires every seam refusal method to have a row, sojsonColumnOperatorgets one per position (operator map, bare value, bare null) on a half-2 transport told exactly one column is JSON-stored. Policy / author / unmarked / merged-arm provenance all hold.Ablation (one-time proofs, via
scripts/ablation-replace.mjs, restore proven by blob hash and emptygit diff HEAD)The suite imports
./turso-driver.jsfrom source (vitest), so nodist/leg applies.turso-driver.ts(this.remoteTransport.setJsonColumnResolver(replaced by a no-op call; anchor 1 to 0, bloba9affc72tob90200ef): the parity suite goes 24 failed / 19 passed of 43, as predicted. Red: all 14 operator-map refusals, bare equality, depth, population, bothcount()pins,u1membership,$notContains, thejsonobject, bind alignment, null equality. Still green, as predicted: the 13 bare-infix rows (both faces refuse as an object comparand whatever the gate), the set check,u10andtagsmembership (substring coincides), the scalar controls, presence.remote-transport.ts(pushJsonMembershipanswers false): 5 failed / 38 passed, exactly the five membership pins whose substring answer differs; every refusal pin stays green.git diff HEADempty, blob equals HEAD in both legs.Filter-semantics compile surfaces, one conclusion per face
driver-sqlapplyFilterCondition(sql-driver.ts): already conformant — it is the contract; the parity suite's local column is its answer, anddriver-sqlite-wasmand local/replicadriver-tursoinherit it. Not edited.driver-tursoRemoteTransport.buildWhereSQL(remote-transport.ts): changed (this PR).service-analyticscompileScopedFilterToSql(read-scope-sql.ts): out of scope — another face; it already reaches the shared membership construct throughcontains-membership-sql.ts(importsjsonMembershipPredicate).service-analyticslowerAnalyticsWhere(strategies/filter-normalizer.ts): out of scope — another face, same shared construct as 3.formulamatchesFilterCondition(matches-filter.ts): out of scope — another face, and seat 2's in-flight #5930 step 4 (domain:engine): the engine-fed faces delete their hand-copied filter meaning (driver-sql, turso remote, memory query, mongodb, formula,having); the memory reference matcher retires (D6) #20822 group 3b owns it.objectqlapplyHaving/matchesHaving(having-filter.ts): out of scope — another face (it already imports the shared set and sentence), and seat 2's #5930 step 4 (domain:engine): the engine-fed faces delete their hand-copied filter meaning (driver-sql, turso remote, memory query, mongodb, formula,having); the memory reference matcher retires (D6) #20822 group 3b owns it.driver-memoryquery path (refuses the family since45ce12a4,filter-refusal.tsimports the shared set) anddriver-mongodbtranslateFieldOperators(mongodb-filter.ts): out of scope — other faces, not touched.Tests and gates (all on HEAD
c67a136eunless noted)pnpm --filter @objectstack/driver-turso exec vitest run --maxWorkers=2: 84 files passed, 2286 tests passed, 33 skipped (exit 0, viaos-verify-lock).pnpm --filter @objectstack/driver-turso typecheck: exit 0;tsc --listFilesincludes both edited test files.pnpm check:driver-conformance: before (base0b12b9ea) 50 covered / 0 DEBT / 0 exempt; after (c67a136e) 50 covered / 0 DEBT / 0 exempt. The ledger did not move.node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commandsderived 63 commands atc67a136e; all 63 were run and reconciled with--ran(each line recording its exit code): 62 run, 1 NOT MEASURED.pnpm check:dual-build-cjs-loadsexited 3 (PREREQUISITE NOT MET: it reads every package'sdist/, and a repo-wide build is CI's); narrowed in its place,requireofdriver-turso's CJS build andimportof its ESM build both load and exposesetJsonColumnResolver.check-plugin-teardown-shape --self-testandcheck:lean-entry-closurefirst exited 3 on prerequisites (a fixture commit outside the shallow clone; objectql'sdist/) and exited 0 after fetching that commit and building objectql's closure.check-adr-0087-registration --base origin/main: the changeset reads as declared-breaking with one disposition,not-required (no-migration-prescription);check-changeset-no-major: no major bump.eslint --no-inline-config --format jsonread 4 files, 0 errors, 0 warnings. All four are ineslint .'s population (--print-configresolves each). The config never enables type-aware linting (the resolvedparserOptionsareecmaVersionandsourceTypeonly, andeslint.config.mjssays so in its header), so this diff cannot move a verdict on any untouched file. The fullpnpm lintis CI's.cli,qa/dogfood,runtime,service-datasource) are declared to CI: none referencesRemoteTransport's API, and the one remote-mode consumer test (date-bucket-parity-turso) aggregates without a JSON-field filter.Changeset
.changeset/21178-remote-json-column-gate.md:@objectstack/driver-tursominor, BREAKING banner,Clause-②: yes (narrowing), ADR-0087 dispositionnot-required (no-migration-prescription), the new optional API named, and the migration text (write$containsfor "holds this member", an$orof$containsfor any-of,$notaround either for the exclusion,$null/$exists/$emptyfor presence).Siblings
json-column-operator-refusal.ts; this PR edits neither that file nor that branch, and its pins compare against the builder's output, so the reword cannot flip them. Whichever of the two lands second mergesmainand re-runs its pins.upsertregions ofremote-transport.tsandturso-driver.ts; this PR is region-disjoint. Whichever lands second mergesmain.Acceptance notes
{ f: null },$eq: null,$ne: null) althoughIS NULLis a well-formed presence question there; this PR matches it on the remote face (that is the parity invariant), and$null/$exists/$emptyremain the presence spellings on both. Noted, not filed: no declared contract says otherwise ($eqis a declared member of the set). Carrier: none.{ owners: { $in: [{ ... }] } }reads the JSON-column sentence remotely and the comparand sentence locally. Both areINVALID_FILTER/ 400.=,in, …) in an operator map are refused on both faces as an object comparand, with different wording per face; pre-existing, not JSON-specific.Generated by Claude Code