fix(driver-memory)!: refuse the equality and ordering family on a declared JSON-stored field, in the SQL family's words (#21066) - #21159
Conversation
…lared JSON-stored field The shape gate in front of the query path and the analytics face now refuses a scalar comparison (the shared JSON_COLUMN_INCOMPATIBLE_OPERATORS set, and implicit equality) aimed at a field declared JSON-stored, with INVALID_FILTER / 400 and the shared refusal text from @objectstack/core. The withheld diagnostic goes to the face's logger at warn. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…write doors and the analytics face Flip the one per-element pin the suite carried ($ne beside $empty on a tags field) to a refusal pin, keep its composition through $null: false, and note the declared population on the scalar-column array suite. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…family refusal Also tag filterFieldDeclarations @internal: it is reachable from the analytics face, not a consumer contract. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
…mory-json-column-family
…mory-json-column-family
…mory-json-column-family
📓 Docs Drift CheckThis PR changes 1 package(s): 4 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
⛔ 6 release-owned page(s) also name something this change touched. These are read-only:
What this run could not see
Coarse fallback — 9 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin a53e51e3e00813bb5881a9194e1a82df6d53935b && git checkout a53e51e3e00813bb5881a9194e1a82df6d53935b
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 2488b98b48f51e2a1bc5b5e50fc1c206c9565291 2eab11e409f28a6d3be235dfabbd53e50bc124ef && git checkout -B drift-repro 2488b98b48f51e2a1bc5b5e50fc1c206c9565291 && git merge --no-ff 2eab11e409f28a6d3be235dfabbd53e50bc124ef
node scripts/docs-audit/affected-docs.mjs --json 2488b98b48f51e2a1bc5b5e50fc1c206c9565291
|
…narrowing) The seat's answer on the card: InMemoryDriver.filterFieldDeclarations is in the published .d.ts, so the declaration follows the precedent the analogous filterContainsTest set. Level and ADR-0087 marker unchanged. Claude-Session: https://claude.ai/code/session_01Ujdtvqs7ree7WyQmEDwEnG Co-authored-by: Claude <noreply@anthropic.com>
Contract reviewServed-tier: PR #21159 on card #21066, judged against triage's direction 5925785069, the claim 5927990211, the report 5929893869 and the seat's answer 5929927010. Inputs: the card thread, the PR body and file list, the net diff against the merge-base with ① Derived judgmentsEvery accept-set and public-surface change the diff implies, each named right or wrong:
② Semver level
③ Boundary flags
Implemented-by: VERDICT: PASS Generated by Claude Code |
Fixes #21066
Clause-②: yes (narrowing)
On a field the object declares JSON-stored (a
multiple: truefield,tags/multiselect/checkboxes, or a structured-JSON type such asjson),driver-memorynow refuses the scalar-comparison family thatdriver-sql'swhererefuses:$eq,$ne,$gt,$gte,$lt,$lte,$between,$in,$ninand implicit equality, whatever the comparand, at any depth. The answer isINVALID_FILTER/ 400 with the same message. The operator set and the sentence are read from@objectstack/core(JSON_COLUMN_INCOMPATIBLE_OPERATORS,jsonColumnOperatorRefusalText, homed by PR #21097). There is no third copy.$contains/$notContains(membership),$null,$existsand$emptykeep answering.What was wrong (H1, measured at
origin/main670680e93throughengine.find)A real
ObjectQLoverInMemoryDriver, #21004's six rows (ownersis amultiple: truelookup,tagsis atagsfield). Every row reproduces the card:whereowners$eq 'u1'd1,d3(per element)INVALID_FILTERowners$in ['u1','u9']d1,d3owners$nin ['u1','u9']d2,d4,d5,d6owners$gt 'u1'd1,d2,d3,d5tags$gt 'red'd3{ owners: 'u1' },$ne,$gte,$lt,$lte,$between,tags $eq,{ owners: null },$eq null,$ne null,$in [],$nin []owners $contains 'u1'/$notContains/$null/$exists/$empty,title $inThe engine hands the driver the operators as written, except
$ne/$nin. Those arrive inside the spec's null-safe lowering ($andof$orof$null: trueand the operator). The gate walks$and/$or/$not, so that shape is refused too.The analytics face (
MemoryAnalyticsService) answered the same per-element rows. Its SQL echo renderedowners = 'u1', which matches no row over the JSON text the SQL family stores. It now refuses inquery()andgenerateSql()alike.What changed
filter-refusal.ts: the shape gate (assertFilterConditionShape) takes an optionalFilterFieldDeclarations(isJsonStoredField,reportWithheld). It has two arms. Implicit equality on a declared JSON-stored field is refused as=, bare. Any operator in the shared set is refused AFTER the existing comparand-shape rules, which isdriver-sql's order (comparand gate, then column-type gate). So an array under$eqor a one-element$betweenstill gets its own refusal first.jsonStoredFieldOperatorErrorbuilds the error from the shared text: this package'sunsupportedFilterErrorenvelope, with the withheld diagnostic handed toreportWithheld(prefixedAt PATH:) before the throw.memory-driver.ts:convertToMongoQuerypassesthis.filterFieldDeclarations(object). The population isisJsonStoredField, the predicate$containsalready forks on (STRUCTURED_JSON_TYPESorisMultiValueField). So the fields where$containsasks membership are exactly the fields where the family is refused. The diagnostic goes to the driver's logger atwarn, the leveldriver-sqluses for its withheld filter diagnostics. That keeps the message's "the full diagnostic is in the server log" true here.memory-analytics.ts:normalizeFilterstakes the cube. It judges awherekey (a cube member) by the field it maps to on the cube's table, the same (table, field path) pairfilterContainsTestreads. Its diagnostic goes to the analytics service's own logger..changeset/21066-memory-json-column-family-refusal.md:@objectstack/driver-memoryminor, BREAKING banner,Clause-②: yes (narrowing), one ADR-0087 markernot-required (no-migration-prescription). No registered id covers a filter operator on a JSON-stored column. The one migration-registry entry that mentions json columns (cel-predicate-one-value-comparand-refused) is the CEL list-comparand surface, not this one.Hypotheses, measured
@objectstack/core'sjson-column-operator-refusal.ts, and both names are read. Before this changedriver-memoryhad NO withheld-diagnostic seam: every refusal it raises (the$null/$existsnon-boolean refusals included) names the field in the message, and nothing in the package logged a diagnostic. This change keeps the shared posture: the message names neither field nor operator, and the diagnostic goes to the server log.driver-sqldecides a JSON column fromjsonFields, filled fromJSON_COLUMN_TYPES.has(type) or isMultiValueField(field).JSON_COLUMN_TYPESisSTRUCTURED_JSON_TYPESplusMULTI_OPTION_TYPESplus the driver-internalobject/arrayaliases. Memory's population is the same predicate less those aliases and less a single-value media field on an unmoved deployment (both recorded onisJsonStoredField). On a schemaless direct call (an object never passed throughsyncSchema), nothing is judged. Every operator answers per element as before, asSqlDriver.isJsonColumnanswersfalsefor a table it was never told about. Pinned. A field declared SCALAR (text) that holds an array is not judged either.@objectstack/formula'sORDERING_OPERATORSdocblock does NOT declare a per-element reading for the query plane. It records a non-alignment ("driver-memory's read, a frozen test driver, compares a stored list element by element and keeps returning those rows ... declared on [finding] driver-memory's own reference matcher has no$fieldarm — a cross-field comparand (bare or withaddDays) reaching it is presumably compared as a literal object rather than resolved or refused (grep reading, to be measured) #15104"). [finding] driver-memory's own reference matcher has no$fieldarm — a cross-field comparand (bare or withaddDays) reaching it is presumably compared as a literal object rather than resolved or refused (grep reading, to be measured) #15104 is the$fieldcross-field reference card, shut asnot_plannedunder the driver-memory investment freeze. It rules nothing about the equality or ordering family on a stored list. So this is a formula-plane record of observed behaviour, not a query-plane contract, and no contract conflict stops the card. That docblock sentence goes stale on declared fields once this lands (see Acceptance notes).$startsWith/$icontainson a multi-valued lookup answer 500 on PostgreSQL and a wrong count on SQLite: the text operators other than$containsreach a JSON column unrefused and unruled #21009 widens the same shared set to the text operators. Both gates here read the set live, and the new suite iteratesJSON_COLUMN_INCOMPATIBLE_OPERATORSintersected with this driver's vocabulary, with a floor of the nine$-spellings. So once both land, memory refuses$startsWith/$endsWith/$icontainson these fields with no edit here, and the suite pins them. Whichever of the two lands second mergesmainand checks the other's members on its face. The suite's$containscontrol is outside [finding]$startsWith/$icontainson a multi-valued lookup answer 500 on PostgreSQL and a wrong count on SQLite: the text operators other than$containsreach a JSON column unrefused and unruled #21009's scope.Pin sweep
memory-20444-empty-operator.test.tshad{ tags: { $empty: true, $ne: null } }givingr2. It is now a refusal pin (code+status+ the shared message). The composition ($emptybeside a has-a-value sibling on one multi-value field) is kept through$null: false, which answersr2.driver-sql/SQLite answers that row too, and refuses the$ne: nullspelling with the same body (measured on the built driver).memory-matcher-scalar-comparand-array-value.test.tspins per-element answers on a column declaredtext. Those cells still hold, and a header note now says the population there is a scalar-declared column.packages/runtime's two ruled consumers). Neither filters a JSON-stored field. No otherINVALID_FILTERpin moves.Tests (final head
13407b76f)pnpm --filter @objectstack/driver-memory exec vitest run --maxWorkers=2: 70 files, 1703 passed. The first run after the implementation, before any test edit: 69 files, 1 red of 1613, the per-element pin flipped above.pnpm --filter @objectstack/driver-memory run typecheck: exit 0.tsc --listFilesincludes both edited test files.memory-21066-json-column-family-refusal.test.ts(89 tests):owners/tags/meta(json);$eq null,$ne null, the engine's$nelowering,$in [],$nin [], under$not, an$orbranch after a holding one,$eqbeside$contains);count/findOne/updateMany/deleteManyrefusing with the table untouched;At filter.$or[1].owners.$gte:diagnostic;query()andgenerateSql(), including thecube.memberspelling, plus its log line and a$containscontrol.node scripts/ablation-replace.mjs, wrap mode, run atde1fef341; the two later merges touched no file in this package; each restore proven blob == HEAD withgit diff HEADempty). The subjects are this package'ssrc, imported relatively, so nodistis involved:filterFieldDeclarations' predicate forced to() => false: 73 red / 37 green of 110 across the new file and 20444. The 17 green in the new file are exactly the answered controls, the declaration-boundary trio, the premise, the comparand-first case and the analytics$containscontrol.node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack(no paths; 7 changed paths, working tree clean) at13407b76f: 60 derived, 60 run, every one exit 0. Reconciled with--ran: "60 derived famil(ies) accounted for — 60 run, 0 NOT-MEASURED (a DERIVED zero — all 60 recorded an exit code and none of them is 3)". The same 60 also ran all-zero at the previous merge head5b75fe461. Atde1fef341,check:dual-build-cjs-loadsexited 3 (PREREQUISITE NOT MET, no dist yet); it measured on both later heads.node scripts/check-driver-conformance.mjs), before and after: byte-identical. 50 covered cells, 0 DEBT, 0 exempt. The shared matrix has no JSON-column or multi-value case-set, so this invariant is held by the per-package pins, not the matrix.Acceptance notes
InMemoryDriver.filterFieldDeclarationsis tagged@internal. It is not private only because the analytics face is another class.FilterFieldDeclarationsis not exported from the package root, but the method does appear in the published.d.ts. fix(driver-memory): $contains on a multi-valued or JSON-stored field is membership, on every face #20984 graded the analogous publicfilterContainsTestas a surface widening (Clause-②: yes (widening)). The seat graded it so (5929927010): the line isyes (narrowing), with the semver (minor) and the ADR-0087 marker unchanged.memory-driver.tsandmemory-analytics.tsare edited. The gate cannot see a declaration on its own, so the plumbing is the minimum the direction needs, and the analytics face calls the same gate. No open PR touched either file when read before the first edit.{ type: 'comparison', field: 'owners', operator: '=', value: 'u1' }) still answers per element on a declared field:d1,d3, measured on the built driver. No seam emits that form (the engine and the protocol hand a driver a FilterCondition), so it is reachable only by a direct driver call. Left alone.driver-sql's, and is literally untrue of this driver and of the engine's per-aggregation face. The prescription ($contains, an$orof$contains) is right on all three. Inherited as [finding] a per-aggregationfilter$ninon a multi-valued field counts the rows it was asked to exclude, and$incounts none, where the samewhereis refused 400: the aggregation evaluator has no JSON-column equality gate #21007 shipped it. [finding]$startsWith/$icontainson a multi-valued lookup answer 500 on PostgreSQL and a wrong count on SQLite: the text operators other than$containsreach a JSON column unrefused and unruled #21009 is the PR that next edits the shared home.@objectstack/formula'sORDERING_OPERATORSdocblock ("driver-memory's read ... keeps returning those rows") is now true only of undeclared objects. It is a comment, and no claim holds that file.turbo2.10.10 to 2.11.5 bump now onmain, every repo-scoped turbo run in an agent session appends a managed "turborepo-agent-rules" block (an HTML-comment-delimited section) toAGENTS.md. These includepnpm exec turbo run build,pnpm check:type-check-debt,check:query-options-erasureandcheck:slot-lookup. It happened repeatedly in this worktree and was restored each time, and every gate derivation above was taken on a clean tree; this PR does not touchAGENTS.md. Tracked as tooling: turbo 2.11.5 writes a managed block into AGENTS.md in every agent worktree; opt out with "agentGuidance": false in turbo.json #21146 (PR chore(turbo): opt out of the agent-guidance block in the root turbo.json #21151).Generated by Claude Code