fix(objectql)!: a no-operator object beneath a relation, structured-JSON or undeclared id column is refused INVALID_FILTER / 400 on every driver (#20745) - #20781
Conversation
…tured-JSON or provisioned column at the engine door (WIP) Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…, live dialects and InMemoryDriver (WIP) Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…; spec prose, docs and changesets Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
…te cell is a measured reading Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
… engine's own refusal Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift CheckThis PR changes 2 package(s): 21 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: ⛔ 5 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails. What this run could not see
Coarse fallback — 138 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 870fcbb63c7b6939f306f79d8f31c36188595b71 && git checkout 870fcbb63c7b6939f306f79d8f31c36188595b71
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 1bcba27d2e73242c6169d24e12031b3a9f52b44d ee17b18fdec9355e633b8f651518d2f30d98a984 && git checkout -B drift-repro 1bcba27d2e73242c6169d24e12031b3a9f52b44d && git merge --no-ff ee17b18fdec9355e633b8f651518d2f30d98a984
node scripts/docs-audit/affected-docs.mjs --json 1bcba27d2e73242c6169d24e12031b3a9f52b44d
|
Contract reviewServed-tier: PR #20781 for card #20745, branch Check-runs on the head, the newer of two readings this round (this comment posted 2026-09-30T05:31Z): 34 runs, 27 ① Derived judgmentsEvery accept-set and public-surface change the diff implies, each judged:
② Semver level
③ Boundary flagsEvery dev flag from the os-dev-report and the PR body, and both
Implemented-by: VERDICT: PASS Generated by Claude Code |
Fixes #20745
Clause-②: no (narrowing)
What this changes
A plain object with no
$-operator key beneath a relation field (the nested-relation form), a structured-JSON field (a whole-value match), or the platform-provisionedidcolumn that the declared field map omits is now refused withINVALID_FILTER/ 400, in the engine's words, before any driver is asked. It holds on every driver, at the three positions the engine judges:where(object form andFilterArraysugar;find/findOne/count/aggregate/update/deleteand the judge-onlyjudgeFilter),aggregations[i].filter, andhaving.Landing site: the door PR #20744 opened for scalars, extended. There is no second door and no second traversal. The no-operator-object arm of the number-comparand door's walk (
walkCondition) already asked one question per field key. It now classifies the column into one of three kinds, each with its own words:packages/objectql/src/no-operator-object-door.ts:noOperatorObjectColumnKindgivesscalar(unchanged:SCALAR_FILTER_HEAD_TYPESplusMULTI_OPTION_TYPES),relation(spec'sREFERENCE_VALUE_TYPES:lookup,master_detail,user,tree),json(spec'sSTRUCTURED_JSON_TYPES), ornull(never judged).provisionedNoOperatorObjectColumncoversid/created_at/updated_atwhen the declared map omits them. The three word builders live here too. ⛔ Nothing in it walks a filter.number-comparand-declared-type-door.ts: the per-key facts carry the judged column instead of a scalar type. The walk, its positions and its boundaries are unchanged.engine.ts/having-filter.ts: comments only.packages/spec/src/data/filter.zod.ts: prose only.FilterCondition's form 4 now states that the engine refuses it and names the route. TheQueryFilterexample stops teaching it. The two "Nested relation" type comments point at the refusal. ⛔ The type and the schema are not narrowed.content/docs/kernel/contracts/data-engine.mdx: the// Nested relation filterexample is replaced by the route that works (query the related object, then$inon its ids), with one paragraph on the refusal and on multi-valued lookups.The words put the verdict and the route first. The REST door truncates a 4xx message at 500 characters (
CLIENT_MESSAGE_MAXinpackages/rest/src/error-response.ts). The first draft of these words, and the #20546 scalar words, put the route past that bound, so a REST caller never saw it. Every kind now reads: position, then verdict, thenThe filter was NOT applied., then the route, then the reasoning. The REST pins assert the route in the response body. The #20546 scalar words were rewritten in the same shape because this change made their middle sentence false: it said a nested-relation condition is something "only a relation field … can carry". Examples of the words, asengine.findthrows them:Before, measured on
origin/maina51920f5fbThe readings come through
POST /api/v1/data/:object/query(the realRestServerroute overObjectStackProtocolImplementationandObjectQL) on InMemoryDriver, SqlDriver on SQLite (better-sqlite3), and SqlDriver on a live PostgreSQL 16.13 started for this run. Three rows: owneru1(region NA) ond1andd3;meta{a:1}/{a:2}/{b:1}.where{ owner: { region: 'NA' } }(lookup, the card)d1,d3meant)INVALID_FILTER, the driver's words ("cannot be bound as a SQL parameter")master_detail, amultiple: truelookup,user,treewhere{ meta: { a: 1 } }(json, the card)d1(deep equality)where{ ship_to: { city: 'Paris' } }(address),{ spec: { k: 1 } }(composite)d1,d3(deep equality)where{ id: { a: 1 } }(the card)where{ owner: {} },{ meta: {} }aggregations[1].filter{ owner: { region: 'NA' } }aggregations[1].filter{ meta: { a: 1 } }having{ owner: { region: 'NA' } }overgroupBy: ['owner']having{ meta: { a: 1 } }overgroupBy: ['meta']DATABASE_ERROR(from the json groupBy itself, see the notes){ owner: { $in: ['u1'] } }; same undermaster_detail,userd1,d3d1,d3d1,d3{ parent: { $in: ['d1'] } }(tree)d2,d3d2,d3d2,d3{ owners: { $in: ['u1'] } }(multiple lookup)d1,d3{ owners: { $contains: 'u1' } }, and$orof$containsd1,d3d1,d3d1,d3{ 'owner.region': 'NA' }INVALID_FIELD(the dotted verdict){ meta: { $contains: 'a' } }INVALID_FILTER(the text-operator door: a JSON value is never a string){ meta: { $null: false } },{ meta: { $exists: true } }{ photo: { url: 'x' } }(image)After, the same run on this branch
Every refused row above answers
400 INVALID_FILTERin the engine's words, on all three drivers, at the path the object sits at (where.owner,where.$not.owner,aggregations[1].filter.owner,having.owner, …). No read of the object runs. The routes ($in,$contains,$null), the dotted verdict, the$contains-on-json refusal and the image control answer exactly as before.Hypotheses (zone 2): which held
REFERENCE_VALUE_TYPESbrings inuserandtreebesidelookup/master_detail, andSTRUCTURED_JSON_TYPESbrings inaddress,composite,repeater,record,locationandvectorbesidejson. See the scope note below. (b) Left unjudged: file and media types (the [finding] The FILTER axis has no DOTTED-path verdict —where: { project_id.name: 'x' }rides its head segment past both doors, where SORT refuses the same spelling (#4256) #8371 carve-out: a legacy stored value is an inline object that memory can still match) andformula(refused one door earlier,INVALID_FIELD). Triage's text covers neither.'owner.region'is served by no driver: it answers400 INVALID_FIELD(the [finding] The FILTER axis has no DOTTED-path verdict —where: { project_id.name: 'x' }rides its head segment past both doors, where SORT refuses the same spelling (#4256) #8371 dotted verdict) on all three. So the refusal names only the related-object query plus ids. One refinement:$inon ids does not work for a multi-valued lookup on SQL, where the driver refuses$inon its JSON column. So the words name$containsper id there (measuredd1,d3on all three).having, relation and JSON columns do appear: a groupBy of the field, or amin/maxof it, carries that field's type (aggregatedRowColumnTypes). A nested-relationhavingkept no group on every driver before.havinghas no field declaration to read, so its relation words name "the related object" and the$inspelling.FilterConditionwithout narrowing another form. Its index signature isany | FieldOperators | FilterCondition, which TypeScript collapses toany. So removing theFilterConditionmember is a no-op, not a separation. At the schema, the nested-relation form and a JSON object comparand are the same shape (a plain object with no$key beneath a key), and only the column's declared type tells them apart. The genericFiltertype's nested arm (a recursiveFilterover an object-typed property's own type) is a separate union member. Removing it would narrowFilterfor every object-typed property, so it is aClause-②: yes (narrowing)change for its own card. ⛔ Neither is separated here.Where this departs from the order or the ruling (named, not silently chosen)
{ id: { a: 1 } }. Triage says theidrow "answers the door's existing unknown-field verdict on every driver". The Pins ruling says every row of the card's table answers the sameINVALID_FILTER. Onorigin/mainthe door has no unknown-field verdict that refuses. Its verdict for an undeclared key is tolerance (the engine's registry-less rule, pinned byGUARD an UNKNOWN field …), which leavesidto the drivers: memory 200 with no rows, SQL 400. Both readings cannot hold at once.idis not an unknown field by the engine's own definitions:find/findOneadd it to their known set, the write gate admits it (PLATFORM_PROVISIONED_COLUMNS), and so do the REST ingress (resolveQueryFields) and the per-aggregation reference names. So this PR judges the three platform-provisioned columns by the type they store, and only when the declared map omits them. That keepsid,created_atandupdated_atin the scalar words, keeps every other undeclared key tolerated, and makes the Pins row true. Reported to the PM as an open question.json.addressandcompositewere measured with the identical split (memory deep-equal rows, SQL 400). The spec publishes one class for them, so the arm judges the class, not one member of it. This is the bounded in-place fix: it is the same defect class, the same mechanical classification as the card, and the same file under this claim, and it adds no new gate family.$containsexample for a JSON field. It is not a route: the text-operator door refuses$containsover a JSON value on every driver (measured above). The JSON words name$nulland a stored field instead.filterwith a JSON object. It was the one cell that already answered alike on every driver (count 1, the engine's own deep equality). It is refused now, so that one filter has one answer at every position. The changeset names it.engine-nested-object-door.test.ts's recording driver by construction: the arm answers before a driver is resolved. The memory readings of the routes were measured (the table above) but are not pinned in a new suite:check:driver-memory-censusrefuses a new test consumer of the in-memory driver without a maintainer ruling. It caught a first draft that put one inpackages/runtime, which was dropped.Tests
The final HEAD is
ee17b18fde, a merge oforigin/maineead9dcf40into the branch.pnpm --filter @objectstack/objectql testonee17b18fde: 339 files / 6719 passed.pnpm --filter @objectstack/rest testonee17b18fde: 231 files / 4469 passed / 71 skipped.pnpm --filter @objectstack/spec testonee17b18fde: 577 files / 17007 passed / 1 todo.test:repoone08fd6883f: spec 45 files / 794, objectql 1 / 5, rest 1 / 8.typecheckfor objectql, rest and spec onee17b18fde: exit 0, including eachcheck:test-typecheckwith its ledger held.pnpm --filter @objectstack/spec check:generated: all 15 artifacts up to date.packages/objectql/src/engine-nested-object-door.test.ts(15 tests). It covers every relation type (single and multiple, with the route per multiplicity and the related object's name), every structured-JSON type, the provisionedid,{}, every verb and the judge,$and/$or/$notand sugar, the per-aggregation filter,having(a lookup groupBy, amaxof a master-detail, a json groupBy), the three REST doors intofindData, the controls (the routes, a file field, an unknown key), and the classification GUARDs over everyFieldType.packages/rest/src/data-nested-object-door.test.ts. SQLite always runs; PostgreSQL and MySQL run whereOS_TEST_POSTGRES_URL/OS_TEST_MYSQL_URLare set. It covers every row of the card's table, and the route asserted inside the REST body (so it must land inside the 500-character bound). It also covers the per-aggregation filter andhaving, the routes answering the rows the nested form meant ($inon the related object's ids givesd1,d3;$containson a multiple lookup givesd1,d3; tree givesd2,d3), and the file control. Local run with a live PostgreSQL 16.13: 8 passed (sqlite 4, live postgres 4) / 4 skipped (mysql, no URL).@objectstack/rest, so the live cells run only locally.$key as a scalar field's filter value answers 200 with no rows on the memory driver andINVALID_FILTER400 on SQLite and PostgreSQL #20546 pins (engine-no-operator-object-door.test.ts,data-no-operator-object-door.test.ts) keep a file field as their only control. Two name-gate controls pinned "the nested-relation form still passes the doors":protocol-explicit-filter-field-gate.test.ts(Data query: an unknown field insidewhere/$filteranswers 200/0 instead of400 INVALID_FIELD— the bare-key door disagrees (#4134's uncovered sibling) #7534) andquery-expression-conformance.test.ts([finding] The FILTER axis has no DOTTED-path verdict —where: { project_id.name: 'x' }rides its head segment past both doors, where SORT refuses the same spelling (#4256) #8371). They now pin what they were for: the answer is the engine'sINVALID_FILTERin the nested-relation words, never the name gate'sINVALID_FIELD....@objectstack/objectqldirection), one08fd6883f:@objectstack/metadata-protocol: 190 files passed, 3 skipped / 2792 passed, 19 skipped.@objectstack/service-analytics: 140 / 3266 passed. Its nested-relationwhereis flattened to cube members before any engine call, and it passed unchanged.@objectstack/plugin-security: 147 files / 3202 passed, 23 skipped.Reverse verification (ablation), from the committed fix. It ran through
scripts/ablation-replace.mjsin WRAP mode, trap-restored. The walk's gateif (facts.column !== null && isNoOperatorObject(value)) {was narrowed back to the #20546 behaviour withfacts.column.kind === 'scalar' && facts.column.provisioned !== true && facts.column.type !== '__ablated_20745__'. On disk the anchor went 1 → 0 and the marker 0 → 1, with blobea19d1959255→3134e87031b1. Then objectql was rebuilt, andablation-dist-preflightfound the marker in 4 built files.$key as a scalar field's filter value answers 200 with no rows on the memory driver andINVALID_FILTER400 on SQLite and PostgreSQL #20546 scalar pin stayed green.whereand aggregate refusals failed on SQLite and live PostgreSQL. The routes, the controls and the [finding] a plain object with no$key as a scalar field's filter value answers 200 with no rows on the memory driver andINVALID_FILTER400 on SQLite and PostgreSQL #20546 file stayed green.ea19d1959255),git diff HEADis empty, and whole-treegit status --porcelainis empty. After a rebuild, the--absentpreflight found the marker absent from all 14 built files. Then both pin sets were green again (249 passed; 16 passed + 8 skipped).Gates
node scripts/pm/dispatch-gates.mjs --commandsatee17b18fde(aftergit fetch, so not stale) derived 110 commands. All 110 were run onee17b18fde.--ranreconciles them: 110 derived, 110 run, 0 NOT-MEASURED, 0 UNRUN, and all exit 0. Among them are these gates:check:adr-0087-registration --base origin/main:not-required (no-migration-prescription)accepted.check:changeset-no-major,check:empty-changeset,check:doc-authoring,check:nul-bytes.check:engine-double-contract(904 pinned),check:where-matcher(440 matchers, 0 silently wrong),check:driver-memory-census,check:cross-package-test-inputs,check:test-source-alias,check:type-check-coverage,check:type-check-debt,check:query-options-erasure.check:dual-build-cjs-loads(105 entry points, 66 packages).check:api-surface/check:docs/check:authorable-surface/check:skill-examples.Lint, narrowed and proven:
pnpm exec eslint --no-inline-config --format jsonover the 11 changed.tsfiles atee17b18fdefound 11 files, 0 errors, 0 warnings. Three facts make this narrowing a measurement:isPathIgnoredanswersfalsefor all 11.parserOptions.projectandprojectServicearenullfor every file, so type-aware linting is not enabled.Changesets
.changeset/20745-nested-object-door.md:@objectstack/objectqlminor, a BREAKING banner,Clause-②: no (narrowing), and the ADR-0087 markernot-required (no-migration-prescription), as the [finding] a plain object with no$key as a scalar field's filter value answers 200 with no rows on the memory driver andINVALID_FILTER400 on SQLite and PostgreSQL #20546 changeset has. It states what an author sees now and the route that works, per kind, with the table. It says it supersedes the "Unchanged" paragraph of the pending scalar-field entry (20546-no-operator-object-on-scalar) for relation and structured-JSON fields..changeset/20745-nested-relation-prose.md:@objectstack/specpatchfor the shipped JSDoc.@objectstack/objectql's root and./coreexports are unchanged.check:api-surfaceis green.Acceptance notes
skills/objectstack-queryteaches the nested-relation form as working:SKILL.md"Nested Relation Filters", the "Filter parent by child conditions" row and thesearchparagraph, andrules/filters.md"Nested Relation Filters". It was already untrue before this change (memory answered no rows, SQL 400). It is a governed Tier H surface outside this claim's file surface, so it is not touched here.groupByof ajsonfield answers 500DATABASE_ERRORon PostgreSQL. On InMemoryDriver it merges every row into one group (n: 3), and on SQLite it gives one group per serialized value.where: { project_id.name: 'x' }rides its head segment past both doors, where SORT refuses the same spelling (#4256) #8371 carve-out and stay unjudged.{ photo: { url: 'x' } }still answers memory 200 with no rows and SQL 400.havingthere is no field declaration, so a relation column's words say "the related object" and give the$inspelling. Ahavingover a multi-valued relation groupBy would get that single-valued spelling.referenceTargetOfnames the related object in the relation words. For a field whosereferencecarrier never went through the schema's parse (a non-string), it throws its ownTypeErrorinstead of the refusal. Parse refuses that shape at the contract door.Generated by Claude Code