fix(objectql)!: refuse a non-numeric string compared against a number field at the engine's filter door, and narrow a numeric one (#20351) - #20501
Conversation
… field at the engine's filter door WIP: the door module and its calls at the collection point (where, both spellings; the per-aggregation filter; having). Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN Co-authored-by: Claude <noreply@anthropic.com>
…the engine Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN Co-authored-by: Claude <noreply@anthropic.com>
…er SqlDriver Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN Co-authored-by: Claude <noreply@anthropic.com>
… set (minor, breaking) Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN Co-authored-by: Claude <noreply@anthropic.com>
…mber-comparand-door
…answers A string on a sum / count / avg column was pinned as kept-no-group beneath the temporal door; the number-comparand door refuses it now, so the row moves to a refusal pin in that door's words. The unknown-token having row moves to a text column, which neither field-aware door judges. Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN Co-authored-by: Claude <noreply@anthropic.com>
…d door now refuses Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN Co-authored-by: Claude <noreply@anthropic.com>
… number door's refusal now Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN Co-authored-by: Claude <noreply@anthropic.com>
…mber-comparand-door
📓 Docs Drift CheckThis PR changes 1 package(s): 5 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:
What this run could not see
Coarse fallback — 17 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 00d4232fcd058e0e2ed54e9c609e838bae2c5f62 && git checkout 00d4232fcd058e0e2ed54e9c609e838bae2c5f62
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 2b24b8b82304e925110800efb0e092845a931d16 09da7a4cc4024292cbd61b24373a13ab1ceb6c57 && git checkout -B drift-repro 2b24b8b82304e925110800efb0e092845a931d16 && git merge --no-ff 09da7a4cc4024292cbd61b24373a13ab1ceb6c57
node scripts/docs-audit/affected-docs.mjs --json 2b24b8b82304e925110800efb0e092845a931d16
|
Contract reviewServed-tier: Read for this record: card #20351 (body and all four comments: triage unlock ① Derived judgmentsThe accept-set change the diff implies. One narrowing, at one seam: a string that the spec's numeric grammar does not read as a number, compared against a declared numeric field (or a numeric aggregated column) at the implicit comparand, the six scalar operators or a member of
Wrong: none found. ② Semver level
Clause-②: no (narrowing) ③ Boundary flagsDev Out-of-scope findings.
The nine deviations. (1) Driver pins: accepted, judgment 9. (2) The RLS premise false at runtime: TRUE at the head, accepted, finding 4 above. (3) The Shape and gates. Draft PR, base Implemented-by: VERDICT: PASS |
Fixes #20351
Clause-②: no (narrowing)
What this adds
Lane (2) of the two-lane route #20336 took on #15661's precedent: the engine door that consults the contract PR #20414 published in
@objectstack/spec/data(filter-number-comparand-declared-type.ts). The contract half is untouched;packages/specis not in this diff.The door,
packages/objectql/src/number-comparand-declared-type-door.ts, beside the text-operator and temporal doors. For each comparand at a judged position on a declared numeric field it asksnumberComparandDoorVerdictand routes the answer:door-refusal: throwsINVALID_FILTER/ 400 (the existinginvalidFilterErrorenvelope) in the contract's words,numberComparandRefusalMessage, before any driver is resolved;narrows: rewrites the numeric string to its number, copy-on-write (the caller's filter is never edited, and a filter with nothing to narrow comes back by reference);passes/deferred: leaves it alone.The door reads no string itself. The grammar, the judged types (
NUMERIC_VALUE_TYPESby identity), the judged operators (NUMBER_COMPARAND_DOOR_SCALAR_OPERATORS/NUMBER_COMPARAND_DOOR_LIST_OPERATORS) and the words are all the spec's.Its calls in
engine.ts, at the collection point only, fifth after the temporal door in the same order everywhere:lowerWhereFilterArray, object form (beforenormalizeFilterComparandTypes) and array form (on the lowered condition). Sofind/findOne/count/aggregate/update/deleteand the judge-onlyjudgeFilter(judgeWhereAdmissioncalls the same function) all inherit it;filter, rooted ataggregations[i].filter, against the object's declared fields;having, after the temporalhavingdoor, over the columnsaggregatedRowColumnClassesclassesnumeric(count/sum/avg, and a groupBy ormin/maxof a numeric field).The
judgeWhereAdmissiondocblock's pipeline list names the new door (comment only).A changeset,
.changeset/20351-number-comparand-door.md:@objectstack/objectqlminor, BREAKING,Clause-②: no (narrowing), a FROM → TO line, and the ADR-0087 dispositionnot-required (no-migration-prescription)in the form PR fix(core,objectql)!: a date or datetime names a year from 0001 to 9999, refused at the comparand door and the write door (#20264) #20469 and PR fix(objectql)!: a number field refuses an array, a boolean or an object with invalid_number (#20309) #20370 used.@objectstack/objectql's root exports are unchanged: the door module is not re-exported fromindex.tsorcore.ts, like its two siblings.What it does to the card's three answers
Measured through
engine.find/engine.aggregateandPOST /api/v1/data/:object/query, three rows (5, 12, 30), on InMemoryDriver, SqlDriver on SQLite and SqlDriver on a local PostgreSQL 16.13 server:numberfield3062e5001: memory · SQLite · PostgreSQLwhere$gt/$eq/ implicit / a$inmember"abc"DATABASE_ERRORINVALID_FILTERwhere$ne "abc"where$eq ""where, REST$gt "{current_user_id}"(resolved to the user's id)filter$gt "abc"/$ne "abc"havingonsum(amount)$gt "abc"/$ne "abc"where$gt "12"/$eq "12"$gt 10(the numeric control)The last-but-one row is the narrowing's point: InMemoryDriver compared
"12"as a string and matched nothing.Premise check, and the order's hypotheses
H1 holds, reproduced at
3062e5001(the table above). SqlDriver's server-side log line on PostgreSQL reads(22P02) … invalid input syntax for type numeric: "abc".H2: the collection point is where the order says, and the new door sits after the temporal door at each call.
judgeFilterpasses through it:judgeWhereAdmissioncallslowerWhereFilterArray(pinned:judgeFilteranswersINVALID_FILTER/ 400 for"abc"and{ ok: true }for"12"). RLS / sharing / tenant predicates do NOT pass through it at runtime. The middleware chain composes them onto the AST after this seam, andplugin-security'sjudgeCompiledComparandsruns only the two field-agnostic faces (rls-compiler.ts, the[#20212]block). A policy predicate reaches this door at authoring instead:validateRlsPredicateEnforceabilityasks the engine'sjudgeFilterwhen the host hands the rule a judge.H3 holds. The verdict is
numberComparandDoorVerdictoverNUMBER_COMPARAND_DOOR_JUDGED_TYPESwith the scalar and list operators, and the words arenumberComparandRefusalMessage. A numeric string is narrowed to its number (the verdict'snarrows, as the contract review's judgment 7 asks). The pins assert the rewritten filter the driver receives, not only the 400s.H4: MySQL is NOT MEASURED. No MySQL server is available in this container. The REST suite carries a MySQL cell, a named skip without
OS_TEST_MYSQL_URL.H5: neither consults the same verdict everywhere.
service-analytics: the ObjectQL strategy sends the caller'swhereintoengine.aggregateand asksjudgeFilterabout the read scope (assertReadScopeAdmittedByEngine), so both inherit the door. The NativeSQL strategy's decline (NativeSQLStrategy.canHandle) declines a cross-field reference and an uninterpretable temporal comparand, but does not consult the number verdict. So a raw-SQL deployment compilesamount > 'abc'itself (read at source, not measured).usingis judged throughjudgeFilter, as above. No lint rule readsnumberComparandDoorVerdict(git grepoverpackages/lint/srcfinds zero hits), so a stored view or report filter comparing a number field with a non-numeric string saves clean and is refused at query time.Both are reported as findings below and are not edited here.
The staged
$emptyrow: pinned at the door aloneNUMBER_COMPARAND_DOOR_CASEScarries PR #20442'sunjudged$emptyrow. The engine suite partitions it out of the end-to-end drive and pins it at the door alone:findNonNumericComparandanswersnull, andnarrowNumberComparandsreturns the same reference. A partition guard asserts the table is split exactly. So the row can neither turn this suite red for a reason that is not the door's, nor vanish unnoticed.The contract's
formularows are partitioned the same way the text door's suite does it: they are pinned in the direction they answer (INVALID_FIELD/ 400 from the #8296 materializable door, one door earlier). The door's own walk is pinned to judgef_formula_numberby itsreturnType.Tests (at
09da7a4cc, the merged head, unless noted)New:
packages/objectql/src/engine-number-comparand-declared-type-door.test.ts, 29 tests. It drives the contract's case table through a realObjectQLand a recording driver, per the contract header:f_formula_numberrow), assertingcode+status+httpStatus, everymustMentionsubstring, and no driver read. All 8 refusal forms and every judged position are covered, both ways;narrowscases, asserting the driver receivesc.expectedFilter()and the caller's filter is untouched;passescases, reaching the driver unchanged;$empty(1) partitions above.Beside the table:
FilterArraysugar, both refused and narrowed;$and/$or/$not;judgeFilter;filter, refused at its path, with numeric strings counting what their numbers count;havingoncount/sum/ a numericmin, refused, narrowed, and a placeholder oncount;findDatadoors (whereobject,$filter, filter AST, implicit query parameter), both ways;New:
packages/rest/src/data-number-comparand-door.test.ts. It runsPOST /api/v1/data/:object/queryandengine.find/engine.aggregateover SqlDriver, with a cell per dialect:where: 9 refused spellings;filter;havingonsumandmax(currency), on the native and the rows path;The SQLite cell always runs. The PostgreSQL cell ran against the local server: 3/3 passed at⚠️ No CI job provisions
09da7a4cc.OS_TEST_POSTGRES_URLfor@objectstack/rest. TheTemporal Conformance (live PG + MySQL)job runsdriver-sql's suite,metadata-protocol'slive-*files and oneruntimefile, and adriver-sql-only pin cannot reach an engine door. So the live cells are red-capable and un-run in CI; the local run above is their measurement.Re-pinned, test side only. Four existing pins asserted the old silent answer for a string on a numeric column:
engine-aggregate-having-temporal-door.test.ts: the three "a string on sum / count / avg keeps no group" rows move to a refusal pin in the number door's words;engine-aggregate-positions.test.ts: the "unknown token on count" row moves to a text column, which neither field-aware door judges, and the count-column case is pinned in the new suite;rest-aggregate-numeric-having.test.ts: three rows move fromKEPTto aREFUSEDtable, SQLite and PostgreSQL both run locally;data-query-having-temporal-door.test.ts: "a string on sum" becomes a number control plus a refusal pin.pnpm --filter @objectstack/objectql exec vitest run --project local --maxWorkers=2: 330 files, 6118 tests passed.--project repo: 1 file, 5 passed.pnpm --filter @objectstack/rest exec vitest run --project local --maxWorkers=2: 219 files, 3930 passed, 40 skipped.--project repo: 1 file, 8 passed.The live PostgreSQL run of the two PostgreSQL-capable REST files: 30 passed (15 live-postgres), 15 skipped (MySQL).
pnpm --filter @objectstack/objectql typecheckandpnpm --filter @objectstack/rest typecheck: exit 0.check:test-typecheckis OK for both, with no debt added (objectql 40 files / 234 errors held; rest 0 / 0).Ablation (reverse verification)
The mutation is in the door's walk, which every position routes through:
if (!meta || numberComparandFieldVerdict(meta) !== 'judged') continue;→if (meta || 'ABLATION_20351') continue;. It is made withscripts/ablation-replace.mjs: anchor 1 → 0, and blobaa4a3247→56a5bdf6.pnpm --filter @objectstack/objectql build,ablation-dist-preflightfound the marker in 4 built files. The objectql door suite went 20 failed / 8 passed; the 8 are the guards and partitions that do not depend on the door firing. The REST door suite went 6 failed / 3 skipped. The SQLite cell answered200withrecords: [], the PostgreSQL cell500 DATABASE_ERROR, and the per-aggregation$in ["5","30"]counted 0 instead of 2: the card's defect, back.aa4a3247= HEAD andgit diff HEADis empty. After a rebuild,ablation-dist-preflight --absentfound the marker absent from all 14 built files and the tree clean. Both suites passed again (28/28 and 6 + 3 skipped at that commit,872d7708b).Gates
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstackat09da7a4ccderives 65 commands, the same list as at the first merged head. All 65 ran with each exit code recorded before any pipe:check:dual-build-cjs-loadsandcheck:type-check-debtanswered exit 3 (PREREQUISITE NOT MET) until the whole workspace was built (turbo run build --filter=!@objectstack/docs, 72/72), then exited 0.dispatch-gates --ran: 65 derived, 65 run, 0 NOT-MEASURED, 0 UNRUN. The branch mergedorigin/maintwice with true merge commits, no rebase and no force-push; the last merge base is45f428d8f.Acceptance notes
havingwords. A numeric aggregated column has no declaredFieldType, so the door hands the verdictnumber(the member of the numeric class the column holds). The spec's words then read "compares a declared number field against … at having.total.$gt". Thenot-a-numberclause ("backends answer it differently (PostgreSQL with a server error)") is thewherefact:havingis evaluated by the engine on every driver, and there it kept no group, or every group under$ne. The words are the contract's, and the path names the position.Datecompared against a number field is not judged (the contract judges strings).$gt true: no rows on memory, every row on SQLite, 500 on PostgreSQL. ADate: no rows · no rows · 500. Both hold on the base and on this branch. Handed to the seat below.driver-mongodb(the door sits in front of it); the NativeSQL analytics path (read at source).skills/**path in the diff).Out of scope, handed to the seat (not filed by this dev)
Datecomparand against a number field answers500 DATABASE_ERRORon PostgreSQL. It isPOST /api/v1/data/:object/querywithwhere: { amount: { $gt: true } }against anumberfield, on a local PostgreSQL 16 server, on the base and on this branch. The contract review of PR feat(spec): the number-comparand declared-type door's contract and the platform's numeric grammar #20414 said to file this only if it answered 500; it does. Dedupe words:boolean comparand number field postgres 500·Date comparand numeric column database_error·non-string comparand declared number type.NativeSQLStrategy.canHandledoes not consult the number verdict, so a raw-SQL analytics deployment does not fall through to this door. Dedupe words:native sql decline number comparand·analytics raw sql non-numeric string.numberComparandDoorVerdict, so a stored view or report filter with a non-numeric string on a number field saves clean and is refused at query time. Dedupe words:stored view filter non-numeric number field lint·authoring number comparand verdict.judgeCompiledComparands) does not consult the number verdict. The authoring judge does, when present. Dedupe words:rls compiled predicate number comparand·policy using string against number field.Generated by Claude Code