fix(spec): refuse a { $field } reference as a $between endpoint at the runtime filter door - #19501
Conversation
…e runtime filter door WIP — implementation and pins; changeset and ledger entry follow. Claude-Session: https://claude.ai/code/session_01AmH9bKvGoLjiY86Q4Z3og2 Co-authored-by: Claude <noreply@anthropic.com>
…rowing Claude-Session: https://claude.ai/code/session_01AmH9bKvGoLjiY86Q4Z3og2 Co-authored-by: Claude <noreply@anthropic.com>
📓 Docs Drift Check5 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 136 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin e9922c97dfb902eb1cfdb234f23767525ec53e99 && git checkout e9922c97dfb902eb1cfdb234f23767525ec53e99
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin f34dda62ed1333ea0d07ac628ca09d830a358f9e 174d72f05c3cf872a0a96512c63ee41ada77ab9e && git checkout -B drift-repro f34dda62ed1333ea0d07ac628ca09d830a358f9e && git merge --no-ff 174d72f05c3cf872a0a96512c63ee41ada77ab9e
node scripts/docs-audit/affected-docs.mjs --json f34dda62ed1333ea0d07ac628ca09d830a358f9e |
Contract reviewServed-tier: VERDICT: FAIL One ground, and it is not the runtime-door refusal — that part is correct, exactly scoped, and survived every attack below. The ground is a sentence the PR ships to users inside the ADR-0087 ledger entry, which measures false on four of the carriers it names. Everything else in this record is a finding for the dev, not a ground. How this was measuredOwn worktrees, own builds, nothing adopted from the card, the PR body or the report. Head at 1. Both doors, before and after — the delta is what the PR says, plus a re-routing it does not say55-row probe, each row read at the runtime door (
So: (i) the refusal is exactly the endpoint case for pairs the runtime door accepted — every newly-refused row is a two-element pair, no null, no blank, at least one non-array object carrying Finding A (not a ground). The PR body's "Accepted before, refused now — and nothing else … That is the whole delta" is over-broad by one door. The bold rows above were already refused at On the shape-vs- 2. Ordering — proved by pairs, not by readingRows 4–6 of the table: 3. Stored data stays readable; the migration TODO exists and does not touch the stack
The claim "only re-saving is refused" is true for the view / page / component filter rules ( 4. The prescriptions work
Finding B (not a ground) — the analytics face's array-form wording IS preempted. The report says the 5. The pins can failAblation: the PR's 6. The card's routeThe refusal lands in 7.
|
| carrier | Zod type of the filter slot | re-save verdict |
|---|---|---|
DashboardSchema widget filter (dashboard.zod.ts:872, :1285) |
FilterConditionSchema |
ACCEPT — document parses green |
DatasetSchema.filter (dataset.zod.ts:191, :337) |
FilterConditionSchema |
ACCEPT — document parses green |
ReportSchema.runtimeFilter (report.zod.ts:239, :316) |
FilterConditionSchema |
no issue at runtimeFilter (only unrelated required-field issues) |
FieldSchema.rollup.filter / relatedListFilter (field.zod.ts:1630, :1525) |
FilterConditionSchema |
no issue at the filter path |
ViewSchema list filter rule / ViewFilterRuleSchema (page, component) |
rule value shaped by operator |
REFUSE @ list.filter.0.value / value ✓ |
NormalizedFilterSchema / FieldOperatorsSchema |
enforced copy | REFUSE ✓ |
FilterConditionSchema is z.record(z.string(), z.unknown()).and(z.object({ $and, $or, $not })) (filter.zod.ts:1704); it never judged an operator map and still does not. The dashboard, dataset, report and rollup filters are therefore not "carriers of FieldOperatorsSchema" at the Zod door, and re-saving one of those documents surfaces nothing. For those carriers the only refusal a stored { $field } endpoint ever meets is the runtime door this PR adds, when the filter is executed. (A TypeScript author gets a compile error from the inferred FieldOperators type; a Studio / JSON / stored document does not.) An upgrader who follows the entry and re-saves their dashboards to find the offending ranges will find none and believe the sweep is done.
The same sentence sits verbatim in the sibling 18.filter-between-blank-endpoint-refused.ts:81 and is equally false there; that is outside this card, but it is why the wording should not be copied a third time.
To act on this without asking: in 18.filter-between-field-reference-endpoint-refused.ts (then pnpm --filter @objectstack/spec gen:migration-registry — tsx scripts/build-migration-registry.ts — to regenerate registry.ts; check:migration-registry must stay exit 0):
- In
surface, split the carrier list into (a) the authoring doors that refuse on save — view / page / component filter rules (ViewFilterRuleSchema) and theNormalizedFilterAST the query faces validate — and (b) theFilterConditionSchemacarriers — dashboard widget filter, dataset filter, reportruntimeFilter, rollupfilter,relatedListFilter— where nothing judges the endpoint at save and the refusal is reached only when the filter is executed throughparseFilterAST/ the engine's shape door (i.e. the door this PR closes). That is also the honest statement of why "both doors" is the right framing. - In
acceptanceCriteria, replace "FieldOperatorsSchema.safeParse and re-saving the document both make the sweep mechanical" with a sentence that names re-save as mechanical only for the (a) carriers, and names the grep (which the entry already prescribes) plus executing the surface as the detector for the (b) carriers. Keep "a range whose BOTH endpoints are references reports both positions" scoped to the schema door — the runtime door throws on the first. - Optional, same edit: the §8 sentence in
reasonnaming the superseded 2026-08-11not-required (no-migration-prescription)disposition.
Nothing in the runtime-door code, its tests, or the changeset needs to change for a PASS.
Out of scope, observed, not filed by this seat
- Pre-existing two-door splits on the same
$betweenslot, both trees:[true,false]and[5n,10n]ACCEPT at the runtime door (the TYPE door's literal set includes boolean and narrows bigint) and REFUSE at the schema door (union is number / Date / string);[ref,[1]]was runtime-ACCEPTED at base because a nested array member is "kept" by the TYPE door. Same family as this card, different spelling; not touched here and not made worse. - A field-level
$notwrapper —{ at: { $not: { $between: [ref,'M'] } } }— is ACCEPTED at both doors on both trees ($notat field level isFilterConditionSchema, loose; the shape door skips non-list$keys). Same for the null and blank carve-outs; pre-existing.
Generated by Claude Code
Contract reviewServed-tier: VERDICT: FAIL — adopted verbatim by this seat at 2026-09-21T05:19Z from the isolated at-tier reviewer's record Why this record was re-rendered — a defect in this seat's dispatch, not in the reviewThe reviewer wrote
The FAIL ground, in one sentence — ⛔ the code is not what failedThe shipped ADR-0087 ledger entry tells an upgrader something that is not true. Its This seat re-measured the ground first-hand before adopting it, ⛔ not from the reviewer's narration:
⇒ an upgrader who follows the entry re-saves every dashboard, finds nothing refused, and concludes the sweep is done. No test goes red because a document lies — which is the whole reason this review exists. ⭐ The reviewer states the fix is confined to the entry's Carriers
Also handed back, ⛔ none of it a FAIL groundFour findings the reviewer recorded for the dev, verbatim in its record: the PR body's 「whole delta」 claim is over-broad by one door (four shapes were already refused by the comparand-TYPE door and now get the reference message instead — convergent, unpinned); the analytics wording is preempted at head, contrary to the dev's report; the 2026-08-11 removal was not a ledger omission (PR #7713's changeset carried Generated by Claude Code |
The entry named four FilterConditionSchema slots as carriers that refuse on save; measured, those documents parse green. Split the carrier list by what actually answers at save, rewrite the re-save sentence per class, and name the 2026-08-11 not-required disposition this entry supersedes. Pin the TYPE-door re-routing and record the $in/$nin gap as a todo. Claude-Session: https://claude.ai/code/session_01AmH9bKvGoLjiY86Q4Z3og2 Co-authored-by: Claude <noreply@anthropic.com>
…tween-field-endpoint-runtime-door
Contract reviewServed-tier: ① Derived judgmentsRound-1 ground (comment 5755750784, head How this was measured. Own detached worktree 1. The delta is what the seat measured — and nothing else moved.
2. The ground, re-read over the real schemas. Document
Every 3. "Executing the surface" — traced per (b) carrier, because it is only a detector if execution reaches a loud door.
4. The new pins are honest. Schema door on the convergent rows ( 5. "Supersedes" — verified. 6. The analytics correction — both forms verified (measurements in §3): FilterArray form 7. The sibling entry — claim verified, leaving it is right. Findings — not grounds, each with its measurement.
② Semver level
③ Boundary flags
Implemented-by: VERDICT: PASS Generated by Claude Code |
Fixes #19377
Clause-②: yes
The runtime filter door now refuses a
{ $field }column reference as a$betweenendpoint, exactly as the authoring schema door already does.The split this closes
RANGE_ENDPOINT_DESCRIPTION— the published endpoint contract shared by both of$between's bounds — has stated verbatim since 2026-08-11 (#7596, ADR-0049 enforce-or-remove) that "A { $field } reference is NOT an endpoint shape: no backend resolves one inside a list". That ruling removedFieldReferenceSchemafrom both endpoint unions and shipped at the schema door alone.Re-measured on this branch's base
4045781fafbefore any edit, ⛔ not adopted from the card:RangeOperatorSchema.safeParse({ $between: [{ $field: 'a' }, 'M'] })success: false— refused, names$between.0parseFilterAST({ f: { $between: [{ $field: 'a' }, 'M'] } })INVALID_FILTER/ 400, names the MIN boundparseFilterAST({ f: { $between: ['A', { $field: 'b' }] } })parseFilterAST([['f', 'between', [{ $field: 'a' }, 'M']]])(lowered array form)parseFilterAST({ f: { $between: ['A', 'M'] } })parseFilterAST({ f: { $between: [null, 1] } })One published sentence, two truth values. The door that passed it is the one an embedder reaches by handing a lowered filter straight to a driver — where nothing resolves the reference: the in-memory matchers compare the raw reference OBJECT so the range silently matches nothing, and both SQL faces refuse the position. The caller got a filter that named a window and answered no rows, instead of a refusal they could act on.
What a reviewer is being asked to accept
Accepted before, refused now — and nothing else: a two-element
$betweencomparand, of correct arity, with neither boundnulland neither bound blank, where at least one bound is a non-array object carrying a$fieldkey. That is the whole delta.Deliberately left alone:
{ $field }reference as the WHOLE comparand of$eq/$ne/$gt/$gte/$lt/$lte— [spec] SqlDriver 将$field编译为列对列比较(cross-field comparison push-down) #5222's shipped column-to-column capability, and the alternative this refusal prescribes. Pinned by a lit control in the same file, for all four ordering operators plus the two-bound spelling{ $gte: { $field: 'a' }, $lte: { $field: 'b' } }the message names.[0, 0],['0', '9']. Same object reference out.$betweenarm, after arity,null(2026-08-31) and blank (2026-09-17). A pair that already carried a refusal keeps the message and the prescription it had —[{ $field: 'a' }, null]still answers with the null predicate,[{ $field: 'a' }, '']still answers with the blank prescription,[{ $field: 'a' }]still answers with the arity sentence. Pinned.{ nope: 1 }— keeps the comparand-TYPE door's own sentence, one step further on. Pinned, so the new refusal cannot be a blanket rejection of object endpoints.$in/$ninMEMBERS carrying a reference. Same 2026-08-11 decision, published bySET_MEMBER_DESCRIPTION, and this door still lowers them unchanged — measured, see Acceptance notes. It is a second split over a different published sentence, needing its own wording and its own ruling; absorbing it silently here is the exact move this card's family exists to refuse. ⛔ It is not pinned in either direction: pinning a measured defect green would read as a ruling nobody made.The refusal's own wording
Its own builder rather than an arm of the null or blank one, because those prescribe a VALUE and the author who wrote a reference was reaching for a column-to-column comparison — a capability the platform has, one operator over. The remedy is a different filter, not a different literal.
INVALID_FILTER/ 400 (ADR-0112 class 1). The longest assembled form measures 456 characters against the unrelaxed 500-character client bound, with the engine'sfind('deal')context prefix included, and the bound test grew the new cases. ⛔ It does NOT offer the in-memory evaluator as an escape —matchesFilterdoes not resolve a list member either, it fails silently rather than loudly, so naming it would send an author to the one path whose answer is a wrong row set; the schema door's twin holds the same line, and a pin asserts the absence.The recogniser is SHAPE — a non-array object carrying a
$fieldkey — spelled as the schema door'sisFieldReferenceShapespells it, ⛔ not the comparand-TYPE door's strictertypeof value.$field === 'string'. It is spelled again in this file rather than imported becausefilter.zod.tsimports this module and the reverse edge would be a cycle; a pin reads BOTH doors on one input set so they cannot drift.The migration conclusion, and the rule behind it
This narrowing owes its own ADR-0087 ledger entry, and it carries one —
filter-between-field-reference-endpoint-refused, registered in this PR.The rule applied: ADR-0087's disposition vocabulary is closed and every exemption is re-verified, so a declared-breaking changeset either registers or claims exactly one named category. Checked in turn against what is on disk:
already-registered— unavailable, and this is the load-bearing finding. It requires ids that pre-date the merge base and cover this change. The 2026-08-11 schema-door removal registered nothing: nosemantic/entry, no retired key, no retired def, nospec-changes.jsonrow, no line in the protocol upgrade guide names$fieldor [spec]FieldReferenceSchemais declared in the$betweenendpoints but NO backend resolves a$fieldinside a list #7596. This is the route PR fix(spec): refuse a blank $between endpoint at the runtime filter door #19374 took for the blank spelling one endpoint over, and it does not transfer, because the blank spelling's schema-door half had already registeredfilter-between-blank-endpoint-refusedunder spec(ui):FieldOperatorsSchemaaccepts a$betweenrange with a blank bound —{$between: [1, '']}parses green, so a half-filled range is not refused at the authoring door #18012. Here the ledger is empty, so this entry is the FIRST record of the transition and covers both doors rather than only the second.no-migration-prescription— refused by a body that carries a migration prescription, and AGENTS.md requires a breaking changeset that removes something an author can write to state FROM → TO. The changeset carries that table.unpublished— refused:@objectstack/specpublishes to npm.runtime-interface-only/type-surface-only— neither describes a published Zod-declared authoring face whose accept/reject behaviour moved.⇒
registered.No D2 conversion, and that is a fact about the shape rather than a decision to decline one. The literal a reference stood for is not recoverable from the reference, and the column-to-column reading has a different OPERATOR SHAPE (two scalar bounds), so producing it would be the platform rewriting one filter into another; dropping the operator would delete a constraint the author wrote and WIDEN the result set silently. The load path was measured, not assumed:
applyConversionsToStoredItem— the one primitive every stored-row rehydration seam calls — returned the same object reference for a stored view carrying{ close_date: { $between: [{ $field: 'contract.start' }, '2026-12-31'] } }, so it neither drops the operator nor refuses the row. Stored views stay readable; what changes is that RE-SAVING one is refused, at the endpoint's own path, with the side named. Same ground as the two nearest narrowings of this surface set,filter-between-blank-endpoint-refusedandfilter-preset-ordering-comparand-refused.The changeset carries the disposition marker in the ADR-0087 comment form the gate reads, naming that entry id;
node scripts/check-adr-0087-registration.mjsreports 1 declared-breaking changeset, carryingregistered filter-between-field-reference-endpoint-refused(new here).File surface — one declared expansion
The claim named
packages/spec/src/data/as the file surface. The code and its pins are inside it. Three files fall outside and are reported rather than taken silently: the changeset (.changeset/, named in the deliverable), the ledger entry (packages/spec/src/migrations/entries/semantic/) and the generatedpackages/spec/src/migrations/registry.tsit concatenates into. The ADR-0087 gate reds a declared-breaking changeset with no valid disposition, and — per the section above — nonot-requiredcategory was available, so the entry is what makes the declared narrowing landable at all.gen:spec-changesandgen:upgrade-guidewere re-run and produced no diff; neither projection carries entries of this major, which matches the sibling precedent.Verification
Every reading below was taken in this worktree.
pnpm --filter @objectstack/spec exec vitest run src/data/filter-comparand-shape.test.ts— 1 file / 65 tests passed.pnpm --filter @objectstack/spec build— green, before and after the change.dist/data/index.mjsafter the change: the three reference rows now REFUSE at both doors, the two literal-range controls still ACCEPT at both, and the null / blank / non-reference-object rows are unchanged.pnpm --filter @objectstack/spec check:migration-registry/check:spec-changes/check:upgrade-guide— all exit 0.node scripts/check-adr-0087-registration.mjs— exit 0.Acceptance notes
Noted while measuring, ⛔ not filed by this dispatch and ⛔ not fixed here:
{ $field }reference as an$in/$ninMEMBER is refused by the schema door and accepted by the runtime door. The same 2026-08-11 ruling one position over, with its own published sentence:SET_MEMBER_DESCRIPTIONstates "A { $field } reference is NOT a member shape: no backend resolves one inside a list", andlistPositionFieldReferenceMessage's own doc names "every$in/$ninmember, and both$betweenendpoints" as one ruling. Measured on this branch at4045781fafand again after this change:FieldOperatorsSchema.safeParse({ $in: [{ $field: 'a' }, 'x'] })answerssuccess: false, whileparseFilterAST({ f: { $in: [{ $field: 'a' }, 'x'] } })returns the filter unchanged, same object reference;$ninbehaves identically. Left alone because the refusal needs its own authored wording and, being a narrowing of a second published face, its own ruling — the same reason this card exists rather than being absorbed into fix(spec): refuse a blank $between endpoint at the runtime filter door #19374. Dedupe words:field reference list member·SET_MEMBER_DESCRIPTION·parseFilterAST runtime door·7596 member shape·two doors disagree.Generated by Claude Code