diff --git a/.changeset/20802-nested-relation-filter-served.md b/.changeset/20802-nested-relation-filter-served.md index ebb4387159d..0546c007566 100644 --- a/.changeset/20802-nested-relation-filter-served.md +++ b/.changeset/20802-nested-relation-filter-served.md @@ -25,4 +25,4 @@ Measured through `POST /api/v1/data/:object/query` on SQLite and PostgreSQL 16 ( | `{ $not: { owner: { region: "NA" } } }` | `INVALID_FILTER` / 400 | `d2`, `d4` | | `{ owner: { region: "APAC" } }` (no owner matches) | `INVALID_FILTER` / 400 | no rows | -On the in-memory driver, a multi-valued relation's `$contains` still matches a stored id by substring per element, so there an id that is a substring of another stored id (`u1` inside `u10`) also matches; SQLite and PostgreSQL match the element. +SQLite, PostgreSQL and the in-memory driver match the element of a multi-valued relation, so an id that is a substring of another stored id (`u1` inside `u10`) does not match it. diff --git a/.changeset/20874-memory-contains-membership.md b/.changeset/20874-memory-contains-membership.md new file mode 100644 index 00000000000..f4401ab9c9a --- /dev/null +++ b/.changeset/20874-memory-contains-membership.md @@ -0,0 +1,21 @@ +--- +"@objectstack/driver-memory": minor +--- + +fix(driver-memory): `$contains` / `$notContains` on a multi-valued or JSON-stored field answer by membership, as the SQL drivers do + +Clause-②: yes (widening) — one new public method on the exported `InMemoryDriver` class, `filterContainsTest`; its return type `MemoryContainsTest` is not re-exported from the package entry. No accepted filter key or operator is added: `$contains` and `$notContains` keep their declared shape. + +On a field whose declaration makes it JSON-stored (`multiple: true` on a `lookup`, `user`, `select`, `radio`, `file` or `image` field, a `multiselect`, `checkboxes` or `tags` field, or a structured type such as `json`), the in-memory driver now answers `{ field: { $contains: v } }` by whole-element membership: some element of the stored array equals `v`. It used to match each element by substring, so `u1` matched a row storing `['u10']` and `'red'` matched a row storing `['redwood']`. A number member answered nothing: `{ nums: { $contains: '1' } }` missed `[1, 2]`. `$notContains` is the exact complement, and a row with no value still satisfies it. A scalar text column keeps the case-exact substring test. + +`driver-sql` gives the same answer on SQLite, PostgreSQL and MySQL; the two drivers were measured over the same fixture. The answer holds on every face of this driver: + +- `find()` and `count()`, in both filter spellings; +- the nested-relation filter on a multi-valued relation, which the engine lowers to one `$contains` per related id; +- `MemoryAnalyticsService`'s query, and its SQL echo, which now renders SQLite's `json_each` membership construct for such a column. + +The comparand is still a string. A number or boolean member is named by its text: `'1'` matches the stored number `1` (and `'1.50'` the number `1.5`), `'true'` matches the boolean `true`, and `'null'` matches a `null` member. A field the driver holds no declaration for, such as a field on an object never passed through `syncSchema`, keeps the substring reading. + +New: `InMemoryDriver.filterContainsTest(object, field, value)` returns the one test every face above lowers `$contains` to. It is a narrow seam for the analytics face, beside `filterSubstringPattern` and `filterComparandStorageForm`. The added public method is why this entry is `minor`. + +**If your tests relied on the old answer:** on the in-memory driver, a filter that matched an id by prefix or a tag by substring now returns only the member rows. That is what SQL already returned in production. Write `$contains` with the whole member value. diff --git a/packages/drivers/driver-memory/src/memory-20874-contains-membership.test.ts b/packages/drivers/driver-memory/src/memory-20874-contains-membership.test.ts new file mode 100644 index 00000000000..2e22ed0cce7 --- /dev/null +++ b/packages/drivers/driver-memory/src/memory-20874-contains-membership.test.ts @@ -0,0 +1,339 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#20874] `$contains` / `$notContains` on a DECLARED JSON-stored field ask + * MEMBERSHIP on every face of `driver-memory` — the rows `driver-sql` answers. + * + * ## The defect, measured on `origin/main` `f6ccca4a` before the diff + * + * The query path lowered `$contains` to an escaped `$regex`, and mingo applies + * a `$regex` to EACH ELEMENT of an array value, so a stored array answered by + * per-element substring. The analytics face borrowed the same pattern + * (`filterSubstringPattern`) and wrapped it in a `$regex` of its own: + * + * | `where` | memory, before | SQLite (`driver-sql`) | memory, now | + * |:--|:--|:--|:--| + * | `{ owners: { $contains: 'u1' } }` over `['u1','u2']`, `['u10']`, `['u3','u1']`, `[]` | 1, **2**, 3 | 1, 3 | 1, 3 | + * | `{ owners: { $notContains: 'u1' } }` | 4 — **2 dropped** | 2, 4 | 2, 4 | + * | `{ tags_: { $contains: 'red' } }` over `['red','blue']`, `['redwood']` | 1, **2** | 1 | 1 | + * | `{ nums: { $contains: '1' } }` over `[1, 2]` (a number member) | **none** | 1 | 1 | + * | the analytics face, the same three | the query path's answer | — | the query path's answer | + * + * Measured through `engine.find` too, with the nested-relation filter + * `{ owners: { region: 'NA' } }` on a multi-valued lookup (owner `u1` is NA, + * `u10` is EU): `d1`, `d3`, **`d5`** before; `d1`, `d3` now, and its `$not` + * admits `d5` again. That engine run was a one-off: this package may not + * import the engine and the engine's packages may not import this driver + * (`check:driver-memory-census`), so what is pinned here is the driver input + * the engine sends — an `$or` of one `$contains` per related id, asserted by + * `@objectstack/objectql`'s `engine-nested-relation-lowering.test.ts`. + * + * ## The fork is the DECLARED field, never the row + * + * The contract (`FILTER_OPERATORS`' `$contains` docblock in `@objectstack/spec`) + * selects the question by the COLUMN: a `multiple: true` field or a + * JSON-stored type asks membership, a scalar string column asks substring. + * `driver-sql` forks on its JSON-column registry, filled from the declaration; + * this driver forks on the declaration `syncSchema` recorded. The two readings + * part on exactly one kind of row, and SQLite decides it: a declared `json` + * field holding the SCALAR string `'u1'` answers no member there (its + * constructs are array-only), where a fork read off the row would have + * answered it by substring. Fixture two holds that row. + * + * ## Why the rows are literal, and mirror `driver-sql`'s + * + * Fixture one is `driver-sql`'s `sql-driver-17590-json-column-membership.test.ts` + * fixture row for row, and the expected ids are that file's, which run on + * SQLite and on live PostgreSQL and MySQL. Fixture two's ids were measured on + * `driver-sql`/SQLite over the same rows before this file was written. The two + * packages cannot import one fixture (neither depends on the other), so the + * shared answer is the literal row set — the same way that file holds its three + * dialect cells to one another. `FILTER_TEXT_CASES` is not the carrier: it has + * no array column, and `driver-mongodb`, which imports every row of it, lowers + * `$contains` to a native `$regex` that MongoDB also applies per element. + */ + +import { describe, it, expect, beforeAll } from 'vitest'; +import type { Cube, FilterCondition } from '@objectstack/spec/data'; +import { InMemoryDriver } from './memory-driver.js'; +import { MemoryAnalyticsService } from './memory-analytics.js'; + +const byId = (a: string, b: string) => a.localeCompare(b); + +/** `driver-sql`'s #17590 fixture, plus its #20874 `owners` column. */ +const MEMBERSHIP = { + object: 'mem20874_membership', + fields: { + label: { type: 'text' }, + tags_: { type: 'tags' }, + picks: { type: 'multiselect' }, + nums: { type: 'select', multiple: true }, + owners: { type: 'lookup', reference: 'mem20874_owner', multiple: true }, + }, + rows: [ + { id: '1', label: 'redwood', tags_: ['red', 'blue'], picks: ['a', 'b'], nums: [1, 2], owners: ['u1', 'u2'] }, + { id: '2', label: 'red', tags_: ['redwood'], picks: ['ab'], nums: [10, 21], owners: ['u10'] }, + { id: '3', label: 'blue', tags_: ['blue'], picks: ['b'], nums: [2], owners: ['u3', 'u1'] }, + { id: '4', label: 'none', tags_: [], picks: [], nums: [], owners: [] }, + ], +} as const; + +/** + * Stored SHAPES the membership reading has to get right, each answered by + * `driver-sql`/SQLite as recorded in {@link SHAPE_CASES}: a declared `json` + * field holding an array, a scalar, an object, a NESTED array, a `null` member, + * a boolean and a fractional number, and a NULL row. + */ +const SHAPES = { + object: 'mem20874_shapes', + fields: { + label: { type: 'text' }, + blob: { type: 'json' }, + picks: { type: 'multiselect' }, + }, + rows: [ + { id: 'e1', label: 'x', blob: ['u1'], picks: ['a'] }, + { id: 'e2', label: 'x', blob: 'u1', picks: ['true'] }, + { id: 'e3', label: 'x', blob: { k: 'u1' }, picks: [] }, + { id: 'e4', label: 'x', blob: [['u1']], picks: ['null'] }, + { id: 'e5', label: 'x', blob: [null], picks: [] }, + { id: 'e6', label: 'x', blob: [true, 1.5], picks: [] }, + { id: 'e7', label: 'x', blob: null, picks: null }, + { id: 'e8', label: 'x', blob: [[null]], picks: [] }, + { id: 'e9', label: 'x', blob: [''], picks: [''] }, + ], +} as const; + +type Fixture = typeof MEMBERSHIP | typeof SHAPES; +type Case = readonly [name: string, where: FilterCondition, expected: readonly string[]]; + +/** Fixture one — `driver-sql`'s #17590 assertions, verbatim, and #20874's. */ +const MEMBERSHIP_CASES: readonly Case[] = [ + ['tags: the member row, never the substring row', { tags_: { $contains: 'red' } }, ['1']], + ['tags: a whole member', { tags_: { $contains: 'redwood' } }, ['2']], + ['tags: two member rows', { tags_: { $contains: 'blue' } }, ['1', '3']], + ['tags: a substring of a member is no member', { tags_: { $contains: 'wood' } }, []], + ['multiselect: the member row', { picks: { $contains: 'a' } }, ['1']], + ['multiselect: a whole member', { picks: { $contains: 'ab' } }, ['2']], + ['multiselect: two member rows', { picks: { $contains: 'b' } }, ['1', '3']], + ['number members are named by their text', { nums: { $contains: '1' } }, ['1']], + ['number members, two rows', { nums: { $contains: '2' } }, ['1', '3']], + ['number members: 10 is not 1', { nums: { $contains: '10' } }, ['2']], + ['number members: a digit of a member is no member', { nums: { $contains: '0' } }, []], + ['lookup: u1 is not a member of [u10] (the card)', { owners: { $contains: 'u1' } }, ['1', '3']], + ['lookup: u10 is its own member', { owners: { $contains: 'u10' } }, ['2']], + ['lookup: $notContains admits [u10] and []', { owners: { $notContains: 'u1' } }, ['2', '4']], + ['a SCALAR text column keeps the substring test (the control)', { label: { $contains: 'red' } }, ['1', '2']], + ['a SCALAR text column, the other control', { label: { $contains: 'wood' } }, ['1']], +]; + +/** Fixture two — each answer measured on `driver-sql`/SQLite over the same rows. */ +const SHAPE_CASES: readonly Case[] = [ + ['json: only a TOP-LEVEL element is a member — not a scalar, an object or a nested array', { blob: { $contains: 'u1' } }, ['e1']], + ['json: $notContains is the exact complement, the NULL row included', { blob: { $notContains: 'u1' } }, ['e2', 'e3', 'e4', 'e5', 'e6', 'e7', 'e8', 'e9']], + ["json: 'null' names a null member, never one inside a nested array", { blob: { $contains: 'null' } }, ['e5']], + ["json: 'true' names a boolean member", { blob: { $contains: 'true' } }, ['e6']], + ["json: '1.50' names the number 1.5", { blob: { $contains: '1.50' } }, ['e6']], + ['json: the empty string names an empty-string member only', { blob: { $contains: '' } }, ['e9']], + ["multiselect: 'true' names the STRING member too", { picks: { $contains: 'true' } }, ['e2']], + ["multiselect: 'null' names the STRING member too", { picks: { $contains: 'null' } }, ['e4']], + ['multiselect: the empty string', { picks: { $contains: '' } }, ['e9']], + ['multiselect: $notContains, the NULL and empty rows included', { picks: { $notContains: 'a' } }, ['e2', 'e3', 'e4', 'e5', 'e6', 'e7', 'e8', 'e9']], +]; + +const FIXTURES: ReadonlyArray = [ + [MEMBERSHIP, MEMBERSHIP_CASES], + [SHAPES, SHAPE_CASES], +]; + +async function seed(fixture: Fixture, { declare = true } = {}): Promise { + const driver = new InMemoryDriver(); + if (declare) await driver.syncSchema(fixture.object, { name: fixture.object, fields: fixture.fields }); + for (const row of fixture.rows) await driver.create(fixture.object, structuredClone({ ...row })); + return driver; +} + +const findIds = async (driver: InMemoryDriver, object: string, where: unknown): Promise => + ((await driver.find(object, { where: where as any })) as any[]).map((r) => String(r.id)).sort(byId); + +/** The single-field, single-operator case as the AST node the other spelling takes. */ +function astOf(where: FilterCondition): unknown { + const [field, ops] = Object.entries(where)[0] as [string, Record]; + const [op, value] = Object.entries(ops)[0] as [string, unknown]; + const operator = op === '$contains' ? 'contains' : op === '$notContains' ? 'not_contains' : null; + if (!operator) throw new Error(`no AST spelling for ${op} in this file`); + return { type: 'comparison', field, operator, value }; +} + +function cubeOf(fixture: Fixture): Cube { + const dimensions: Record = { id: { label: 'Id', type: 'string', sql: 'id' } }; + for (const field of Object.keys(fixture.fields)) dimensions[field] = { label: field, type: 'string', sql: field }; + return { + name: 'members', + title: 'Members', + sql: fixture.object, + measures: { count: { label: 'Count', type: 'count', sql: 'id' } }, + dimensions, + } as unknown as Cube; +} + +const analyticsQuery = (where: unknown) => + ({ cube: 'members', measures: ['members.count'], dimensions: ['members.id'], where }) as any; + +for (const [fixture, cases] of FIXTURES) { + describe(`[#20874] ${fixture.object} — $contains on a declared JSON-stored field is membership, on every face`, () => { + let driver: InMemoryDriver; + let service: MemoryAnalyticsService; + beforeAll(async () => { + driver = await seed(fixture); + service = new MemoryAnalyticsService({ driver, cubes: [cubeOf(fixture)] }); + }); + + it('stored every row — the premise of every answer below', async () => { + expect(await findIds(driver, fixture.object, {})).toEqual(fixture.rows.map((r) => r.id).sort(byId)); + }); + + for (const [name, where, expected] of cases) { + it(`query path: ${name}`, async () => { + expect(await findIds(driver, fixture.object, where)).toEqual([...expected]); + }); + + it(`query path, AST spelling: ${name}`, async () => { + expect(await findIds(driver, fixture.object, astOf(where))).toEqual([...expected]); + }); + + it(`count(): ${name}`, async () => { + expect(await driver.count(fixture.object, { where: where as any })).toBe(expected.length); + }); + + // The invariant across this package's faces: the same row set as + // `find()`, never a third, quieter answer (#5374). + it(`analytics face: ${name}`, async () => { + const result = await service.query(analyticsQuery(where)); + expect(result.rows.map((r: any) => String(r['members.id'])).sort(byId)).toEqual([...expected]); + }); + } + }); +} + +/** + * The analytics face's SQL ECHO, EXECUTED — on a real SQLite engine (sql.js) + * over the same rows in `driver-sql`'s stored form (a JSON-stored field holds + * its JSON text). The echo's job is reproducing execution + * (`memory-analytics-echo-operator-coverage.test.ts`), so once the `$match` it + * describes asks membership, a `GLOB '*u1*'` echo over the text `["u10"]` would + * return the row the chart excludes. + */ +describe('[#20874] the analytics echo renders the membership that ran', () => { + const echoes: Array<{ fixture: Fixture; cases: readonly Case[]; db: any; service: MemoryAnalyticsService }> = []; + + beforeAll(async () => { + const mod: any = await import('sql.js'); + const initSqlJs = mod.default ?? mod; + // sql.js locates its own `.wasm` under Node; the explicit path is the + // echo-coverage file's belt-and-braces, taken when it resolves. + let locateFile: ((file: string) => string) | undefined; + try { + const { createRequire } = await import('node:module'); + const { dirname, join } = await import('node:path'); + const dir = dirname(createRequire(import.meta.url).resolve('sql.js/package.json')); + locateFile = (file: string) => join(dir, 'dist', file); + } catch { + locateFile = undefined; + } + const SQL = await initSqlJs(locateFile ? { locateFile } : undefined); + for (const [fixture, cases] of FIXTURES) { + const columns = ['id', ...Object.keys(fixture.fields)]; + const db = new SQL.Database(); + db.run(`CREATE TABLE ${fixture.object} (${columns.map((c) => `${c} TEXT`).join(', ')});`); + const insert = db.prepare(`INSERT INTO ${fixture.object} VALUES (${columns.map(() => '?').join(', ')})`); + for (const row of fixture.rows as ReadonlyArray>) { + insert.run(columns.map((c) => { + const v = row[c]; + if (c === 'id' || c === 'label') return v; + return v == null ? null : JSON.stringify(v); + })); + } + insert.free(); + const driver = await seed(fixture); + echoes.push({ fixture, cases, db, service: new MemoryAnalyticsService({ driver, cubes: [cubeOf(fixture)] }) }); + } + }); + + const echoIds = async (db: any, service: MemoryAnalyticsService, where: unknown): Promise => { + const { sql } = await service.generateSql(analyticsQuery(where)); + const stmt = db.prepare(sql); + const out: string[] = []; + while (stmt.step()) out.push(String(Object.values(stmt.getAsObject())[0])); + stmt.free(); + return out.sort(byId); + }; + + it('the SQLite engine has json_each and both tables hold every row (the premise)', async () => { + for (const { fixture, db, service } of echoes) { + expect(db.exec(`SELECT count(*) FROM json_each('[1,2]')`)[0].values[0][0]).toBe(2); + expect(await echoIds(db, service, {})).toEqual(fixture.rows.map((r) => r.id).sort(byId)); + } + }); + + it('every case: the echoed statement returns the rows the face executed', async () => { + for (const { cases, db, service } of echoes) { + for (const [name, where, expected] of cases) { + expect(await echoIds(db, service, where), name).toEqual([...expected]); + } + } + }); + + it('a JSON-stored column echoes the membership construct, a scalar column the GLOB it always did', async () => { + const { service } = echoes[0]!; + const member = (await service.generateSql(analyticsQuery({ owners: { $contains: 'u1' } }))).sql; + expect(member).toContain('json_each(CASE WHEN json_valid(owners)'); + expect(member).toContain(`IN ('"u1"')`); + expect(member).not.toContain('GLOB'); + const negated = (await service.generateSql(analyticsQuery({ owners: { $notContains: 'u1' } }))).sql; + expect(negated).toContain('(owners IS NULL OR NOT EXISTS ('); + const scalar = (await service.generateSql(analyticsQuery({ label: { $contains: 'red' } }))).sql; + expect(scalar).toContain("label GLOB '*red*'"); + }); +}); + +/** + * The nested-relation filter on a multi-valued relation, as the engine hands it + * to a driver: an `$or` of one `$contains` per related id, and its `$not`. + * With owner `u1` the only NA owner, `{ owners: { region: 'NA' } }` becomes the + * first `where` below — and the row holding `['u10']` must not answer it. + */ +describe('[#20874] the nested-relation lowering, as a driver receives it', () => { + let driver: InMemoryDriver; + beforeAll(async () => { driver = await seed(MEMBERSHIP); }); + + it('an $or of $contains per related id answers the member rows only', async () => { + expect(await findIds(driver, MEMBERSHIP.object, { $or: [{ owners: { $contains: 'u1' } }] })).toEqual(['1', '3']); + expect(await findIds(driver, MEMBERSHIP.object, { $or: [{ owners: { $contains: 'u1' } }, { owners: { $contains: 'u2' } }] })) + .toEqual(['1', '3']); + }); + + it('its $not admits the row whose id only PREFIXES a member, and the empty row', async () => { + expect(await findIds(driver, MEMBERSHIP.object, { $not: { $or: [{ owners: { $contains: 'u1' } }] } })).toEqual(['2', '4']); + }); +}); + +/** + * The population is the DECLARATION. A field with none keeps the substring + * reading it always had — `driver-sql` answers a table it was never told about + * the same way — so the scalar control holds on an undeclared object too. + */ +describe('[#20874] the fork is the declared field', () => { + it('a scalar text column keeps substring whether or not the object was declared', async () => { + for (const declare of [true, false]) { + const driver = await seed(MEMBERSHIP, { declare }); + expect(await findIds(driver, MEMBERSHIP.object, { label: { $contains: 'red' } }), `declare=${declare}`).toEqual(['1', '2']); + } + }); + + it('a declared json field holding a SCALAR string has no member — the row a value-shape fork would answer', async () => { + const driver = await seed(SHAPES); + expect(await findIds(driver, SHAPES.object, { blob: { $contains: 'u1' } })).not.toContain('e2'); + expect(await findIds(driver, SHAPES.object, { blob: { $notContains: 'u1' } })).toContain('e2'); + }); +}); diff --git a/packages/drivers/driver-memory/src/memory-analytics.ts b/packages/drivers/driver-memory/src/memory-analytics.ts index cc0aaa76402..9de9198bd6f 100644 --- a/packages/drivers/driver-memory/src/memory-analytics.ts +++ b/packages/drivers/driver-memory/src/memory-analytics.ts @@ -20,7 +20,7 @@ import { // call: `isFilterAST` gates the shape, `parseFilterAST` lowers it. See // {@link lowerWhereFilterArray}. ⛔ No second FilterArray parser in this face. import { isFilterAST, parseFilterAST, VALID_AST_OPERATORS } from '@objectstack/spec/data'; -import type { InMemoryDriver } from './memory-driver.js'; +import type { InMemoryDriver, MemoryContainsTest } from './memory-driver.js'; import { Logger, createLogger, @@ -238,15 +238,23 @@ interface MongoPredicateInput { /** The operands as authored. For operands that are not comparands. */ readonly raw: readonly unknown[]; /** - * A comparand as a literal-substring pattern, built by the DRIVER's own rule - * (`filterSubstringPattern`) rather than re-derived here. + * The test `$contains` asks of this member's column, built by the DRIVER's + * own rule (`filterContainsTest`) rather than re-derived here — `$notContains` + * wraps it in `$not`, as the live query path does. * * [#7723] Case-EXACT, because that rule is: `filterSubstringPattern` carried * an `i` flag until #7723 took it off, putting the `$contains` family on the * #4706 Q2 = A answer across every face of this package. Borrowing the rule * rather than restating it is what made that one edit reach this face too. + * + * [#20874] Borrowed WHOLE now, not as a pattern. On a declared JSON-stored + * field the rule asks MEMBERSHIP (an `$elemMatch`), not substring, and this + * face used to wrap `filterSubstringPattern` in a `$regex` of its own — so + * `u1` matched a stored `['u10']` here exactly as it did on `find()`. Taking + * the predicate rather than a piece of it is what makes the membership reading + * reach this face with the same edit, the lesson #7723 records above. */ - readonly substring: (value: unknown) => RegExp; + readonly containment: (value: unknown) => MemoryContainsTest; /** * [#6520] A comparand as an ASCII-case-insensitive literal-substring pattern — * `$icontains`' fold, which is NOT {@link substring}'s. @@ -335,8 +343,9 @@ const CUBE_OPERATOR_TO_MONGO_PREDICATE: Readonly ({ $in: [...comparands] }), notIn: ({ comparands }) => ({ $nin: [...comparands] }), - // A pattern, not a comparand: `raw`, and the driver's own substring rule. - contains: ({ raw, substring }) => ({ $regex: substring(raw[0]) }), + // A pattern, not a comparand: `raw`, and the driver's own rule — membership on + // a declared JSON-stored field, the substring everywhere else (#20874). + contains: ({ raw, containment }) => containment(raw[0]), // [#6520] The case-INSENSITIVE twin, folding ASCII and nothing else. It takes // `asciiSubstring`, not `substring`: the neighbour above folds Unicode, so // reusing it here would answer `CAFÉ` for `café` on this face while the SQL @@ -344,8 +353,9 @@ const CUBE_OPERATOR_TO_MONGO_PREDICATE: Readonly ({ $regex: asciiSubstring(raw[0]) }), // The fix this issue is about. `{$not: }` constrains nothing; the // negation has to wrap a pattern, which is exactly what the live query path - // builds for `$notContains` (`memory-driver.ts` `normalizeFieldOperators`). - notContains: ({ raw, substring }) => ({ $not: { $regex: substring(raw[0]) } }), + // builds for `$notContains` (`memory-driver.ts` `normalizeFieldOperators`) — + // and, since #20874, the SAME test it builds, membership or substring. + notContains: ({ raw, containment }) => ({ $not: containment(raw[0]) }), // [#13195] A presence flag, not a comparand — and "present" means HAS A // VALUE (`!= null`), never key presence: #5298 leg 3 / #5369, landed in PR // #5962, ruled onto this face 2026-08-30. It used to emit `{$exists: }` @@ -386,6 +396,52 @@ interface SqlPredicateInput { * See {@link globSubstringPattern} for why GLOB and not LIKE. */ readonly globSubstring: (value: unknown) => string; + /** + * [#20874] The stored members a `$contains` comparand names on this column, + * or `null` when the column asks the SUBSTRING question — read off the + * DRIVER's test (`filterContainsTest`), the one its `$match` twin executes. + * See {@link sqliteMembershipPredicate}. + */ + readonly members: (value: unknown) => readonly unknown[] | null; +} + +/** + * [#20874] The SQLite rendering of a `$contains` MEMBERSHIP test — `driver-sql`'s + * own SQLite construct (`jsonMembershipPredicate`), with literals where that + * one binds. + * + * The echo's job is reproducing execution ({@link globSubstringPattern}). Once + * the `$match` twin asks membership on a declared JSON-stored column, a + * `GLOB '*u1*'` echo over the stored text `["u10"]` would return the row the + * chart excludes — so this renders the question that ran: + * + * - `json_each` over the column, guarded by `json_valid` so a cell holding + * bare text has no members instead of raising — the guard `driver-sql` keeps + * for the same reason; + * - `typeof(os_member.key) = 'integer'` keeps it array-only: an array element + * has an INTEGER key, an object member a TEXT key and a scalar root a NULL + * one — the same "a scalar or object has no member" the `$elemMatch` twin + * answers; + * - each element's JSON TEXT compared with each member's, the `CASE` spelling + * the three JSON literals by type name because SQLite surfaces `true` as the + * INTEGER 1 and `json_quote` would render it `1`. + * + * The members' JSON texts are exactly the texts `driver-sql` binds, because + * both sides read the comparand the same way (`containsMemberCandidates` in + * `memory-driver.ts`, `jsonMembershipCandidates` in `driver-sql`). + */ +function sqliteMembershipPredicate( + column: string, + members: readonly unknown[], + literal: (value: unknown) => string, +): string { + const texts = members.map((member) => literal(JSON.stringify(member))).join(', '); + return ( + `EXISTS (SELECT 1 FROM json_each(CASE WHEN json_valid(${column}) THEN ${column} ELSE '[]' END) AS os_member ` + + `WHERE typeof(os_member.key) = 'integer' AND CASE os_member.type ` + + `WHEN 'true' THEN 'true' WHEN 'false' THEN 'false' WHEN 'null' THEN 'null' ` + + `ELSE json_quote(os_member.value) END IN (${texts}))` + ); } type SqlPredicateBuilder = (input: SqlPredicateInput) => string; @@ -537,10 +593,20 @@ const CUBE_OPERATOR_TO_SQL_PREDICATE: Readonly `${column} GLOB ${globSubstring(raw[0])}`, - notContains: ({ column, raw, globSubstring }) => - `(${column} IS NULL OR ${column} NOT GLOB ${globSubstring(raw[0])})`, + // A pattern, not a comparand: `raw`, and the shared GLOB substring rule — or, + // on a declared JSON-stored column, the membership the `$match` twin runs + // (#20874, {@link sqliteMembershipPredicate}). The negation stays null-safe + // either way: `NOT EXISTS` is never UNKNOWN, but the NULL row still has to be + // admitted by name, since `json_each(NULL)` is not a row set to negate. + contains: ({ column, raw, globSubstring, members, literal }) => { + const set = members(raw[0]); + return set ? sqliteMembershipPredicate(column, set, literal) : `${column} GLOB ${globSubstring(raw[0])}`; + }, + notContains: ({ column, raw, globSubstring, members, literal }) => { + const set = members(raw[0]); + const test = set ? `NOT ${sqliteMembershipPredicate(column, set, literal)}` : `${column} NOT GLOB ${globSubstring(raw[0])}`; + return `(${column} IS NULL OR ${test})`; + }, // [#6520] The case-INSENSITIVE twin. SQLite's `lower()` folds ASCII and // nothing else — measured in #6518: `lower('CAFÉ')` is `'cafÉ'` — so it is // `$icontains`' fold (#4706 Q1 = A) rather than the Unicode one, and it goes @@ -1595,10 +1661,11 @@ export class MemoryAnalyticsService implements IAnalyticsService { // predicate over `is_active` or `closed_at` selects the same rows // `find()` selects instead of none / all of them. const storageForm = this.storageFormFor(cube, filter.member); + const table = this.extractTableName(cube.sql); const predicate = this.mongoPredicateBuilder(filter.operator)({ comparands: filter.values.map(storageForm), raw: filter.values, - substring: (value) => this.driver.filterSubstringPattern(value), + containment: (value) => this.driver.filterContainsTest(table, fieldPath, value), // [#6520] `$icontains`' fold, from the spec's shared definition rather // than from the driver's Unicode-folding `filterSubstringPattern`. asciiSubstring: (value) => new RegExp(asciiCaseInsensitiveRegexSource(String(value))), @@ -1663,12 +1730,19 @@ export class MemoryAnalyticsService implements IAnalyticsService { } const fieldPath = this.resolveFieldPath(cube, entry.member); const storageForm = this.storageFormFor(cube, entry.member); + const table = this.extractTableName(cube.sql); clauses.push(this.sqlPredicateBuilder(entry.operator)({ column: fieldPath, comparands: entry.values.map(storageForm), raw: entry.values, literal: (value) => this.toSqlLiteral(value), globSubstring: (value) => this.toSqlLiteral(globSubstringPattern(value)), + members: (value) => { + // [#20874] Read off the very test the `$match` exit runs, so the echo + // and the chart can never name two different member sets. + const test = this.driver.filterContainsTest(table, fieldPath, value); + return '$elemMatch' in test ? test.$elemMatch.$in : null; + }, })); } return clauses; diff --git a/packages/drivers/driver-memory/src/memory-driver.ts b/packages/drivers/driver-memory/src/memory-driver.ts index 4df636dc892..d376dadbd86 100644 --- a/packages/drivers/driver-memory/src/memory-driver.ts +++ b/packages/drivers/driver-memory/src/memory-driver.ts @@ -13,6 +13,10 @@ import { hasDanglingLikeEscape, hasNulInLikePattern, likePatternToRegExp } from // [#20444] The `$empty` operator's ONE expansion — the field's declared row of // the ruled 「is empty」 table, asked of the spec by the live query path. import { expandEmptyOperator, type ValueShapeFieldDef } from '@objectstack/spec/data'; +// [#20874] The JSON-stored population — the declared fields on which +// `$contains` asks MEMBERSHIP — from the spec's value-shape classes, the same +// two `driver-sql`'s JSON-column registry is built from. +import { STRUCTURED_JSON_TYPES, isMultiValueField } from '@objectstack/spec/data'; import type { DriverQuery, IDataDriver } from '@objectstack/spec/contracts'; import { Logger, createLogger, compensatedSum } from '@objectstack/core'; import { Query, Aggregator } from 'mingo'; @@ -127,6 +131,9 @@ interface LoweredWrite { * `$not` is written by `$notContains` and by NOTHING else — it is covered here * by construction rather than curatively, which is the point of ranging over * the vocabulary instead of over the three operators that had been noticed. + * [#20874] `$elemMatch` joins it on the same terms: written by `$contains` on a + * JSON-stored field and by nothing else (an author cannot write `$elemMatch` — + * the shape gate refuses it), so it is never contested either. * * ## The rule, and why it is this one * @@ -325,6 +332,64 @@ interface MemoryTransaction { snapshot: Record; } +/** + * [#20874] The JSON NUMBER grammar, spelled out — the pattern `driver-sql`'s + * `jsonMembershipCandidates` tests a comparand against, for its reason: + * `Number()` also accepts `'0x10'`, `' 1 '`, `'Infinity'` and `''`, none of + * which is a JSON number, and admitting them would make the member set depend + * on JS coercion rules no SQL dialect shares. + */ +const JSON_NUMBER_TEXT = /^-?(?:0|[1-9][0-9]*)(?:\.[0-9]+)?(?:[eE][+-]?[0-9]+)?$/; + +/** + * [#20874] The stored MEMBERS a `$contains` / `$notContains` comparand names on + * a JSON-stored field — the in-memory twin of `driver-sql`'s + * `jsonMembershipCandidates`, which reads one comparand the same way for all + * three SQL dialects. + * + * The contract declares the comparand a STRING (`FILTER_OPERATORS`' `$contains` + * docblock in `@objectstack/spec`), so a member stored as a JSON number or + * boolean is named by its TEXT: `'1'` names the string `'1'` OR the number `1`, + * `'true'` the string OR `true`, `'null'` the string OR `null`. The number is + * read through `Number()`, so `'1.50'` names a stored `1.5`, as it does on + * every SQL dialect. ⛔ Not a lenient alias: one declared comparand type read + * against one stored shape, decided in the driver so every backend gets the + * same reading. + * + * `String(value)` is the rendering the substring reading gives the comparand + * ({@link InMemoryDriver.filterSubstringPattern}) and the one `driver-sql` + * gives it, so the two readings never disagree about WHICH text was asked for. + * Each candidate's `JSON.stringify` is exactly the JSON text `driver-sql` + * compares a stored element against, which is what lets the analytics face's + * SQLite echo name the same set (`memory-analytics.ts`). + * + * @see InMemoryDriver.filterContainsTest — where the population decides + * whether a comparand is read this way at all. + */ +function containsMemberCandidates(value: unknown): ContainsMember[] { + const text = String(value); + if (text === 'true') return [text, true]; + if (text === 'false') return [text, false]; + if (text === 'null') return [text, null]; + if (JSON_NUMBER_TEXT.test(text)) { + const parsed = Number(text); + if (Number.isFinite(parsed)) return [text, parsed]; + } + return [text]; +} + +/** [#20874] A stored member a `$contains` comparand can name. */ +type ContainsMember = string | number | boolean | null; + +/** + * [#20874] The un-negated test `$contains` lowers to on this driver — + * membership on a declared JSON-stored field, the substring pattern elsewhere. + * See {@link InMemoryDriver.filterContainsTest}. + */ +export type MemoryContainsTest = + | { $elemMatch: { $in: ContainsMember[]; $not: { $type: 'array' } } } + | { $regex: RegExp }; + /** * [#20444] Each field's declared value shape — the `type` and `multiple` slice * `expandEmptyOperator` reads — for the fields that declare a `type`. The RAW @@ -1428,8 +1493,12 @@ export class InMemoryDriver implements IDataDriver { // returned rows it excludes, which on an RLS read scope is over-reach // rather than a loose filter (#3948). `escapeRegex` stays: the comparand // was always literal, and that half was never the defect. + // + // [#20874] `contains` / `not_contains` take the ONE test the `$`-spelling + // takes ({@link filterContainsTest}): membership on a declared JSON-stored + // field, the case-exact substring everywhere else. case 'contains': - return { [field]: { $regex: new RegExp(this.escapeRegex(value)) } }; + return { [field]: this.filterContainsTest(object, field, value) }; // [#7536] `like` / `ilike` are NOT `contains`, and sharing this arm with // it was the memory-face twin of the wire defect #7536 closed: the // comparand was regex-ESCAPED (so a caller's `%` matched a literal percent @@ -1451,7 +1520,7 @@ export class InMemoryDriver implements IDataDriver { }; } case 'notcontains': case 'not_contains': - return { [field]: { $not: { $regex: new RegExp(this.escapeRegex(value)) } } }; + return { [field]: { $not: this.filterContainsTest(object, field, value) } }; case 'startswith': case 'starts_with': return { [field]: { $regex: new RegExp(`^${this.escapeRegex(value)}`) } }; case 'endswith': case 'ends_with': @@ -1580,7 +1649,7 @@ export class InMemoryDriver implements IDataDriver { if (Object.keys(rest).length === 0) continue; fieldOps = rest; } - const normalized = this.normalizeFieldOperators(fieldOps, this.temporalKind(object, key), key, here); + const normalized = this.normalizeFieldOperators(fieldOps, this.temporalKind(object, key), key, here, object); // [#13524] Lowered writes whose mingo key was already taken by a // sibling operator on the same field. They cannot be merged without one // of the two constraints silently overwriting the other, so each @@ -1634,8 +1703,17 @@ export class InMemoryDriver implements IDataDriver { * * `field` and `path` are carried only so a refusal can name the position it * refused — the vocabulary itself is enforced one level up (#5324). + * [#20874] `object` is carried for `$contains` / `$notContains` alone: which + * question they ask is the FIELD's declared storage shape + * ({@link filterContainsTest}), and `field` names it only together with `object`. */ - private normalizeFieldOperators(ops: Record, kind?: TemporalFieldKind, field = '', path = 'filter'): Record { + private normalizeFieldOperators( + ops: Record, + kind?: TemporalFieldKind, + field = '', + path = 'filter', + object?: string, + ): Record { const store = (v: any) => coerceTemporalValue(v, kind); const regexConditions: Record[] = []; /** @@ -1660,11 +1738,21 @@ export class InMemoryDriver implements IDataDriver { // method up (`convertConditionToMongo`) and for the same reason — see // the note there. The comparand stays `escapeRegex`-literal; only the // Unicode-folding `i` flag is gone. - case '$contains': - regexConditions.push({ $regex: new RegExp(this.escapeRegex(val)) }); + // + // [#20874] `$contains` / `$notContains` ask the ONE test + // {@link filterContainsTest} builds: MEMBERSHIP on a declared JSON-stored + // field, the substring pattern everywhere else. The substring test + // still joins `regexConditions`, so it composes with `$startsWith` / + // `$endsWith` exactly as before; the membership test lowers to + // `$elemMatch`, a key no other operator writes. + case '$contains': { + const test = this.filterContainsTest(object, field, val); + if ('$elemMatch' in test) put('$elemMatch', test.$elemMatch); + else regexConditions.push(test); break; + } case '$notContains': - put('$not', { $regex: new RegExp(this.escapeRegex(val)) }); + put('$not', this.filterContainsTest(object, field, val)); break; case '$startsWith': regexConditions.push({ $regex: new RegExp(`^${this.escapeRegex(val)}`) }); @@ -2225,6 +2313,10 @@ export class InMemoryDriver implements IDataDriver { * [#5374] The pattern a `$contains` / `$notContains` comparand becomes — the * substring rule itself, for the analytics (cube) face. * + * [#20874] On a scalar column. A declared JSON-stored field asks MEMBERSHIP + * instead, so the analytics face now takes the whole predicate + * ({@link filterContainsTest}), which uses this for the substring half. + * * Same reasoning as {@link filterComparandStorageForm} one method up, on the * other half of what a `contains` predicate needs. This driver's rule is * `escapeRegex` and NO flags ({@link normalizeFieldOperators}): the comparand @@ -2260,6 +2352,85 @@ export class InMemoryDriver implements IDataDriver { return new RegExp(this.escapeRegex(value as string)); } + /** + * [#20874] Is `field` of `object` DECLARED JSON-stored — the population on + * which `$contains` / `$notContains` ask MEMBERSHIP rather than SUBSTRING? + * + * The contract (`FILTER_OPERATORS`' `$contains` docblock, `@objectstack/spec`) + * selects the question by the COLUMN: on a `multiple: true` field or a + * JSON-stored type, `$contains: v` asks whether `v` is a member of the stored + * array; on a scalar string column it stays the substring test. The storage + * shape is DECLARED metadata, so this reads the declaration {@link syncSchema} + * recorded ({@link valueShapes}) and never the row: `driver-sql` forks on its + * JSON-column registry the same way (`SqlDriver.isJsonColumn`), and a fork read + * off each row's value would answer a declared JSON field holding a scalar + * string by substring where every SQL dialect answers no member. + * + * The population is the spec's JSON-stored classes — `STRUCTURED_JSON_TYPES` + * and every multi-valued field (`isMultiValueField`, which covers + * `MULTI_OPTION_TYPES`) — the two halves `driver-sql`'s registry is built + * from. Two members of that registry are deliberately not here: its + * driver-internal `object` / `array` aliases (introspected external columns, + * not an authorable `type`), and a SINGLE-VALUE media field, a JSON column + * only on a deployment that has not moved its media columns (ADR-0104 + * addendum); this driver stores the bare id, the moved end-state. + * + * **A field with no recorded declaration answers `false`**, exactly as + * `SqlDriver.isJsonColumn` answers for a table it was never told about: an + * object never synced, or a field its schema does not name, keeps the + * substring reading it has always had. + */ + private isJsonStoredField(object: string | undefined, field: string): boolean { + const shape = object ? this.valueShapes.get(object)?.get(field) : undefined; + if (!shape) return false; + return STRUCTURED_JSON_TYPES.has(shape.type) || isMultiValueField(shape); + } + + /** + * [#20874] The one un-negated test `$contains` asks of `field` on `object` — + * behind every spelling of the operator on this package: the `$`-spelling and + * the AST spelling of the query path, and the analytics face, which wraps it + * in `$not` for `$notContains` exactly as the query path does and renders its + * SQL echo from the members it names. + * + * - **A declared JSON-stored field ({@link isJsonStoredField}) → MEMBERSHIP**: + * `{ $elemMatch: { $in: members, … } }` — some element of the stored array + * IS one of the members the comparand names ({@link containsMemberCandidates}). + * `$elemMatch` is array-only by construction, so a stored scalar, an object + * or `null` has no member — the answer `driver-sql` gives on every dialect + * (its constructs are array-only too). The `$not: { $type: 'array' }` clause + * keeps a NESTED array out: mingo applies `$in` through an element that is + * itself an array, so `[['u1']]` would otherwise answer `'u1'`, where SQLite + * compares the element's JSON text `["u1"]` and does not. Measured on mingo + * 7.2 against `driver-sql`/SQLite over one fixture: with the clause the two + * answer the same rows on every shape, `[null]` and `[[null]]` included. + * - **Anything else → `{ $regex }`, the case-exact literal SUBSTRING** (#6682), + * {@link filterSubstringPattern} unchanged. + * + * Negated, it composes with the NULL rule rather than replacing it: `$not` + * over either test admits a row whose field is null or missing (#5298), which + * is `driver-sql`'s `col IS NULL OR NOT (…)`. + * + * Public for the analytics face, for the reason {@link filterComparandStorageForm} + * and {@link filterSubstringPattern} are: a face that re-derived the population + * or the member reading would be a second place for the two faces to drift + * apart (#5374). Returned whole rather than as a pattern because the two + * readings do not share a shape — the analytics face used to take + * {@link filterSubstringPattern} and wrap it in a `$regex` of its own, which is + * how it inherited the per-element substring answer along with the query path. + * + * ⚠️ On a field this driver holds no declaration for, the substring pattern + * still reaches mingo, which applies a `$regex` to each element of an array + * value — the population's deliberate `false`, the same one `driver-sql` keeps + * for a table it was never told about. + */ + filterContainsTest(object: string | undefined, field: string, value: unknown): MemoryContainsTest { + if (this.isJsonStoredField(object, field)) { + return { $elemMatch: { $in: containsMemberCandidates(value), $not: { $type: 'array' } } }; + } + return { $regex: this.filterSubstringPattern(value) }; + } + /** * The form a record takes in the backing table: no own key holding * `undefined`, then every declared temporal field in its storage form. diff --git a/packages/drivers/driver-sql/src/sql-driver-17590-json-column-membership.test.ts b/packages/drivers/driver-sql/src/sql-driver-17590-json-column-membership.test.ts index 0ba181037ae..45a1a00c2ee 100644 --- a/packages/drivers/driver-sql/src/sql-driver-17590-json-column-membership.test.ts +++ b/packages/drivers/driver-sql/src/sql-driver-17590-json-column-membership.test.ts @@ -93,18 +93,23 @@ const FIELDS: Record> = { tags_: { type: 'tags' }, picks: { type: 'multiselect' }, nums: { type: 'select', multiple: true }, + // [#20874] A multi-valued LOOKUP — the shape the nested-relation filter + // lowers onto (`$contains` per related id) and the one `driver-memory` + // pins the same rows for (`memory-20874-contains-membership.test.ts`). + owners: { type: 'lookup', reference: 'os17590_owner', multiple: true }, }; /** * Rows where substring and membership DISAGREE on every column — see the head * note. `redwood`/`ab`/`[10, 21]` are the rows a substring emitter returns and - * a membership construct does not. + * a membership construct does not; `['u10']` is the row a per-element + * substring answers for `u1` (#20874). */ const ROWS = [ - { id: '1', label: 'redwood', tags_: ['red', 'blue'], picks: ['a', 'b'], nums: [1, 2] }, - { id: '2', label: 'red', tags_: ['redwood'], picks: ['ab'], nums: [10, 21] }, - { id: '3', label: 'blue', tags_: ['blue'], picks: ['b'], nums: [2] }, - { id: '4', label: 'none', tags_: [], picks: [], nums: [] }, + { id: '1', label: 'redwood', tags_: ['red', 'blue'], picks: ['a', 'b'], nums: [1, 2], owners: ['u1', 'u2'] }, + { id: '2', label: 'red', tags_: ['redwood'], picks: ['ab'], nums: [10, 21], owners: ['u10'] }, + { id: '3', label: 'blue', tags_: ['blue'], picks: ['b'], nums: [2], owners: ['u3', 'u1'] }, + { id: '4', label: 'none', tags_: [], picks: [], nums: [], owners: [] }, ] as const; const ALL_IDS = ['1', '2', '3', '4']; @@ -127,7 +132,13 @@ function declareMembershipCell(cell: DialectCell): void { knexInstance = driver.getKnex(); await knexInstance.schema.dropTableIfExists(OBJECT); await driver.initObjects([{ name: OBJECT, fields: FIELDS } as never]); - for (const row of ROWS) await driver.create(OBJECT, { ...row, tags_: [...row.tags_], picks: [...row.picks], nums: [...row.nums] }, BYPASS); + for (const row of ROWS) { + await driver.create( + OBJECT, + { ...row, tags_: [...row.tags_], picks: [...row.picks], nums: [...row.nums], owners: [...row.owners] }, + BYPASS, + ); + } }, LIVE_CELL_TIMEOUT_MS); afterAll(async () => { @@ -175,6 +186,19 @@ function declareMembershipCell(cell: DialectCell): void { expect(await ids({ nums: { $contains: '0' } })).toEqual([]); }, LIVE_CELL_TIMEOUT_MS); + /** + * [#20874] An id that is a PREFIX of another stored id is not its member. + * `u1` inside `['u10']` is the row a per-element substring answers and the + * membership construct does not — the case the nested-relation filter + * reaches (`$contains` per related id on a multi-valued relation). + * `driver-memory` pins these literal rows over the same fixture. + */ + it('$contains over a multi-valued lookup answers the MEMBER ids, never an id that prefixes another (u1 / u10)', async () => { + expect(await ids({ owners: { $contains: 'u1' } })).toEqual(['1', '3']); + expect(await ids({ owners: { $contains: 'u10' } })).toEqual(['2']); + expect(await ids({ owners: { $notContains: 'u1' } })).toEqual(['2', '4']); + }, LIVE_CELL_TIMEOUT_MS); + /** * The other half of the contract sentence, and the negative control: a * SCALAR string column keeps the substring test, so `red` still answers the diff --git a/packages/rest/src/data-nested-object-door.test.ts b/packages/rest/src/data-nested-object-door.test.ts index de467f2ba46..551f106faf6 100644 --- a/packages/rest/src/data-nested-object-door.test.ts +++ b/packages/rest/src/data-nested-object-door.test.ts @@ -23,10 +23,11 @@ * The InMemoryDriver cells were measured on this branch and are not pinned * here: this package does not depend on the in-memory driver, and that * driver's test consumers are a ruled, closed census - * (`check:driver-memory-census`). They answered every row above alike, except - * the multi-valued arm where one stored id contains another as a substring - * (`u1` / `u10`): the in-memory driver matches `$contains` per element by - * substring — the gap `FILTER_OPERATORS`' `$contains` docblock records for it. + * (`check:driver-memory-census`). They answered every row above alike. The + * multi-valued arm where one stored id contains another as a substring + * (`u1` / `u10`) differed until #20874 made the in-memory driver answer + * `$contains` on a multi-valued field by membership; that driver pins the + * case over the driver input below (`memory-20874-contains-membership.test.ts`). * `@objectstack/objectql`'s `engine-nested-relation-lowering.test.ts` pins the * driver input the engine sends, which is the same on every driver. * diff --git a/packages/spec/src/data/filter.zod.ts b/packages/spec/src/data/filter.zod.ts index db70c63be64..e26330ebcc6 100644 --- a/packages/spec/src/data/filter.zod.ts +++ b/packages/spec/src/data/filter.zod.ts @@ -1012,17 +1012,21 @@ export const RangeOperatorSchema = lazySchema(() => z.object({ * SQLite. Measured on better-sqlite3, live PostgreSQL 16.13 and live MySQL * 8.0.46 over one fixture whose rows make substring and membership disagree. * `driver-sqlite-wasm` and `driver-turso` inherit it. - * - **`driver-memory` — DOES NOT ANSWER IT YET, and reads `multiple: true` - * two ways of its own.** Measured on the same fixture: its live query path - * matches a stored array by substring PER ELEMENT (so `['redwood']` answers - * `$contains: 'red'`, the same over-match the SQL family just lost) and - * answers NOTHING at all for a `multiple: true` NUMBER, while its reference - * matcher answers no array at all. That whole axis — every non-equality arm - * over a stored array, in both directions — was measured on a tracking card - * that recorded the semantics as undecided; this ruling is the decision it - * was missing. ⚠️ So an application whose tests run on the in-memory double - * and whose production runs SQL still gets two answers from one filter here. - * That card is gone: measure `driver-memory` for the open set, ⛔ not this text. + * - **`driver-memory` — ANSWERS the membership contract, on every face.** Its + * live query path (both filter spellings) and its analytics face fork on the + * field's DECLARED storage shape, read from the schema `syncSchema` recorded + * (`STRUCTURED_JSON_TYPES`, or `isMultiValueField`): membership there, the + * substring test on a scalar column. Measured over `driver-sql`'s own + * fixture, row for row: `['redwood']` no longer answers `'red'`, `['u10']` no + * longer answers `'u1'`, and the stored number `1` answers `'1'`. The + * analytics face's SQL echo renders SQLite's `json_each` construct for the + * same question. A field with no recorded declaration keeps the substring + * reading, as a table `driver-sql` was never told about does. + * + * Only `$contains` / `$notContains` are ruled here. The other text operators + * over a stored array are not: measured on the same fixture, `$startsWith` + * still answers per element on `driver-memory` and over the serialized text on + * SQLite, so those two backends still disagree there. * * The comparand stays a STRING on every column ({@link CONTAINS_DESCRIPTION}), * so a member that is stored as a JSON number or boolean is named by its text: @@ -1039,7 +1043,6 @@ export const RangeOperatorSchema = lazySchema(() => z.object({ * @see https://github.com/objectstack-ai/objectstack/issues/6520 (the JS faces — landed) * @see https://github.com/objectstack-ai/objectstack/issues/17590 (the membership reading — the SQL family landed) * @see https://github.com/objectstack-ai/objectstack/issues/7398 (the refusal whose prescription this spelling is) - * @see https://github.com/objectstack-ai/objectstack/issues/17286 (driver-memory's stored-array axis — open) */ /** * The comparand contract the four CASE-SENSITIVE members of this family share