diff --git a/.changeset/20600-last-day-unbounded-above.md b/.changeset/20600-last-day-unbounded-above.md new file mode 100644 index 00000000000..632bba2a3f0 --- /dev/null +++ b/.changeset/20600-last-day-unbounded-above.md @@ -0,0 +1,36 @@ +--- +'@objectstack/spec': minor +'@objectstack/core': minor +'@objectstack/driver-sql': patch +'@objectstack/driver-turso': patch +'@objectstack/driver-memory': patch +'@objectstack/driver-mongodb': patch +'@objectstack/formula': patch +'@objectstack/objectql': patch +'@objectstack/service-analytics': patch +--- + +fix(spec,drivers): a `datetime` filter `$lte '9999-12-31'`, or a `$between` whose maximum is that day, includes the whole last supported day on every backend (#20600) + +Clause-②: yes (widening) — three new exports on `@objectstack/spec` (`data`) and `@objectstack/core`: the constant `UNBOUNDED_ABOVE`, its type `UnboundedAbove` and the guard `isUnboundedAbove`; `nextUtcCalendarDay` answers the constant for one input that used to answer a string. Nothing any door accepted before is refused, and nothing is removed or renamed. + +**BREAKING for TypeScript and JavaScript callers of `nextUtcCalendarDay`** (`@objectstack/spec/data`, re-exported by `@objectstack/core`): its return type gains a member and its answer for one input changes from a string to a symbol, landing in the launch window as `minor` (the lockstep convention: the bump level is not the carrier, this banner and the disposition below are). No filter an author writes and no stored row changes meaning except that a whole-day upper bound on `9999-12-31` now includes that day. + +`9999-12-31` is the last day of the supported years (0001..9999). A bare-day upper bound on a `datetime` field — `$lte`, a `$between` maximum, an analytics `dateRange` end — means that whole day, and is compiled as "before the next day's midnight". That day has no next day with a `YYYY-MM-DD` spelling: `nextUtcCalendarDay('9999-12-31')` answered the five-digit `'10000-01-01'`, which sorts below `'2026-…'` as text. So on SQLite, where a `datetime` column is ISO text, `$lte '9999-12-31'` and `$between ['2026-01-01', '9999-12-31']` answered no rows; PostgreSQL parsed the bound as an instant and answered them. The memory and mongo drivers, the analytics strategies and the draft preview built their bound from the same answer, and `formula`'s RLS `check` evaluator compared a `'2026-…'` value against it and denied the write. + +Every supported value is at most the last millisecond of `9999-12-31`, so that day's whole-day bound bounds nothing. `nextUtcCalendarDay('9999-12-31')` now answers `UNBOUNDED_ABOVE`, a symbol that is neither `null` ("not a calendar day", which would compile the day's midnight and miss the rest of it) nor a string, and every backend compiles no upper bound for it: + +- `$lte` / `<=` on that day asks only that the value is not null: `IS NOT NULL` on the SQL drivers and the analytics echo, `$ne: null` on the memory and mongo drivers. +- A `$between` / `between` whose maximum is that day, and an explicit analytics `dateRange` ending on it, keep only their minimum. +- The type-blind `formula` `check` evaluator and the draft preview admit every value that denotes an instant, and compare any other value as written. +- `$gte`, `$gt`, `$lt` and `$eq` on that day are unchanged: they anchor to its midnight, as on every other day. `9999-12-30` and every earlier day compile the same bound as before. + +Measured through `POST /api/v1/data/:object/query`, rows at `2026-07-15T14:00Z`, `9999-12-30T10:00Z`, `9999-12-31T00:00Z`, `T10:00Z` and `T23:59:59.999Z`: on SQLite, `$lte '9999-12-31'` and `$between ['2026-01-01', '9999-12-31']` answered none of them and now answer all five; `$between ['9999-12-31', '9999-12-31']` answered none and now answers the three on that day. PostgreSQL 16 answers the same before and after. `$lte '9999-12-30'` answers the first two rows on both, before and after. + +**If your code stops compiling.** `nextUtcCalendarDay` now returns `string | UnboundedAbove | null`, where `UnboundedAbove` is a `symbol` with a structural brand. TypeScript refuses that member in a template literal (TS2731), a relational comparison (TS2469) and a `string` parameter (TS2345), so code that used the answer as a day string no longer compiles until it handles the last day. Test the answer with `isUnboundedAbove(answer)` (or `typeof answer === 'symbol'`) first: on its false branch the answer is `string | null` as before, and on its true branch there is no upper bound to compile. `answer === UNBOUNDED_ABOVE` compares correctly but does not narrow, because the branded type is not a unit type. The type is structural on purpose: `@objectstack/spec` ships `./data` as `index.d.mts` and `index.d.ts`, and a `unique symbol` would be two unrelated types in a program that meets both. + +**If your JavaScript code handled the answer as text.** For `'9999-12-31'` it is now a registered symbol (`Symbol.for('objectstack.calendarDay.unboundedAbove')`), not `'10000-01-01'`: a template literal or a relational comparison on it throws a `TypeError`, and better-sqlite3 and `pg` refuse to bind it. Every other input answers exactly as before. + +The shared temporal conformance kit (`TEMPORAL_ROWS` / `TEMPORAL_CASES` in `@objectstack/spec/data`) gains the row `z_last` (`9999-12-31T10:00:00.000Z`) and five last-day cases, so every backend it drives is held to this answer; three existing `$gte` / `$gt` cases now also expect `z_last`. + + diff --git a/packages/core/src/utils/analytics-date-range.test.ts b/packages/core/src/utils/analytics-date-range.test.ts index 62aa519a77d..528976728f2 100644 --- a/packages/core/src/utils/analytics-date-range.test.ts +++ b/packages/core/src/utils/analytics-date-range.test.ts @@ -316,8 +316,10 @@ describe('#17341 — the END of every calendar window is one CALENDAR day after const where = `${preset} @ ${tz ?? 'UTC'} @ ${now.toISOString()}`; const lastDay = String(resolveFilterToken(prescribedEnd, { now, timezone: tz })); const stopsBefore = nextUtcCalendarDay(lastDay); - expect(stopsBefore, where).not.toBeNull(); - const expected = new Date(zonedDateStartToUtcMs(stopsBefore!, tz)).toISOString(); + // A day: never `null`, and never `UNBOUNDED_ABOVE` — the + // sweep's instants sit nowhere near 9999-12-31 (#20600). + expect(typeof stopsBefore, where).toBe('string'); + const expected = new Date(zonedDateStartToUtcMs(stopsBefore as string, tz)).toISOString(); expect(resolveAnalyticsDateRangePreset(preset, { now, timezone: tz }).end, where) .toBe(expected); } diff --git a/packages/core/src/utils/datetime.ts b/packages/core/src/utils/datetime.ts index f041cc4befa..dc023b37876 100644 --- a/packages/core/src/utils/datetime.ts +++ b/packages/core/src/utils/datetime.ts @@ -192,8 +192,12 @@ export function zonedWallClockToUtcMs(parts: WallClockParts, tz?: string): numbe * * Re-exported here so the published `@objectstack/core` surface is unchanged * for the drivers and analytics strategies that already import it from here. + * [#20600] `UNBOUNDED_ABOVE`, its type `UnboundedAbove` and the guard + * `isUnboundedAbove` travel with the helper: the constant is the helper's answer + * for `9999-12-31`, and every caller of one narrows it with the guard. */ -export { nextUtcCalendarDay, utcInstantMs } from '@objectstack/spec/data'; +export { nextUtcCalendarDay, utcInstantMs, UNBOUNDED_ABOVE, isUnboundedAbove } from '@objectstack/spec/data'; +export type { UnboundedAbove } from '@objectstack/spec/data'; /** * Granularity of a canonical date-bucket key. Mirrors `@objectstack/spec`'s diff --git a/packages/drivers/driver-memory/src/memory-analytics.ts b/packages/drivers/driver-memory/src/memory-analytics.ts index eff07c732b2..f0f5ffb6613 100644 --- a/packages/drivers/driver-memory/src/memory-analytics.ts +++ b/packages/drivers/driver-memory/src/memory-analytics.ts @@ -11,6 +11,7 @@ import { Logger, createLogger, nextUtcCalendarDay, + isUnboundedAbove, // [#16322] The ONE lowering of the closed `dateRange` preset vocabulary and // the ONE refusal for a string outside it, shared with the SQL analytics // path so the two backends cannot answer one input differently again. @@ -266,8 +267,11 @@ const CUBE_OPERATOR_TO_MONGO_PREDICATE: Readonly { const nextDay = nextUtcCalendarDay(comparands[0]); + if (isUnboundedAbove(nextDay)) return { $ne: null }; return nextDay != null ? { $lt: nextDay } : { $lte: comparands[0] }; }, // The list operators take the WHOLE list. An empty one is a real predicate — @@ -458,8 +462,11 @@ const CUBE_OPERATOR_TO_SQL_PREDICATE: Readonly { const nextDay = nextUtcCalendarDay(comparands[0]); + if (isUnboundedAbove(nextDay)) return `${column} IS NOT NULL`; return nextDay != null ? `${column} < ${literal(nextDay)}` : `${column} <= ${literal(comparands[0])}`; @@ -991,17 +998,27 @@ export class MemoryAnalyticsService implements IAnalyticsService { // // Anything else -- a full timestamp the CALLER wrote -- keeps // instant semantics and stays INCLUSIVE, byte for byte as before. - const widenedDay = resolved.endExclusive ? null : nextUtcCalendarDay(end); + // + // [#20600] A caller's bare end on the last supported day has no next + // day to stop before: every value is inside it, so the window keeps + // its start alone, in both spellings. + const widened = resolved.endExclusive ? null : nextUtcCalendarDay(end); + const unbounded = isUnboundedAbove(widened); + const widenedDay = isUnboundedAbove(widened) ? null : widened; const upperString = resolved.endExclusive ? end : widenedDay; const upperDate = widenedDay != null ? new Date(`${widenedDay}T00:00:00.000Z`) : (resolved.endExclusive ? new Date(end) : null); - const stringBounds = upperString != null - ? { $gte: start, $lt: upperString } - : { $gte: start, $lte: end }; - const dateBounds = upperDate != null - ? { $gte: new Date(start), $lt: upperDate } - : { $gte: new Date(start), $lte: new Date(end) }; + const stringBounds = unbounded + ? { $gte: start } + : upperString != null + ? { $gte: start, $lt: upperString } + : { $gte: start, $lte: end }; + const dateBounds = unbounded + ? { $gte: new Date(start) } + : upperDate != null + ? { $gte: new Date(start), $lt: upperDate } + : { $gte: new Date(start), $lte: new Date(end) }; pipeline.push({ $match: { $or: [ diff --git a/packages/drivers/driver-memory/src/memory-driver-calendar-day-upper-bound.test.ts b/packages/drivers/driver-memory/src/memory-driver-calendar-day-upper-bound.test.ts index 9bf2cf935e5..12a5a148baa 100644 --- a/packages/drivers/driver-memory/src/memory-driver-calendar-day-upper-bound.test.ts +++ b/packages/drivers/driver-memory/src/memory-driver-calendar-day-upper-bound.test.ts @@ -17,7 +17,7 @@ */ import { describe, it, expect, beforeEach } from 'vitest'; -import { parseFilterAST } from '@objectstack/spec/data'; +import { parseFilterAST, type FilterCondition } from '@objectstack/spec/data'; import { InMemoryDriver } from './memory-driver.js'; import { MemoryAnalyticsService } from './memory-analytics.js'; import type { Cube } from '@objectstack/spec/data'; @@ -153,3 +153,103 @@ describe('MemoryAnalyticsService — dateRange window (#4042)', () => { expect(result.rows[0]?.count).toBe(4); }); }); + +/** + * [#20600] `9999-12-31`, the last supported day, has no next day: every + * supported value is inside its whole-day bound, so the whole-day widening + * compiles NO upper bound — `$lte` / `<=` ask only for a value (`$ne: null`), a + * `$between` / `between` / `dateRange` keeps its minimum. The spec's helper + * used to answer the five-digit `'10000-01-01'`, and every ISO-string row + * sorts above it, so each of those answered no rows. `9999-12-30` is the + * control: an ordinary bound. + */ +describe('[#20600] InMemoryDriver — a bare-day upper bound on the last supported day', () => { + let driver: InMemoryDriver; + + beforeEach(async () => { + driver = new InMemoryDriver({ + initialData: { + task: [ + { id: 'c26', title: 'c26', created_at: '2026-07-15T14:00:00.000Z' }, + { id: 'prev', title: 'prev', created_at: '9999-12-30T10:00:00.000Z' }, + { id: 'open', title: 'open', created_at: '9999-12-31T00:00:00.000Z' }, + { id: 'mid', title: 'mid', created_at: '9999-12-31T10:00:00.000Z' }, + { id: 'last', title: 'last', created_at: '9999-12-31T23:59:59.999Z' }, + { id: 'none', title: 'none', created_at: null }, + ], + }, + }); + await driver.connect(); + }); + + /** `where` · the ids it answers, sorted. */ + const CASES: ReadonlyArray FilterCondition | undefined, readonly string[]]> = [ + ["$lte '9999-12-31'", () => ({ created_at: { $lte: '9999-12-31' } }), ['c26', 'last', 'mid', 'open', 'prev']], + ["$between ['2026-01-01', '9999-12-31']", () => ({ created_at: { $between: ['2026-01-01', '9999-12-31'] } }), ['c26', 'last', 'mid', 'open', 'prev']], + ["$between ['9999-12-31', '9999-12-31']", () => ({ created_at: { $between: ['9999-12-31', '9999-12-31'] } }), ['last', 'mid', 'open']], + ["$not $lte '9999-12-31'", () => ({ $not: { created_at: { $lte: '9999-12-31' } } }), ['none']], + // The lowered `$ne` shares a key an author can write: both constraints survive (#13524). + ["$lte '9999-12-31' beside an author's $ne", () => ({ created_at: { $lte: '9999-12-31', $ne: '9999-12-31T10:00:00.000Z' } }), ['c26', 'last', 'open', 'prev']], + ["the AST spelling <= '9999-12-31'", () => parseFilterAST([['created_at', '<=', '9999-12-31']]), ['c26', 'last', 'mid', 'open', 'prev']], + ["the AST spelling between max '9999-12-31'", () => parseFilterAST([['created_at', 'between', ['9999-12-31', '9999-12-31']]]), ['last', 'mid', 'open']], + ["$lte '9999-12-30' (control)", () => ({ created_at: { $lte: '9999-12-30' } }), ['c26', 'prev']], + ["$between ['2026-01-01', '9999-12-30'] (control)", () => ({ created_at: { $between: ['2026-01-01', '9999-12-30'] } }), ['c26', 'prev']], + ["$gte '9999-12-31' (unchanged)", () => ({ created_at: { $gte: '9999-12-31' } }), ['last', 'mid', 'open']], + ["$lt '9999-12-31' (unchanged)", () => ({ created_at: { $lt: '9999-12-31' } }), ['c26', 'prev']], + ]; + + it('compiles no upper bound on 9999-12-31; 9999-12-30 is a bound; the lower-bound operators do not move', async () => { + const got: Record = {}; + for (const [name, where] of CASES) got[name] = ids(await driver.find('task', { where: where() })); + expect(got).toEqual(Object.fromEntries(CASES.map(([name, , want]) => [name, want]))); + }); + + it('the analytics face: a dateRange ending 9999-12-31 keeps its start alone, for BOTH stored forms', async () => { + const analytics = new InMemoryDriver({ + initialData: { + task: [ + { id: 's_before', created_at: '2026-06-30T23:00:00.000Z' }, + { id: 's_in', created_at: '2026-07-10T08:00:00.000Z' }, + { id: 's_last_day', created_at: '9999-12-31T21:40:00.000Z' }, + { id: 'd_in', created_at: new Date('2026-07-10T12:00:00Z') }, + { id: 'd_last_day', created_at: new Date('9999-12-31T09:15:00Z') }, + { id: 'd_before', created_at: new Date('2026-04-19T10:00:00Z') }, + ], + }, + }); + await analytics.connect(); + const service = new MemoryAnalyticsService({ + driver: analytics, + cubes: [{ + name: 'tasks', title: 'Tasks', sql: 'task', + measures: { count: { label: 'Count', type: 'count', sql: 'id' } }, + dimensions: { created_at: { label: 'Created', type: 'time', sql: 'created_at' } }, + } as unknown as Cube], + }); + const count = async (dateRange: [string, string]) => + (await service.query({ cube: 'tasks', measures: ['count'], timeDimensions: [{ dimension: 'created_at', dateRange }] } as any)).rows[0]?.count; + + expect(await count(['2026-07-01', '9999-12-31'])).toBe(4); // s_in, s_last_day, d_in, d_last_day + expect(await count(['2026-07-01', '9999-12-30'])).toBe(2); // the control: the last day is out + }); + + it('the analytics face: a cube-filter lte on 9999-12-31 asks only for a value, in the pipeline and the echo', async () => { + const service = new MemoryAnalyticsService({ + driver, + cubes: [{ + name: 'tasks', title: 'Tasks', sql: 'task', + measures: { count: { label: 'Count', type: 'count', sql: 'id' } }, + dimensions: { + id: { label: 'Id', type: 'string', sql: 'id' }, + created_at: { label: 'Created', type: 'time', sql: 'created_at' }, + }, + } as unknown as Cube], + }); + const q = (day: string) => ({ cube: 'tasks', measures: ['count'], dimensions: ['id'], where: { created_at: { $lte: day } } } as any); + + expect(ids((await service.query(q('9999-12-31'))).rows)).toEqual(['c26', 'last', 'mid', 'open', 'prev']); + expect(ids((await service.query(q('9999-12-30'))).rows)).toEqual(['c26', 'prev']); + expect((await service.generateSql(q('9999-12-31'))).sql).toContain('created_at IS NOT NULL'); + expect((await service.generateSql(q('9999-12-30'))).sql).toContain("created_at < '9999-12-31'"); + }); +}); diff --git a/packages/drivers/driver-memory/src/memory-driver.ts b/packages/drivers/driver-memory/src/memory-driver.ts index d8cf37b9c5a..d1e188562d5 100644 --- a/packages/drivers/driver-memory/src/memory-driver.ts +++ b/packages/drivers/driver-memory/src/memory-driver.ts @@ -14,7 +14,7 @@ import { hasDanglingLikeEscape, hasNulInLikePattern, likePatternToRegExp } from // the ruled 「is empty」 table, asked of the spec by the live query path. import { expandEmptyOperator, type ValueShapeFieldDef } from '@objectstack/spec/data'; import type { DriverQuery, IDataDriver } from '@objectstack/spec/contracts'; -import { Logger, createLogger, nextUtcCalendarDay } from '@objectstack/core'; +import { Logger, createLogger, nextUtcCalendarDay, isUnboundedAbove } from '@objectstack/core'; import { Query, Aggregator } from 'mingo'; import { assertSingleTenantPosture, @@ -112,7 +112,7 @@ interface LoweredWrite { * | lowered key | written by | * |---|---| * | `$eq` | `$eq`, `$null: true`, `$exists: false` | - * | `$ne` | `$ne`, `$null: false`, `$exists: true` | + * | `$ne` | `$ne`, `$null: false`, `$exists: true`, `$lte` (the LAST supported day, `9999-12-31` — #20600: no bound, a value) | * | `$gte` | `$gte`, `$between` | * | `$lte` | `$lte`, `$between` | * | `$lt` | `$lt`, `$lte` (BARE CALENDAR DAY — #4042's half-open rewrite), `$between` (bare-day max) | @@ -1397,7 +1397,11 @@ export class InMemoryDriver implements IDataDriver { // compiles half-open (`< 2026-07-29`), which is also order-equivalent // to `<=` for plain `YYYY-MM-DD` date values — so no field-type lookup // is needed, exactly the argument the preview evaluator uses. + // [#20600] On the last supported day there is no next day: every value + // is inside the bound, so what `<=` still asks is a value (`$ne: null`, + // the `is_not_null` arm below). const nextDay = nextUtcCalendarDay(value); + if (isUnboundedAbove(nextDay)) return { [field]: { $ne: null } }; return { [field]: nextDay != null ? { $lt: store(nextDay) } : { $lte: store(value) } }; } case 'in': @@ -1469,7 +1473,10 @@ export class InMemoryDriver implements IDataDriver { case 'between': if (Array.isArray(value) && value.length === 2) { // Bare-day max → half-open, inheriting `<=`'s whole-day rule (#4042). + // [#20600] A max on the last supported day bounds nothing: the range + // keeps its minimum alone. const nextDay = nextUtcCalendarDay(value[1]); + if (isUnboundedAbove(nextDay)) return { [field]: { $gte: store(value[0]) } }; return { [field]: nextDay != null ? { $gte: store(value[0]), $lt: store(nextDay) } @@ -1716,7 +1723,10 @@ export class InMemoryDriver implements IDataDriver { if (!Array.isArray(val) || val.length !== 2) throw malformedBetweenError(field, val, `${path}.$between`); put('$gte', store(val[0])); // Bare-day max → half-open, inheriting `$lte`'s whole-day rule (#4042). + // [#20600] A max on the last supported day bounds nothing: the + // range keeps its minimum alone. const betweenNextDay = nextUtcCalendarDay(val[1]); + if (isUnboundedAbove(betweenNextDay)) break; if (betweenNextDay != null) put('$lt', store(betweenNextDay)); else put('$lte', store(val[1])); break; @@ -1728,8 +1738,13 @@ export class InMemoryDriver implements IDataDriver { // [#13524] `$lt` here is a key an AUTHOR can also write — this arm is // the member of the clobber class no card had named. See // {@link assembleLoweredWrites}. + // [#20600] On the last supported day there is no next day: every + // value is inside the bound, so what `$lte` still asks is a value — + // `$ne: null`, the lowering `$null: false` takes below. Collected like + // every other write, so an author's own `$ne` survives beside it. const nextDay = nextUtcCalendarDay(val); - if (nextDay != null) put('$lt', store(nextDay)); + if (isUnboundedAbove(nextDay)) put('$ne', null); + else if (nextDay != null) put('$lt', store(nextDay)); else put('$lte', store(val)); break; } diff --git a/packages/drivers/driver-mongodb/src/mongodb-filter.test.ts b/packages/drivers/driver-mongodb/src/mongodb-filter.test.ts index 1031555aced..73f2069ccee 100644 --- a/packages/drivers/driver-mongodb/src/mongodb-filter.test.ts +++ b/packages/drivers/driver-mongodb/src/mongodb-filter.test.ts @@ -83,6 +83,37 @@ describe('MongoDB Filter Translator', () => { created_at: { $lte: '2026-07-28T12:00:00.000Z' }, }); }); + + // [#20600] 9999-12-31, the last supported day, has no next day: every value + // is inside its whole-day bound, so no upper bound is compiled. The helper + // answered the five-digit '10000-01-01', a bound below every stored value. + it('a bare-day $lte on the last supported day asks only for a value', () => { + expect(translateFilter({ created_at: { $lte: '9999-12-31' } })).toEqual({ + created_at: { $ne: null }, + }); + // The control: the day before is an ordinary half-open bound. + expect(translateFilter({ created_at: { $lte: '9999-12-30' } })).toEqual({ + created_at: { $lt: '9999-12-31' }, + }); + }); + + it('a $between whose max is the last supported day keeps its min alone', () => { + expect(translateFilter({ created_at: { $between: ['2026-04-29', '9999-12-31'] } })).toEqual({ + created_at: { $gte: '2026-04-29' }, + }); + expect(translateFilter({ created_at: { $between: ['2026-04-29', '9999-12-30'] } })).toEqual({ + created_at: { $gte: '2026-04-29', $lt: '9999-12-31' }, + }); + }); + + it("the lowered $ne beside an author's own $ne keeps both (#13524)", () => { + expect(translateFilter({ created_at: { $lte: '9999-12-31', $ne: '9999-12-31T10:00:00.000Z' } })).toEqual({ + $and: [ + { created_at: { $ne: '9999-12-31T10:00:00.000Z' } }, + { created_at: { $ne: null } }, + ], + }); + }); }); /** diff --git a/packages/drivers/driver-mongodb/src/mongodb-filter.ts b/packages/drivers/driver-mongodb/src/mongodb-filter.ts index 30452b5bd00..a5defbce141 100644 --- a/packages/drivers/driver-mongodb/src/mongodb-filter.ts +++ b/packages/drivers/driver-mongodb/src/mongodb-filter.ts @@ -24,7 +24,7 @@ */ import type { Filter } from 'mongodb'; -import { nextUtcCalendarDay } from '@objectstack/core'; +import { nextUtcCalendarDay, isUnboundedAbove } from '@objectstack/core'; import { StandardErrorCode } from '@objectstack/spec/api'; // [#5659] The Filter Protocol's boolean identity reduction, shared with // driver-sql, driver-memory and the flow linter and proven against the same @@ -1000,7 +1000,7 @@ interface LoweredWrite { * | lowered key | written by | * |---|---| * | `$eq` | `$eq`, `$null: true`, `$exists: false` | - * | `$ne` | `$ne`, `$null: false`, `$exists: true` | + * | `$ne` | `$ne`, `$null: false`, `$exists: true`, `$lte` (the LAST supported day, `9999-12-31` — #20600: no bound, a value) | * | `$gte` | `$gte`, `$between` | * | `$lte` | `$lte`, `$between` | * | `$lt` | `$lt`, `$lte` (BARE CALENDAR DAY — #4042's half-open rewrite), `$between` (bare-day max) | @@ -1158,8 +1158,13 @@ function translateFieldOperators( // [#13524] `$lt` here is a key an AUTHOR can also write — this arm is a // member of the clobber class that no card had named. See // {@link assembleLoweredWrites}. + // [#20600] On the last supported day there is no next day: every + // value is inside the bound, so what `$lte` still asks is a value — + // `$ne: null`, the lowering `$exists: true` takes above. Collected like + // every other write, so an author's own `$ne` survives beside it. const nextDay = nextUtcCalendarDay(value); - if (nextDay != null) put('$lt', store(nextDay)); + if (isUnboundedAbove(nextDay)) put('$ne', null); + else if (nextDay != null) put('$lt', store(nextDay)); else put('$lte', store(value)); break; } @@ -1249,7 +1254,10 @@ function translateFieldOperators( case '$between': { if (!isBetweenRange(value)) throw malformedBetweenError(field, value, `${path}.$between`); put('$gte', store(value[0])); + // [#20600] A max on the last supported day bounds nothing: the range + // keeps its minimum alone. const betweenNextDay = nextUtcCalendarDay(value[1]); + if (isUnboundedAbove(betweenNextDay)) break; if (betweenNextDay != null) put('$lt', store(betweenNextDay)); else put('$lte', store(value[1])); break; diff --git a/packages/drivers/driver-sql/src/sql-driver-calendar-day-upper-bound.test.ts b/packages/drivers/driver-sql/src/sql-driver-calendar-day-upper-bound.test.ts index 82c26734193..c95856fb807 100644 --- a/packages/drivers/driver-sql/src/sql-driver-calendar-day-upper-bound.test.ts +++ b/packages/drivers/driver-sql/src/sql-driver-calendar-day-upper-bound.test.ts @@ -23,7 +23,7 @@ */ import { describe, it, expect, beforeEach, afterEach } from 'vitest'; -import { parseFilterAST } from '@objectstack/spec/data'; +import { parseFilterAST, UNBOUNDED_ABOVE, type FilterCondition } from '@objectstack/spec/data'; import { SqlDriver } from '../src/index.js'; import { LegacyStorageDriver } from '../src/legacy-datetime-storage.testkit.js'; @@ -280,3 +280,95 @@ describe('calendarDayUpperBoundRewrite — dialect and boundary matrix', () => { expect(d.betweenRewrite('t', 'at', ['2026-04-29'])).toBeNull(); // malformed → caller's error }); }); + +/** + * [#20600] `9999-12-31`, the last supported day, has no next day: every + * supported value is inside its whole-day bound, so `$lte` and a `$between` + * max on that day compile NO upper bound. The spec's helper used to answer the + * five-digit `'10000-01-01'`, which this driver bound as `'+010000-…'` / the + * five-digit text; a SQLite column (ISO text) sorts every row above it, so + * both answered no rows. `9999-12-30` is the control: an ordinary bound. + */ +describe('[#20600] a bare-day upper bound on the last supported day', () => { + it('the rewrite answers UNBOUNDED_ABOVE on every dialect for $lte and <=, and the between max carries it', () => { + const lowerByClient: Record = { + 'better-sqlite3': '2026-01-01T00:00:00.000Z', + pg: '2026-01-01T00:00:00.000Z', + mysql2: '2026-01-01 00:00:00.000', + }; + for (const [client, lower] of Object.entries(lowerByClient)) { + const d = makeProbe(client); + d.seedDatetime('t', 'at'); + d.seedDate('t', 'on'); + expect(d.rewrite('t', 'at', '$lte', '9999-12-31'), client).toBe(UNBOUNDED_ABOVE); + expect(d.rewrite('t', 'at', '<=', '9999-12-31'), client).toBe(UNBOUNDED_ABOVE); + expect(d.betweenRewrite('t', 'at', ['2026-01-01', '9999-12-31']), client).toEqual({ lower, upper: UNBOUNDED_ABOVE }); + // The control: the day before is an ordinary bound. + expect((d.rewrite('t', 'at', '$lte', '9999-12-30') as any)?.op, client).toBe('$lt'); + // Outside the calendar-day-on-datetime cell nothing changes. + expect(d.rewrite('t', 'on', '$lte', '9999-12-31'), client).toBeNull(); + expect(d.rewrite('t', 'at', '$gte', '9999-12-31'), client).toBeNull(); + expect(d.rewrite('t', 'at', '$lte', '9999-12-31T10:00:00.000Z'), client).toBeNull(); + } + }); + + describe('row results — canonical text and the un-backfilled legacy column', () => { + const LAST_DAY_ROWS = [ + ['c26', '2026-07-15T14:00:00.000Z'], + ['prev', '9999-12-30T10:00:00.000Z'], + ['open', '9999-12-31T00:00:00.000Z'], + ['mid', '9999-12-31T10:00:00.000Z'], + ['last', '9999-12-31T23:59:59.999Z'], + ] as const; + + /** `where` on `at` · the ids it answers, sorted — `none` has no value. */ + const CASES: ReadonlyArray = [ + ["$lte '9999-12-31'", { at: { $lte: '9999-12-31' } }, ['c26', 'last', 'mid', 'open', 'prev']], + ["$between ['2026-01-01', '9999-12-31']", { at: { $between: ['2026-01-01', '9999-12-31'] } }, ['c26', 'last', 'mid', 'open', 'prev']], + ["$between ['9999-12-31', '9999-12-31']", { at: { $between: ['9999-12-31', '9999-12-31'] } }, ['last', 'mid', 'open']], + ["$not $lte '9999-12-31'", { $not: { at: { $lte: '9999-12-31' } } }, ['none']], + ["$or [$lte '9999-12-31', title 'none']", { $or: [{ at: { $lte: '9999-12-31' } }, { title: 'none' }] }, ['c26', 'last', 'mid', 'none', 'open', 'prev']], + ["$lte '9999-12-30' (control)", { at: { $lte: '9999-12-30' } }, ['c26', 'prev']], + ["$between ['2026-01-01', '9999-12-30'] (control)", { at: { $between: ['2026-01-01', '9999-12-30'] } }, ['c26', 'prev']], + ["$gt '9999-12-31' (unchanged)", { at: { $gt: '9999-12-31' } }, ['last', 'mid']], + ["$lt '9999-12-31' (unchanged)", { at: { $lt: '9999-12-31' } }, ['c26', 'prev']], + ]; + + const run = async (driver: SqlDriver) => { + const got: Record = {}; + for (const [name, where] of CASES) got[name] = ids(await driver.find('task', { where })); + expect(got).toEqual(Object.fromEntries(CASES.map(([name, , want]) => [name, want]))); + }; + + it('canonical text: $lte and the $between max on 9999-12-31 compile no upper bound', async () => { + const driver = new SqlDriver({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true }); + try { + await driver.initObjects([{ name: 'task', fields: { title: { type: 'string' }, at: { type: 'datetime' } } }]); + for (const [id, at] of LAST_DAY_ROWS) await driver.create('task', { id, title: id, at }, { bypassTenantAudit: true }); + await driver.create('task', { id: 'none', title: 'none' }, { bypassTenantAudit: true }); + await run(driver); + } finally { + await driver.disconnect?.(); + } + }); + + it('legacy mixed storage: the same answers through the normalised column expression', async () => { + const driver = new LegacyStorageDriver({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true }); + try { + await driver.initObjects([{ name: 'task', fields: { title: { type: 'string' }, at: { type: 'datetime' } } }]); + // Both pre-#3912 storage forms: an INTEGER epoch and zone-naive TEXT. + await driver.seedLegacyRows('task', 'at', [ + ...LAST_DAY_ROWS.map(([id, at], i) => ({ + id, + title: id, + at: i % 2 === 0 ? Date.parse(at) : at.replace('T', ' ').replace('Z', ''), + })), + { id: 'none', title: 'none', at: null }, + ]); + await run(driver); + } finally { + await driver.disconnect?.(); + } + }); + }); +}); diff --git a/packages/drivers/driver-sql/src/sql-driver.ts b/packages/drivers/driver-sql/src/sql-driver.ts index 9da3c48e310..2462eccd9c0 100644 --- a/packages/drivers/driver-sql/src/sql-driver.ts +++ b/packages/drivers/driver-sql/src/sql-driver.ts @@ -90,7 +90,13 @@ import { declareTargetedTable, } from '@objectstack/types'; import { postureEnforcesWall } from '@objectstack/spec/security'; -import { nextUtcCalendarDay, temporalStorageForm } from '@objectstack/core'; +import { + nextUtcCalendarDay, + temporalStorageForm, + UNBOUNDED_ABOVE, + isUnboundedAbove, + type UnboundedAbove, +} from '@objectstack/core'; import { applyIndexKeyParts, buildIndexName, @@ -15395,15 +15401,23 @@ export class SqlDriver implements IDataDriver { * keeps instant semantics — only the day-granular STRING carries * calendar-day intent); * - `$gte` / `$gt` / `$lt` keep their midnight anchoring (correct today). + * + * [#20600] `9999-12-31`, the last supported day, has no next day to stop + * before: every supported value is at most its last millisecond, so the + * spec's helper answers `UNBOUNDED_ABOVE` and this passes it on for the two + * emitters below to compile NO upper bound. It used to be the five-digit + * `'10000-01-01…'`, which a SQLite column (ISO text) sorts above every row, + * so `$lte '9999-12-31'` answered no rows there. */ protected calendarDayExclusiveUpperBound( table: string | null, field: string, value: unknown, - ): unknown | null { + ): unknown | UnboundedAbove | null { if (this.temporalFieldKind(table, field) !== 'datetime') return null; const next = nextUtcCalendarDay(value); if (next == null) return null; + if (isUnboundedAbove(next)) return UNBOUNDED_ABOVE; return this.storageDatetimeValue(`${next}T00:00:00.000Z`); } @@ -15412,16 +15426,21 @@ export class SqlDriver implements IDataDriver { * with a bare `YYYY-MM-DD` on a `datetime` column becomes `$lt`/`<` against * {@link calendarDayExclusiveUpperBound}. Returns `null` — "not applicable, * compile as-is" — for every other operator/comparand/column combination. + * + * [#20600] `UNBOUNDED_ABOVE` for the last supported day: the comparison has + * no bound to compile, and what it still asks is that the column has a value + * (a comparison never holds for NULL) — the caller compiles `IS NOT NULL`. */ protected calendarDayUpperBoundRewrite( table: string | null, field: string, op: string, value: unknown, - ): { op: string; value: unknown } | null { + ): { op: string; value: unknown } | UnboundedAbove | null { if (op !== '$lte' && op !== '<=') return null; const upper = this.calendarDayExclusiveUpperBound(table, field, value); if (upper == null) return null; + if (isUnboundedAbove(upper)) return UNBOUNDED_ABOVE; return { op: op === '$lte' ? '$lt' : '<', value: upper }; } @@ -15432,13 +15451,14 @@ export class SqlDriver implements IDataDriver { * `whereBetween` is inclusive on both ends, so it inherits the same * midnight-anchored upper bound `$lte` had. Returns `null` when the range is * malformed (caller keeps its descriptive error) or the rewrite does not - * apply. + * apply. [#20600] `upper` is `UNBOUNDED_ABOVE` when the max is the last + * supported day: the range keeps only its minimum. */ protected calendarDayBetweenRewrite( table: string | null, field: string, value: unknown, - ): { lower: unknown; upper: unknown } | null { + ): { lower: unknown; upper: unknown | UnboundedAbove } | null { if (!Array.isArray(value) || value.length !== 2) return null; const upper = this.calendarDayExclusiveUpperBound(table, field, value[1]); if (upper == null) return null; @@ -16951,18 +16971,31 @@ export class SqlDriver implements IDataDriver { if (rawOp === '$between') { const dayRange = this.calendarDayBetweenRewrite(table, localField, opValue); if (dayRange) { + // [#20600] A max on the last supported day bounds nothing: the + // range keeps its minimum alone. + const bounded = !isUnboundedAbove(dayRange.upper); (builder as any)[method]((qb: any) => { if (columnExpr) { this.applyNormalizedComparison(qb, 'and', columnExpr, '$gte', dayRange.lower); - this.applyNormalizedComparison(qb, 'and', columnExpr, '$lt', dayRange.upper); + if (bounded) this.applyNormalizedComparison(qb, 'and', columnExpr, '$lt', dayRange.upper); } else { - qb.where(field, '>=', dayRange.lower).andWhere(field, '<', dayRange.upper); + qb.where(field, '>=', dayRange.lower); + if (bounded) qb.andWhere(field, '<', dayRange.upper); } }); continue; } } const rewrite = this.calendarDayUpperBoundRewrite(table, localField, rawOp, opValue); + if (isUnboundedAbove(rewrite)) { + // [#20600] `$lte` on the last supported day: no upper bound, so the + // comparison asks only that the column has a value — the `IS NOT + // NULL` the `$ne: null` arm below spells. The raw column, not + // `columnExpr`: a legacy-repair expression is NULL exactly when + // the column is. + (builder as any)[logicalOp === 'or' ? 'orWhereNotNull' : 'whereNotNull'](field); + continue; + } const op = rewrite?.op ?? rawOp; const coerced = rewrite ? rewrite.value : this.coerceFilterValue(table, localField, opValue); if (columnExpr && this.applyNormalizedComparison(builder, logicalOp, columnExpr, op, coerced)) continue; diff --git a/packages/drivers/driver-turso/src/turso-driver.ts b/packages/drivers/driver-turso/src/turso-driver.ts index 361ab116ec1..a2d78a5e78f 100644 --- a/packages/drivers/driver-turso/src/turso-driver.ts +++ b/packages/drivers/driver-turso/src/turso-driver.ts @@ -37,6 +37,7 @@ import { import { StandardErrorCode } from '@objectstack/spec/api'; import type { DriverQuery } from '@objectstack/spec/contracts'; import type { DriverOptions, FilterCondition } from '@objectstack/spec/data'; +import { UNBOUNDED_ABOVE, isUnboundedAbove, type UnboundedAbove } from '@objectstack/spec/data'; import type { Client } from '@libsql/client'; import { RemoteTransport } from './remote-transport.js'; import { @@ -2576,12 +2577,21 @@ export class TursoDriver extends SqlDriver { break; } out.$gte = this.temporalFilterValue(object, field, raw[0]); - Object.assign(out, this.toRemoteUpperBound(object, field, '$lte', raw[1])); + // [#20600] A max on the last supported day bounds nothing: the range + // keeps its minimum alone. + const upper = this.toRemoteUpperBound(object, field, '$lte', raw[1]); + if (!isUnboundedAbove(upper)) Object.assign(out, upper); break; } - case '$lte': - Object.assign(out, this.toRemoteUpperBound(object, field, op, raw)); + case '$lte': { + // [#20600] `$lte` on the last supported day has no bound to send; what + // it still asks is that the column has a value, which the transport's + // `$null: false` arm spells `IS NOT NULL` — the reading local mode's + // emitter gives the same rewrite. + const upper = this.toRemoteUpperBound(object, field, op, raw); + Object.assign(out, isUnboundedAbove(upper) ? { $null: false } : upper); break; + } case '$in': case '$nin': out[op] = Array.isArray(raw) @@ -2616,15 +2626,18 @@ export class TursoDriver extends SqlDriver { * * `calendarDayUpperBoundRewrite` is the inherited authority for that rule and * already scopes itself to `datetime`, so `date`/`time` columns compile - * byte-identically to before. + * byte-identically to before. [#20600] `UNBOUNDED_ABOVE` — the last + * supported day, whose whole-day bound bounds nothing — is handed back for + * the caller to compile no upper bound. */ private toRemoteUpperBound( object: string, field: string, op: string, raw: unknown, - ): Record { + ): Record | UnboundedAbove { const rewritten = this.calendarDayUpperBoundRewrite(object, field, op, raw); + if (isUnboundedAbove(rewritten)) return UNBOUNDED_ABOVE; if (rewritten) return { [rewritten.op]: rewritten.value }; return { [op]: this.temporalFilterValue(object, field, raw) }; } diff --git a/packages/formula/src/matches-filter-temporal-conformance.test.ts b/packages/formula/src/matches-filter-temporal-conformance.test.ts index c04d21f5f7a..2ee0838e354 100644 --- a/packages/formula/src/matches-filter-temporal-conformance.test.ts +++ b/packages/formula/src/matches-filter-temporal-conformance.test.ts @@ -75,7 +75,8 @@ describe('matchesFilterCondition — temporal conformance on a native-writer pos // identically and must answer identically. const bound = new Date('2026-07-28T00:00:00.000Z'); const got = TEMPORAL_ROWS.filter((r) => matchesFilterCondition(r, { at: { $gte: bound } } as any)).map((r) => r.id); - expect(got).toEqual(['c_open', 'd_mid', 'e_late', 'f_next', 'g_eom']); + // `z_last` (9999-12-31, #20600) is after the bound like `f_next` and `g_eom`. + expect(got).toEqual(['c_open', 'd_mid', 'e_late', 'f_next', 'g_eom', 'z_last']); }); }); diff --git a/packages/formula/src/matches-filter.test.ts b/packages/formula/src/matches-filter.test.ts index 3afbf26ffe5..63c22bcf405 100644 --- a/packages/formula/src/matches-filter.test.ts +++ b/packages/formula/src/matches-filter.test.ts @@ -187,4 +187,32 @@ describe('matchesFilterCondition — calendar-day upper bounds (ADR-0053 D-D, #3 expect(m(at('2026-07-28T09:15:00.000Z'), { created_at: { $lte: null } as never })).toBe(false); expect(m(at('2026-07-28T09:15:00.000Z'), { created_at: { $between: ['2026-04-29', null] } as never })).toBe(false); }); + + // [#20600] 9999-12-31, the last supported day, has no next day: every instant + // is inside its whole-day bound. It compared against the five-digit + // '10000-01-01', which every '2026-…' value sorts above, so the check DENIED + // every write it should have admitted. + it('on the last supported day, every instant is admitted — string or Date', () => { + expect(m(at('2026-07-15T14:00:00.000Z'), { created_at: { $lte: '9999-12-31' } })).toBe(true); + expect(m(at('9999-12-31T23:59:59.999Z'), { created_at: { $lte: '9999-12-31' } })).toBe(true); + expect(m({ created_at: new Date('9999-12-31T10:00:00.000Z') }, { created_at: { $lte: '9999-12-31' } })).toBe(true); + expect(m({ signed_on: '9999-12-31' }, { signed_on: { $lte: '9999-12-31' } })).toBe(true); + expect(m(at('2026-07-15T14:00:00.000Z'), { created_at: { $between: ['2026-01-01', '9999-12-31'] } })).toBe(true); + expect(m(at('9999-12-31T10:00:00.000Z'), { created_at: { $between: ['9999-12-31', '9999-12-31'] } })).toBe(true); + // The min still bounds. + expect(m(at('2025-12-31T23:59:59.999Z'), { created_at: { $between: ['2026-01-01', '9999-12-31'] } })).toBe(false); + }); + + it('…9999-12-30 is still a bound (the control)', () => { + expect(m(at('9999-12-30T10:00:00.000Z'), { created_at: { $lte: '9999-12-30' } })).toBe(true); + expect(m(at('9999-12-31T10:00:00.000Z'), { created_at: { $lte: '9999-12-30' } })).toBe(false); + }); + + it('…a value that denotes no instant keeps the comparison as written — no schema here says it is temporal', () => { + expect(m({ code: 'zzz' }, { code: { $lte: '9999-12-31' } })).toBe(false); + expect(m({ code: '5000' }, { code: { $lte: '9999-12-31' } })).toBe(true); + expect(m({ code: true }, { code: { $lte: '9999-12-31' } })).toBe(false); + expect(m({ code: null }, { code: { $lte: '9999-12-31' } })).toBe(false); + expect(m({}, { code: { $lte: '9999-12-31' } })).toBe(false); + }); }); diff --git a/packages/formula/src/matches-filter.ts b/packages/formula/src/matches-filter.ts index 80c6fbcf6d9..91fee436a9d 100644 --- a/packages/formula/src/matches-filter.ts +++ b/packages/formula/src/matches-filter.ts @@ -114,7 +114,7 @@ import { // the spec and shared by every JS evaluation face — so a `check` evaluated here // and the same predicate compiled to SQL by `read-scope-sql.ts` fold the same // domain. -import { nextUtcCalendarDay, utcInstantMs, asciiCaseInsensitiveContains } from '@objectstack/spec/data'; +import { nextUtcCalendarDay, utcInstantMs, asciiCaseInsensitiveContains, isUnboundedAbove } from '@objectstack/spec/data'; // [#7536] `$like`/`$ilike`'s pattern language, likewise defined once in the // spec: this face evaluates the pattern in JS, `driver-sql` compiles the same // one to `LIKE`/`GLOB`, and a translation written twice would agree on the day @@ -850,10 +850,20 @@ function assertComparableReference( * `YYYY-MM-DD` value, so no field-type lookup is needed — which matters here, * because this evaluator sees a bare record and has no schema to consult. * A full-ISO or non-string bound keeps exact-instant semantics. + * + * [#20600] `9999-12-31`, the last supported day, has no next day to compare + * against (`UNBOUNDED_ABOVE`): every instant the platform stores is on or + * before it, so a value that denotes an instant ({@link utcInstantMs}) is + * inside the bound, and any other value keeps the comparison as written — no + * schema here says it is temporal, and the check must not admit a value the + * operands do not justify. The five-digit `'10000-01-01'` this compared + * against before sorted below every `'2026-…'` value, so the bound DENIED + * every write it should have admitted. */ function lteBound(actual: unknown, bound: unknown): boolean { if (bound == null) return false; const nextDay = nextUtcCalendarDay(bound); + if (isUnboundedAbove(nextDay)) return utcInstantMs(actual) !== null || order(actual, bound, (a, b) => a <= b); if (nextDay != null) return order(actual, nextDay, (a, b) => a < b); return order(actual, bound, (a, b) => a <= b); } diff --git a/packages/objectql/src/having-filter.ts b/packages/objectql/src/having-filter.ts index 1f3d8fd9109..02cfa699b24 100644 --- a/packages/objectql/src/having-filter.ts +++ b/packages/objectql/src/having-filter.ts @@ -151,7 +151,7 @@ import { isEmptyFilterValue } from '@objectstack/spec/data'; // reading of a bare-day upper bound on a `datetime` column (ADR-0053 D-D), from // the spec, where that rule is declared. import { temporalStorageForm, type TemporalComparandKind } from '@objectstack/core'; -import { nextUtcCalendarDay } from '@objectstack/spec/data'; +import { nextUtcCalendarDay, UNBOUNDED_ABOVE, isUnboundedAbove, type UnboundedAbove } from '@objectstack/spec/data'; // [#7047] The ADR-0112 envelope this face's refusals used to omit. Shared with // `filter-comparand-shape.ts` rather than re-declared here — see the note on // {@link invalidFilterError} and on {@link unknownOperator} below. @@ -1311,10 +1311,19 @@ function listHolds(list: readonly unknown[], value: unknown): boolean { * semantics). The same decision both drivers' `where` emitters take * (`SqlDriver.calendarDayUpperBoundRewrite`, `driver-memory`'s `$lte` arm), * read from the spec's `nextUtcCalendarDay`. + * + * [#20600] `UNBOUNDED_ABOVE` for `9999-12-31`, the last supported day: every + * supported value is inside its whole day, so the callers compare against NO + * upper bound — `$lte` asks only for a value, a `$between` keeps its minimum. + * The drivers compile the same (`IS NOT NULL`, `$ne: null`). */ -function wholeDayUpperBound(bound: unknown, kind: TemporalComparandKind | undefined): unknown { +function wholeDayUpperBound( + bound: unknown, + kind: TemporalComparandKind | undefined, +): unknown | UnboundedAbove { if (kind !== 'datetime') return undefined; const next = nextUtcCalendarDay(bound); + if (isUnboundedAbove(next)) return UNBOUNDED_ABOVE; return next === null ? undefined : temporalStorageForm(next, 'datetime'); } @@ -1413,6 +1422,11 @@ function checkCondition( case '$lt': if (!ordered(stored, form(target), (a, b) => a < b)) return false; break; case '$lte': { const dayAfter = wholeDayUpperBound(target, kind); + if (isUnboundedAbove(dayAfter)) { + // [#20600] No upper bound: what `$lte` still asks is a value. + if (stored === null || stored === undefined) return false; + break; + } if (dayAfter !== undefined ? !ordered(stored, dayAfter, (a, b) => a < b) : !ordered(stored, form(target), (a, b) => a <= b)) return false; @@ -1421,10 +1435,14 @@ function checkCondition( case '$between': { if (!Array.isArray(target)) break; const dayAfter = wholeDayUpperBound(target[1], kind); + // [#20600] A max on the last supported day bounds nothing: the range + // keeps its minimum alone. if (ordered(stored, form(target[0]), (a, b) => a < b) - || (dayAfter !== undefined - ? ordered(stored, dayAfter, (a, b) => a >= b) - : ordered(stored, form(target[1]), (a, b) => a > b))) return false; + || (isUnboundedAbove(dayAfter) + ? false + : dayAfter !== undefined + ? ordered(stored, dayAfter, (a, b) => a >= b) + : ordered(stored, form(target[1]), (a, b) => a > b))) return false; break; } case '$in': if (!Array.isArray(target) || !listHolds(target.map(form), stored)) return false; break; diff --git a/packages/rest/src/data-query-calendar-day-last-day.test.ts b/packages/rest/src/data-query-calendar-day-last-day.test.ts new file mode 100644 index 00000000000..bf1b7bb7b96 --- /dev/null +++ b/packages/rest/src/data-query-calendar-day-last-day.test.ts @@ -0,0 +1,195 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [#20600] A bare `YYYY-MM-DD` upper bound on a `datetime` field — `$lte`, or a + * `$between` maximum — on `9999-12-31`, the last day of the supported years + * (0001..9999), includes that whole day, at the public door + * `POST /api/v1/data/:object/query`, over a real `SqlDriver`. + * + * The widening (ADR-0053 D-D) compiles `$lte day` to `< nextUtcCalendarDay(day)` + * midnight UTC. For the last day the helper answered `'10000-01-01'`, and the + * driver compiled a bound in a five-digit year. On SQLite the column is ISO + * text, where that bound sorts below `'2026-…'`, so it excluded every row; + * PostgreSQL parsed it as an instant and answered. Every supported value is at + * most the last millisecond of `9999-12-31`, so the helper now answers + * `UNBOUNDED_ABOVE` for that day and the driver compiles no upper bound: `$lte` + * keeps every row that has a value, and a `$between` keeps only its minimum. + * Measured through this door on the base, the process in America/New_York: + * + * | `where opened_at` | SQLite (base) | PostgreSQL 16 (base) | the whole day | + * |:--|:--|:--|:--| + * | `$lte '9999-12-31'` | none | every row with a value | every row with a value | + * | `$between ['2026-01-01', '9999-12-31']` | none | every row with a value | every row with a value | + * | `$between ['9999-12-31', '9999-12-31']` | none | `open`, `mid`, `last` | `open`, `mid`, `last` | + * | `$lte '9999-12-30'` (control) | `c26`, `prev` | `c26`, `prev` | the same | + * + * The lower-bound operators and equality do not read the widening and answer + * the same before and after: `$gte` / `$gt` / `$lt` anchor the day to its + * midnight, and `$eq` is that midnight instant. A row with no value stays out + * of every bound, and a `$not` over the unbounded `$lte` answers exactly it. + * + * ## The dialect axis of THIS file + * + * The SQLite cell always runs. The PostgreSQL cell runs where + * `OS_TEST_POSTGRES_URL` is set and is a named skip otherwise; no CI job + * provisions it for this package (the shared temporal conformance table in + * `@objectstack/spec` carries the same last-day cases to the live PostgreSQL + * and MySQL job). It owns one table, dropped before and after. An in-memory + * cell is not here: `@objectstack/driver-memory` has no binding in this + * package. The helper's own pins are + * `packages/spec/src/data/calendar-day.test.ts`. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import { ObjectQL } from '@objectstack/objectql'; +import { SqlDriver } from '@objectstack/driver-sql'; +import { ObjectStackProtocolImplementation } from '@objectstack/metadata-protocol'; +import { RestServer } from './rest-server'; + +const OBJECT = 'rest_calendar_day_20600'; +const HOST_ZONE = 'America/New_York'; + +const LEDGER = { + name: OBJECT, + label: 'Ledger 20600', + fields: { + memo: { name: 'memo', type: 'text' as const }, + opened_at: { name: 'opened_at', type: 'datetime' as const }, + }, +}; + +/** id · the instant written through the door (`null`: no value). */ +const ROWS: ReadonlyArray = [ + ['c26', '2026-07-15T14:00:00.000Z'], // the card's control row + ['prev', '9999-12-30T10:00:00.000Z'], // the day before the last day + ['open', '9999-12-31T00:00:00.000Z'], // the last day's first instant + ['mid', '9999-12-31T10:00:00.000Z'], // the card's row + ['last', '9999-12-31T23:59:59.999Z'], // the last supported millisecond + ['none', null], // no value: outside every bound +]; + +const WITH_A_VALUE = ['c26', 'last', 'mid', 'open', 'prev']; + +/** `where` · the ids it answers, sorted. */ +const QUERIES: ReadonlyArray, readonly string[]]> = [ + // The widening, on the last day: no upper bound. + ["$lte '9999-12-31'", { opened_at: { $lte: '9999-12-31' } }, WITH_A_VALUE], + ["$between ['2026-01-01', '9999-12-31']", { opened_at: { $between: ['2026-01-01', '9999-12-31'] } }, WITH_A_VALUE], + ["$between ['9999-12-31', '9999-12-31']", { opened_at: { $between: ['9999-12-31', '9999-12-31'] } }, ['last', 'mid', 'open']], + ["$not $lte '9999-12-31'", { $not: { opened_at: { $lte: '9999-12-31' } } }, ['none']], + // The day before: a bound, as on every other day (the control). + ["$lte '9999-12-30' (control)", { opened_at: { $lte: '9999-12-30' } }, ['c26', 'prev']], + ["$between ['2026-01-01', '9999-12-30'] (control)", { opened_at: { $between: ['2026-01-01', '9999-12-30'] } }, ['c26', 'prev']], + // The operators that do not read the widening: that day's midnight. + ["$gte '9999-12-31'", { opened_at: { $gte: '9999-12-31' } }, ['last', 'mid', 'open']], + ["$gt '9999-12-31'", { opened_at: { $gt: '9999-12-31' } }, ['last', 'mid']], + ["$lt '9999-12-31'", { opened_at: { $lt: '9999-12-31' } }, ['c26', 'prev']], + ["$eq '9999-12-31'", { opened_at: { $eq: '9999-12-31' } }, ['open']], +]; + +interface Cell { + id: 'sqlite' | 'pg'; + label: string; + env: string | null; + config: () => Record | null; +} + +const CELLS: readonly Cell[] = [ + { id: 'sqlite', label: 'sqlite', env: null, config: () => ({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true }) }, + { + id: 'pg', + label: 'live postgres', + env: 'OS_TEST_POSTGRES_URL', + config: () => (process.env.OS_TEST_POSTGRES_URL ? { client: 'pg', connection: process.env.OS_TEST_POSTGRES_URL } : null), + }, +]; + +function createMockServer() { + const noop = () => {}; + return { get: noop, post: noop, put: noop, delete: noop, patch: noop, use: noop, listen: async () => {}, close: async () => {} }; +} + +function makeRes() { + const res: any = { + write: () => true, end: () => {}, + header: () => res, + status: (code: number) => { res._status = code; return res; }, + json: (body: any) => { res._json = body; return res; }, + }; + return res; +} + +const originalTz = process.env.TZ; + +for (const cell of CELLS) { + const config = cell.config(); + describe.skipIf(!config)( + `[#20600] a bare-day upper bound on 9999-12-31 has no upper bound, at the public door — ${cell.label}${config ? '' : ` (skipped: set ${cell.env} to run this cell)`}`, + () => { + let engine: ObjectQL; + let driver: any; + let call: (method: string, path: string, params: Record, body: unknown) => Promise<{ status: number; body: any }>; + + beforeAll(async () => { + // A host whose zone is not UTC, so a host-zone reading of a bound would show. + process.env.TZ = HOST_ZONE; + expect(Intl.DateTimeFormat().resolvedOptions().timeZone, 'the host zone really changed').toBe(HOST_ZONE); + + driver = new SqlDriver(config as any); + if (cell.id !== 'sqlite') await driver.execute(`drop table if exists ${OBJECT}`).catch(() => {}); + engine = new ObjectQL(); + engine.registerDriver(driver, true); + await engine.init(); + engine.registry.registerObject(LEDGER as any); + await engine.syncSchemas(); + + const protocol = new ObjectStackProtocolImplementation(engine as any); + const rest = new RestServer(createMockServer() as any, protocol as any, { api: { requireAuth: false } } as any); + (rest as any).resolveExecCtx = async () => ({ userId: 'test-user' }); + rest.registerRoutes(); + call = async (method, path, params, body) => { + const route = rest.getRoutes().find((r: any) => r.method === method && r.path === path); + expect(route, `${method} ${path}`).toBeDefined(); + const res = makeRes(); + await route!.handler({ params, body, query: {}, headers: {} } as any, res); + return { status: res._status ?? 200, body: res._json }; + }; + + for (const [id, openedAt] of ROWS) { + const record = openedAt === null ? { id, memo: 'm' } : { id, memo: 'm', opened_at: openedAt }; + const created = await call('POST', '/api/v1/data/:object', { object: OBJECT }, record); + expect(created.status, `create ${id} ${openedAt}: ${JSON.stringify(created.body)}`).toBe(201); + } + }); + + afterAll(async () => { + if (cell.id !== 'sqlite') await driver?.execute(`drop table if exists ${OBJECT}`).catch(() => {}); + try { await engine?.destroy(); } catch { /* noop */ } + if (originalTz === undefined) delete process.env.TZ; + else process.env.TZ = originalTz; + }); + + const idsWhere = async (where: Record) => { + const res = await call('POST', '/api/v1/data/:object/query', { object: OBJECT }, { where }); + expect(res.status, JSON.stringify(res.body)).toBe(200); + return (res.body.records as Array<{ id: string }>).map((r) => r.id).sort(); + }; + + it('every row reads back the instant it was written — the rows the bounds are measured against', async () => { + const res = await call('POST', '/api/v1/data/:object/query', { object: OBJECT }, { where: { memo: 'm' } }); + expect(res.status, JSON.stringify(res.body)).toBe(200); + const stored = Object.fromEntries( + (res.body.records as Array<{ id: string; opened_at?: string | null }>).map((r) => [r.id, r.opened_at ?? null]), + ); + expect(stored).toEqual(Object.fromEntries(ROWS)); + }); + + it('$lte and the $between maximum on 9999-12-31 compile no upper bound; 9999-12-30 is a bound; the lower-bound operators do not move', async () => { + // Every reading first, then one comparison, so a red run shows every cell. + const got: Record = {}; + for (const [name, where] of QUERIES) got[name] = await idsWhere(where); + expect(got).toEqual(Object.fromEntries(QUERIES.map(([name, , want]) => [name, want]))); + }); + }, + ); +} diff --git a/packages/services/service-analytics/src/__tests__/objectql-daterange.test.ts b/packages/services/service-analytics/src/__tests__/objectql-daterange.test.ts index 81bda02fd53..2ab168a4afa 100644 --- a/packages/services/service-analytics/src/__tests__/objectql-daterange.test.ts +++ b/packages/services/service-analytics/src/__tests__/objectql-daterange.test.ts @@ -427,6 +427,29 @@ describe('ObjectQLStrategy.generateSql — window rendering (#3650)', () => { expect(sql).toContain('(close_date >= $2 AND close_date < $3)'); expect(params).toEqual(['won', '2026-01-01', '2026-02-01']); }); + + // [#20600] 9999-12-31, the last supported day, has no next day: the driver + // compiles no upper bound for it, so the echo renders none. It used to bind + // the five-digit '10000-01-01' as the upper bound — SQL that answers no rows + // on SQLite, where the column is ISO text that sorts above it. + it('renders a window ending on the last supported day with no upper bound; 9999-12-30 keeps one', async () => { + const svc = makeService([]); + const echo = (end: string) => svc.generateSql!({ + cube: 'sales', + dimensions: ['stage'], + measures: ['revenue'], + timeDimensions: [{ dimension: 'close_date', dateRange: ['2026-01-01', end] }], + }); + + const last = await echo('9999-12-31'); + expect(last.sql).toContain('(close_date >= $1)'); + expect(last.sql).not.toContain('close_date <'); + expect(last.params).toEqual(['2026-01-01']); + + const control = await echo('9999-12-30'); + expect(control.sql).toContain('(close_date >= $1 AND close_date < $2)'); + expect(control.params).toEqual(['2026-01-01', '9999-12-31']); + }); }); describe('ObjectQLStrategy — cross-object FK-expand carries the window (#3650 × #3654)', () => { diff --git a/packages/services/service-analytics/src/preview-evaluator.ts b/packages/services/service-analytics/src/preview-evaluator.ts index 126bb1dbb31..7980b2044f1 100644 --- a/packages/services/service-analytics/src/preview-evaluator.ts +++ b/packages/services/service-analytics/src/preview-evaluator.ts @@ -35,6 +35,7 @@ import { nextUtcCalendarDay, resolveAnalyticsDateRangeString, utcInstantMs, + isUnboundedAbove, } from '@objectstack/core'; import { explicitDateRangeWindow } from './date-range-array-arm.js'; // [#19810] The `where` door's refusal envelope — `INVALID_FILTER` / 400, @@ -92,9 +93,17 @@ function compare(a: unknown, b: unknown): number { * matters here, because the preview sees drafted rows with no schema. * * Shared by `$lte` and the max of `$between` so the two cannot drift apart. + * + * [#20600] `9999-12-31`, the last supported day, has no next day to compare + * against (`UNBOUNDED_ABOVE`): every instant the platform stores is on or + * before it, so a value that denotes an instant ({@link utcInstantMs}) is + * inside the bound, and any other value keeps the comparison as written — the + * same reading `formula`'s `check` evaluator gives, so the two type-blind + * surfaces answer one bound alike. */ function lteBound(value: unknown, bound: unknown): boolean { const nextDay = nextUtcCalendarDay(bound); + if (isUnboundedAbove(nextDay)) return utcInstantMs(value) !== null || compare(value, bound) <= 0; if (nextDay != null) return compare(value, nextDay) < 0; return compare(value, bound) <= 0; } @@ -671,16 +680,20 @@ export function evaluateAnalyticsQueryOverRows( // ⛔ Neither reaches a RESOLVED preset window: it states its own upper // reading and is never a bare day — the ten calendar presets stop BEFORE // their end instant, the three rolling ones end at NOW and reach it. + // [#20600] A bare end on the last supported day has no next day to stop + // before: every value is inside it, so the window keeps its start alone. const nextDay = explicit ? nextUtcCalendarDay(end) : null; filtered = filtered.filter((r) => { const v = String(r[field] ?? ''); const inUpper = endExclusive ? v < end - : nextDay != null - ? v < nextDay - : explicit - ? v <= `${end}~` - : v <= end; + : isUnboundedAbove(nextDay) + ? true + : nextDay != null + ? v < nextDay + : explicit + ? v <= `${end}~` + : v <= end; return v >= start && inUpper; }); } diff --git a/packages/services/service-analytics/src/strategies/native-sql-strategy.ts b/packages/services/service-analytics/src/strategies/native-sql-strategy.ts index 1742fbc131c..16607fbe9f5 100644 --- a/packages/services/service-analytics/src/strategies/native-sql-strategy.ts +++ b/packages/services/service-analytics/src/strategies/native-sql-strategy.ts @@ -18,7 +18,7 @@ import { declaredValueShapeResolver, whereEmptyLeafSql } from '../empty-operator import { datasetInvalidError, invalidMemberError } from '../dataset-refusal.js'; import { type LikeShape } from '../like-pattern.js'; import { textMatchPredicateSql, sqlDialectFor } from '../text-match-sql.js'; -import { nextUtcCalendarDay, resolveAnalyticsDateRangeString } from '@objectstack/core'; +import { nextUtcCalendarDay, resolveAnalyticsDateRangeString, isUnboundedAbove } from '@objectstack/core'; import { explicitDateRangeWindow } from '../date-range-array-arm.js'; /** @@ -543,13 +543,20 @@ export class NativeSQLStrategy implements AnalyticsStrategy { // `[a, b]` a CALLER wrote keeps the inclusive reading it has always // had — the #16179 separation, on this side too. const nextDay = resolved ? null : nextUtcCalendarDay(range[1]); - const upperExclusive = resolved ? resolved.endExclusive : nextDay != null; params.push(this.coerceTemporal(ctx, td2, range[0])); const lower = `${column} >= $${params.length}`; - params.push(this.coerceTemporal(ctx, td2, nextDay ?? range[1])); - whereClauses.push( - `(${lower} AND ${column} ${upperExclusive ? '<' : '<='} $${params.length})`, - ); + // [#20600] A bare end on the last supported day has no next day to + // stop before: every value is inside it, so the window keeps its + // start alone. + if (isUnboundedAbove(nextDay)) { + whereClauses.push(`(${lower})`); + } else { + const upperExclusive = resolved ? resolved.endExclusive : nextDay != null; + params.push(this.coerceTemporal(ctx, td2, nextDay ?? range[1])); + whereClauses.push( + `(${lower} AND ${column} ${upperExclusive ? '<' : '<='} $${params.length})`, + ); + } } } } @@ -1235,6 +1242,10 @@ export class NativeSQLStrategy implements AnalyticsStrategy { // Equivalent to `<=` for a `date` column, so no column-type lookup needed. if (operator === 'lte') { const nextDay = nextUtcCalendarDay(values[0]); + // [#20600] On the last supported day there is no next day: every value is + // inside the bound, so what `lte` still asks is a value — the `set` arm's + // `IS NOT NULL`. + if (isUnboundedAbove(nextDay)) return `${rawCol} IS NOT NULL`; if (nextDay != null) { params.push(this.coerceTemporal(ctx, target, nextDay)); return `${this.temporalColumn(ctx, target, rawCol)} < $${params.length}`; diff --git a/packages/services/service-analytics/src/strategies/objectql-strategy.ts b/packages/services/service-analytics/src/strategies/objectql-strategy.ts index dae26260124..f71f818c557 100644 --- a/packages/services/service-analytics/src/strategies/objectql-strategy.ts +++ b/packages/services/service-analytics/src/strategies/objectql-strategy.ts @@ -28,7 +28,7 @@ import { declaredValueShapeResolver, whereEmptyLeafSql } from '../empty-operator import { invalidMemberError } from '../dataset-refusal.js'; import { type LikeShape } from '../like-pattern.js'; import { textMatchPredicateSql, sqlDialectFor } from '../text-match-sql.js'; -import { nextUtcCalendarDay, resolveAnalyticsDateRangeString } from '@objectstack/core'; +import { nextUtcCalendarDay, resolveAnalyticsDateRangeString, isUnboundedAbove } from '@objectstack/core'; import { explicitDateRangeWindow } from '../date-range-array-arm.js'; import { rebucketCrossObject, @@ -546,6 +546,13 @@ export class ObjectQLStrategy implements AnalyticsStrategy { // the final day's rows and cannot reproduce the result. for (const { field, bounds } of this.dateRangeBounds(cube, query)) { const nextDay = nextUtcCalendarDay(bounds.$lte); + // [#20600] A bare end on the last supported day renders no upper bound, + // because the driver compiles none for it. + if (isUnboundedAbove(nextDay)) { + params.push(bounds.$gte); + whereParts.push(`(${field} >= $${params.length})`); + continue; + } params.push(bounds.$gte, nextDay ?? bounds.$lte); whereParts.push( `(${field} >= $${params.length - 1} AND ${field} ${nextDay ? '<' : '<='} $${params.length})`, diff --git a/packages/spec/api-surface/data.json b/packages/spec/api-surface/data.json index ad3eaa9db8b..cfe2929ec49 100644 --- a/packages/spec/api-surface/data.json +++ b/packages/spec/api-surface/data.json @@ -737,9 +737,11 @@ "TursoDriverSpec (const)", "TursoTransportMode (type)", "TursoTransportModeSchema (const)", + "UNBOUNDED_ABOVE (const)", "UNRESOLVED_PLACEHOLDER_REFUSED (const)", "URL_CREDENTIAL_QUERY_PARAM_REFUSED (const)", "URL_EMBEDDED_CREDENTIAL_REFUSED (const)", + "UnboundedAbove (type)", "UniqueScope (type)", "UniqueScopeSchema (const)", "UnknownAuthoringKeyFinding (interface)", @@ -850,6 +852,7 @@ "isTenancyDisabled (function)", "isTextFilterOperator (function)", "isTitleEligible (function)", + "isUnboundedAbove (function)", "isUniqueDeclared (function)", "isVirtualSearchField (function)", "judgeImportMappingTarget (function)", diff --git a/packages/spec/export-origins/data.json b/packages/spec/export-origins/data.json index a0b42e4624d..e2eb504b6f3 100644 --- a/packages/spec/export-origins/data.json +++ b/packages/spec/export-origins/data.json @@ -724,9 +724,11 @@ "TursoDriverSpec": "src/data/driver/turso.zod.ts#TursoDriverSpec (const)", "TursoTransportMode": "src/data/driver/turso.zod.ts#TursoTransportMode (type)", "TursoTransportModeSchema": "src/data/driver/turso.zod.ts#TursoTransportModeSchema (const)", + "UNBOUNDED_ABOVE": "src/data/calendar-day.ts#UNBOUNDED_ABOVE (const)", "UNRESOLVED_PLACEHOLDER_REFUSED": "src/data/driver/common.zod.ts#UNRESOLVED_PLACEHOLDER_REFUSED (const)", "URL_CREDENTIAL_QUERY_PARAM_REFUSED": "src/data/driver/common.zod.ts#URL_CREDENTIAL_QUERY_PARAM_REFUSED (const)", "URL_EMBEDDED_CREDENTIAL_REFUSED": "src/data/driver/common.zod.ts#URL_EMBEDDED_CREDENTIAL_REFUSED (const)", + "UnboundedAbove": "src/data/calendar-day.ts#UnboundedAbove (type)", "UniqueScope": "src/data/field.zod.ts#UniqueScope (type)", "UniqueScopeSchema": "src/data/field.zod.ts#UniqueScopeSchema (const)", "UnknownAuthoringKeyFinding": "src/data/authoring-key-lint.ts#UnknownAuthoringKeyFinding (interface)", @@ -837,6 +839,7 @@ "isTenancyDisabled": "src/data/object.zod.ts#isTenancyDisabled (function)", "isTextFilterOperator": "src/data/filter-text-operator-declared-type.ts#isTextFilterOperator (function)", "isTitleEligible": "src/data/display-name.ts#isTitleEligible (function)", + "isUnboundedAbove": "src/data/calendar-day.ts#isUnboundedAbove (function)", "isUniqueDeclared": "src/data/field.zod.ts#isUniqueDeclared (function)", "isVirtualSearchField": "src/data/search-fields.ts#isVirtualSearchField (function)", "judgeImportMappingTarget": "src/data/import-mapping-target.ts#judgeImportMappingTarget (function)", diff --git a/packages/spec/src/data/calendar-day.test.ts b/packages/spec/src/data/calendar-day.test.ts index aadb5f0422a..dbe5ea7619a 100644 --- a/packages/spec/src/data/calendar-day.test.ts +++ b/packages/spec/src/data/calendar-day.test.ts @@ -9,7 +9,7 @@ */ import { describe, expect, it } from 'vitest'; -import { nextUtcCalendarDay, utcInstantMs } from './calendar-day'; +import { nextUtcCalendarDay, utcInstantMs, UNBOUNDED_ABOVE, isUnboundedAbove } from './calendar-day'; describe('nextUtcCalendarDay', () => { it('advances one calendar day', () => { @@ -107,3 +107,71 @@ describe('[#20550] years 0001..0099 are calendar days, not 1900..1999', () => { expect(utcInstantMs('0100-02-29')).toBeNull(); }); }); + +/** + * [#20600] `9999-12-31`, the last day of the supported years, has no next day + * with a `YYYY-MM-DD` spelling. The helper answered the five-digit + * `'10000-01-01'`, which sorts below `'2026-…'` as text, so a SQLite `datetime` + * `$lte '9999-12-31'` answered no rows. It now answers `UNBOUNDED_ABOVE`: every + * supported value is inside that day's whole-day bound, so an emitter compiles + * no upper bound. The answer is neither `null` ("not a calendar day", which + * would compile the day's midnight and miss the rest of it) nor a string. + */ +describe('[#20600] the last supported day answers UNBOUNDED_ABOVE', () => { + it('9999-12-31 answers UNBOUNDED_ABOVE — not a five-digit day, not null', () => { + const next = nextUtcCalendarDay('9999-12-31'); + expect(next).toBe(UNBOUNDED_ABOVE); + expect(next).not.toBeNull(); + expect(typeof next).toBe('symbol'); + expect(nextUtcCalendarDay(' 9999-12-31 ')).toBe(UNBOUNDED_ABOVE); // trimmed, like every day + }); + + it('the days before it still answer the next day (the control)', () => { + expect(nextUtcCalendarDay('9999-12-30')).toBe('9999-12-31'); + expect(nextUtcCalendarDay('9999-11-30')).toBe('9999-12-01'); + expect(nextUtcCalendarDay('9998-12-31')).toBe('9999-01-01'); + }); + + it('it is the one real day that does: every answer is a bare day or that value', () => { + // Walk the last year day by day; only its final day has no next day. + let unbounded = 0; + for (let ms = Date.parse('9999-01-01T00:00:00.000Z'); ms <= Date.parse('9999-12-31T00:00:00.000Z'); ms += 86_400_000) { + const day = new Date(ms).toISOString().slice(0, 10); + const next = nextUtcCalendarDay(day); + if (isUnboundedAbove(next)) unbounded += 1; + else expect(next, day).toMatch(/^\d{4}-\d{2}-\d{2}$/); + } + expect(unbounded).toBe(1); + }); + + it('refusals are unchanged: an instant on that day, an impossible day and a five-digit day stay null', () => { + expect(nextUtcCalendarDay('9999-12-31T10:00:00.000Z')).toBeNull(); + expect(nextUtcCalendarDay(new Date('9999-12-31T00:00:00.000Z'))).toBeNull(); + expect(nextUtcCalendarDay('9999-12-32')).toBeNull(); + expect(nextUtcCalendarDay('10000-01-01')).toBeNull(); + }); + + it('is one registered symbol, so a second bundled copy of the helper answers the same value', () => { + expect(UNBOUNDED_ABOVE).toBe(Symbol.for('objectstack.calendarDay.unboundedAbove')); + }); + + it('isUnboundedAbove is true for that answer alone, and narrows it away', () => { + expect(isUnboundedAbove(nextUtcCalendarDay('9999-12-31'))).toBe(true); + expect(isUnboundedAbove(UNBOUNDED_ABOVE)).toBe(true); + expect(isUnboundedAbove(Symbol.for('objectstack.calendarDay.unboundedAbove'))).toBe(true); + // An unregistered symbol with the same description is another value. + expect(isUnboundedAbove(Symbol('objectstack.calendarDay.unboundedAbove'))).toBe(false); + for (const other of [nextUtcCalendarDay('9999-12-30'), null, undefined, '10000-01-01', '9999-12-31', {}]) { + expect(isUnboundedAbove(other), String(other)).toBe(false); + } + // The false branch is `string | null` to the compiler: a template literal + // compiles there, and would not on the unnarrowed answer (TS2731). + const next = nextUtcCalendarDay('9999-12-30'); + const bound = isUnboundedAbove(next) || next === null ? null : `${next}T00:00:00.000Z`; + expect(bound).toBe('9999-12-31T00:00:00.000Z'); + }); + + it("utcInstantMs still reads 9999-12-31 as that day's midnight UTC", () => { + expect(utcInstantMs('9999-12-31')).toBe(Date.parse('9999-12-31T00:00:00.000Z')); + }); +}); diff --git a/packages/spec/src/data/calendar-day.ts b/packages/spec/src/data/calendar-day.ts index f03557d5bb7..d573a71e4ce 100644 --- a/packages/spec/src/data/calendar-day.ts +++ b/packages/spec/src/data/calendar-day.ts @@ -18,6 +18,7 @@ * |---|---| * | `$gte` / `$gt` / `$lt` | that day's `00:00:00.000` — already correct as written | * | `$lte`, a `$between` max, a `dateRange` end | the WHOLE day → compile `< nextUtcCalendarDay(day)` | + * | the same, on `9999-12-31` | the WHOLE day → compile NO upper bound ({@link UNBOUNDED_ABOVE}) | * * Half-open, never an inclusive `23:59:59.999`: the latter re-opens the gap at * whatever sub-millisecond precision a dialect keeps (Postgres stores @@ -25,21 +26,72 @@ * already emit. On a `date` column `< nextDay` is order-equivalent to * `<= day` under plain `YYYY-MM-DD` text ordering, which is what lets emitters * that cannot see the column type apply it unconditionally. + * + * The last row is the one day with no next day to stop before. The supported + * years are 0001..9999 (the comparand and write doors refuse the rest), so + * every supported value is at most the last millisecond of `9999-12-31`, and + * the whole-day bound of that day bounds nothing: `$lte '9999-12-31'` holds for + * every value there is, and a `$between` whose maximum is that day keeps only + * its minimum. The day after it has no bare-day spelling: `'10000-01-01'` sorts + * below `'2026-…'` as text, so a SQLite column compared against it answered no + * rows, and `null` would compile the day's midnight and miss the rest of it. + * So the helper answers {@link UNBOUNDED_ABOVE}, and each emitter compiles no + * upper bound for it — for a lone `$lte`, "the value is not null", the one + * condition the comparison it replaces carried beyond its bound. + */ + +/** + * The type of {@link UNBOUNDED_ABOVE}: a `symbol` carrying a STRUCTURAL brand. + * + * Structural, not `unique symbol`, and that is the point. A `unique symbol` is + * nominal to the declaration that spells it, and this package ships the `./data` + * entry's declarations twice (`dist/data/index.d.mts` under `import`, + * `dist/data/index.d.ts` under `require`), each declaring its own copy. A + * program that reaches the sentinel through both files saw two unrelated + * `unique symbol`s: the comparison against the constant was refused as having + * "no overlap" (TS2367), and the guard stopped narrowing (TS2339) — measured in + * `@objectstack/dogfood`, whose program maps one workspace package to source and + * the rest to built types. Two copies of this alias are one type wherever they meet, so the answer + * of the helper from one file and the constant or guard from another agree. + * + * Still a `symbol`, so the last day stays a member every caller must handle: + * TypeScript refuses it in a template literal (TS2731), in a relational + * comparison (TS2469) and where a `string` is expected (TS2345). ⛔ Never widen + * it into `string`: a string-typed sentinel compiles and sorts silently as a + * bound — the defect this answer exists to end. The brand key names no property + * the value really has; it only makes the type distinct from every other + * `symbol`. Narrow it with {@link isUnboundedAbove} (or `typeof … === 'symbol'`): + * because the type is not a unit type, `=== UNBOUNDED_ABOVE` compares, but does + * not narrow. */ +export type UnboundedAbove = symbol & { readonly __objectstackCalendarDayBound: 'unbounded-above' }; /** - * The calendar day after a bare `YYYY-MM-DD` string — the exclusive upper bound - * of that day. + * The answer {@link nextUtcCalendarDay} gives for `9999-12-31`, the last day of + * the supported years: that day's whole-day bound is past every supported + * value, so an emitter compiles NO upper bound for it (see the module note). * - * Returns `null` for anything that is not a valid bare calendar day. That - * refusal is load-bearing in two directions: - * - a full ISO timestamp or a `Date` keeps **instant** semantics and must not - * be widened, so callers get `null` and compile their original bound; - * - an impossible day (`2026-02-30`, `2026-13-01`) is rejected rather than - * rolled over, so a caller falls back to the untranslated comparand instead - * of silently querying a date the author never wrote. + * A symbol, so no caller can compile it as a bound by accident, and distinct + * from `null` ("not a calendar day"); its type is {@link UnboundedAbove}. + * Registered with `Symbol.for`, so every bundled copy of this module (the ESM + * and CJS builds, the browser builds) answers the same VALUE, whichever one a + * caller's import resolved to. The structural type is what makes the copies' + * declarations one TYPE as well. */ -export function nextUtcCalendarDay(value: unknown): string | null { +export const UNBOUNDED_ABOVE = Symbol.for('objectstack.calendarDay.unboundedAbove') as UnboundedAbove; + +/** + * Is `value` {@link UNBOUNDED_ABOVE}? The guard every reader of + * {@link nextUtcCalendarDay} shares: it narrows the answer to `string | null` + * on its false branch in any program, whichever declaration file each import + * resolved to — which `=== UNBOUNDED_ABOVE` cannot do (see {@link UnboundedAbove}). + * An unregistered `Symbol('objectstack.calendarDay.unboundedAbove')` is not it. + */ +export function isUnboundedAbove(value: unknown): value is UnboundedAbove { + return value === UNBOUNDED_ABOVE; +} + +export function nextUtcCalendarDay(value: unknown): string | UnboundedAbove | null { if (typeof value !== 'string') return null; const day = value.trim(); const m = /^(\d{4})-(\d{2})-(\d{2})$/.exec(day); @@ -50,7 +102,14 @@ export function nextUtcCalendarDay(value: unknown): string | null { // into March, so the round-trip is what proves the input was a real calendar // day — in every year the four digits spell, 0001..0099 included. if (fmtUtcDay(start) !== day) return null; - return fmtUtcDay(utcMidnight(y, mo - 1, d + 1)); + const next = fmtUtcDay(utcMidnight(y, mo - 1, d + 1)); + // The one real day whose successor has no `YYYY-MM-DD` spelling is + // 9999-12-31: `fmtUtcDay` pads a year to four digits and never truncates, so + // it spells the day after as the five-digit `'10000-01-01'`. The shape ends + // where the supported years end (the doors refuse a year above 9999 — + // `@objectstack/core`'s `isOutsideTemporalYearRange` is that range's one + // statement), so this reads the shape, not a second copy of the range. + return /^\d{4}-/.test(next) ? next : UNBOUNDED_ABOVE; } /** diff --git a/packages/spec/src/data/date-range-presets.test.ts b/packages/spec/src/data/date-range-presets.test.ts index 177299dcee1..99741182445 100644 --- a/packages/spec/src/data/date-range-presets.test.ts +++ b/packages/spec/src/data/date-range-presets.test.ts @@ -157,8 +157,10 @@ const midnightUtc = (day: string): number => Date.parse(`${day}T00:00:00.000Z`); */ function prescribedDayCount(startDay: string, endDay: string): number { const exclusiveEnd = nextUtcCalendarDay(endDay); - expect(exclusiveEnd, `${endDay} must be a real calendar day`).not.toBeNull(); - return (midnightUtc(exclusiveEnd!) - midnightUtc(startDay)) / DAY_MS; + // A day: never `null`, and never `UNBOUNDED_ABOVE` — no preset ends on + // 9999-12-31, the one day without a next day (#20600). + expect(typeof exclusiveEnd, `${endDay} must be a real calendar day before the last one`).toBe('string'); + return (midnightUtc(exclusiveEnd as string) - midnightUtc(startDay)) / DAY_MS; } describe('the prescribed window covers exactly the days the preset names (#17014)', () => { diff --git a/packages/spec/src/data/temporal-conformance.ts b/packages/spec/src/data/temporal-conformance.ts index 2bae1acfe94..b9882f36995 100644 --- a/packages/spec/src/data/temporal-conformance.ts +++ b/packages/spec/src/data/temporal-conformance.ts @@ -217,6 +217,7 @@ export const TEMPORAL_ROWS: readonly TemporalRow[] = [ { id: 'f_next', at: '2026-07-29T00:00:00.000Z', on: '2026-07-29', writerForm: 'native', why: 'next midnight — the exclusive edge a half-open bound must NOT keep' }, { id: 'g_eom', at: '2026-07-31T23:59:59.999Z', on: '2026-07-31', writerForm: 'wire', why: 'last representable instant of a month — month rollover' }, { id: 'h_leap', at: '2024-02-29T12:00:00.000Z', on: '2024-02-29', writerForm: 'native', why: 'leap day — February rollover' }, + { id: 'z_last', at: '9999-12-31T10:00:00.000Z', on: '9999-12-31', writerForm: 'native', why: 'the last supported day (years 0001..9999), after its midnight — the one day with no next day, so its whole-day upper bound bounds nothing (#20600)' }, ] as const; /** Which declared field type a case filters on. */ @@ -314,7 +315,7 @@ export const TEMPORAL_CASES: readonly TemporalCase[] = [ kind: 'datetime', filter: { at: { $gte: '2026-07-28' } }, tokenFilter: { at: { $gte: '{today}' } }, - expected: ['c_open', 'd_mid', 'e_late', 'f_next', 'g_eom'], + expected: ['c_open', 'd_mid', 'e_late', 'f_next', 'g_eom', 'z_last'], note: 'A LOWER bound anchors to 00:00 — c_open is included precisely because the bound is inclusive of that instant.', }, { @@ -334,7 +335,7 @@ export const TEMPORAL_CASES: readonly TemporalCase[] = [ kind: 'date', filter: { on: { $gt: '2026-07-28' } }, tokenFilter: { on: { $gt: '{today}' } }, - expected: ['f_next', 'g_eom'], + expected: ['f_next', 'g_eom', 'z_last'], note: 'The boundary day itself is out; the mirror-image of widening a lower bound.', }, @@ -352,7 +353,7 @@ export const TEMPORAL_CASES: readonly TemporalCase[] = [ field: 'at', kind: 'datetime', filter: { at: { $gte: '2026-07-28T09:15:00.000Z' } }, - expected: ['d_mid', 'e_late', 'f_next', 'g_eom'], + expected: ['d_mid', 'e_late', 'f_next', 'g_eom', 'z_last'], note: 'd_mid is exactly the bound and inclusive.', }, @@ -392,6 +393,50 @@ export const TEMPORAL_CASES: readonly TemporalCase[] = [ note: 'Negative epoch ms. The #3773 family: any surface that assumes a datetime is a non-negative epoch, or reads one as a Julian day, breaks here first.', }, + // ── The last supported day: no next day, so no upper bound (#20600) ─────── + { + name: 'datetime: a bare-day $lte on the last supported day has no upper bound', + field: 'at', + kind: 'datetime', + filter: { at: { $lte: '9999-12-31' } }, + expected: ['a_epoch', 'a_old', 'b_prev', 'c_open', 'd_mid', 'e_late', 'f_next', 'g_eom', 'h_leap', 'z_last'], + note: 'Every supported value is at most 9999-12-31T23:59:59.999Z, so the whole-day bound of that day bounds nothing. Its next day spelled as text is the five-digit 10000-01-01, which sorts below 2026-… — a backend that compiled it answered no rows at all.', + }, + { + name: 'datetime: a $between whose max is the last supported day keeps only its min', + field: 'at', + kind: 'datetime', + filter: { at: { $between: ['2026-07-29', '9999-12-31'] } }, + dateRange: ['2026-07-29', '9999-12-31'], + expected: ['f_next', 'g_eom', 'z_last'], + note: 'The min still anchors to midnight (f_next is exactly on it); the max bounds nothing, so z_last (10:00 on the last day) is in.', + }, + { + name: 'datetime: the last supported day as a one-day $between', + field: 'at', + kind: 'datetime', + filter: { at: { $between: ['9999-12-31', '9999-12-31'] } }, + dateRange: ['9999-12-31', '9999-12-31'], + expected: ['z_last'], + note: 'The "today" shape on the last day: from its midnight, with no upper bound.', + }, + { + name: 'datetime: the day before the last day is still a bound', + field: 'at', + kind: 'datetime', + filter: { at: { $lte: '9999-12-30' } }, + expected: ['a_epoch', 'a_old', 'b_prev', 'c_open', 'd_mid', 'e_late', 'f_next', 'g_eom', 'h_leap'], + note: 'The control: 9999-12-30 has a next day, so the bound stops at 9999-12-31T00:00:00.000Z and z_last stays out.', + }, + { + name: 'date: $lte on the last supported day keeps every day', + field: 'on', + kind: 'date', + filter: { on: { $lte: '9999-12-31' } }, + expected: ['a_epoch', 'a_old', 'b_prev', 'c_open', 'd_mid', 'e_late', 'f_next', 'g_eom', 'h_leap', 'z_last'], + note: 'A backend that widens without seeing the column type widens a date column too; on the last day that must still keep every day, 9999-12-31 included.', + }, + // ── Equality on the `date` column — the ADR's original defect (#1874) ──── { name: 'date: equality against a resolved day matches the whole day',