fix(spec,drivers): a datetime $lte or $between maximum on 9999-12-31 includes the whole last supported day (#20600)

.changeset/20600-last-day-unbounded-above.md

@@ -0,0 +1,36 @@
+---
+'@objectstack/spec': minor
+'@objectstack/core': minor
+'@objectstack/driver-sql': patch
+'@objectstack/driver-turso': patch
+'@objectstack/driver-memory': patch
+'@objectstack/driver-mongodb': patch
+'@objectstack/formula': patch
+'@objectstack/objectql': patch
+'@objectstack/service-analytics': patch
+---
+
+fix(spec,drivers): a `datetime` filter `$lte '9999-12-31'`, or a `$between` whose maximum is that day, includes the whole last supported day on every backend (#20600)
+
+Clause-②: yes (widening) — three new exports on `@objectstack/spec` (`data`) and `@objectstack/core`: the constant `UNBOUNDED_ABOVE`, its type `UnboundedAbove` and the guard `isUnboundedAbove`; `nextUtcCalendarDay` answers the constant for one input that used to answer a string. Nothing any door accepted before is refused, and nothing is removed or renamed.
+
+**BREAKING for TypeScript and JavaScript callers of `nextUtcCalendarDay`** (`@objectstack/spec/data`, re-exported by `@objectstack/core`): its return type gains a member and its answer for one input changes from a string to a symbol, landing in the launch window as `minor` (the lockstep convention: the bump level is not the carrier, this banner and the disposition below are). No filter an author writes and no stored row changes meaning except that a whole-day upper bound on `9999-12-31` now includes that day.
+
+`9999-12-31` is the last day of the supported years (0001..9999). A bare-day upper bound on a `datetime` field — `$lte`, a `$between` maximum, an analytics `dateRange` end — means that whole day, and is compiled as "before the next day's midnight". That day has no next day with a `YYYY-MM-DD` spelling: `nextUtcCalendarDay('9999-12-31')` answered the five-digit `'10000-01-01'`, which sorts below `'2026-…'` as text. So on SQLite, where a `datetime` column is ISO text, `$lte '9999-12-31'` and `$between ['2026-01-01', '9999-12-31']` answered no rows; PostgreSQL parsed the bound as an instant and answered them. The memory and mongo drivers, the analytics strategies and the draft preview built their bound from the same answer, and `formula`'s RLS `check` evaluator compared a `'2026-…'` value against it and denied the write.
+
+Every supported value is at most the last millisecond of `9999-12-31`, so that day's whole-day bound bounds nothing. `nextUtcCalendarDay('9999-12-31')` now answers `UNBOUNDED_ABOVE`, a symbol that is neither `null` ("not a calendar day", which would compile the day's midnight and miss the rest of it) nor a string, and every backend compiles no upper bound for it:
+
+- `$lte` / `<=` on that day asks only that the value is not null: `IS NOT NULL` on the SQL drivers and the analytics echo, `$ne: null` on the memory and mongo drivers.
+- A `$between` / `between` whose maximum is that day, and an explicit analytics `dateRange` ending on it, keep only their minimum.
+- The type-blind `formula` `check` evaluator and the draft preview admit every value that denotes an instant, and compare any other value as written.
+- `$gte`, `$gt`, `$lt` and `$eq` on that day are unchanged: they anchor to its midnight, as on every other day. `9999-12-30` and every earlier day compile the same bound as before.
+
+Measured through `POST /api/v1/data/:object/query`, rows at `2026-07-15T14:00Z`, `9999-12-30T10:00Z`, `9999-12-31T00:00Z`, `T10:00Z` and `T23:59:59.999Z`: on SQLite, `$lte '9999-12-31'` and `$between ['2026-01-01', '9999-12-31']` answered none of them and now answer all five; `$between ['9999-12-31', '9999-12-31']` answered none and now answers the three on that day. PostgreSQL 16 answers the same before and after. `$lte '9999-12-30'` answers the first two rows on both, before and after.
+
+**If your code stops compiling.** `nextUtcCalendarDay` now returns `string | UnboundedAbove | null`, where `UnboundedAbove` is a `symbol` with a structural brand. TypeScript refuses that member in a template literal (TS2731), a relational comparison (TS2469) and a `string` parameter (TS2345), so code that used the answer as a day string no longer compiles until it handles the last day. Test the answer with `isUnboundedAbove(answer)` (or `typeof answer === 'symbol'`) first: on its false branch the answer is `string | null` as before, and on its true branch there is no upper bound to compile. `answer === UNBOUNDED_ABOVE` compares correctly but does not narrow, because the branded type is not a unit type. The type is structural on purpose: `@objectstack/spec` ships `./data` as `index.d.mts` and `index.d.ts`, and a `unique symbol` would be two unrelated types in a program that meets both.
+
+**If your JavaScript code handled the answer as text.** For `'9999-12-31'` it is now a registered symbol (`Symbol.for('objectstack.calendarDay.unboundedAbove')`), not `'10000-01-01'`: a template literal or a relational comparison on it throws a `TypeError`, and better-sqlite3 and `pg` refuse to bind it. Every other input answers exactly as before.
+
+The shared temporal conformance kit (`TEMPORAL_ROWS` / `TEMPORAL_CASES` in `@objectstack/spec/data`) gains the row `z_last` (`9999-12-31T10:00:00.000Z`) and five last-day cases, so every backend it drives is held to this answer; three existing `$gte` / `$gt` cases now also expect `z_last`.
+
+<!-- adr-0087: not-required (no-migration-prescription) Nothing an author writes moves — no spec key, no stored row and no accept set changes, so `objectstack migrate meta` has nothing to reach — and what moves is one published function's return type and its answer for one input, whose channel is the caller's compiler and the banner above. -->

packages/core/src/utils/analytics-date-range.test.ts

@@ -316,8 +316,10 @@ describe('#17341 — the END of every calendar window is one CALENDAR day after
                     const where = `${preset} @ ${tz ?? 'UTC'} @ ${now.toISOString()}`;
                     const lastDay = String(resolveFilterToken(prescribedEnd, { now, timezone: tz }));
                     const stopsBefore = nextUtcCalendarDay(lastDay);
-                    expect(stopsBefore, where).not.toBeNull();
-                    const expected = new Date(zonedDateStartToUtcMs(stopsBefore!, tz)).toISOString();
+                    // A day: never `null`, and never `UNBOUNDED_ABOVE` — the
+                    // sweep's instants sit nowhere near 9999-12-31 (#20600).
+                    expect(typeof stopsBefore, where).toBe('string');
+                    const expected = new Date(zonedDateStartToUtcMs(stopsBefore as string, tz)).toISOString();
                     expect(resolveAnalyticsDateRangePreset(preset, { now, timezone: tz }).end, where)
                         .toBe(expected);
                 }

packages/core/src/utils/datetime.ts

@@ -192,8 +192,12 @@ export function zonedWallClockToUtcMs(parts: WallClockParts, tz?: string): numbe
  *
  * Re-exported here so the published `@objectstack/core` surface is unchanged
  * for the drivers and analytics strategies that already import it from here.
+ * [#20600] `UNBOUNDED_ABOVE`, its type `UnboundedAbove` and the guard
+ * `isUnboundedAbove` travel with the helper: the constant is the helper's answer
+ * for `9999-12-31`, and every caller of one narrows it with the guard.
  */
-export { nextUtcCalendarDay, utcInstantMs } from '@objectstack/spec/data';
+export { nextUtcCalendarDay, utcInstantMs, UNBOUNDED_ABOVE, isUnboundedAbove } from '@objectstack/spec/data';
+export type { UnboundedAbove } from '@objectstack/spec/data';
 
 /**
  * Granularity of a canonical date-bucket key. Mirrors `@objectstack/spec`'s

packages/drivers/driver-memory/src/memory-analytics.ts

@@ -11,6 +11,7 @@ import {
   Logger,
   createLogger,
   nextUtcCalendarDay,
+  isUnboundedAbove,
   // [#16322] The ONE lowering of the closed `dateRange` preset vocabulary and
   // the ONE refusal for a string outside it, shared with the SQL analytics
   // path so the two backends cannot answer one input differently again.
@@ -266,8 +267,11 @@ const CUBE_OPERATOR_TO_MONGO_PREDICATE: Readonly<Record<CubeOperator, MongoPredi
   // A bare-day `lte` bound means "through that whole day" (#4042; the SQL twin
   // is #3777): compile half-open so timestamp values on the final day stay in.
   // Order-equivalent to `$lte` for plain `YYYY-MM-DD` values.
+  // [#20600] On the last supported day there is no next day: every value is
+  // inside the bound, so what `lte` still asks is a value (the `set` row below).
   lte: ({ comparands }) => {
     const nextDay = nextUtcCalendarDay(comparands[0]);
+    if (isUnboundedAbove(nextDay)) return { $ne: null };
     return nextDay != null ? { $lt: nextDay } : { $lte: comparands[0] };
   },
   // The list operators take the WHOLE list. An empty one is a real predicate —
@@ -458,8 +462,11 @@ const CUBE_OPERATOR_TO_SQL_PREDICATE: Readonly<Record<CubeOperator, SqlPredicate
   // Half-open on a bare-day bound, exactly as the mingo row above is (#4042; the
   // SQL twin is #3777). `<= '2026-01-02'` drops that day's timestamped rows,
   // which is measurable as an echo one row NARROWER than the chart it describes.
+  // [#20600] …and `IS NOT NULL` on the last supported day, as the mingo row
+  // above answers `$ne: null` there.
   lte: ({ column, comparands, literal }) => {
     const nextDay = nextUtcCalendarDay(comparands[0]);
+    if (isUnboundedAbove(nextDay)) return `${column} IS NOT NULL`;
     return nextDay != null
       ? `${column} < ${literal(nextDay)}`
       : `${column} <= ${literal(comparands[0])}`;
@@ -991,17 +998,27 @@ export class MemoryAnalyticsService implements IAnalyticsService {
           //
           // Anything else -- a full timestamp the CALLER wrote -- keeps
           // instant semantics and stays INCLUSIVE, byte for byte as before.
-          const widenedDay = resolved.endExclusive ? null : nextUtcCalendarDay(end);
+          //
+          // [#20600] A caller's bare end on the last supported day has no next
+          // day to stop before: every value is inside it, so the window keeps
+          // its start alone, in both spellings.
+          const widened = resolved.endExclusive ? null : nextUtcCalendarDay(end);
+          const unbounded = isUnboundedAbove(widened);
+          const widenedDay = isUnboundedAbove(widened) ? null : widened;
           const upperString = resolved.endExclusive ? end : widenedDay;
           const upperDate = widenedDay != null
             ? new Date(`${widenedDay}T00:00:00.000Z`)
             : (resolved.endExclusive ? new Date(end) : null);
-          const stringBounds = upperString != null
-            ? { $gte: start, $lt: upperString }
-            : { $gte: start, $lte: end };
-          const dateBounds = upperDate != null
-            ? { $gte: new Date(start), $lt: upperDate }
-            : { $gte: new Date(start), $lte: new Date(end) };
+          const stringBounds = unbounded
+            ? { $gte: start }
+            : upperString != null
+              ? { $gte: start, $lt: upperString }
+              : { $gte: start, $lte: end };
+          const dateBounds = unbounded
+            ? { $gte: new Date(start) }
+            : upperDate != null
+              ? { $gte: new Date(start), $lt: upperDate }
+              : { $gte: new Date(start), $lte: new Date(end) };
           pipeline.push({
             $match: {
               $or: [

packages/drivers/driver-memory/src/memory-driver-calendar-day-upper-bound.test.ts

@@ -17,7 +17,7 @@
  */
 
 import { describe, it, expect, beforeEach } from 'vitest';
-import { parseFilterAST } from '@objectstack/spec/data';
+import { parseFilterAST, type FilterCondition } from '@objectstack/spec/data';
 import { InMemoryDriver } from './memory-driver.js';
 import { MemoryAnalyticsService } from './memory-analytics.js';
 import type { Cube } from '@objectstack/spec/data';
@@ -153,3 +153,103 @@ describe('MemoryAnalyticsService — dateRange window (#4042)', () => {
     expect(result.rows[0]?.count).toBe(4);
   });
 });
+
+/**
+ * [#20600] `9999-12-31`, the last supported day, has no next day: every
+ * supported value is inside its whole-day bound, so the whole-day widening
+ * compiles NO upper bound — `$lte` / `<=` ask only for a value (`$ne: null`), a
+ * `$between` / `between` / `dateRange` keeps its minimum. The spec's helper
+ * used to answer the five-digit `'10000-01-01'`, and every ISO-string row
+ * sorts above it, so each of those answered no rows. `9999-12-30` is the
+ * control: an ordinary bound.
+ */
+describe('[#20600] InMemoryDriver — a bare-day upper bound on the last supported day', () => {
+  let driver: InMemoryDriver;
+
+  beforeEach(async () => {
+    driver = new InMemoryDriver({
+      initialData: {
+        task: [
+          { id: 'c26', title: 'c26', created_at: '2026-07-15T14:00:00.000Z' },
+          { id: 'prev', title: 'prev', created_at: '9999-12-30T10:00:00.000Z' },
+          { id: 'open', title: 'open', created_at: '9999-12-31T00:00:00.000Z' },
+          { id: 'mid', title: 'mid', created_at: '9999-12-31T10:00:00.000Z' },
+          { id: 'last', title: 'last', created_at: '9999-12-31T23:59:59.999Z' },
+          { id: 'none', title: 'none', created_at: null },
+        ],
+      },
+    });
+    await driver.connect();
+  });
+
+  /** `where` · the ids it answers, sorted. */
+  const CASES: ReadonlyArray<readonly [string, () => FilterCondition | undefined, readonly string[]]> = [
+    ["$lte '9999-12-31'", () => ({ created_at: { $lte: '9999-12-31' } }), ['c26', 'last', 'mid', 'open', 'prev']],
+    ["$between ['2026-01-01', '9999-12-31']", () => ({ created_at: { $between: ['2026-01-01', '9999-12-31'] } }), ['c26', 'last', 'mid', 'open', 'prev']],
+    ["$between ['9999-12-31', '9999-12-31']", () => ({ created_at: { $between: ['9999-12-31', '9999-12-31'] } }), ['last', 'mid', 'open']],
+    ["$not $lte '9999-12-31'", () => ({ $not: { created_at: { $lte: '9999-12-31' } } }), ['none']],
+    // The lowered `$ne` shares a key an author can write: both constraints survive (#13524).
+    ["$lte '9999-12-31' beside an author's $ne", () => ({ created_at: { $lte: '9999-12-31', $ne: '9999-12-31T10:00:00.000Z' } }), ['c26', 'last', 'open', 'prev']],
+    ["the AST spelling <= '9999-12-31'", () => parseFilterAST([['created_at', '<=', '9999-12-31']]), ['c26', 'last', 'mid', 'open', 'prev']],
+    ["the AST spelling between max '9999-12-31'", () => parseFilterAST([['created_at', 'between', ['9999-12-31', '9999-12-31']]]), ['last', 'mid', 'open']],
+    ["$lte '9999-12-30' (control)", () => ({ created_at: { $lte: '9999-12-30' } }), ['c26', 'prev']],
+    ["$between ['2026-01-01', '9999-12-30'] (control)", () => ({ created_at: { $between: ['2026-01-01', '9999-12-30'] } }), ['c26', 'prev']],
+    ["$gte '9999-12-31' (unchanged)", () => ({ created_at: { $gte: '9999-12-31' } }), ['last', 'mid', 'open']],
+    ["$lt '9999-12-31' (unchanged)", () => ({ created_at: { $lt: '9999-12-31' } }), ['c26', 'prev']],
+  ];
+
+  it('compiles no upper bound on 9999-12-31; 9999-12-30 is a bound; the lower-bound operators do not move', async () => {
+    const got: Record<string, string[]> = {};
+    for (const [name, where] of CASES) got[name] = ids(await driver.find('task', { where: where() }));
+    expect(got).toEqual(Object.fromEntries(CASES.map(([name, , want]) => [name, want])));
+  });
+
+  it('the analytics face: a dateRange ending 9999-12-31 keeps its start alone, for BOTH stored forms', async () => {
+    const analytics = new InMemoryDriver({
+      initialData: {
+        task: [
+          { id: 's_before', created_at: '2026-06-30T23:00:00.000Z' },
+          { id: 's_in', created_at: '2026-07-10T08:00:00.000Z' },
+          { id: 's_last_day', created_at: '9999-12-31T21:40:00.000Z' },
+          { id: 'd_in', created_at: new Date('2026-07-10T12:00:00Z') },
+          { id: 'd_last_day', created_at: new Date('9999-12-31T09:15:00Z') },
+          { id: 'd_before', created_at: new Date('2026-04-19T10:00:00Z') },
+        ],
+      },
+    });
+    await analytics.connect();
+    const service = new MemoryAnalyticsService({
+      driver: analytics,
+      cubes: [{
+        name: 'tasks', title: 'Tasks', sql: 'task',
+        measures: { count: { label: 'Count', type: 'count', sql: 'id' } },
+        dimensions: { created_at: { label: 'Created', type: 'time', sql: 'created_at' } },
+      } as unknown as Cube],
+    });
+    const count = async (dateRange: [string, string]) =>
+      (await service.query({ cube: 'tasks', measures: ['count'], timeDimensions: [{ dimension: 'created_at', dateRange }] } as any)).rows[0]?.count;
+
+    expect(await count(['2026-07-01', '9999-12-31'])).toBe(4); // s_in, s_last_day, d_in, d_last_day
+    expect(await count(['2026-07-01', '9999-12-30'])).toBe(2); // the control: the last day is out
+  });
+
+  it('the analytics face: a cube-filter lte on 9999-12-31 asks only for a value, in the pipeline and the echo', async () => {
+    const service = new MemoryAnalyticsService({
+      driver,
+      cubes: [{
+        name: 'tasks', title: 'Tasks', sql: 'task',
+        measures: { count: { label: 'Count', type: 'count', sql: 'id' } },
+        dimensions: {
+          id: { label: 'Id', type: 'string', sql: 'id' },
+          created_at: { label: 'Created', type: 'time', sql: 'created_at' },
+        },
+      } as unknown as Cube],
+    });
+    const q = (day: string) => ({ cube: 'tasks', measures: ['count'], dimensions: ['id'], where: { created_at: { $lte: day } } } as any);
+
+    expect(ids((await service.query(q('9999-12-31'))).rows)).toEqual(['c26', 'last', 'mid', 'open', 'prev']);
+    expect(ids((await service.query(q('9999-12-30'))).rows)).toEqual(['c26', 'prev']);
+    expect((await service.generateSql(q('9999-12-31'))).sql).toContain('created_at IS NOT NULL');
+    expect((await service.generateSql(q('9999-12-30'))).sql).toContain("created_at < '9999-12-31'");
+  });
+});