Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .changeset/20263-having-temporal-comparand-door.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
---
"@objectstack/objectql": minor
---
Expand Down Expand Up @@ -28,14 +28,14 @@

- The same walk and the same predicate, `isUninterpretableTemporalComparand` in `@objectstack/core`, that the door runs on `where` and on each per-aggregation `filter`. A change to that rule reaches `having` with it.
- The kind is the aggregated column's class, the one the `addDays` rule already reads: `min` / `max` of a `date`, `datetime` or `time` field keeps that kind, a groupBy projection of such a field takes its kind, and a `day` bucket is a `date`. `count`, `count_distinct`, `sum` and `avg`, a `week` / `month` / `quarter` / `year` bucket, and every other column are not temporal, so they are not judged.
- Every comparison and set operator's comparand, each `$in` / `$nin` member and `$between` endpoint, and the implicit-equality slot, under `$and`, `$or` and `$not`. As on `where`, a `{placeholder}` string, the empty string, `null` and a `{ $field }` reference are not judged. `having` does not resolve placeholders, and did not before, so the refusal's remedy names none.
- Every comparison and set operator's comparand, each `$in` / `$nin` member and `$between` endpoint, and the implicit-equality slot, under `$and`, `$or` and `$not`. As on `where`, a `{placeholder}` string, the empty string, `null` and a `{ $field }` reference are not judged. `having` resolves placeholders from the same release (#20334), after this door, so the refusal's remedy on a `date` or `datetime` column is the `where` refusal's and names them, e.g. `{30_days_ago}` / `{current_month_start}`.
- The text operators (`$contains`, `$notContains`, `$startsWith`, `$endsWith`, `$icontains`) are not judged. On `where` the text-operator declared-type door answers them first, and that door does not front `having`.
- The door runs after every other `having` door, so a clause one of them refuses (an unknown operator, a key naming no column, a comparand of no comparable type, an `addDays` pair, an array in the equality slot) keeps that refusal and its words.

The refusal follows the `where` door's words. It names the `having` path, the column, what the column aggregates and its kind, and the comparand, for example: `` `having` on 'last_placed' (max(placed_on), a date column) compares against "not-a-date" at having.last_placed.$lt ``. Like the `where` refusal, it names the column's kind, and otherwise only what the query carries.

**Who is affected.** `having` is a request-only key (`QuerySchema.having`, `EngineAggregateOptions.having`), and no metadata type stores it. Every `having` in this repository's docs and published skills compares a numeric aggregation alias, which is not judged. Callers of `engine.aggregate` and of the REST aggregate query in a deployment were NOT measured.

**Fix.** Compare a `date` column with a `YYYY-MM-DD` day, a `datetime` column with an ISO-8601 instant, a bare day or epoch milliseconds, and a `time` column with an `HH:MM` or `HH:MM:SS` wall clock.
**Fix.** Compare a `date` column with a `YYYY-MM-DD` day, a `datetime` column with an ISO-8601 instant, a bare day or epoch milliseconds, either one with a relative-date placeholder the resolver knows (`{30_days_ago}`, `{current_month_start}`; `having` resolves them from the same release, #20334), and a `time` column with an `HH:MM` or `HH:MM:SS` wall clock.

**Unchanged**, measured identical before and after on the three drivers, both paths and both doors: every `where` and per-aggregation `filter` answer; every `having` on a temporal column whose comparand the rule reads (a `YYYY-MM-DD` day, an ISO instant, an epoch-millisecond number or string, an in-range `Date`, a zone-naive instant, a wall clock, an extended-year instant on a `datetime` column, which that rule reads); `{today}`-style placeholders, known or not; the empty and the whitespace-only string; `null`, `$exists`, `$in` / `$nin`, `$between`, `$not` / `$or` / `$and` and `{ $field }` references; `$contains` and `$startsWith`; every `count` / `sum` / `avg` column, a string comparand included; a `month` bucket; and every existing `having` refusal, in its words.
33 changes: 33 additions & 0 deletions .changeset/20334-aggregate-positions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
"@objectstack/objectql": minor
---

fix(objectql)!: `having` on `engine.aggregate` resolves `{placeholder}` tokens through the resolver `where` uses, so an unknown one is refused `FILTER_TOKEN_UNKNOWN` / 400 instead of keeping no group with a 200; and the per-aggregation `filter`'s temporal and text-operator refusals name `aggregations[i].filter` instead of `where` (#20334)

Clause-②: no (narrowing)

<!-- adr-0087: not-required (no-migration-prescription) this change adds no transition to migrate. A refused `having` comparand is a `{placeholder}` the resolver cannot resolve: a token outside the vocabulary, which `having` compared as its own literal text, or a context token the request carries no value for. Neither was ever an accepted spelling with a meaning to preserve, and there is no spelling it can be mechanically rewritten to: which token or which literal the author meant is an authoring decision, and the refusal lists the resolvable tokens. `having` is a request-only key that no metadata type stores, so there is no stored document for `objectstack migrate meta` to rewrite. The tables below record the answer each input had and has; they prescribe no rewrite. -->

**BREAKING**: this narrows what `having` accepts on `engine.aggregate`, and on the REST aggregate query (`POST /data/:object/query`) that forwards it there. A `{placeholder}` in `having` is now resolved by the same resolver, with the same refusals, as one in `where`, once per query, before any driver is asked for a row, on both the native `driver.aggregate()` path and the in-memory fallback. It ships as `minor` under the launch-window convention for accept-set narrowings.

Measured before and after through `engine.aggregate` and `POST /data/:object/query`, on InMemoryDriver, SqlDriver on SQLite and SqlDriver on PostgreSQL 16, on both `having` paths, four groups whose `max(placed_on)` falls between 2026-01-02 and 2026-03-01:

| `having` | before | now | the same token in `where` |
|:--|:--|:--|:--|
| an unknown token: `{ last_placed: { $gte: '{not_a_token}' } }`, or a near miss such as `'{TODAY}'`, on any column, under `$and` / `$or` / `$not` or as an `$in` member | 200, keeps no group | `FILTER_TOKEN_UNKNOWN` / 400, no read | the same refusal, in the same words |
| `'{current_user_id}'` with no user on the request, `'{current_org_id}'` with no active organization, `'{record_id}'` always | 200, keeps no group | `FILTER_TOKEN_UNRESOLVED` / 400, no read (a REST request with no user is answered 401 before it reaches the engine, as before) | the same refusal |
| a known token: `{ last_placed: { $gt: '{current_year_start}' } }` on `max(placed_on)` | compared as its own text, which sorts after every digit: keeps no group | compares as `'2026-01-01'`: keeps all four | resolved |
| any known token, as a comparand, an `$in` member or a `$between` endpoint, under `$and` / `$or` / `$not` | compared as its own text (on a date or datetime column, `$gt` / `$gte` kept no group and `$lt` / `$lte` every group) | the groups the resolved value keeps, the same as that value written out | resolved |
| `{ customer_id: '{current_user_id}' }` on a groupBy key, as user `c2` | keeps no group | keeps `c2` | resolved |

Tokens resolve the way they do in `where`: a date macro to a `YYYY-MM-DD` day (a sub-day macro to an ISO instant) in the request's timezone, `{current_user_id}` and `{current_org_id}` from the request. A string that only contains braces (`'a{b}c'`) is not a placeholder and compares as written, as in `where`. The `having` doors run first, as `where`'s do: a clause an earlier `having` door refuses (an unknown key, an unknown operator, a comparand its temporal column cannot read) keeps that refusal, in that door's words.

**The per-aggregation `filter`'s refusals name their position.** A comparand a declared temporal field cannot read, and a text operator aimed at a field that never holds a string, in `aggregations[1].filter` said `at where.placed_on.$gt` / `at where.amount.$contains`, a `where` the author did not write. They now say `at aggregations[1].filter.placed_on.$gt` / `at aggregations[1].filter.amount.$contains`, as that filter's list-shape and comparand-type refusals already did. The code (`INVALID_FILTER`), the status (400) and every other word are unchanged at the engine. Over REST, the message keeps its existing 500-character bound: a `date` field's refusal of a string such as `'not-a-date'`, which fitted within it, now loses the end of its remedy (`"{current_month_s…` at the shortest path), and the refusals that already exceeded the bound still do.

**The `having` temporal refusal's remedy is `where`'s.** A string a `date` or `datetime` aggregated column cannot read (`{ last_placed: { $lt: 'last_30_days' } }` on `max(placed_on)`) was refused with a remedy that named the literal forms only (`Write a "YYYY-MM-DD" calendar day.` on a `date` column), written when `having` resolved no placeholder. Now that it resolves them, the refusal ends in the remedy the same comparand gets in `where`, which names the placeholder: `Write a "YYYY-MM-DD" calendar day, or a relative-date placeholder the resolver knows, e.g. "{30_days_ago}" / "{current_month_start}".` on a `date` column, and the `where` remedy for a `datetime` field on a `datetime` column. The code (`INVALID_FILTER`), the status (400), what is refused and every word before the remedy are unchanged, and so are a `time` column's refusal and the refusal of a number or `Date` whose year falls outside 0000 to 9999. Over REST the message keeps its 500-character bound, which the longer remedy now reaches, measured with an object named `ledger_having`. A `date` column's refusal still arrives whole in every cell measured, the longest at 498 characters (`'+010000-01-01T00:00:00.000Z'` on `max(placed_on)`), and it stays whole while the object name, the column, what it aggregates, the comparand as quoted and its path take at most 90 characters together. A `datetime` column's refusal no longer fits whatever those are, because its fixed words and remedy alone take 502 characters: over REST it now ends inside the remedy, before the placeholder it names (`…epoch milliseconds, or a relative-date pl…` on `min(opened_at)`), as the `where` refusal for a `datetime` field already did. A preset name such as `'last_30_days'` does not reach this refusal over REST: the query schema refuses it first (`VALIDATION_FAILED`), before and after.

**Who is affected.** `having` is a request-only key (`QuerySchema.having`, `EngineAggregateOptions.having`), and no metadata type stores it. The five `having` clauses in this repository's docs and published skills compare a numeric aggregation alias with a number, and none carries a placeholder; no runtime code or example app in this repository composes a `having`. Callers of `engine.aggregate` and of the REST aggregate query in a deployment were NOT measured.

**Fix.** For an unknown token, write one the resolver knows (the refusal lists them: `{today}`, `{current_quarter_start}`, `{30_days_ago}`, `{current_user_id}`, …) or the literal value. For `{current_user_id}` / `{current_org_id}`, send the request with a user or an active organization.

**Unchanged**, measured identical before and after on the three drivers, both paths and both doors: every `where` answer, placeholders and refusals included; every `having` that carries no placeholder, and its refusals in their words other than the `date` / `datetime` temporal refusal's remedy above; every per-aggregation `filter` answer other than the two refusals' paths above, placeholders included.
Original file line number Diff line number Diff line change
Expand Up @@ -247,9 +247,32 @@ describe('[#20263] having — a comparand its column cannot read is refused befo
await expectHavingRefusal(() => ({ d: { $gt: Y10000 } }), [{ field: 'opened_at', dateGranularity: 'day', alias: 'd' }]);
});

it('the remedy names no placeholder: having resolves none, so it would send the author to a literal', async () => {
for (const having of [{ last_placed: { $lt: 'x' } }, { first_opened: { $lt: 'x' } }]) {
expect(await expectHavingRefusal(() => having)).not.toContain('{30_days_ago}');
// [#20334] `having` resolves placeholders through `where`'s resolver, so its
// remedy is `where`'s: it names the relative-date placeholder that a caller
// holding a preset name (`last_30_days`) needs, on the two kinds that take one.
it('the remedy names the placeholder the resolver knows, in the where twin\'s words', async () => {
const CASES: ReadonlyArray<readonly [Record<string, unknown>, Record<string, unknown>, string]> = [
[{ last_placed: { $lt: 'last_30_days' } }, { placed_on: { $lt: 'last_30_days' } },
"`having` on 'last_placed' (max(placed_on), a date column) compares against \"last_30_days\" at "
+ 'having.last_placed.$lt, which is not a date value this platform can interpret.'],
[{ first_opened: { $lt: 'last_30_days' } }, { opened_at: { $lt: 'last_30_days' } },
"`having` on 'first_opened' (min(opened_at), a datetime column) compares against \"last_30_days\" at "
+ 'having.first_opened.$lt, which is not a datetime value this platform can interpret.'],
];
const after = (message: string, marker: string): string => {
expect(message).toContain(marker);
return message.slice(message.indexOf(marker) + marker.length);
};
for (const [having, where, firstSentence] of CASES) {
// INVALID_FILTER / 400 on both paths, empty or populated, no read.
const message = await expectHavingRefusal(() => having);
expect(message.startsWith(`aggregate('${OBJECT}'): ${firstSentence} `)).toBe(true);
const remedy = after(message, 'The `having` was NOT applied. ');
expect(remedy).toContain('"{30_days_ago}"');
const twin = await whereTwinOf(where);
expect(twin.err?.code).toBe('INVALID_FILTER');
expect(twin.err?.status).toBe(400);
expect(remedy).toBe(after(twin.err!.message, 'The filter was NOT applied. '));
}
});
});
Expand Down Expand Up @@ -306,7 +329,6 @@ describe('[#20263] having — what the door leaves alone answers exactly as befo
['a string on count — not temporal', { n: { $gt: 'not-a-date' } }, []],
['a string on avg — not temporal', { mean: { $gt: 'not-a-date' } }, []],
['a {placeholder} is stepped around, as on where', { last_placed: { $lte: '{today}' } }, ['c1', 'c2', 'c3', 'c4']],
['an unknown {placeholder} too', { last_placed: { $gte: '{not_a_token}' } }, []],
['the empty string (its own card)', { last_placed: { $gt: '' } }, ['c1', 'c2', 'c3', 'c4']],
['null in the equality slot', { last_placed: null }, []],
['$exists', { last_placed: { $exists: true } }, ['c1', 'c2', 'c3', 'c4']],
Expand All @@ -325,6 +347,19 @@ describe('[#20263] having — what the door leaves alone answers exactly as befo
});
}

// [#20334] An unknown one is stepped around by this door too, and is then
// refused one layer down by the token resolver, in its own code, as on
// `where` (it kept no group with a 200 before `having` resolved tokens).
it('an unknown {placeholder} is not this door\'s verdict: FILTER_TOKEN_UNKNOWN from the resolver, before any read', async () => {
for (const path of ['native', 'rows'] as const) {
const { engine, reads } = await makeEngine(path, ROWS);
const { err } = await outcome(() => engine.aggregate(OBJECT, query(path, { last_placed: { $gte: '{not_a_token}' } })));
expect(err?.code, path).toBe('FILTER_TOKEN_UNKNOWN');
expect(err?.status, path).toBe(400);
expect(reads, path).toEqual({ aggregate: 0, find: 0 });
}
});

it('a coarser bucket is a text label, and is not judged', async () => {
const kept = await keptGroups(
{ m: { $gt: 'not-a-date' } },
Expand Down
Loading
Loading