Skip to content

fix(objectql)!: a groupBy on a structured-JSON field is refused INVALID_FIELD / 400 at the engine aggregate door, on every driver (#20783) - #20804

Merged
objectstack-fleet[bot] merged 6 commits into
mainfrom
claude/issue-20783-groupby-json-refused
Sep 30, 2026
Merged

objectstack-fleet[bot] merged 6 commits into
mainfrom
claude/issue-20783-groupby-json-refused

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #20783
Clause-②: no (narrowing)

What this changes

A groupBy entry that names a declared structured-JSON field (json, composite, repeater, record, location, address, vector) is now refused by engine.aggregate with INVALID_FIELD / 400, in the engine's words, before any driver is asked. It holds on every driver and for every caller that reaches the engine: the REST query door, a flow or hook calling the engine in-process, and the analytics strategy that lowers a cube query onto engine.aggregate. Both entry spellings are judged: the field name (groupBy[0]) and the { field } object (groupBy[0].field), a dateGranularity bucket included.

The words, as POST /api/v1/data/:object/query returns them (the route is inside the 500 characters the REST door keeps, and the REST pin asserts it there):

aggregate('rest_group_by_json_20783'): groupBy[0] names 'meta', a declared json field — a structured-JSON value, which the engine does not group by. The query was NOT run. Group by a field that stores one scalar value: store the part you group on in a field of its own and group by that field. A JSON document is no group key the drivers share: one merged every row into a single group, one grouped each serialized document apart, one refused the statement.

The thrown error carries code: 'INVALID_FIELD', status and httpStatus 400, field, fields (every offending entry), object and param: 'groupBy'.

Landing site, as the order expected: packages/objectql's aggregate admission. No driver file, no serialization rule.

  • packages/objectql/src/group-by-structured-json-door.ts (new): assertGroupByNamesNoStructuredJsonField(object, schema, groupBy). The class is the spec's STRUCTURED_JSON_TYPES (the set PR fix(objectql)!: a no-operator object beneath a relation, structured-JSON or undeclared id column is refused INVALID_FILTER / 400 on every driver (#20745) #20781's JSON arm judges), never a list minted here. Not judged: an undeclared name (the engine's registry-less tolerance; the REST ingress answers an unknown name INVALID_FIELD first), a host with no field map, and every other type.
  • packages/objectql/src/engine.ts: one call at the entry of aggregate, right after rejectCredentialAggregation (which reads the same groupBy entries), so a protected field keeps that refusal's words. aggregate is the only engine verb that takes groupBy: find refuses the key (ENGINE_FIND_OPTION_KEYS).
  • packages/objectql/src/number-comparand-declared-type-door.ts: comment only. Its sentence "a json … groupBy … is judged now" at having became false for a json groupBy, which no longer reaches having.

Why INVALID_FIELD, an existing code. The verdict is about the named field's type at a position. That is the question the REST ingress answers with INVALID_FIELD for an unknown groupBy name (assertGroupByFieldsExist), the search axis answers with INVALID_FIELD for a field whose type it cannot scan, and the engine's assertFilterIsMaterializable answers with INVALID_FIELD for a virtual field ("this verdict is about the NAME's type"). INVALID_FILTER is the engine's value-shape envelope and groupBy is not a filter; INVALID_QUERY is the ingress's malformed-shape code, and the entry here is well formed.

Before, measured on origin/main 7a09eee1b1

Through POST /api/v1/data/:object/query (the real RestServer route over ObjectStackProtocolImplementation and ObjectQL) with { groupBy: [FIELD], aggregations: [{ function: 'count', alias: 'n' }] }, and through engine.aggregate directly (same answers). Drivers: InMemoryDriver, SqlDriver on SQLite (better-sqlite3), and SqlDriver on a private PostgreSQL 16.13 started for this run. Three rows: title x, x, y; meta {a:1}, {a:2}, {b:1}; and one differing value per row under every other structured-JSON field.

groupBy InMemoryDriver SQLite PostgreSQL 16
title (text, the control) 200, x 2 · y 1 same same
meta (json, the card) 200, one group {a:1}, n 3 200, one group per serialized document (3) 500 DATABASE_ERROR ("could not identify an equality operator for type json")
composite, repeater, record, location, address 200, one group, n 3 200, one group per serialized document 500
vector 200, one group per array ([1,2] 2 · [3,4] 1) 200, one group per serialized array 500
{ field: 'meta' } 200, one group, n 3 200, 3 groups 500
{ field: 'meta', dateGranularity: 'month' } 200, one null bucket, n 3 200, one null bucket, n 3 500 ("cannot cast type json to timestamp with time zone")
['title', 'meta'] 200, 2 groups 200, 3 groups 500

After, the same run on this branch (43ae6c1fcc)

Every structured-JSON row above answers 400 INVALID_FIELD in the engine's words on all three drivers, naming the position (groupBy[0], groupBy[0].field, groupBy[1] for the mixed entry), the field and its declared type. No read of the object runs. The title control answers x 2 · y 1 on all three, unchanged. The InMemoryDriver cells of this run came from a scratch script over the built packages. They are not committed: check:driver-memory-census refuses a new test consumer of that driver without a ruling, so the committed memory cell is the recording driver below, by construction.

Hypotheses (zone 2): which held

  • H1: held. aggregate resolves groupBy entries against the declared field map before the driver in rejectCredentialAggregation (credential and internal fields) and, for having, in aggregatedRowColumnTypes. The refusal sits beside the first, at the verb's entry, so it runs before the per-aggregation filter and having doors and before any driver is resolved. Code: INVALID_FIELD, reasons above. The REST ingress also resolves groupBy names (assertGroupByFieldsExist in metadata-protocol), but only for the REST path. The engine is the one door every caller shares, as triage directed.

  • H2: held, refined. Every member of STRUCTURED_JSON_TYPES answers per driver today, and none answers one way. Six members split exactly like json (memory one merged group, SQLite per serialized document, PostgreSQL 500). vector splits differently on memory (one group per array, not one merged group) and still 500 on PostgreSQL, so it does not answer one way either. So the class is refused, not json alone.

  • H3: held, and it was NOT already refused. A date-bucketed { field, dateGranularity } over a json field passed the REST ingress (a known field, a valid granularity) and answered one null bucket on memory and SQLite and 500 on PostgreSQL. It is refused now with the { field } form's words, since no granularity makes a JSON document a date.

  • H4: measured. The analytics face is partly the same door. Measured through AnalyticsService.query wired with AnalyticsServicePlugin's own auto-bridges (executeAggregate to engine.aggregate, executeRawSql to engine.execute), with a cube dimension sql: 'meta' on a json field:

    cube query InMemoryDriver SQLite PostgreSQL 16
    dimensions: [meta], before 200, one merged group (raw SQL unsupported, fell back to engine.aggregate) 200, one group per serialized document (native SQL, engine not reached) 500 DATABASE_ERROR (native SQL)
    dimensions: [meta], after 400 INVALID_FIELD (this door) unchanged unchanged
    timeDimensions: [{ meta, granularity: month }], before 200, one null bucket 200, one null bucket 500
    the same, after 400 INVALID_FIELD (the native strategy declines a granularity, so engine.aggregate serves it) 400 400

    So the ObjectQL strategy (any cube query on a driver without raw SQL, and any bucketed time dimension) reaches this door and is folded in with no packages/services edit. NativeSQLStrategy does not: on SQL drivers it compiles GROUP BY "meta" by hand and bypasses the engine. That half is a sibling card (reported to the PM below, not filed here). The dataset compiler only checks that a dimension's field is declared (assertDeclared), so a dataset dimension on a json field compiles into the same cube dimension and splits the same way. The memory cube face (MemoryAnalyticsService) has zero production constructors (git grep "new MemoryAnalyticsService" outside tests: 0 hits at 7a09eee1b1), so it reaches a driver only in tests. It was not edited and not measured for this shape.

Producers measured (for "refuse, don't define")

Zero producers group by a structured-JSON field. Every grouping / groupBy / groupByField / dimensions target in examples/ at 7a09eee1b1 is one of status, priority, created_at, account, stage, total, region, sales_region, progress, issued_on, industry, category, signed_on, health, completed_date, close_date, and none is structured-JSON (the showcase's structured-JSON fields are field-zoo's f_*, account.hq, account.support_config and task.location). The same count over the platform packages names object_name, user_id, id, action, topic, provider_id, organization_id, namespace, kind, actor_id and phone, plus two dynamic engine callers in service-analytics (see the acceptance notes).

Tests

  • New packages/objectql/src/engine-group-by-json-door.test.ts (6 tests, recording driver, so the in-memory cell by construction). It covers every structured-JSON type with the full envelope (code, status, httpStatus, field, fields, object, param) and the position, and asserts zero reads. It covers the { field } form, a date bucket, an entry after a scalar one and two offenders, and the REST door into findData. Controls: text, number, a multiple: true select, an image field and an undeclared name all reach the driver, and a structured-JSON field as an aggregated column is unchanged. GUARDs: the judged types equal STRUCTURED_JSON_TYPES over every FieldType, and there is no verdict without a field map, for an undeclared name, or for an entry naming no field.
  • New packages/rest/src/data-group-by-json-door.test.ts: SQLite always, PostgreSQL / MySQL where OS_TEST_POSTGRES_URL / OS_TEST_MYSQL_URL are set. Every structured-JSON type answers 400 INVALID_FIELD with the route in the REST body and zero reads. The object and bucket forms and the mixed entry answer the same 400. The control title answers x 2 · y 1 from the driver. ⚠️ As with the sibling door suites, no CI job sets those URLs for @objectstack/rest, so the live cells run only locally. Local run with the private PostgreSQL 16.13: 6 passed (sqlite 3, live postgres 3) / 3 skipped (mysql, no URL).
  • Fixture triage in engine-nested-object-door.test.ts (PR fix(objectql)!: a no-operator object beneath a relation, structured-JSON or undeclared id column is refused INVALID_FILTER / 400 on every driver (#20745) #20781's pin): its having case over groupBy: ['meta'] pinned the branch this PR closes, because a json groupBy never reaches having now. It is replaced, not respelled. The JSON arm at having is still reached through a max of a json field (having: { top_meta: { a: 1 } }), which answers the same INVALID_FILTER whole-value words. No other fixture groups by a structured-JSON field. Measured by a grep over every test that both groups and declares a structured-JSON type; the downstream suites below stayed green.
  • pnpm --filter @objectstack/objectql test at 43ae6c1fcc: 342 files / 6739 passed.
  • pnpm --filter @objectstack/rest test at 43ae6c1fcc (with the live PostgreSQL URL set): 232 files / 4516 passed / 33 skipped.
  • typecheck for @objectstack/objectql and @objectstack/rest at 43ae6c1fcc: exit 0, including each check:test-typecheck (objectql's ledger held at 40 files / 234 errors / 65 signatures; rest 0), so both new test files compile.
  • Downstream consumers (...@objectstack/objectql direction), at cafaf885d8: @objectstack/service-analytics 141 files / 3267 passed; @objectstack/metadata-protocol 190 files passed, 3 skipped / 2792 passed, 19 skipped. The other consumers are declared to CI.

Reverse verification (ablation), from the committed fix at 1eacad5296. It ran through scripts/ablation-replace.mjs in WRAP mode, trap-restored. The engine's call assertGroupByNamesNoStructuredJsonField(object, this._registry.getObject(object), query.groupBy); was fed (query as { __ablated_20783__?: unknown }).__ablated_20783__ (always undefined) instead of query.groupBy. On disk the anchor went 1 to 0 and the marker 0 to 1, with blob 67198fc4a8fc to 837fc9257e37. objectql was rebuilt, and ablation-dist-preflight found the marker in 4 built files.

  • Predicted direction: red. Observed: red.
  • objectql pins: 3 failed / 18 passed. Every refusal case failed. The controls, both GUARDs and the whole engine-nested-object-door.test.ts stayed green.
  • rest pins: 4 failed / 2 passed / 3 skipped. The refusal cases failed on SQLite and live PostgreSQL, and the title control stayed green.
  • Restore leg: blob equals HEAD (67198fc4a8fc), git diff HEAD is empty, and whole-tree git status --porcelain is empty. After a rebuild, the --absent preflight found the marker absent from all 14 built files, and the tree reading was clean. Then both pin sets were green again (21 passed; 6 passed + 3 skipped).

Gates

node scripts/pm/dispatch-gates.mjs --commands at 43ae6c1fcc (the merge of origin/main 96e724475c, which touches none of this diff's packages) derived 65 commands over the 7 changed paths. All 65 were run on 43ae6c1fcc, and --ran reconciles them: 65 derived, 65 run, 0 NOT-MEASURED, 0 UNRUN, all exit 0. Among them:

  • check:adr-0087-registration --base origin/main: not-required (no-migration-prescription) accepted.
  • check:changeset-no-major, check:empty-changeset, check:doc-authoring, check:nul-bytes, check:issue-citations.
  • check:engine-double-contract, check:where-matcher, check:driver-memory-census, check:cross-package-test-inputs, check:test-source-alias, check:type-check-coverage, check:type-check-debt.
  • check:query-options-erasure. It caught a first draft of the REST pin that erased an engine.aggregate options bag to any (test surface 236 to 237). Fixed by typing it (cafaf885d8), and the surface is back at 236.
  • check:dual-build-cjs-loads, after a full turbo run build of ./packages/* and ./packages/*/*.

Lint, narrowed and proven: pnpm exec eslint --no-inline-config --format json over the 6 changed .ts files at 43ae6c1fcc found 6 files, 0 errors, 0 warnings. Three facts make this narrowing a measurement:

  • The checked population comes from eslint's own config: isPathIgnored answers false for all 6.
  • The file count comes from the JSON output: 6 results.
  • Untouched files cannot change verdict: parserOptions.project and projectService are null for every file, so type-aware linting is not enabled.

Changeset

.changeset/20783-groupby-structured-json-refused.md: @objectstack/objectql minor, a BREAKING banner, Clause-②: no (narrowing), and exactly one ADR-0087 marker, not-required (no-migration-prescription), in the form PR #20781's changeset uses. It carries no rewrite table and no arrow: the body states what an author sees now, why, who is affected and what is unchanged. No export or published type changes: the door module is internal, and @objectstack/objectql's root and ./core exports are unchanged.

Acceptance notes

  • Reported to the PM, not filed here (same family, measured):
    • The native-SQL analytics face (triage's H4 sibling). A cube or dataset dimension on a structured-JSON field through NativeSQLStrategy answers one group per serialized document on SQLite and 500 on PostgreSQL. It never reaches this door. The fix lands in packages/services/service-analytics, which this claim does not touch.
    • groupBy on a multiple: true select through POST /api/v1/data/:object/query: memory answers one group per array, SQLite one per serialized array, and PostgreSQL 500 (json equality). Not this card's class, and a multi-value field has a plausible other meaning (a bucket per member), so it is not refused here.
    • count_distinct over a json field through the same door: memory 3, SQLite 3, PostgreSQL 500. AGGREGATE_FIELD_TYPE_COMPATIBILITY accepts count_distinct for every FieldType on the ground that every backend gives one answer. PostgreSQL does not for json.
  • service-analytics has two dynamic engine.aggregate callers: resolveFkAttr (groupBy: ['id', attr], a cross-object dimension attribute) and the display-label pass (groupBy: ['id', displayField]). If that attribute or display field is structured-JSON, they now answer this 400. Before, by reading and not measured: memory and SQLite grouped under id first, so one row per record, and PostgreSQL would have answered the same json-equality 500. No example or platform dataset names such a dimension (the census above).
  • A refusal relayed through the analytics engine path names the engine's position (groupBy[0]), not the cube member the caller wrote (c20783.meta). Carrier: the native-SQL sibling card, which touches the same face.
  • The native-SQL analytics path returned PostgreSQL's count as a string ("2") where SQLite returned a number, in the stand-in wiring above. Observed only; not measured through the HTTP door.

Generated by Claude Code

…gine's aggregate door

A groupBy entry naming a json, composite, repeater, record, location,
address or vector field is refused INVALID_FIELD / 400 before any driver
is asked, in both entry spellings.

Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY
Co-authored-by: Claude <noreply@anthropic.com>
… on SQLite and live SQL

Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY
Co-authored-by: Claude <noreply@anthropic.com>
…d of erasing them

Claude-Session: https://claude.ai/code/session_01DEvba2nBuD4tWzfq8r8NFY
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/objectql, touching 6 documentable anchor(s). ⚠️ 1 changed file(s) yielded no anchor (packages/objectql/src/number-comparand-declared-type-door.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

14 hand-written doc(s) NAME something this change touched and may need an implementation-accuracy re-verification:

  • content/docs/api/client-sdk.mdx (via data.query (sdk, the route ledger binds it to POST /api/v1/data/:object/query))
  • content/docs/api/data-api.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/api/error-catalog.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/api/wire-format.mdx (via /api/v1/data/:object/query (route, a path literal in a comment on a changed line))
  • content/docs/data-modeling/index.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/data-modeling/queries.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField), /api/v1/data/:object/query (route, a path literal in a comment on a changed line))
  • content/docs/deployment/validating-metadata.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/kernel/contracts/data-engine.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/kernel/runtime-services/data-service.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField), data.query (sdk, the route ledger binds it to POST /api/v1/data/:object/query))
  • content/docs/protocol/kernel/error-handling.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/protocol/objectql/query-syntax.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/protocol/objectui/concept.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/protocol/objectui/layout-dsl.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/ui/react-pages.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))

⛔ 3 release-owned page(s) also name something this change touched. These are read-only:

  • content/docs/releases/implementation-status.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/releases/v15.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))
  • content/docs/releases/v17/17-0.mdx (via groupBy (literal, a string literal in assertGroupByNamesNoStructuredJsonField))

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/objectql/src/number-comparand-declared-type-door.ts) — pages documenting those are invisible to this run
  • 2 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 17 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 96e724475c476d018c2b6d13dcd117ade14d0aa0 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 8c3eed2f8b8b31473f732fd3acbe50bb1444b93d — the merge of head 43ae6c1fcc7b4770385757f8fc3b439ca3a80382 into base 96e724475c476d018c2b6d13dcd117ade14d0aa0, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 8c3eed2f8b8b31473f732fd3acbe50bb1444b93d && git checkout 8c3eed2f8b8b31473f732fd3acbe50bb1444b93d
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 96e724475c476d018c2b6d13dcd117ade14d0aa0 43ae6c1fcc7b4770385757f8fc3b439ca3a80382 && git checkout -B drift-repro 96e724475c476d018c2b6d13dcd117ade14d0aa0 && git merge --no-ff 43ae6c1fcc7b4770385757f8fc3b439ca3a80382

node scripts/docs-audit/affected-docs.mjs --json 96e724475c476d018c2b6d13dcd117ade14d0aa0

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 96e724475c476d018c2b6d13dcd117ade14d0aa0 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 43ae6c1fcc7b4770385757f8fc3b439ca3a80382
Local-runs: none

PR #20804 (card #20783), isolated read-only review. The PR's head was still the sha above when read, twice. Inputs and nothing else: the card body and its three comments (triage 5905038653, the claim 5905436757, the os-dev-report 5906760989); the PR body, its 7-file list and the net diff against main (merge base 96e724475c, +574 / -2); PR #20781 (#20745, merged as 4b4ee88fb) as the same door's precedent; the head's check-runs. Nothing was built, run or re-run.

① Derived judgments

  • The refused set — right. engine.aggregate throws before any driver when a groupBy entry names a declared field whose type is in the spec's STRUCTURED_JSON_TYPES (json, composite, repeater, record, location, address, vector), in both entry spellings: a bare name at groupBy[i], the { field, dateGranularity } object at groupBy[i].field, a date bucket included. The class is imported from @objectstack/spec/data, never a list minted in the door, and the GUARD pin walks every FieldType against it. Triage wrote "a json field", the claim "(the structured-JSON class)", fix(objectql)!: a no-operator object beneath a relation, structured-JSON or undeclared id column is refused INVALID_FILTER / 400 on every driver (#20745) #20781 judged the same class at the same door, and the dev measured every member splitting per driver (H2). Refusing the class, not one member, is the right reading of the order.
  • What stays accepted — right. A text control (pinned on the recording driver, on SQLite, and on live PostgreSQL where a URL is set), number, image, an undeclared name (the engine's registry-less tolerance; the REST ingress assertGroupByFieldsExist in metadata-protocol/src/protocol.ts answers an unknown name INVALID_FIELD first, verified), a host with no field map, and a structured-JSON field as an aggregated column. A multiple: true select is deliberately left accepted: it is not in STRUCTURED_JSON_TYPES, triage named the JSON class only, and a multi-value field has a plausible other meaning (a bucket per member). The PostgreSQL 500 it still answers is reported as an out-of-scope finding with reach and evidence — the right disposal: no scope expansion, no buried defect.
  • The code — right. INVALID_FIELD / 400 with field, fields, object, param: 'groupBy', status and httpStatus. Triage asked for "an existing INVALID_* code, naming the field and its type". INVALID_FIELD is the code the ingress already answers for an unknown groupBy name, with the same envelope keys, the code assertFilterIsMaterializable answers for a name whose type cannot be filtered, and the code the search axis answers for a type it cannot scan (all verified in the tree). The verdict is about the named field's type at a position, so INVALID_FILTER (a filter's value shape) and INVALID_QUERY (a malformed shape) would both be wrong.
  • Where it stands — right. One call at aggregate's entry, after rejectCredentialAggregation (a credential or internal field keeps that refusal's words) and before the per-aggregation filter and having doors and before any driver is resolved. aggregate is the only engine verb taking groupBy (ENGINE_FIND_OPTION_KEYS and the passthrough key set carry no groupBy; the ScopedRepo.aggregate facade forwards to the same method). The REST query door reaches it through findData's engine.aggregate branch, and the analytics auto-bridge executeAggregate reaches it through engine.aggregate (service-analytics/src/plugin.ts). No driver file, no serialization rule: triage's ⛔ holds.
  • Public surface — none. packages/objectql/src/index.ts is not in the diff; the door module is referenced only from engine.ts and the pin. No export, type or schema moves; EngineAggregateOptions and QuerySchema.groupBy still parse the entry. The narrowing is runtime behaviour only.
  • The refusal words — true, with one imprecision. Position, verdict, "The query was NOT run.", the route, then the reason, in that order, so the route sits inside the REST door's 500-character CLIENT_MESSAGE_MAX (packages/rest/src/error-response.ts); the REST pin asserts the route in the body. Every sentence is true for json and its five twins. For vector the reason clause "one merged every row into a single group" overstates the in-memory driver, which by the dev's own table grouped per array rather than into one group; the class-level claim (no group key the drivers share) still holds for it, and the verdict, envelope and route are right for every member. Immaterial to this verdict; a wording tweak can ride the sibling analytics card.
  • The comment-only edit in number-comparand-declared-type-door.ts — true. The appended sentence is true: a json groupBy no longer reaches having, a min / max of a json field does (aggregatedRowColumnTypes gives a min / max alias its field's declared type). The [finding] a no-operator object under a lookup, master_detail or json field answers per driver: the declared nested-relation filter returns no rows on memory and a 400 on SQL, and a json object comparand deep-equals on memory and is refused on SQL #20745 sentence before it is left standing and only qualified after, which reads awkwardly, but the paragraph as a whole is now correct. Comment only, no behaviour.
  • The fixture triage in engine-nested-object-door.test.ts — right. The old having case grouped by meta, which pinned exactly the branch this door now closes earlier; the replacement (a max of meta aliased top_meta, having on top_meta) keeps fix(objectql)!: a no-operator object beneath a relation, structured-JSON or undeclared id column is refused INVALID_FILTER / 400 on every driver (#20745) #20781's JSON arm at having reached, with the pin's expected words ("carries a json value", "whole-value match") unchanged. Replaced, not weakened.
  • The PR body's factual sentences — true where the tree can answer them: the assertGroupByFieldsExist claim; the three INVALID_FIELD precedents; ENGINE_FIND_OPTION_KEYS; CLIENT_MESSAGE_MAX = 500; the NativeSQLStrategy granularity decline; resolveFkAttr (groupBy: ['id', attr]) and the display-label pass (groupBy: ['id', displayField]); zero production new MemoryAnalyticsService; the merge of origin/main 96e724475c bringing nothing under packages/objectql or packages/rest (it moved two files under packages/spec, a dependency, not one of this diff's packages); the 7-file list at +574 / -2. The before/after driver readings, the ablation, the local suite counts and the narrowed lint are the dev's own readings, not re-run here; the gate verdicts are the head's check-runs.
  • Check-runs on the head, as read for this record: 32 runs, 24 success, 3 skipped (Console Pin Gate, Build Docs, Packed-tarball smoke — path-filtered or opt-in), 5 in_progress (Test Core 1/6, 4/6, 5/6; Lint & Repo Gates; Type Check · workspace), 0 failed. Of the seven required contexts, Build Core, Dogfood Regression Gate, Temporal Conformance (live PG + MySQL) and Governed Surface Queue Guard are success; Lint & Repo Gates, TypeScript Type Check and Test Core were still running. Check Changeset and Check PR Size are success. Nothing red at the moment of reading. This PASS is on the diff; arming waits for those three to conclude green, as always.

② Semver level

  • .changeset/20783-groupby-structured-json-refused.md: @objectstack/objectql minor, a BREAKING banner, the line Clause-②: no (narrowing) and exactly one ADR-0087 marker, not-required (no-migration-prescription) — the same form as fix(objectql)!: a no-operator object beneath a relation, structured-JSON or undeclared id column is refused INVALID_FILTER / 400 on every driver (#20745) #20781's 20745-nested-object-door.md. Right: the diff narrows what a published package accepts at runtime and widens nothing (no new accepted input, no new export), so the declaration is no with the (narrowing) arm, which is BREAKING and takes at least minor under the launch-window convention (check:changeset-no-major). patch would be wrong, major refused, skip-changeset wrong. The PR body carries the same line, so the two carriers agree.
  • The marker's category holds on the body: no FROM / TO arrow, no rewrite table, no migration heading. The route ("store the part you group on in a field of its own and group by that field") is prose a reader applies by hand, the same shape fix(objectql)!: a no-operator object beneath a relation, structured-JSON or undeclared id column is refused INVALID_FILTER / 400 on every driver (#20745) #20781's accepted marker rides on, and the marker's own argument (no authorable key, spelling, export or stored shape moves; no mechanical rewrite exists because which scalar part of the document was meant is not in the query) is true of the diff. Check Changeset on the head is success; check:adr-0087-registration and check:changeset-no-major sit in Lint & Repo Gates, still in_progress when read.
  • @objectstack/rest gains a test file only and publishes nothing new: no changeset owed. No other package moves. The changeset's "Who is affected" names the REST door, in-process engine.aggregate callers and the analytics aggregate path; its "Unchanged" paragraph is true of the diff (the multiple: true select, file fields, aggregated JSON columns and undeclared names all still reach the driver, pinned as controls).

③ Boundary flags

Every dev flag from the os-dev-report (5906760989) and the PR body, answered:

  • The memory pin is by construction (a recording driver), with real InMemoryDriver cells from an uncommitted scratch script. Accepted. check:driver-memory-census (ledger scripts/driver-memory-census.ledger.json) admits no new test consumer of @objectstack/driver-memory without a ruling; the door answers before a driver is resolved, so a recording driver asserting zero reads is the whole memory claim; fix(objectql)!: a no-operator object beneath a relation, structured-JSON or undeclared id column is refused INVALID_FILTER / 400 on every driver (#20745) #20781 took the same route at the same door. The InMemoryDriver before/after readings stay the dev's, as the card's own readings were.
  • Fixture triage in engine-nested-object-door.test.ts. Answered under ①: right, replaced not weakened.
  • origin/main merged once (96e724475c), suites and gates re-run on the merge commit. In order under the multi-agent discipline; main brought nothing into packages/objectql or packages/rest.
  • The analytics face: the ObjectQL strategy folded in, NativeSQLStrategy left as the sibling card. Right against triage ("folded in only if the same door serves it; otherwise it becomes a sibling card"). The auto-bridge lowers onto engine.aggregate, so that half is served by this door with no packages/services edit; the native strategy compiles its own GROUP BY and never reaches the engine, and its fix lands in packages/services/service-analytics, outside this claim. Escalated to the PM: file the sibling card from out_of_scope_findings[0] (reach, evidence and dedupe words are there); the engine-position wording (groupBy[0] rather than the cube member) and the PostgreSQL string count ride it as carriers, as the report proposes.
  • Out of scope: groupBy on a multiple: true select (PostgreSQL 500) and count_distinct over a json field (PostgreSQL 500). Right to leave out: neither is this card's class, the select has a plausible other meaning, and count_distinct is a question for the spec's AGGREGATE_FIELD_TYPE_COMPATIBILITY table. Both carry reach, evidence and dedupe words. Escalated to the PM: file them, as one family card or two (the report proposes one family; the seat's call).
  • resolveFkAttr and the display-label pass now answer this 400 for a structured-JSON attribute or display field. Answered: a consequence of the one shared door, not a second narrowing. The changeset's "Who is affected" covers the analytics aggregate path, no example or platform dataset names such a dimension (the census), and the reading that memory and SQLite answered one row per record before is by reading, not measured, which the report says. No carrier beyond the sibling analytics card is needed.
  • Process notes (the --maxWorkers=2 after a bare --, the PostgreSQL data dir under /tmp, the eslint probe copied then deleted, a clean git status after each): acknowledged; none touches the diff.
  • open_questions: none filed; nothing to answer.
  • Triage's pins (memory, SQLite, PostgreSQL, with a text control): present as ordered — memory by construction, SQLite always, PostgreSQL where OS_TEST_POSTGRES_URL is set. No CI job sets that URL for @objectstack/rest, so the live PostgreSQL cell is the dev's local run (6 passed / 3 skipped), the same caveat fix(objectql)!: a no-operator object beneath a relation, structured-JSON or undeclared id column is refused INVALID_FILTER / 400 on every driver (#20745) #20781 carried and the PR body states.

Implemented-by: claude/issue-20783-groupby-json-refused
Reviewed-by: session_01DEvba2nBuD4tWzfq8r8NFY

VERDICT: PASS


Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review September 30, 2026 08:09
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Sep 30, 2026
Merged via the queue into main with commit 157baa7 Sep 30, 2026
36 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-20783-groupby-json-refused branch September 30, 2026 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] groupBy on a json field answers 500 DATABASE_ERROR on PostgreSQL, one merged group on memory, and one group per serialized value on SQLite

2 participants