Skip to content

fix(objectql): the rows path adds sum / avg with compensated summation, as SQLite does - #20543

Merged
objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-20489-rows-path-compensated-sum
Sep 29, 2026
Merged

objectstack-fleet[bot] merged 5 commits into
mainfrom
claude/issue-20489-rows-path-compensated-sum

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #20489

Clause-②: no

What changed

The engine's rows path (packages/objectql/src/in-memory-aggregation.ts, applyInMemoryAggregation) now adds sum / avg with Kahan-Babuska-Neumaier compensation. That is the direction triage 5875711059 ruled: fix the one face the platform owns, so the rows path agrees with SQLite native on both SQLite paths and gives the more accurate answer.

  • One helper, compensatedSum, is a transcription of SQLite's kahanBabuskaNeumaierStep plus its finalizers' overflow guard: when the error term is non-finite, the plain running sum is returned. The sum arm and the avg arm both call it. Before, each arm had its own naive reduce.
  • toNumber, the null / non-numeric handling, the empty group (sum 0, avg null) and the answer's type (a JS number) are untouched.
  • Out of scope, as the claim set it: driver-sql's native aggregate, PostgreSQL / MySQL arithmetic, driver-memory, and packages/spec.

Head measured: 8ad4a9d4a. It is a true merge of origin/main at 03b19d9cf, on top of the two commits of this branch.

H1: the defect, before and after

Base b2b6a0643, SQLite 3.53.4 (better-sqlite3), driver-sql. One number column. The rows path is forced by a filtered sibling count. Both engine.aggregate and POST /api/v1/data/:object/query were read, and they gave the same values in every cell:

group native sum / avg rows path, base rows path, 8ad4a9d4a
0.1, 0.2, 0.3 0.6 / 0.19999999999999998 0.6000000000000001 / 0.20000000000000004 0.6 / 0.19999999999999998
1e16, 1, -1e16 1 / 0.3333333333333333 0 / 0 1 / 0.3333333333333333
1e16, 0.5, -1e16 0.5 / 0.16666666666666666 0 / 0 0.5 / 0.16666666666666666
0.1, 0.2 0.30000000000000004 / 0.15000000000000002 same same
1, 2, 3, 40, 500 546 / 109.2 same same

having { s: { $eq: 0.6 } }, engine and REST, and { a: { $eq: 0.19999999999999998 } }, engine:

  • base: keeps the group on native and no group on rows;
  • after: keeps the group on both. The REST pin also holds the a case through REST, on both paths.

H1 holds as the card stated it.

H3: the compensated fold against SQLite native

SQLite's native sum / avg was measured on three engines: better-sqlite3 3.53.4 (driver-sql), sql.js 3.49.1 (driver-sqlite-wasm) and @libsql/client 3.45.1 (driver-turso). better-sqlite3 and sql.js were read over a REAL and a NUMERIC column, libsql over a REAL column. The JS fold was run over the same doubles. The 2^53 row was read on better-sqlite3 alone.

fixture naive (rows path before) Neumaier (rows path after) SQLite native, all three engines
0.1, 0.2, 0.3 0.6000000000000001 / 0.20000000000000004 0.6 / 0.19999999999999998 0.6 / 0.19999999999999998
1e16, 1, -1e16 0 / 0 1 / 0.3333333333333333 1 / 0.3333333333333333
1e16, 0.5, -1e16 0 / 0 0.5 / 0.16666666666666666 0.5 / 0.16666666666666666
1e20, 1, -1e20 0 / 0 1 / 0.3333333333333333 1 / 0.3333333333333333
0.1, 0.2 (two-addend control) 0.30000000000000004 / 0.15000000000000002 the same the same
1, 2, 3, 40, 500 (integers) 546 / 109.2 the same the same
2^53, 1, 1 9007199254740992 9007199254740994 9007199254740994
  • No fixture disagrees with SQLite. A randomized cross-check agrees too: 5,000 groups of 3 to 22 values (cents, tenths, raw doubles, mixed sign and magnitude up to 1e19) against better-sqlite3. The Neumaier sum and avg matched SQLite on all 5,000 groups; the naive fold disagreed on 1,601 of them.
  • SQLite's integer affinity. Under NUMERIC, SQLite stores 1e16 as an INTEGER and sums it in exact int64. The answer is the same 1.
  • Integers beyond 2^53 move. Integers are unchanged while the running total stays within 2^53. Beyond it, the compensated total is the exact total SQLite also answers, where the naive fold dropped the 1s (last row).

H2: census of rows folds of sum / avg in packages/**

site own fold? reading over 0.1, 0.2, 0.3, head 8ad4a9d4a
objectql applyInMemoryAggregation, sum and avg arms the fold this PR changes. H2 partly held: two naive reduce copies in one function, now one helper 0.6 / 0.19999999999999998
per-aggregation filter shares it: filters the bucket's rows, then the same arms pinned in the unit test
having no fold: it reads the aggregated row whichever path produced it inherits
roll-up summary (summary-aggregate.ts), and service-analytics ObjectQLStrategy no fold: both call engine.aggregate inherit the engine's path
driver-memory data face (memory-driver.ts, sum / avg arm) own naive fold 0.6000000000000001 / 0.20000000000000004
driver-memory analytics face (memory-analytics.ts, mingo $sum / $avg) own fold (mingo) 0.6000000000000001 / 0.20000000000000004
service-analytics draft preview (preview-evaluator.ts, sum / avg arms) own naive fold 0.6000000000000001 / 0.20000000000000004
service-analytics dataset-executor.ts computeDerived sum not a rows fold: it adds measure columns within one row n/a
service-analytics NativeSQLStrategy, driver-mongodb $sum database's own arithmetic not measured

Three sites keep their own fold. They are measured and reported, not edited, as the dispatch requires. One reading needs stating plainly:

  • driver-memory now splits by path. engine.aggregate on driver-memory answers 0.6000000000000001 on its native path and 0.6 on the rows path. having { s: { $eq: 0.6 } } keeps the group on the rows path only.
  • Before this PR, both of its paths were naive. The split on that driver is new here; SQLite's split is gone.
  • It goes back to the seat as a family finding. The suggested route: hoist the fold to @objectstack/core, as bucketDateKey was, and adopt it on driver-memory's two faces and the preview.
  • The changeset states it.

H4: what stays as it was (pinned)

  • null adds nothing to sum and is left out of the avg count.
  • A non-numeric cell reads as 0 and counts in avg. A numeric string reads as its number. A boolean reads as 0 / 1.
  • The empty group, with no rows or only nulls: sum 0, avg null.
  • A non-finite total (Infinity, -Infinity, overflow, NaN) is Object.is-equal to the naive answer.
  • The answer's type is a number.

H5: the residual

Residual, stated. PostgreSQL and MySQL add sum / avg natively in double without compensation. That arithmetic is the database's own, and this PR does not wrap it. So over three or more fractions, their native path can still differ from the rows path in the last place. For 0.1 + 0.2 + 0.3, #20387's measurement on live PostgreSQL 16.13 and MySQL 8.0.46 gave native 0.6000000000000001; the rows path is now 0.6. An exact $eq on a fractional sum compares doubles, and a cross-dialect last-place difference remains there: compare with a range.

Tests

New pins

  • packages/objectql/src/in-memory-aggregation-compensated-sum.test.ts: 11 cases, covering the H3 fixtures, groupBy plus a per-aggregation filter, and the H4 cases.
  • packages/rest/src/rest-aggregate-compensated-sum.test.ts: 4 cases on SQLite, through the engine and REST. It proves each path really ran (a spy on driver.aggregate), checks that find() reads back the doubles, pins native equal to rows for every group, and pins having $eq / $in on both paths.

Reverse verification, from the committed fix

  • Tool: scripts/ablation-replace.mjs (wrap mode). It swapped return Number.isFinite(c) ? s + c : s; for const ablation20489 = s; return ablation20489;, which is the naive running sum.
    • Anchor count went 1 to 0, blob ff8d2385a69f to 89ea11d3bc18.
    • Then pnpm --filter @objectstack/objectql build, and ablation-dist-preflight.mjs @objectstack/objectql ablation20489 found the marker in 4 built files.
  • Mutated leg: objectql unit test 6 failed / 5 passed of 11; REST pin 2 failed / 2 passed of 4. The expected direction, red.
  • Restore leg: the restore was proven (blob equal to HEAD, git diff HEAD empty).
    • Then a rebuild, and --absent found the marker absent from all 14 built files, with a clean tree.
    • Unit 11/11 and REST 4/4.

Suites

suite head result
pnpm --filter @objectstack/objectql test db9d27230 334 files / 6651 tests passed
pnpm --filter @objectstack/rest test db9d27230 224 files / 4242 passed, 43 skipped
typecheck for both packages, check:test-typecheck included db9d27230 OK; both new test files are in the tsconfig.test.json programs (--listFilesOnly)
targeted re-run: objectql aggregation files 72/72, REST pin plus rest-aggregate-numeric-having 16 passed / 24 skipped 8ad4a9d4a green

The merge brought driver-sql, cli, lint, platform-objects and spec changes, and none in objectql or rest. After it, the @objectstack/rest^... closure was rebuilt before the targeted re-run.

Gates

  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at 8ad4a9d4a printed 63 commands. All 63 exit 0, and each exit code was written to a file before any pipe.
  • check:dual-build-cjs-loads and check:type-check-debt first exited 3 (PREREQUISITE NOT MET: no dist/).
    • They were re-run after pnpm exec turbo run build --concurrency=2 --filter='./packages/*' --filter='./packages/*/*' (71 tasks), and both passed.
    • The dist-reading gates were re-run over the full build too.
  • --ran: 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN.
  • CI-only lanes, NOT MEASURED locally: the 5 path-scheduled CI jobs, the workspace type-check lanes and the wide-population families that dispatch-gates names.

Lint

  • Repo-wide pnpm lint is CI's.
  • A narrowed run, measured: eslint --no-inline-config --format json over the three changed TS files counted 3 files, 0 errors and 0 warnings.
    • Population: eslint.config.mjs files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']; the changeset is not a linted extension.
    • Invariance: the config enables no type-aware linting (no parserOptions.project), so this diff cannot move an untouched file's verdict.

Acceptance notes

All three notes from the first round were corrected in patch round 1, text only (below):

  • The AGGREGATE_ACCUMULATION residual comment in packages/drivers/driver-sql/src/sql-driver.ts ("as the rows path adds them", "on SQLite's native face alone", "every other face 0.6000000000000001").
  • The header sentence of sql-driver-20387-aggregate-double-accumulation.test.ts that called rowsPathSum the rows path's arithmetic.
  • The pending .changeset/20387-aggregate-one-double.md residual paragraph, as a DELIBERATE CORRECTION.

Still standing, not edited (outside the admitted text): the aggregate() inline comment in sql-driver.ts ("so one query answers one number on every face", beside the accumulatesInDouble call). It was already an overclaim before this PR, since SQLite's native sum compensated. It points at AGGREGATE_ACCUMULATION, which now states the residual. Carrier: none.

DELIBERATE CORRECTION

Note: .changeset/20387-aggregate-one-double.md. It is pending on main, so it ships in the same release as this PR.

Sentence corrected: the "Residual, stated." paragraph, and only that paragraph. It read:

  • "Double sums are added in row order, one after another, as the rows path adds them."
  • "So for a group of three or more fractions, SQLite's native answer can still differ from every other face in the last place (0.1 + 0.2 + 0.3: SQLite 0.6, every other face 0.6000000000000001). This was already true of SQLite's two paths before this change."

What changed under it: this PR makes the engine's rows path add with compensated summation, as SQLite 3.43+ does. The rows path now answers 0.6, not 0.6000000000000001, so "every other face" and "SQLite's native face alone" are false once this PR lands.

It now reads: PostgreSQL and MySQL add without compensation, while SQLite and, since #20489, the rows path compensate. So the PostgreSQL / MySQL native answer can differ from SQLite's and the rows path's in the last place (0.1 + 0.2 + 0.3: PostgreSQL / MySQL native 0.6000000000000001, SQLite and the rows path 0.6). Before #20489, SQLite's own two paths differed there too. Two addends cannot differ.

Every other sentence was checked against this PR and left unchanged:

  • Line 16, "11 / 9 answered 1.2222222222222222, where every other face answers 1.2222222222222223", stays true. Integer addends within 2^53 sum exactly under both folds. Measured through the built rows path at 1c4767883: applyInMemoryAggregation over 1,1,1,1,1,1,1,2,2 answers sum 11 and avg 1.2222222222222223, and over 1, 2, 2 answers 1.6666666666666667.
  • The title and line 10, "SQLite and the engine's rows path add JS doubles", stay true: both still add doubles, now compensated.
  • The 0.1 + 0.2 statements (base lines 11-12, 38-40 and 48-50) are two addends, which no compensation moves.

Gate: check-empty-changeset is red on exactly this one name, by design ("DELIBERATE CORRECTION ... do NOT restore it"). ⛔ No skip-changeset; the note is not restored. The at-tier review is the written confirmation, as the seat answer asks.

Patch round 1

New head: 1c4767883. One push, carrying a true merge of origin/main at 288611e3e (spec-only commits) as 925368f00, then one text-only commit. Seat answer 5881988201 ruled the open question A.

Edits:

  1. .changeset/20387-aggregate-one-double.md: the DELIBERATE CORRECTION above.
  2. packages/drivers/driver-sql/src/sql-driver.ts: the AGGREGATE_ACCUMULATION residual comment now states PostgreSQL / MySQL native against SQLite and the rows path.
  3. packages/drivers/driver-sql/src/sql-driver-20387-aggregate-double-accumulation.test.ts: the header sentence says the expected values are find()'s rows added in row order, which is the rows path's compensated sum too over this file's fixtures (two addends, and integers within 2^53). "a compensated sum ... fail" is removed, since it was never true for two addends. The one-line docs of rowsPathSum / rowsPathAvg make the same statement, because the header links them.
  4. .changeset/20489-rows-path-compensated-sum.md: it no longer says it "replaces" the driver-sql residual. It says the driver-sql entry states the same residual, and it keeps the driver-memory split and the PostgreSQL / MySQL residual.

Comment-only proof for items 2 and 3: every changed line under packages/drivers/driver-sql/ is a comment line (git diff filtered to non-comment lines is empty).

No @objectstack/driver-sql line added to this PR's changeset. Why none is needed:

  • Check Changeset counts the changesets a PR adds, not the packages it touches.
  • All 69 publishable packages are one fixed group in .changeset/config.json, so the @objectstack/objectql patch releases driver-sql in lockstep anyway.
  • The edited comment does not ship. @objectstack/driver-sql's built dist/index.{js,mjs,d.ts,d.mts} carry 0 hits for "Kahan", "20489" or "every other face". The positive control, AGGREGATE_ACCUMULATION, is found in those same four files.

Checks at 1c4767883, each exit code recorded before any pipe:

check exit reading
node scripts/check-empty-changeset.mjs --base origin/main 1 red on exactly one name, .changeset/20387-aggregate-one-double.md (one ::error line), by design; the empty-frontmatter half is green
node scripts/check-changeset-fixed.mjs 0 fixed group in sync with 69 public packages
node scripts/check-changeset-no-major.mjs --base origin/main 0 no major bump
node scripts/check-adr-0087-registration.mjs --base origin/main 0 no declared-breaking changeset
pnpm check:nul-bytes 0 green
node scripts/check-issue-citations.mjs 0 3 citations resolve
pnpm --filter @objectstack/driver-sql typecheck 0 the test file is in its program (--listFilesOnly)
vitest run src/sql-driver-20387-aggregate-double-accumulation.test.ts (driver-sql) 0 5 passed, 2 skipped (the PostgreSQL / MySQL cells, no server here)
pnpm check:driver-conformance, check:object-def-param-keys, check:tenant-chokepoint 0, 0, 0 the three families dispatch-gates newly derives for the driver-sql paths
objectql aggregation tests, and the REST compensated-sum pin 0 72/72 and 4/4

The other 63 derived families were run at 8ad4a9d4a (above). This round changes only text on top of a spec-only merge, and CI re-runs them.


Generated by Claude Code

…mpensation, as SQLite does

The engine's rows path (in-memory-aggregation.ts) folded sum and avg
naively, while SQLite 3.43+ compensates, so one query answered two doubles
on SQLite depending on the path engine.aggregate took (0.1 + 0.2 + 0.3:
native 0.6, rows 0.6000000000000001). Both arms now add through one
compensated fold transcribed from SQLite's kahanBabuskaNeumaierStep and
its finalizers' overflow guard.

Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN
Co-authored-by: Claude <noreply@anthropic.com>
… residual it leaves

Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/m documentation Improvements or additions to documentation tests tooling labels Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

2 anchor(s) derived from 2 changed package(s); no hand-written page names any of them. ⚠️ 1 changed file(s) yielded no anchor (packages/drivers/driver-sql/src/sql-driver.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/drivers/driver-sql/src/sql-driver.ts) — pages documenting those are invisible to this run
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 23 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json ba5927f714af7516105706b36a05cedf34d5fa1b → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 1b2e388b6b50f2fc4e8f967879943a2ddcac84ec — the merge of head 1c476788382b5de4e5aec4c5cebb71fe14452f53 into base ba5927f714af7516105706b36a05cedf34d5fa1b, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 1b2e388b6b50f2fc4e8f967879943a2ddcac84ec && git checkout 1b2e388b6b50f2fc4e8f967879943a2ddcac84ec
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin ba5927f714af7516105706b36a05cedf34d5fa1b 1c476788382b5de4e5aec4c5cebb71fe14452f53 && git checkout -B drift-repro ba5927f714af7516105706b36a05cedf34d5fa1b && git merge --no-ff 1c476788382b5de4e5aec4c5cebb71fe14452f53

node scripts/docs-audit/affected-docs.mjs --json ba5927f714af7516105706b36a05cedf34d5fa1b

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

…te and its comments

The rows path now adds with compensated summation, so the residual the
pending driver-sql double-accumulation note, the AGGREGATE_ACCUMULATION
comment and the driver-sql test header stated ("SQLite's native face
alone", "every other face 0.6000000000000001") no longer held. Each now
states it as PostgreSQL / MySQL native against SQLite and the rows path.
Text only; no logic moves.

Claude-Session: https://claude.ai/code/session_01N8TPEsoJxPsdSdNKGnNGEN
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: CONTRACT_REVIEW_TIER
Head-sha: 1c476788382b5de4e5aec4c5cebb71fe14452f53
Local-runs: none

Inputs read: card #20489 (body; comments 5875711059 triage, 5881410875 claim, 5881969729 os-dev-report round 1, 5881988201 seat answer, 5882195153 os-dev-report patch 1), PR #20543 (body, 7-file list, net diff against merge-base 288611e3e of origin/main, its one bot comment), #20387 (body and comments, incl. the #20486 os-dev-report 5875171240, the ACCEPT 5875542725 and the landing record 5875956131), PR #20486 (body; its at-tier record 5875498653), card #20544 (body), and the check-runs on the head (read twice; the last read is at the end of ③). Read-only git and REST GETs only; nothing built, run or re-run. The head is two true merges plus three branch commits: 24706c959 (fix + two tests) and db9d27230 (changeset) on the H1 base b2b6a0643, merge 8ad4a9d4a (parents db9d27230, 03b19d9cf), merge 925368f00 (parents 8ad4a9d4a, 288611e3e), then 1c4767883 (text only, four files). The #20486 squash fc0db22bc is an ancestor of the merge base, so triage's "serial after PR #20486" is met.

① Derived judgments

1. compensatedSum (packages/objectql/src/in-memory-aggregation.ts:317): a faithful transcription, right. The step t = s + r; c += Math.abs(s) greater than Math.abs(r) ? (s - t) + r : (r - t) + s; s = t is SQLite's kahanBabuskaNeumaierStep line for line, including SQLite's own strictly-greater branch (fabs(s) strictly greater than fabs(r)) rather than the textbook greater-or-equal; at the tie the two branches are equal (with equal magnitudes s + r is exact, so both error terms are 0), so the spelling is immaterial and the code follows SQLite's. The finalizer Number.isFinite(c) ? s + c : s is sumFinalize / avgFinalize's sqlite3IsOverflow(rErr) guard (NaN or an infinity is exactly "not finite"): s is the naive running total in the same order (t = s + r each step), so a non-finite total is Object.is-equal to the old answer, and once s is non-finite c is too (the s - t term is NaN), pinned across five shapes. Both arms take it: sum is compensatedSum(values.map(toNumber)), where toNumber(null) is 0 as before (adding 0 moves neither s nor c), and avg is compensatedSum(nums) / nums.length over the same non-null toNumber list as before, the same fold divided by the same count, which is SQLite's (rSum + rErr) / cnt. Nothing else in applyInMemoryAggregation moved: the diff's only code lines are the two arms, a three-line comment and the helper; count, count_distinct, min, max, the empty group (sum 0 since the fold over nothing returns 0 + 0, avg null via nums.length === 0), collectValues, toNumber, the per-aggregation filter and bucketDateValue are byte-unchanged, and compensatedSum is module-private (index.ts:361 still exports applyInMemoryAggregation and bucketDateValue only). Two addends: after [a] the state is s = a, c = 0; after b, t = fl(a + b) and c is the exact rounding error of that one addition (the branch always subtracts from the larger magnitude, Fast2Sum-exact), and t + c is mathematically a + b, which rounds to t again, so compensatedSum([a, b]) === a + b for every finite pair. Integers whose partial sums stay within 2^53: every addition exact, c stays 0, the naive answer returns. 2^53, 1, 1: c carries the two dropped 1s and s + c is 9007199254740994, the total SQLite answers over REAL (Neumaier) and NUMERIC (exact int64) alike, as the PR states for that row. Not transcribed, and rightly so: SQLite's int64 path for INTEGER-typed values (the JS fold is doubles only; within 2^53 the two are identical, beyond it the one-double policy declares the loss).

2. The pins: cover what the card and the brief ask, and weaken nothing. The objectql unit file (11 cases): the card's fixture with the discriminating naiveSum assertion beside it; the 1e16 and 1e16 + 0.5 cancellations; the two-addend control (four pairs plus the 0.30000000000000004 literal); integers within 2^53 (two sets, Number.isInteger); 2^53, 1, 1; groupBy with a per-aggregation filter on both arms plus an unfiltered sibling; null in sum and avg; non-numeric, numeric-string and boolean cells; the empty group with no rows and with only nulls; five non-finite shapes held Object.is-equal to the naive answer; the number type. The REST pin (4 cases, better-sqlite3 :memory: through a real SqlDriver): the spy proves native calls driver.aggregate once and rows never calls it (the rows path is forced by a filtered sibling count whose predicate { g: { $ne: '' } } excludes no row, so both paths add identical operands); find() reads back the doubles written; every group's sum / avg equals the SQLite-native literals on both paths through engine.aggregate and POST /api/v1/data/:object/query; and having $eq 0.6, $eq 0.19999999999999998, $eq 1, $in [0.5, 546] keep the same group on both paths, engine and REST, with the residual pinned as $eq 0.3 keeping nothing and $eq 0.30000000000000004 keeping two. So having { s: { $eq: 0.6 } } is pinned to keep the group on both SQLite paths. The spy sits in the first case only, but every case builds its options through the same grouped() helper, so the path proof carries by construction. No existing assertion changed or weakened: the net diff touches no existing test line except comments in the 20387 file, and git grep at the merge base finds no test in packages/** or apps/** pinning 0.6000000000000001, 0.20000000000000004 or 0.19999999999999998 (the sole hit is the sql-driver.ts comment this PR corrects).

3. The DELIBERATE CORRECTION of .changeset/20387-aggregate-one-double.md: confirmed, and this record is the written confirmation Check Changeset asks for. The note is pending on the merge base: blob 7194e30ee at 288611e3e and at the origin/main tip ba5927f71 at read time, head blob 691961c7a; the squash that added it (fc0db22bc) is an ancestor of the merge base, so it ships in the same release as this PR. It is the only foreign changeset edited (git diff --name-status on .changeset/: M the 20387 note, A the 20489 note), and the only hunk in it is lines 42-47, five lines out and six in, the "Residual, stated." paragraph and nothing else. What changed under it: this PR makes the rows path compensate, so "as the rows path adds them" and "every other face 0.6000000000000001" became false. The rewritten paragraph, sentence by sentence: (1) "On PostgreSQL and MySQL the double sums are added in row order, one after another, without compensation." TRUE for a serial plan (sum(float8) / SUM(DOUBLE); PR #20486 measured 0.6000000000000001 over 0.1, 0.2, 0.3 on live PostgreSQL 16.13 and MySQL 8.0.46; the PostgreSQL parallel-aggregate note from record 5875498653 stands as a note). (2) "SQLite 3.43 and later adds with compensated summation, and since #20489 so does the engine's rows path." TRUE, the diff. (3) "So for a group of three or more fractions, the PostgreSQL and MySQL native answer can still differ from SQLite's and the rows path's in the last place" TRUE. (4) "(0.1 + 0.2 + 0.3: PostgreSQL / MySQL native 0.6000000000000001, SQLite and the rows path 0.6)." TRUE (PG / MySQL from #20486's live measurement; SQLite from the card and the three engines the dev read; the rows path from the unit pin). (5) "Before #20489, SQLite's own two paths differed there too." TRUE, the card's table. (6) "Two addends cannot differ." TRUE, the proof in item 1. Line 16, "11 / 9 answered 1.2222222222222222, where every other face answers 1.2222222222222223", still holds: the nine group is seven 1s and two 2s, every addition exact, c stays 0, sum 11 and 11 / 9 is 1.2222222222222223 in double on the rows path, the same 11 / 9 the driver-sql test pins at line 190 on all three cells; SQLite sums the integers in int64 and divides the same way. The title and line 10 ("SQLite and the engine's rows path add JS doubles") stay TRUE; lines 11-12, 38-40 and 49-51 are two-addend statements, unchanged and TRUE. The gate: Check Changeset is red on this one name (twice, two triggers), by design; the gate's own remedy for this class is "do NOT restore it -- say so on the PR and get it confirmed" (FOREIGN_CORRECTION_REMEDY), pr-automation.yml says to name the note and what changed and leave the check red, and landing-operations.md makes the same-head at-tier PASS record that confirmation provided it names the note and judges each rewritten sentence, which this record has now done. No skip-changeset label is on the PR (labels: documentation, size/m, tests, tooling), and the note is not restored.

4. The driver-sql edits: comment-only, and each now TRUE. Re-derived from the net diff: filtering every changed line under packages/drivers/driver-sql/ to non-comment lines leaves nothing (each + / - line is a * line or a one-line /** ... */). sql-driver.ts lines 1596-1604, the AGGREGATE_ACCUMULATION residual: PostgreSQL / MySQL scan-order without compensation, TRUE; SQLite 3.43+ compensated "and since #20489 so does the engine's rows path (in-memory-aggregation.ts, compensatedSum)", TRUE (the file and the name exist at this head); the last-place difference "between the PostgreSQL / MySQL native faces and those two", TRUE; the example, TRUE; "Two addends cannot differ, which is why the pin is 0.1 + 0.2", TRUE. The 20387 test header, lines 28-32: "computed from find()'s own rows, added in row order (rowsPathSum), plus the literal the triage named, so a decimal answer or a string each fail", TRUE (rowsPathSum is that reduce; the literals sit at 161-162; toBe fails a string); "Over these fixtures, two addends, and integer-valued columns within 2^53, that sum is also the rows path's, which adds with compensation since #20489: the two folds cannot differ there", TRUE: rowsPathSum / rowsPathAvg are the oracle only on pin (0.1, 0.2), three (1, 2, 2), nine (seven 1s, two 2s) and the 0 / 1 flags, while the big group (2^53 + 1, 1) is pinned to a literal at line 197 and never to rowsPathSum. The rowsPathSum / rowsPathAvg one-liners say the same, TRUE. The unedited aggregate() inline comment (lines 10432-10434, "so one query answers one number on every face"): acceptable to leave. It is FALSE in letter for three or more fractional addends, and it was already FALSE at the merge base, since SQLite's native face compensated while the rows path did not; this PR neither wrote nor falsified it, it is outside the admitted text (seat answer 5881988201 item 3 admitted the residual comment and the header sentence only), it is a comment in unpublished source, and its pointer lands on AGGREGATE_ACCUMULATION, whose docblock now states the residual in full. Recorded as an acceptance note, carrier the next touch of aggregate(); not a FAIL.

5. The new driver-memory split: stated honestly, and #20544 is the right carrier. The 20489 changeset says the in-memory driver "still adds naively in its own aggregate, so on that driver the two paths can now differ in the same last place": "now" declares the split new, "the same last place" ties it to the 0.6 / 0.6000000000000001 example above it, and the PR body's H2 states the numbers and that both of its paths were naive before. The finding carries class (a) and a reach: measured at engine.aggregate on driver-memory at 8ad4a9d4a, so it passes the filing gate. #20544 was filed bare by the seat from it: it names the three faces (memory-driver.ts's sum / avg arm, memory-analytics.ts's mingo $sum / $avg, preview-evaluator.ts), the hoist of compensatedSum to @objectstack/core after the bucketDateKey precedent, "serial after PR #20543", the pins to write and a dedupe, and it sits open and unlabelled for triage. That is consistent with triage 5875711059 ("the one face the platform owns", the rows path) and with the claim's own ⛔ on driver-memory ("measured and reported, not edited"). The H2 census table is the dev's measurement, carried to #20544 and consistent with that card's table; not re-measured here.

② Semver level

@objectstack/objectql: patch, Clause-②: no, no arm — right. Clause ② asks whether the card widens an accept set or enlarges the public surface (clause2-line.mjs); this diff adds no key and no export (compensatedSum is module-private, index.ts:361 is unchanged), applyInMemoryAggregation's signature and accepted inputs are unchanged, and no input once accepted is now refused, so no narrowing arm is owed either. What moves is the answer's VALUE on one path, in the last place for three or more fractional addends and beyond 2^53 for integers, toward SQLite's own arithmetic and the more accurate total, under triage's direction: a bug fix, patch per AGENTS.md. No BREAKING banner and no ADR-0087 marker is owed, and none is carried; check-changeset-no-major and check-adr-0087-registration were 0 locally, and Lint & Repo Gates, which carries them in CI, is in progress at the final read. A @objectstack/driver-sql line: not owed, the dev is right. AGENTS.md owes a changeset to "anything that publishes"; the driver-sql change is comment-only source and publishes nothing. AGGREGATE_ACCUMULATION is a non-exported const (sql-driver.ts:1610), so its docblock cannot reach dist/index.d.ts, and the JS bundle is esbuild through the root tsup.config.ts, which drops non-legal comments; that is consistent with the dev's 0-hit reading of dist/, which this review does not rebuild, and it is moot for semver, since a comment is not an API surface even where it ships. Structurally, .changeset/config.json has one fixed group of 69 that holds @objectstack/objectql, @objectstack/driver-sql and @objectstack/driver-memory, so driver-sql bumps in lockstep with the objectql patch, and the corrected 20387 note is itself a @objectstack/driver-sql: patch entry, so driver-sql's CHANGELOG will carry the residual naming #20489 in this same release; a second driver-sql changeset would say only what that corrected note already says.

The 20489 changeset, each sentence: the title, TRUE; Clause-②: no, TRUE (above); the two-path sentence (driver's own aggregate, or the rows path for a per-aggregation filter, a non-UTC date bucket, or a driver without native aggregation), TRUE by the file's own header and the spy, the filter trigger being the one proven here; "SQLite 3.43 and later adds with Kahan-Babuska-Neumaier compensation; the rows path added naively", TRUE; "So on SQLite one query answered two doubles depending on the path", TRUE; the table, TRUE (the card, the pins); "transcribed from SQLite's own, so on SQLite both paths answer the same double, through engine.aggregate and POST /api/v1/data/:object/query alike", TRUE (the REST pin); "1e16 + 1 - 1e16 is 1, where the naive fold answered 0", TRUE; the "Unchanged" sentence (two addends, integers within 2^53, a non-finite total, null and non-numeric cells, the empty group, a JS number), TRUE; the residual's four sentences: PostgreSQL / MySQL native without compensation and the database's own, TRUE; the last-place difference with the example, TRUE; "The @objectstack/driver-sql entry for the double accumulation states the same residual: ... PostgreSQL / MySQL native against SQLite and the rows path", TRUE at this head; the driver-memory sentence, TRUE (① item 5); "compare with a range", TRUE.

③ Boundary flags

Open question 1 (land as is, or widen to the hoist): the seat ruled A, and A is right. Triage scoped the card to the rows path; the claim's ⛔ forbids editing driver-memory; the family close-out is on #20544, which reads "serial after PR #20543" so the helper lands first. This PR lands.

Round 1 deviations 1-4: accepted, each verified. (1) The three branch commits carry Claude-Session: and Co-authored-by: Claude and no model-named trailer; AGENTS.md governs the repo's trailers, and the pre-push trailer check passed on each push. (2) The --maxWorkers=2 flag possibly dropped: process only; the whole package ran, and Test Core is the verdict (2/6 and 4/6 green, four shards in progress). (3) PostgreSQL / MySQL not measured at this head: stated as NOT MEASURED, the residual cites #20486's live measurement, and Temporal Conformance (live PG + MySQL) is green on this head; it runs the driver-sql suite live, whose 20387 cells pin two addends and integers by design, so the 3+ residual stays a stated one. (4) The stale text outside the surface: closed by the patch round, all three named texts corrected at this head.

Patch-round deviations 1-3. (1) The rowsPathSum / rowsPathAvg one-liners reworded beyond the admitted header sentence: comment-only, linked by the header, and TRUE; inside the admission's "comment text only, no logic moves". (2) The aggregate() inline comment left standing: judged in ① item 4, acceptable, acceptance note, carrier the next touch. (3) The worktree recreated and pnpm install re-run: process only.

Out-of-scope findings. Round 1 [0], the driver-memory family: filed as #20544, the right carrier (① item 5). Round 1 [1], the 20387 note's false sentence: done in the patch round as the DELIBERATE CORRECTION (① item 3). Patch round [0], the aggregate() inline comment: acceptable, carrier none (① item 4).

Added by this review, carrier the next touch of that file, not a FAIL: the 20387 test header's line 26 still says having-filter.ts compares with ==; record 5875498653 on PR #20486 judged that FALSE in letter (formula's looseEq, strict === for numbers, identical in effect) and said it "rides the next touch of that file". This PR is that next touch and did not carry it, since it was outside the admitted text. No pin depends on it.

PR-body sentences: TRUE, with three notes. (i) "a true merge of origin/main at 288611e3e (spec-only commits)" is TRUE of the two commits (05077d4c2, 288611e3e, both fix(spec)) and imprecise of the tree: that merge also brought a CLI test, scripts/regen-artifacts.mjs, .gitattributes, two changesets and the audit ledgers beside packages/spec (22 of 116 files); none is in this PR's net diff, so nothing here rests on it. (ii) The local suite, gate, ablation, lint, dist/ and randomized cross-check counts, and the three-engine SQLite readings, are the dev's transcript and are not re-run here (read-only); the check-runs below are the verdict. (iii) The H2 census beyond the objectql arms is measured, not re-measured, and carried to #20544. Each of these reproduces: Fixes #20489 as the sole closing keyword (the claim, single-writer and part-of gates green, each twice); Clause-②: no; the H1 base b2b6a0643; 8ad4a9d4a a true merge of 03b19d9cf on the branch's two commits; the first merge bringing no file under packages/objectql or packages/rest (0 files, verified); the comment-only proof (re-derived); "Check Changeset counts the changesets a PR adds" (the workflow's discriminator on A rows); "All 69 publishable packages are one fixed group" (one group, 69 members); the DELIBERATE CORRECTION section's quoted before-text (byte-equal to the merge-base blob) and its "only that paragraph" claim (one hunk); the line-16 reading (11 / 9 and 5 / 3 as the built rows path answers them, exact integer sums); "It was already an overclaim before this PR"; and the having readings before and after.

Governed surface: none (no .claude/**, docs/adr/**, skills/**, AGENTS.md or CLAUDE.md in the file list; Governed Surface Queue Guard green); 449 changed lines, under the human-merge threshold. The DELIBERATE CORRECTION needs this record regardless of tier.

Check-runs on 1c476788382b5de4e5aec4c5cebb71fe14452f53, final read 2026-09-29T02:08:55Z: 39 runs — 26 success, 5 skipped (Auto Label and Check PR Size on their second trigger, Build Docs, Console Pin Gate, Packed-tarball smoke: the rostered skips), 2 failure (both Check Changeset, the expected red on the corrected 20387 note, by design; not a required context, so it blocks no merge), 6 in_progress: Lint & Repo Gates, Test Core (1/6), (3/6), (5/6), (6/6), Type Check · workspace. Named gates green: Temporal Conformance (live PG + MySQL); Build Core; Type Check · source gates, consumer gates, debt ledger; Dogfood Regression Gate (1/3, 2/3, 3/3 and the roll-up); Dogfood Verify CLI; Governed Surface Queue Guard; Test Core (2/6), (4/6); Check Documentation Links; Flag docs affected by code changes; the claim, single-writer and part-of gates. No red other than Check Changeset. The PR is still a draft at this head, mergeable_state: blocked on the in-progress runs. The landing act reads the six in-progress runs to completion; a red among them re-owes this record.

Implemented-by: claude/issue-20489-rows-path-compensated-sum
Reviewed-by: session_01N8TPEsoJxPsdSdNKGnNGEN

VERDICT: PASS

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review September 29, 2026 02:17
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Sep 29, 2026
Merged via the queue into main with commit 8538edf Sep 29, 2026
41 of 43 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-20489-rows-path-compensated-sum branch September 29, 2026 02:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

2 participants