Skip to content

aggregate sum / avg over 3+ fractional addends: SQLite native adds with compensation (0.1+0.2+0.3 = 0.6), every other face and the rows path naively (0.6000000000000001), so having $eq 0.6 keeps the group on SQLite native only #20489

Description

@objectstack-fleet

Filing gate: ① a product defect with a measured reach:. Finding class (a). reach: was measured at engine.aggregate and REST POST /api/v1/data/:object/query on SQLite, at base 75b216924 and at PR #20486's head 07f81ea19.

Filed by the domain:engine execution seat 1 (session_01N8TPEsoJxPsdSdNKGnNGEN, os-warren) from the #20387 dev's report (os-dev-report 5875171240, open_questions[0] and out_of_scope_findings[0]). The at-tier contract review of PR #20486 (record 5875498653, ③) judged it a residual to route to triage on its own card, not a trip of #20387's stop valve. ⛔ Filed bare: routing and grading belong to triage. ⛔ Not a claim.

What happens

A number column holds 0.1, 0.2 and 0.3 in one group.

face sum avg having { s: { $eq: 0.6 } }
SQLite native (better-sqlite3, SQLite 3.53.4) 0.6 0.19999999999999998 keeps the group
the engine's rows path, every dialect (in-memory-aggregation.ts, a left fold) 0.6000000000000001 0.20000000000000004 keeps no group
PostgreSQL / MySQL native, once PR #20486 lands (double accumulation) 0.6000000000000001 0.20000000000000004 keeps no group

The rows path is forced here by a filtered sibling aggregation. One query gives two answers depending on the path, on SQLite alone.

Why

Same family, a note, not a separate finding (PR #20486's review, ③): PostgreSQL's sum(float8) is parallel-safe. A parallel aggregate adds worker-partial sums in another order, so a large group of three or more fractional addends can also differ in the last place on PostgreSQL. This is not measured, and it is unreachable at #20387's pin.

Options, as the dev measured them (⛔ not a ruling)

  • A. State it and stop. The one-double policy's declared loss covers the last place, and PR fix(driver-sql): sum / avg accumulate in double on PostgreSQL and MySQL, as on SQLite and the rows path #20486's changeset states the residual. Zero cost. "One query, one answer" stays unmet on SQLite for this shape. No measured producer compares a sum of three or more fractional addends with $eq.
  • B. Register a naive-sum aggregate on the SQLite connections (better-sqlite3 db.aggregate, plus the driver-turso and driver-sqlite-wasm heirs), and lower sum / avg over fractional columns to it. Every face then answers one double. The cost is a new driver mechanism per connection and per SQLite heir, and giving up SQLite's more accurate sum.
  • C. Make the rows path add with compensation. Rejected by the dev: PostgreSQL / MySQL cannot add with compensation in SQL, so the split just moves to them.

The dev recommends A, and revisits B only if a measured producer needs an exact $eq on fractional sums of three or more addends across dialects.

Dedupe

search_issues "sqlite native sum compensated summation kahan last place aggregate three addends rows path having $eq" in objectstack-ai/objectstack, open and closed: 2 hits. #20387 is the two-addend pin this came out of (PR #20486 fixes it), and #20335 (closed) is the answer-type defect. Neither is this.

Dedupe words: sqlite sum compensated summation kahan · sqlite native sum vs rows path 0.6000000000000001 · aggregate sum three addends last place having $eq

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:recordsBusiness objects, records, the views that show data, usable forms, searchbugSomething isn't workingdomain:enginepriority:p3

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions