Skip to content

perf: Optimize min(DISTINCT x) aggregations - #25126

Closed
mkleen wants to merge 1 commit into
apache:mainfrom
mkleen:min-max-distinct
Closed

mkleen wants to merge 1 commit into
apache:mainfrom
mkleen:min-max-distinct

Conversation

@mkleen

@mkleen mkleen commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Simplify min(DISTINCT x) to min(x) in a grouped query makes the query need 1000x less memory on the following setup:

SET datafusion.execution.target_partitions = 1;

CREATE TABLE t AS
SELECT v % 2000 AS g, (v * 48271) % 999983 AS x, v % 1000 AS y
FROM (SELECT unnest(generate_series(0, 3999999)) AS v);
query smallest limit that completes
SELECT g, min(DISTINCT x) FROM t GROUP BY g 192M, fails at 160M
SELECT g, min(x) FROM t GROUP BY g 192K, fails at 160K

The aggregation function min is duplicate insensitive, therefore we can do the following simplification:

  • min(DISTINCT x) is identical min(x)

This also prevents that the optimizer rule SingleDistinctToGroupBy fires which leads to a more efficient plan:

EXPLAIN FORMAT indent SELECT g, min(DISTINCT x) FROM t GROUP BY g;

Before this change:

Projection: t.g, min(alias1) AS min(DISTINCT t.x)
  Aggregate: groupBy=[[t.g]], aggr=[[min(alias1)]]
    Aggregate: groupBy=[[t.g, t.x AS alias1]], aggr=[[]]
      TableScan: t projection=[g, x]

after:

Aggregate: groupBy=[[t.g]], aggr=[[min(t.x) AS min(DISTINCT t.x)]]
   TableScan: t projection=[g, x]                                                 

Running SELECT g, min(DISTINCT x) FROM t GROUP BY g against both branches:

┌──────────────────┬─────────┬───────────────┐
│      version     │ median  │     range     │
├──────────────────┼─────────┼───────────────┤
│ main             │ 0.149 s │ 0.143–0.151 s │
├──────────────────┼─────────┼───────────────┤
│ branch           │ 0.013 s │ 0.013–0.016 s │
└──────────────────┴─────────┴───────────────┘ 

The query is roughly 11x faster with 1000x less memory usage.

DuckDB is following the same approach:

D explain SELECT g, min(DISTINCT x) FROM t GROUP BY g;

┌─────────────────────────────┐
│┌───────────────────────────┐│
││ Unoptimized Logical Plan  ││
│└───────────────────────────┘│
└─────────────────────────────┘
┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│        Expressions:       │
│             g             │
│      min(DISTINCT x)      │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         AGGREGATE         │
│    ────────────────────   │
│         Groups: g         │
│                           │
│        Expressions:       │
│      min(DISTINCT x)      │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│          SEQ_SCAN         │
│    ────────────────────   │
│          Table: t         │
│   Type: Sequential Scan   │
└───────────────────────────┘

┌─────────────────────────────┐
│┌───────────────────────────┐│
││  Optimized Logical Plan   ││
│└───────────────────────────┘│
└─────────────────────────────┘
┌───────────────────────────┐
│         PROJECTION        │
│    ────────────────────   │
│        Expressions:       │
│             g             │
│      min(DISTINCT x)      │
│                           │
│        ~2,322 rows        │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         PROJECTION        │
│    ────────────────────   │
│        Expressions:       │
│__internal_decompress_integ│
│     ral_bigint(#0, 0)     │
│             #1            │
│                           │
│        ~2,322 rows        │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         AGGREGATE         │
│    ────────────────────   │
│         Groups: g         │
│    Expressions: min(x)    │  <- distinct is removed 
│                           │
│        ~2,322 rows        │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│         PROJECTION        │
│    ────────────────────   │
│        Expressions:       │
│__internal_compress_integra│
│     l_usmallint(#0, 0)    │
│             #1            │
│                           │
│      ~4,000,000 rows      │
└─────────────┬─────────────┘
┌─────────────┴─────────────┐
│          SEQ_SCAN         │
│    ────────────────────   │
│          Table: t         │
│   Type: Sequential Scan   │
│                           │
│      ~4,000,000 rows      │
└───────────────────────────┘

What changes are included in this PR?

  • Simplify min(distinct x) to min(x)
  • Unalias the expression in SingleDistinctToGroupBy
  • Tests

What is the testing strategy for this PR?

  • Existing tests pass
  • New tests/slt added

Are there any user-facing changes?

More memory efficient queries

@mkleen mkleen changed the title feat: Remove distinct from min/max aggregations feat: Simplify min(distinct x)/max(distinct x) to min(x)/max(x) Sep 9, 2026
@mkleen mkleen changed the title feat: Simplify min(distinct x)/max(distinct x) to min(x)/max(x) feat: Simplify min(distinct x) to min(x) Sep 9, 2026
@mkleen mkleen changed the title feat: Simplify min(distinct x) to min(x) feat: Simplify min(DISTINCT x) to min(x) Sep 9, 2026
@github-actions github-actions Bot added optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Sep 9, 2026
@mkleen
mkleen force-pushed the min-max-distinct branch 2 times, most recently from 3da9184 to 532d434 Compare September 9, 2026 20:10
@codecov-commenter

codecov-commenter commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.91%. Comparing base (7e5f40a) to head (5b8a629).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main   #25126    +/-   ##
========================================
  Coverage   81.91%   81.91%            
========================================
  Files        1132     1132            
  Lines      421117   421273   +156     
  Branches   421117   421273   +156     
========================================
+ Hits       344961   345102   +141     
- Misses      55767    55771     +4     
- Partials    20389    20400    +11     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@mkleen mkleen changed the title feat: Simplify min(DISTINCT x) to min(x) feat: Avoid SingleDistinctToGroupBy to fire for grouped min(DISTINCT x) queries Sep 10, 2026
@mkleen mkleen changed the title feat: Avoid SingleDistinctToGroupBy to fire for grouped min(DISTINCT x) queries feat: Avoid SingleDistinctToGroupBy rule for grouped min(DISTINCT x) queries Sep 10, 2026
@mkleen
mkleen force-pushed the min-max-distinct branch 2 times, most recently from 003da84 to b2db45f Compare September 10, 2026 09:29
02)--Projection: aggregate_test_100.c1, count(alias1) AS count(DISTINCT aggregate_test_100.c2), min(alias1) AS min(DISTINCT aggregate_test_100.c2), sum(alias2) AS sum(aggregate_test_100.c3), max(alias3) AS max(aggregate_test_100.c4)
03)----Aggregate: groupBy=[[aggregate_test_100.c1]], aggr=[[count(alias1), min(alias1), sum(alias2), max(alias3)]]
04)------Aggregate: groupBy=[[aggregate_test_100.c1, aggregate_test_100.c2 AS alias1]], aggr=[[sum(CAST(aggregate_test_100.c3 AS Int64)) AS alias2, max(aggregate_test_100.c4) AS alias3]]
02)--Projection: aggregate_test_100.c1, count(alias1) AS count(DISTINCT aggregate_test_100.c2), min(alias2) AS min(DISTINCT aggregate_test_100.c2), sum(alias3) AS sum(aggregate_test_100.c3), max(alias4) AS max(aggregate_test_100.c4)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes are related due to an extra alias by this simplification.

@mkleen
mkleen marked this pull request as ready for review September 10, 2026 09:37
@mkleen mkleen changed the title feat: Avoid SingleDistinctToGroupBy rule for grouped min(DISTINCT x) queries feat: Make a grouped min(DISTINCT x) query use 1000x less memory Sep 10, 2026
@mkleen mkleen changed the title feat: Make a grouped min(DISTINCT x) query use 1000x less memory feat: Optimize grouped min(DISTINCT x) query Sep 10, 2026
@mkleen mkleen changed the title feat: Optimize grouped min(DISTINCT x) query feat: Optimize min(DISTINCT x) aggreations Sep 10, 2026
@mkleen mkleen changed the title feat: Optimize min(DISTINCT x) aggreations feat: Optimize min(DISTINCT x) aggregations Sep 10, 2026
@mkleen

mkleen commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

@adriangb Would you have time for a review? Thanks a lot!

@mkleen mkleen changed the title feat: Optimize min(DISTINCT x) aggregations perf: Optimize min(DISTINCT x) aggregations Sep 10, 2026
@mkleen

mkleen commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

run benchmarks

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5616665559-2304-2hcsn 6.12.94+ #1 SMP Fri Jul 17 09:42:57 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing min-max-distinct (db160ce) to 0015a75 (merge-base) diff

Run configuration
run benchmark tpch

Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5616665559-2303-t5s9b 6.12.94+ #1 SMP Fri Jul 17 09:42:57 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing min-max-distinct (db160ce) to 0015a75 (merge-base) diff

Run configuration
run benchmark tpcds

Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c5616665559-2302-m9rvt 6.12.94+ #1 SMP Fri Jul 17 09:42:57 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing min-max-distinct (db160ce) to 0015a75 (merge-base) diff

Run configuration
run benchmark clickbench_partitioned

Results will be posted here when complete


File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing min-max-distinct (db160ce) to 0015a75 (merge-base) diff

Run configuration
run benchmark tpch
CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details

Comparing HEAD and min-max-distinct
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Query     ┃     HEAD ┃ min-max-distinct ┃    Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ QQuery 1  │ 37.75 ms │         38.29 ms │ no change │
│ QQuery 2  │ 18.37 ms │         18.87 ms │ no change │
│ QQuery 3  │ 28.90 ms │         28.83 ms │ no change │
│ QQuery 4  │ 17.69 ms │         17.60 ms │ no change │
│ QQuery 5  │ 35.16 ms │         35.91 ms │ no change │
│ QQuery 6  │ 15.88 ms │         15.83 ms │ no change │
│ QQuery 7  │ 42.35 ms │         40.52 ms │ no change │
│ QQuery 8  │ 41.08 ms │         40.99 ms │ no change │
│ QQuery 9  │ 49.46 ms │         49.98 ms │ no change │
│ QQuery 10 │ 42.05 ms │         41.84 ms │ no change │
│ QQuery 11 │ 13.83 ms │         13.83 ms │ no change │
│ QQuery 12 │ 23.99 ms │         24.02 ms │ no change │
│ QQuery 13 │ 40.79 ms │         40.07 ms │ no change │
│ QQuery 14 │ 24.94 ms │         24.89 ms │ no change │
│ QQuery 15 │ 30.48 ms │         31.12 ms │ no change │
│ QQuery 16 │ 13.80 ms │         13.81 ms │ no change │
│ QQuery 17 │ 70.33 ms │         70.98 ms │ no change │
│ QQuery 18 │ 59.52 ms │         60.15 ms │ no change │
│ QQuery 19 │ 32.95 ms │         33.32 ms │ no change │
│ QQuery 20 │ 31.17 ms │         32.03 ms │ no change │
│ QQuery 21 │ 57.21 ms │         56.76 ms │ no change │
│ QQuery 22 │ 14.21 ms │         14.52 ms │ no change │
└───────────┴──────────┴──────────────────┴───────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary               ┃          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD)               │ 741.93ms │
│ Total Time (min-max-distinct)   │ 744.17ms │
│ Average Time (HEAD)             │  33.72ms │
│ Average Time (min-max-distinct) │  33.83ms │
│ Queries Faster                  │        0 │
│ Queries Slower                  │        0 │
│ Queries with No Change          │       22 │
│ Queries with Failure            │        0 │
└─────────────────────────────────┴──────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and min-max-distinct
--------------------
Benchmark tpch_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query     ┃                           HEAD ┃               min-max-distinct ┃       Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1  │ 37.75 / 40.18 ±2.69 / 44.91 ms │ 38.29 / 38.98 ±0.93 / 40.79 ms │    no change │
│ QQuery 2  │ 18.37 / 18.86 ±0.27 / 19.19 ms │ 18.87 / 19.44 ±0.33 / 19.82 ms │    no change │
│ QQuery 3  │ 28.90 / 29.38 ±0.33 / 29.76 ms │ 28.83 / 29.27 ±0.53 / 30.29 ms │    no change │
│ QQuery 4  │ 17.69 / 18.63 ±0.85 / 19.73 ms │ 17.60 / 17.93 ±0.31 / 18.41 ms │    no change │
│ QQuery 5  │ 35.16 / 37.01 ±1.85 / 40.49 ms │ 35.91 / 36.54 ±0.56 / 37.39 ms │    no change │
│ QQuery 6  │ 15.88 / 17.56 ±1.89 / 20.69 ms │ 15.83 / 17.02 ±1.29 / 19.02 ms │    no change │
│ QQuery 7  │ 42.35 / 44.58 ±1.27 / 46.30 ms │ 40.52 / 43.63 ±2.30 / 46.06 ms │    no change │
│ QQuery 8  │ 41.08 / 42.46 ±1.76 / 45.84 ms │ 40.99 / 41.75 ±0.88 / 43.48 ms │    no change │
│ QQuery 9  │ 49.46 / 51.08 ±1.41 / 52.89 ms │ 49.98 / 51.01 ±0.84 / 51.94 ms │    no change │
│ QQuery 10 │ 42.05 / 42.53 ±0.32 / 43.00 ms │ 41.84 / 42.79 ±0.60 / 43.43 ms │    no change │
│ QQuery 11 │ 13.83 / 14.09 ±0.32 / 14.71 ms │ 13.83 / 14.95 ±1.20 / 17.14 ms │ 1.06x slower │
│ QQuery 12 │ 23.99 / 24.89 ±1.06 / 26.92 ms │ 24.02 / 24.47 ±0.37 / 24.97 ms │    no change │
│ QQuery 13 │ 40.79 / 43.46 ±2.98 / 48.85 ms │ 40.07 / 41.30 ±1.48 / 44.15 ms │    no change │
│ QQuery 14 │ 24.94 / 25.44 ±0.29 / 25.85 ms │ 24.89 / 25.19 ±0.19 / 25.41 ms │    no change │
│ QQuery 15 │ 30.48 / 31.41 ±0.74 / 32.67 ms │ 31.12 / 32.08 ±0.90 / 33.78 ms │    no change │
│ QQuery 16 │ 13.80 / 14.12 ±0.19 / 14.30 ms │ 13.81 / 14.01 ±0.17 / 14.32 ms │    no change │
│ QQuery 17 │ 70.33 / 74.12 ±2.19 / 77.01 ms │ 70.98 / 72.82 ±2.93 / 78.64 ms │    no change │
│ QQuery 18 │ 59.52 / 61.30 ±1.85 / 64.76 ms │ 60.15 / 61.29 ±1.82 / 64.91 ms │    no change │
│ QQuery 19 │ 32.95 / 33.89 ±0.70 / 35.12 ms │ 33.32 / 34.23 ±0.89 / 35.64 ms │    no change │
│ QQuery 20 │ 31.17 / 31.76 ±0.63 / 32.96 ms │ 32.03 / 32.21 ±0.21 / 32.63 ms │    no change │
│ QQuery 21 │ 57.21 / 58.49 ±0.96 / 59.72 ms │ 56.76 / 57.37 ±0.57 / 58.24 ms │    no change │
│ QQuery 22 │ 14.21 / 14.59 ±0.25 / 14.90 ms │ 14.52 / 14.68 ±0.12 / 14.86 ms │    no change │
└───────────┴────────────────────────────────┴────────────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃ Benchmark Summary               ┃          ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ Total Time (HEAD)               │ 769.81ms │
│ Total Time (min-max-distinct)   │ 762.97ms │
│ Average Time (HEAD)             │  34.99ms │
│ Average Time (min-max-distinct) │  34.68ms │
│ Queries Faster                  │        0 │
│ Queries Slower                  │        1 │
│ Queries with No Change          │       21 │
│ Queries with Failure            │        0 │
└─────────────────────────────────┴──────────┘

Resource Usage

tpch — base (merge-base)

Metric Value
Wall time 5.0s
Peak memory 1.1 GiB
Avg memory 462.3 MiB
CPU user 21.5s
CPU sys 1.9s
Peak spill 0 B

tpch — branch

Metric Value
Wall time 5.0s
Peak memory 1.2 GiB
Avg memory 509.5 MiB
CPU user 21.3s
CPU sys 1.7s
Peak spill 0 B

File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing min-max-distinct (db160ce) to 0015a75 (merge-base) diff

Run configuration
run benchmark tpcds
CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details

Comparing HEAD and min-max-distinct
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ min-max-distinct ┃       Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 1  │    5.52 ms │          5.62 ms │    no change │
│ QQuery 2  │   82.60 ms │         80.10 ms │    no change │
│ QQuery 3  │   28.50 ms │         28.62 ms │    no change │
│ QQuery 4  │  474.85 ms │        475.58 ms │    no change │
│ QQuery 5  │   51.83 ms │         51.90 ms │    no change │
│ QQuery 6  │   35.70 ms │         35.91 ms │    no change │
│ QQuery 7  │   74.77 ms │         74.88 ms │    no change │
│ QQuery 8  │   35.71 ms │         36.11 ms │    no change │
│ QQuery 9  │   52.37 ms │         51.82 ms │    no change │
│ QQuery 10 │   61.94 ms │         62.38 ms │    no change │
│ QQuery 11 │  292.07 ms │        288.26 ms │    no change │
│ QQuery 12 │   28.46 ms │         27.95 ms │    no change │
│ QQuery 13 │  118.17 ms │        117.71 ms │    no change │
│ QQuery 14 │  414.95 ms │        416.26 ms │    no change │
│ QQuery 15 │   56.80 ms │         57.37 ms │    no change │
│ QQuery 16 │    6.69 ms │          6.81 ms │    no change │
│ QQuery 17 │   79.48 ms │         79.45 ms │    no change │
│ QQuery 18 │  102.39 ms │        104.80 ms │    no change │
│ QQuery 19 │   41.14 ms │         40.53 ms │    no change │
│ QQuery 20 │   35.02 ms │         35.01 ms │    no change │
│ QQuery 21 │   16.80 ms │         16.84 ms │    no change │
│ QQuery 22 │   63.14 ms │         62.92 ms │    no change │
│ QQuery 23 │  308.07 ms │        308.98 ms │    no change │
│ QQuery 24 │  190.96 ms │        192.59 ms │    no change │
│ QQuery 25 │  108.89 ms │        109.57 ms │    no change │
│ QQuery 26 │   47.98 ms │         48.42 ms │    no change │
│ QQuery 27 │    6.08 ms │          6.06 ms │    no change │
│ QQuery 28 │   56.59 ms │         60.29 ms │ 1.07x slower │
│ QQuery 29 │   98.30 ms │         96.91 ms │    no change │
│ QQuery 30 │   32.17 ms │         31.85 ms │    no change │
│ QQuery 31 │  110.16 ms │        110.63 ms │    no change │
│ QQuery 32 │   20.00 ms │         20.17 ms │    no change │
│ QQuery 33 │   37.62 ms │         37.66 ms │    no change │
│ QQuery 34 │    9.82 ms │          9.76 ms │    no change │
│ QQuery 35 │   71.97 ms │         72.11 ms │    no change │
│ QQuery 36 │    5.63 ms │          5.85 ms │    no change │
│ QQuery 37 │    6.62 ms │          6.84 ms │    no change │
│ QQuery 38 │   61.05 ms │         61.54 ms │    no change │
│ QQuery 39 │   89.11 ms │         88.28 ms │    no change │
│ QQuery 40 │   23.47 ms │         23.76 ms │    no change │
│ QQuery 41 │   11.04 ms │         10.96 ms │    no change │
│ QQuery 42 │   23.52 ms │         23.45 ms │    no change │
│ QQuery 43 │    5.05 ms │          5.01 ms │    no change │
│ QQuery 44 │    9.31 ms │          9.25 ms │    no change │
│ QQuery 45 │   38.76 ms │         38.76 ms │    no change │
│ QQuery 46 │   12.09 ms │         11.95 ms │    no change │
│ QQuery 47 │  224.61 ms │        223.25 ms │    no change │
│ QQuery 48 │   95.01 ms │         94.70 ms │    no change │
│ QQuery 49 │   70.27 ms │         70.60 ms │    no change │
│ QQuery 50 │   58.66 ms │         58.21 ms │    no change │
│ QQuery 51 │   91.20 ms │         90.75 ms │    no change │
│ QQuery 52 │   23.71 ms │         23.52 ms │    no change │
│ QQuery 53 │   28.78 ms │         28.76 ms │    no change │
│ QQuery 54 │   54.20 ms │         54.07 ms │    no change │
│ QQuery 55 │   22.81 ms │         23.10 ms │    no change │
│ QQuery 56 │   38.14 ms │         38.33 ms │    no change │
│ QQuery 57 │  173.82 ms │        174.44 ms │    no change │
│ QQuery 58 │  110.77 ms │        111.65 ms │    no change │
│ QQuery 59 │  118.84 ms │        117.28 ms │    no change │
│ QQuery 60 │   38.47 ms │         38.95 ms │    no change │
│ QQuery 61 │   12.26 ms │         12.14 ms │    no change │
│ QQuery 62 │   45.59 ms │         45.64 ms │    no change │
│ QQuery 63 │   29.35 ms │         28.86 ms │    no change │
│ QQuery 64 │  361.04 ms │        365.46 ms │    no change │
│ QQuery 65 │  121.21 ms │        121.73 ms │    no change │
│ QQuery 66 │   78.29 ms │         78.28 ms │    no change │
│ QQuery 67 │  238.65 ms │        239.64 ms │    no change │
│ QQuery 68 │   11.81 ms │         12.01 ms │    no change │
│ QQuery 69 │   56.29 ms │         56.63 ms │    no change │
│ QQuery 70 │  105.22 ms │        104.98 ms │    no change │
│ QQuery 71 │   35.07 ms │         34.95 ms │    no change │
│ QQuery 72 │ 1792.10 ms │       1857.20 ms │    no change │
│ QQuery 73 │    9.78 ms │          9.58 ms │    no change │
│ QQuery 74 │  166.30 ms │        168.21 ms │    no change │
│ QQuery 75 │  145.53 ms │        146.75 ms │    no change │
│ QQuery 76 │   35.14 ms │         35.67 ms │    no change │
│ QQuery 77 │   60.31 ms │         61.47 ms │    no change │
│ QQuery 78 │  219.19 ms │        222.00 ms │    no change │
│ QQuery 79 │   66.00 ms │         66.40 ms │    no change │
│ QQuery 80 │   96.77 ms │        100.17 ms │    no change │
│ QQuery 81 │   25.43 ms │         26.01 ms │    no change │
│ QQuery 82 │   15.94 ms │         16.06 ms │    no change │
│ QQuery 83 │   33.63 ms │         34.11 ms │    no change │
│ QQuery 84 │   29.12 ms │         29.45 ms │    no change │
│ QQuery 85 │  102.64 ms │        102.83 ms │    no change │
│ QQuery 86 │   24.53 ms │         24.85 ms │    no change │
│ QQuery 87 │   60.66 ms │         61.32 ms │    no change │
│ QQuery 88 │   62.23 ms │         63.05 ms │    no change │
│ QQuery 89 │   35.14 ms │         35.03 ms │    no change │
│ QQuery 90 │   16.89 ms │         16.95 ms │    no change │
│ QQuery 91 │   44.47 ms │         44.54 ms │    no change │
│ QQuery 92 │   29.15 ms │         28.82 ms │    no change │
│ QQuery 93 │   49.89 ms │         49.89 ms │    no change │
│ QQuery 94 │   38.06 ms │         37.22 ms │    no change │
│ QQuery 95 │   80.06 ms │         79.84 ms │    no change │
│ QQuery 96 │   23.65 ms │         23.94 ms │    no change │
│ QQuery 97 │   52.26 ms │         52.37 ms │    no change │
│ QQuery 98 │   41.95 ms │         42.47 ms │    no change │
│ QQuery 99 │   69.26 ms │         69.65 ms │    no change │
└───────────┴────────────┴──────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary               ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)               │ 9213.29ms │
│ Total Time (min-max-distinct)   │ 9301.25ms │
│ Average Time (HEAD)             │   93.06ms │
│ Average Time (min-max-distinct) │   93.95ms │
│ Queries Faster                  │         0 │
│ Queries Slower                  │         1 │
│ Queries with No Change          │        98 │
│ Queries with Failure            │         0 │
└─────────────────────────────────┴───────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and min-max-distinct
--------------------
Benchmark tpcds_sf1.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                  HEAD ┃                      min-max-distinct ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1  │           5.52 / 6.04 ±0.97 / 7.99 ms │           5.62 / 6.12 ±0.93 / 7.97 ms │     no change │
│ QQuery 2  │        82.60 / 82.84 ±0.27 / 83.19 ms │        80.10 / 80.58 ±0.28 / 80.92 ms │     no change │
│ QQuery 3  │        28.50 / 28.82 ±0.21 / 29.15 ms │        28.62 / 28.80 ±0.16 / 29.08 ms │     no change │
│ QQuery 4  │     474.85 / 477.03 ±2.13 / 480.44 ms │     475.58 / 478.94 ±2.88 / 483.17 ms │     no change │
│ QQuery 5  │        51.83 / 52.54 ±0.46 / 53.07 ms │        51.90 / 52.85 ±0.84 / 54.09 ms │     no change │
│ QQuery 6  │        35.70 / 36.39 ±0.45 / 37.09 ms │        35.91 / 36.26 ±0.24 / 36.60 ms │     no change │
│ QQuery 7  │        74.77 / 75.05 ±0.18 / 75.25 ms │        74.88 / 75.35 ±0.45 / 76.18 ms │     no change │
│ QQuery 8  │        35.71 / 36.11 ±0.24 / 36.40 ms │        36.11 / 36.41 ±0.29 / 36.96 ms │     no change │
│ QQuery 9  │        52.37 / 54.74 ±2.44 / 59.04 ms │        51.82 / 53.71 ±1.96 / 56.57 ms │     no change │
│ QQuery 10 │        61.94 / 62.57 ±0.33 / 62.89 ms │        62.38 / 62.47 ±0.11 / 62.68 ms │     no change │
│ QQuery 11 │     292.07 / 298.14 ±5.66 / 308.26 ms │     288.26 / 293.04 ±3.46 / 297.30 ms │     no change │
│ QQuery 12 │        28.46 / 28.95 ±0.41 / 29.44 ms │        27.95 / 28.61 ±0.42 / 29.11 ms │     no change │
│ QQuery 13 │     118.17 / 119.39 ±1.35 / 121.95 ms │     117.71 / 118.42 ±0.48 / 119.13 ms │     no change │
│ QQuery 14 │     414.95 / 422.02 ±4.67 / 429.21 ms │     416.26 / 420.85 ±3.77 / 426.04 ms │     no change │
│ QQuery 15 │        56.80 / 58.68 ±2.26 / 63.06 ms │        57.37 / 59.36 ±2.27 / 63.78 ms │     no change │
│ QQuery 16 │           6.69 / 6.81 ±0.13 / 7.05 ms │           6.81 / 6.95 ±0.17 / 7.28 ms │     no change │
│ QQuery 17 │        79.48 / 80.49 ±1.34 / 83.12 ms │        79.45 / 80.90 ±1.46 / 83.68 ms │     no change │
│ QQuery 18 │     102.39 / 104.26 ±1.47 / 106.02 ms │     104.80 / 106.45 ±1.54 / 108.37 ms │     no change │
│ QQuery 19 │        41.14 / 41.32 ±0.16 / 41.53 ms │        40.53 / 41.37 ±0.43 / 41.76 ms │     no change │
│ QQuery 20 │        35.02 / 35.88 ±0.57 / 36.54 ms │        35.01 / 35.65 ±0.44 / 36.29 ms │     no change │
│ QQuery 21 │        16.80 / 17.06 ±0.25 / 17.48 ms │        16.84 / 17.23 ±0.29 / 17.55 ms │     no change │
│ QQuery 22 │        63.14 / 63.49 ±0.35 / 64.08 ms │        62.92 / 63.21 ±0.27 / 63.62 ms │     no change │
│ QQuery 23 │     308.07 / 310.86 ±3.59 / 317.79 ms │     308.98 / 312.21 ±4.82 / 321.78 ms │     no change │
│ QQuery 24 │     190.96 / 193.05 ±1.92 / 196.38 ms │     192.59 / 197.15 ±4.92 / 206.51 ms │     no change │
│ QQuery 25 │     108.89 / 110.26 ±1.69 / 113.02 ms │     109.57 / 113.01 ±4.19 / 121.25 ms │     no change │
│ QQuery 26 │        47.98 / 49.15 ±1.30 / 51.61 ms │        48.42 / 49.39 ±0.88 / 50.76 ms │     no change │
│ QQuery 27 │           6.08 / 6.31 ±0.24 / 6.78 ms │           6.06 / 6.21 ±0.19 / 6.51 ms │     no change │
│ QQuery 28 │        56.59 / 59.91 ±1.68 / 61.05 ms │        60.29 / 61.01 ±0.46 / 61.53 ms │     no change │
│ QQuery 29 │       98.30 / 99.83 ±1.45 / 102.60 ms │       96.91 / 99.71 ±2.28 / 103.67 ms │     no change │
│ QQuery 30 │        32.17 / 32.50 ±0.44 / 33.33 ms │        31.85 / 32.30 ±0.26 / 32.61 ms │     no change │
│ QQuery 31 │     110.16 / 111.67 ±2.38 / 116.39 ms │     110.63 / 112.38 ±2.17 / 116.21 ms │     no change │
│ QQuery 32 │        20.00 / 20.43 ±0.44 / 21.07 ms │        20.17 / 20.37 ±0.12 / 20.52 ms │     no change │
│ QQuery 33 │        37.62 / 38.07 ±0.42 / 38.84 ms │        37.66 / 37.97 ±0.21 / 38.27 ms │     no change │
│ QQuery 34 │         9.82 / 10.07 ±0.27 / 10.58 ms │         9.76 / 10.02 ±0.25 / 10.37 ms │     no change │
│ QQuery 35 │        71.97 / 72.42 ±0.39 / 72.97 ms │        72.11 / 72.72 ±0.38 / 73.10 ms │     no change │
│ QQuery 36 │           5.63 / 5.80 ±0.24 / 6.26 ms │           5.85 / 6.01 ±0.19 / 6.37 ms │     no change │
│ QQuery 37 │           6.62 / 6.80 ±0.09 / 6.87 ms │           6.84 / 6.87 ±0.03 / 6.90 ms │     no change │
│ QQuery 38 │        61.05 / 61.57 ±0.35 / 62.15 ms │        61.54 / 63.14 ±1.83 / 66.68 ms │     no change │
│ QQuery 39 │        89.11 / 89.73 ±0.65 / 90.89 ms │        88.28 / 89.68 ±0.86 / 90.51 ms │     no change │
│ QQuery 40 │        23.47 / 23.63 ±0.13 / 23.78 ms │        23.76 / 23.88 ±0.08 / 23.98 ms │     no change │
│ QQuery 41 │        11.04 / 11.16 ±0.11 / 11.37 ms │        10.96 / 11.11 ±0.13 / 11.33 ms │     no change │
│ QQuery 42 │        23.52 / 23.77 ±0.23 / 24.16 ms │        23.45 / 23.66 ±0.28 / 24.22 ms │     no change │
│ QQuery 43 │           5.05 / 5.18 ±0.16 / 5.50 ms │           5.01 / 5.14 ±0.12 / 5.36 ms │     no change │
│ QQuery 44 │           9.31 / 9.37 ±0.06 / 9.47 ms │         9.25 / 10.45 ±2.16 / 14.76 ms │  1.11x slower │
│ QQuery 45 │        38.76 / 40.11 ±1.60 / 43.23 ms │        38.76 / 39.22 ±0.56 / 40.16 ms │     no change │
│ QQuery 46 │        12.09 / 12.41 ±0.46 / 13.31 ms │        11.95 / 12.17 ±0.18 / 12.47 ms │     no change │
│ QQuery 47 │     224.61 / 227.62 ±2.87 / 231.29 ms │     223.25 / 226.33 ±2.02 / 229.45 ms │     no change │
│ QQuery 48 │        95.01 / 95.47 ±0.29 / 95.82 ms │        94.70 / 95.35 ±0.42 / 96.00 ms │     no change │
│ QQuery 49 │        70.27 / 72.25 ±2.01 / 75.86 ms │        70.60 / 71.30 ±0.44 / 71.71 ms │     no change │
│ QQuery 50 │        58.66 / 59.61 ±1.40 / 62.38 ms │        58.21 / 58.83 ±0.66 / 60.08 ms │     no change │
│ QQuery 51 │        91.20 / 93.42 ±1.40 / 95.56 ms │        90.75 / 91.94 ±0.71 / 92.96 ms │     no change │
│ QQuery 52 │        23.71 / 25.27 ±2.61 / 30.49 ms │        23.52 / 24.32 ±0.51 / 25.04 ms │     no change │
│ QQuery 53 │        28.78 / 29.15 ±0.33 / 29.59 ms │        28.76 / 28.97 ±0.20 / 29.29 ms │     no change │
│ QQuery 54 │        54.20 / 55.45 ±1.20 / 57.65 ms │        54.07 / 54.95 ±1.11 / 57.06 ms │     no change │
│ QQuery 55 │        22.81 / 23.19 ±0.24 / 23.54 ms │        23.10 / 23.35 ±0.18 / 23.66 ms │     no change │
│ QQuery 56 │        38.14 / 38.61 ±0.30 / 39.07 ms │        38.33 / 38.62 ±0.31 / 39.11 ms │     no change │
│ QQuery 57 │     173.82 / 175.20 ±2.07 / 179.28 ms │     174.44 / 177.41 ±2.58 / 181.58 ms │     no change │
│ QQuery 58 │     110.77 / 112.50 ±1.68 / 115.68 ms │     111.65 / 113.10 ±1.14 / 114.98 ms │     no change │
│ QQuery 59 │     118.84 / 119.95 ±0.89 / 121.42 ms │     117.28 / 117.69 ±0.46 / 118.49 ms │     no change │
│ QQuery 60 │        38.47 / 39.17 ±0.36 / 39.49 ms │        38.95 / 39.82 ±0.73 / 40.73 ms │     no change │
│ QQuery 61 │        12.26 / 13.32 ±1.71 / 16.71 ms │        12.14 / 12.32 ±0.19 / 12.68 ms │ +1.08x faster │
│ QQuery 62 │        45.59 / 46.56 ±1.08 / 48.65 ms │        45.64 / 46.48 ±0.88 / 48.10 ms │     no change │
│ QQuery 63 │        29.35 / 29.60 ±0.18 / 29.91 ms │        28.86 / 29.02 ±0.14 / 29.20 ms │     no change │
│ QQuery 64 │     361.04 / 364.66 ±2.37 / 368.35 ms │     365.46 / 369.82 ±6.08 / 381.60 ms │     no change │
│ QQuery 65 │     121.21 / 124.21 ±1.82 / 126.25 ms │     121.73 / 123.95 ±1.60 / 126.64 ms │     no change │
│ QQuery 66 │        78.29 / 79.75 ±1.21 / 81.58 ms │        78.28 / 79.55 ±1.39 / 82.22 ms │     no change │
│ QQuery 67 │     238.65 / 241.78 ±2.34 / 245.66 ms │     239.64 / 244.97 ±3.54 / 250.07 ms │     no change │
│ QQuery 68 │        11.81 / 12.01 ±0.18 / 12.32 ms │        12.01 / 12.22 ±0.15 / 12.48 ms │     no change │
│ QQuery 69 │        56.29 / 56.68 ±0.25 / 57.07 ms │        56.63 / 57.25 ±0.35 / 57.70 ms │     no change │
│ QQuery 70 │     105.22 / 108.35 ±3.71 / 115.26 ms │     104.98 / 110.98 ±5.80 / 119.67 ms │     no change │
│ QQuery 71 │        35.07 / 35.69 ±0.68 / 36.76 ms │        34.95 / 35.19 ±0.23 / 35.50 ms │     no change │
│ QQuery 72 │ 1792.10 / 1826.14 ±34.92 / 1892.15 ms │ 1857.20 / 1955.62 ±57.33 / 2028.02 ms │  1.07x slower │
│ QQuery 73 │         9.78 / 10.28 ±0.35 / 10.67 ms │         9.58 / 10.08 ±0.41 / 10.52 ms │     no change │
│ QQuery 74 │     166.30 / 170.17 ±2.11 / 172.57 ms │     168.21 / 171.56 ±2.18 / 174.52 ms │     no change │
│ QQuery 75 │     145.53 / 148.58 ±3.23 / 154.22 ms │     146.75 / 148.07 ±1.55 / 150.96 ms │     no change │
│ QQuery 76 │        35.14 / 35.68 ±0.46 / 36.40 ms │        35.67 / 35.99 ±0.28 / 36.35 ms │     no change │
│ QQuery 77 │        60.31 / 60.77 ±0.28 / 61.15 ms │        61.47 / 62.25 ±1.02 / 64.26 ms │     no change │
│ QQuery 78 │     219.19 / 225.70 ±5.79 / 235.00 ms │     222.00 / 227.22 ±6.50 / 238.35 ms │     no change │
│ QQuery 79 │        66.00 / 69.07 ±4.05 / 76.98 ms │        66.40 / 67.12 ±0.52 / 67.90 ms │     no change │
│ QQuery 80 │       96.77 / 99.24 ±1.79 / 102.18 ms │     100.17 / 103.84 ±5.80 / 115.38 ms │     no change │
│ QQuery 81 │        25.43 / 25.72 ±0.25 / 26.06 ms │        26.01 / 26.25 ±0.15 / 26.46 ms │     no change │
│ QQuery 82 │        15.94 / 16.10 ±0.15 / 16.33 ms │        16.06 / 16.37 ±0.19 / 16.61 ms │     no change │
│ QQuery 83 │        33.63 / 35.33 ±2.77 / 40.87 ms │        34.11 / 34.38 ±0.23 / 34.80 ms │     no change │
│ QQuery 84 │        29.12 / 29.65 ±0.38 / 30.20 ms │        29.45 / 31.10 ±2.39 / 35.85 ms │     no change │
│ QQuery 85 │     102.64 / 104.12 ±1.53 / 106.98 ms │     102.83 / 104.15 ±1.06 / 105.90 ms │     no change │
│ QQuery 86 │        24.53 / 25.17 ±0.43 / 25.78 ms │        24.85 / 25.37 ±0.28 / 25.64 ms │     no change │
│ QQuery 87 │        60.66 / 62.66 ±2.59 / 67.75 ms │        61.32 / 62.45 ±0.66 / 63.22 ms │     no change │
│ QQuery 88 │        62.23 / 62.77 ±0.36 / 63.34 ms │        63.05 / 64.30 ±2.24 / 68.78 ms │     no change │
│ QQuery 89 │        35.14 / 35.65 ±0.38 / 36.25 ms │        35.03 / 36.21 ±1.35 / 38.72 ms │     no change │
│ QQuery 90 │        16.89 / 17.04 ±0.18 / 17.33 ms │        16.95 / 17.20 ±0.20 / 17.48 ms │     no change │
│ QQuery 91 │        44.47 / 46.22 ±2.80 / 51.80 ms │        44.54 / 44.88 ±0.20 / 45.08 ms │     no change │
│ QQuery 92 │        29.15 / 29.86 ±0.54 / 30.50 ms │        28.82 / 29.40 ±0.33 / 29.80 ms │     no change │
│ QQuery 93 │        49.89 / 50.75 ±1.03 / 52.78 ms │        49.89 / 50.84 ±0.94 / 52.50 ms │     no change │
│ QQuery 94 │        38.06 / 38.40 ±0.29 / 38.76 ms │        37.22 / 38.53 ±1.35 / 41.08 ms │     no change │
│ QQuery 95 │        80.06 / 80.63 ±0.53 / 81.61 ms │        79.84 / 81.44 ±1.13 / 83.15 ms │     no change │
│ QQuery 96 │        23.65 / 24.77 ±1.67 / 28.09 ms │        23.94 / 24.20 ±0.18 / 24.45 ms │     no change │
│ QQuery 97 │        52.26 / 53.38 ±0.92 / 54.93 ms │        52.37 / 53.86 ±1.63 / 56.80 ms │     no change │
│ QQuery 98 │        41.95 / 42.87 ±0.63 / 43.74 ms │        42.47 / 43.01 ±0.32 / 43.35 ms │     no change │
│ QQuery 99 │        69.26 / 69.78 ±0.40 / 70.35 ms │        69.65 / 70.29 ±0.56 / 71.32 ms │     no change │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary               ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)               │ 9368.71ms │
│ Total Time (min-max-distinct)   │ 9519.09ms │
│ Average Time (HEAD)             │   94.63ms │
│ Average Time (min-max-distinct) │   96.15ms │
│ Queries Faster                  │         1 │
│ Queries Slower                  │         2 │
│ Queries with No Change          │        96 │
│ Queries with Failure            │         0 │
└─────────────────────────────────┴───────────┘

Resource Usage

tpcds — base (merge-base)

Metric Value
Wall time 50.0s
Peak memory 2.1 GiB
Avg memory 1.4 GiB
CPU user 203.9s
CPU sys 5.7s
Peak spill 0 B

tpcds — branch

Metric Value
Wall time 50.0s
Peak memory 1.9 GiB
Avg memory 1.3 GiB
CPU user 204.9s
CPU sys 5.5s
Peak spill 0 B

File an issue against this benchmark runner

@adriangbot

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing min-max-distinct (db160ce) to 0015a75 (merge-base) diff

Run configuration
run benchmark clickbench_partitioned
CPU Details (lscpu)
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details

Comparing HEAD and min-max-distinct
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ min-max-distinct ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │    1.40 ms │          1.25 ms │ +1.12x faster │
│ QQuery 1  │   12.69 ms │         11.86 ms │ +1.07x faster │
│ QQuery 2  │   37.92 ms │         36.22 ms │     no change │
│ QQuery 3  │   32.97 ms │         30.83 ms │ +1.07x faster │
│ QQuery 4  │  267.03 ms │        222.53 ms │ +1.20x faster │
│ QQuery 5  │  279.25 ms │        273.93 ms │     no change │
│ QQuery 6  │    1.29 ms │          1.25 ms │     no change │
│ QQuery 7  │   13.08 ms │         12.99 ms │     no change │
│ QQuery 8  │  326.57 ms │        327.31 ms │     no change │
│ QQuery 9  │  449.15 ms │        450.61 ms │     no change │
│ QQuery 10 │   75.86 ms │         68.52 ms │ +1.11x faster │
│ QQuery 11 │   87.82 ms │         80.00 ms │ +1.10x faster │
│ QQuery 12 │  268.17 ms │        265.71 ms │     no change │
│ QQuery 13 │  361.32 ms │        360.60 ms │     no change │
│ QQuery 14 │  282.53 ms │        281.94 ms │     no change │
│ QQuery 15 │  271.46 ms │        273.50 ms │     no change │
│ QQuery 16 │  616.56 ms │        708.69 ms │  1.15x slower │
│ QQuery 17 │  619.20 ms │        715.94 ms │  1.16x slower │
│ QQuery 18 │ 1277.68 ms │       1292.63 ms │     no change │
│ QQuery 19 │   27.25 ms │         29.02 ms │  1.07x slower │
│ QQuery 20 │  518.39 ms │        520.19 ms │     no change │
│ QQuery 21 │  514.81 ms │        513.42 ms │     no change │
│ QQuery 22 │  989.51 ms │        998.29 ms │     no change │
│ QQuery 23 │ 3082.45 ms │       3060.30 ms │     no change │
│ QQuery 24 │   43.04 ms │         40.89 ms │ +1.05x faster │
│ QQuery 25 │  117.00 ms │        110.68 ms │ +1.06x faster │
│ QQuery 26 │   43.61 ms │         41.08 ms │ +1.06x faster │
│ QQuery 27 │  541.31 ms │        521.59 ms │     no change │
│ QQuery 28 │ 2920.87 ms │       2895.37 ms │     no change │
│ QQuery 29 │   41.51 ms │         41.70 ms │     no change │
│ QQuery 30 │  305.99 ms │        311.07 ms │     no change │
│ QQuery 31 │  286.12 ms │        313.43 ms │  1.10x slower │
│ QQuery 32 │  957.78 ms │       1122.58 ms │  1.17x slower │
│ QQuery 33 │ 1468.70 ms │       1460.78 ms │     no change │
│ QQuery 34 │ 1485.24 ms │       1469.66 ms │     no change │
│ QQuery 35 │  320.84 ms │        284.50 ms │ +1.13x faster │
│ QQuery 36 │   69.37 ms │         66.05 ms │     no change │
│ QQuery 37 │   35.98 ms │         35.86 ms │     no change │
│ QQuery 38 │   40.07 ms │         45.47 ms │  1.13x slower │
│ QQuery 39 │  142.18 ms │        143.74 ms │     no change │
│ QQuery 40 │   13.81 ms │         14.10 ms │     no change │
│ QQuery 41 │   13.47 ms │         13.49 ms │     no change │
│ QQuery 42 │   12.96 ms │         12.93 ms │     no change │
└───────────┴────────────┴──────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary               ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)               │ 19274.21ms │
│ Total Time (min-max-distinct)   │ 19482.48ms │
│ Average Time (HEAD)             │   448.24ms │
│ Average Time (min-max-distinct) │   453.08ms │
│ Queries Faster                  │         10 │
│ Queries Slower                  │          6 │
│ Queries with No Change          │         27 │
│ Queries with Failure            │          0 │
└─────────────────────────────────┴────────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and min-max-distinct
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                  HEAD ┃                      min-max-distinct ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │          1.40 / 4.41 ±5.83 / 16.07 ms │          1.25 / 4.01 ±5.45 / 14.91 ms │ +1.10x faster │
│ QQuery 1  │        12.69 / 12.99 ±0.24 / 13.23 ms │        11.86 / 11.96 ±0.09 / 12.07 ms │ +1.09x faster │
│ QQuery 2  │        37.92 / 38.92 ±1.45 / 41.79 ms │        36.22 / 36.49 ±0.22 / 36.76 ms │ +1.07x faster │
│ QQuery 3  │        32.97 / 33.47 ±0.70 / 34.85 ms │        30.83 / 31.31 ±0.30 / 31.67 ms │ +1.07x faster │
│ QQuery 4  │     267.03 / 268.93 ±2.13 / 272.82 ms │     222.53 / 226.87 ±2.83 / 230.45 ms │ +1.19x faster │
│ QQuery 5  │    279.25 / 294.25 ±12.92 / 311.12 ms │     273.93 / 275.98 ±1.94 / 279.49 ms │ +1.07x faster │
│ QQuery 6  │           1.29 / 1.43 ±0.23 / 1.88 ms │           1.25 / 1.40 ±0.22 / 1.83 ms │     no change │
│ QQuery 7  │        13.08 / 13.27 ±0.13 / 13.46 ms │        12.99 / 13.20 ±0.18 / 13.50 ms │     no change │
│ QQuery 8  │     326.57 / 331.28 ±2.71 / 333.76 ms │     327.31 / 328.91 ±1.77 / 332.28 ms │     no change │
│ QQuery 9  │    449.15 / 485.09 ±35.03 / 530.39 ms │     450.61 / 457.14 ±4.32 / 463.66 ms │ +1.06x faster │
│ QQuery 10 │        75.86 / 77.01 ±1.09 / 78.62 ms │        68.52 / 71.47 ±4.73 / 80.88 ms │ +1.08x faster │
│ QQuery 11 │        87.82 / 88.92 ±0.88 / 90.06 ms │        80.00 / 80.73 ±0.74 / 81.81 ms │ +1.10x faster │
│ QQuery 12 │    268.17 / 288.95 ±19.28 / 320.90 ms │     265.71 / 274.03 ±5.88 / 281.85 ms │ +1.05x faster │
│ QQuery 13 │    361.32 / 383.28 ±17.29 / 408.44 ms │     360.60 / 372.44 ±9.01 / 383.10 ms │     no change │
│ QQuery 14 │     282.53 / 290.53 ±8.62 / 306.10 ms │     281.94 / 290.14 ±9.09 / 307.52 ms │     no change │
│ QQuery 15 │     271.46 / 276.81 ±4.30 / 282.97 ms │    273.50 / 305.23 ±34.78 / 366.40 ms │  1.10x slower │
│ QQuery 16 │     616.56 / 622.67 ±3.90 / 628.01 ms │     708.69 / 712.22 ±5.43 / 723.04 ms │  1.14x slower │
│ QQuery 17 │    619.20 / 636.81 ±21.06 / 677.79 ms │     715.94 / 725.19 ±9.39 / 740.28 ms │  1.14x slower │
│ QQuery 18 │ 1277.68 / 1333.32 ±45.09 / 1412.00 ms │ 1292.63 / 1350.10 ±74.79 / 1491.30 ms │     no change │
│ QQuery 19 │       27.25 / 37.81 ±14.71 / 65.49 ms │        29.02 / 30.26 ±0.91 / 31.35 ms │ +1.25x faster │
│ QQuery 20 │    518.39 / 528.89 ±11.36 / 545.09 ms │    520.19 / 544.68 ±17.55 / 572.89 ms │     no change │
│ QQuery 21 │     514.81 / 522.05 ±5.49 / 530.07 ms │     513.42 / 524.65 ±6.87 / 532.37 ms │     no change │
│ QQuery 22 │    989.51 / 999.72 ±7.12 / 1010.01 ms │   998.29 / 1007.08 ±9.46 / 1025.40 ms │     no change │
│ QQuery 23 │ 3082.45 / 3138.86 ±54.77 / 3239.12 ms │ 3060.30 / 3117.72 ±45.53 / 3199.29 ms │     no change │
│ QQuery 24 │        43.04 / 49.06 ±6.99 / 61.73 ms │        40.89 / 42.28 ±2.20 / 46.57 ms │ +1.16x faster │
│ QQuery 25 │     117.00 / 118.35 ±1.35 / 120.73 ms │     110.68 / 112.76 ±1.82 / 115.20 ms │     no change │
│ QQuery 26 │        43.61 / 45.56 ±2.55 / 50.43 ms │        41.08 / 41.93 ±1.08 / 44.06 ms │ +1.09x faster │
│ QQuery 27 │     541.31 / 547.33 ±4.76 / 555.61 ms │     521.59 / 534.65 ±8.10 / 543.40 ms │     no change │
│ QQuery 28 │ 2920.87 / 3011.23 ±64.98 / 3087.36 ms │ 2895.37 / 2954.82 ±40.33 / 3014.90 ms │     no change │
│ QQuery 29 │       41.51 / 50.85 ±14.14 / 78.26 ms │       41.70 / 55.41 ±14.65 / 77.94 ms │  1.09x slower │
│ QQuery 30 │    305.99 / 322.64 ±18.01 / 352.64 ms │    311.07 / 330.23 ±15.01 / 347.17 ms │     no change │
│ QQuery 31 │    286.12 / 296.15 ±11.22 / 317.59 ms │    313.43 / 330.68 ±19.25 / 365.83 ms │  1.12x slower │
│ QQuery 32 │   957.78 / 982.10 ±16.32 / 1003.84 ms │ 1122.58 / 1153.23 ±33.76 / 1218.83 ms │  1.17x slower │
│ QQuery 33 │ 1468.70 / 1549.56 ±50.70 / 1619.64 ms │ 1460.78 / 1593.05 ±93.25 / 1738.88 ms │     no change │
│ QQuery 34 │ 1485.24 / 1511.63 ±20.78 / 1532.05 ms │ 1469.66 / 1523.96 ±38.49 / 1587.71 ms │     no change │
│ QQuery 35 │    320.84 / 345.01 ±23.89 / 388.11 ms │    284.50 / 317.17 ±37.98 / 390.09 ms │ +1.09x faster │
│ QQuery 36 │        69.37 / 77.60 ±5.31 / 83.48 ms │        66.05 / 69.64 ±3.66 / 74.97 ms │ +1.11x faster │
│ QQuery 37 │        35.98 / 38.41 ±2.70 / 42.35 ms │        35.86 / 39.35 ±3.69 / 46.41 ms │     no change │
│ QQuery 38 │        40.07 / 43.89 ±2.97 / 49.09 ms │        45.47 / 49.22 ±2.09 / 51.57 ms │  1.12x slower │
│ QQuery 39 │     142.18 / 153.29 ±9.56 / 165.61 ms │     143.74 / 151.26 ±4.21 / 155.17 ms │     no change │
│ QQuery 40 │        13.81 / 14.45 ±0.45 / 14.98 ms │        14.10 / 16.21 ±3.73 / 23.67 ms │  1.12x slower │
│ QQuery 41 │        13.47 / 15.09 ±2.87 / 20.82 ms │        13.49 / 14.92 ±2.30 / 19.51 ms │     no change │
│ QQuery 42 │        12.96 / 13.03 ±0.06 / 13.13 ms │        12.93 / 14.11 ±1.76 / 17.61 ms │  1.08x slower │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary               ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)               │ 19894.89ms │
│ Total Time (min-max-distinct)   │ 20138.08ms │
│ Average Time (HEAD)             │   462.67ms │
│ Average Time (min-max-distinct) │   468.33ms │
│ Queries Faster                  │         15 │
│ Queries Slower                  │          9 │
│ Queries with No Change          │         19 │
│ Queries with Failure            │          0 │
└─────────────────────────────────┴────────────┘

Resource Usage

clickbench_partitioned — base (merge-base)

Metric Value
Wall time 100.0s
Peak memory 11.2 GiB
Avg memory 4.6 GiB
CPU user 1015.1s
CPU sys 74.6s
Peak spill 0 B

clickbench_partitioned — branch

Metric Value
Wall time 105.0s
Peak memory 11.6 GiB
Avg memory 4.3 GiB
CPU user 1021.7s
CPU sys 78.4s
Peak spill 0 B

File an issue against this benchmark runner

@mkleen

mkleen commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

The changes in clickbench_partitioned are probably just noise, since there is no min(DISTINCT) in any of the queries.

@nuno-faria

Copy link
Copy Markdown
Contributor

Wouldn't this work for max as well?

@adriangb

Copy link
Copy Markdown
Contributor

Wouldn't this work for max as well?

It looks like @neilconway predicted this one: #22644.

It sounds like what we need is

A trait method on AggregateUDFImpl::distinct_handling() -> DistinctHandling

/// How an aggregate function treats the `DISTINCT` modifier.
///
/// Mathematically, `Ignored` means the function's merge operation is
/// idempotent (its state forms a semilattice): f(S ⊎ S) = f(S), so
/// removing duplicates from the input cannot change the result.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DistinctHandling {
    /// The result is the same with or without `DISTINCT`, so the planner
    /// is free to drop it. `min`, `max`, `bool_and`, `bit_or`, ...
    Ignored,
    /// The accumulator honours `AccumulatorArgs::is_distinct` and
    /// deduplicates its input. `count`, `sum`, `avg`, `array_agg`, ...
    Honored,
    /// The accumulator does not implement `DISTINCT`. Planning
    /// `f(DISTINCT ...)` is an error. `corr`, `regr_*`, `nth_value`, ...
    Unsupported,
}

pub trait AggregateUDFImpl {
    /// How this function treats the `DISTINCT` modifier.
    ///
    /// Return [`DistinctHandling::Ignored`] for duplicate-insensitive
    /// functions so that `f(DISTINCT x)` is planned as `f(x)`, and
    /// [`DistinctHandling::Unsupported`] if the accumulator does not
    /// read `is_distinct`, so that `DISTINCT` is rejected at planning
    /// time rather than silently ignored.
    fn distinct_handling(&self) -> DistinctHandling {
        DistinctHandling::Honored
    }
}

We would set this to Unsupported for Correlation, Covariance, Regr, ApproxPercentileCont, NthValue, Stddev, Variance and ApproxMedian.

An optimizer rule that runs before SingleDistinctToGroupBy to remove DISTINCT where it wasn't needed so that SingleDistinctToGroupBy doesn't fire

@jayzhan211

Copy link
Copy Markdown
Contributor

Wouldn't this work for max as well?

It looks like @neilconway predicted this one: #22644.

It sounds like what we need is

A trait method on AggregateUDFImpl::distinct_handling() -> DistinctHandling

/// How an aggregate function treats the `DISTINCT` modifier.
///
/// Mathematically, `Ignored` means the function's merge operation is
/// idempotent (its state forms a semilattice): f(S ⊎ S) = f(S), so
/// removing duplicates from the input cannot change the result.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DistinctHandling {
    /// The result is the same with or without `DISTINCT`, so the planner
    /// is free to drop it. `min`, `max`, `bool_and`, `bit_or`, ...
    Ignored,
    /// The accumulator honours `AccumulatorArgs::is_distinct` and
    /// deduplicates its input. `count`, `sum`, `avg`, `array_agg`, ...
    Honored,
    /// The accumulator does not implement `DISTINCT`. Planning
    /// `f(DISTINCT ...)` is an error. `corr`, `regr_*`, `nth_value`, ...
    Unsupported,
}

pub trait AggregateUDFImpl {
    /// How this function treats the `DISTINCT` modifier.
    ///
    /// Return [`DistinctHandling::Ignored`] for duplicate-insensitive
    /// functions so that `f(DISTINCT x)` is planned as `f(x)`, and
    /// [`DistinctHandling::Unsupported`] if the accumulator does not
    /// read `is_distinct`, so that `DISTINCT` is rejected at planning
    /// time rather than silently ignored.
    fn distinct_handling(&self) -> DistinctHandling {
        DistinctHandling::Honored
    }
}

We would set this to Unsupported for Correlation, Covariance, Regr, ApproxPercentileCont, NthValue, Stddev, Variance and ApproxMedian.

An optimizer rule that runs before SingleDistinctToGroupBy to remove DISTINCT where it wasn't needed so that SingleDistinctToGroupBy doesn't fire

+1 for this

@mkleen

mkleen commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the feedback!

@mkleen

mkleen commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

Superseeded by #25288

@mkleen mkleen closed this Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants