Skip to content

Explain ObjectHashAggregate fallback when Comet shuffle is disabled #5500

Description

@sunchao

Describe the bug

CometObjectHashAggregate returns None when Comet shuffle is disabled without attaching a fallback reason. This is a pre-existing diagnostics gap identified while reviewing #5421, not a regression from that PR's aggregate-buffer repair.

The shuffle guard in the reviewed source bypasses doConvert and does not call withFallbackReason. The strict unexplained-fallback check can therefore report an unannotated decline, or a reason collected from another node can obscure this particular cause.

Steps to reproduce

Add a focused serde/planner regression for ObjectHashAggregateExec, such as grouped collect_list over an eligible Parquet input with spark.sql.execution.useObjectHashAggregateExec=true and spark.comet.shuffle.enabled=false. Check the shuffle-disabled guard directly and the resulting extended-explain reason; run the regression with spark.comet.explain.fallback.strict.enabled both enabled and disabled.

The missing annotation is verified from the guard's source. A separate end-to-end strict-mode reproduction has not been run for this issue.

Expected behavior

The deliberate decline should carry an accurate shuffle-disabled reason, without changing aggregate execution eligibility or masking an unrelated missing explanation.

Additional context

The review discussion explicitly separates this issue from the buffer-compatibility fix. #5499 tracks stale fallback-reason provenance; this issue covers a specific missing reason at its source.

Activity

  1. 0lai0 commented on Aug 28, 2026

    @0lai0
    Contributor

    take

  2. added
    bugSomething isn't working
    priority:lowMinor issues, test failures, tooling, cosmetic
    area:aggregationHash aggregates, aggregate expressions
    and removed on Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:aggregationHash aggregates, aggregate expressionsbugSomething isn't workingpriority:lowMinor issues, test failures, tooling, cosmetic

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions