Skip to content

Struct-valued expressions other than column() emit a single output name #312

Open

Description

@nielspardon

ExpressionReference.output_names holds field names in depth-first order, so a struct-valued expression needs one name for itself plus one per nested field. Apart from column(), the expression builders in src/substrait/builders/extended_expression.py always produce a single name, whatever the output type is. A projected struct-valued function, cast, literal or if_then therefore yields a RelRoot.names / inferred NamedStruct with too few names, and nothing raises an error:

from substrait.builders.extended_expression import cast, column
from substrait.builders.plan import read_named_table, select
from substrait.builders.type import i64, named_struct, struct
from substrait.extension_registry import ExtensionRegistry
from substrait.type_inference import infer_plan_schema

registry = ExtensionRegistry(load_default_extensions=False)
ns = named_struct(
    names=["a", "s", "x", "y"],
    struct=struct([i64(), struct([i64(), i64()])], nullable=False),
)
plan = select(
    read_named_table("t", ns), [cast(column("s"), struct([i64(), i64()]))]
)(registry)

list(plan.relations[-1].root.names)                  # ['cast(s)']  needs 3 names
list(infer_plan_schema(plan, registry=registry).names)  # ['cast(s)']  for a struct<i64, i64>

Fixing this means deriving each builder's output type and padding output_names to type_num_names of it. The spec does not say what the nested fields of a computed struct should be called, so before picking a scheme, check what the sibling bindings (substrait-java, substrait-rs) generate and state the choice in the PR. alias() renames only output_names[0], which is correct once the nested names are present.

Follow-up to #311, which fixed select, project and aggregate to carry a struct column's full name list.

Activity

  1. nielspardon commented on Oct 13, 2026

    @nielspardon
    MemberAuthor

    This also covers aggregate() measures. In #313, grouping expressions keep their full output_names list, but measures still take only output_names[0] (src/substrait/builders/plan.py, the measure half of the names list in aggregate()). A struct-returning aggregate function given a full alias list is therefore cut back to one name. When the builders here start emitting the nested names, switch measures to the full list as well. No standard extension has an aggregate that returns a struct, so this only shows up with a user-registered extension.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions