Skip to content

[C++] Add dynamic dispatch on ARM64 #47769

Closed
Closed
[C++] Add dynamic dispatch on ARM64#47769
Assignees
AntoinePrv
Labels
Component: C++Type: enhancement
Milestone
25.0.0

Description

@pitrou
pitrou
opened on Oct 9, 2025

Describe the enhancement requested

On ARM64, Arrow C++ currently enables NEON at compile-time, and doesn't support any form of dynamic dispatch.

We should add dynamic dispatch for SVE and SVE2, and look into optimizing specific operations (such as those used when reading Parquet files) for SVE and/or SVE2 vectors.

About SVE vs. SVE2

According to https://developer.arm.com/documentation/102340/0100/Introducing-SVE2, "the main difference between SVE2 and SVE is the functional coverage of the instruction set. SVE was designed for HPC and ML applications. SVE2 extends the SVE instruction set to enable data-processing domains beyond HPC and ML."

More details can be found in https://developer.arm.com/documentation/102340/0100/New-features-in-SVE2

Testing on SVE2

It's not easy to find cloud instances that have SVE2 enabled. Possible picks:

  • Axion-based instances on Google Cloud
  • Graviton4-based instances on AWS (but not earlier generations of Graviton CPUs)
  • Cobalt 100-based instances on Azure

Component(s)

C++

Activity

  1. pitrou
    added
    Type: enhancement
    on Oct 9, 2025
  2. github-actions
    added
    Component: C++
    on Oct 9, 2025

  3. AntoinePrv commented on Apr 10, 2026

    @AntoinePrv
    AntoinePrv
    on Apr 10, 2026
    Collaborator

    I've been looking into this.

    First xsimd does not currently support SVE2, but as Antoine said, it is not easy to find instances, meaning the demand is likely limited at the present time.

    Second, SVE introduce a significant departure from the general SIMD model, which is that the register/batch size is a dynamic constant. One can write SVE code, and it can work with 256 or 2048 register sizes, sort of a built-in dynamic dispatch!
    The issue with this is that xsimd had not yet adapted this model, and requires to know the size at compile time (there is compiler option to force it to a given size). Regardless of this, few of our algorithm (integer unpacking, byte stream split that I know of) also currently require to know the size at compile time.

    As such I suggest we introduce hard coded macros ARROW_HAVE_RUNTIME_SVE(256|512), ARROW_RUNTIME_SIMD_LEVEL+=SVE256,SVE512 (these two sizes feel like a good start).
    We currently have ARROW_HAVE_SVE${SVE_VECTOR_BITS}, which is gated behind if(SVE_VECTOR_BITS) (else ARROW_HAVE_SVE_SIZELESS). This may be departing from this, as we would need to supply multiple ones in the dynamic case (but only one in the static case), though we can make it work with some more CMake checks.

    I am not aware of any code in Arrow using this macros, perhaps some are using it with auto vectorization capabilities.

    What do you think @pitrou, seems like a good direction?
    Also CC @raulcd on whether we have an SVE CI (otherwise how to set it up).

  4. pitrou commented on Apr 10, 2026

    @pitrou
    pitrou
    on Apr 10, 2026
    MemberAuthor

    AFAIK, the only SVE vector sizes in the wild are 128 and 256, so we should choose our macros accordingly.

    This may be departing from this, as we would need to supply multiple ones in the dynamic case (but only one in the static case), though we can make it work with some more CMake checks.

    I'm not sure sure it's possible to enable two different SVE sizes at the same time, so we may need distinct compilations for vector width anyway.

    Edit: using gcc it's probably possible to pass only -msve-vector-bits option at a time, so we do need a distinct compilation per SVE vector size. Which means we must treat SVE128 and SVE256 as distinct SIMD targets, just like AVX2 and AVX512.

  5. pitrou commented on Apr 10, 2026

    @pitrou
    pitrou
    on Apr 10, 2026
    MemberAuthor

    Also CC @raulcd on whether we have an SVE CI (otherwise how to set it up).

    Can probably run lscpu on one of our ARM64 Linux jobs.

  6. raulcd commented on Apr 14, 2026

    @raulcd
    raulcd
    on Apr 14, 2026
    Member

    We should check what features do GitHub runner arm64 CPU provide. If they don't have the specific CPU capabilities we want to test we could potentially spin up an AWS instance and add a nightly job using runs-on to test those.

  7. pitrou commented on Apr 14, 2026

    @pitrou
    pitrou
    on Apr 14, 2026
    MemberAuthor

    We have the answer here (assuming that it's always the same instance type, which is of course not guaranteed):
    https://github.com/apache/arrow/actions/runs/24392607522/job/71242267674?pr=49736

    
Architecture:                            aarch64
CPU op-mode(s):                          32-bit, 64-bit
Byte Order:                              Little Endian
CPU(s):                                  4
On-line CPU(s) list:                     0-3
Vendor ID:                               ARM
Model name:                              Neoverse-N2
Model:                                   0
Thread(s) per core:                      1
Core(s) per socket:                      4
Socket(s):                               1
Stepping:                                r0p0
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16

    So it has SVE and even SVE2, and the SVE vector length is 128 bits according to various sources.

  8. pitrou
    added a commit that references this issue on Apr 14, 2026

    MINOR: [CI][C++] Show machine info in Linux GHA jobs (#49736)

    36c498d

  9. AntoinePrv commented on Apr 14, 2026

    @AntoinePrv
    AntoinePrv
    on Apr 14, 2026
    Collaborator

    So we'd want to dynamic dispatch to SVE128? Not sure how it compares to Neon.

  10. pitrou commented on Apr 14, 2026

    @pitrou
    pitrou
    on Apr 14, 2026
    MemberAuthor

    So we'd want to dynamic dispatch to SVE128? Not sure how it compares to Neon.

    Yes, to SVE-128 and SVE-256 (and perhaps SVE2-128 and SVE2-256). Finding out how it compares to Neon is part of the interest, and that will certainly be informative to our client.

    cc @cyb70289

  11. cyb70289 commented on Apr 15, 2026

    @cyb70289
    cyb70289
    on Apr 15, 2026
    Contributor

    SVE-128 is often no faster than Neon for similar instructions (e.g., vaddq -> svadd). Actually, it's sometimes slightly slower than Neon because most SVE operations require an additional predicate argument to activate only partial lanes, even though it's always all true for traditional SIMD coding.
    SVE-256 will definitely help. AFAIK only Gravtion3 supports SVE-256 (no SVE2). Gravtion4 and 5 returns to SVE-128 (with SVE2).

    Some new instructions introduced in SVE can be useful. E.g., SVE2 svmatch can boost special token detection in CSV parser [1], by matching multiple tokens in one shot [2].

    [1] https://github.com/apache/arrow/blob/main/cpp/src/arrow/csv/lexing_internal.h#L177
    [2] https://developer.arm.com/community/arm-community-blogs/b/architectures-and-processors-blog/posts/multi-token-search-strings-svmatch-instruction

  12. pitrou commented on Apr 15, 2026

    @pitrou
    pitrou
    on Apr 15, 2026
    MemberAuthor

    SVE-128 is often no faster than Neon for similar instructions (e.g., vaddq -> svadd). Actually, it's sometimes slightly slower than Neon because most SVE operations require an additional predicate argument to activate only partial lanes, even though it's always all true for traditional SIMD coding. SVE-256 will definitely help. AFAIK only Gravtion3 supports SVE-256 (no SVE2). Gravtion4 and 5 returns to SVE-128 (with SVE2).

    I see, thanks. Unfortunately, it seems most SVE implementations nowadays are 128-bit?

    Some new instructions introduced in SVE can be useful. E.g., SVE2 svmatch can boost special token detection in CSV parser [1], by matching multiple tokens in one shot [2].

    Is there an exhaustive list of new instructions in SVE (vs Neon) and SVE2 (vs SVE) somewhere? I couldn't find anything on my own.

  13. cyb70289 commented on Apr 15, 2026

    @cyb70289
    cyb70289
    on Apr 15, 2026
    Contributor

    I see, thanks. Unfortunately, it seems most SVE implementations nowadays are 128-bit?

    Yes. Looks some hardware seek for more execution units (higher parallelism), not wider vector.

    Is there an exhaustive list of new instructions in SVE (vs Neon) and SVE2 (vs SVE) somewhere? I couldn't find anything on my own.

    You can check ACLE for related documents. Arm C Language Extension for SVE lists SVE and SVE2 intrinsics respectively.
    No SVE vs Neon comparison, AFAIK there's not much new features (predicated loop might be one but not widely used due to incompatible to traditional simd coding).
    Arm intrinsics is also a good reference.

  14. github-actions
    assigned
    AntoinePrv
    on Apr 15, 2026
  15. pitrou
    added a commit that references this issue on Apr 28, 2026

    GH-47769: [C++] SVE dynamic dispatch (#49756)

    d1a8cf4
  16. pitrou
    added this to the 25.0.0 milestone on Apr 28, 2026

  17. pitrou commented on Apr 28, 2026

    @pitrou
    pitrou
    on Apr 28, 2026
    MemberAuthor

    Issue resolved by pull request 49756
    #49756

  18. pitrou
    closed this as completedon Apr 28, 2026
  19. Mottl
    added 2 commits that reference this issue on May 26, 2026

    MINOR: [CI][C++] Show machine info in Linux GHA jobs (apache#49736)

    37af50f

    apacheGH-47769: [C++] SVE dynamic dispatch (apache#49756)

    1d76806
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Component: C++Type: enhancement

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions