[HN Gopher] Prefix sums at gigabytes per second with ARM NEON
___________________________________________________________________
Prefix sums at gigabytes per second with ARM NEON
Author : mfiguiere
Score : 63 points
Date : 2026-03-08 20:22 UTC (5 days ago)
(HTM) web link (lemire.me)
(TXT) w3m dump (lemire.me)
| vardump wrote:
| What's going on with SVE[2] support in the ARM land? It's weird
| that even Apple's M5 still doesn't support it (other than
| SME[2]).
| nubinetwork wrote:
| The radxa orion o6 apparently supports it...
| adrian_b wrote:
| All the ARM cores designed by the Arm company and launched
| since 2022, which support variants of the Armv9-A ISA, support
| SVE2. This means that all medium price or high price
| smartphones that were introduced during the last 4 years have
| SVE2 support.
|
| However, in embedded computers typically only extremely old
| cores are used, not newer than Cortex-A78 (2021), so these
| normally do not have SVE2 support. The exceptions are the new
| and extremely expensive NVIDIA Thor, intended for automotive
| applications (with Neoverse V3AE cores) and a CPU made by a
| Chinese company with Cortex-A720 cores, which is available in
| several single-board computers or Mini-ITX motherboards.
|
| A few of the latest Arm-based server CPUs, for instance AWS
| Graviton5, support SVE2.
|
| Apple seems to believe that the SVE2 ISA (derived from an
| extension of Aarch64 from Fujitsu) is not good, so they promote
| the SME/SME2 extension, which appears to be derived from a
| former proprietary ISA extension implemented in older Apple
| CPUs.
|
| For single-thread applications, where Apple CPUs are better
| than the competition, SME2 can provide significantly higher
| performance than SVE2.
|
| However, the SME2 performance for multi-threaded applications
| is much less impressive, not because the SME2 ISA has any
| defect, but because SME2 is executed in a separate dedicated
| core, which is shared by a cluster of normal CPU cores, so SME2
| performance does not scale much when more cores are used,
| because a CPU might have only 1 SME2 core for each 4 or 8
| normal cores.
|
| This might contribute to the fact that the Apple CPUs have
| exceptional single-thread performance, but a multi-threaded
| performance that is not better than that of the competitors.
|
| When I first heard about SVE/SVE2, I thought that it was great,
| but nowadays I am much less enthusiastic about it. I believe
| that the original goal, of writing programs that run on any
| CPU, regardless of the widths of its vector registers and of
| its vector execution units, is futile.
|
| It is not possible to reach the maximum performance allowed by
| the hardware in a width-agnostic program. So now I believe that
| what is needed is not hardware support for ignoring the width,
| but better software tools that allow an easier writing of
| programs that are parametrized with hardware characteristics
| like the width of a cache line and the width of vector or
| matrix registers, from which a compiler should generate optimal
| code when the hardware parameters are given.
|
| Even if I believe that SVE2 is not good enough to allow the
| programmer to ignore the implemented width, it still has some
| important improvements over the older Arm SIMD instructions, so
| it must be preferred on any CPU than supports it. When SME2 is
| available, like on Apple or on the latest generation of Arm
| cores launched in 2025, it is likely to be preferable to SVE2,
| unless latency is more important than throughput.
|
| SME2 is intended to offer better throughput than SVE2 and
| better latency than the GPU. For maximum throughput, the GPU is
| preferable, if applicable. For minimum latency, SVE2 is the
| best.
| dzaima wrote:
| > This means that all medium price or high price smartphones
| that were introduced during the last 4 years have SVE2
| support.
|
| Except Qualcomm chipsets, which disable SVE even if all ARM
| cores used support it. ("Snapdragon 8 Elite Gen 5" supposedly
| finally supports SVE? but that's like only half a year old)
| my123 wrote:
| Qualcomm was odd like that for a long time yeah.
|
| And yes the Gen 5 chips (8, 8 Elite and X2) do implement
| SVE2 and SME.
| my123 wrote:
| SME2 is restricted in scope to matrix multiply workloads and
| isn't really designed for anything else.
|
| The point of streaming SVE is to have a way to pre/post
| process data on the way in or out of a matrix multiply.
|
| A list that I have around of chips which support various
| levels of SVE:
|
| For SVE(1) deployment, chips that have it: - Fujitsu A64fx -
| AWS Graviton3
|
| SVE2: - Snapdragon X2, 8/8 Elite Gen 5 and later - MediaTek
| Dimensity 9000 and later - NVIDIA Tegra Thor and later,
| NVIDIA "N1" or later (GB10 is an "N1x" SKU) - Samsung Exynos
| 2200 or later - AWS Graviton4, Microsoft Cobalt 100, Google
| Axion (and newer chips) - CIX P1
|
| SME(1) instead of SME2:
|
| - Snapdragon X2, 8/8 Elite Gen 5
|
| SME2:
|
| - Apple M4, A18 and later - Samsung Exynos 2600 - MediaTek
| Dimensity 9500
|
| Note that the Snapdragon 8/8 Elite Gen 5 and X2 support sve2
| but not svebitperm.
| hayley-patton wrote:
| As not mentioned in the article, if you want the general form of
| this algorithm, it is a Hillis-Steele prefix sum:
| <https://en.wikipedia.org/wiki/Prefix_sum#Algorithm_1:_Shorte...>
| mlochbaum wrote:
| I don't think this really describes neon_prefixsum_fast as a
| whole? The algorithm does use a Hillis-Steele sum on sums of 4
| values, but each of these is computed with a sequential sum,
| interleaving those with a transposed order. In terms of what's
| added to what, it's actually quite a bit like my "Sequential
| broadcasting" picture from [0]. The reference I'd use for a
| general form is "Parallel Scan as a Multidimensional Array
| Problem"[1], breaking 16 elements into a 4x4 array; the paper
| describes how the scan splits into a row-wise scan, plus values
| obtained from an _exclusive_ scan on carries from the rows.
|
| [0]
| https://mlochbaum.github.io/BQN/implementation/primitive/fol...
|
| [1] https://ashinkarov.github.io/pubs/2022-scan.html
| Jeffrin-dev wrote:
| The interleaved load trick is clever, i never thought about using
| vld4 that way. always assumed SIMD would struggle with sequential
| dependencies like this since each value depends on the previous
| one curious how this holds up on older ARM chips, like would you
| see similar gains or does it depend heavily on the M4's specific
| pipeline. also wonder if there's a similar approach for AVX on
| x86 or if the instruction set makes it more awkward.
| flykespice wrote:
| Couldn't this be written in a C-pure way so that compilers can
| take advantadge of vector optimization and produce equally
| optimized code?
|
| I have been discouraged to write hand-written assembly SIMD code,
| because netizents say you can barely outsmart compiler-optimized
| assembly code nowadays..
___________________________________________________________________
(page generated 2026-03-13 23:01 UTC)