[HN Gopher] Everything I've learned building the fastest Arm des...
___________________________________________________________________
Everything I've learned building the fastest Arm desktop
Author : alphabettsy
Score : 115 points
Date : 2023-10-27 14:13 UTC (8 hours ago)
(HTM) web link (www.jeffgeerling.com)
(TXT) w3m dump (www.jeffgeerling.com)
| a_vanderbilt wrote:
| 128 cores is impressive, but the single core performance is
| painfully low compared to the M series.
| pjmlp wrote:
| Outside classical C UNIX applications, most language runtimes
| are multihreaded anyway.
|
| Windows has been multithreaded for decades across all layers,
| even classical Win32 written in C is using multiple threads.
|
| Android, ChromeOS and macOS likewise.
|
| In the end, single core performance being that relevant is
| quite niche case in modern software stacks.
| zerohp wrote:
| Having multiple threads does not mean that they are all doing
| equally useful work. Single threaded performance is
| absolutely critical for a desktop machine.
|
| Even in multithreaded desktop applications, it's rare to see
| them effectively use more than 8 threads.
| geerlingguy wrote:
| There are some tasks where single core definitely limits
| the performance (some games especially). For most of the
| 'compute' oriented tasks like CAD/3D, LLMs, etc. multicore
| is great, and the slow single core speed doesn't seem to
| get in the way.
|
| I would still rather have 128 M1-class cores than 128
| Neoverse-N1 cores :)
| sroussey wrote:
| I would rather have 10 M series and take the rest of the
| area for cache.
| pjmlp wrote:
| Yeah, on classical UNIX tradition.
|
| macOS, ChromeOS, Windows and Android are heavily
| multithreaded, even when there is one main application
| thread, the underlying OS APIs are using auxiliary threads.
|
| Easily observable in any system debugger.
| cozzyd wrote:
| Python enters the chat.
| pjmlp wrote:
| If we ignore the threads that might be executing in the
| background for GC, async io support, and in some runtimes
| JIT compiler.
| aseipp wrote:
| To be fair the Neoverse N-series was designed for datacenter
| style usage where lower clock frequencies and going wide is
| more common even on x86. For client uses some of the things the
| Apple M-series does for single core performance may or may not
| be worse; in particular 128-bit cache lines would potentially
| add a lot more false-sharing, and the really big-and-fast per-
| core L2 is probably not practical even if you added the 16kb
| page size that Apple uses to achieve it (L2 sizes it didn't get
| any better with AmpereOne either, though). Those are big, good
| single-core wins that make sense for client SKUs. I think
| you'll always have some of those tradeoffs with CPUs like this.
|
| Hopefully Qualcomm Oryon will spice things up on the client
| side a bit. Maybe we can get some real HEDT designs after that.
| ihattendorf wrote:
| That's all true, but then maybe it's not "the fastest Arm
| desktop" if it's 25% faster in multi-core workload but the M2
| Ultra is _168% faster in single core_ (that's 168% faster as
| in 2.68x as fast, not 68% faster). There are plenty of
| workloads where the M2 Ultra will be significantly more
| performant.
|
| I'm excited for the future of ARM desktops, but that also
| means they need to significantly increase single core
| performance. It looks like maybe the Snapdragon X Elite will
| get us there? Or at least significantly closer.
| geerlingguy wrote:
| AmpereOne uses a customized CPU core too, and I haven't had
| a chance to test on it yet to see how it compares. Still
| probably far below the latest Qualcomm or Apple core
| designs, but should be better than the Altra/Altra Max
| series.
|
| I would love to see an EPYC-style Arm CPU with M1 or better
| cores.
| bhouston wrote:
| This is very promising.
|
| The benchmark results are great, with the exception of the single
| core speeds. We just need non-Apple ARM to catch up in the single
| core speeds across the board -- they seem particularly slow here
| compared to everyone else. Even Intel and AMD I think outdo Apple
| ARM now on single core.
|
| I think it is a bit of a shame that Apple's top of the line M1/M2
| go to 24 cores and the rumoured M3 CPUs only go up to 32 cores at
| most. I think the Ultra should have 64 cores personally.
| Rendering and video people want that.
| 015a wrote:
| > Even Intel and AMD I think outdo Apple ARM now on single
| core.
|
| Yeah they do currently. But, if the single-core performance
| jump we see between A15 and M2 (same architecture), about 25%,
| can be replicated between A17 and M3, we should expect Apple to
| retake the single-core performance lead, as M2 is only ~8-12%
| behind. Its not unreasonable to expect a perf jump like this,
| given its mostly just shoving more watts through the same core
| architecture, and Apple's tremendous efficiency lead gives them
| more headroom to do that than Intel's 14th gen over 13th.
|
| Its also a totally valid question whether M3 will be based on
| A17 or A16. I'm not sure if we have strong rumors to point one
| way or the other. We'll know more next week.
| CharlesW wrote:
| Also the slowest, according to the Cinebench single-core numbers.
|
| But it can only get better from here. I'm glad that PC users are
| starting to benefit from Apple's "desktop ARM" revolution.
| jakobson14 wrote:
| I seriously doubt ARM will hit consumer desktops any time soon.
| Servers, where people care about power efficiency? Yes.
| Laptops, where people care about power efficiency? Yes.
|
| But on "gaming rigs" where people are content to eat nvidia's
| absolute bed-shitting power consumption, and where replacing
| ram is a rallying cry? Not going to happen. It'll take until
| they're dragged along by the other two segments, sometime in
| 2-3 hardware generations. Maybe 2029-30 at the earliest.
| CharlesW wrote:
| An interesting aspect of this is that gaming
| notebooks/laptops already outsell gaming desktops by ~2:11,
| and that ratio will grow as desktops continue to become niche
| devices.
|
| I like your estimate, but I think it's on the conservative
| side by at least a couple of years. The migration from x86 to
| ARM will be slow until it isn't.
|
| 1 https://www.statista.com/statistics/1003576/gaming-pc-
| shipme...
| andrewmutz wrote:
| Anyone know the cost of a box like this? Couldnt find it in the
| article
| aseipp wrote:
| You can spec out the prices here, which are the same boxes used
| in this article. The entry price is about $2k USD for the
| 32-core SKU: https://www.ipi.wiki/products/com-hpc-ampere-
| altra#
|
| The Dev Kit is probably what most people here would want, but
| if you probe around the site you can also find the fully loaded
| (PSU, case, etc) option too.
| geerlingguy wrote:
| Yeah sorry about that; there was one link to the dev kit
| buried in the text, but I've added links to the Dev Platform
| and Dev Kit near the top now.
| zamalek wrote:
| $2000 for 32 core up to $3300 for 128. It's non-trivial to get
| hold of. The biggest hurdle Ampere has is distribution (at
| least if the plan is to move from dev kits to consumer
| hardware), they are obnoxiously enterprise.
| stephencanon wrote:
| I'm very confused by the talk of "teraflops". Author references
| linpack (which is a double precision benchmark), but then says
| "if you really care about teraflops, you need a graphics card"
| and installs a 4070 Ti, which does like 600 gigaflops in double
| (IIRC). This would make more sense if we were talking about
| single precision (4070 does ~40 TFlops single?); which is it?
|
| The original 1.2 teraflop CPU number also smells funny; 128 cores
| * 2 neon units / core * 2 doubles / neon unit * 2 flops / cycle *
| 2.8 billion cycles / second = 2.8 TFlops peak, an even halfway
| decent BLAS will get you to 80% of that, for 2.3 TFlops. Double
| that number for single precision. Either a completely untuned
| BLAS or benchmarking a problem that's much too small for the CPU
| under test.
| geerlingguy wrote:
| Caveat: I consider myself a noob at benchmarking, after
| spending probably 1,000 hours on it over the years.
|
| The CPU benchmark used HPL linpack following a standard-ish
| Top500-style benchmark, mostly because I think it's fun to see
| how various single CPU systems compare to historic
| 'supercomputers' on the official lists.
|
| There are different ways to calculate (and benchmark) flops,
| the way I'm doing it is with this open source project:
| https://github.com/geerlingguy/top500-benchmark -- which can
| use OpenBLAS, Blis, or ATLAS, and I've tried all three un-
| tuned.
|
| I also worked with some Ampere engineers to run their optimized
| version for Ampere Altra Max:
| https://github.com/AmpereComputing/HPL-on-Ampere-Altra
|
| On their own server systems with 8 channels of memory they
| topped out around 1.6 Tflops. There are other tests you can run
| and get more or less Tflops, but I based my own results on the
| top500 test.
|
| See more discussion about the results and testing in the
| following links:
|
| https://github.com/geerlingguy/top500-benchmark/issues/19
| https://github.com/geerlingguy/top500-benchmark/issues/17
| https://github.com/geerlingguy/top500-benchmark/issues/10
|
| (And see some of the issues linked back to the Ampere repo in
| those issues.)
|
| Edit: And regarding the video card mention--it was meant more
| as a generic reference (e.g. running A100 or 4090 since you can
| go much further there than Altra Max), and not specifically to
| the 4070 Ti... but I can see how that is not as clear!
| dansalvato wrote:
| Hey Jeff, I loved the video. I know it was more about
| "current state of the tech" and less about what we should
| actually be buying, but it would be very cool to hear more
| about how each of these setups are priced on the scale of
| "dollar per unit of performance" or something like that. (Or
| maybe that's not fair to do, since most consumer software
| can't handle all those cores yet?)
|
| I'm also curious whether you think Apple's decisions on
| memory architecture (despite being non-upgradeable) will have
| a leg up in the long run. You mentioned that memory bandwidth
| tops out around 174GB/s. Although you handily beat the Mac
| Pro in a multicore benchmark thanks to core count, one of the
| Mac Pro's claims to fame is its memory bandwidth of 800GB/s,
| as well as its unified memory architecture.
| geerlingguy wrote:
| As with all things, it's a tradeoff. HBM on servers is
| similar to Apple's choice, and Xeon, EPYC, Nvidia H100, and
| some other designs incorporate it. There are good things
| (performance) and bad things (price/non-upgradeability)
| about it. Best of both worlds would he chip on module plus
| expansion slots, so the fast RAM is like L4 cache.
| jakobson14 wrote:
| That's essentially how things are likely to go with CXL,
| though the latency isn't likely to be quite as good as on
| a dedicated DIMM connection or even as "good" as it was
| with IBM's OMI. The future (imminent in enterprise and
| sometime around when PCIe 6.0 hits consumer machines)
| looks to be mostly a combination of HBM and CXL memory.
| radar1310 wrote:
| You can bring it to its knees with MSFS2020.
| LastNevadan wrote:
| I lost him when he said this:
|
| _And RAM goes a lot deeper too. Look at these two sticks of RAM.
| See how the one on the right has twice the number of memory
| modules? That allows the individual stick of RAM to pump through
| data more quickly than the one on the left, even though both of
| them are rated at DDR 3200 and CL22._
|
| The bus width is defined by the DIMM standard. A 64-bit DIMM
| (which is what he is showing) always has a 64-bit data interface,
| regardless of what devices are on the DIMM. The DIMM with fewer
| devices on it just has 16-bit-wide microchips on it. That doesn't
| have anything to do with the latency or clock speed of the chips
| themselves.
|
| And it's strange that he calls each microchip a "module". I would
| call that a microchip or a device. A "module" is an assembly that
| contains multiple microchips.
|
| I just get the vibe that he's someone skilled at putting systems
| together but doesn't have any real engineering background.
| redundantly wrote:
| He's referring to Rank Interleaving, and he's correct:
|
| https://frankdenneman.nl/2015/02/20/memory-deep-
| dive/#:~:tex....
|
| You've put off the vibe of someone that assumes they're correct
| without double checking themselves ;)
| dabedee wrote:
| From the article: "The GPU can be seen by Windows, but Nvidia
| only publishes Arm drivers for Linux, not Windows. So in device
| manager you just see a Basic Display Adapter, and it can't really
| do anything."
|
| Never thought I'd live in a world where drivers are published for
| Linux first. This is great.
| VHRanger wrote:
| I mean, arm drivers for GPUs are used a ton in scientific
| computing and deep learning.
|
| Nvidia sells ARM servers with 8-16 GPUs to my knowledge.
|
| So that makes sense
| baq wrote:
| Nvidia also wanted to buy ARM and has just announced their
| own ARM CPU, so yeah.
| 7734128 wrote:
| And they have been pushing Tegra for more than a decade.
| justin66 wrote:
| Their embedded SoCs include GPUs as well.
| buescher wrote:
| Wasn't there a Fujitsu A64FX workstation or am I totally
| misremembering?
| benj111 wrote:
| Am I weird that my first thought about a 128 core CPU is all the
| scrolling in htop?
___________________________________________________________________
(page generated 2023-10-27 23:01 UTC)