[HN Gopher] macOS 26.2 enables fast AI clusters with RDMA over T...
___________________________________________________________________
macOS 26.2 enables fast AI clusters with RDMA over Thunderbolt
Author : guiand
Score : 133 points
Date : 2025-12-12 20:41 UTC (2 hours ago)
(HTM) web link (developer.apple.com)
(TXT) w3m dump (developer.apple.com)
| nodesocket wrote:
| Can we get proper HDR support first in macOS? If I enable HDR on
| my LG OLED monitor it looks completely washed out and blacks are
| grey. Windows 11 HDR works fine.
| Razengan wrote:
| Really? I thought it's always been that HDR was notorious on
| Windows, hopeless on Linux, and only really worked in a plug-
| and-play manner on Mac, unless your display has an incorrect
| profile or something/
|
| https://www.youtube.com/shorts/sx9TUNv80RE
| heavyset_go wrote:
| Works well on Linux, just toggle a checkmark in the settings.
| masspro wrote:
| MacOS does wash out SDR content in HDR mode specifically on
| non-Apple monitors. An HDR video playing in windowed mode
| will look fine but all the UI around it has black and white
| levels very close to grey.
|
| Edit: to be clear, macOS itself (Cocoa elements) is all SDR
| content and thus washed out.
| Starmina wrote:
| That's intended behavior for monitor limited in peak
| brightness
| nodesocket wrote:
| I don't think so. Windows 11 has a HDR calibration
| utility that allows you to adjust brightness and HDR and
| it maintains blacks being perfectly black (especially
| with my OLED). When I enable HDR on macOS whatever
| settings I try, including adjusting brightness and
| contrast on the monitor the blacks look completely washed
| out and grey. HDR DOES seem to work correctly on macOS
| but only if you use Mac displays.
| masspro wrote:
| That's the statement I found last time I went down this
| rabbit hole, that they don't have physical brightness
| info for third-party displays so it just can't be done
| any better. But I don't understand how this can lead to
| making the black point terrible. Black should be the one
| color every emissive colorspace agrees on.
| adastra22 wrote:
| Huh, so that's why HDR looks like shit on my Mac Studio.
| m-ack-toddler wrote:
| AI is arguably more important than whatever gaming gimmick
| you're talking about.
| simonw wrote:
| I follow the MLX team on Twitter and they sometimes post about
| using MLX on two or more joined together Macs to run models that
| need more than 512GB of RAM.
|
| A couple of examples:
|
| Kimi K2 Thinking (1 trillion parameters):
| https://x.com/awnihannun/status/1986601104130646266
|
| DeepSeek R1 (671B):
| https://x.com/awnihannun/status/1881915166922863045 - that one
| came with setup instructions in a Gist:
| https://gist.github.com/awni/ec071fd27940698edd14a4191855bba...
| awnihannun wrote:
| For a bit more context, those posts are using pipeline
| parallelism. For N machines put the first L/N layers on machine
| 1, next L/N layers on machine 2, etc. With pipeline parallelism
| you don't get a speedup over one machine - it just buys you the
| ability to use larger models than you can fit on a single
| machine.
|
| The release in Tahoe 26.2 will enable us to do fast tensor
| parallelism in MLX. Each layer of the model is sharded across
| all machines. With this type of parallelism you can get close
| to N-times faster for N machines. The main challenge is latency
| since you have to do much more frequent communication.
| liuliu wrote:
| But that's only for prefilling right? Or is it beneficial for
| decoding too (I guess you can do KV lookup on shards, not
| sure how much speed-up that will be though).
| andy99 wrote:
| I'm hoping this isn't as attractive as it sounds for non-
| hobbyists because the performance won't scale well to parallel
| workloads or even context processing, where parallelism can be
| better used.
|
| Hopefully this makes it really nice for people that want the
| experiment with LLMs and have a local model but means well
| funded companies won't have any reason to grab them all vs
| GPUs.
| codazoda wrote:
| I haven't looked yet but I might be a candidate for something
| like this, maybe. I'm RAM constrained and, to a lesser
| extent, CPU constrained. It would be nice to offload some of
| that. That said, I don't think I would buy a cluster of Macs
| for that. I'd probably buy a machine that can take a GPU.
| pstuart wrote:
| I imagine that M5 Ultra with Thunderbolt 5 could be a decent
| contender for building plug and play AI clusters. Not cheap, but
| neither is Nvidia.
| whimsicalism wrote:
| nvidia is absolutely cheaper per flop
| FlacksonFive wrote:
| To acquire, maybe, but to power?
| whimsicalism wrote:
| machine capex currently dominates power
| amazingman wrote:
| Sounds like an ecosystem ripe for horizontally scaling
| cheaper hardware.
| adastra22 wrote:
| FLOPS are not what matters here.
| whimsicalism wrote:
| also cheaper memory bandwidth. where are you claiming that
| M5 wins?
| Infernal wrote:
| I'm not sure where else you can get a half TB of 800GB/s
| memory for < $10k. (Though that's the M3 Ultra, don't
| know about the M5). Is there something competitive in the
| nvidia ecosystem?
| whimsicalism wrote:
| I wasn't aware that M3 Ultra offered a half terabyte of
| unified memory, but an RTX5090 has double that bandwidth
| and that's before we even get into B200 (~8TB/s).
| 650REDHAIR wrote:
| You could get x1 M3 Ultra w/ 512gb of unified ram for the
| price of x2 RTX 5090 totaling 64gb of vram not including
| the cost of a rig capable of utilizing x2 RTX 5090.
| baq wrote:
| at current memory prices today's _cheap_ is yesterday 's
| _obscenely expensive_ - Apple 's current RAM upgrade prices are
| cheap
| jeffbee wrote:
| Very cool. It requires a fully-connected mesh so the scaling
| limit here would seem to be 6 Mac Studio M3 Ultra, up to 3TB of
| unified memory to work with.
| PunchyHamster wrote:
| I'm sure someone will figure out how to make thunderbolt
| switch/router
| huslage wrote:
| I don't believe the standard supports such a thing. But I
| wonder if TB6 will.
| novok wrote:
| Now we need some hardware that is rackmount friendly, an OS that
| is not fidly as hell to manage in a data center or headless
| server and we are off to the races! And no, custom racks are not
| 'rackmount friendly'.
| joeframbach wrote:
| So, the Powerbook Duo Dock?
| btown wrote:
| It would be incredibly ironic if, with Apple's relatively stable
| supply chain relative to the chaos of the RAM market these days
| (projected to last for years), Apple compute became known as a
| _cost-effective_ way to build medium-sized clusters for
| inference.
| andy99 wrote:
| It's gonna suck if all the good Macs get gobbled up by
| commercial users.
| mschuster91 wrote:
| it's not like regular people can afford this kind of Apple
| machine anyway.
| teaearlgraycold wrote:
| It already is depending on your needs.
| timsneath wrote:
| Also see https://www.engadget.com/ai/you-can-turn-a-cluster-of-
| macs-i...
| geerlingguy wrote:
| This implies you'd run more than one Mac Studio in a cluster, and
| I have a few concerns regarding Mac clustering (as someone who's
| managed a number of tiny clusters, with various hardware):
|
| 1. The power button is in an awkward location, meaning
| rackmounting them (either 10" or 19" rack) is a bit cumbersome
| (at best)
|
| 2. Thunderbolt is great for peripherals, but as a semi-permanent
| interconnect, I have worries over the port's physical
| stability... wish they made a Mac with QSFP :)
|
| 3. Cabling will be important, as I've had tons of issues with TB4
| and TB5 devices with anything but the most expensive Cable
| Matters and Apple cables I've tested (and even then...)
|
| 4. macOS remote management is not nearly as efficient as Linux,
| at least if you're using open source / built-in tooling
|
| To that last point, I've been trying to figure out a way to, for
| example, upgrade to macOS 26.2 from 26.1 remotely, without a GUI,
| but it looks like you _have_ to use something like Screen Sharing
| or an IP KVM to log into the UI, to click the right buttons to
| initiate the upgrade.
|
| Trying "sudo softwareupdate -i -a" will install minor updates,
| but not full OS upgrades, at least AFAICT.
| eurleif wrote:
| I have no experience with this, but for what it's worth, looks
| like there's a rack mounting enclosure available which
| mechanically extends the power switch:
| https://www.sonnetstore.com/products/rackmac-studio
| wlesieutre wrote:
| For #2, OWC puts a screw hole above their dock's thunderbolt
| ports so that you can attach a stabilizer around the cord
|
| https://www.owc.com/solutions/thunderbolt-dock
|
| It's a poor imitation of old ports that had screws on the
| cables, but should help reduce inadvertent port stress.
|
| The screw only works with limited devices (ie not the Mac
| Studio end of the cord) but it can also be adhesive mounted.
|
| https://eshop.macsales.com/item/OWC/CLINGON1PK/
| crote wrote:
| That screw hole is just the regular locking USB-C variant, is
| it not?
|
| See for example:
|
| https://www.startech.com/en-jp/cables/usb31cctlkv50cm
| timc3 wrote:
| It's been terrible for years/forever. Even Xserves didn't
| really meet the needs of a professional data centre. And it's
| got worse as a server OS because it's not a core focus. Don't
| understand why anyone tries to bother - apart from this MLX use
| case or as a ProRes render farm.
| crote wrote:
| iOS build runner. Good luck developing cross-platform apps
| without a Mac!
| colechristensen wrote:
| There are open source MDM projects, I'm not familiar but
| https://github.com/micromdm/nanohub might do the job for OS
| upgrades.
| givemeethekeys wrote:
| Would this also work for gaming?
| AndroTux wrote:
| No
| storus wrote:
| Is there any way to connect DGX Sparks to this via USB4? Right
| now only 10GbE can be used despite both Spark and MacStudio
| having vastly faster options.
| daft_pink wrote:
| Hoping Apple has secured plentiful DDR5 to use in their machines
| so we can buy M5 chips with massive amounts of RAM soon.
| colechristensen wrote:
| Apple tends to book its fab time / supplier capacity years in
| advance
| reaperducer wrote:
| As someone not involved in this space at all, is this similar to
| the old MacOS Xgrid?
|
| https://en.wikipedia.org/wiki/Xgrid
| reilly3000 wrote:
| dang I wish I could share md tables.
|
| Here's a text edition: For $50k the inference hardware market
| forces a trade-off between capacity and throughput:
|
| * Apple M3 Ultra Cluster ($50k): Maximizes capacity (3TB). It is
| the only option in this price class capable of running 3T+
| parameter models (e.g., Kimi k2), albeit at low speeds (~15 t/s).
|
| * NVIDIA RTX 6000 Workstation ($50k): Maximizes throughput (>80
| t/s). It is superior for training and inference but is hard-
| capped at 384GB VRAM, restricting model size to <400B parameters.
|
| To achieve both high capacity (3TB) and high throughput (>100
| t/s) requires a ~$270,000 NVIDIA GH200 cluster and data center
| infrastructure. The Apple cluster provides 87% of that capacity
| for 18% of the cost.
___________________________________________________________________
(page generated 2025-12-12 23:00 UTC)