[HN Gopher] A beginner's guide to deploying LLMs with AMD on Win...
___________________________________________________________________
A beginner's guide to deploying LLMs with AMD on Windows using
PyTorch
Author : beckford
Score : 107 points
Date : 2025-10-06 13:15 UTC (4 days ago)
(HTM) web link (gpuopen.com)
(TXT) w3m dump (gpuopen.com)
| behnamoh wrote:
| I have a philosophy for which I have mixed feelings because I
| like it in principle despite it making me worse off in some other
| ways: Devs should punish companies that clearly
| don't give a shit about them.
|
| When I see AMD, I think of a firm that heavily prioritized their
| B2B business over B2C. Not just gamers, but a lot of LLM
| enthusiasts have been calling AMD to offer _something_ comparable
| to 4090 /5090, and don't mess up drivers and software
| compatibility.
|
| AMD's response? Nothing. They do build AI chips but they're for
| the megacorps. At one point I wanted to save for a MI300x. But
| AMD only sells them in batches of x8! (not factorial lol).
|
| I've had it with AMD, and would rather penalize them (even if it
| costs them pennies) out of principle.
|
| I had similar thoughts about Apple MLX a while back and wrote a
| mean post about it on reddit:
| https://www.reddit.com/r/LocalLLaMA/comments/19cdd9z/dont_ta...
|
| But since then the cracked MLX team has kept delivering so much
| that I've now completely changed my mind about it. That, coupled
| with Apple's failed AI attempts and the recent addition of MatMul
| in A-chips (and ofc in M5), gives me hope about MLX's future and
| now I develop for it!
| almostgotcaught wrote:
| > I've had it with AMD, and would rather penalize them (even if
| it costs them pennies) out of principle.
|
| strong jilted, jealous, lover vibes on this. just curious - how
| do you plan to carry out this revenge scheme on a multi-
| billion, multi-national, megacorp, other than angry posts on
| social media?
| mcraiha wrote:
| You can poison the LLM training data
| https://news.ycombinator.com/item?id=45529587
| almostgotcaught wrote:
| what's your point? AMD isn't in the business of selling LLM
| services?
| kouteiheika wrote:
| I feel exactly the same as OP. Personally, I just ignore
| them. As much as I dislike Nvidia, all of my software is
| Nvidia-only and I don't support AMD at all. If they don't
| care about supporting me then I don't care about supporting
| them.
|
| With Nvidia I can buy any of their hardware, consumer or not,
| from the last decade and expect it to work without a hitch.
| Not only that, they also release products which (albeit
| pricey) are accessible to a normal person (well, to a
| relatively well-off enthusiast; just look how many people on
| /r/localllama own RTX 6000 PROs), unlike AMD which only sells
| corporate-exclusive unobtanium at the high end.
|
| I have really high hopes for Tenstorrent's next product. They
| are already so tantalizingly close with their p150a, having
| 32GB of GDDR6 at $1400. If they double the VRAM and double
| the memory speed - I'm sold.
| hodgehog11 wrote:
| I kind of feel the same way, but my take may not be quite as
| cynical. The way I see it, some companies just act really
| stupid, and there's a lot more room for forgiveness when you
| suspect incompetence than active disregard. There's a lot of
| talk about how amazing Lisa Su is as a leader, and while it's
| true that she has executed brilliantly on the goals she
| dictated for the company, the record will show that those goals
| were rather short-sighted in the long run.
|
| It's been painfully obvious since at least 2018 that GPUs were
| going to explode in value as Nvidia developed an increasingly
| large monopoly on AI compute. In response to this, AMD put
| almost no-one on the driver stack. At the time, it was
| baffling. Now it's a near Blockbuster-level failure in
| corporate decision making. They reap what they sow.
|
| But what will benefit the consumer in the long run is to break
| the monopoly. There are no real sides to take here other than
| "not Nvidia". If AMD provides a decent product, it makes sense
| to buy it to show that the support is there. It's not been
| sunshine and roses, but my recent experience with the 7900 XTX
| has been positive and substantially better than anything they
| had to offer in 2020. The fact that the AMD stack is still so
| weak is not a product of their lack of improvement, but rather
| how astonishingly far behind they were.
|
| Having said that, the recent GPU prices from AMD have been
| baffling. If they don't improve, they'll continue to be in the
| dust.
| janalsncm wrote:
| What will likely break the monopoly is a Chinese competitor
| producing a worse but usable product within the next couple
| of years. AMD will be the next Compaq.
| LogicFailsMe wrote:
| Indeed, but it will take more than a few years, the entire
| industry negging GPUs for a decade and half comes at a high
| price for them. And looking back historically, dominant
| companies like Intel had been sucking wind for over a
| decade before its ineptitude caught up with it. Microsoft
| was repeatedly punching itself in the face until Satya
| Nadella took over and pulled it back from the brink just by
| getting them to stop doing so. And a quarter century later,
| CSCO has nearly recovered to its dotcom peak stock value.
|
| But Nvidia isn't inept yet, and despite some of its people
| being just as arrogant as Googlers at Google's prime, they
| still deliver a vastly better product and ecosystem than
| anyone else. The Chinese will crack that, but the first few
| generations are going to be a rough ride. Just check out
| the iterations on any knockoff of DeWalt tools they sell
| with cheaper parts until they nail the price/performance
| ratio to perfection to see how they roll. But also, Nvidia
| remains a moving target with near bottomless pockets now.
| Good luck with that.
| abbadadda wrote:
| Have you looked at AMD's financials? They are fine. Intel
| is massively fucking up in the CPU space and they're
| benefiting from that in a big way. Yes, they're late to
| GPUs, but every percentage of market share they do make
| inroads on is raising the boat. They're not sinking by any
| means and there are more revenue streams for AMD than just
| CPUs/GPUs.
| patapong wrote:
| OpenAI now suddenly has a potentially hundreds of billions of
| dollars in incentives to improve the GPU Ai stack for AMD. I
| am quite intrigued to see what that might lead to.
| buyucu wrote:
| AMD is bad, but still preferable to paying the Nvidia tax.
|
| Hopefully Huawei will go global with their new chips soon, and
| we can all get cheap compute.
| fransje26 wrote:
| Cue the 1000% tariff on said affordable Huawei chip.
| pharrington wrote:
| I mean I also want AMD to simply make the better video card.
| I'd give them advice if I could, but that ain't my field of
| expertise.
| z3ratul163071 wrote:
| it's not only the support. they are unable to write good
| software.
|
| period.
| breakingcups wrote:
| Luckily you have NVIDIA to fall back on, with their affordable,
| consumer-focused LLM chips...
| overfeed wrote:
| I too love Nvidia for prioritizing B2C over B2B! /s
| zby wrote:
| What do you think about: https://unlockgpu.com/? I put it on
| hold because the shareholder resolutions has to propose some
| concrete board level changes - and it seems (or seemed a few
| months ago) that AMD really started implementing such changes.
| pjmlp wrote:
| They love gamers, just not on GNu/Linux.
|
| Playstation, XBox, Windows, gaming handhelds.
| _aavaa_ wrote:
| The steam deck is famously Linux and uses AMD
| pjmlp wrote:
| GNU/Linux like proper laptops and desktops, not gaming
| handhelds.
|
| 100% of the gamers that don't overlap with FOSS folks,
| don't care SteamOS is somehow based on the Linux kernel,
| the Steam Runtime and a Win32/DirectX translation layer,
| otherwise there would not be games to play.
| tuna74 wrote:
| Calling a Linux distro Gnu/Linux today is wrong because
| the actual Gnu code running is such a small part of
| everything.
|
| SteamOS uses Linux as much as Linux distros like Fedora
| and Arch and other Linux based systems like Android.
|
| A lot of gamers are also FOSS folks although not as
| hardcore as RMS.
| pjmlp wrote:
| Not at all, it makes a point, ChromeOS, WebOS, Android,
| are also based on the Linux kernel and hardly considered
| Linux distributions for everyday use, in a way similar to
| GNU/Linux expectations.
|
| How little do game studios care to port their titles from
| Android/NDK into SteamOS, even though both are "Linux".
| tuna74 wrote:
| Android and iOS games are usually not interesting for
| SteamOS gamers.
|
| What really makes ChromeOS different from Ubuntu except
| it is not developed in the open?
| _aavaa_ wrote:
| What, prey tell, is a "proper" laptop and desktop?
|
| Plug a usb dock into the steam deck and you have a
| "proper" computer.
| tokai wrote:
| What do you mean, AMD has been the go too for linux gamers
| for a long time.
| pjmlp wrote:
| OP was talking about which customers AMD actually cares
| about, apparently not enough about GNU/Linux gamers.
| yonaguska wrote:
| I haven't been following the market until recently, but isn't
| the AMD Ryzen Max AI a consumer friendly AI option? Is it just
| not serious enough relative to the Nvidia offerings?
| curt15 wrote:
| Does ROCm work on Ryzem Max AI?
| jakogut wrote:
| It works on the GPU, and the NPU is supposed to work on
| Windows using a framework called lemonade (I haven't
| tried), but the NPU is not supported with the same software
| stack on Linux yet.
| overfeed wrote:
| > Devs should punish companies that clearly don't give a shit
| about them.
|
| Don't get involved in parasocial relationships with
| corporations - if they were human, they'd all be amoral
| psychopaths with a harrowing addiction to profits.
|
| > When I see AMD, I think of a firm that heavily prioritized
| their B2B business over B2C
|
| That's ironic to read as a gamer who saw Nvidia roll-over to
| scalpers and Bitcoin farms. Selling out to big tech would have
| been better, frankly - there's some hope of technical
| cooperation to improve the product. Then again, corporations
| are not our friends.
|
| AMD will continue to have my custom as long as they have better
| bang-for-buck compared to Intel or Nvidia.
| drnick1 wrote:
| If you can follow these instructions, then you are also able to
| install Linux and can work in an environment with first class
| support for development and AI.
| Klaster_1 wrote:
| Recently, I wanted to TTS a book with abogen. I main Windows
| and it does everything I want from it, mostly gaming and web
| dev. It didn't work with my 7900XTX because of Pytorch
| incompatibility. Even Zluda didn't help because it lacked a
| specific feature necessary for the app. I just want to use
| locally AI powered software. AMD/Windows combo sucks for this.
| pjmlp wrote:
| These instructions assume there is already a working OS where
| everything is supported.
|
| Some of us consider modern IDEs and advanced debugging tools a
| much better support for development.
|
| https://bitsavers.org/pdf/xerox/parc/techReports/CSL-89-4_UN...
| user_7832 wrote:
| Anyone might have any idea whether it's easy (and worth it) to
| use the NPU or GPU on an earlier gen AMD? I have a 7480u, and CPU
| interference is quite alright but I've been wondering if the iPGU
| (or inbuilt and presumably puny) NPU might have any advantages.
| leopoldj wrote:
| The blog doesn't verify if the code is actually using the GPU.
| The code will work perfectly fine on CPU, albeit slowly. You
| should run this to be sure: python -c "import
| torch; print(torch.cuda.is_available())"
|
| Strange that torch.cuda.is_available() is used for AMD also.
|
| Use rocm-smi to be double sure.
___________________________________________________________________
(page generated 2025-10-10 23:01 UTC)