[HN Gopher] AMD Expands AI Product Lineup with GPU-Only Instinct...
___________________________________________________________________
AMD Expands AI Product Lineup with GPU-Only Instinct Mi300X with
192GB Memory
Author : mfiguiere
Score : 96 points
Date : 2023-06-13 18:35 UTC (4 hours ago)
(HTM) web link (www.anandtech.com)
(TXT) w3m dump (www.anandtech.com)
| lvl102 wrote:
| To be honest, if I am building a CUDA alternative from scratch,
| I'd build it for Apple Silicon. Apple will have a monopoly on
| TSM's state of the art.
| rbzz wrote:
| AMD relations with TSMC are definitely much more tight than
| Nvidia's. AMD has been helping with developing HPC-optimized
| nodes and modern packaging techniques (3d-stacking and
| chiplets), while Nvidia tried to force TSMC to drop prices by
| going to Samsung fabs for 1.5 generations (did not go too well
| for them).
|
| The problem with AMD is definitely not modern node access, but
| software, and with some investment it can probably change
| really fast.
|
| Apple, on the other hand, is definitely not going to release
| standalone apple silicon products (GPUs or CPUs), so I don't
| really believe for Apple Silicon as an AI platform (apart from
| on-device inferrence).
| brucethemoose2 wrote:
| Maybe they have something in the oven?
|
| I am skeptical too, but hardware design takes a _looong_
| time, and surely Apple sees the writing on the wall and the
| potential of their own hardware.
| brucethemoose2 wrote:
| The issue is Apple has zero presence in datacenters, and they
| dont seem very interested in changing that.
|
| And they are optimizing more for power efficiency in multimedia
| workloads than raw AI throughout at _any_ cost like the AI ASIC
| makers.
| smy20011 wrote:
| One thing AMD can do is working with ggml to make llama.cpp
| running on AMD GPUs. Compile modern ML framework is a quite
| complex due to number of OPs. However, running LLMs does not need
| a lot of OPs. Just einsum & relu & softmax. Having LLM works with
| llama.cpp could be done by a team within a week or so.
| mastax wrote:
| AMD ROCm vs Nvidia CUDA has been discussed to death, but I'm
| curious how AMD fares compared to some of the AI training
| accelerator vendors. I think it would be much more damning if AMD
| were worse than some upstart, because the upstart wouldn't have
| the huge resource advantage and decade long head start of Nvidia.
| From my limited experience it seems like Google TPU and Cerebras
| are much nicer to use for AI training, from the standpoint of
| driver and software stability, documentation, and ecosystem
| support.
|
| Perhaps that's not a fair comparison. From what I know AMD and
| NVIDIA use GPGPU cores (now with AI-focused instructions) plus
| separate AI-specific accelerator blocks. Conceptually, GPGPU +
| NPU on one die. NPUs can be much simpler than general-purpose
| GPUs. So AMD's driver and software stack likely needs to be an
| order of magnitude more complex than the NPU vendors' in order to
| accommodate other non-AI use cases. But to an end user it doesn't
| really matter _why_ it sucks, only that it does.
| brucethemoose2 wrote:
| ...Not precisely. AMD implements the "AI" matrix instructions
| in the shaders themselves, not as big seperate blocks like
| Nvidia. But unlike Nvidia, the instructions are different on
| the consumer (RDNA3) and server line. I dont know anything
| about building rocm, but supporting rdna must indeed make
| things more difficult.
|
| Intel takes this approach too.
| zwaps wrote:
| If you want to go an run inference/train on one of these fresh
| and sweet LLMs using bitsandbytes/PEFT, which is really where the
| excitement is at, you gotta use CUDA pretty much. This is the
| story now. And the story for the innovation before was the same.
| Use CUDA, or wait for AMD to catch up a year late and with a
| worse version of everything.
|
| I mean, sure, you could compile your stuff to XLA or just, I
| dunno, set up 800 of these cards and train the whole thing on
| ROCM. But then would you really, really use AMD instead of some
| TPUs?
|
| AMD made Machine Learning either impossible, unsupported or a
| chore on most of their hardware for years. Their stack sucks and
| no one wanted to implement it, and in fact they rarely supported
| most of their own GPUs.
|
| Yes you have GPUs. But we need also drivers. And software. People
| have been yelling at AMD about this for years and years.
|
| Instead, AMD has made it clear that Machine Learning is not a
| priority for the company. Hence, they deserve the lack of
| traction. Investing in AMD hardware for ML has literally been a
| mistake at every point in recent history. Imagine if you bought a
| bunch of ( _insert last Gen card which is no longer supported by
| their stack_ ) how dumb you'd look.
|
| Releasing a GPGPU card now? Honestly, why bother? No one is gonna
| buy it.
| thereisnospork wrote:
| This is a naive question, but how hard would it be for AMD to
| make their cards/firmware CUDA compatible? Feels like that's
| what they would need to do to sell hardware in the space, other
| than banking on sufficiently severe shortages.
| JonChesterfield wrote:
| Cuda runs in the layer above firmware. Compiling cuda to
| amdgpu isa could be done but might invite lawsuits.
|
| There's a language called HIP which is a fairly close
| approximation to cuda. You can probably convert one to the
| other with determination and regex. The GPUs themselves are
| fundamentally different in ways that hopefully don't matter
| to your application (warp synchronisation is the big one in
| my opinion, but I suspect cuda applications ignore it and
| just live with the race conditions).
| mindcrime wrote:
| There's more to machine learning and AI than just training the
| latest LLM's though. Speaking for myself, I am very interested
| in supporting AMD and the ROCm ecosystem, whatever AMD's past
| sins may be. I'm building a machine now which will be based on
| a high-end consumer GPU from AMD. Not going for something like
| this as it's almost certainly way out of my budget, but perhaps
| in the future.
|
| So basically, I'm betting that AMD has had (or is having) a
| change of heart and is genuinely committed to AI/ML on their
| products. Time may ultimately prove me wrong, but so be it if
| that proves to be the case.
|
| And FWIW, one reason for my optimism is that, whatever you
| think about the state of ROCm today, they are clearly investing
| heavily into the platform and continually working on it. You
| can see that just from looking at the activity on their Github
| repos:
|
| https://github.com/orgs/ROCmSoftwarePlatform/repositories
|
| There is constant activity and has been for some time, which I
| take as a good sign. Yes, it's just one signal among many that
| one could consider, but I think it's an important one.
| zwaps wrote:
| Yeah but you see, we have been here before. "This time we are
| serious about ML/AI..." And if you went and bought an AMD
| card then, you'd have been wrong.
|
| My example about LLMs was just to show that AMD is simply not
| part of the conversation. Three month before you could have
| made the same point about another approach.
|
| And still, if you'd actually have to risk money, you and me
| both know you'd never invest in AMD hardware for AI or start
| developing on it something high stakes.
|
| I mean, look at geohot he tried and just gave up entirely and
| AMD.
| mindcrime wrote:
| _And still, if you 'd actually have to risk money, you and
| me both know you'd never invest in AMD hardware for AI or
| start developing on it something high stakes._
|
| I assume you meant "you" in the royal sense. Since I
| personally am, in point of fact, investing in AMD hardware
| for AI. Yeah, it's a gamble, but that's my style. And I
| have to admit, part of it is driven by ideological reasons
| (ROCm being open source) and a simple desire to support AMD
| since I want them to become a serious competitor to NVIDIA.
| I believe that's an outcome that would benefit everybody.
|
| _I mean, look at geohot he tried and just gave up entirely
| and AMD._
|
| I have to admit, I don't find that particularly compelling
| in any regard. Nothing against geohot, he's clearly a smart
| dude, but... I'm not judging a company based on his
| interactions with them. _shrug_
| zwaps wrote:
| I wish you luck then. Right now, you need CUDA to run
| things like bf16 or quantization, which are truly
| enablers in this space.
|
| So for now, I don't see AMD getting any traction. And
| apparently, the quality of the drivers has not been
| improving. Time will tell.
| mindcrime wrote:
| Agreed. And FWIW, I'm not saying I use AMD kit
| exclusively. I'm fine with using NVIDIA hardware and CUDA
| when needed. In addition to the AMD based box I'm
| building for the lab at home, I'm also building a sister
| box which will be nearly identical, except using an
| NVIDIA GPU. I am an ideologue but I'm also at least a
| little bit pragmatic. :-)
| jiggawatts wrote:
| Can someone who works at AMD please print this out, roll it up,
| and smack some senior managers on the nose with it?
|
| NVIDIA is about to walk off with a trillion dollars because
| nobody at AMD "gets it".
|
| With no meaningful competition, NVIDIA will gouge as hard as
| they can. Such as charging $50K for a card that's not too
| different to a 4090 but with more memory.
| JonChesterfield wrote:
| AMD took aim at HPC and shipped Frontier. The ROCm stack is
| quite HPC themed because that was the driving project.
| Porting to run AI models is a work in progress but the back
| end is the same compiler stack that was written for graphics
| (largely games consoles iiuc) then upgraded for HPC, it'll
| get there.
| mindcrime wrote:
| _NVIDIA is about to walk off with a trillion dollars because
| nobody at AMD "gets it"._
|
| I think they "get it" OK. Whether or not they can formulate a
| viable strategy and execute it is one question, but they get
| the idea that "AI is important" and they know where they
| stand vis-a-vis NVIDIA.
|
| This is another reason I'm willing to invest some time and
| money in working with AMD products for AI/ML. History has
| shown us their ability to go toe-to-toe with a seemingly
| unassailable industry titan before, and they came out in
| pretty good shape then.
|
| https://www.forbes.com/sites/iainmartin/2023/05/31/lisa-
| su-s...
| stevenhuang wrote:
| Been playing with Stable Diffusion on 6750XT 12GB for a while
| now with minimal issues (low memory issues sometimes on higher
| res, but it's less of a problem using grid upscaler).
|
| ROCm is getting there, slowly.
| rbanffy wrote:
| Honestly, I find the x86+GPU parts more interesting. One could
| make a very Apple-like PC with a SoM and very little glue around
| it (a bit like what you can do with some top-of-the-line Xeons
| that have HBM chiplets). And with between 1 and 6 x86 chiplets,
| the Mi300 could have between 8 and 96 Zen 4 cores.
|
| These things are so interesting it's a shame they aren't cheap.
| the_pwner224 wrote:
| An AMD Ryzen APU + RAM soldered onto the motherboard is very
| similar to what Apple's doing.
|
| Afaik, because the memory controller is part of the CPU, the
| CPU-RAM connection on the consumer chips is entirely passive -
| just copper traces on the motherboard with no ICs inbetween?
| NekkoDroid wrote:
| IIRC the APUs aren't exactly typical APUs with shared memory,
| but more a CPU with a few GPU cores attached, operating
| independently memory whise (but I might be wrong).
| JonChesterfield wrote:
| The ones that have been shipping for a few years have one
| block of ddr that GPU and CPU cores both access. You can do
| things like synchronise code running in a thread on the CPU
| with a wavefront running in a kernel on the GPU using
| atomic operations on that shared memory.
| brucethemoose2 wrote:
| AMD and Intel are both making quad channel APUs akin to the M2
| Pro.
|
| https://videocardz.com/newz/intel-arrow-lake-p-with-320eu-gp...
|
| (Sorry, I cannot find the AMD rumor link atm)
|
| But TBH the hybrid design is less interesting than you think,
| just because nothing really takes advantage of it. Hence Intel
| canceled their datacenter APU in favor of a pure Falcon Shores
| GPU due to a lack of interest from customers.
| rbanffy wrote:
| > just because nothing really takes advantage of it.
|
| This is precisely why AMD, Intel, and Nvidia should think
| about making workstation-class machines with the lowest end
| of these - because until more people have one to play with,
| there won't be much to do with them.
| brucethemoose2 wrote:
| There were rumors of an AMD one... That also never
| materialized.
|
| Its a chicken and egg problem, I think. A big APU is so
| expensive that it doesn't really make sense without a very
| specific workload (like in a console), and the workloads
| dont really appear without the APUs.
| sliken wrote:
| AMD halo strix is the future APU with a 256 bit wide
| interface. Boggles my mind with a multi-year GPU shortage
| that AMD didn't bring a wider memory interface to iGPUs.
| Obviously they can do it since the ps5 and Xbox X have been
| shopping for some time.
|
| Looks like someone ported llama to apples metal v3 already
| and are getting 5 tok/s on a 65b model.
| brucethemoose2 wrote:
| AMD should have re-used the Xbox APU, but other than that
| it makes no economic sense?
|
| The tape out cost would be huge, the die would be huge.
| Either the mobo/socket would be super expensive and niche,
| or consumers would be pissed about non expandable RAM.
|
| Laptop OEMs didn't even want the Steam Deck chip or
| Broadwell-edram back then, much less a big expensive APU.
| digitallyfree wrote:
| The issue with AMD and AI is, as always, the software stack. Even
| if the hardware is great ROCM simply doesn't have industry
| traction and accessiblity.
| wing-_-nuts wrote:
| Doesn't have the traction _for now_. Cloud providers (ms,
| google, amz, etc) are quickly tiring of paying Nvidia monopoly
| premiums for their gpu hardware. Google has already invested in
| tpus and it wouldn 't surprise me at all if they got together
| to fund ROCm development or even went so far as to develop
| their own NN asics.
|
| Cuda is great, but it's not _strictly_ necessary for much of
| the latest AI / ML developments.
| gremlinsinc wrote:
| Intel, AMD, Google, Amazon, etc should team up to create some
| sort of standards/consortium around an open source CUDA
| alternative, something that anyone who can fabricate chips
| could use, and the consortium could have their own team of
| devs/researchers to make improvements / next gen versions of
| their CUDA alternative.
|
| Something like the way chrome vs chromium is, or even a
| foundation like the linux foundation, where you have multiple
| distros contributing packages/etc back into the ecosystem.
| bfeynman wrote:
| they are already doing that with XLA, google has TPUs,
| amazon has tranium/inferentia. common interface in future
| that you basically just cast model to `.toDevice` of an
| enum of accelerated computing types seems to be the goal.
| mepian wrote:
| They already did: https://www.khronos.org/sycl/
| zwaps wrote:
| ROCm is so terrible the cloud providers rolled out their own
| chips rather than use AMD which has perfectly good GPUs and
| the worst software stack ever.
| mook wrote:
| ROCm still doesn't support consumer GPUs; that means people
| building random things (as opposed to more serious work
| things) won't be using their stack, so none of the innovation
| will be there.
|
| It may be possible to use it with consumer GPUs anyway, but
| many won't try because it's not officially supported.
|
| https://rocm.docs.amd.com/en/latest/release/gpu_os_support.h.
| .. https://developer.nvidia.com/cuda-gpus
| YetAnotherNick wrote:
| > Cloud providers (ms, google, amz, etc) are quickly tiring
| of paying Nvidia monopoly premiums for their gpu hardware.
|
| I think cloud providers love exclusivity(Nvidia MSRP is
| significantly higher than it is available to clouds) and
| based on pricing compared to competitors like lambdalabs they
| have highest profit margin on GPU instances. Also based on
| availability, they likely have the highest utilisation. They
| definitely wouldn't want to commoditize the space. Google
| already has TPU that they could scale and sell to everyone
| but it would make the margins significantly smaller if they
| do it.
| michelb wrote:
| Bit of a noob on these things here, but why 192GB? Why not, say,
| 256GB? I noticed the new mac pro also caps out at 192GB, is there
| a specific reason for that number?
| zargon wrote:
| The article says that HBM3 memory comes in 24GB stacks. (Not
| 32GB.)
| wmf wrote:
| The largest DRAM dies are 24 gigabit.
| Dylan16807 wrote:
| In this case it's because memory scaling has slowed down, so
| the manufacturers have introduced sizes that are halfway
| between powers of two. It would simply take too long to wait
| for 2x before making a new product line.
|
| So the biggest HBM you can get is 24GB, and 8 of them is a
| reasonable max.
| brucethemoose2 wrote:
| The MI300 has 8x 24GB stacks of memory, which is physically the
| absolute max they can do. And those 24GB stacks are brand new.
|
| I believe the Apple M series is limited by package space? The
| LPDDR5(X?) bus is not physically/electrically limited to 192GB
| like the MI300.
| thewataccount wrote:
| Does anyone know what type of api you'd use for this? I know AMD
| has Rocm for their dedicated GPUs but it's barely usable from
| what I've heard.
| stuckinhell wrote:
| ROCm just doesn't have the support these days. Supposedly
| people are working on getting stable diffusion working on it.
| https://www.videogames.ai/2022/11/06/Stable-Diffusion-AMD-GP...
|
| But it's just too much of investment for me on something that
| MAY work. I ended up just buying a 4080rtx
| pja wrote:
| Stable Diffusion worked fine for me with Rocm & my RX580
| (once I had compiled a custom torch library IIRC).
|
| But I don't know whether it works with the more recent RDNA2
| cards.
| pohuing wrote:
| It works without a hitch on my rx6900xt. Only pain was
| getting the amdpro drivers
| thewataccount wrote:
| Yeah that's kinda what I mean. I've heard to not even
| consider AMD for ML applications specifically ROCM, so I'm
| curious if these chips will use ROCM as their primary api or
| not.
|
| It "technically" works, but their own examples crash, you get
| a fraction of the performance you'd expect for the level of
| hardware you have, chicken&egg problem with little other
| software having good support of it, etc.
| delusional wrote:
| I'm running stable diffusion on my 7900xtx and it's working
| fine. I had to screw around a little bit to get the newest
| ROCm and torch libraries since they aren't packed on my OS,
| but it wasn't that bad. I made a docker image if anybody is
| struggling to get it working:
| https://hub.docker.com/r/delusional/sd-rx7900xtx
| thewataccount wrote:
| Out of curiosity, how many it/s do you get with DPM2 at
| 512x512 with a batch size of 1, and then the it/s for
| whatever the max batch size you can fit?
| brucethemoose2 wrote:
| Stable Diffusion works fine on rocm (and intel OpenVINO), the
| issue is out-of-the-box support in popular UIs.
|
| TBH the whole space is kinda a mess. Tons of optimizations
| (like most ML compilers), even on Nvidia cards, are left on
| the table because the SD UI devs just dont have the
| throughout or motivation to implement them.
|
| At the other end, hardware makers, ml compiler devs,
| researchers and such are making quick demos, but are not
| making any integration attempts for popular frameworks.
|
| There is no one in the middle, so we are stuck with PyTorch
| eager mode and a perception that it only works on big Nvidia
| GPUs.
| sp332 wrote:
| This tools page makes me think they are sticking with ROCm
| compilers and OpenCL.
| https://docs.amd.com/category/compilers_and_tools They have a
| tool called "hipify" that supposedly converts CUDA code to
| something else. But the fact it's not just integrated as a CUDA
| compiler makes it look like they don't really trust it to work.
| Also they misspelled it.
| kcb wrote:
| HIP itself is a C++ language that can be utilized directly.
| It's extremely similar to CUDA. I think the use case AMD
| desires is the reverse. That software is written in HIP then
| converted to CUDA to also run on Nvidia GPUs.
| ilaksh wrote:
| Is it possible to do inference with Falcon 40B on this type of
| hardware or similar?
| brucethemoose2 wrote:
| Yes, but would be massive overkill. Falcon 40B takes ~35GB of
| VRAM to load now, and probably less in the future with better
| quant from llama.cpp and such.
|
| https://huggingface.co/TheBloke/falcon-40b-instruct-GPTQ
|
| Large context size is becoming less of an issue now too.
|
| But maybe it would be good for batched inference?
| ilaksh wrote:
| I was thinking maybe RunPod or Modal could use it to handle
| multiple containers per CPU, possibly including some that are
| much lighter like stable diffusion.
| brucethemoose2 wrote:
| If its multiple llm client requests, batched inference
| would be massively more efficient.
|
| And for multiple services... Probably just better to run
| multiple cheap instances and/or load dynamically? The MI300
| is super expensive.
| ilaksh wrote:
| How expensive is it compared to A100?
| zwaps wrote:
| I mean the 35GB version of Falcon is maybe not something
| you'd want to use in production
|
| Also ironically, this version of Falcon will require CUDA.
| seydor wrote:
| They showed Falcon 40B running on the mi300x
| ilaksh wrote:
| Hm. Do you have a link to that?
|
| It would be nice if this became on option for something like
| RunPod or Modal. Especially if it could be slightly cheaper
| than the Nvidia hardware somehow.
| seydor wrote:
| https://www.youtube.com/live/l3pe_qx95E0?feature=share&t=56
| 4...
| [deleted]
| fcanesin wrote:
| MI300A looks really nice, if only one anyone could buy it and do
| a DIY "Mac Studio"-like linux machine. Imagine just swapping it
| for the next generation in 2 or 3 years and keeping all the rest:
| peripherals, case, motherboard and etc.
| rbanffy wrote:
| And the x86/GPU version would be an awesome workstation. Just
| add M.2 and ethernet (and USB-C with video) and we are in
| business.
|
| As for a Hackintosh, I'd imagine an Nvidia Grace or Grace
| Hopper as a good option, even though the 500+W TDP would
| require a MacPro-sized heatsink. And it can have up to 960GB of
| RAM.
|
| edit: got a lot of details confused between the MI300 and the
| Grace.
| bratao wrote:
| I'm genuinely rooting for AMD to develop a competitive
| alternative to NVIDIA. Currently, NVIDIA dominates the machine
| learning landscape, and there doesn't seem to be a justifiable
| reason for the price discrepancy between the RTX 4090 and the
| A100.
| valyagolev wrote:
| comma.ai dude is making a whole company to achieve just that
| https://geohot.github.io/blog/jekyll/update/2023/05/24/the-t...
| synthos wrote:
| Interesting, seems like it's very similar to Modular's
| runtime portion of their business case
| brucethemoose2 wrote:
| Its not so black and white. The A100 is exponentially more
| difficult to assemble even though it is older. The silicon is
| far more specialized. You pay a price for such a "fat" node and
| server hardware guarantees.
|
| At the same time, the cost _is_ outrageous. Its not a low
| volume product.
|
| Also, a 48GB 4090 (or 3090) would be trivial. So would a 48GB
| 7900. Its not done for purely anticompetitive reasons, one that
| AMD _and_ Nvidia are unfortunately happy to go along with.
| densh wrote:
| 48Gb 4090 exists already, just with a different name,
| branding, and pricing strategy: https://www.nvidia.com/en-
| us/design-visualization/rtx-6000/
|
| 48Gb 3090: https://www.nvidia.com/en-us/design-
| visualization/rtx-a6000/
| BaculumMeumEst wrote:
| > Also, a 48GB 4090 (or 3090) would be trivial. So would a
| 48GB 7900. Its not done for purely anticompetitive reasons,
| one that AMD and Nvidia are unfortunately happy to go along
| with.
|
| Is this why Intel started taking dGPU production more
| seriously in recent years?
| [deleted]
| cycomanic wrote:
| I'm always surprised that people bring up cost as a main
| factor determining price, especially here on HN. Maybe it
| would be on a commoditized market with many competitors,
| which the GPUarket clearly isn't.
|
| As most economics 101 lecture would tell you, price is
| determined by supply and demand and in this case Nvidia
| essentially maximizing their profit based on the demand and
| segmentation of the market.
| brucethemoose2 wrote:
| Its not that simple, as Nvidia is supply constrained and
| _technically_ not a monopoly. And they have to manage a
| gaming market and share supply from TSMC /Samsung with with
| other markets.
| wmf wrote:
| There kind of are 48 GB 4090s and 7900s; they're just under
| different names.
| brucethemoose2 wrote:
| Yeah, but Quadros (and the AMD equivalents) are
| outrageously priced too.
|
| Nvidia and AMD used to "let" their manufacturers double up
| the vram on gaming cards, but no more. Its pure, collusive,
| anticompetitive market segmentation.
| splitrocket wrote:
| what names?
| ChuckNorris89 wrote:
| _> a justifiable reason for the price discrepancy between the
| RTX 4090 and the A100_
|
| 1. they're a publicly listed company operating in a free market
| in an industry they themselves helped develop who's purpose is
| to make returns for their investors, not be liked by the
| public, they don't need to justify their pricing to their
| buyers. They're not selling essentials for survival like
| insulin, baby formula, or housing, they can charge as much as
| the market will bear for their consumer electronics products.
| Don't like the pricing? Don't buy it. Simple. Buy from the
| competition instead or older generations off the second hand
| market that fit your budget.
|
| 2. the price justification is that cutting edge silicon is and
| will always be in short supply, and buyers of the silicon in
| A100 form, like datacenters, use it to make money, therefore
| it's an investment that will yield returns, therefore they can
| justify spending way more money to outbid the gamers who buy
| the same silicon in RTX 4090 form and don't use it to make
| money but use it to play games therefore for them the product
| is worth less and it makes Nvidia smaller margins than what
| selling it to datacenters can. It's basic price segmentation
| that's been going on for decades.
|
| I also don't like the GPU pricing situation but that's the
| market reality I can't change and downvoting the messenger
| won't change it either. My 2 cents.
| ActorNightly wrote:
| Yeah but the issue is that they can make more money if they
| actually did affordable ML hardware and didn't purposefully
| gimp the gaming cards.
|
| LLms running on standalone affordable boxes could become a
| ubiquitous home accessory, not to mention the potential for
| things like being able to leverage learning from widespread
| use to retrain the models, like Tesla does with their fleet.
| ChuckNorris89 wrote:
| _> Yeah but the issue is that they can make more money if
| they actually did affordable ML hardware and didn't
| purposefully gimp the gaming cards._
|
| It's a dick move but it's not illegal (at least not now).
| Their purpose is to make greater returns for their
| investors, not make affordable ML hardware for every
| consumer.
|
| _> LLms running on standalone affordable boxes could
| become a ubiquitous home accessory_
|
| They could be, but why is it Nvidia's fault if they don't
| happen? They're in the business of making money, not
| fulfilling dreams, and currently they can barely fulfill
| their orders for the datacenter customers. Consumers could
| move the LLM stuff on cheaper Apple HW or Intel or AMD SoCs
| if they can't outbid the datacenter companies for Nvidia
| silicone.
|
| Seems like a market opening for Nvidia's competitors to
| price cut them. If they don't exploit it and let Nvidia
| dominate, it's their fault. It's not Nvidia's fault their
| competitors were asleep at the wheel since they launched
| CUDA in 2007 and were helping researchers put their GPUs to
| use for parallel computing for over 15 years.
| gremlinsinc wrote:
| If AMD were smart(er), their cpu's are awesome at
| least... They would team up with intel, google, apple,
| etc to create a machine learning open cuda alternative,
| such that all GPU's could essentially be used by ML
| software equally, I think Nvidia's real hold on AI is
| that most AI libs use CUDA, and nobody else can really
| compete when all the tools use X, it's like if firefox
| launched with their own different HTML syntax because
| HTML was proprietary, and every other browser did the
| same thing.
|
| Something like chrome : chromium dynamic would work
| wonders for bringing a sort of standard, not to mention
| more minds could create better and better
| implementations.
| [deleted]
| ornitorrincos wrote:
| They did try, apple made OpenCL and released it to
| khronos to make a standard, but never got wide
| support(well, they got plenty of manufacturers on board,
| not so much users).
|
| By that time nvidia already had an advantage with CUDA
| and their opencl support always lagged.
| WheatMillington wrote:
| NVIDIA are not stupid, they understand their marginal
| revenue and marginal cost analysis, and they obviously
| decided they're making the most money doing what they're
| doing. It's asinine to believe we, with access to almost
| none of the requisite data, could tell NVIDIA how to turn a
| better profit.
| scld wrote:
| It's because you're responding as if OP said it wasn't
| justifiable legally. They're just saying that AMD is, in
| their estimation, making a bad business decision.
| ChuckNorris89 wrote:
| That's was really not his point. His point was that Nvidia
| is price gouging and I explained him why they can get away
| with it since the current market conditions favor their
| products a lot.
|
| AMD has been consistently making bad business decisions in
| the GPU space for years now. What's new? They're pricing
| their hardware near NVidia but with less performance and
| way less features and poor track record, especially for the
| ML users.
| wwtrv wrote:
| Nvidia is price gouging because there is no credible
| competition. That's the entire explanation and there is
| nothing more to it.
| ChuckNorris89 wrote:
| Why are people here continuously surprised by this and
| keep asking the same redundant questions: _" Why does
| Nvidia charge so much?"_ I just don't get it.
|
| Thinking that Nvidia is some evil villain doing it to
| consumer out of spite, when they're just doing what any
| other company in their dominant position would do: charge
| as much as the market will bear.
|
| Apple also doesn't have to charge you $200 for
| configuring the 512GB SSD over the 256 SSD which only
| costs them an extra $5 NAND chip, but they do it because
| they can. So does Nvidia and any other company in a
| dominant position with virtually no competition.
| The_Colonel wrote:
| > His point was that Nvidia is price gouging
|
| You might be reading into their comment too much, they
| don't really blame NVidia, just wish for more
| competition. IMHO you're being downvoted because you come
| across as needlessly confrontational to an innocent
| comment.
| ChuckNorris89 wrote:
| I sound confrontational because it seems like the upvoted
| poster didn't understand the laws of supply and demand
| and neither do the people who upvoted it to the top, or
| was just karma farming by asking redundant/self
| explanatory questions hoping to get votes from the mobs
| with their pitchforks out.
|
| Maybe I'm wrong and OP really didn't understand the
| curent supply/demand issues which is why I gave a lengthy
| explanation of how it looks.
| [deleted]
___________________________________________________________________
(page generated 2023-06-13 23:03 UTC)