[HN Gopher] Micron Kicks Off Production of HBM3E Memory
___________________________________________________________________
Micron Kicks Off Production of HBM3E Memory
Author : 0xd1r
Score : 93 points
Date : 2024-02-27 10:37 UTC (1 days ago)
(HTM) web link (www.anandtech.com)
(TXT) w3m dump (www.anandtech.com)
| brucethemoose2 wrote:
| How about higher capacity GDDR6X?
|
| 48GB consumer cards (or 96GB pro cards) would sell like hotcakes
| if AMD/Intel dare to break the artificial VRAM segmentation
| status quo.
| teaearlgraycold wrote:
| Build it and people will start porting their CUDA stuff to run
| on other architectures.
| Brananarchy wrote:
| You over estimate the 'semi-pro' market for graphics cards.
| Gamers are barely willing to pay for 20GB. There's no market
| for consumer cards with an order of magnitude more RAM until
| games are built to use that memory.
| cinntaile wrote:
| 48GB cards would sell like hotcakes. The problem is that they
| would sell way less cards aimed at professionals, where they
| have much higher margins.
| AnthonyMouse wrote:
| Intel doesn't sell a lot of graphics cards whatsoever
| though. Be the first to offer 64GB of VRAM for under $1000
| and that could change pretty fast.
| xadhominemx wrote:
| Not without CUDA unfortunately.
| KeplerBoy wrote:
| Don't underestimate the amount of shit people would be
| willing to deal with to make stuff work.
|
| A capable GPU with 24+ GB would sell if it significantly
| undercuts Nvidia. Just look at geohot building his
| tinyboxes with AMD cards.
| xadhominemx wrote:
| I would personally love that project but there are
| already so many versioning issues in the space it would
| be a nightmare if ROCm randomly broke things all the
| time.
| KeplerBoy wrote:
| I agree, ROCm seems to be a mess from the outside, but
| I'm glad people are putting in the effort.
| AnthonyMouse wrote:
| And we're talking about Intel here. AMD is going to price
| competitively against Nvidia but they'd still rather you
| buy a $20,000 MI300 than a hypothetical 128GB Radeon for
| $2000.
|
| Intel could very easily just put a buttload of VRAM on
| their existing GPUs to stick it to their competitors and
| make out like bandits. All they'd have to do is charge a
| Big markup instead of an Enterprise markup. And Intel has
| a better history of not making broken libraries.
| ianbutler wrote:
| This is for ML, not gamers. There is an entirely different
| market here.
| usrusr wrote:
| But "basement ML" is a thing, the market of people who are
| interested in PC gaming but not to the point of being
| lifestyle gamers who throw every cent they can spare at
| that altar. The GPU they bought long before the pandemic is
| still running every game they throw at it, but they never
| completely stop eyeing the new stuff. Dipping their toes in
| ML, even if it's just getting through 80% of some stable
| diffusion setup tutorial, can be a very welcome excuse to
| upgrade their gaming. A card sold for gaming but with
| generously overprovisioned VRAM (ideally in the range of
| the lowest bin of the biggest or second-biggest chip I
| think) could match that market segment very well - and it
| would not only compete with other price points, it would
| actually increase the market by some buyers (those who
| would not upgrade without the "ML excuse").
| HeWhoLurksLate wrote:
| I swear I'm getting Deja Vu right now, I coulda sworn I've seen
| this thread before. There's gonna be a guy commenting that "you
| don't need it" somebody else saying "but I want it!" And a few
| trying to figure out the economics of it and whether or not it
| makes any sense.
|
| Personally I'd love to have as much VRAM as possible (and as
| high a bandwidth as is possible too) to mess around with
| simulations in- but that's _definitely_ a pro workload.
|
| I'd love to see like a flagship card have a stupid amounts of
| VRAM spec option - like an RTX 4090 with 32-48gb of VRAM just
| to see what happens with it on the market.
| omneity wrote:
| A friend just got an M3 Max with 128GB (V)RAM and he's
| extremely happy with it (AI workloads wise). That could be an
| option if you can run your simulations on macOS.
| AnthonyMouse wrote:
| That machine with 128GB is $5000, with 48GB is still well
| over $3000, and has as much memory bandwidth as a $400 GPU.
| At the current spot price, 128GB of GDDR6 is <$400 and 48GB
| is <$150, implying that they could be paired with any
| existing <$1000 GPU to produce something significantly
| faster for dramatically less money. If anyone could be
| bothered to make one.
| omneity wrote:
| I agree with you, that's how it should be, but that's not
| how it currently is.
|
| Looking at what's available right now. You need 3 A100
| 40GB to get this amount of VRAM which will cost you way
| north of 20000$.
|
| Doing it with A6000s is still about 15k$.
|
| There's not that many high VRAM options out there you
| know..
| AnthonyMouse wrote:
| If all you want is VRAM you can get old P40s with 24GB
| for $175 and it's 144GB for $1050. Then you need a big
| machine to put six of them in but that doesn't cost
| $4000.
|
| But all of this is kludges. The Radeon RX 7900 XTX has
| more than twice the memory bandwidth of the M3 Max with
| much better performance per watt than an array of P40s.
| What you want is that with more VRAM, not any of this
| misery.
| omneity wrote:
| That checks out in principle, but given that P40 doesn't
| support NVLink, I wouldn't count too much on using six of
| them together in a performant manner.
|
| But yeah the best option remains an MI300 if you can
| afford that.
| a_wild_dandan wrote:
| Yeah, my M2 MacBook has 96GB @400GB/s. For $4k or so, it
| feels like cheating. Does it beat 4x24GB NVIDIA cards?
| Absolutely not! It's slower and occasionally runs into
| CUDA-moat software issues. But the capability to daily
| drive Mixtral 8x7 locally, with great token speeds, is
| _phenomenal_.
| AnthonyMouse wrote:
| NVLink is most needed for training. For inference a lot
| of the popular models can usefully be run on multiple
| GPUs without it:
|
| https://www.reddit.com/r/LocalLLaMA/comments/142rm0m/llam
| acp...
| hydroreadsstuff wrote:
| https://semiconductor.samsung.com/news-events/tech-blog/a-br...
| hkgirjenk wrote:
| Don't you think Nvidia analyzed, sliced and diced the market to
| figure out how to maximize profits?
|
| AMD is doing the same thing, the only high memory cards they
| put out (MI300) are for data centers.
| andy_ppp wrote:
| It's possible to come up with many strategies and different
| companies will. Why are you so sure that Nvidia's strategy is
| right for AMD or Intel who need to offer differentiation to
| get over the CUDA moat?
| xadhominemx wrote:
| NVIDIA has actually not really sliced and diced the market.
| They only sell Cadillacs, which is fine for now because
| they're the only game in town.
| hmottestad wrote:
| And Samsung just launched 36 GB packages of HBM3E:
| https://www.anandtech.com/show/21278/samsung-launches-12hi-3...
|
| Combined with this[1] interesting paper from summer 2023 on HBM
| combined with Xeon processors which would now allow for 144 GB on
| a single CPU. In theory at least.
|
| 1: https://lenovopress.lenovo.com/lp1738-implementing-intel-
| hig...
| hiddencost wrote:
| > launched
|
| Not true. They announced completion of development, with
| intention to begin mass production in H1. So a couple months
| out, at least.
|
| > Micron's memory roadmap for AI is further solidified with the
| upcoming release of a 36 GB 12-Hi HBM3E product in March 2024.
|
| So likely competitive timing with Samsung for 36/12.
| hmottestad wrote:
| Went by the title, but yeah, announced is more fitting.
| omneity wrote:
| Would this explain why I started seeing so many ads from HBM
| manufacturers with no product an individual like me can purchase?
| (I'm not even close to hardware, professionally)
|
| Not naming the company but seems like HBM manufacturers might be
| going all-in to benefit from Nvidia's stock surge.
| fcanesin wrote:
| By the article (March production) new GPU eng. samples should be
| flying in H1 which means new high memory GPUs coming end of the
| year. 2025 will be extremely exciting for ML.
| spintin wrote:
| This is an empty article:
|
| - CAS latency?
|
| - Wattage?
|
| Also: https://piped.video/watch?v=2G4_RZo41Zw (is this memory
| same size as 5nm?)
| jdblair wrote:
| The linked press release is a better source, anandtech didn't add
| much.
|
| https://investors.micron.com/news-releases/news-release-deta...
| mechagodzilla wrote:
| What is the read latency like on HBM3E?
___________________________________________________________________
(page generated 2024-02-28 23:02 UTC)