[HN Gopher] For people who still don't get how bad DIMMs are
___________________________________________________________________
For people who still don't get how bad DIMMs are
Author : tosh
Score : 39 points
Date : 2022-06-30 15:29 UTC (7 hours ago)
(HTM) web link (twitter.com)
(TXT) w3m dump (twitter.com)
| h2odragon wrote:
| This gives me a vision of a "slotcket" thing with CPU and a
| _bunch_ of RAM on it, like 128GB for today 's CPUs. gang these up
| on your expensive bus for bigger systems, package them in ceramic
| blocks with fluid ports, and... oh that's what they're talking
| about for "HPC" now innit.
|
| human manipulable components are too big for the interconnect and
| density we want now... and those are _already_ too small for easy
| cooling.
| Avlin67 wrote:
| I have 1TB/s on my Radeon VII
| verall wrote:
| If you need a ton of memory bandwidth and have the money then you
| build your product around HBM2 or similar. There are routers,
| switches, IPS, etc built like this today. Most consumer computer
| systems only need big memory bandwidth for graphics, which is why
| video cards have fast on-board memory. Apple of course doesn't
| want to buy another vendor's graphics hardware and has experience
| designing their own bespoke hardware, so they designed an
| integrated system which of course then needs big memory
| bandwidth. It's an interesting bet that is certainly paying out
| in terms of power efficiency, but it's not some impossible feat
| of engineering.
|
| Intel is married to the commodity server market and their
| laptop/mobile products have always been trickle-down, because
| they are a giant megacorp whose roadmaps are dictated by their
| biggest profit centers. Even if they designed this kind of
| system, they wouldn't be able to capture most of the profit from
| it like Apple can, since they don't sell memory or systems, they
| sell CPUs. After the mobile debacle, Microsoft pivoted to SaaS
| and is way too chickenshit to start designing hardware. Apple is
| the only company incentivized to produce a product like this for
| consumers. This is not going to suddenly become profitable for
| the commodity server market; and it doesn't have to; it's not a
| market Apple is particularly interested in.
| rektide wrote:
| > _If you need a ton of memory bandwidth and have the money
| then you build your product around HBM2 or similar_
|
| What does similar mean? I'm not sure I agree.
|
| The M1 is basically a classic big-iron architecture,
| miniaturized with chiplets. The top M1 Max config is
| effectively 8x channel/512-bit-wide, just like a current epyc.
| It's faster though because it's 5.2GT/s LPDDR5 memory (400GBps)
| vs 3.2GT/s DDR5, and it's much smaller because it appears as 4x
| stacks on package. (Next gen Genoa is up to 12x channels of
| 5.2GT/s DDR5 fwiw.) But it's still a conventional play,
| basically, with new packaging.
|
| > _Intel is married to the commodity server market and their
| laptop /mobile products have always been trickle-down, because
| they are a giant megacorp whose roadmaps are dictated by their
| biggest profit centers._
|
| Generally, yes! Notably sometimes innovation is first delivered
| in mobile and "trickles up," but basically it's because mobile
| is a safer easier space to innovate/explore/learn in (as it's
| much less investment cost to make the much smaller/lighter
| chips there). Generally you're right about what leads the
| business.
|
| > _Even if they designed this kind of system, they wouldn 't be
| able to capture most of the profit from it like Apple can,
| since they don't sell memory or systems, they sell CPUs._
|
| It's definitely a little intimidating figuring out how to sell
| bundled on ram. The technical benefits are hard to ignore
| though. Total cost of ownership should improve for customers
| notably, via less intense/simpler motherboards, via low power
| consumption stemming from much more direct channels & stronger
| signal integrity to main memory. But you're right, I think
| there's a lot of unappealing & unwanted aspects to becoming a
| company that ships ram. It'll be hard to resist/ignore for long
| though.
|
| AMD hasn't shown DRAM related interest yet, but their 5800X3D
| has 96MB stacked L3 v-cache on a separate chiplet. Their
| upcoming Genoa cpus will have ~1GB stacked L3 V-cache, again at
| absurd terabits-per-second speed, also via chiplets. AMD is
| pushing the memory heirarchy wider, is doing something like
| this M1-ening, but at a at higher-throughput lower-latency but
| smaller-capacity points, while leaving the conventional RAM
| channels also available. But they could, pretty easily I'd
| wager, snap back & deliver integrated DDR if they wanted, saw
| the need.
|
| Now that there are groups like Universal Chiplet Interconnect
| Express, this tech is kind of semi-commodified, available for
| purchase, to a degree. Yes, the mainstream market may lag. But
| with the capabilities arriving, it's hard to imagine this tech
| staying out of the data-centers for too long. Once we see some
| hyperscalers or upstarts delivering chips with integrated DDR,
| I expect the appeal will be irresistible/unsuppressable. It'll
| become obvious/known that this is good tech that is table
| stakes for selling chips, like it or not.
| comprambler wrote:
| Indeed, any spec of HBM is not going to be comparable to an
| ondie memory controller having to reach out to individual DIMMS
| on a traditional motherboard. I appreciate the work this person
| is putting in to Asahi linux, but on the whole this is an
| unfair comparison.
| snake_plissken wrote:
| I don't really understand the point being made here. Is it that
| high-end bespoke Apple hardware is faster than the equivalent
| commodity hardware? Or is the person trying to explain why the
| memory chips are integrated directly into the chipset (making
| them extremely fast at the expense of being not so easy to
| upgrade/replace)?
| tommek4077 wrote:
| The second. He wants to make clear, why it is a good idea for
| Apple to integrate RAM.
| wolpoli wrote:
| Sounds like that Intel/AMD will need to move toward integrating
| RAM into the CPU package.
| wmf wrote:
| The bandwidth is not really for the CPU; it's for the GPU and
| GPUs have been using soldered RAM forever. Maybe Marcan should
| compare against a 3090.
| wolpoli wrote:
| If bandwidth isn't the bottleneck in real world workloads,
| then I am not sure what point Marcan is trying to make by
| highlighting this particular characteristic.
| webmobdev wrote:
| That's ok. I'll choose the ability to self-repair and upgrade,
| even with the "limited" bandwidth of user-replaceable RAM, over
| soldered or "integrated" RAM and SSD for planned obsolescence
| anytime.
|
| The base model aluminium Intel Mac Minis started out with 2GB
| RAM, then 4 GB and finally 8 GB - all over a span of 8 years.
| Many of these models would be useless today if you couldn't
| upgrade the RAM at all. In fact, the 2014 Mac Mini models with
| soldered RAM (LPDDR) received a very poor reception and Apple was
| forced to backtrack and go back to user-replaceable RAM with the
| next Mac Minis ( https://everymac.com/systems/apple/mac_mini/mac-
| mini-aluminu... ).
|
| Many of the early adopters of Apple Silicon M1 devices noticed
| heavy usage of their SSD, which many attributed to heavy swapping
| due to the limited (non-upgradable) RAM in the M1 devices. Apple
| finally said it was a bug (
| https://appleinsider.com/articles/21/06/04/apple-resolves-m1... )
| and recommended upgrading macOS. But many of these early adopters
| are less than satisfied and still spend a lot of anxious time
| worrying whether their _soldered_ SSD 's life has reduced by a
| year or two because of the increased usage. Their concerns are
| real because with soldered SSDs, you obviously have to do a very,
| very costly repair through Apple if the soldered SSD chips conk
| out.
|
| Imagine if you have to replace your whole CPU / SoC to upgrade
| just your RAM because the RAM is now integrated with the CPU /
| SoC!? (This is ignoring that even CPUs are now unnecessarily
| being soldered in many Apple and non-Apple desktops and not
| easily upgradeable ...).
|
| At some point, the cost-benefit analysis of such soldered /
| integrated parts doesn't work out in favour of us consumers. And
| that's what we need to consider before blindly giving up ease of
| repair for some brief performance increase. (While I have
| specifically highlighted Apple because they are the only one
| currently using integrated RAM with their CPU / SoCs, my
| criticism is directed at the others too, including Dell, HP,
| Lenovo etc. who are following the same path of planned
| obsolescence by soldering components to make them less repairable
| and reduce their life. Unless we start supporting repairability -
| like with https://frame.work/ devices - we consumers are in a
| very slippery slope of losing our right to own and repair our
| devices).
| acuozzo wrote:
| > That's ok. I'll choose the ability to self-repair and upgrade
|
| How do you upgrade your L2 cache? You're not still using a 486,
| are you?
|
| Did you protest the Pentium as well?
| rektide wrote:
| What an excellent simple post. Amazing how our of sync, how
| ridiculous the situation is. The board real-estate for DIMMs is
| out of control. I'd say smaller form factors would be so
| excellent, but short of serious change it feels like an
| incommensurate ask, like it's not going to get us to really
| better.
|
| Yesterday there was an article on Intel's newest integrated
| silicon photonics, this time having some of the photonics on
| stacked chiplets on the chip[1]. It's a different sort of
| integration and micro-miniaturization, but feels very much the
| same. Not needing a big ole NIC, not needing to have extra, power
| sucking PCIe transcievers at each end, by moving capabilities
| onto the chip complex would be amazing. I'd love to see ultra-
| dense servers of the future where there are dozens of cpus on a
| sled, with basically just power & optical io feeding each chip.
| We've bounced off the "disaggregated rack" a couple times now,
| but it also feels like a major phase transition has been building
| for a while to.
|
| [1] https://news.ycombinator.com/item?id=31924382
| Lammy wrote:
| I'm so tired of this person's constant fawning over the shiny and
| fast but proprietary ARM Macs like the raw performance is the
| only criterion that should matter to everyone else too.
| wmf wrote:
| Reverse engineering Apple's proprietary stuff is literally his
| job now.
| draw_down wrote:
| wngzro wrote:
| LTT just made a video covering similar points -
| https://youtu.be/LFQ3LkVF5sM
| incahoots wrote:
| Good looking out!
| citrin_ru wrote:
| I still don't get it after reading this tweet. What would be more
| informative to me - a table for M1 and a system with the same RAM
| size but DIMMs which would contain: 1. random access latency 2.
| linear access bandwidth 3. price
| wmf wrote:
| 1. basically the same
|
| 2. basically the same
|
| 3. basically the same
|
| The advantage to Apple's soldered RAM is that the motherboard
| is 10x smaller.
| tinus_hn wrote:
| So where is the bottleneck? Is it the length of the lanes? The
| number of lanes? Or something else?
| controversial97 wrote:
| A combination of capacitance that has to be charged, crosstalk
| and characteristic impedance discontinuities.
|
| The signals move between the cpu and the ram at about one
| nanosecond per foot. A nanosecond is one full cycle of a 1GHz
| clock.
|
| The characteristic impedance of the connection is set by the
| physical size of the pcb track and the dielectric constant of
| the pcb material.
|
| As the signals move between the cpu and ram at nearly the speed
| of light if they hit an impedance discontinuity some of the
| energy is reflected back. That messes up the signal.
|
| Making a connector with hundreds of pins that works reliably
| and does not cause a significant impedance discontinuity is
| hard.
|
| The lower the impedance of the connection, the more power is
| required to drive data through it.
|
| A bit error rate of one in a billion would corrupt data or
| cause a software crash daily so signal integrity is a hard
| problem.
| goosedragons wrote:
| Except ONE of those sockets has the same memory ceiling as an M1
| Studio Ultra. So while you need a lot of space to get the same
| bandwidth you still need a lot of space to get a high memory
| ceiling. DRAM chips are only so small.
|
| Right now the Mac Pro has 2TB memory limit. Apple's either going
| to have to commit considerable board space to DRAM, use sockets
| or just completely obliterate the max amount of RAM a M* series
| Mac Pro can have.
___________________________________________________________________
(page generated 2022-06-30 23:03 UTC)