[HN Gopher] The Great CPU Stagnation
___________________________________________________________________
The Great CPU Stagnation
Author : greghn
Score : 176 points
Date : 2023-05-18 15:33 UTC (7 hours ago)
(HTM) web link (databasearchitects.blogspot.com)
(TXT) w3m dump (databasearchitects.blogspot.com)
| cubefox wrote:
| It isn't just CPUs either. For GPUs, a 2018 analysis [1]
| estimated that FLOPS per dollar only doubled every 3.9 years.
| With recent disappointing improvements in new graphics card
| price/performance, I expect this number got even worse now.
|
| [1] http://mediangroup.org/gpu.html
| lynx23 wrote:
| The perfect time to shed ourselves of the idea that "optimisation
| is a waste of dev-time". Mobile computing was the last godsent t
| actually rethink performance a little bit, but we still have a
| lot of relatively low-hanging fruit. I sometimes dream about a
| month-of-no-new-features, where everyone would just have a bit of
| time to clean up and improve on existing stuff.
| RajT88 wrote:
| OS and device manufacturers don't necessarily want to optimize.
| Optimization would reduce the hardware replacement cycle,
| cutting into their profits.
|
| Microsoft gets pressure from their hardware partners to keep up
| the hardware replacement cycle, and they themselves of course
| get a cut of that via OS licenses.
|
| Phone manufacturers seemingly invented "Always On Displays" to
| also cause people to start to think they needed a new phone or
| battery just a year/18 months into owning their device. Christ
| what a waste of power for such little value delivered! My wife
| said her new S23+ would barely last a day - so I turned off the
| always on display option, and now it lasts 2.5/3 daysish of
| normal usage.
|
| It's a slightly different story with laptops, where battery
| life is an important feature. You'd think battery life would be
| important for phones too.
| tempestn wrote:
| Huh, the always on display on my S23 non-plus was only a 10%
| battery hit over a day or so. Still disabled it, of course.
|
| Also nice that they've finally added an option on Samsung
| phones to only charge to 85%, but it would be nice if it had
| smarter options like on iOS. Eg, an option to charge up to
| 100% just before you expect to wake up, so you have the full
| charge but it doesn't sit at 100% all night degrading the
| battery.
| eppsilon wrote:
| iOS has an Optimized Battery Charging option that does
| something like that. You can't set a schedule, though -
| it's based on your usage patterns.
| danielvaughn wrote:
| More like a year - the performance rot runs deep.
| MichaelZuo wrote:
| Considering that the NeXTStep OS ran on a 1120 x 832, 12 bit,
| display in 1990 with 12 MB of archaic memory and 1.5 MB of
| VRAM, I'd say more then a year. Judging by modern app memory
| usage.
|
| Even with the latest and greatest 4k, HDR, wide colour,
| display, no app should ideally use more then 256 MB of memory
| by those standards, unless it's even more complex.
| Arrath wrote:
| Recently watching a Gamer's Nexus video where they tested a
| new built Voodoo 6 5000, I was impressed watching them
| install some ancient edition of Windows in which to do the
| testing, and how snappy and responsive the interface was.
| danielvaughn wrote:
| My first computer was a 95 Packard Bell. I was 12 years
| old at the time, so I might be misremembering, but I
| swear the interface responsiveness felt _immediate_.
| danielvaughn wrote:
| We might need to block all software features for the next
| decade so we can figure out what's going on.
| jorvi wrote:
| This is why I loved every console pre-PS4, from a technical
| standpoint. The amount of performance (think God of War 2 or
| Black) devs were able to squeeze out of the PS2 was nothing
| short of staggering.
| dietrichepp wrote:
| A more extreme version of that is "permacomputing"--
| https://permacomputing.net/projects/
|
| I still regularly use old systems for fun. I grew up with Macs,
| so that's the point of reference. When I use these old systems,
| I pine for a few specific things I'm used to on newer systems--
| but it almost feels like nitpicking.
|
| The things that I _really_ want from a computer are pretty
| basic. Like good, consistent copy /paste and drag 'n' drop,
| good autosave, good file browser, that sort of thing. It seems
| like new, half-baked stuff got dropped in our laps before the
| basics really got perfected.
| ianlevesque wrote:
| It's been a little frustrating watching iOS rip out literally
| all file management at launch and then slowly reintroduce it
| all feature by feature one at a time on iPad. Recently you
| can even mount network shares and use USB drives again.
| didgetmaster wrote:
| It would be nice if every developer had to run their latest
| build on 10 year old hardware while testing it out.
|
| Instead of their high-powered development machine with the
| latest CPU, tons of high speed memory, and the fastest SSD;
| they would get to experience what many of their customers have
| to endure on slower hardware with capacity constraints.
|
| Nothing spurs optimization like seeing first hand how your code
| creeps along on slow hardware.
| whynotmaybe wrote:
| Don't say it too loud, friend of mine worked at a company
| where everyone had the same computer.
|
| I5 Gen 3, 500g HDD and 8gig of ram.
|
| Yes, outlook runs fine for the secretary, visual studio not
| so fine for debugging.
| didgetmaster wrote:
| I wasn't suggesting that all the developers have to develop
| and compile their code on antiquated hardware. They should
| still have fast development machines.
|
| What I was saying that if the developers ONLY run their
| software on their high-powered computers and never try it
| on slow hardware, they generally resort to the 'it runs
| fine on my machine' response when customers start
| complaining about performance.
| ilyt wrote:
| *and* slow-ish internet.
|
| We had few surprises when some newbie dev noticed that the
| site doesn't work quite as well outside of 1Gbit connection
| with 1ms ping to the app server...
| tambourine_man wrote:
| Google cache version
|
| http://webcache.googleusercontent.com/search?q=cache:FBaO-jB...
| bhouston wrote:
| This "stagnation" is nothing like the stagnation during AMD's
| poorly performing Bulldozer era (the post Athlon era) where they
| were consistently beat by Intel's offerings and there was a
| general lack of innovation in the prosumer space.
|
| During that era for the most part Intel's i7 prosumer CPUs
| started with 4 cores with the Bloomfield Nehalem chips in 2008
| (which at the time were awesome and a game changer) and ended
| with 4 cores with the Kaby Lake-S in 2017. It really only changed
| in 2017 with AMD Ryzen forcing Intel to actually increase core
| count.
|
| 2008 Nehalem benchmark:
| https://cpu.userbenchmark.com/SpeedTest/778/IntelR-CoreTM-i7...
|
| 2017 Kaby Lake-S benchmark: https://cpu.userbenchmark.com/Intel-
| Core-i7-7700/Rating/3887
|
| When I compare the two, it shows an effective 20% speed increase,
| although microbenchmarks show a 50% increase. That is a
| stagnation.
|
| During that era it felt like a lost decade. I don't miss it.
| semi-extrinsic wrote:
| Can confirm, I bought a pair of desktops for work in 2012 with
| Ivy Brige i7's that could "turbo boost" to 3.9 GHz indefinitely
| without overclock. I did not feel a real upgrade need until the
| 32 core Threadripper machines came out in 2018.
| amiga-workbench wrote:
| I only just upgraded from my i5 3570K to a Ryzen 7 5800X3D.
|
| Not because anything I was currently doing with my computer
| was becoming too slow, but because I wanted to do new things
| (VR). A shame, I wanted to run that thing into the ground.
|
| I've got my old system sat spare, I'm not sure what to do
| with it.
| dmarchand90 wrote:
| Nice processor choice... vrchat in mind? Haha
| ilyt wrote:
| my i7-4790 served me nicely for 8 years. Only upgraded
| because now most games were CPU-limited, even on "only" GTX
| 1070. But I went "all in" "I don't want to upgrade for
| quite a long time" with 7800X3D. Maybe GPU upgrade in 2-3
| years...
| jeron wrote:
| the one good thing to come out of that era was the "moar cores"
| meme
| bhouston wrote:
| heh... https://9gag.com/gag/aRPw3xM
| ketralnis wrote:
| he's wearing a silly hat because it's predated by this http
| s://www.reddit.com/r/gaming/comments/tc4aw/make_more_hat...
| frozenport wrote:
| Compute power continues to increase as evidenced by faster
| accelerators/gpus and top500 flop counts.
| paulddraper wrote:
| "When you can't scale up, scale out."
|
| GPUs have strong potential for improvement, and moving workloads
| to them helps on multiple fronts: performance, cost, power
| consumption.
| mcguire wrote:
| _If_ your workload is the embarrassingly parallel type that
| GPUs are designed for.
| kayson wrote:
| We've known about this for a long time. Everyone expected it to
| happen. There are some key upcoming technologies that have the
| potential to cause a step in scaling (CFETs, backside power
| delivery) but it's still not going to be anywhere near Moore's
| law levels. I think this is part of why GPU power is skyrocketing
| and why Apple, Qualcomm, and the like are trying to shift towards
| services.
| qayxc wrote:
| > I think this is part of why GPU power is skyrocketing and why
| Apple, Qualcomm, and the like are trying to shift towards
| services.
|
| IMHO it's only a very small part of why GPU power consumption
| is going up. The main reason is the completely unnecessary
| chase for the performance crown.
|
| From personal testing: my GPU manages to get 95% of its peak
| performance while being power limited to 80%. So the in order
| to squeeze the last 5% of performance out of the device, 20%
| more power is pushed through it. It stays above 99% peak
| performance while being power limited to ~87%.
|
| But even just looking at the raw numbers paints a different
| picture. About 12 years ago, a high-end GPU (e.g. GTX 480) had
| a power draw of 250W at a theoretical peak FP32 performance of
| 1,345 GFLOPS. This year's RTX 4070 has a theoretical peak
| performance of 29.15 TFOPS at 200W, so we went from 5.38
| GFLOPS/W to 145.75 GFLOPS/W in 12 years - a 27x improvement in
| efficiency and a ~22x improvement in raw performance.
|
| Now let's compare that to the numbers from a decade ago: a GTX
| 580 from 2010 had a power rating of 244W at 49.41 GTexel/s. A
| Geforce2 Ultra from 2000 used about 10W at 2.0 GTexel/s. So we
| went from 0.2 GTexel/s/W to - you've guessed it - 0.2
| GTexel/s/W, so same efficiency with a ~27x increase in
| performance over a decade, though the efficiency is only a
| guess, since neither GFLOPS nor official power draw figures are
| readily available for 2000-era hardware.
|
| Fast forward a few years so we can get reliable power draw
| numbers and comparable performance in GFLOPS, we have the high
| end GeForce 8800 GTX at 155W for 345.6 GFLOPS in 2006. Ten
| years later, the comparable model would have been the GTX 1080
| from 2016 with 180W at 8.873 TFLOPS. So 2.2 GFLOPS/W versus
| 49.3 GFLOPS/W or a 22x increase in efficiency and a ~26x
| increase in performance over the course of a decade.
|
| So during the past 23 years, power efficiency steadily
| improved, while raw performance increase also showed no signs
| of slow down in the GPU space. This is given the same generous
| time frames, to account for the occasional generational leap.
| pixl97 wrote:
| I would also think that GPU workloads have something to do
| with it. Almost none are serial workflows, and instead highly
| parallel work.
|
| GPU workloads will eventually run into the same scaling
| limits. That is we will be unable to speed up each execution
| unit any further, or the primary work we give the GPU will
| not be able to be split into more threads and accomplish
| useful work.
| qayxc wrote:
| Good point about the workload. On the other hand, typical
| GPU tasks seem to be scalable basically ad-infinitum as
| graphics moved from fixed-function pipelines to per-pixel
| shaders to raytracing and path tracing.
|
| So maybe GPUs still have some room until they run into the
| same problem as CPUs.
| pklausler wrote:
| Multi-GPU nodes with fast coherent interconnects exist.
| Yoric wrote:
| Well, and photonics, quantum computing, etc. But they're not
| there yet.
| [deleted]
| moffkalast wrote:
| Well the main problem is resistance isn't it? Most of the power
| "used" is to get electrons to flow fast enough for the logic
| gates to settle for a specific clock frequency and the
| resistive losses to heat.
|
| The only real way forward that isn't a temporary workaround
| seems finding a new type of semiconductor that has lower
| overall resistance than silicon. Whoever figures out how to
| dope graphene and produce wafers without defects will probably
| make trillions.
| timerol wrote:
| tl;dr New materials can help, but "resistive losses" aren't
| really the driving factor.
|
| The energy is a mix of leakage current and active current.
| Leakage current can be thought of as resistance - it's how
| much current flows through a transistor that's off. This can
| be better based on the material, but gets harder with smaller
| transistors. (Thinking about quantum tunneling as a
| resistance is good to get intuition, but not good enough to
| help solve the problem. A material with a lower bulk
| resistivity will not help here.)
|
| Active current is based on capacitance. Each FET has a little
| capacitor that needs to be charged and discharged every time
| the logic is switched - that adds up. Lowering the
| capacitance of each FET would reduce the energy required to
| switch it, but generally comes with bad tradeoffs. High-k
| dielectrics increase the capacitance, all other things being
| equal. But all other things are not equal, and they are used
| to create better performing FETs with lower power leakage.
| saltcured wrote:
| I thought leakage current would be the "DC" loss that is
| independent of frequency, like we had in old bipolar logic.
| Isn't it fair to characterize the cmos/fet switching losses
| as resistance to moving the charges around?
|
| I understand leakage will go up if we increase voltages to
| support higher switching speeds, but aren't there still a
| lot of losses that happen with logic transitions and reduce
| when the states are stable, even if voltages are held
| constant?
|
| I realize it we can't move charges around for free, but in
| some fantasy superconducting-fet logic circuit, wouldn't
| the power consumption be reduced? I.e. much of the waste is
| resistive losses while charging and discharging those
| gates.
| timerol wrote:
| > Isn't it fair to characterize the cmos/fet switching
| losses as resistance to moving the charges around?
|
| Not really. It makes more sense to think about it as
| filling and emptying capacitors. You are charging the
| gate capacitance up to the supply voltage, then dumping
| that charge to discharge the gate to 0 again. The energy
| of each capacitance that gets charged and dumped is
| CV^2/2, which happens for each logic transition.
|
| > I realize it we can't move charges around for free, but
| in some fantasy superconducting-fet logic circuit,
| wouldn't the power consumption be reduced?
|
| If there was no resistance when distributing charge, it
| would help a bit, but not enough to change the clock
| frequency by more than 20%, assuming that the fantasy
| superconducting-fet had normal leakage and gate
| capacitance.
| saltcured wrote:
| So the charge is work and the discharge is waste?
|
| I guess I am entertaining the idea of an idealized
| Maxwell-demon CMOS circuit, if we could bounce the charge
| between gates with very little work to just pump the
| charge back and forth.
| timerol wrote:
| That's a reasonable way to think about it - you take
| energy from the supply voltage to charge the gate
| capacitor when the logic line goes high, then dump it
| when the logic line goes low.
|
| If you had a lossless bidirectional voltage converter
| circuit for each gate capacitance, then you could charge
| the capacitor from the supply and discharge it back into
| the supply, removing any switching losses.
| vlovich123 wrote:
| I think two major revolutions would be optical and
| reversible computing. The former would significantly shrink
| the heat generated which is a huge bottleneck but is very
| hard to build generic computing out of and expensive. The
| latter would basically result in computing obtaining a new
| theoretical lower bound on energy required but is purely
| research with no known approaches for actually building the
| things.
|
| Asynchronous clockless designs might also drastically cut
| the power budget but those have failed to find adoption for
| some reason.
| orbital-decay wrote:
| Clockless designs did find their use, just not for the
| entire chips. Certain parts of modern CPUs are
| asynchronous.
| vlovich123 wrote:
| Yeah, I'm just a bit surprised it didn't go further. Do
| you know what the reasons were that they couldn't make
| the entire thing clockless?
| moffkalast wrote:
| From what I understand while these two do contribute a lot
| to power usage, they don't really contribute that much to
| heating by themselves? Leakage should happen all the same
| in a processor that's completely idle and those typically
| don't heat up much. For higher clock speeds specifically I
| still don't see how lower resistance isn't key.
| timerol wrote:
| > Leakage should happen all the same
|
| Modern processors are very careful about this, and
| actively turn off the supply voltage to large parts of
| the die to prevent extra leakage current. The funny-but-
| appropriate name for this is "dark silicon"
| https://en.wikipedia.org/wiki/Dark_silicon
| moffkalast wrote:
| > at 8 nm technology nodes, the amount of dark silicon
| may reach up to 50-80%
|
| Damn TIL, I never would've expected that. But I guess it
| makes sense to use a few of the older, larger transistors
| that don't leak as much to power off a section of the
| smaller leaky ones while they're not performing any
| operations.
| thfuran wrote:
| >while these two do contribute a lot to power usage, they
| don't really contribute that much to heating
|
| Those are the same thing. Or at least close enough as
| makes no practical difference. Only an extremely tiny
| fraction of the power used but a CPU is becoming anything
| other than heat.
| moffkalast wrote:
| Then we actually agree? You don't get heating without
| resistance, ergo resistance is the main problem. MRIs
| don't have any problems sending a thousand amps through
| their coils.
| kayson wrote:
| You're right that the resistance is where the heat is
| dissipated, but lowering the resistance does not actually
| change the amount of heat. Transistor switching can be
| modeled as a step input to an RC circuit [1]. If you
| integrate the power through the resistor to infinity,
| you'll see that the value of the resistor drops out.
|
| Intuitively, you might think of it like this: to charge a
| capacitor (or transistor) up to a certain voltage, you
| need a fixed number of electrons. That number of
| electrons will always pass through the resistor and
| generate heat based on their energy. Even if you change
| the resistor value, its still the same number of
| electrons, and the same amount of energy.
|
| What does change with resistance, though, is the time
| over which the power is dissipated. In practice, you have
| to make sure the resistors are small enough such that you
| can achieve your desired clock speed.
|
| There are actual resistive losses too, but they're mainly
| related to power delivery.
|
| [1] https://en.wikipedia.org/wiki/RC_circuit#Time-
| domain_conside...
| LordDragonfang wrote:
| When talking about resistance and materials, it's also
| important to note that silicon has relatively low optimal
| operating temperatures compared to some of the other
| semiconductors available. This limits the amount of voltage
| you can pump into it (because the resistance mean higher V
| leads to heat), and voltage correlates with clock frequency.
| GaN has already seen success in chargers, and silicon carbide
| is another promising material. We can't achieve the low level
| of defects needed for small process nodes yet, though.
|
| Disclaimer: I'm not a material scientist, so this is probably
| only partly correct.
| Kon-Peki wrote:
| > We've known about this for a long time. Everyone expected it
| to happen.
|
| Absolutely. The 2006 "A View from Berkeley" is _still_ a great
| paper [1]. And we still have a long ways to go on this
| recommendation:
|
| > To maximize application efficiency, programming models should
| support a wide range of data types and successful models of
| parallelism
|
| We are still stuck in the winner-take-all mindset when it comes
| to software development.
|
| [1]
| https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-...
| Reason077 wrote:
| It helps that those services are a great recurring revenue
| stream, too.
| cesaref wrote:
| I'm not convinced this is the right criteria to compare these
| processors (core count and purchase cost).
|
| When looking at these high core count processors, the typical use
| case is for a server in a data centre, and these sorts of
| applications run 24/7 and the cost of power is a massive part of
| the TCO. I think you have to address power per gflop when
| evaluating performance for these parts, as this is the criteria
| they were designed against.
|
| I think the processors are costed in consideration of the TCO of
| a 2U dual socket machine with a 2-3 year expected lifespan. They
| will be designed and costed to show year on year improvements.
|
| Oh, and i'm not sure inflation was included as it will be
| relevant over the timescales involved.
| nologic01 wrote:
| In fact the CPU stagnation is worse. In contrast to the shrinking
| chip size and higher frequencies of past scaling laws, CPU core
| count does not easily translate to performance gains. It needs
| specialized and careful designed software to tap the capability.
|
| In fact if CPU core count _did_ translate more easily to
| performance gains I think already with the existing CPU 's we'd
| have a fairly signficant one-time boost.
|
| Maybe somebody has statistical survey of how much of the existing
| deployed CPU core count is typically used?
| nabakin wrote:
| I was looking for this comment and I think you're exactly
| right.
|
| Over the years, Moore's Law became a household term for
| computer performance doubling every couple of years. Under that
| definition, Moore's Law died in 2005 with Dennard Scaling so
| for most intents and purposes, Moore's Law has been dead for a
| long time.
|
| It only held under the more restrictive definition of
| performance for tasks that were able to be parallelized
| perfectly, but even that has now been broken.
|
| You could also argue that Moore's Law died in 2005 because the
| term CPU used to refer to what we now know as a CPU 'core' and
| the term was redefined.
|
| Ultimately, what matters is that the performance the end user
| experiences hasn't been doubling every 2 years since 2005.
| JonChesterfield wrote:
| Or it needs multiple processes running at the same time.
| Notably running lots of C++ compilers simultaneously works
| great and really likes the high core count machines.
| rbanffy wrote:
| > It needs specialized and careful designed software to tap the
| capability.
|
| I've been suggesting engineers get more cores of lower speed to
| gain insight on what will be performant a few years down the
| road since I saw my first Xeon Phi.
|
| It's been a while since clock speeds got higher (IBM has been
| pushing 5GHz in their highest end for the past couple years now
| and it doesn't seem likely they'll cross 6 anytime soon), but
| we get more cores every year. We now have 4-core entry-level
| machines and 2-core/4-thread ones are the bottom of the barrel,
| with a decent one being 8-core. Ampere just announced a
| 192-core server beast.
|
| And then we have another thing: performance for most users has
| been "good enough" for the past couple decades. I haven't
| gotten a new computer just because it had a faster CPU since
| the early 2000's - they usually turn to dust well before they
| become too slow to use. My wife will need to upgrade her
| Macbook soon-ish for regulatory reasons (when Apple EOLs and
| stops patching macOS 12) and her laptop is still going strong.
| Considering that, there is little advantage in making all but
| the most demanding software more parallel.
|
| This leaves the high-end, the stuff that needs a POWER10 or a
| Telum to run at acceptable speeds, and the cloud vendors, who'd
| kill to be able to serve 1% more VMs per kilowatt because 1% of
| their revenue is the GDP of a small country.
| robocat wrote:
| > and the cloud vendors, who'd kill to be able to serve 1%
| more VMs per kilowatt
|
| If Google's _only_ expense were electricity, and profit
| margin was 50%, then saving 1% on power bill would increase
| profits by 1%.
|
| I skimmed the Alphabet annual report: saw $260 revenue, $80
| profit, $110 operating expenses (electricity and staff). Say
| $10 on power, then 1% is $0.1 - improve profits a bit over
| 0.1%.
|
| Anyone know how many $/year Google spends on power?
| phkahler wrote:
| >> Maybe somebody has statistical survey of how much of the
| existing deployed CPU core count is typically used?
|
| My guess is very few cores are used on average. I did some
| testing with Solvespace to see which build options contributed
| most to performance:
|
| https://github.com/solvespace/solvespace/issues/972
|
| Obviously using OpenMP for multi-core was the big win. But
| what's not shown is that in typical usage (not the test I ran)
| if you're dragging some geometry around it will use all cores
| (in my case 4 cores / 8 threads) at about 50 percent
| utilization. That percentage probably drops as more cores are
| thrown at it due to Amdahl's Law. In other words, throwing
| double the cores at it will give a good boost to a lot of code
| that is already taking less than half the time (wall clock
| time, not CPU time).
|
| We added OpenMP to a number of functions for significant
| performance gains. And in fact, any remining single-thread
| operation that gets the parallel treatment is likely to have a
| significant impact on overall performance since that is where
| most of the time is spent now. At this point we're more focused
| on features and bugs.
|
| Algorithmic improvements are possible and I'd like to do those
| in the future, but they are much harder to do than sprinkling
| some #pragmas around critical loops. That will improve the
| scalability though, where multithreading really did not.
| jandrewrogers wrote:
| CPU core count does translate to performance gains but the
| popular software architecture idioms most people use are
| incapable of taking advantage of large numbers of cores. The
| gap between the performance you typically see and what is
| possible with proper software architecture and performance
| engineering is orders of magnitude in scale. A _lot_ of
| performance and scalability is left on the table.
|
| We've known how to scale software on large silicon for a long
| time, but as an industry we mostly can't be bothered (or lack
| the skills) to do it.
| JamesBarney wrote:
| The simple reason is devs are more expensive than servers for
| most applications.
| pjmlp wrote:
| It will hardly change as long as scripting languages keep
| being used for full blown applications.
| inconceivable wrote:
| multi-core architectures have been optimized for web servers
| i.e. tons of clients hitting relatively simple backend
| processes. also, my day-to-day gui workstation (ancient xeon
| mac) gets far more core usage than my dev server (threadripper)
| which is honestly kind of a waste of money, but it's shiny so i
| bought it anyway. it should last me a decade+ just like my gui
| workstation has.
|
| now that single-core GPU/CPU/TPU/whatever performance is back
| on the front burner i think we'll see some horsepower and
| compiler improvements over the next few years. luckily the i/o
| problem has made great strides in the meantime so
| network/memory/storage will be there to support it, unlike in
| the past. ecc ram is also plummeting in cost, so that's good.
| duped wrote:
| > CPU core count does not easily translate to performance
| gains.
|
| In theory, performance scales logistically with the number of
| parallel processors (Amdahl's law).
|
| In practice, the limit is (and has always been) memory and i/o.
| That's why Apple silicon kicks everyone's ass.
|
| If we want faster computers, the biggest gains are not to be
| found in making processors do more work. It's in designing
| systems (not just CPUs) that don't let the CPU wait around to
| do work.
| lostmsu wrote:
| > That's why Apple silicon kicks everyone's ass.
|
| ??? But it doesn't: https://browser.geekbench.com/processor-
| benchmarks https://browser.geekbench.com/mac-benchmarks
| eppsilon wrote:
| *kicks ass per-watt
| owenmarshall wrote:
| Doesn't Geekbench overwhelmingly measure raw CPU
| performance, whereas GP was talking about overall system
| performance? Isn't this chalk and cheese?
| ketralnis wrote:
| > In practice, the limit is (and has always been) memory and
| i/o
|
| Absolutely not. In practise the limit is (1) how many cores
| are actually _used_ by programs and (2) how much work is put
| into making anything fast at all, ever. We're using web
| frontends powered by python backends over a network. The vast
| majority of programs use nowhere near the resoures available
| to them.
| duped wrote:
| Those backends are spending most of their time waiting to
| do work, which is my point.
| ketralnis wrote:
| But even the amount of time they're doing "work" they're
| mostly managing refcounts and walking MRO chains and
| formatting strings and doing hashtable lookups to find
| local variables at runtime. Python code is 20-40x slower
| than C code. That other 19/20ths of the time is the
| bottleneck, not the memory bandwidth of the actual work.
| Same with your electron frontend.
| sangnoir wrote:
| > It needs specialized and careful designed software to tap the
| capability.
|
| I wouldn't exactly call Golang[1] "specialized", but it does
| make multiprocessing easier than most languages.
|
| 1.Or Erlang or Elixir
| nologic01 wrote:
| I only use go through hugo but its fast enough I never
| checked how many cores it uses :-)
| sremani wrote:
| This has to be coupled with 'The Great Software Decay'. I hope we
| will have 'real' innovation in Software Engineering where people
| take writing performant code seriously and many Software
| Engineers actually know what a compiler does and what the CPU
| does etc.
|
| The frameworks have been helpful but at the same time -- rounding
| buttons is not software engineering.
| qayxc wrote:
| The problem is that many algorithms simply cannot be
| parallelised well and that ever since the 90s the iron is cheap
| whereas the programmer's time is expensive.
|
| So instead of spending weeks or even months on trying to
| squeeze the last bit of performance out of an application
| that's "good enough" performance-wise, developers can use that
| time to roll out features or fix bugs instead, i.e. generating
| value for their customers.
|
| It's simply a question of economics.
| mensetmanusman wrote:
| We are off the exponential curve until a completely new method is
| discovered, which of course might not exist.
|
| Maybe our great^n grandchildren will then have lives very similar
| to the great^n+1 grandchildren thereafter, just like the older
| days!
| captainbland wrote:
| I'd say the desktop CPU situation is doing pretty alright. My
| 10400F is feeling _very_ modest compared to pretty much
| everything in the 13xxx and ryzen 7000 lineup. Frustrating given
| how deadend the chipset I got was.
| IMSAI8080 wrote:
| The answer to this might be more application specific
| accelerators and 3D stacking. You can't afford to have all of a
| chip switched on at once because of the power consumption and
| dissipation, so you build more optimised accelerators and keep
| flipping between them as you encounter different pieces of code
| that may benefit from each accelerator. Only a fraction of the
| device is ever in use at once. You 3D stack the chips to get more
| transistors in the same space. NAND flash chips are already 3D
| stacked, often with 100 layers or more.
| senkora wrote:
| The industry term for the idea that most of your chip is
| switched off at any given time is "dark silicon".
| kzrdude wrote:
| What about silicon that's perpetually switched off because of
| defects and binning (Like apple's 7 core graphics offerings
| and so on), that's an even darker silicon?
| bee_rider wrote:
| It would be cool if chiplets got to the point where, at least,
| an OEM... someone Dell sized... could actually differentiate
| themselves by mix-and-matching a group of accelerators.
| Bringing back consumer-visible differentiation in CPUs (other
| than Apple against the world) would be nice for the market I
| think.
| trynumber9 wrote:
| I'm still impressed by 64 core Genoa. It is 1.7x the total
| performance (cores * clock rate * IPC) compared to 64 core Milan
| in the table presented. That's not even two years apart.
|
| I know the die sizes didn't decrease much, around 10% or so. And
| R&D increase is surely surpassing inflation.
| AbsoluteCabbage wrote:
| At least now people will start to focus on how bloated and slow
| software has become in the meantime. And stop referring to it as
| "tech".
| freedomben wrote:
| I (unfortunately) doubt it. I think the reality with our
| current situation is that only hobbyists and open source people
| care enough about that to do anything about it. Everyone else
| is doing Scrum so hard that they can't think of much outside
| the current sprint, and any "pre-mature optimization" is evil
| and must be avoided. The result will be more of the same bloat.
| Competition won't help because everybody is using the same
| bloated foundations and nobody will invest more than a few days
| in the foundation because it's not "product work."
|
| The best we can hope for I think is that open source will
| create frameworks/foundations on top of which people can then
| try to build. Elixir Phoenix has been that to some extent,
| basically taking the rails philosophy but making it super light
| and fast (my Phoenix APIs run with 40MiB of memory and response
| times ~1ms). Maybe those sorts of advancements can save us, but
| I can't think of a way to address the browser that way and
| realistically right now the browser is a huge area of the
| bloat. A ton of code that runs in the browser is terribly
| optimized, but even the base is quite big.
| f6v wrote:
| > Everyone else is doing Scrum so hard that they can't think
| of much outside the current sprint
|
| This is a gross oversimplification. While there're
| inefficiencies and process abuses, this doesn't mean nobody
| cares about speed and resources. It might look that way to
| purists who're focused only on tech part of the businesses.
| jvanderbot wrote:
| That's exponential era practices. But we're hitting the top
| of the S curve. Believing that exponential era practices will
| continue indefinitely into the coming linear/level era is as
| naive as believing exponential growth continues forever.
|
| The real question is what happens first: change in software
| to adapt to slow growing compute or change in architecture to
| revitalize Moore's law.
| GrumpyYoungMan wrote:
| Oh, I don't think it's quite that grim. When development
| teams can no longer easily spend their way out of their
| inefficient code, being able to write high performance code
| becomes a competitive advantage. I'd even go so far as to say
| that individual developers looking to get ahead in their
| careers should pay less attention to flashy tech fads and
| more at foundational stuff like how to diagnose performance
| problems and write efficient code.
| ActorNightly wrote:
| Its not like biological creatures are highly optimized.
| pixl97 wrote:
| Eh, for the amount of work your brain accomplishes, it is
| insanely hyper optimized at around 20 watts. We don't have
| human level AI/image processing quite yet, but it would take
| hundreds of thousands of watts to accomplish the same thing
| at this time.
| eppp wrote:
| They have numerous problems. They are wrong a lot and die
| constantly as prime examples.
| mhh__ wrote:
| Processors used to spend most of their time sorting, after a
| period of wasteful bullshit they may return to hotspot's
| dominating again in the form of AI inference.
| moffkalast wrote:
| Won't we have dedicated analog/optical/whatever inference
| accelerators by that point as well?
| joshuakogut wrote:
| Compute time is cheaper than man hours.
| hindsightbias wrote:
| Containers killed the performance star.
| hamandcheese wrote:
| I don't see how this could be true. Containers are just
| processes with some extra permissions applied.
| durandal1 wrote:
| What seemingly has stagnated even more is our ability to write
| fast and responsive software. We have oceans of compute power,
| blazingly fast I/O, and yet, it's expected and accepted for
| mundane tasks to take hundreds of milliseconds to complete.
| christkv wrote:
| What's the power use vs performance look like over the same
| period?
| sspiff wrote:
| This analysis is wrong because it is focusing on a limited number
| of high end chips.
|
| You can buy CPUs that cost a fraction of any of those listed that
| will absolutely demolish even the best chips from 6 years ago
| today. All while consuming drastically less power for that
| performance as well.
|
| Even mobile chips over 2 years old are within margin of error
| performance distance to those entry level Naples chips listed, at
| a fraction of the cost and power consumption.
|
| I would not call that stagnation.
| anonymousDan wrote:
| Thanks, I was hoping someone would comment as to why the
| article might be wrong. I also wonder what the trends look like
| in terms of performance per watt?
| williamkuszmaul wrote:
| Related recent paper in Science:
| https://www.science.org/doi/10.1126/science.aam9744
| nickdothutton wrote:
| I found the single core SPECInt figures for the last couple of
| decades very interesting. The curve flattens. Apologies for link
| to the bird site, but I'm mobile and it's the fastest way I can
| find it, I think some of your will enjoy the diagram.
|
| https://twitter.com/nickdothutton/status/1194978743250538496...
| mhh__ wrote:
| IPC data seems a bit sus
| formerly_proven wrote:
| It's the average IPC improvement from the AMD marketing slides
| - Zen 2 over Zen, 15%, Zen 3 over Zen 2, 19% and Zen 4 over Zen
| 3 at 13%, so Zen 3 over Zen is 37% and Zen 4 over Zen is 55%
| (maybe a typo in the article)
| jeffbee wrote:
| Zen 1 was terrible and is not a useful baseline. It was
| obsolete at launch. Nobody bought them except companies who
| were paid to take them and agree to be in a press release. It
| was literally half the speed of Skylake on server workloads.
| ahartmetz wrote:
| Zen 1 was slightly slower per core, but had twice the
| amount of cores that Intel was willing to sell for desktop
| platforms! Kind of a big deal for use cases such as
| compiling C++ code...
| phkahler wrote:
| >> Zen 1 was terrible and is not a useful baseline.
|
| First off, it's a perfect baseline when comparing AMD chips
| since that time. Zen 1 was similar to Intel performance-
| wise, winning some benchmarks and losing some.
|
| Second, the Raven Ridge (Zen 1+) APUs were IMHO excellent
| performance for the price at the time - even against Intel.
| I have not felt the need to build a new system since the
| Mellori_ITX:
|
| https://github.com/phkahler/mellori_ITX
| formerly_proven wrote:
| > Regrettably, when considering cost per core, this impressive
| trend appears to have stalled, ushering in an era of CPU
| stagnation.
|
| In this case, don't let the author see $/GFLOPS for the last 10
| years of GPUs!
| bhouston wrote:
| I think you should factor in inflation. Inflation has been
| significant recently.
|
| Also you are not comparing to Intel's cost per core which would
| show that this pricing issue is not new. I think you just didn't
| notice it before.
| IshKebab wrote:
| Inflation has only been significant in the last 2 years max.
| Reason077 wrote:
| The Great _x86_ CPU Stagnation? ARM seems to have made some great
| leaps forward in recent years.
| rbanffy wrote:
| I don't have a feeling ARM offerings are significantly cheaper
| than x86. There really isn't much magic, a transistor costs the
| same to manufacture under a given process, regardless of the
| ISA.
|
| In that, RISC-V might get some edge, because ARM licensing is
| expensive, but I don't think licensing is a significant cost
| for the x86 crowd.
|
| OTOH, the server ARM people are really pushing it:
| https://www.semianalysis.com/p/sound-the-siyrn-ampereone-192...
|
| But Altra's ain't cheap.
| paulddraper wrote:
| ARM has seen increased adoption, but the chips themselves
| aren't much different than 4-5 years ago.
| freedomben wrote:
| not really, there have been some _systems_ built on top of ARM
| chips that have been impressive, but the chip hardware tech
| hasn 't changed much differently than their x86 counterparts
| neogodless wrote:
| Top AMD consumer x86 CPU over 4.5 generations:
|
| https://www.cpubenchmark.net/compare/2966vs3238vs3598vs3862v...
|
| I think the article is just about server CPUs getting better
| but the price is keeping them from being significantly better
| for the price. Of course, in these markets, any increase in
| power is _justified_ such that the buyers are not price
| sensitive.
| wmf wrote:
| Coincidentally Ampere One was announced today and it looks
| pretty mediocre.
| Rafuino wrote:
| What looks mediocre about it? They're making BIG claims about
| density and power usage at scale
| wmf wrote:
| If they doubled performance and doubled power that's not
| good.
| f6v wrote:
| I'm ignorant about this. Can a Qualcomm or Samsung chip match
| Intel's raw power?
| mhh__ wrote:
| The leading Arm chip (by apple) is arguably the best in
| class.
| f6v wrote:
| I'm well aware of Apple's success since I'm surrounded by
| their devices at home. It seems to me it can't be
| attributed only to ARM. What I wonder is where're someone
| can match Intel on Intel's playing field, like supplying
| ARM chips to laptop manufacturers that are better than
| Intel.
| EscapeFromNY wrote:
| Apple's chips are very power efficent, yes, but the article
| and parent are talking/asking about raw power. Despite all
| the Intel hate the past 5 years, I don't think there's been
| even a moment where another company's leading chip has
| outperformed Intel's in benchmarks.
|
| See for example https://www.cpubenchmark.net/compare/4922vs
| 5022vs5008vs5189/...
| Reason077 wrote:
| The M2 Pro isn't doing too bad when you consider you're
| comparing a laptop chip with 14+ hour battery life
| against Intel's latest and greatest, high-TDP _desktop_
| CPU with twice as many cores. On a performance-per-core
| basis it 's not even far behind.
| e44858 wrote:
| If the TDP is an accurate measure, then the M1 appears to
| be about as efficient as 12th gen Intel:
| https://www.cpubenchmark.net/power_performance.html#all-
| cpu
| Reason077 wrote:
| I wonder if there's some shenanigans going on there. Max
| TDP in an M1 is surely far lower than the _max_ TDP of an
| i7-1255U, which it outperforms. Most M1 systems don 't
| even have fans but can perform at max performance for
| extended times. U-series i7s can also be run fan-less,
| but performance will be compromised?
| rocket_surgeron wrote:
| >125W
|
| And that is a lie, to achieve those scores it turbos and
| consumes 300W+ at turbo.
|
| Today, right now, is a moment where AMD's enterprise
| product are outperforming Intel's in benchmarks.
|
| Except for a very small set of specific use cases, I
| think anyone recommending Xeon for enterprise solutions
| is professionally negligent.
| neogodless wrote:
| Umm, I added a readily available consumer CPU from a
| different company to your comparison chart.
|
| https://www.cpubenchmark.net/compare/5022vs5008vs5189vs50
| 31/...
| qayxc wrote:
| Much of Apple's success in the area seems to stem from the
| fact that they're simply buying other parties out of the
| latest manufacturing processes, e.g.
| https://appleinsider.com/articles/23/05/15/apple-has-a-
| stran...
|
| So if others want to compete, they'll always be a few years
| behind, since the fab capacity is reserved to Apple. So any
| competitor either has to magically improve the architecture
| dramatically - which they can't, since that would require
| an architectural license which Apple has, but most others
| don't - or find a fab that can compete with TSMC's latest
| tech both in terms of price and available volume.
| [deleted]
| hospitalJail wrote:
| Up until this year, CPU was never my limiting factor. We always
| used multithreaded or GPUs.
|
| This year has seen a need for a decent CPU for occasions where we
| must single thread. Both Local AI and python development.
|
| For so long, we never maxed out our CPU.
| lostmsu wrote:
| Just don't get stuck in Python.
| IAmPaigeAT wrote:
| Ugh, you know it really wouldn't be so bad if they would just go
| back to selling cpus like they're meant to be kept around for
| more than a year. The thing that has me really ticked off at
| Intel right now is the i915 sriov capabilities introduced in 11th
| gen which were meant to supersede GVT-G. I didn't even know GVT-G
| was a thing until it was already gone. They didn't even make a
| driver that supports 11th gen and 12th is apparently supported in
| some capacity but I'm not understanding it very well myself. I
| damn sure wouldn't buy a 12th gen though if its a thing you think
| you might actually care about. If 11th gen is any example of how
| 12th will turn out, any development will be left up to one person
| who miraculously understands i915 well enough to develop for it,
| doing for free and doing a good job but not well enough to keep
| up with linus/linux~HEAD; needless to say if 11th gen support for
| i915 sriov does ever get merged the cheap low quality
| construction of the hardware will probably be starting to break
| down, seems like most of it is by the time you unbox it anyway
| these days :(
| IAmPaigeAT wrote:
| There's actually 2, theres the intel lts one, which i guess
| they did do something it just never amounted to anything,
| explained here:
|
| https://github.com/intel/linux-intel-lts/issues/33
|
| The ongoing development:
|
| https://github.com/strongtz/i915-sriov-dkms
| cl0ckt0wer wrote:
| The cores/money comparison doesn't looks like it accounts for
| inflation.
| huijzer wrote:
| Indeed. I was also wondering why people would buy newer chips
| if they don't show performance increases per dollar.
| bee_rider wrote:
| I think the main reason people get a new CPU, regardless of
| the performance, is because the device that it came in has
| gotten too old (wear-and tear: screen, batteries, etc).
| rbanffy wrote:
| I'm pretty sure that when you adjust for inflation, you get
| more performance per dollar, but I'd need to do math and I
| don't have the numbers from my head.
|
| Does Google Sheets provide a "inflation-adjusted dollar"
| function?
| capitalsigma wrote:
| There are overheads in server workloads that scale with the
| number of machines (network traffic,
| serializing/deserializing requests). There are also fixed
| costs per server that don't scale with core count, or at
| least scale sublinearly (storage, physical data center space,
| motherboard, ease of maintenance). So running 10 machines
| with 100 cores can be cheaper and more performant than
| running 1,000 machines with 1 core even if $/core is higher.
| And of course individual cores can be beefier: wider SIMD
| units, application-specific extensions like bfloat support
| for ML workloads, etc.
|
| Of course Moore's law is slowing down, but cores/$ is an
| extremely silly metric to use
| pixl97 wrote:
| Because the vendor will no longer give the the old chips?
| That an lack of support around security.
| meindnoch wrote:
| We might need to stop writing JavaScript? Please no :'(
| crabbone wrote:
| There were multiple mentions in this thread contrasting GPU with
| CPU... I think, there's some conceptual gap here. GPUs are made
| of the same thing as CPUs, they have all the same problems CPUs
| have. They emphasize vectorized operations, and some less
| important (for this thread) stuff, like video encoding /
| decoding, but, by and large, the "PU" part is not accidentally
| the same in both acronyms.
|
| The difference comes from usage. CPUs are shared by processes and
| threads that are designed to be unaware of each other, or to be
| even hostile. At the same time, a lot of programs are built in
| such a way that they don't exploit the parallelism available to
| them through CPU, or, even if they do, they do it in a very
| clumsy way (through a bunch of wrappers with their own
| limitations).
|
| To contrast this, GPU programs typically use the whole GPU at
| once, and are written with parallelism in mind, with little to no
| wrappers.
|
| Similarly, because the basic unit of CPU usage is a process, and
| the model of using CPUs is that processes aren't allowed to know
| about each other by default, the memory use becomes more
| involved, inter-process communication becomes more involved,
| permissions, access to network etc. -- all this complicates and
| slows down programs which want to use CPUs.
|
| But, if, somehow, there was an OS that could use GPU to run
| processes on it, use VRAM for code / data of those processes etc.
| -- we'd have the same problems.
| tedsanders wrote:
| Indeed. GPUs are not much better. For example, the NVIDIA H100
| SXM has roughly the same price per transistor as the A100 SXM.
| The gains between generations came from (1) better chip design,
| (2) higher clock speed / power consumption, and (3) more
| transistors. Cost per transistor did not meaningfully improve.
| cubefox wrote:
| GPUs have a much higher typical utilization than CPUs, I
| think that's what the previous poster wanted to say.
| rektide wrote:
| Zen 4C is somewhat delayed but should be an interesting shift on
| these great numbers. It's a significsntly smaller core, first for
| servers, then latter big/little consumer parts.
|
| Maybe the IPC or GHz really will be significantly lower, but I
| tend to think things like cache size will be the biggest hit, and
| that cache size change wouldn't show up in these graphs.
| Essentially, same number of transistors, but more compute less
| cache is my guess. But perhaps the cores really are smaller &
| narrower & the IPC * GHz rating doesnt budge much!
___________________________________________________________________
(page generated 2023-05-18 23:01 UTC)