[HN Gopher] The end of Moore's Law forced YouTube to make its ow...
___________________________________________________________________
The end of Moore's Law forced YouTube to make its own video chip
Author : redbell
Score : 96 points
Date : 2022-08-27 13:12 UTC (9 hours ago)
(HTM) web link (www.techspot.com)
(TXT) w3m dump (www.techspot.com)
| pstoll wrote:
| We talk power, not servers, at work because it's the limiting
| resource at any scale. "How many MWatts do you need next year".
| Its not that you can't do these workloads on CPUs or GPUs - you
| can. But you can do it at 10x-30x less _power_ on custom silicon.
|
| That's why Amazon did Graviton3; Google did TPUs and VPUs. Ditto
| on personal computer with Apple & M1/M2.
| sthreeaway wrote:
| An important caveat is that this is practically and
| theoretically true for "embarrassingly parallel" (in the term-
| of-art, not perjorative sense) compute problems, and only if
| one sets aside supply-chain issues.
|
| It's theoretically (but rarely practically) true for network-
| bound problems of all sorts.
|
| It's not theoretically (and usually not practically) true for
| storage-bound problems. (Though Wh/bytes-at-rest is a useful
| bounding metric to understand if you're trying to build a
| storage business. It's never zero if the _integrity_ of a byte
| at rest is at all important. Integrity is complicated.)
|
| But in any case, supply chain issues tend to dominate as the
| limiting factor at the largest scale in my experience. This has
| always been true, even since before the current clusterfuck.
|
| Source: no comment.
| fezfight wrote:
| You skipped over the entire smart phone era.
| pstoll wrote:
| Sure - add the Apple A chips; Qualcomm chips for
| communications, etc...
| fomine3 wrote:
| It's not obvious from this article why they need their own
| original chip. Video transcoding is a quite common task done by
| dedicated chip (or circuit, like QSV).
|
| Perhaps because they push new codecs VP9 and AV1, these are also
| general format but Google is a developer and early adapter. Also
| perhaps their use case (batch processing, transmitted many times)
| isn't very suitable for existing transcoding chips that might be
| optimized for realtime transcoding.
| r3013 wrote:
| The actual papers are very detailed.
|
| https://ieeexplore.ieee.org/document/9567040
|
| https://dl.acm.org/doi/abs/10.1145/3445814.3446723
|
| They basically have computing per energy as an efficiency spec
| and then they show how that spec increased by 20-33x compared
| to an "Intel Skylake" machine.
|
| Google also added some features, like a custom speed vs quality
| tuning thing, and single and multi-output transcoding.
|
| What was interesting to me is the video core was made with
| "Catapult, a C++ HLS flow from Siemens".
| lerp wrote:
| There's also a recording of the conference presentation at
| https://www.youtube.com/watch?v=vdXCVuY2CJs.
| IshKebab wrote:
| That is interesting. I guess it makes the most sense if you
| already have a full C++ implementation, which they do.
| j45 wrote:
| Compute per watt is a great way to look at it.
|
| A very trivial example of this is the concept in homelab
| circles to compare a cpus geekbench score to its watts to its
| price. Cost per point.
| yaantc wrote:
| The article [1] is a bit old (2010) but still relevant to
| answer your question. It compares the efficiency of a HD.264
| implemented first on a general purpose CPU, then on a SIMD+VLIW
| DSP, then on the same DSP with additional optimized
| instruction, then on this platform with macro memory based
| accelerators, and lastly a pure ASIC implementation. There are
| significant efficiency gains at each steps, the big gap is
| between optimized ops and macro accelerators.
|
| Flexibility do have a price. With efficiency gains due to new
| nodes getting lower (and pricer!), it's normal to see more
| dedicated hardware accelerators.
|
| [1]
| https://courses.cs.washington.edu/courses/cse591n/10au/Paper...
| sp332 wrote:
| I think it's the batch thing. They can put in one input file
| and get out multiple formats and resolutions. Also, they have
| very specific requirements like only encoding 5 seconds at a
| time for DASH streaming, so they can build the chips with a
| specific amount of RAM and leave out features like B-frames or
| whatever they're not using.
| brigade wrote:
| Typically one chip has the hardware for a total encode
| throughput of like 4k60 pixels per second, optimized for the
| possibility of one real-time 4k60 encode. The chip also
| includes an entire GPU and/or a bunch of CPU cores. Putting
| more fixed encode HW instead of GPU or CPU cores is more useful
| and efficient for YouTube, and at the same time they don't need
| individual encodes to be real-time, so it becomes possible to
| design the hardware for higher quality if you don't need to
| guarantee 16ms frame encode times (maybe? at least, "realtime"
| is the reason I got when asked why hardware can't use recon for
| intra mode decision...)
|
| Intel went partly this way by slapping three 4-core Xeons on a
| PCIe card (VCA2), abandoned it, then announced variants of Xe
| to replace it. But ultimately they're repurposing their
| existing silicon and it's not as important for Intel as it is
| for Youtube.
| dmw_ng wrote:
| You can get similar functionality very cheaply on AWS from the
| vt1 instance type. They're a PITA to work with, which is probably
| why they're still around $160/mo. for enough transcode to power a
| small city on spot. These cards can transcode a full ladder (5
| resolutions) of a 60 minute 1080p video in around 4 minutes, or
| the full ladder for 8 60 minute videos in parallel in about 30
| minutes. Those instances are based on off-the-shelf cards that
| cost somewhere around $2000 each.
|
| Guessing Google went the custom route for wider codec support,
| but standard hardware for this stuff is readily available in the
| professional broadcast industry
| hot_gril wrote:
| Would be funny if Google did custom hardware entirely for
| VP8/9, since VT1 on the face of it doesn't seem to support
| that. But more likely, Google didn't want to rely on a
| competitor, and in general with datacenters that have a
| reputation for doing things their own more-optimized way even
| if it's a hassle.
| alexklarjr wrote:
| It must be a law of common sense that if all you do is encoding
| every stupid video on planet, you will need an encoding
| accelerator because gpu manufacturers likes to lock down and
| charge alot for tiny part of silicon that does it.
| wtallis wrote:
| > because gpu manufacturers likes to lock down and charge alot
| for tiny part of silicon that does it.
|
| Those shenanigans have a far smaller effect than the
| fundamental problem with buying a large GPU when you have no
| use for 80% of what's on that chip. That's obviously not going
| to be economical at large scale (especially not during a GPU
| shortage driven by demand for exactly the portions of a GPU
| that you never use).
| cosmiccatnap wrote:
| ...or they needed a chip that supported av1 and despite all the
| major tech companies being a part of the working group for years
| none of them had implemented it because Nvidia has a stranglehold
| on the industry because it makes money from h26x codecs
| deepsquirrelnet wrote:
| On an unrelated note, I find it funny that we call things like
| "Moore's law" laws at all. Everything in science is a theory now,
| and these strictly observational "laws" are borne on much shakier
| grounds.
|
| Meanwhile, everybody expects the law to break at some point
| (relatively soon, if not already). So what is it then? The
| observation formerly known as Moore's law? Seems a bit of a silly
| way to chronicle contemporary, short lived phenomena.
| Frost1x wrote:
| I used to think similarly about various "laws" but if you think
| about it for a moment, laws in what I suspect is their oldest
| usage refer to governance from a society laws. Those laws
| aren't immutable even though we pretend they are, they flex
| quite often. In some countries like the US they're designed to
| be inherently mutable. Laws can change, drop, be invalidated
| and so on. So it's not too inaccurate to call fairly solid
| theories that seem to behave in a stable manner as a law
| because laws are by no means absolute, either.
|
| It was clear not long after Moore's law that it likely wouldn't
| continue indefinitely unless we could work at the quantum scale
| somehow, there was no path for scaling beyond that. We could
| extend that to Planck scale if we wanted to reach absolute
| limits of understanding but you can only double density so many
| times unless we don't understand space and matter that well.
| deepsquirrelnet wrote:
| However, speaking of governance, what governs Moore's "law"?
|
| Unlike the "laws of nature" or "laws of society" there is no
| such clear expectation of a governance in this case. Moore's
| law is a rate observation. If it's governed by anything, it
| represents our global investment in technological
| advancement.
| Frost1x wrote:
| Well with laws in society we do know sort of axomatically
| where they come from because they're human defined and we
| can reference the base. So I can't argue there too much,
| although there's plenty of squishy area around the edges
| where law isn't clearly defined but cases are attempted to
| be interpreted which boils down to some set of conclusions
| based on some argumentative evidence.
|
| When it comes to laws of nature I would argue there's no
| clear governance in the laws of nature either. Why is the
| speed of light casualty? Why does space and time behave
| according to general relativity? Heck, I don't even know
| why my software behaves the way it does sometimes and it
| _is_ well defined but it 's so complex and behaviors become
| emergent that even that struggles to find a definite cause.
|
| Laws are often just observational trends, pretty much
| always in science. Science itself isn't definitive and is
| structured under that assumption of change. All laws are is
| a culmination of defining a pattern our best understanding
| of observations and prodding at the universe. What governs
| the law of conservation of mass (hint it was taught as a
| "law" when I was in school but clearly isn't after nuclear
| physics and GR discoveries). Even Newton's "laws" are
| flawed as we know now and were just very amazing pattern
| approximations that apply to most but not all cases--what
| governs these are really GR/SR, but we don't know what
| governs them. Take black holes at the event horizon and
| deeper--we really don't know what's going on, we're just
| taking our well defined and studied sets of known patterns
| and trying to imagine and extrapolate what goes on. I
| wouldn't fault Moore's law too much, it may lack some of
| the deep rigor we expect in smaller sgstems but it was a
| pretty good and useful observation for quite some time.
|
| We need to shed the idea that science will always give us
| the deepest underlying why, because philosophically it may
| not. It may get us very close and is a useful endeavor but
| we may never understand the causes of the most core
| relationships we discover. We like to think of laws like
| axioms in mathematics but they simply aren't in pretty much
| every case I can think of.
| barrysteve wrote:
| Got a better name for those waves of temporarily useful theory?
| mdrachuk wrote:
| Meme? Moore's meme.
| nyokodo wrote:
| > I find it funny that we call things like "Moore's law" laws
| at all
|
| Because "Moore's contemporary short lived phenomena" isn't as
| catchy? Words can be used in all sorts of non-literal ways.
| mr_mitm wrote:
| Moore conjecture? Moore approximation? Moore prediction? Moore
| relation?
| MaxBarraclough wrote:
| Perhaps the _Moore curve_?
| scoopertrooper wrote:
| > I find it funny that we call things like "Moore's law" laws
| at all.
|
| I think it had more to do with 'Moore' rhyming with the word
| 'law'.
|
| There hasn't been a radical shift in the use of the word 'law'
| over time.
| wonnage wrote:
| wouldn't Moore's Lore rhyme a bit better
| maxboone wrote:
| It's a law, not every law needs to be true / can be refuted or
| superseded.
|
| So it'd still be Moore 's law, just not true and relevant
| anymore.
| pdimitar wrote:
| Maybe we'll need co-processors in the future again?
|
| Although when you think about it, TPUs / VPUs etc. being
| pluggable to a PCIe slot is mostly getting us there?
| BooneJS wrote:
| We've already got them! They're integrated into the "System on
| Chip" or as chiplets on the package. Just considering Apple's
| M1: https://en.wikipedia.org/wiki/Apple_M1
|
| Because pure-software implementations of algorithms are too
| "expensive" in terms of power and/or latency, the M1 package
| has logic blocks that implements a GPU, a Neural Network
| processor, and H.264/H.265/VP9/JPEG/ProRes codecs. When Apple
| quotes battery life, the fine print says "1080P HD video with
| brightness set 8 clicks from bottom". Powering down most of the
| cores and lighting up just one of the codecs is clearly a good
| way to achieve 18 hours of battery life.
|
| AMD and Nvidia sell GPU "co-processors" that plays games and
| has been used for machine learning in "GPU mode" (think games
| using "software rendering"), but Nvidia GPUs now have "Tensor
| Cores" to accelerate and decrease the electrical power required
| for some of the linear algebra functions common with ML. Laptop
| chips from Intel and AMD will have embedded video on package to
| eliminate an external GPU.
| diydsp wrote:
| Moore's law is not over. It's not about density. It's about
| quantity of transistors on a chip.
| r3013 wrote:
| It depends how literally one wants to take the law.
|
| The "law" is doubling density every two years.
|
| So strictly speaking it is done because it has taken longer
| than two years. But if you choose the looser interpretation and
| simple read it as, transistor density increases with time, then
| it's not dead.
|
| Probably the bigger deal is cost per transistor is actually
| going up with the new nodes.
|
| https://www.fabricatedknowledge.com/p/the-rising-tide-of-sem...
|
| and
|
| https://www.eetimes.com/moores-law-dead-by-2022-expert-says/
| diydsp wrote:
| People often misinterpret this because they think Moore was
| only talking about technology. However the _number_ of
| transistors is more interesting than the technology alone
| because it represents _what people actually do_ with chips.
| e.g. GPUs added to CPUs, and later AI added to the M1. These
| additions may not have been much more dense technologically,
| but the needs and desires of users increase as well as the
| technology. The tech doesn 't have to improve 2x to reach 2x,
| because peoples' needs for AI, games, etc. is also
| increasing. And that's more useful in anticipating and
| predicting markets than the mere tech process.
|
| So if you still think you're right, kindly post your
| citations. Here are mine:
|
| "Moore's law is a term used to refer to the observation made
| by Gordon Moore in 1965 that the number of transistors in a
| dense integrated circuit (IC) doubles about every two years.
| "[0]
|
| "Moore's law is the observation that the number of
| transistors in a dense integrated circuit (IC) doubles about
| every two years"[1]
|
| "Moore's Law refers to Gordon Moore's perception that the
| number of transistors on a microchip doubles every two
| years..."[2]
|
| [0] https://www.synopsys.com/glossary/what-is-moores-law.html
|
| [1] https://en.wikipedia.org/wiki/Moore%27s_law
|
| [2] https://www.investopedia.com/terms/m/mooreslaw.asp
| diydsp wrote:
| Everyone downvoting, please post your disputatious
| citations.
| docfort wrote:
| Perhaps the bigger deal here is the loss of Dennard scaling
| in the 2000s. This is due to heat density at any given
| point on the chip. After all, increased transistor density
| could lead you to using all those transistors, but the
| thermal conductivity of all the materials involved isn't
| changing much, so there's a natural limit to the
| electrically active transistor density. You're welcome to
| increase transistor density, so long as you keep the number
| of active transistors roughly constant (not quite true
| since smaller transistors produce less heat).
|
| You could keep transistor density fixed, but still double
| the number of transistors in a chip ... by using bigger
| chips. And that is naturally more expensive because they
| are more likely to be defective and require more
| sophisticated packaging. 3D stacking of smaller chips is
| another approach, since we usually only talk about the 2D
| area of a chip. This requires more advanced packaging, but
| substantially increases the likelihood of building a
| functional device.
|
| The loss of Dennard scaling makes it more interesting to
| investigate ASICs because CPUs won't necessarily eat your
| lunch in two years. And circuit cleverness is one
| contributor to Moore's Law that Moore talked about in his
| 1975 address.
|
| So Moore's Law is in some ways dead (number of
| instantaneously useful transistors is NOT doubling every
| two years) and other ways alive (useful work is improving
| rapidly so long as you can decompose your workload into
| leaning on the accelerators like YouTube did), depending on
| how you measure it.
| _hypx wrote:
| Even for transistor density, it isn't quite true. The
| transistors aren't getting any smaller like they use to. A
| lot of growth now involves stacking transistors on top of
| each other. 3D transistors lets you pack more of them closer
| together. While this allows for further increases in
| transistor density, you can argue that it is no longer an
| exponential function anymore.
| wtallis wrote:
| > A lot of growth now involves stacking transistors on top
| of each other.
|
| Approximately _none_ of the growth thus far involves
| stacking transistors on top of each other, unless you count
| 3D NAND flash memory. FinFETs and upcoming GAAFETs are less
| planar than traditional transistor designs, but still don
| 't get you two or more separate transistors stacked on top
| of each other. Actual 3D stacking of transistors on a
| single wafer is still nothing more than a hot topic for
| R&D, and stacking multiple dies full of transistors is
| pretty much only done with memory so far.
| onlyrealcuzzo wrote:
| Koomey's Law is where it's at - performance per watt growth
| is slowing - but it's still doubling every ~2.6 years.
| yaantc wrote:
| Moore's 1965 paper was literally about "the complexity for
| minimum component costs" [1]. So you're right that transistors
| on a chip is more applicable then just die density. But as too
| often nowadays you left out the "minimum cost" aspect. Moore's
| law is not only about tech performance, it's about economics.
| Everybody knew the "minimum cost" part would fall the first,
| and it did.
|
| When the least cost offering improves, when things get better
| and cheaper, everyone follows. The lowering cost per function
| increases the addressable market, which can then sustain
| increasing investments. This is the virtuous cycle that moved
| the industry forward for a long time.
|
| When the economic part stops, when you can get more transistors
| but only at a higher cost, then the virtuous cycle stops.
| Technical progress do continue, but you will have less players
| at each new step: only those who can absorb the increasing
| costs will keep on playing at the leading edge. And production
| costs will still rise. If the market gets smaller due to
| increasing prices, and the production cost keep on raising, you
| will see a slow down as it will take more time to amortize
| investments.
|
| The big players are worried about this: slow down has in itself
| the specter of followers catching up (eventually...), leading
| to commoditization. We're definitely not there yet, but it's
| not a good perspective. Definitely bad for their stock. So the
| cost has been taken out. At first the NRE part was dropped of
| the equation, to consider only the cost per transistor. For
| this one can also be creative: considering the fabs
| amortization, or after? Makes a big difference (see the issue
| car makers have in getting new investments at "old" nodes:
| message from the big guys is move to smaller nodes, even if
| it's not always convenient). But in the end it's been simpler
| to just drop the least cost aspect completely. The industry PR
| had good results on this ;)
|
| For small to medium fabless players, Moore's law has been over
| for a while. For the biggest players it's also over now:
| there's still technical progress on transistors per device, but
| at an increasing cost.
|
| It's not always a smooth process: if you're an Apple, you can
| gain on the cost by going vertical (good-bye Intel margin). But
| it's a bold move not for all, and it's a one off gain.
|
| Moore's law is over. Tech progress is not. For HN readers who
| can afford the increasing cost of the devices and are reading
| regularly from the big silicon vendors that "Moore's law is
| still going strong yoo-hoo", when it's been neutered to
| transistors count only, it may be easy to miss.
|
| [1] https://newsroom.intel.com/wp-
| content/uploads/sites/11/2018/...
| IshKebab wrote:
| Moore's law is definitely over. Chips are still getting faster,
| but that's not the same thing.
| diydsp wrote:
| Are you going by vibes, rumors, Chinese whispers, or facts?
|
| > definitely
|
| Would you care to post some definite citations? Or would you
| like to dispute my citations listed above?
| CharlesW wrote:
| It doesn't seem dead quite yet.
|
| https://static1.makeuseofimages.com/wordpress/wp-
| content/upl...
| yaantc wrote:
| This is the number of transistors per device only. There is
| no notion of "minimum cost point" as in Moore's 1965 paper,
| where the "law" comes from. See my other post in this
| thread on this and why the economic part of Moore's law is
| as important as the tech part (density).
| CharlesW wrote:
| But that these chips don't cost a million dollars shows
| that the cost per transistor has plunged commensurately,
| no?
| yaantc wrote:
| It shows that the difference in efficiency of doing a
| custom ASIC at Youtube scale saves more money that what's
| required to design and manufacture their custom chip. But
| this is different from the trend cost per transistor
| (particularly for smaller players!).
|
| And actually, this is the case _because_ new processes
| efficiency gains are lower than in the past. It 's
| something that's been said before (and I read about it on
| HN, it was just years back and I don't have the reference
| at hand now ;), but when process improvements were fast
| making a custom ASIC didn't really make sense: by the
| time the ASIC was designed and debugged and deployed, a
| few years were spent and a recent CPU was nearly as good
| as the custom ASIC for no risk and less effort.
|
| Doing an ASIC is hard and takes time, and having a slower
| pace of improvement for new nodes efficiency makes it
| easier to justify the long deployment. There is now time
| to use such an ASIC with a noticeable gain, and absorb
| the cost and gain. Still, this is for a Youtube or Google
| (TPU) or Amazon (Graviton) or Apple. You need this scale
| still to absorb the development cost.
___________________________________________________________________
(page generated 2022-08-27 23:02 UTC)