[HN Gopher] Analyzing Modern Nvidia GPU Cores
___________________________________________________________________
Analyzing Modern Nvidia GPU Cores
Author : mfiguiere
Score : 169 points
Date : 2025-05-05 23:38 UTC (23 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| winwang wrote:
| I hope this can help shed the misconception that GPUs are only
| good at linear algebra and FP arithmetic, which I've been hearing
| a whole lot!
|
| Edit: learned a bunch, but the "uniform" registers and 64-bit
| (memory) performance are some easy standouts.
| remcob wrote:
| It's well known GPUs are good at cryptography. Starting with
| hash functions (e.g. crypto mining) but also zero knowledge
| proofs and multi party computation.
| saagarjha wrote:
| They're not particularly good at cryptography, but they are
| good at highly parallel tasks like trying a bunch of hashes.
| qwertox wrote:
| Wasn't it well known that CUDA cores are programmable cores?
| winwang wrote:
| Haha, if you're the type to toss out the phrase "well known",
| then yes!
| YetAnotherNick wrote:
| In a sense, GPUs are only great at matrix-matrix
| multiplication. For anything else you would only get 7% of the
| FLOPs/s compared to it(989 vs 67 TFLOP/s for H100)[1].
|
| [1]: https://www.nvidia.com/en-in/data-center/h100/
| winwang wrote:
| lol, I haven't thought about it like that, true. though of
| course, I mean compared to CPUs :P
|
| I try and use tensor cores for non-obvious things every now
| and then. The most promising so far seems to be for linear
| arithmetic in Datalog, but that's just matrix-vector/gemv
| harperlee wrote:
| Could you expand the Datalog example? I'm quite interested
| cma wrote:
| That link says "* With sparsity". For extremely sparse
| matrixes you can get more than 989 TFLOPS on CPU, if we're
| counting elided operations in TFLOPS.
| YetAnotherNick wrote:
| I am counting FP16/BF16 without sparsity, which is used in
| majority of AI.
| cma wrote:
| That change checks out then. They didn't see much need
| for FP16 outside of that so no longer run it at double
| FP32 rate outside of tensor cores (unless I'm mixing that
| up with AMD).
|
| Other forms of sparsity are heavily used at training time
| now, like block compression in Deepseek.
| randomgermanguy wrote:
| Funnily, they're far from being optimal for GEMM ops
| (especially in terms of power consumption).
|
| For GEMM you need to visit each row/vec n-times so theres a
| bunch of data-reuse going on, which isn't optimal for GPUs
| since you can't keep that all so close to your processing-
| units. And while the tensor-cores kinda implement this i
| think they don't quite scale up to a full sized systolic
| array, which is you would want for larger matrix
| multiplications.
|
| Also just a simpler view: with GPUs most of their silicon is
| spent NOT tensor-core, so just from that you know its not
| optimal i guess.
|
| Just referring to that FLOP/s number doesn't really mean much
| nowadays with tensor-cores and sparsity.
|
| In my eyes the big win of GPUs are that not only are they
| pretty good at GEMMs but also really good at a lot of other
| easily parallelizable tasks PLUS they're comparatively easy
| to program ^^
| dist-epoch wrote:
| From your comment I've learned that you never did GPU graphical
| programming :)
|
| "uniform registers" exist for about 20 years now.
| rnrn wrote:
| I think you are confusing uniform registers with the uniform
| keyword in RSL / GLSL / HLSL?
|
| maybe some vendors have had an equivalent to uniform
| registers for 20 years, but per the articles' references they
| are new in nvidia GPUs in turing (2018)
| dist-epoch wrote:
| They are the same thing. The uniform keyword in shading
| languages is implemented using the uniform registers.
|
| I don't know what Nvidia did in 2018, maybe they opened up
| access to the uniform registers to CUDA code.
|
| I made Grok research this topic:
|
| > In conclusion, research strongly suggests that the
| "uniform" keyword in GLSL is implemented in hardware using
| NVIDIA's "uniform registers," as evidenced by NVIDIA's own
| documentation on the Turing architecture and historical
| practices of mapping uniforms to constant registers. While
| explicit links can be limited due to proprietary details,
| the combination of technical presentations, community
| discussions, and historical context supports this
| connection. The uniform register file, with its capacity
| and usage in shader instructions, aligns with GLSL's
| uniform functionality, ensuring efficient data access
| during shader execution.
|
| https://grok.com/share/c2hhcmQtMg%3D%3D_358362f3-21e2-4fe0-
| a...
| Firadeoclus wrote:
| While you can use uniform registers to implement the
| uniform keyword from shading languages, the two are not
| the same. Uniform registers are not constants, and they
| are only uniform/shared across one warp. Nvidia
| architectures before Turing did not have uniform
| registers.
| pjc50 wrote:
| I didn't get that at all - to me this looks like a very smart
| investigation into instruction latency and the precise
| mechanics of out-of-order execution (no reference made to
| speculative or branch prediction, though?) without looking at
| what the instructions do in detail.
|
| GPUs can certainly do bulk integer arithmetic but most use
| cases prefer FP. Maybe for DSP fixed-point is ideal.
| gmays wrote:
| The special sauce:
|
| > "GPUs leverage hardware-compiler techniques where the compiler
| guides hardware during execution."
| kookamamie wrote:
| > NVIDIA RTX A6000
|
| Unfortunately that's already behind the latest GPU by two
| generations. You'd have these after A6000: 6000 Ada, Pro 6000.
| flowerthoughts wrote:
| It's a major step forward compared to 2006.
|
| A6000 was released in 2020: https://www.techpowerup.com/gpu-
| specs/rtx-a6000.c3686
| KeplerBoy wrote:
| Nvidia's Quadro naming scheme really is bad these days, isn't
| it?
|
| I bet there are plenty of papers out there claiming to have
| used a RTX 6000 instead of a RTX 6000 Ada gen.
| kookamamie wrote:
| The naming scheme is horrible, to be quite frank.
|
| To understand this, consider these names in the order of
| release time: Quadro RTX 6000, RTX A6000, RTX 6000 Ada, RTX
| Pro 6000, RTX Pro 6000 Max-Q.
| pjmlp wrote:
| Still better than most folks have access to.
|
| I bet I can do more CUDA with my lame GeForce MX 150 from 2017,
| than what most people can reach for to do ROCm, and that is how
| NVidia keeps being ahead.
| kookamamie wrote:
| Yeah, kind of. I have an 6000 Ada and 5090 here.
| pjmlp wrote:
| On a laptop?
|
| Because that is part of my point, that is a laptop GPU.
| kookamamie wrote:
| Oh no, there are high-end desktops. You're right, laptops
| completely different profiles for these things.
| gitroom wrote:
| Haha honestly I always thought GPUs were mostly number crunchers,
| but there's way more under the hood than I realized. Wondering
| now if anyone really gets the full potential of these cores, or
| if we're all just scratching the surface most days?
| Dlemo wrote:
| There are very good performance tools for GPUs.
|
| I don't think GPU utilization is a real bottleneck in most
| cases.
| dist-epoch wrote:
| Yet DeepSeek managed to get huge improvement by optimizing
| GPU code.
| nabla9 wrote:
| >Overall, we can conclude that GPUs are hardware-compiler
| codesign where the compiler guides the hardware in handling
| dependencies and introduces hints that can improve performance
| and energy.
|
| New architectures rely on the compiler to handle register data
| dependencies, and controlling register file cache allocation
| policy.
| dist-epoch wrote:
| This is an age-old idea, RISC compilers were supposed to do
| this too, the mythical "sufficiently smart compiler"
|
| https://wiki.c2.com/?SufficientlySmartCompiler
| pjc50 wrote:
| > New architectures
|
| [citation needed] - which architectures?
| nabla9 wrote:
| Citation is the paper we are discussing. It also mentions the
| architectures.
|
| There is so much stuff you miss when you don't follow the
| links ;)
___________________________________________________________________
(page generated 2025-05-06 23:02 UTC)