[HN Gopher] Ask HN: Resources for GPU Compilers?
       ___________________________________________________________________
        
       Ask HN: Resources for GPU Compilers?
        
       Hi folks, I've done some cpu compilers for x86, RISC-V, LLVM
       (mostly for fun and some for profit). I'm eager to learn more about
       GPU/TPU compilers and have been looking into Triton and XLA. What
       resources would be useful to learn HPC and/or gpu compilers in
       depth (or any adjacent areas)?  Any such books, courses etc will be
       much appreciated.
        
       Author : zvikinoza
       Score  : 19 points
       Date   : 2024-09-02 19:24 UTC (3 hours ago)
        
       | danielt3 wrote:
       | https://theartofhpc.com/
       | 
       | Great book series on the subject of HPC. Not sure if it actually
       | touches GPU. Great material anyway. BONUS: it's free!
        
       | surfingdino wrote:
       | https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index...
        
       | hansvm wrote:
       | If you're already familiar with compilers in the abstract, start
       | by implementing some high-level solutions leveraging a GPU, some
       | low-level performance-optimized kernels, and build up a bit of
       | intuition. The high-level code depends on your goals, but for
       | low-level code maybe try optimizing something equivalent to a
       | binary tree (both with leaves smaller and larger than 4kb),
       | something benefiting from operator fusion (e.g., a matmul
       | followed by an element-wise exponential), and something
       | benefiting from deeply understanding the memory hierarchy (e.g.,
       | multiplying two very large square matrices, and also inner/outer
       | producting two very narrow matrices).
       | 
       | From there, hopefully you'll have the intuition to actually
       | evaluate whether a given resource being recommended here is any
       | good.
        
       | raphlinus wrote:
       | Faith Ekstrand has an impressive track record of compiler work
       | and has written a few blog posts[1], [1a]. Her Mastodon[2] is
       | also worth a follow.
       | 
       | SPIR-V is important in the compute shader space, especially
       | because DXIL and Metal's AIR are similar. I'm going to link three
       | articles critical of SPIR-V: [3], [4], [5].
       | 
       | WebGPU [6] is interesting for a number of reasons, largely
       | because they're trying to actually nail down the semantics, and
       | also make it safe (see the uniformity analysis [7] in particular,
       | which is a very "compiler" approach to a GPU-specific problem).
       | Both Tint and naga projects are open source, with lots of high
       | quality discussion in the issue trackers.
       | 
       | Shader languages suck, and we really need a good one. Promising
       | approaches are Circle [8] (which is C++ based, very advanced but
       | not open source), and Slang [9] (an evolution of HLSL). The Vcc
       | work (also related to [4]) is worth studying.
       | 
       | Best of luck! This is a fascinating, if frustrating, space, and
       | there's lots of room to improve things.
       | 
       | [1]: https://www.gfxstrand.net/faith/blog/
       | 
       | [1a]: https://www.collabora.com/news-and-
       | blog/blog/2024/04/25/re-c...
       | 
       | [2]: https://mastodon.gamedev.place/@gfxstrand
       | 
       | [3]: https://kvark.github.io/spirv/2021/05/01/spirv-horrors.html
       | 
       | [4]: https://xol.io/blah/the-trouble-with-spirv/
       | 
       | [5]: https://themaister.net/blog/2022/08/21/my-personal-hell-
       | of-t...
       | 
       | [6]: https://github.com/gpuweb/gpuweb
       | 
       | [7]: https://www.w3.org/TR/2022/WD-WGSL-20220505/#uniformity-
       | over...
       | 
       | [8]: https://www.circle-lang.org/site/index.html
       | 
       | [9]: https://github.com/shader-slang/slang
        
       ___________________________________________________________________
       (page generated 2024-09-02 23:01 UTC)