https://airlied.blogspot.com/2023/07/tinygrad-rusticl-aco-why-not.html Skip to main content Search This Blog [ ] [Search] Dave Airlie Linux Graphics blog tinygrad + rusticl + aco: why not? * Get link * Facebook * Twitter * Pinterest * Email * Other Apps July 13, 2023 I recently came across tinygrad as a small powerful nn framework that had an OpenCL backend target and could run LLaMA model. I've been looking out for rusticl workloads, and this seemed like a good one, and I could jump on the AI train, and run an LLM in my house! I started it going on my Radeon 6700XT with the latest rusticl using radeonsi with the LLVM backend, and I could slowly interrogate a model with a question, and it would respond. I've no idea how performant it is vs ROCm yet which seems to be where tinygrad is more directed, but I may get to that next week. While I was there though I decided to give the Mesa ACO compiler backend a go, it's been tied into radeonsi recently, and I done some hacks before to get compute kernels to run. I reproduced said hacks on the modern code and gave it a run. tinygrad comes with a benchmark script called benchmark_train_efficientnet so I started playing with it to see what low hanging fruit I could find in an LLVM vs ACO shootout. The bench does 10 runs, the first is where lots of compilation happens, the last is well primed cache wise. There are the figures from the first and last runs with a release build of llvm and mesa. (and the ACO hacks). LLVM: 215.78 ms cpy, 12245.04 ms run, 120.33 ms build, 12019.45 ms realize, 105.26 ms CL, -0.12 loss, 421 tensors, 0.04 GB used, 0.94 GFLOPS 10.25 ms cpy, 221.02 ms run, 83.50 ms build, 36.25 ms realize, 101.27 ms CL, -0.01 loss, 421 tensors, 0.04 GB used, 52.11 GFLOPS ACO: 71.10 ms cpy, 3443.04 ms run, 112.58 ms build, 3214.13 ms realize, 116.34 ms CL, -0.04 loss, 421 tensors, 0.04 GB used, 3.35 GFLOPS 10.36 ms cpy, 234.90 ms run, 84.84 ms build, 36.51 ms realize, 113.54 ms CL, 0.05 loss, 421 tensors, 0.04 GB used, 49.03 GFLOPS So ACO is about 4 times faster to compile but produces binaries that are less optimised. The benchmark produces 148 shaders: LLVM: 126 Max Waves: 16 6 Max Waves: 10 5 Max Waves: 9 6 Max Waves: 8 5 Max Waves: 4 ACO: 96 Max Waves: 16 36 Max Waves: 12 2 Max Waves: 10 10 Max Waves: 8 4 Max Waves: 4 So ACO doesn't quite get the optimal shaders for a bunch of paths, even with some local hackery I've done to make it do better.[1] I'll investigate ROCm next week maybe, got a bit of a cold/flu, and large GPU stacks usually make me want to wipe the machine after I test them :-P [1] https://gitlab.freedesktop.org/airlied/mesa/-/commits/ radeonsi-rusticl-aco-wip * Get link * Facebook * Twitter * Pinterest * Email * Other Apps Comments Post a Comment Popular posts from this blog Fedora 38 LLVM vs Team Fortress 2 (TF2) April 23, 2023 F38 just released and seeing a bunch of people complain that TF2 dies on AMD or other platforms when lavapipe is installed. Who's at fault? I've no real idea. How to fix it? I've no real idea. What's happening? AMD OpenGL drivers use LLVM as the backend compiler. Fedora 38 updated to LLVM 16. LLVM 16 is built with c++17 by default. C++17 introduces new "operator new/delete" interfaces[1]. TF2 ships with it's own libtcmalloc_minimal.so implementation, tcmalloc expects to replace all the new/delete interfaces, but the version in TF2 must not support or had incorrect support for the new align interfaces. What happens is when TF2 probes OpenGL and LLVM is loaded, when DenseMap initializes, one "new" path fails to go into tcmalloc, but the "delete" path does, and this causes tcmalloc to explode with "src /tcmalloc.cc:278] Attempt to free invalid pointer" Fixing it? I'll talk to Valve and see if we can work out something, LLVM 16 Read more vulkan video decoding: radv status December 14, 2022 I've been working the past couple of weeks with an ffmpeg developer (Lynne) doing Vulkan video decode bringup on radv. The current status of this work is in a branch[1]. This work is all against the current EXT decode beta extensions in the spec. Khronos has released the final specs for these extensions. This work is rebased onto the final KHR form and is in a merge request for radv[2]. This contains an initial implementation of H264 and H265 decoding for AMD GPUs from TONGA to NAVI2x. It passes the basic conformance tests but fails some of the more complicated ones, but it has decoded the streams we've been throwing at it using ffmpeg. Building: git clone https:// gitlab.freedesktop.org/airlied/mesa git checkout radv-vulkan-video-prelim-decode mkdir build meson build -Dvulkan-beta =true -Dvulkan-drivers=amd -Dvideo-codecs=h264dec,h265dec --prefix= cd build ninja ninja install Running: export VK_ICD_FILENAMES=/share/vulkan/icd.d/radeon_icd.x86_64.json Read more LPC 2022 Accelerators BOF outcomes summary September 16, 2022 At Linux Plumbers Conference 2022, we held a BoF session around accelerators. This is a summary made from memory and notes taken by John Hubbard. We started with defining categories of accelerator devices. 1. single shot data processors, submit one off jobs to a device. (simpler image processors) 2. single-user, single task offload devices (ML training devices) 3. multi-app devices (GPU, ML/ inference execution engines) One of the main points made is that common device frameworks are normally about targeting a common userspace (e.g. mesa for GPUs). Since a common userspace doesn't exist for accelerators, this presents a problem of what sort of common things can be targetted. Discussion about tensorflow, pytorch as being the userspace, but also camera image processing and OpenCL. OpenXLA was also named as a userspace API that might be of interest to use as a target for implementations. There was a discussion on what to call the subsystem and where to place it in the tree. It was ag Read more Powered by Blogger Theme images by Michael Elkan Dave Airlie Visit profile Archive * July 20231 * May 20232 * April 20232 * March 20231 * February 20231 * January 20232 * December 20223 * September 20222 * July 20221 * May 20221 * February 20221 * November 20214 * August 20211 * July 20211 * April 20213 * March 20211 * November 20201 * October 20201 * August 20201 * May 20201 * May 20191 * December 20181 * July 20181 * March 20181 * February 20181 * December 20171 * November 20171 * October 20171 * August 20171 * July 20172 Show more Show less Report Abuse