[HN Gopher] Running GPT-2 in WebGL: Rediscovering the Lost Art o...
___________________________________________________________________
Running GPT-2 in WebGL: Rediscovering the Lost Art of GPU Shader
Programming
Author : nathan-barry
Score : 75 points
Date : 2025-05-27 18:02 UTC (4 hours ago)
(HTM) web link (nathan.rs)
(TXT) w3m dump (nathan.rs)
| nathan-barry wrote:
| A few weeks back, I implemented GPT-2 using WebGL and shaders.
| Here's a write-up over how I made it, covering how I used
| textures and frame buffer objects to store and move around
| weights and outputs from calculations while using WebGL.
| vessenes wrote:
| Request -- big bold link to a working web page right at the
| top? I read the page, I read your github, and I saw
| instructions to clone and run a node site, and was like "..nah"
| I think github pages will serve this up for free if you like.
|
| p.s. Cool!
| nathan-barry wrote:
| Yeah, that would have been a good thing to set up. Main thing
| to add would be loading the weights into the browser
| nathan-barry wrote:
| Here's a link to the github repo. At the top of the README it
| has a demo of GPT-2 running and the visualizations of the
| attention matrices and transformer block outputs
|
| Repo: https://github.com/nathan-barry/gpt2-webgl
| pjmlp wrote:
| Kudos for going down the WebGL route and not the Chrome only
| WebGPU approach that most likely some people expect.
|
| It is going to take at least yet another year, for WebGPU 1.0
| to be available on stable versions on other browsers, Chrome
| still hasn't stable WebGPU on GNU/Linux, and it is already much
| far ahead with extensions that most likely won't be on the 1.0
| MVP of the other browsers.
|
| Interesting article.
| rezmason wrote:
| Nice writeup! I'm a fan of shader sandwiches like yours. Judging
| from the stated limitations and conclusion, I bet this would
| benefit tremendously from a switch to WebGPU. That's not a
| criticism! If anything, you're effectively demonstrating that
| WebGL, a sometimes frustrating but mature ubiquitous computing
| platform, can be a valuable tool in the hands of the ambitious.
| Regardless, I hope to fork your repo and try a WebGPU port. Good
| stuff!
| nathan-barry wrote:
| Thanks for the comment! I did this as a final project in a
| graphics class where we mainly used WebGL for all the
| assignments. Seeing the improvements a WebGPU port would bring
| would be cool to see!
| GloamingNiblets wrote:
| Out of curiosity, what degree and level is this class? Super
| cool for a final project
| summarity wrote:
| For a webgpu version there's tokenhawk (pure implementation)
| flakiness wrote:
| CUDA is better for sure, but the pure functional nature of the
| traditional shader is conceptually much simpler and I kind of
| relish the simplicity. There is no crazy tiling or anything. Just
| per-pixel parallelism [1]. It won't be as fast as those real,
| highly-tuned kernels, but it's still nice to see something simple
| that does something non-trivial. It reminded me of the early
| "GPGPU" days (early 2000s?)
|
| [1] https://github.com/nathan-
| barry/gpt2-webgl/blob/main/src/gpt...
| 3abiton wrote:
| > GPGPU
|
| I haven't heard this one in a long while
| sigmoid10 wrote:
| Fun fact: Long before the dawn of the GPU deep learning hype
| (and even before CUDA was a thing), a bunch of CS nerds from
| Korea managed to train a neural network on an ATI (now AMD)
| Radeon 9700 Pro using nothing but shaders [1]. They saw an even
| bigger performance improvement than Hinton and his group did
| for AlexNet 8 years later using CUDA.
|
| [1]
| https://ui.adsabs.harvard.edu/abs/2004PatRe..37.1311O/abstra...
| math_dandy wrote:
| Cool, I had not heard about this. Adding this paper to my
| machine learning teaching bibliography.
|
| Even though the start of the deep learning renaissance is
| typically dated to 2012 with Alexnet, things were in motion
| week before that. As you point out, GPU training was
| validated at least 8 years previously. Concurrently, some
| very prescient researchers like Li were working hard to
| generate large scale datasets like ImageNet (CVPR 2009,
| https://www.image-
| net.org/static_files/papers/imagenet_cvpr0...). And in 2012
| it all came together.
| sigmoid10 wrote:
| Since shaders were designed for, well, shading, this early
| experiment was more of an academic playground exercise than
| useful research. But AlexNet still wasn't the first deep
| neural network trained using CUDA. It had already been done
| three years earlier:
| https://dl.acm.org/doi/10.1145/1553374.1553486
|
| The ImageNet competition had also been around since 2010.
| So the ingredients were actually all there before.
| nickpsecurity wrote:
| People have complained Nvidia dominates the market. Many were
| looking at their older GPU's for cheap experimentation. One idea
| I had was just using OpenCL to use all the cross-platform support
| it has. Even some FPGA's support it.
|
| Good to see GPT-2 done with shader programming. A port of the
| techniques used in smaller models, like TinyLlama or Gemma-2B,
| might lead to more experimentation with older or cheaper hardware
| [on-site].
| bigyabai wrote:
| Nvidia dominates training moreso than inference, and mostly
| because of their hardware efficiency and not entirely because
| of CUDA. To dominate Nvidia, you have to beat their TSMC
| investments, beat their design chops and match their software
| support. The shortlist consisting of companies that can do that
| reads "Apple" and nobody else, which is exactly the sort of
| captive market Nvidia wants to conquer.
|
| OpenCL has been pretty handy for inference on older cards, but
| I'd argue it's relevance is waning. llama.cpp has Vulkan
| compute now which requires a smaller featureset for hardware to
| support. Many consumer devices skip OpenCL/CUDA altogether and
| delegate inference to an NPU library.
| tsurba wrote:
| 1.5-2 years ago I did some training for a ML paper on 4 AMD
| MI250x (each is essentially 2 gpus so 8 in total really, each
| with 64GB VRAM) on LUMI.
|
| My Jax models and the baseline PyTorch models were quite easy
| to set up there, and there was not a noticeable perf
| difference to 8x A100s (which I used for prototyping on our
| university cluster) in practice.
|
| Of course it's just a random anecdote, but I don't think
| nvidia is actually _that_ much ahead.
| pjmlp wrote:
| CUDA dominates and almost no one cares about OpenCL, exactly
| because it is stuck on C99 model, never cared for Fortran, they
| only adopted a bytecode format when the war was already lost,
| SYCL/DPC++ is mostly an Intel thing and after they acquired
| CodePlay, Intel/AMD kind of always fail on delivering good
| OpenCL tooling, now they decided to reboot the whole oneAPI
| effort as UXL Foundation,....
|
| NVidia has a lot to thank the competition why they happen to
| dominate the market.
| cmovq wrote:
| > gl.drawArrays(gl.TRIANGLES, 0, 6);
|
| Using 2 tris for this isn't ideal because you will get duplicate
| fragment invocations along the horizontal seam where the
| triangles meet. It is slightly more efficient to use one larger
| triangle extending outside the viewport, the offscreen parts will
| be clipped and not generate any additional fragments.
|
| [1]: https://wallisc.github.io/rendering/2021/04/18/Fullscreen-
| Pa...
| lerp-io wrote:
| i think u can use webgpu compute shaders now
___________________________________________________________________
(page generated 2025-05-27 23:00 UTC)