[HN Gopher] Using Libc for GPUs
___________________________________________________________________
Using Libc for GPUs
Author : hochmartinez
Score : 80 points
Date : 2024-12-11 15:29 UTC (3 days ago)
(HTM) web link (libc.llvm.org)
(TXT) w3m dump (libc.llvm.org)
| almostgotcaught wrote:
| this was presented at llvm this year
| https://www.youtube.com/watch?v=4TxGWis1mws - it was a nice talk.
| kelsey98765431 wrote:
| Running normal c directly on gpu has been the dream for a long
| time. this looks excellent
| C-programmer wrote:
| Genuinely, why?
|
| - For new code, all of the functions
| [here](https://libc.llvm.org/gpu/support.html#libc-gpu-support)
| you can do without just fine.
|
| - For old code: * Your project is large enough
| that you are likely use using an unsupported libc function
| somewhere. * Your project is small enough that you
| would benefit from just implementing a new kernel yourself.
|
| I am biased because I avoid the C standard library even on the
| CPU, but this seems like a technology that raises the floor not
| the ceiling of what is possible.
| tredre3 wrote:
| > this seems like a technology that raises the floor not the
| ceiling of what is possible.
|
| In your view, how is making GPU programming easier a bad
| thing?
| convolvatron wrote:
| that's clearly not a bad thing. however encouraging people
| to run mutating, procedural code with explicit loops and
| aliasing maybe isn't the right path to get there.
| particularly if you just drag forward all the weird old
| baggage with libc and its horrible string conventions.
|
| I think any programming environment that treats a gpu as a
| really slow serial cpu isn't really what you want(?)
| quotemstr wrote:
| What if it encourages people to write parallel and
| functional code on CPUs? That'd be a good thing.
| Influence works both ways.
|
| The bigger problem is that GPUs have various platform
| features (shared memory, explicit cache residency and
| invalidation management) that CPUs sadly don't yet. Sure,
| you _could_ expose these facilities via compiler
| intrinsics, but then you end up code that might be
| syntactically valid C but is alien both to CPUs and human
| minds
| JonChesterfield wrote:
| One thing this gives you is syscall on the gpu. Functions
| like sprintf are just blobs of userspace code, but others
| like fopen require support from the operating system (or
| whatever else the hardware needs you to do). That plumbing
| was decently annoying to write for the gpu.
|
| These aren't gpu kernels. They're functions to call from
| kernels.
| almostgotcaught wrote:
| > One thing this gives you is syscall on the gpu
|
| i wish people in our industry would stop (forever,
| completely, absolutely) using metaphors/allusions. it's a
| complete disservice to anyone that isn't in on the trick.
| it doesn't give you syscalls. that's impossible because
| there's no sys/os on a gpu and your actual os does not
| (necessarily) have any way to peer into the address
| space/schedular/etc of a gpu core.
|
| what it gives you is something that's working really really
| hard to pretend be a syscall:
|
| > Traditionally, the C library abstracts over several
| functions that interface with the platform's operating
| system through system calls. The GPU however does not
| provide an operating system that can handle target
| dependent operations. Instead, we implemented remote
| procedure calls to interface with the host's operating
| system while executing on a GPU.
|
| https://libc.llvm.org/gpu/rpc.html.
| quotemstr wrote:
| It's a matter of perspective. If you think of the GPU as
| a separate computer, you're right. If you think of it as
| a coprocessor, then the use of RPC is just an
| implementation detail of the system call mechanism, not a
| semantically different thing.
|
| When an old school 486SX delegates a floating point
| instruction to a physically separate 487DX coprocessor,
| is it executing an instruction or doing an RPC? If RPC,
| does the same instruction start being a real instruction
| when you replace your 486SX with a 486DX, with an
| integrated GPU? The program can't tell the difference!
| almostgotcaught wrote:
| > It's a matter of perspective. If you think of the GPU
| as a separate computer, you're right.
|
| this perspective is a function of exactly one thing: do
| you care about the performance of your program? if not
| then sure indulge in whatever abstract perspective you
| want ("it's magic, i just press buttons and the lights
| blink"). but if you don't care about perf then why are
| you using a GPU at all...? so for people that aren't just
| randomly running code on a GPU (for shits and giggles),
| the distinction is very significant between "syscall" and
| syscall.
|
| people who say these things don't program GPUs for a
| living. there are no abstractions unless you don't care
| about your program's performance (in which case why are
| you using a GPU at all).
| quotemstr wrote:
| Not everything in every program is performance critical.
| A pattern I've noticed repeatedly among CUDAheads is the
| idea that "every cycle matters" and therefore we should
| uglify and optimize even cold parts of our CUDA programs.
| That's as much BS on GPU as it is on CPU. In CPU land, we
| moved past this sophomoric attitude decades ago. The GPU
| world might catch up one day.
|
| Are you planning on putting fopen() in an inner loop or
| something? LOL
| nickysielicki wrote:
| genuinely asking: where else should ML engineers focus
| their time, if not on looking at datapath bottlenecks in
| either kernel execution or the networking stack?
| almostgotcaught wrote:
| > A pattern I've noticed repeatedly among CUDAheads is
| the idea that "every cycle matters" and therefore we
| should uglify and optimize even cold parts of our CUDA
| programs
|
| I don't know what a "cudahead" is but if you're gonna
| build up a strawman just to chop it down have at it.
| Doesn't change anything about my point - these aren't
| syscalls because there's no sys. I mean the dev here
| literally spells it out correctly so I don't understand
| why there's any debate.
| oivey wrote:
| The whole reason CUDA/GPUs are fast is that they
| explicitly don't match the architecture of CPUs. The
| truly sophomoric attitude is that all compute devices
| should work like CPUs. The point of CUDA/GPUs is to
| provide a different set of abstractions than CPUs that
| enable much higher performance for certain problems.
| Forcing your GPU to execute CPU-like code is a bad
| abstraction.
|
| Your comment about putting fopen in an inner loop really
| betrays that. Every thread in your GPU kernel is going to
| have to wait for your libc call. You're really confused
| if you're talking about hot loops in a GPU kernel.
| JonChesterfield wrote:
| The "proper syscall" isn't a fast thing either. The
| context switch blows out your caches. Part of why I like
| the name syscall is it's an indication to not put it on
| the fast path.
|
| The implementation behind this puts a lot of emphasis on
| performance, though the protocol was heavilt simplfied in
| upstreaming. Running on pcie instead of the APU systems
| makes things rather laggy too. Design is roughly a mashup
| of io_uring and occam, made much more annoying by the GPU
| scheduler constraints.
|
| The two authors of this thing probably count as people
| who program GPUs for a living for what it's worth.
| JonChesterfield wrote:
| Well, I called it syscall because it's a void function of
| 8 u64 arguments which your code stumbles into, gets
| suspended, then restored with new values for those
| integers. That it's a function instead of an instruction
| doesn't change the semantics. My favourite of the uses of
| that is to pass six of those integers to the x64 syscall
| operation.
|
| This isn't misnaming. It's a branch into a trampoline
| that messes about with shared memory to give the effect
| of the x64 syscall you wanted, or some other thing that
| you'd rather do on the cpu.
|
| There's a gpu thing called trap which is closer in
| behaviour to what you're thinking of but it's really
| annoying to work with.
|
| Side note, RPC has a terrible rep for introducing failure
| modes into APIs, but that's completely missing here
| because pcie either works or your machine is gonna have
| to reboot. There are no errors on the interface that can
| be handled by the application.
| almostgotcaught wrote:
| > Well, I called it syscall because it's a void function
| of 8 u64 arguments which your code stumbles into, gets
| suspended, then restored with new values for those
| integers
|
| I'm put it really simply: is there a difference (in perf,
| semantics, whatever) between using this "syscalls" to
| implement fopen on GPU and using a syscall to implement
| fopen on CPU? Note that's a rhetorical question because
| we both already know that the answer is yes. So again
| you're just playing slight of hand in calling them
| syscalls and I'll emphasize: this is a slight of hand
| that the dev himself doesn't play (so why would I take
| your word over his).
| JonChesterfield wrote:
| Wonderfully you don't need to trust my words, you've got
| my code :)
|
| If semantics are different, that's a bug/todo. It'll have
| worse latency than a CPU thread making the same kernel
| request. Throughput shouldn't be way off. The GPU writes
| some integers to memory that the CPU will need to read,
| and then write other integers, and then load those again.
| Plus whatever the x64 syscall itself does. That's a bunch
| of cache line invalidation and reads. It's not as fast as
| if the hardware guys were on board with the strategy but
| I'm optimistic it can be useful today and thus help
| justify changing the hardware/driver stack.
|
| The whole point of libc is to paper over the syscall
| interface. If you start from musl, "syscall" can be a
| table of function pointers or asm. Glibc is more
| obstructive. This libc open codes a bunch of things, with
| a rpc.h file dealing with synchronising memcpy of
| arguments to/from threads running on the CPU which get to
| call into the Linux kernel directly. It's mainly
| carefully placed atomic operations to keep the data
| accesses well defined.
|
| There's also nothing in here which random GPU devs can't
| build themselves. The header files are (now) self
| contained if people would like to use the same mechanism
| for other functionality and don't want to handroll the
| data structure. The most subtle part is getting this to
| work correctly under arbitrary warp divergence on volta.
| It should be an out of the box thing under openmp early
| next year too.
| rowanG077 wrote:
| Why does a perf difference factor into it. There is no
| requirement for a syscall to be this fast or else it
| isn't a syscall. If you have a hot loop you shouldn't be
| putting a syscall in it, not even on the CPU.
| nickysielicki wrote:
| https://developer.nvidia.com/blog/simplifying-gpu-
| applicatio...
| benatkin wrote:
| Remember that TFA is LLVM. In LLVM, C is just a frontend. The
| dream is there because a lot of code is already written in C.
| JonChesterfield wrote:
| > Genuinely, why?
|
| > ... this seems like a technology that raises the floor not
| the ceiling of what is possible.
|
| The root cause reason for this project existing is to show
| that GPU programming is not synonymous with CUDA (or the
| other offloading languages).
|
| It's nominally to help people run existing code on GPUs.
| Disregarding that use case, it shows that GPUs can actually
| do things like fprintf or open sockets. This is obvious to
| the implementation but seems largely missed by application
| developers. Lots of people think GPUs can only do floating
| point math.
|
| Especially on an APU, where the GPU units and the CPU cores
| can hammer on the same memory, it is a travesty to persist
| with the "offloading to accelerator" model. Raw C++ isn't an
| especially sensible language to program GPUs in but it's
| workable and I think it's better than CUDA.
| rbanffy wrote:
| > Lots of people think GPUs can only do floating point
| math.
|
| IIRC, every Raspberry Pi is brought up by the GPU setting
| up the system before the CPU is brought out of reset and
| the bootloader looks for the OS.
|
| > it is a travesty to persist with the "offloading to
| accelerator" model.
|
| Operating systems would need to support heterogeneous
| processors running programs with different ISAs accessing
| the same pools of memory. I'd _LOVE_ to see that. It 'd be
| extremely convenient to have first-class processes running
| on the GPU MIMD cores.
|
| I'm not sure there is much research done in that space. I
| believe IBM mainframe OSs have something like that because
| programmers are exposed to the various hardware assists
| that run as coprocessors sharing the main memory with the
| OS and applications.
| als0 wrote:
| > I'm not sure there is much research done in that space.
|
| There is. And the finest example I can think of is
| Barrelfish https://barrelfish.org
| krackers wrote:
| >Disregarding that use case, it shows that GPUs can
| actually do things like fprintf or open sockets.
|
| Can you elaborate on this? My mental model of GPU is
| basically like a huge vector coprocessor. How would things
| like printf or sockets work directly from the GPU when they
| require syscalls to trap into the OS kernel? Given that the
| kernel code is running on the CPU, that seems to imply that
| there needs to be a handover at some point. Or conversely
| even if there was unified memory and the GPU could directly
| address memory-mapped peripherals, you'd basically need to
| reimplement drivers wouldn't you?
| einpoklum wrote:
| > The root cause reason for this project existing is to
| show that GPU > programming is not synonymous with CUDA (or
| the other offloading > languages).
|
| 1. The ability to use a particular library does not reflect
| much on which languages can be used.
|
| 2. One you have PTX as a backend target for a compiler,
| obviously you can use all sorts of languages on the
| frontend - which NVIDIA's drivers and libraries won't even
| know about. Or you can just use PTX as your language -
| making your point that GPU programming is not synonymous
| with CUDA C++.
|
| > It's nominally to help people run existing code on GPUs.
|
| I'm worried you might be right. But - we should really not
| encourage people to run existing CPU-side code on GPUs,
| that's rarely (or maybe never?) a good idea.
|
| > Raw C++ isn't an especially sensible language to program
| GPUs in > but it's workable and I think it's better than
| CUDA.
|
| CUDA is an execution ecosystem. The programming language
| for writing kernel code is "CUDA C++", which _is_ C++, plus
| a few builtins functions ... or maybe I'm misunderstanding
| this sentence.
| amelius wrote:
| I wonder what C would look like if CPUs would evolve into what
| GPUs are today.
|
| What if a CPU had assembly instructions for everything a GPU
| can do? Would compiler/language designers support them?
| gpderetta wrote:
| there is no reason to wonder: https://ispc.github.io/
| quotemstr wrote:
| I've never understood why people say you "can't" do this or
| that on GPU. A GPU is made of SMs, and each SM is just a CPU
| with very wide SIMD pipes and very good hyperthreading. You can
| take one thread of a warp in a SM and do exactly the same
| things a CPU would do. Would you get 1/32 potential
| performance? Sure. But so what? Years ago, we did plenty of
| useful work with less than 1/32 of a modest CPU, and we can
| again.
|
| One of the more annoying parts of the Nvidia experience is PTX.
| I. I know perfectly well that your CPU/SM/whatever has a
| program counter. Let me manipulate it directly!
| bee_rider wrote:
| I think people are just saying it isn't very cost effective
| to use a whole GPU as 1/32 of a modest (modern?) CPU.
| quotemstr wrote:
| Cars are slow and inefficient in reverse gear but a car
| that couldn't drive in reverse would be broken.
| bee_rider wrote:
| You put "can't" in quotes, so I guess you are quoting
| somebody, but I don't see where the quote is from, so I'm
| not sure what they actually meant.
|
| But I suspect they are using "can't" informally. Like:
| You can't run a drag race in reverse. Ok, you technically
| could, but it would be a silly thing to do.
| Archit3ch wrote:
| No direct Metal target.
| almostgotcaught wrote:
| this is an LLVM project... you want this to work on Metal, ask
| apple to add a Metal backend to LLVM
|
| https://github.com/llvm/llvm-project/tree/main/llvm/lib/Targ...
| rbanffy wrote:
| I am surprised there isn't.
| JonChesterfield wrote:
| No Intel either. The port would be easy - gpuintrin.h abstracts
| over the intrinsics, provide an implementation for those, write
| a loader in terms of opencl or whatever if you want to run the
| test suite.
|
| The protocol needs ordered load/store on shared memory but
| nothing else. I wrote a paper trying to make it clear that
| load/store on shmem was sufficient which doesn't seem to be
| considered persuasive. It's specifically designed to tolerate
| architectures doing slopping things with cache invalidation. It
| could run much faster with fetch_or / fetch_and instructions
| (as APUs have, but PCIe does not). It could also hang off DMA
| but that isn't implemented (I want to have the GPU push packets
| over the network without involving the x64 CPU at all).
| gdiamos wrote:
| Nice to see this finally after 15 years.
| amelius wrote:
| I hate libc. It's such a common cause of versioning problems on
| my system.
|
| Can't they just stop making new versions of it?
| wbl wrote:
| Dynamic linking has some benefits, but many detractors.
| rbanffy wrote:
| It's nice not having to recompile all your software because
| of a CVE impacting libc, or any other fundamental component
| of the system.
| fuhsnn wrote:
| I wonder how GPU is going to access an unknown size NULL
| terminated string in system RAM, the strchr() source looks like
| normal C++. In my minimal Vulkan GPGPU experience the data need
| to be bound to VkDeviceMemory to be accessible through PCI bus
| with compute shader, is LLVM libc runtime doing similar set-ups
| in the background, and if so, is it faster than glibc's hand-
| tuned AVX implementation?
| JonChesterfield wrote:
| This is libc running on the GPU, not a libc spanning CPU and
| GPU. The primary obstruction to doing that is persuading people
| to let go of glibc. The spec of "host runs antique glibc, GPU
| runs some other thing that interops transparently with glibc"
| is a nightmare of hacks and tragedy.
|
| What would be relatively easy to put together is llvm libc
| running on the host x64 and also llvm libc running on the GPU.
| There's then the option to do things like malloc() on the GPU
| and free() the same pointer on the CPU. Making it genuinely
| seamless also involves persuading people to change what
| function pointers are, do some work on the ABI, and preferably
| move to APUs because PCIe is unhelpful.
|
| There's an uphill battle to bring people along on the journey
| of "program the things differently". For example, here's a
| thread trying to drum up enthusiasm for making function
| pointers into integers as that makes passing them between
| heterogenous architectures far easier
| https://discourse.llvm.org/t/rfc-function-pointers-as-
| intege....
| einpoklum wrote:
| I am pretty sure this is just a gimmick. I would not call libc
| code in a GPU kernel. It would mean dragging in a whole bunch of
| stuff I don't want, and cant control or pick-and-choose. That
| makes sense for regular processes on the CPU; it does _not_ make
| sense in code you run millions of times in GPU threads.
|
| I see people saying they've "dreamed" of this or have waited so
| long for this to happen... well, my friends, you should not have;
| and I'm afraid you're in for a disappointment.
| JonChesterfield wrote:
| It uses a few mb of contiguous shared memory and periodically
| calling a function from a host thread. Unless you only want
| sprintf or similar in which case neither is needed. The unused
| code deadstrips pretty well. Won't help compilation time.
| Generally you don't want libc calls in numerical kernels doing
| useful stuff - the most common request was for printf as a
| debugging crutch, I mostly wanted mmap.
___________________________________________________________________
(page generated 2024-12-14 23:00 UTC)