[HN Gopher] Exo: Exocompilation for productive programming of ha...
___________________________________________________________________
Exo: Exocompilation for productive programming of hardware
accelerators
Author : gnabgib
Score : 73 points
Date : 2025-03-14 18:35 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| erdaniels wrote:
| I'm not the target audience but the GitHub and website getting
| started page feel so poorly explained. What the hell is a
| schedule?
| gnabgib wrote:
| This MIT article covers it a bit more (with a slightly too
| generic title) _High-performance computing, with much less
| code_ https://news.mit.edu/2025/high-performance-computing-
| with-mu... (https://news.ycombinator.com/item?id=43357091)
| imtringued wrote:
| https://github.com/exo-lang/exo/blob/main/examples/avx2_matm...
|
| I personally am not convinced.
| lostmsu wrote:
| Agreed. It looks like if you need to optimize it would be
| much easier to just modify the code directly. The result will
| also be more readable and therefore easier to support in the
| future.
| ajb wrote:
| A schedule is the order in which machine instructions get
| executed.
|
| So, I've done this professionally (written assembler code, and
| then scheduled it manually to improve performance). Normally
| you don't need to do that these days, as even mobile CPUs use
| out-of-order cores which dynamically schedule at runtime.
|
| It's only going to be useful if you're writing code for some
| machine that doesn't do that (they give examples of TPU etc)
| almostgotcaught wrote:
| > out-of-order cores which dynamically schedule at runtime.
|
| OOO architectures don't reschedule dynamically - that's
| impossible - they just have multiple instruction buffers that
| can issue the instructions. So scheduling is still important
| for OOO it's just at the level of DDG instead of literally
| linear order in the binary.
|
| Edit: just want to emphasize
|
| > It's only going to be useful if you're writing code for
| some machine that doesn't do that
|
| There is no architecture for which instruction scheduling
| isn't crucial.
| ajb wrote:
| If you're talking about modifying the DDG, I would not call
| that scheduling. Because then you need to do serious work
| to prove that your code is actually doing the same thing.
| But I haven't spent a lot of time in the compiler world, so
| maybe they do call it that. Perhaps you could give your
| definition?
| achierius wrote:
| > There is no architecture for which instruction scheduling
| isn't crucial.
|
| In my experience doing back-end compiler work, it's
| definitely last on the list of major concerns. Obviously
| you can't ignore it, but it's not where any significant
| gains are coming from, and anything 'fancy' you do there is
| likely to be picked up by future generations of hardware.
| almostgotcaught wrote:
| > but it's not where any significant gains are coming
| from
|
| i have no clue what you're saying here - if your
| scheduler fucks up your pipelining you're gonna have a
| bad time (conversely if you're scheduler recognizes
| pipelining opportunities you're gonna have a good time).
| as usual anyone who says something like "in my experience
| it's not a big deal" _simply does not have enough
| experience_.
| almostgotcaught wrote:
| "[compilation] for productive programming of hardware
| accelerators"
|
| But 93% of the codebase is Python lol. Whatever you think about
| Python, it is not a systems programming language. Takeaway: this
| is not a serious compiler project (and it's clearly not, it's a
| PhD student project).
|
| Deeper take: this is just a DSL that behind the scenes calls a
| (SMT) solver for tuning (what they call "scheduling"). There are
| a million examples of this approach for every architecture under
| the sun. My org is literally building out the same thing right
| now. Performance is directly a function of how good your model of
| the architecture is. At such a high-level it's very likely to
| produce _suboptimal_ results because you have no control over ISA
| /intrinsic level decisions. Lower-level implementations are much
| more robustly "powerful".
|
| https://dl.acm.org/doi/10.1145/3332373
| rscho wrote:
| Well, this is clearly an attempt at abstracting the kind of
| low-level stuff you describe. Perhaps it doesn't work (yet),
| but that shouldn't prevent people from trying ? Involving a SMT
| solver suggests that the solver is doing the heavy-lifting, not
| python. PhDs often produce inapplicable stuff, but they are
| also the basis for industry/application R&D, such as what your
| org is doing... PhDs are the slaves of science. They make stuff
| happen for peanuts in return and deserve our respect for that,
| even if what happens is oftentimes a dead-end. It's really sad
| seeing people shitting on PhDs.
| pclmulqdq wrote:
| Unfortunately, the comment you are responding to is more
| correct on this than I think you are. The python thing was
| stupid, though - a lot of high-performance code gets written
| in libraries like numpy (that call C 99.99% of the time) or
| pytorch (JIT-ed before executing) that keep Python our of the
| critical path.
|
| The problem with this research is that many similar ideas
| have been tried before in the contexts of supercomputers or
| CPU compilers. They ultimately all fail because they end up
| (1) being more work to program in, and (2) not being any
| faster because real life happens. Networks drop packets,
| clock frequencies jitter, and all sorts of non-determinism
| happens when you have large scale. A static scheduler forces
| you to stall the whole program for any of these faults. All
| the gain you got by painstakingly optimizing things goes
| away.
|
| PhD theses, in a _best-case_ scenario, are the basis for new
| applications. Most of them amount to nothing. This one
| belongs on that pile. The sad part about that is that it isn
| 't the student's fault that the professor sent them down a
| research direction that is guaranteed to amount to nothing of
| use. This one is on the professor.
| fancyfredbot wrote:
| I don't think the paper is about statically scheduled
| architectures. In fact they mention it's for modern
| accelerators. These switch between threads in a dynamic way
| rather than stalling. The scheduling being referred to
| seemed to mean the order in which instructions should be
| fed to a potentially dynamic scheduler to enable efficient
| usage of caches etc.
|
| So I'm not sure you can dismiss it as a thesis which will
| amount to nothing on the basis that static scheduling is a
| bad idea!
|
| I could easily have missed something though. It's not a
| particularly clear or succinct write-up and I have only
| read some of it. If it does say that it only works for
| strictly deterministic in-order architectures somewhere
| please can you point out where?
| QuadmasterXLII wrote:
| any sufficiently powerful compiler is going to run an
| interpreted language at compile time, and there's no reason it
| can't be Python instead of C++ template metaprograms or CMake
| fancyfredbot wrote:
| Your take seems to contradict the article? You say SMT solvers
| give "no control over ISA/intrinsic level decisions" but their
| design.md says "user-defined scheduling operations can
| encapsulate common optimization patterns and hardware-specific
| transformations". Are they wrong about this? Can you explain
| why?
| saagarjha wrote:
| Maybe a takeaway that boils down to "how could anyone ever
| write a compiler on Python" is the wrong one to have.
| alex7o wrote:
| Halide does something similar but as C++ for it dsl langauge:
| https://halide-lang.org/
| gotoeleven wrote:
| The Exo docs mention Halide as an example of a language similar
| to Exo but is "lowering-based", while Exo is "rewrite-based."
| This seems to mean that Halide is more of a DSL where what you
| want is specified at a higher level while Exo is a set of
| transformations you can apply to an existing kernel of code
| (though at least in the examples the kernel of code is written
| in python and then somehow C is generated from it, after
| transformations are applied). Do you know what the relative
| strengths of these two approaches are?
|
| Also, are there any languages in this vein that take more of a
| declarative (as opposed to imperative) approach?
| alex7o wrote:
| I have mostly a passing interest in the topic, however from
| what I understand Halide first lowers to an intermediate
| language then they do the scheduling and optimisation. While
| exo, does the optimisations directly on the python code. Also
| Halide needs you to tell it how to lower the code and how to
| do the scheduling. Which is something that exo can determine
| itself.
| stanleykm wrote:
| You guys might want to explain what this is a little better. My
| first question was "what do they mean by 'hardware
| accelerators'?"
|
| It wasnt until I saw the matmul example that I realized this is
| (probably, its still unclear) for GPUs.
| Netcob wrote:
| Probably any hardware that requires you to use some
| intermediate code. Nowadays probably some TPU or NPU or
| whatever.
|
| This made me think of what "accelerators" I've come across
| (just as a regular consumer):
|
| In the late 90s, that's what we called GPUs - "3D
| accelerators". For a short time, they were cards separate from
| the actual graphics card, and often would involve routing your
| VGA cable through that thing. Before it all merged into one
| device. I was very slightly disappointed as a kid that I
| narrowly missed that time, but bilinear filtering and higher
| resolutions and framerates helped me get over the fact I
| couldn't cram more cool weird tech into my PCI slots.
|
| Then you had sound cards with "audio accelerators" using
| technologies like EAX. All that eventually migrated back to the
| CPU I think.
|
| For a while you could by a "physics accelerator" for PhysX,
| then acquired by nvidia and moved to the GPU using CUDA. I
| never had one, but one time I kept around an older GPU after
| upgrading as a dedicated physx processor. Now that's the only
| way to run older 32bit games with physx turned up, since that's
| not supported in 5000 series GPUs.
|
| Finally, I got this "Coral TPU", a USB device with 4GB RAM (and
| I think around 4 TOPS or something?) for very efficient
| inferencing. There are some open source projects supporting
| this, like frigate, which lets you process object detection
| with multiple surveillance camera streams on a raspberry pi. I
| never really used it though.
|
| And of course now we have NPUs as sub-systems in CPUs and GPUs.
| I'd love to have a dedicated PCIe card with that, but of course
| having yet another computer architecture with dozens/hundreds
| of GB of redundant RAM is kind of a bummer.
| k_bx wrote:
| Spent 5 minutes browsing but still couldn't figure this out.
| Would it be able to target FPGA?
| DiabloD3 wrote:
| Neat idea, but why is this implemented in Python instead of a
| more suitable language? Also, there are many DSLs and APIs
| already for this, some even supported by industries and provided
| as part of the SDKs for various hardware blobs, so why Exo?
|
| Like, I hate SYCL, Kokkos, and all the other similar things as
| much as the next guy, but major companies and orgs support and
| use those already.
___________________________________________________________________
(page generated 2025-03-15 23:01 UTC)