[HN Gopher] Tinygrad and rusticl and aco: why not?
___________________________________________________________________
Tinygrad and rusticl and aco: why not?
Author : pantalaimon
Score : 43 points
Date : 2023-07-14 10:58 UTC (1 days ago)
(HTM) web link (airlied.blogspot.com)
(TXT) w3m dump (airlied.blogspot.com)
| rhn_mk1 wrote:
| Rusticl seems a bit hidden for all the (positive) noise it
| creates.
|
| I'd like to try it out, but:
|
| - are there any distributions shipping it?
|
| - what hardware does it support?
|
| I can guess that distros are not shipping it due to some
| stability concerns (which I could not see so far). I'm also
| guessing that despite being a Gallium driver, some features will
| depend on the capabilities of the underlying hardware. Can I hope
| that my AMD APU will finally get some OpenCL support?
| pantalaimon wrote:
| > what hardware does it support?
|
| That's the nice thing about it, it supports most hardware
| supported by Mesa going back even to r600 GPUs [0]. So anything
| Intel or AMD should work.
|
| My guess for as to why distributions are not enabling it yet
| might be due to the Rust build dependency that might require a
| newer Rust version than what's provided by the distribution.
|
| [0]
| https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20...
| brucethemoose2 wrote:
| > Can I hope that my AMD APU will finally get some OpenCL
| support?
|
| What APU is it? OpenCL support goes back pretty far.
| rhn_mk1 wrote:
| I haven't found anything for the Vega 8 that would run on
| Linux.
| brucethemoose2 wrote:
| ? I have had OpenCL on my 4900HS for awhile, just through
| the Mesa OpenCL driver.
|
| I believe my really old Llano (2011) APU even had OpenCL
| support. It was a selling point at the time.
| sspiff wrote:
| Same, it just works on my Debian powered home server with
| 5800H with just the Mesa ICD installed.
|
| Not that it really matters for me, I only run wgpu
| compute stuff over Vulkan for my experiments.
| rhn_mk1 wrote:
| Hm, now I have to start asking questions of my OS.
|
| For what it's worth, my Llano kept hanging up when I
| tried.
| brucethemoose2 wrote:
| You are probsbly just missing the driver, some linux
| distros dont ship it by default.
|
| If you are on Windows... _shrug_.
| sitkack wrote:
| XDC 2022 | Rusticl: An OpenCL implementation written in Rust |
| Karol Herbst
|
| https://www.youtube.com/watch?v=Es7UmvNAEvk
| jchw wrote:
| > large GPU stacks usually make me want to wipe the machine after
| I test them
|
| Core value proposition for NixOS/GuixSD right here. That said,
| the amount of time you have to dedicate to figuring out how to
| make everything work well for your use cases might negate the
| benefits a bit :)
| colordrops wrote:
| Once you figure it out you never have to figure it out again.
|
| And there is likely someone else who already figured out
| something as common as this so you could just import their
| config.
| jchw wrote:
| I love NixOS, but if you're a heavy Linux user the odds that
| you do something fairly unique in your personal setup gets
| very high. For me, to give you an example of just one little
| microcosm, I have probably the only combination of SwayWM,
| gnome-keyring and NixOS in the world that works the exact way
| mine does, and it took quite a while to land on it (and
| sometimes time to maintain it.) It's actually pretty simple
| and elegant now, but getting to where I am today felt
| anything but simple.
|
| Back when I first started (5 years ago,) drivers for my label
| printer were not available, and I stumbled around trying to
| figure out how to make a derivation for them. Ultimately I
| succeeded, but for a n00b it did feel pretty ridiculously
| hard to try to figure it out while not even having much of a
| background in pure FP languages less the oddity that is Nix.
| I never got around to adding it to Nixpkgs, but thankfully
| somebody else did.
|
| There is indeed solutions for a LOT of things. Is home
| manager worth the time? Not sure, I just use global configs
| instead of home dir configs for most things. There's multiple
| solutions for secrets but I have a few different Nix
| configuration repositories (for different sets of machines,
| not one for each machine; I only do this when it feels like
| it makes sense, like to keep servers for some org separate
| from my home machines) and not all of them have SOPS setup
| because while it works, bootstrapping it always has me
| guessing a little to make sure the way I'm setting it up
| actually makes sense.
|
| Flakes. They're good, but I started before everyone
| recommended them for everything. The switching cost of flakes
| was not huge, but nor was it trivial, as I had to adjust a
| lot of workflows for it. This kind of negates the "you never
| have to figure it out again" thing. On the contrary, NixOS is
| a moving and vastly complicated machine, and it indeed feels
| like things need to be re-figured out quite often. Rust
| dependency management in Nixpkgs is sure to change in the
| future: right now it can't handle all of the situations that
| Cargo can and often requires including a copy of the lockfile
| in the Nixpkgs tree which seems unideal.
|
| Worth it still? Yes, because I can throw my boot drive out
| the window and have a machine back up and running in like 10
| minutes. All of my desktop machines are very consistent,
| because any configuration that matters to me goes into
| declarative and versioned configuration. The concept of
| "hosing" my OS doesn't exist anymore, and even if a workflow
| does break, I can trivially bisect it to figure out why. Nix
| tools are super complicated, but they are super powerful too.
| NixOS is Linux with superpowers. All of the cool stuff that
| open source could theoretically enable you to do, NixOS makes
| _practical_.
|
| But is it perfect? No. To this day, you will not find a
| derivation for plenty of very useful closed source software.
| No VMWare Workstation, IDA Pro, etc. AppImage binaries that
| run other AppImages don't work with appimage-run; though I
| have a workaround with Nix-LD that does work, I've never seen
| anyone else doing it, so I suspect not too many people do
| this. Most open source software is there, but it can be a bit
| hit or miss at times. You'll occasionally find places where
| missing dependencies or hardcoded paths in the upstream
| sources lead to subtly broken or suboptimal things.
|
| So if you only manage one machine with NixOS or you don't
| have a lot of time to invest upfront, or you have some use
| cases that are pathologically difficult to deal with, it's
| not always practical for everyone.
|
| I think if NixOS immediately sells someone, they're in.
| They'll _get_ it. If not? Maybe in a couple more years. For
| me, it was on first sight.
| bsder wrote:
| > I think if NixOS immediately sells someone, they're in.
| They'll get it. If not? Maybe in a couple more years. For
| me, it was on first sight.
|
| I disagree.
|
| I _GET_ the value proposition of Nix. It 's source control
| for your operating system and environment. This is
| _valuable_. The problem is that the activation energy for
| Nix is _WAY_ higher than it ever was for source control.
|
| Recommending Nix is more like giving a beginner the advice
| "You really need to learn a scripting language." I
| regularly get pushback from junior engineers about this.
| However, it is almost uniformly one of the things that sets
| apart the average programmer from the actively good. To be
| fair, I get less pushback now that Python is more common.
|
| Nix is _even worse that this_. With scripting languages,
| you don 't have to understand the universe in order to get
| "Baby's First" to work. With Nix, it's basically "The
| fields of Elysium are laid out ahead of you over there on
| the other side of that river. Oh, but the river is a half
| mile wide raging torrent of steaming sewage. But, trust us,
| you'll be much happier after you swim through it."
|
| Yes, I know I will be happier. But swimming that half mile
| of sewage is a _LOT OF PAIN_ (from what I can see: about 6
| solid months of frustration before you "Git Gud(tm)").
|
| That's my view of Nix.
| jchw wrote:
| I'm not saying that everyone who doesn't use NixOS just
| doesn't get it. I'm saying just about anyone who is
| immediately sold is probably correct.
|
| From a romantic point of view there's obviously a lot of
| cool ideas in NixOS. However, it's not _just_ that that
| drew me in. I suffered through trying to find good ways
| to manage my configurations, things like Ansible and
| SaltStack, and dotfiles repos, and just felt unfulfilled
| by any of them: they 're all just not proper solutions.
| Kludges _at best_ , that solve a fraction of the problem
| _at most_.
|
| So when I saw NixOS, I _got_ it. Not only did I get why
| it 's good--someone who's just experienced enough would
| see that--but I immediately realized it solves the
| _exact_ problem that I have, and that I was willing to go
| great lengths to get that solution.
|
| I don't think anyone who is not a fairly experienced
| programmer with a lot of time and effort to burn will
| have a good time with Nix as it is today, though. I hope
| whatever comes next after NixOS answers that part,
| because I think everyone deserves what NixOS can make
| possible.
| colordrops wrote:
| All fair points, i was a bit hasty with my comment.
|
| Do you mind sharing your configs for gnome keyring with
| sway? I'm on sway and would love to use that.
| aseipp wrote:
| Ehh, the reality is not quite that rose-tinted; in practice
| you have to put the elbow grease in, still. It's normally not
| quite as simple as porting someones config IME, because often
| it's wrapped up with other things they need. I've been a
| NixOS user and committer for 10 years. Things do in fact
| change and sometimes you have to learn a new thing, often.
| I'm dating myself here but I still remember when the Great
| Haskell Package overhaul happened, in an attempt to start
| vendoring Haskell packages directly from upstream Hackage;
| before that there was totally different infrastructure and
| every package was added manually. That broke a lot of things.
|
| Similarly, I only started using Flakes last year after years
| and years of development. Rust packaging is still pretty bad
| and non-granular, unless you use cargo-based generators which
| all have issues, and none are upstream. If you are using even
| more marginal languages, your experiences will probably range
| from bad to worse. NixOS is constant evolving and things do
| get deprecated and changed. You do have to learn new things.
|
| What you _don 't_ have to deal with as often is just, things
| randomly breaking. And with Flakes you don't have to deal
| with ambient Nixpkgs changes. I have shell scripts using nix-
| shell I haven't had to touch in years, despite incompatible
| changes in the tools used by the script. I still have a
| walled off NixOS machine running a 15.03pre-release, IIRC. I
| could probably upgrade it but the software on it isn't going
| to break. I suspect the disk will fail first (which is fine
| for this machine.)
|
| I think the best analogy is this: tools like Nix, Guix etc
| let you move mountains as an individual. That's impressive.
| It still requires you to get your hands dirty and actually
| push the thing, which is still work.
| fbdab103 wrote:
| Cannot vouch for it, but the nixified.ai project is precisely
| geared towards this idea. Having Nix builds to run machine
| learning environments[0]
|
| [0] https://nixified.ai/
| imtringued wrote:
| No mention what ACO is.
|
| The benchmark is just a dump of the console output with no
| explanation what the individual phases mean.
|
| If I am going to be honest, this is a terrible blog post.
| sva_ wrote:
| AMD COmpiler
| mathisfun123 wrote:
| > No mention what ACO is.
|
| Griping about initialisms has gotta be hn's favorite pass-time
| _by far_. It 's as if we _don 't_ live in a world where things
| can be looked up with the flick of wrist. It's as if _we 're
| not_ discussing this on a website entirely premised on making
| information access/retrieval more efficient.
|
| > Mesa ACO compiler
|
| More than enough disambiguation that the first hit tells me
| _exactly_ what it is.
|
| > If I am going to be honest, this is a terrible blog post.
|
| Actually, I was wrong: cheap disparagement is hn's favorite
| pass-time.
| brucethemoose2 wrote:
| I have griped about this before, but I don't really get the hype
| around tinygrad... GitHub is littered with ML frameworks, some of
| them (Like MLIR/TVM) with very good working demos right now, and
| other smaller scale ones (like GGML) are SOTA in very specific
| niches.
| pseg134 wrote:
| And did you gain any knowledge the last time you "griped" about
| it? If not then who are these "gripe" sessions even serving?
| brucethemoose2 wrote:
| > And did you gain any knowledge
|
| ...No.
|
| I am not against tinygrad or anything, but I can't find any
| projects using it as a backend. And I can't find benchmarks
| for, say, the stable diffusion or llama demos.
|
| But I _am_ frustrated that it seems to be attracting lots of
| attention on social media, while other ML frameworks
| /projects targeting wide hardware support, and already being
| used as backends in places, are flying under the radar.
|
| And Hotz's very brief spat with AMD that I saw on his YouTube
| channel also raised my eyebrows.
| vegarab wrote:
| comma.ai uses Tinygrad as its backend
| tinco wrote:
| What was his spat with AMD about? I think he complained
| about AMD's drivers being shit on the Lex Fridman podcast
| but that's not eye brow raising right?
| brucethemoose2 wrote:
| Because of unfixed rocm bugs, Hotz stated that Tinygrad
| was going to abandon AMD support on GitHub, and was very
| upset about it in the following YouTube video.
|
| This is when Lisa Su apparently stepped in and patched
| things up.
| xiphias2 wrote:
| Maybe you should look at the code base before throwing a
| tantrum on a hacking blog.
|
| Try adding a new feature to Tensorflow / PyTorch, do the
| same for TinyGrad and you won't have this question anymore.
| sakras wrote:
| Just curious, have you tried adding a feature to
| TinyGrad? Or are you arguing in the abstract because
| TinyGrad is small?
|
| Because I personally have tried, and found the code to be
| so code-golf and spaghetti-like that I gave up, it just
| wasn't worth my time (even for the $1000 bounty).
| xiphias2 wrote:
| I did small changes, took part of the Twitch sessions and
| thinking and understanding it. I contributed the main
| ideas for Kernel search, and was happy to get George out
| of a mini depression with some quick fun wins. He almost
| gave it up, but now the progress every day is crazy.
|
| The way to implement anything new and cool is to read it
| all first.
|
| I love the way convolutions and matmuls are both
| expressed using ShapeTracker and simple mul operation for
| example. No other framework has that elegance.
|
| Until you understand that part, it's not worth trying to
| change it.
| voz_ wrote:
| Pytorch is pretty easy to add stuff to.
| steeve wrote:
| MLIR is really cool imho
| dist1ll wrote:
| Calling a compiler infrastructure project like MLIR an "ML
| framework" feels a bit disparaging.
| brucethemoose2 wrote:
| No, but its an increasingly popular middle layer for other
| frameworks.
| mellosouls wrote:
| _I don 't really get the hype around tinygrad_
|
| geohot
|
| whether that validates it or not, I think thats the reason.
| WinLychee wrote:
| It's super easy to install and use, and you can comprehend it
| in its entirety _because_ it's such a tiny project. Pop open
| the source code and browse around! The genius is the
| simplicity.
___________________________________________________________________
(page generated 2023-07-15 23:02 UTC)