[HN Gopher] Show HN: Chili. Rust port of Spice, a low-overhead p...
___________________________________________________________________
Show HN: Chili. Rust port of Spice, a low-overhead parallelization
library
Author : dragostis
Score : 186 points
Date : 2024-09-19 13:04 UTC (9 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| n8henrie wrote:
| Not related to the content, but as a port of "spice" -- being
| from New Mexico, we would usually refer to a spicy pepper as (eg
| green) "chile," or a chunky dish with beans and meat as (eg
| Texas) "chili." Looks like the AP Stylebook agrees, though
| Webster's makes it seem less clear, so you can probably get away
| with it either way.
| skipkey wrote:
| If it has beans it isn't Texas chili...
| andyferris wrote:
| Further abroad (Australia), Chile is a country and chili is the
| hot pepper, so I suspect the (English) language here is highly
| regional.
|
| Eg I've even lived in North American and never come across that
| spelling for the pepper.
| anamexis wrote:
| The "chile" spelling is from Spanish, particularly Mexican
| and Central American Spanish. The word is derived from
| Nahuatl (Aztec).
|
| That said, I'm in northern US and I would spell the pepper
| "chili"
| ithkuil wrote:
| And the name of the country Chile has no relation to the
| nahuatl word for pepper but comes from a word in a south
| American language (likely Aymara)
| darby_nine wrote:
| Oddly I associate `chile` with the pepper and `chili` with
| the stew. Curiously chili pepper seems to be the common
| spelling around where I live so I have no clue where I
| picked this up.
| saghm wrote:
| Yeah, I've lived in the northeast US my whole life and have
| never seen "Chile" used for anything but the country before;
| the pepper and the food are both spelled "chili" in my
| experience. That being said, pronouncing the country's name
| as a homophone of "chili" is a bit of an anglicism too, since
| from what I remember of Spanish in high school, the more
| accurate pronunciation would be something like "chee-lay"
| rather than "chill-ee". I wouldn't be shocked if both the
| spelling "chili" and the pronunciation of it had roots in the
| name of the country and in English usage drifted over time
| (sort of like the pronunciation of the name of the city
| "Amarillo" in Texas).
| aidenn0 wrote:
| Wikipedia says "Chili peppers, also spelled chile or chilli"
| ralegh wrote:
| We call them chilli in the Uk
| twic wrote:
| True: https://www.rhs.org.uk/vegetables/chilli-pepper/grow-
| your-ow...
| hombre_fatal wrote:
| https://en.wikipedia.org/wiki/Chili_pepper
| beepbooptheory wrote:
| Was just arguing with a friend the other day about distinction
| between a "chili" and what New Mexican's have in "green chile
| stew." I have had what by at least strong family-resemblance is
| a chili but had essentially an adovada meat base, but would
| never call something chili that had potatoes or any visible
| vegetables. But in NM, even frito pies get the (superior)
| "green chile stew" topping.
| thayne wrote:
| > or a chunky dish with beans and meat as (eg Texas) "chili."
|
| My understanding was that the name of the dish was short for
| "chili con carne" i.e. chili (the pepper) with meat
| dangoodmanUT wrote:
| I love this, the overhead of rayon always made it hard to gauge
| whether to use it for small operations (or could be small
| operations). This seems to solve that, at least for relatively
| short workloads
| bee_rider wrote:
| Out of curiosity, how do Spice/Chili/Rayon compare against the
| boring answer, OpenMP?
|
| I guess they must have some really neat capabilities.
| nestorD wrote:
| Rayon is known to have a higher overhead for the simple, and
| arguably most common case in numerical codes, static block size
| scheduling. Things get better, and you start to benefit from
| rayon, as work gets less balanced.
| thayne wrote:
| Well, for one thing OpenMP is a c/c++ library, and chili and
| rayon are rust libraries.
|
| It's probably possible to use openmp from rust, but the
| interface probably isn't as nice, you would need some kind of
| translation layer from rust closures and iterators to the
| OpenMP API
| bee_rider wrote:
| Plus Fortran!
|
| There'd definitely be some work translating Rust concepts
| over. It might be impossible/not worth it. But maybe it would
| be nice to have Rust, C, and Fortran all talking to the same
| runtime?
| marmaduke wrote:
| That's what I was wondering. Iiuc openmp uses work stealing and
| the spice readme talks about the inefficiency of that:
|
| https://github.com/judofyr/spice?tab=readme-ov-file#work-ste...
|
| so I'd be interested in seeing a benchmark.
| marmaduke wrote:
| Reading the implementation details kinda got me wanting to do
| a c11 threads based port.
| bee_rider wrote:
| Thanks. That's some pretty clever stuff, with a nice
| intuitive description.
| zbowling wrote:
| > a "low-overhead" parallelization library
|
| Who is out here building high-overhead parallelization libraries?
| internetter wrote:
| Relatively, Rayon is higher overhead
| zamalek wrote:
| I have almost certainly built high overhead parallelization
| primitives during my junior years.
| andrepd wrote:
| Yeah, why are those bastards writing slow code? They should
| just write fast code instead
| bravetraveler wrote:
| Check out my high-overhead replacement for _' cat'_ in BASH
| ~ $ type cat cat is a function cat ()
| { while read; do printf '%s\n'
| "$REPLY"; done < "$@" }
|
| It probably butchers things. Why? I got bored in a meeting and
| someone accidentally posted _' cat'_ to Teams
| gfs wrote:
| I would love to see a blog post or more detail about how this
| implementation works and why it can beat rayon in some cases.
| judofyr wrote:
| The README of Spice has detailed explanation of the approach:
| https://github.com/judofyr/spice
| jnordwick wrote:
| The definitive source on how to spell "chili":
|
| https://redhotchilipeppers.com/
| jmakov wrote:
| Is there anything like ray.io in rust? Nobody running
| computations on clusters in rust?
| aidanhs wrote:
| The company I work at (Hadean) used to have this as a product -
| think erlang-like multi machine IPC, with automatic acquisition
| of cloud resources and language integration for Rust, C, C++,
| Python. Pretty easy to point it at some machines and get them
| running a distributed application (as in simulation or big
| data).
|
| But infrastructure for developers is hard to make money with -
| developers like to build it themselves and people holding the
| purse strings point at kubernetes and say "that's free". So we
| just use it as an internal platform for a distributed
| simulation engine and it works pretty well.
|
| I did an analysis of removing it (it's a lot of bespoke code
| that we have to maintain for something that isn't our actual
| product) and I think you could probably implement something on
| top of Nomad that's close enough...but then Nomad went BSL and
| Kubernetes is a big complexity shift.
|
| So...if anyone knows of something out there let me know, I'd
| love to be able to use it outside of work :)
| jmakov wrote:
| ray.io seems to be doing pretty well financially...
| aidanhs wrote:
| Right, because Anyscale found a niche that distributed
| compute matters in (AI) and built great libraries/hosted
| platforms/services around that. I would venture that the
| money they make from people who pare back things to just
| ray core is ~0, which is why it's open source.
|
| Put another way - building such a platform doesn't preclude
| commercial success, but (at least for us) it isn't
| sufficient. Fly.io might be able to pull it off if they
| want to explore that direction imo.
|
| Fwiw if you dig around in the ray core codebase (as I did
| when I was doing competitor analysis years ago) you can use
| the core C code from other languages to build such a
| platform for Rust if you like - they had Java and C++
| interfaces at the time, but I haven't looked in the last 5
| years.
| aspensmonster wrote:
| The old EE in me got excited for a second there, thinking someone
| had made a Rust port of
| [SPICE](https://en.wikipedia.org/wiki/SPICE).
___________________________________________________________________
(page generated 2024-09-19 23:00 UTC)