[HN Gopher] Async Rust in Three Parts
___________________________________________________________________
Async Rust in Three Parts
Author : oconnor663
Score : 65 points
Date : 2024-10-23 14:23 UTC (8 hours ago)
(HTM) web link (jacko.io)
(TXT) w3m dump (jacko.io)
| Animats wrote:
| It's amusing that the Rust Playground lets you run a hundred
| threads. That's generous of them. There's a ceiling below 1000,
| though. The author points out, "On my Linux laptop I can spawn
| almost 19k threads before I hit this crash, but the Playground
| has tighter resource limits." Large servers can go higher than
| that.
|
| The thread-based I/O example with the compute bound poll loop is
| kind of strange.
|
| "Join" isn't really that useful when you have unrelated threads
| running finite tasks. Usually, you let the thread do its
| thing,finish, put results on a queue, and let the thread exit.
| Then it doesn't matter who finishes first. Rust join is actually
| optional. You don't have to join to close out a thread and reap
| the thread resources. It's not like zombies in Unix/Linux, where
| some resources are tied up until the parent process calls wait().
|
| Loops where you join all the threads that are supposedly finished
| are usually troublesome. If somebody gets stuck, the joiner
| stalls. Clean exits from programs with lots of channels are
| troublesome. Channels with multiple senders don't close until all
| senders exit, which can be hard to arrange when something detects
| an error.
|
| In Rust, the main thread is special. (I consider this
| unfortunate, but web people like it, because inside browsers, the
| main thread is very special.) If the main thread exits, all the
| other threads are silently killed.
| diath wrote:
| > Loops where you join all the threads that are supposedly
| finished are usually troublesome. If somebody gets stuck, the
| joiner stalls.
|
| That makes sense if the main thread is actually doing any
| useful work, but when its only job is to spawn threads and wait
| for them to finish before exiting, then it's a pretty common
| idiom.
| willglynn wrote:
| > In Rust, the main thread is special. (I consider this
| unfortunate, but web people like it, because inside browsers,
| the main thread is very special.) If the main thread exits, all
| the other threads are silently killed.
|
| Rust inherits this from `pthread_detach()`:
| The detached attribute merely determines the behavior of the
| system when the thread terminates; it does not prevent the
| thread from being terminated if the process
| terminates using exit(3) (or equivalently, if the
| main thread returns).
| wahern wrote:
| The main thread is special because that's how the runtime
| works on Unix. In particular, when "main" exits, the process
| exits. This is required by the C standard. It's also
| fundamentally built into how Unix processes work, as certain
| global variables, like argv and environ strings, typically
| are stored on the main thread's stack, so if the main thread
| is destroyed those references become invalid.
|
| In principle Rust could have defined its environment to not
| make the main thread special, but then it would need some
| additional runtime magic on Unix systems, including having
| the main thread poll for all other threads to exit, which in
| turn would require it to add a layer of indirection to the
| system's threading runtime (e.g. wrapping pthreads) to be
| able to track all threads.
| kelnos wrote:
| > _In principle Rust could have defined its environment to
| not make the main thread special..._
|
| Not to mention they'd have to be _very_ careful with what
| they do on the main thread (e.g. allocating memory via
| malloc() is out), since there are quite a few things that
| are not safe to do (like fork() that 's not immediately
| followed by exec()) in a multi-threaded program. So even a
| "single-threaded" Rust program would become multi-threaded,
| and assume all those problems.
| oconnor663 wrote:
| > Rust join is actually optional.
|
| I was recently surprised to learn that returning from main()
| with background threads still running is more or less UB in
| C++, because those threads can race against static destructors:
| https://www.reddit.com/r/cpp/comments/1fu0y6n/when_a_backgro...
| . C doesn't have this issue, though, as far as I know?
| dwattttt wrote:
| atexit enters the chat
| shepmaster wrote:
| > the Rust Playground lets you run a hundred threads
|
| It's more that we don't do anything to prevent it, other than
| coarse process-wide memory / CPU time limits. IIRC, Rust-
| spawned threads on Linux use 2MiB of stack space by default, so
| that seems like a likely cap.
|
| Note that the playground is only 2 cores and you are sharing
| with everyone else, so you aren't likely to really benefit.
| littlestymaar wrote:
| > Note that the playground is only 2 cores and you are
| sharing with everyone else
|
| This is amazing, I use it all the time with no performance
| issues so I expected it to be much beefier to support many
| simultaneous users.
|
| How many users does it serve? (Monthly or daily user and/or
| compilation job sent). And what tricks are used to keep it
| working? (I suspect it can re-use already compiled binaries
| of all supported dependencies and only need to compile the
| user's code and link it, but is there other clever
| strategies?)
| dwattttt wrote:
| > Loops where you join all the threads that are supposedly
| finished are usually troublesome. If somebody gets stuck, the
| joiner stalls. Clean exits from programs with lots of channels
| are troublesome. Channels with multiple senders don't close
| until all senders exit, which can be hard to arrange when
| something detects an error.
|
| I wish join-with-timeout was a more common/supported operation.
| alilleybrinker wrote:
| In part two, the author explains trait objects in a way that is,
| I think, a little misleading.
|
| They're right that trait objects are dynamically sized types,
| which means they can't be passed by value to functions, but wrong
| that they need to be boxed; they can instead be put behind a
| reference. Both of the following are valid types.
| type DynFutureBox = Pin<Box<dyn Future<Output = ()>>>; type
| DynFutureRef<'f> = Pin<&'f dyn Future<Output = ()>>;
|
| You can see this in the Rust Playground here: https://play.rust-
| lang.org/?version=stable&mode=debug&editio...
| LoganDark wrote:
| Technically trait objects aren't entirely a thing at all.
| They're a concept that only makes sense in the concept of a
| pointer (references are safe pointers, `Box`es are smart
| pointers). You can refer to something as a trait object but the
| trait of the object is a property of the pointer and not the
| object. So if you have some struct that implements a trait you
| can cast a pointer to that struct to a pointer to a trait
| object, but that struct never stops being the struct, a trait
| object is just a different way of referring to the struct.
___________________________________________________________________
(page generated 2024-10-23 23:00 UTC)