[HN Gopher] JEP proposed to target JDK 19: 425: Virtual Threads ...
___________________________________________________________________
JEP proposed to target JDK 19: 425: Virtual Threads (Preview)
Author : SemanticStrengh
Score : 197 points
Date : 2022-05-02 15:58 UTC (7 hours ago)
(HTM) web link (mail.openjdk.java.net)
(TXT) w3m dump (mail.openjdk.java.net)
| mike_hearn wrote:
| What this means is: Loom ships in preview form in September, and
| if all goes well will probably graduate to a fully supported
| feature a year later. So it'll be September 2023 at the earliest
| before this can used. During that period you have to pass extra
| command line arguments to opt-in.
|
| The good news is that long term supported releases got more
| frequent in recent times and the next one is due in ... September
| 2023. So there hopefully won't be any delays for people who have
| to wait for an LTS release.
| pron wrote:
| Preview features [1] are fully supported -- and are part of the
| official Java SE specification -- they're just not finalised or
| "permanent" (and are, therefore, disabled by default). I.e. the
| API might experience some changes due to feedback, so those who
| wish to use it acknowledge, by means of the --enable-preview
| flag, that they accept the possibility of the API changing.
| However, an API is marked as Preview (as opposed to
| incubator[2]) only when it is close to being finalised, meaning
| that we believe it can be finalised in two releases (one year).
|
| [1]: https://openjdk.java.net/jeps/12
|
| [2]: https://openjdk.java.net/jeps/11
| dang wrote:
| Recent and related:
|
| _Achieving 5M persistent connections with Project Loom virtual
| threads_ - https://news.ycombinator.com/item?id=31214253 - April
| 2022 (142 comments)
| sydthrowaway wrote:
| Isnt this the Google Balloon
| sidpatil wrote:
| No, that's Loon.
|
| https://x.company/projects/loon/
| djanogo wrote:
| The world biggest deployment of Java API is android, I wish
| Google had just bought out Java instead of copying it's API's,
| fighting it in court, and now using kotlin (with it's _IMO_ ugly
| syntactic sugar) to avoid legal issues.
| SemanticStrengh wrote:
| It's only a matter of time before the amount of bullshit at
| Google reach a too strong threshold and are obligated to switch
| to full openjdk vs their abandonware. Unfortunately they are so
| mediocre they have made an incompatible bytecode so they will
| need some patches/translation layers atop jdk. BTW they are
| claiming JDK 11 support for Android 13. Of course this is a lie
| that only apply to the stdlib and not runtime support such as
| e.g constant dynamics.
| pron wrote:
| > The world biggest deployment of Java API is android
|
| Not even close. The entire mobile space -- Android and iOS
| combined -- is drastically smaller than just Java alone (not
| counting Android). You can confirm this by going to job
| websites that allow you to analyse their postings. Java,
| together with its sister leading languages, JavaScript and
| Python, is so popular that it's in a completely different
| ballgame than most languages.
| uluyol wrote:
| Why would Oracle sell Java to Google? I'm pretty sure that a
| main attraction for them in buying it was the possibility of
| collecting $$$ from Google.
| vips7L wrote:
| I believe OP means that Google had the opportunity to buy Sun
| but didn't.
| twic wrote:
| One question: virtual threads are lightweight, but are the
| executors which create them?
|
| That is, should i organise my code so i create a single executor
| and use it across many operations, or is it okay to create and
| destroy executors all over the place?
|
| And are the executors threadsafe - both for use from virtual
| threads, and from native ones?
|
| As an aside, these are things which is important to know, but
| which are very rarely documented. We had some headaches a while
| ago because we wrote code which treated the new HttpClient as
| lightweight when it is very much not.
| kjeetgill wrote:
| The heaviest part of executors are the task queues and the
| threads themselves. If you're using 2 or 3 executors over a
| single one, you're presumably using the same amount of space in
| the queue and number of threads, so the "weight" doesn't change
| _except_ that the _queues are concurrent data structures_ so
| can be sources of contention ... in really odd hyperactive
| scenarios.
|
| Executors tend to be units of management (await()/shutdown())
| etc. and monitoring, so that'll be your main reason to use more
| than just one for everything.
| Traubenfuchs wrote:
| Can you interpret this as a global infinitely scaling thread
| pool?
|
| Or as syntactic sugar for interacting with such a threadpool?
|
| I guess I still haven't understood what this gives you as opposed
| to submitting lambdas to a threadpool. Besides maybe more usefull
| threadlocals.
| RcouF1uZ4gsC wrote:
| Although I am not a Java user, this is great news. Asynchronous
| code, even with a await, adds a lot of complexity. It leads to
| the "color" problem of functions, as well as increased complexity
| in languages without garbage collection (my understanding is that
| async is what drove a lot of the requirement for Pin in Rust.
|
| There are also advantages in debug ability and performance
| tracing for threads.
|
| It would be nice to have (virtual) threads that are lightweight
| and efficient enough that we did not need async for writing
| highly concurrent software.
|
| I wonder if 20 years from now, programmers will look at async,
| kind of like current programmers look at segmented memory or
| manual register allocation --- something that was necessary for
| performance in a bygone era, but is now not needed.
| pjmlp wrote:
| Lets not forget that Modula-2 already had co-routines, Ada
| tasks, Active Oberon active objects, Concurrent Pascal, and
| plenty of other examples.
|
| If anything, this is yet another example of the decades how old
| concepts still take to become widespread across all major
| mainstream languages.
| olliej wrote:
| Are these like "fibers" or whatever that weird not-a-full thread
| thing on windows?
|
| I genuinely don't know as I have never actually seen them being
| used, but they always seem to be described as a "lightweight"
| thread
| exabrial wrote:
| correct. These are "threads" that are managed by the JVM, and
| "context switches" do not involve a syscall.
| theptip wrote:
| I found the JEP to be very well-written. A good summarizing
| paragraph:
|
| > Virtual threads are a lightweight implementation of threads
| that is provided by the JDK rather than the OS. They are a form
| of user-mode threads, which have been successful in other
| multithreaded languages (e.g., goroutines in Go and processes in
| Erlang). User-mode threads even featured as so-called "green
| threads" in early versions of Java, when OS threads were not yet
| mature and widespread. However, Java's green threads all shared
| one OS thread (M:1 scheduling) and were eventually outperformed
| by platform threads, implemented as wrappers for OS threads (1:1
| scheduling). Virtual threads employ M:N scheduling, where a large
| number (M) of virtual threads is scheduled to run on a smaller
| number (N) of OS threads
| binarynate wrote:
| With Loom, what's the equivalent of the following Promise
| concepts?
|
| 1. Promise constructor (to turn an existing callback-based or
| async operation into one that can be blocked with Loom). JS
| example: await new Promise(resolve =>
| someApiThatTakesACallback(resolve));
|
| 2. Methods like Promise.all() and Promise.any() to run multiple
| async operations concurrently. JS example: const
| [result1, result2] = await Promise.all([doSomething1(),
| doSomething2()]);
| twic wrote:
| Here are my guesses.
|
| 1. Same as with normal threads:
| CompletableFuture<Foo> future = new CompletableFuture<>();
| someApiThatTakesACallback(future::complete); Foo result =
| future.get(); // this can take a timeout
|
| 2. If you mean that the operations are blocking and should be
| called in parallel using virtual threads, the equivalent of
| Promise.any:: ExecutorService executor =
| Executors.newVirtualThreadPerTaskExecutor(); Future<Foo>
| future1 = executor.submit(this::doSomething1);
| Future<Bar> future2 = executor.submit(this::doSomething2);
| Foo result1 = future1.get(); Bar result2 = future2.get();
|
| There is a version of that using ExecutorService::invokeAll,
| but i'm not sure it's any more concise, because you still need
| to unpack the futures one by one.
|
| For Promise.any: ExecutorService executor =
| Executors.newVirtualThreadPerTaskExecutor(); Foo result =
| executor.invokeAny(List.of(this::doSomething1,
| this::doSomething2));
|
| The asymmetry between invokeAny and invokeAll here is slightly
| grating, but makes sense.
|
| Note that in both cases, you probably already have an executor
| lying around that you can use for this.
| binarynate wrote:
| Thanks! Your examples are super helpful.
| christophilus wrote:
| My guess for the latter is something like goroutines and
| channels, though I'm not sure. So: spin up a thread per
| "doSomething" and then join those in your main thread.
| teh64 wrote:
| For the first one, I believe this is the correct method [2],
| which takes a Callable (the other submit methods take a
| Runnable, which does not seem to be generic).
|
| For the second one, you can use an implementation of
| ExecutorService[1], which has the methods invokeAll and
| invokeAny.
|
| Futures in Java are similar to Promises in Javascript, in that
| they represent a calculation that might not be completed yet.
|
| 1.
| https://docs.oracle.com/en/java/javase/17/docs/api/java.base...
| 2.
| https://docs.oracle.com/en/java/javase/17/docs/api/java.base...
| ackfoobar wrote:
| The parent comment meant `callbackTakingFunction: ((Res,
| Throwable) -> ()) -> ()`, not `blockingFunction: () -> Res`
|
| `ExecutorService#submit` works with the latter.
| binarynate wrote:
| Thank you!
| ackfoobar wrote:
| 1. Callback interop
|
| I was expecting something like call/cc, or
| `suspend(Cancellable)Coroutine` in Kotlin. But judging from the
| list of preview items[1] it seems the only way is to have the
| callback write the result to a Future and blocking wait on it.
|
| 2.1 Promise.all() can be done with `.map(future =>
| future.get())` But it is a bit more complicated than that. [2]
|
| 2.2 Promise.any()
|
| This is somewhat relevant
| http://mail.openjdk.java.net/pipermail/loom-dev/2020-Februar...
|
| [1]
| https://download.java.net/java/early_access/loom/docs/api/pr...
|
| [2] https://kotlin.github.io/kotlinx.coroutines/kotlinx-
| coroutin... "This function is not equivalent to deferreds.map {
| it.await() } which fails only when it sequentially gets to wait
| for the failing deferred, while this awaitAll fails immediately
| as soon as any of the deferreds fail."
| binarynate wrote:
| Thanks!
| nimish wrote:
| Congrats! Loom is excellent.
| [deleted]
| abledon wrote:
| If only Java had a slick CLI tool like "dotnet" for C#. man that
| thing can do so much, so easily
| SemanticStrengh wrote:
| You mean JShell? (yes the naming is perfect)
| abledon wrote:
| that seems to be a REPL. dotnet cli is more broad, great
| developer ergonomics imo. terse/concise design
| https://docs.microsoft.com/en-us/dotnet/core/tools/
|
| e.g. to make a new console app.... ``` c#
| dotnet new console java go install extra
| maven program (if your team uses maven)... mvn
| archetype:generate
| -DarchetypeGroupId=org.apache.maven.archetypes
| -DarchetypeArtifactId=maven-archetype-quickstart
| -DarchetypeVersion=1.4
|
| ```
| twic wrote:
| No, the dotnet tool is more like Gradle or Cargo. You say
| "dotnet new" to set up a new project, "dotnet build" to build
| it, "dotnet nuget push" to release to an artifact repository,
| etc.
|
| The JDK doesn't have this because many of the concerns
| involved are outside its remit. Java does not define a
| project layout, a build process, or a way to acquire or
| distribute libraries.
|
| But the Java world does have Gradle and Maven, which are, for
| better or for worse, the options to satisfy abledon's desire.
| SemanticStrengh wrote:
| There is also Jlink and yeah I get your point, Gradle has
| moderate complexity and is extremely featureful.
| bestinterest wrote:
| Java will join the (imo) better concurrency path of higher level
| languages with green like threads, Golang (goroutine), Erlang,
| Java (Loom, virtual threads) & Ruby (soon under Ractors?).
|
| I really think C# & Python will be jealous of the languages
| mentioned and puts them in an odd spot from language design
| perspective.
| nu11ptr wrote:
| As much as I love Rust for serious code and Python for
| scripting, I do much prefer the green thread model to
| async/await in general.
| kmac_ wrote:
| Green threads and async/await are orthogonal concepts. Both
| can be used at the same time and separately. Jep 425 brigns
| VM threads, but the API remains same as the old one. Without
| deep language extensions any concurrency improvements, in
| terms of usage, aren't possible. C#, Rust, JS made the
| successful leap, and imo Go route is very interesting.
| abledon wrote:
| for anyone interested in Rust's choice to not use the green
| thread model:
|
| https://www.reddit.com/r/rust/comments/7x0icm/regarding_gree.
| ..
|
| https://github.com/rust-
| lang/rfcs/blob/master/text/0230-remo...
|
| > Initially, Rust supported only the green threading model.
| Later, native threading was added and ultimately became the
| default.
| nu11ptr wrote:
| I get why Rust opted for native threads vs. green threads
| as a default, esp. when they want to promote zero cost
| abstractions. What the above links don't cover, and what
| I'm interested in, is why async/await instead of M:N/green
| threads for the secondary "light weight" threading model.
| Was async/await easier to make "zero cost"? Or what was the
| reason?
| amaranth wrote:
| In theory async/await can be compiled to something as
| efficient as if you wrote all the non-blocking logic by
| hand so yeah, it's a "zero cost" abstraction.
|
| For green threads the stdlib would have to be designed
| around them (again) or they'd feel like something bolted
| on the side with two sets of APIs for everything. This is
| also the case for async/await but that means green
| threads don't have an advantage there. If you have to
| worry about Task vs Thread, make sure you all the right
| APIs from each, figure out what to do for TLS in Tasks,
| etc then you may as well just use the one that is more
| efficient.
| fabioz wrote:
| Well,
|
| https://github.com/python-greenlet/greenlet
|
| has been available for quite some time in Python (gevent
| probably being its most used flavor).
|
| i.e.: just because it's not in the standard library doesn't
| mean it's not available (so, you can actually choose whether
| you'd like to use async/await or greenlets).
| azinman2 wrote:
| Because it's not in the standard library means there are
| various issues with compatibility. Everything needs to be
| designed/used for greenlets in mind. It's not quite the same
| as the VM itself saying we've sorted it all out.
| fabioz wrote:
| Well, whenever you add threads to the mix (native or
| virtual threads) if you don't design for it, I don't think
| things pan out properly (but I guess that I agree that if
| someone designed with native threads going for virtual
| threads where the same APIs can now make IO async switch
| threads it's probably easier to go that route).
|
| I must say that the main usage I personally had of
| greenlets didn't have it in mind initially (it was used in
| an existing application where making the coloring wasn't
| really feasible as it was a huge app already and adopting
| green threads was _much_ less work).
| criddell wrote:
| If it's a better model, why wouldn't C# and Python adopt it?
| jayd16 wrote:
| Yeah I really don't see an issue with C# building a virtual
| thread system if the paradigm really shifts that way. Its
| aggressively pragmatic in that way.
|
| In theory, such a system could be implemented on C# with the
| same sort of gotchas as Java virtual threads. I don't think
| there's a fundamental design conflict.
| SemanticStrengh wrote:
| Kotlin state of the art coroutines should be mentioned too
| deepsun wrote:
| Isn't Kotlin coroutines have the same coloring problem
| (functions must be marked with "suspend").l?
|
| Although the coloring problem is only a problem if red
| functions are harder to use. Not sure marking all functions
| with "suspend" would make it any worse, besides infecting the
| codebase.
| SemanticStrengh wrote:
| The main coloring problem beyond async is the wrapping of
| return types in futures, which Kotlin groundbreakingly make
| transparent.
| ackfoobar wrote:
| Here the Kotlin lead argued that it is not a "problem".
|
| https://elizarov.medium.com/how-do-you-color-your-
| functions-...
| vips7L wrote:
| I don't really see how that's not a problem. While not
| having to specifically await you still need to mark
| functions as suspend and only call suspending functions
| from other suspending functions.
|
| I am also not sure how this explicit marking would work
| with interfaces. Can you create an interface that can be
| implemented by both suspending and synchronous functions?
| SemanticStrengh wrote:
| > Can you create an interface that can be implemented by
| both suspending and synchronous functions? I don't
| remember.
|
| I do know you can at least specify suspend in an
| interface method to enforce its implementers to be
| suspending too. Note that this is e.g impossible in
| typescript..
| ackfoobar wrote:
| > you still need to mark functions as suspend and only
| call suspending functions from other suspending
| functions.
|
| You restated the idea of function colouring without
| further elaborating why it is a problem.
|
| If your thread can afford to block and want to call a
| suspend function, use `runBlocking`. If your thread
| cannot block, having `suspend` in the type just saves you
| from a bug.
| SemanticStrengh wrote:
| runBlocking can't run regular functions right? Othwerwise
| what's preventing you from making the main() runblocking
| and have all functions under a context that allow both
| types?
|
| If runblocking can't allow non-suspending function then
| you got a problem because that mean a method implementing
| an interface can either be suspending or not and
| therefore for one of two implementation, fail.
| ackfoobar wrote:
| You can absolutely do this.
|
| `fun main() = runBlocking<Unit> {`
|
| https://kotlinlang.org/docs/composing-suspending-
| functions.h...
| SemanticStrengh wrote:
| Then you'll have to explain why that isn't the default
| main behavior then. Synchronous functions cannot call
| async functions (without defining N runblocking scopes)
| but a runblocking main would make kotlin seems color-less
| and should be seen as best practice. What's the catch?
| ackfoobar wrote:
| > would make kotlin seems color-less
|
| Quoting the post:
|
| > Having to mark asynchronous functions with suspend
| modifier is a small price to pay, but in return you get
| better insight into your code.
|
| Also you can do `suspend fun main()`
| vips7L wrote:
| > You restated the idea of function colouring without
| further elaborating why it is a problem
|
| Did you not read my second paragraph?
| ackfoobar wrote:
| > I am also not sure
|
| It felt like a question more than a complaint.
|
| > create an interface that can be implemented by both
| suspending and synchronous functions
|
| If interface has a suspending method, the implementations
| will also be suspending. But you can choose not to make
| any suspend calls in the method body. Not being able to
| say "this particular implementation of a red interface is
| blue" has never bothered me.
| pgh wrote:
| This is very reminiscent of the m:n thread support of Java on
| Solaris.
| https://docs.oracle.com/cd/E19620-01/805-4031/6j3qv1oej/inde...
| SemanticStrengh wrote:
| What is old is new
| reggieband wrote:
| > When code running in a virtual thread calls a blocking I/O
| operation in the java.* API, the runtime performs a non-blocking
| OS call and automatically suspends the virtual thread until it
| can be resumed later.
|
| How does it know that an operation is blocking on I/O? Is this
| limited to stuff baked into the language or standard library?
|
| What is the mechanism of suspension and resumption? They mention
| elsewhere that any platform thread could pick up any virtual
| thread so I assume they must be storing the stack somewhere. Is
| there a cost transferring stacks on virtual threads across
| platform threads? Does this introduce new security implications
| if there are less OS level restrictions on memory access between
| platform threads?
|
| What happens in cases where an application has user defined
| platform threads? How does the system determine what platform
| threads are available to the virtual threading system?
|
| I think this is probably the right decision for Java and I agree
| with all of their motivations. I personally like the explicitness
| of async/await, however I assume Java devs are very familiar with
| threading in general. I believe this allows an easier path to
| migrating existing Java code.
| kevincox wrote:
| > Is there a cost transferring stacks on virtual threads across
| platform threads?
|
| You shouldn't need to "transfer" the stack. There isn't really
| a "platform stack", that is just whatever your stack pointer is
| pointing at. So it is perfectly fine to allocate many stacks
| and switch between them within one OS thread just by changing
| the stack pointer.
|
| Of course if you are allocating a "full stack" then the main
| question is what is the point? IIUC the biggest cost of threads
| is the memory allocated to the stack. So unless you are doing
| something clever you don't get much benefit. There are many
| approaches here but I guess it is up to the runtime to pick
| one.
| jeremyjh wrote:
| > How does it know that an operation is blocking on I/O? Is
| this limited to stuff baked into the language or standard
| library?
|
| Most libraries use the I/O primitives from the platform
| standard libraries, so the behaviour is going to trickle out
| from there. If you aren't using those libraries, the only other
| way to do I/O would be to use native code such as via JNI, and
| the runtime would schedule that on a thread pool and so it
| would tie up an OS thread for the duration of a function
| invocation.
| zmmmmm wrote:
| thanks, this explains the mysterious magic that I think is
| probably bugging a lot of people.
|
| It will be interesting to see how much of a coloring problem
| this creates, if any. I guess the great thing about having it
| be standardised and baked into the language / VM is it will
| quickly become de facto best practice to make any such
| library code compliant with virtual threads. And for all its
| various downsides, the java ecosystem has always leaned away
| from native code and treated it as a last resort rather than
| leaning into it like Python etc. So likely it will not be a
| huge issue the way the coloring problem is in the async/await
| situation.
| nu11ptr wrote:
| From
| http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.h...:
|
| "Whereas the OS can support up to a few thousand active threads,
| the Java runtime can support millions of virtual threads. Every
| unit of concurrency in the application domain can be represented
| by its own thread, making programming concurrent applications
| easier. Forget about thread-pools, just spawn a new thread, one
| per task. You've already spawned a new virtual thread to handle
| an incoming HTTP request, but now, in the course of handling the
| request, you want to simultaneously query a database and issue
| outgoing requests to three other services? No problem -- spawn
| more threads. You need to wait for something to happen without
| wasting precious resources? Forget about callbacks or reactive
| stream chaining -- just block. Write straightforward, boring
| code. All the benefits threads give us -- control flow, exception
| context, debugging flow, profiling organization -- are preserved
| by virtual threads; only the runtime cost in footprint and
| performance is gone. There is no loss in flexibility compared to
| asynchronous programming because, as we'll see, we have not ceded
| fine-grained control over scheduling."
|
| This seems so obvious in hindsight as the "right way". I wonder
| why we went down that whole async/await craze with so many
| languages?
| pkulak wrote:
| Well, this kinda thing isn't free. You need a rather large and
| complicated runtime to make it work. Fine for the JVM, which is
| already a large and complicated runtime, but it turns out that
| making a runtime like this optional is not really possible. So
| languages that don't have a runtime are stuck with async/await,
| which can be zero-cost.
|
| And, to be fair, languages like Erlang and Go went this route
| LONG ago.
| [deleted]
| SemanticStrengh wrote:
| Except GraalVM bring support to almost any language
| Comevius wrote:
| Garbage collectors and preemptively scheduled virtual
| threads require a runtime. Even GraalVM Native Image has a
| large runtime for this reason. There is no free lunch here.
| pjmlp wrote:
| All high level languages have a runtime, the only
| difference among them is how big it is in practice.
|
| Even Assembly can be considered to have one, in case of
| microcoded CPUs.
| SemanticStrengh wrote:
| I'm just saying the cost of developing the runtime can
| now easily be mutualized between multiple languages
| thfuran wrote:
| That's some hefty sleight of hand.
| jpgvm wrote:
| It's much easier to implement async/await, especially in
| languages that don't have as pervasive a runtime as JVM and/or
| have substantial amounts of native code extensions that can't
| be managed, i.e Python/JS/Ruby.
| fabioz wrote:
| Well,
|
| https://github.com/python-greenlet/greenlet
|
| has been available for quite some time in Python (gevent
| probably being its most used flavor).
|
| Note that it it actually predates the async/await approach
| which was incorporated into the Python language (so, in
| Python it was implemented as a third-party library -- even
| async/await had an implementation based on Python 2 using
| yield and some decorators:
| https://pypi.org/project/trollius/).
| jpgvm wrote:
| Yeah I am familiar but using gevent w/monkey-patching is
| nowhere near the same experience as using Loom. Not to
| mention that if you patch threading to make it more gevent
| friendly you can also run into all sorts of fun with
| locking that wasn't designed for it etc.
|
| I really do think Python should have instead adopted gevent
| as it's async approach instead of asyncio and async/await
| etc.
| fabioz wrote:
| I must say that I definitely don't like the gevent
| monkey-patching myself -- I prefer to just use the
| different APIs, although yes, that can end up in a
| blocking call when there should be none, but that hasn't
| been a problem I've been bitten by so far -- although
| it's been a while since I had to use it too ;)
| nu11ptr wrote:
| Yeah I would imagine anything blocking would need special
| support, which is definitely a downside. Async/await is no
| different in that regard, but since explicitely marked, you
| know anything that isn't "awaited" and yet blocks is gonna
| break things, so might be easier to avoid mishaps in
| practice.
| merb wrote:
| because green threads and async/await have both upsides and
| downsides. if it would've been as easy as the comment would've
| described it, they would've shipped it in jdk 8, but they
| didn't. they ship a preview in jdk 19.
|
| rust's rfc explains some of the drawbacks:
| https://github.com/rust-lang/rfcs/blob/master/text/0230-remo...
| (which basically explains why a preview took until jdk 19)
|
| // Edit: also: void handle(Request request,
| Response response) { var url1 = ... var url2 =
| ... try (var executor =
| Executors.newVirtualThreadPerTaskExecutor()) { var
| future1 = executor.submit(() -> fetchURL(url1));
| var future2 = executor.submit(() -> fetchURL(url2));
| response.send(future1.get() + future2.get()); } catch
| (ExecutionException | InterruptedException e) {
| response.fail(e); } }
|
| is not better or worse than: async
| Task<IActionResult> Handle() { var url1 =
| ... var url2 = ... try {
| var requestTask1 = FetchURL(url1); var
| requestTask2 = FetchURL(url2); return Ok((await
| requestTask1) + (await requestTask2)); }
| catch (Exception ex) { return
| BadRequest(); } }
|
| of course one uses "colored" functions the other not, but both
| have one thing in common: a bad programmer can make serious
| mistakes with both.
| pron wrote:
| > is not better or worse than
|
| It is better for two reasons:
|
| 1. Virtual threads are threads as far as the Java platform is
| concerned, meaning you get the same troubleshooting (stack
| traces), debugging, and profiling support by the runtime and
| its tools as for platform (OS-backed) threads.
|
| 2. There is no split-world of APIs, with separate and largely
| incompatible flavours, that need to be developed for two
| constructs that are semantically (almost) equivalent.
|
| You are correct, however, that implementing user-mode threads
| is harder than implementing async/await. While the latter
| requires only changes to the frontend compiler (until you
| want to add tooling support), the former requires deep
| changes to the backend.
| merb wrote:
| > 1. Virtual threads are threads, meaning you get the same
| troubleshooting (stack traces), debugging, and profiling
| support by the runtime and its tools as for "platform"
| (i.e. OS-backed) threads.
|
| I never had any problem with debugging and profiling
| Futures/async/await. Neither in Rust/C#/Scala. And stack
| traces like Exceptions were never a problem, in fact go
| panics were way more troublesome... (and still are, most
| often you dont run into them...) Most of the time depending
| how "concurrent" your program is, it also makes just no
| sense to debug them anyway, if you deal with tons of
| concurrent code you should learn to make good log
| statements, it's bread and butter, especially once your in
| production.
|
| > 2. There is no split-world of APIs, with separate and
| largely incompatible flavours, that need to be developed
| for two constructs that are semantically (almost)
| equivalent.
|
| well yeah... but the question is why do you try to hide it?
| that programmers don't see that the code might run
| concurrent? most goroutine bugs I've seen happened because
| of the hidden concurrency. and it's odd that an important
| concept gets hidden, for the sake of split-world apis. heck
| thats exactly why there is a type system, to express the
| meaning of code. with green threads it's not really obvious
| on first sight what the code exactly expresses.
|
| (P.S.: I switched pretty early in my career to Scala, now
| to Rust/Golang/C# so maybe I'm more biased when it comes to
| a strong type system and I prefer a strong typesystem with
| all it's quirks, i've seen nasty bugs with goroutines and
| with async/await, so I doubt there is any real world
| benefit over the two worlds)
| pron wrote:
| > I never had any problem with debugging and profiling
| Futures/async/await.
|
| When you profile a server written in the asynchornous
| style (with, say, JFR), the server can be under heavy
| load and yet the profile will only show idle thread
| pools. I don't know what you mean by having no problem
| profiling, but the Java platform offers no mechanism to
| profile asynchronous code.
|
| > but the question is why do you try to hide it?
|
| We don't. Virtual threads are threads, and just like
| threads today Java or Scala or Rust, you need some
| explicit operation to perform a concurrent task on some
| other thread.
|
| > and I prefer a strong typesystem
|
| Great, but that has nothing to do with threads. Rust and
| Scala support threads, too, and do so without any
| indication in the subroutine's type for blocking
| operations.
| pie_flavor wrote:
| If you assume it's not concurrent when it is, that's the
| programmer's fault. This is true of both goroutines and
| threads in equal capacity; distinguishing between the two
| won't stop that from happening. If you assume it is
| concurrent when it isn't, nothing bad happens. The point
| of 'function color' is that hiding it implies it needs to
| be shown, which it doesn't. Rust hides from you which
| register a local variable is stored in, because it is
| thoroughly irrelevant to you and exposing it would
| require you to care when Rust is perfectly capable of
| managing it for you.
|
| There are legitimate reasons to prefer a colored approach
| to asynchrony. But the two points _are_ good points and
| you should prefer the colored approach only when said
| legitimate reasons outweigh those points given the
| language 's design.
| Thaxll wrote:
| Troubleshooting async is a pain in every language I
| worked with , that being node or Java ( vertx ). It's
| just terrible to get the real error, stack trace that
| don't even show the real issue, in Java I remember many
| times getting 150 lines of useles stacktraces, still
| figuring out from where the error was thrown.
|
| A panic in Go will always show you exactly where the
| problem is in the first lines of the panic.
| nu11ptr wrote:
| To be fair, the comment above is about how easy it is for the
| _user_. Having read docs on go's scheduler design I don't
| think many would argue it is easy to write such a runtime for
| the core devs.
| merb wrote:
| I added a snippt which also explains the user side, I doubt
| either has a benefit. both can be misued. I already seen
| bad go code because of this and I've also seen bad C# code.
| In fact go code is harder to reason because of this, since
| you do not know exactly when a function spawns a new
| goroutine.
| nu11ptr wrote:
| I don't think it is a huge difference, but I still prefer
| the straight threading model. It uses less keywords,
| looks like regular blocking code, and most importantly,
| there is exactly "one" world - not a sync vs. async world
| that have to coexist. I'm speaking generically, not
| specific to Java (which I have not written in many
| years).
|
| > In fact go code is harder to reason because of this,
| since you do not know exactly when a function spawns a
| new goroutine.
|
| This is no different in the async/await world, as you
| never know what function will launch a new "task" (this
| concept exists in Rust/Python and I assume others). If
| you don't need a new "thread", both models allow waiting
| for multiple things simultaneously.
| kodablah wrote:
| > I wonder why we went down that whole async/await craze with
| so many languages?
|
| For single-threaded event loops because most of those languages
| did not want to put the concept of thread safety onto the
| developer (e.g. JS and Python). And even the ones that do
| concern the user w/ thread safety don't have an intermediate
| representation and VM to automatically sequence instructions
| (e.g. Rust).
| sporkland wrote:
| Eric Brewer (CAP theorem) wrote about it in 2003:
| https://web.stanford.edu/class/cs240e/papers/threads-hotos-2...
| orasis wrote:
| async/await is FAR less error prone than explicit threaded
| concurrency.
| jayd16 wrote:
| Having more threads doesn't solve inter-thread concurrency like
| async/await does. With async/await you can achieve concurrency
| while ensuring there parallelism as well as is no parallelism
| (to avoid explicit critical sections).
|
| I don't think you could, for example, use this model of many
| virtual threads with a traditional GUI framework as designs are
| still predominately single threaded by nature.
|
| Said another way, cooperative scheduling has its place and will
| continue to.
|
| I'm excited to see some threaded UI designs shake out of this,
| though.
| kjeetgill wrote:
| Yea, it won't solve those things. I think the appeal for
| languages that already used locks/queues/atomics with their
| existing threads don't find this to be too, too disruptive.
| gorjusborg wrote:
| async/await doesn't solve anything with concurrency, working
| on a single thread does.
|
| you can do that as a design choice in languages that aren't
| exclusively single threaded.
| akvadrako wrote:
| That quote is wrong. Linux can support over a million threads.
| Of course they can't all be "active" since there isn't more
| than a few hundred hardware threads, but that's also true of
| green threads.
| chrisseaton wrote:
| > Linux can support over a million threads.
|
| How many servers have a spare half-terabyte of RAM to do that
| though?
|
| ((512 * 1024 * 1_000_000) / 1024 / 1024 / 1024 = ~500)
| CraigJPerry wrote:
| In large envs, 512gb ram servers were circa 2015. The
| servers being racked in those envs today can have > 10Tb
| and it would be unlikely to rack one with less than 2Tb,
| that would be a specialised node for some specific
| workload.
| chrisseaton wrote:
| If you have a 2 TB server, which most cloud instances for
| example are not, that's still 1/4 of your RAM... just for
| stacks. That can't be an efficient use of RAM.
| CraigJPerry wrote:
| Cells in hardware RAM are only used to hold actual data
| (and some are wasted to alignment). I may claim to have
| reserved 512gb of ram for stacks, but unless I'm actually
| using the space (in which case there's no issue) then
| physical ram is not being consumed.
| chrisseaton wrote:
| Doesn't Java pre-commit stacks?
|
| And even if it doesn't... those TLB entries are also not
| free.
| rossjudson wrote:
| Last time I looked at this in detail:
|
| The JVM commits a suitable number of pages for the stack.
| Stack growth can mean new page commits to expand the
| stack. After a thread is _idle_ for some defined period
| of time, the JVM _may_ release back _some_ of the pages
| used to expand the stack.
|
| If you actually want to take advantage of this behavior,
| you have to pay attention to which threads are being
| scheduled to do work -- because if you randomly choose a
| thread from a pool they're _all_ doing work, and the JVM
| thinks none of them are idle. You need to focus on
| particular threads.
|
| And at the end of the day if you're allocating a certain
| number of threads to do "task-like" work, you need to be
| prepared to deal with the expansion of thread
| stack/overhead.
|
| So overall...I like this new proposal. Building
| cooperative scheduling deep into the libraries makes
| sense, as does breaking the 1:1 ratio to OS threads.
|
| In other cooperative scheduling libraries I've seen, that
| 1:1 ratio is _retained_ , and this inevitably leads to a
| schism in programming: Those who believe that the
| "lightweight" thread truly is, and those who have learned
| otherwise.
| CraigJPerry wrote:
| As in write to each requested memory location to
| guarantee physical allocation? No, that would be really
| slow. It's not like heap memory where you probably want
| to memset it first, with stack you have a stack pointer
| so you know whats been written by you and what's random
| garbage.
|
| Java just requests the memory and linux just smirks and
| always says "yes sure, whatever you need" - it can do
| that even while the OOM killer is trying to decide which
| process looks like it has completed the most useful work
| so far :-)
|
| TLB entries dont really come into it (the size isnt an
| issue from our stack usage but even if it were, we could
| use larger pages).
|
| TLB flushes could be an issue with that many threads
| floating around but that's a different story and not
| really a memory usage so much as a performance and
| latency issue.
| chrisseaton wrote:
| > As in write to each requested memory location to
| guarantee physical allocation?
|
| No you use the advisory bits when you map the memory.
|
| It does this so it isn't unable to commit more memory
| while trying to handle an exception.
|
| > Java just requests the memory and linux just smirks and
| always says "yes sure, whatever you need"
|
| Linux doesn't ignore even LOCKED does it? And it
| certainly can't ignore actual writes to pages, which is
| what for example pretouch_memory does.
| CraigJPerry wrote:
| >> No you use the advisory bits when you map the memory.
|
| Is that really what's happening when we create another
| thread? That doesn't fit with my understanding here. When
| i create a new thread, i expect its stack to be allocated
| before the break so mlock wouldn't come into it. Even on
| a jvm with Xms smaller than Xmx (i.e. so the jvm could
| have to request to move the break and grow the heap to
| accomodate my new threads private jvm stack), I'm
| expecting you would have to explicitly request this
| behaviour somehow (because it's quite unneighbourly and
| probably counter productive for most apps). I don't know
| how you would do that, maybe there's a call available
| under the unsafe packages?
|
| I'm conscious I'm saying the jvm - i mean hotspot.
|
| >> so it isn't unable to commit more memory while trying
| to handle an exception
|
| Again this doesn't fit with my understanding - you can
| receive a further OOME while handling an exception and
| that's just bad luck.
|
| >> Linux doesn't ignore even LOCKED does it? And it
| certainly can't ignore actual writes to pages
|
| Sure but I'm going on the expectation that doesn't apply
| here
| chrisseaton wrote:
| > When i create a new thread, i expect its stack to be
| allocated before the break
|
| Why? Stacks are just memory like any other. You can put
| them wherever you want.
|
| > so the jvm could have to request to move the break and
| grow the heap
|
| The JVM allocates heap space using mmap, not by moving
| the break.
|
| > you can receive a further OOME while handling an
| exception and that's just bad luck
|
| If you're being reasonable you don't - the JVM maintains
| emergency heap storage (pre-committed!) space so it can
| keep allocating even when out-of-memory.
| CraigJPerry wrote:
| P.s is that you up ticking all my comments? Now i feel
| un-neighbourly not having done it back... fixing now
| akvadrako wrote:
| First of all, 512GB of RAM isn't that unusual. If you are
| supporting a million clients, most applications will need a
| beefy server. If only 0.1% of your clients are active,
| that's 1000 cores.
|
| Secondly, you don't need 512kb per thread. The kernel will
| only use pages that are actually used, so by default new
| threads will use 4kb.
| chrisseaton wrote:
| I _think_ , but can't prove right now, that Java pre-
| commits the stack so it has enough head-room to deal with
| exceptions. Maybe it shouldn't, but it currently does.
|
| And the point still-stands - hardware threads are heavier
| than virtual threads.
| ec109685 wrote:
| It allocates the entire stack, but it doesn't write into
| it, so it doesn't take up resident memory until used.
| _old_dude_ wrote:
| The problem is not Linux per se, it's the fact that
| pthread_create [1] reserves 2M for the stack, so the process
| runs out of memory if you have 1 million threads (1M * 2M ==
| 2T)
|
| [1] https://man7.org/linux/man-
| pages/man3/pthread_create.3.html
| akira2501 wrote:
| Well.. by default. You can set it to the minimum if you
| like, which is 16384 bytes.
| akvadrako wrote:
| It only reserves virtual memory.
| cogman10 wrote:
| Which has a real nasty habit of becoming real memory as
| applications process stuff.
| ec109685 wrote:
| There's OS level context switching as threads move on and off
| the CPU that you don't have to pay with the green threads
| approach.
| stonemetal12 wrote:
| >Every unit of concurrency in the application domain can be
| represented by its own thread, making programming concurrent
| applications easier.
|
| Funny, I have never heard anyone describe threads as the thing
| that made concurrent programming easier, or more threads would
| make it easier.
|
| What makes concurrent programming easier is thread interaction
| models that make it hard to lock the process.
|
| async\await, give you concurrency without the need to for the
| developer to explicitly write lock code. Much easier than
| having to manually handle that interaction.
| cogman10 wrote:
| > async\await, give you concurrency without the need to for
| the developer to explicitly write lock code.
|
| async\await in no way relieve the developer from needing to
| worry about locks. If you have shared mutable memory, you
| have locks. If you aren't considering that, then you've got
| broken code.
|
| > What makes concurrent programming easier is thread
| interaction models that make it hard to lock the process.
|
| Yes, the way you do that is by focusing on message passing
| and immutable data when you can get away with it and,
| ideally, prebuilt thread safe datastructures when you can't.
|
| What async\await buys is lightweight threading when
| lightweight threading isn't available. It allows you to have
| millions of concurrent processes running at the same time.
| HOWEVER, the cost of this is the "colored function" problem.
| You HAVE to mark up your code to let the compiler/framework
| know "this is code that can block and thus needs to be able
| to give up control and resume". This problem is difficult
| because you can't simply call an async function from a non-
| async function. You have to do some wrapping/juggling to make
| everything play nice.
|
| The reason lightweight threading is nice is because you no
| longer have the colored function problem. From any reference
| or context you can say
| `CompletableFuture.supplyAsync(()->calculateValue())` and
| have a new concurrent action spawned. If that action does IO
| or whatever, it doesn't hog a thread, it just yields (like
| async/await does) and lets another task move forward.
|
| The only reason for async/await is OS threads are expensive
| and have pretty large negative implications on operating
| systems.
| Nullabillity wrote:
| > async\await in no way relieve the developer from needing
| to worry about locks. If you have shared mutable memory,
| you have locks. If you aren't considering that, then you've
| got broken code.
|
| The huge difference is that futures (and thus async/await)
| change the primary concurrency concept you think about into
| "wait until this value is available" rather than managing
| the locks and shared memory yourself.
|
| The latter is still available (and still as footgunny as
| ever) but aren't as much of a problem when better options
| are more ergonomic.
|
| Futures also compose far better than threads do, especially
| when you want to wait for "any of X" rather than "all of
| X".
| cogman10 wrote:
| > The huge difference is that futures (and thus
| async/await) change the primary concurrency concept you
| think about into "wait until this value is available"
| rather than managing the locks and shared memory
| yourself.
|
| Java has had Futures for a while (and pretty good ones
| since Java 8's addition of "CompletableFuture")
|
| That being said, the biggest issue with Java Futures has
| not been the futures themselves, but rather the
| management of the ThreadPools for when you want to do IO
| in the futures.
|
| As pron points out, virtual threads make that WAY better
| to work with.
| pron wrote:
| I don't understand the distinction you make between
| futures and threads. Virtual threads make working with
| futures more pleasant.
| SemanticStrengh wrote:
| JS async is lock free though.
| ackfoobar wrote:
| You will need locks if you need mutual exclusion across
| an `await`.
| cogman10 wrote:
| JS has an uber lock. JS has a strict requirement that
| only one thread can run javascript code. That means that
| whenever you get to that `await` block, you've
| effectively established a whole application lock which
| prevents other awaits from proceeding (until your code
| yields). In otherwords JS isn't lock free, it's got 1
| really big lock that covers all user code.
|
| If you really wanted to, you could simulate this behavior
| in threaded languages by having a global lock you acquire
| before you run anything.
|
| It's the python GIL problem [1]
|
| Languages like kotlin, rust, C++, or C# with async/await
| MUST concern themselves with locks because they don't
| acquire whole application locks anytime they run a sliver
| of code.
|
| [1] https://realpython.com/python-gil/
| SemanticStrengh wrote:
| Great explanation in general except web workers are a
| thing and sharedarraybuffer allow shared memory between
| them. Thats why js has atomics. Although while js can
| have concurent races via shared memory, it doesn't expose
| locks IIRC? If so that's bad
| cogman10 wrote:
| Web workers are certainly weird... and it's generally
| best to ignore them :D
|
| But if you don't, then yeah, you sort of run right into
| the need for an actual lock since you can have shared
| memory in concurrently executing interpreters. Entire:
| Web Lock [1] Specifically designed to resolve this issue.
|
| In typical JS programming this is not an issue, but if
| you are trying to really abuse that javascript VM then
| this is the way you'd do it.
|
| [1] https://developer.mozilla.org/en-
| US/docs/Web/API/Web_Locks_A...
| SemanticStrengh wrote:
| Oh thank u web locks are a thing then. Well in all
| seriousness there are many valid uses for web workers,
| e.g. For offline processing of localstorage
| jayd16 wrote:
| >async\await in no way relieve the developer from needing
| to worry about locks. If you have shared mutable memory,
| you have locks.
|
| This isn't really true. For example, UI code often uses
| async/await for the concurrency but a single UI thread to
| prevent locking. Because this design is only concurrent and
| not parallel you don't need to worry about thread safety
| while still getting a way to yield execution to other code.
| noncoml wrote:
| > This seems so obvious in hindsight as the "right way". I
| wonder why we went down that whole async/await craze with so
| many languages?
|
| Hindsight? There are many of us that have been advocating
| against the async/await madness.
|
| I said it before and I will say it again. Async is today what
| OOP used to be in the nineties.
| kjeetgill wrote:
| OOP isn't that bad in moderation! My personal proprietary
| blend looks like 5%/20%/35%/40%
| declarative/functional/OOP/imperative these days. I'm
| definitely wary of letting too much functional _or_ OOP
| zealotry. I try to avoid all the monad /applicative/functor
| stuff _and_ too many abstract base classes, excessive
| interface indirection.
|
| But I am a Java pleb.
| zmmmmm wrote:
| OOP is bad in the way that many magnificently successful
| things are bad. People only work up the energy to fully
| critique things when they are so pervasive and ubiquitous.
| At which point they are entirely associated with their
| flaws and none of their positive attributes which are all
| treated as orthodoxy.
| pron wrote:
| > I wonder why we went down that whole async/await craze with
| so many languages?
|
| We addressed this, albeit very briefly, in the Alternatives
| section of the JEP: https://openjdk.java.net/jeps/425
|
| There are multiple reasons:
|
| 1. Languages that don't already have threads have an implicit
| assumption built into all existing code that program state
| cannot change concurrently. This is why scheduling points need
| to be marked explicitly, and why adding threads -- whether
| user-mode or OS -- might break existing code in some very
| tricky ways. That's the case of JavaScript.
|
| 2. Some languages target an IR without control over its
| backend, making an efficient implementation of user-mode
| threads difficult, if not impossible. async/await requires only
| changes to the frontend compiler. That's the case of Kotlin,
| and, perhaps to a lesser extent, Rust.
|
| 3. Some languages have technical features that make
| implementing user-mode threads efficiently more difficult than
| in others. Pointers into the stack and careful control over
| memory allocation make this more challenging in languages like
| C++ and Rust than in Java.
| SemanticStrengh wrote:
| A major issue with loom is that it consume much more %CPU
| https://github.com/ebarlas/project-loom-
| comparison/blob/main... Edit no it is more efficient although
| it consume surprisingly high gpus at higher throughput than
| the others.
| didibus wrote:
| Wouldn't that be expected when it also delivers more
| throughput and better latencies? It's handling more
| requests concurrently, so I'd expect the CPU usage to be
| higher, how else could it serve more requests faster
| otherwise?
| SemanticStrengh wrote:
| Yes indeed I just find the consumption increase a but
| abrupt after 10K
| brokencode wrote:
| Looking at the graphs, it uses less CPU for a given
| throughout, so it's actually more efficient for CPU. It
| also provides lower latency and higher max throughout. It
| does seem to require more memory, though.
| pron wrote:
| We expect to improve the memory consumption significantly
| in future releases. Some things had to be cut to make
| this release.
| SemanticStrengh wrote:
| The existing data already looks excellent. I wonder if
| you could leverage SIMD/the vector api for speeding up
| some things. Or if value types will have an impact.
| closeparen wrote:
| How does async/await mitigate #1? Interleaved execution is
| enough to give you data races; you don't need actual
| parallelism.
| pron wrote:
| Yes, but it requires a special call site (transitively, all
| the way up the stack) that permits the interleaving, and so
| cannot sneak into existing code that might implicitly
| assume no interleaving.
| signal11 wrote:
| Ron, thank you for your work on Loom, it's very exciting and
| I'm looking forward to using it in production code!
|
| As an aside, Ron had a JUG talk 6 months ago which I found
| really helpful where he went into more detail about why they
| chose this approach: https://youtu.be/KmMU5Y_r0Uk (27m20s
| mark, and from 2m50 there's a more general introduction to
| Loom).
|
| I'm sure there are other videos/papers as well, but this was
| a pretty good overview of Java vs other languages' approach
| to async.
| noncoml wrote:
| All these are implementation details.
|
| The programmers should be seen as "users" of the language.
|
| What you give here is a list of excuses on why system X
| doesn't do what is best for its users.
| vbezhenar wrote:
| As I grow older, implementation details are all that
| matters to me in the end. I hate go language, IMO it's ugly
| and terrible to work with. But its compiler and toolset are
| golden and I'll use it just because of its implementation
| details. I don't have time to wait until language
| developers will implement implementation details I need, if
| ever. I need to ship software tomorrow.
| rileyphone wrote:
| The compiler and toolset are the user facing aspects of a
| language like go, how the parser works or internal
| functions in the standard library would be the
| implementation details.
| noncoml wrote:
| Hmmm. Implementation details are the stuff that are not
| visible to you as a "user" of the language and the
| toolset.
|
| I don't know why people confuse this so much
| ecnahc515 wrote:
| Implementation details are often also defacto features,
| because the behavior may be relied upon by users. Unless
| you designed the language up front to consider these
| things, it's often very much a challenge to tell your users
| that their code is broken, especially if you did not have a
| language specification clarifying it.
|
| For languages like Python for example, this is a big issue,
| and reason why alternative concurrency patterns to
| async/await haven't made much progress.
| noncoml wrote:
| Totally agree. But that was not the point of GP
| kjeetgill wrote:
| Choices come with trade-offs and in turn make languages
| suitable for different use-cases and users. There's no
| uniform "best for it's users" choices that apply to all
| languages.
|
| Well, ironically, the exception that applies to all
| languages being: "does my code still work?" choices...
| which is what pron was addressing.
| noncoml wrote:
| I can buy the argument that async/await is a design
| choice. In which case this is what is "best if the user"
|
| But the GP replied to a comment that was claiming threads
| of execution are better than async. So in the context of
| the reply the threads are the "best for the the user"
| noncoml wrote:
| Damn folks. Downvoting is not meant for showing your
| disagreement
|
| Targeting a specific IR _is_ an implementation detail.
|
| Having explicit pointers and thread are not mutually
| exclusive concepts. It's the implementation details that
| make them difficult.
|
| HN can be so annoying sometimes
| yCombLinks wrote:
| It's not about disagreement with your opinion, it's about
| your rudeness presenting it.
| pie_flavor wrote:
| No, explicit pointers are not an implementation detail. Nor
| is knowing when you are on an OS thread in a language where
| you are likely to integrate with OS functions that depend
| on what thread you are on. Different languages exist for
| different purposes requiring them to solve problems in
| different ways.
| noncoml wrote:
| Two out of three arguments of the GP are implementation
| details. You can tell that they are implementation
| details by the way the language is used.
|
| "It's difficult to do A given B"
|
| As a _user_ I don't care about B. I just want A.
|
| For your example, the concept of explicit pointers are
| orthogonal to threads. That's why you can have OS threads
| with explicit pointers.
|
| Just because it's difficult to make them work together
| doesn't mean they are incompatible as concepts.
|
| It's still an implementation detail.
|
| For example, during Win95 one could argue it's impossible
| for a crash program not to crash the whole system.
|
| As a user I don't care what's going under the hood. I
| just don't like it when my Windows 95 app can crash the
| system. It _is_ an implementation detail
| kevincox wrote:
| I think you are missing a key benefit of async/await. It can
| be implemented incredibly efficiently. This is because it is
| "stackless". Or in other words you know the exact amount of
| "stack" space required and you can allocate this exactly
| instead of allocating a real stack which can be very much
| larger.
|
| For example if I want to implement a "sleep" with async/await
| I probably only need to store the wake time as state, if I
| want to make a virtual thread to do the same I likely need to
| allocate a large stack just in case I use it.
|
| Of course this can be mitigated with stack caching, small
| stacks, segmented stacks or other tricks. But doing this is
| still more expensive than knowing how much "stack" you need
| up-front and allocating only that.
| pron wrote:
| That's not a benefit of async/await as the same could be
| done with user-mode threads. In fact, that's what we do
| with virtual threads. But it might be a benefit of
| async/await in _some particular languages_.
| kibwen wrote:
| _> But it might be a benefit of async /await in some
| particular languages._
|
| Rather than saying it's a benefit for particular
| languages, I'd say it's a benefit in particular contexts,
| e.g. in contexts where you don't have a heap. Of course
| it's true that some (most) languages don't support such
| contexts at all (for a host of good reasons), but the
| languages that do are shaped by that decision.
| pron wrote:
| The use case of interest here is having many concurrent
| operations (hundreds of thousands or millions). If you
| don't have a heap, where do you store the (unbounded
| number of) async/await frames? There are other use-cases
| where stackless coroutines are useful without being
| plentiful -- e.g. generators -- but that's not the use-
| case we're targeting here (and is probably a use-case of
| lower importance in general).
|
| Many languages/runtimes want just a single
| coroutine/continuation construct to cover both
| concurrency and generators -- which is a good idea in
| principle -- but then they, especially low-level
| languages, optimise for the less useful of the two. I've
| seen some very cool demos of C++ coroutines that are
| useful for _very_ narrow domains, and yet they offer a
| single construct that sacrifices the more common, more
| useful, usage for the less common one.
|
| There was one particular presentation about context-
| switching coroutines in the shadow of cache misses. It
| was extremely impressive, yet amounted to little more
| than a party trick. For one, it was extremely sensitive
| to precise sizing of the coroutine frames, which goes
| against the point of having a simple, transparent
| language construct, and for another, it simplifies small
| code that has to be very carefully written and optimised
| to the instruction level even after the simplification.
| kibwen wrote:
| Indeed, I am (perhaps a bit sloppily) using "particular
| contexts" to refer to particular use cases. And while
| your use case is the C5M problem, since we're bringing up
| other languages (which optimize for different contexts) I
| think it's worth emphasizing that these features also
| lend themselves to other use cases. Here's an example of
| using Rust's async/await on embedded devices, for reasons
| other than serving millions of concurrent connections:
| https://ferrous-systems.com/blog/async-on-embedded/
|
| _> Many languages /runtimes want just a single
| coroutine/continuation construct to cover both
| concurrency and generators -- which is a good idea in
| principle -- but then they, especially low-level
| languages, optimise for the less useful of the two._
|
| Notably Rust appears to be the opposite here, as it is
| first focusing on providing higher-level async/await
| support rather than providing general coroutine support,
| but its async/await is implemented atop a coroutine
| abstraction which it does hope to expose directly
| someday.
|
| I'm sure you don't need to be told most of this, but I
| bring all this up to help answer the more general
| question of why not every language builds in a green
| thread runtime, and why one approach is not necessarily
| strictly superior to another.
| kevincox wrote:
| Can you explain how this is done? Is the current stack
| copied onto the heap (to the size it currently is)? How
| are new frames allocated once a thread is suspended?
| pron wrote:
| A portion of the stack is copied to the heap when the
| virtual thread is suspended, and upon successive yields
| those "stack chunks" are either reused or new ones
| allocated and form a linked list. When resuming a virtual
| thread, however, we don't copy its entire stack back from
| the heap to the stack, but we do it lazily, by installing
| a "return barrier" by patching the return address, so as
| you return from a method, its caller (or several callers)
| is lazily "thawed" back from the heap. This copying of
| small chunks of memory into a region that's likely in the
| cache is very efficient.
|
| The entire mechanism is rather efficient because in Java
| we don't have pointers into the stack, so we don't need
| to pin anything to a specific address, and stacks can be
| freely moved around.
| SemanticStrengh wrote:
| I wonder the implications and opportunities for
| https://github.com/microsoft/openjdk-
| proposals/blob/main/sta...
| vbezhenar wrote:
| OS virtual memory ensures that overhead will not be that
| big. OS will allocate memory page by page as software
| touches corresponding virtual addresses. So thread stack
| will use only as much memory as its maximum stack usage
| requires (rounded by page size). Async/await of course is
| more efficient, but in real world native stacks might be
| good enough, especially when RAM is not very expensive.
| addaon wrote:
| "Rounded by page size" is a pretty huge caveat here,
| though, no? With a 4 kB minimum page size on most
| platforms, 5M threads is 20 GB of stack virtual mappings,
| minimum. And cycling through those threads even once will
| make every page of that 20 GB resident.
| dboreham wrote:
| Big thank you from me -- so good to see someone pushing back
| hard against the async insanity.
| olliej wrote:
| > This seems so obvious in hindsight as the "right way". I
| wonder why we went down that whole async/await craze with so
| many languages?
|
| That's basically what async swift does?
|
| It is also orthogonal to async/await.
|
| Async/await is a mechanism a programming language uses to allow
| you to write straight line code without having to write
| blocking code. You _could_ resolve that by creating a hulking
| thread, but that just means you need to await a thread result
| :)
|
| Also JS isn't going anywhere, and it seems unlikely it will
| ever get multiple execution threads
| exabrial wrote:
| I think Java has been working on getting the API "correct" for a
| number of years. I appreciate the pace at which they are moving!
| Once this is released, we're going to have it for a very long
| time
| jpgvm wrote:
| Rate of new features out of the Java team has been incredibly
| impressive.
| homarp wrote:
| see https://news.ycombinator.com/item?id=31214253 "Achieving 5M
| persistent connections with Project Loom virtual threads"
| alberth wrote:
| Erlang.
|
| Curious to see how Loom concurrency compares to FP languages like
| Erlang.
| ferdowsi wrote:
| Congratulations to everyone who thought this through and
| implemented it. I'll go so far as to say that Project Loom makes
| Java a more desirable web programming language to me than any
| language that has colored asynchronousity, like Rust, C#, Python,
| etc. Async/await is one of the worst design patterns to emerge in
| the last few years and I'm glad that the tech world is coming to
| its senses.
___________________________________________________________________
(page generated 2022-05-02 23:01 UTC)