[HN Gopher] Helidon Nima: A Java microservices framework based o...
___________________________________________________________________
Helidon Nima: A Java microservices framework based on virtual
threads
Author : philonoist
Score : 45 points
Date : 2023-08-18 15:42 UTC (1 days ago)
(HTM) web link (helidon.io)
(TXT) w3m dump (helidon.io)
| canvascritic wrote:
| Oooh this looks promising. Seeing helidon nima's take on java
| microservices with virtual threads is certainly a different
| direction. Way way back, while working with a java microservices
| codebase, we were stuck with OJ (ObscuraJ, to be honest I'm not
| sure if it was publicly available or internal, was so many years
| ago) and that was pure hell. The configuration overhead,
| especially the cumbersome dynamic routing setup, and layers upon
| layers of indirection and dependency injection was a headache.
| nima's approach piques my interest, albeit with a bit of caution,
| will need to dig into the source more
| twic wrote:
| I am looking forwarding to trying Nima. The blocker for me is
| IntelliJ supporting Java 21, which i believe will be this winter
| [1]; i don't absolutely need IntelliJ, but i am very lazy.
|
| At the moment, my apps are using either the JDK HttpServer, which
| is easy to use, but of questionable robustness and lacking
| websocket support, or a Netty-based server, which works very well
| and supports websockets, but is often awkward to program against
| because it's asynchronous. Nima should combine a nice simple
| synchronous API with a fairly luxurious feature set and good
| scalability.
|
| Of course, it will turn out there are things terribly wrong with
| it, because this is real life. But for now i'm optimistic!
|
| [1] https://intellij-support.jetbrains.com/hc/en-
| us/community/po...
| gbear605 wrote:
| The IntelliJ Early Access Program will be out sometime before
| then, if you're willing to go for a potentially unstable IDE
| experience.
| Two4 wrote:
| Non-MicroProfile Helidon uses a reactive interface for most
| things, which lends itself well to async operations. It a very
| nice, simple and powerful abstraction that makes async
| relatively easy. IMO anything webserver or client should be
| async when virtual threads aren't available, due to Java having
| the tendency to quickly build up a ton of threads when you use
| the one-thread-per-request model. That being said, virtual
| threads does make that model viable at scale, so reactive async
| probably isn't as necessary.
| strangemonad wrote:
| I've had no problems scaling jetty to 400-500 qps per node on
| small thread pools.
| Two4 wrote:
| As soon as you start talking thread pools, you're talking
| about async task dispatching at some level, even if it's
| abstracted away at the API level. What I'm talking about is
| using a virtual thread, not event-driven shuttling of
| stateful objects between threads via task queues, to
| complete a request from start to finish.
| mikmoila wrote:
| Jetty can be used with virtual threads:
| https://webtide.com/jetty-12-virtual-threads-support/
| twic wrote:
| Reactive APIs are horrible, and precisely what I'm trying to
| avoid.
| Svenskunganka wrote:
| The main problem I have with reactive programming in Java, at
| least with Project Reactor which is the only one I've used is
| the cryptic stack traces. You can regain some information by
| enabling runtime instrumentation to be used in development,
| but has too much of a performance hit to be used in
| production. It doesn't produce much better stack traces
| either. Trying to read a profiler flamegraph is nigh
| impossible for reactive code. For the debugger, you need to
| manually insert breakpoints in each step of the chain,
| because otherwise you'll just be looking around the reactive
| library's internal code.
|
| The other problems I have is that as soon as you need to do
| anything blocking (e.g talking to SQLite or RocksDB over
| JNI), the whole thing falls apart in terms of threading.
| Doing a HTTP request somewhere in a reactive chain that does
| some blocking operations - well shit now you're blocking the
| reactor-http-netty-* threads, which are fixed so you're worse
| off than with thread-per-request at that point. Or even just
| using Caffeine cache, or any caching library really, but
| Caffeine is one of few that prevents async stampede - well
| shit now you're running on ForkJoin pool during cache misses,
| but "reactive" threads on cache hits. You can see how this
| quickly becomes a tangled mess of thread-hopping and avoiding
| blocking event loops, and it's extremely complicated to get
| this right with how subscribeOn/publishOn works and how the
| profiler can't tell you which threads runs what code because
| the stack traces are filled with garbage.
|
| You also opt out of a lot of what Java offers in terms of
| synchronization. Using the synchronization keyword is a big
| no-no, the JVM can park the thread while waiting for the
| lock. Now that thread cannot schedule other tasks just
| because that one task ran into lock contention, and it's
| highly likely a fixed pool of threads. You're basically left
| with CAS, AtomicLong/AtomicBoolean/etc, and that's it. I've
| even seen BlockHound, project reactor's tool to help you find
| if you are blocking somewhere, trigger "Blocking call to
| LockSupport.park()!" during a ConcurrentHashMap lookup. Add
| to this that most caching libraries are implemented on top of
| ConcurrentHashMap. Getting async right in reactive code has a
| huge mental overhead and you need intricate knowledge of the
| libraries you depend on, the platform you run on and what it
| is that actually enables async in the kernel, and where it is
| unavailable. Good example of this; that UUID library you use
| uses a CSPRNG. Have you set the JVM flag to read it from the
| non-blocking /dev/urandom rather than the default /dev/random
| on Linux?
|
| I think reactive programming in Java is decent as long as all
| you're really doing is accepting/performing HTTP requests and
| talking to a database that has a reactive client connector.
| In essence, anything that strictly deals with networking,
| since that's all that can really be made truly async
| currently, thanks to epoll/kqueue/iocp on linux/mac/windows
| respectively. Hopefully io_uring will broaden that to file IO
| as well eventually, but unfortunately that's linux-only for
| now.
|
| Contrast this with Go's scheduler that just takes care of
| this for you - no need to think about it. Or Rust + Tokio,
| where it is explicit with tokio::spawn_blocking and offering
| async counterparts to std Mutex, RWLock, Channels, etc that
| doesn't block the current thread on lock contention.
| skinowski wrote:
| This is one of the things that bother me about Java; IDE
| dependency.
| skinowski wrote:
| If a developer says "The blocker for me is IntelliJ
| supporting Java 21", clearly there's an IDE dependency.
| Powerful enough for a developer not to adopt a new framework.
| twic wrote:
| It's entirely possible to write Java without an IDE. I've
| done it, and one of my colleagues has been using VSCode
| quite happily (we've bullied him into stopping because it
| doesn't have a formatter and he keeps checking in wonky
| code - but that's another story).
|
| But writing _any_ language is much more productive with a
| good IDE. For me, the added value in using Java 21 now
| rather than in a few months is not enough to outweigh
| giving up the use of an IDE.
| xcv123 wrote:
| No it is straightforward to setup a Maven project manually
| and use Vim if that's your preference.
| zmmmmm wrote:
| But then they complain they had to manually type an import
| statement ... before going back to their opinionating on
| how bad IDEs are.
|
| I find the whole notion of software engineers rejecting
| software applications being useful as a concept while it
| being the primary focus of their own profession quite
| fascinating.
| twic wrote:
| There are some weirdly hair-shirt beliefs about
| development practices in this industry.
| skinowski wrote:
| If a developer does not want to try this framework
| because an IDE does not yet support a particular version
| of java, then the situation has gone beyond an IDE being
| just useful.
|
| Personally I use various IDEs, light weight IDE/Editors
| and even vi/vim depending on the language & situation.
|
| Think of it this way:
|
| If I asked my team to develop a project/POC with this
| framework and they came back to me saying that they have
| to wait for (or prefer to wait for, or whatever) jet
| brains to add java 21 support to do this, I would not be
| very happy.
| pjmlp wrote:
| IDE productivity, and it started with Smalltalk and Lisp
| Machines, was adopted by C++, Visual Basic and Delphi, among
| several 4GLs, several years before Java was invented.
|
| Everyone is free to use Java in vi with make, if they feel
| happy doing so.
|
| In fact, there were hardly any Java IDEs when the language
| was released in 1996, which were quickly provided by
| Smalltalk, Delphi and C++ vendors.
| lolinder wrote:
| It's not so much that you can't do Java without an IDE--Java
| is verbose in part because it doesn't assume that you have an
| IDE to give you context--it's that IntelliJ is such a good
| IDE that it's hard to go back to weaker tooling once you've
| become comfortable with it.
|
| All my co-workers use VS Code for TypeScript, but I feel
| crippled when I can't use WebStorm. Not because I can't
| program without it, but because I'm missing a powerful force
| multiplier.
| lenkite wrote:
| Thats because Java IDE's are _ridiculously_ powerful. When I
| work in vscode and Python, I am always like: This is so much
| easier when I am using Java and Intellij.
|
| It is not mere IDE _Dependency_. It is IDE _Supremacy_. Java
| is the leader of the IDE master race - the other PL IDE 's
| need to squint to see how far ahead Java IDE's are.
| rubicon33 wrote:
| I'll take Scala + Akka over this. Underrated multithreading
| framework to say the least.
| mikmoila wrote:
| Of course, this is not fully comparable to Akka, but you might
| find this interesting: https://github.com/ebarlas/game-of-life-
| csp
| sgt wrote:
| This is a killer "app" for Java 21. There's now little reason to
| choose Kotlin or some other language to build efficient API's
| etc.
| jfengel wrote:
| Was there ever? What made Kotlin fundamentally different in
| that respect?
|
| Kotlin is great for fixing some legacy design oopsies (Java
| really should have been smarter about null and it can't be
| fixed without breaking legacy code), but I wouldn't expect it
| to do something that was impossible or it infeasible in Java.
| Tainnor wrote:
| Coroutines are one of the most prominent features of Kotlin,
| so it does make sense to bring this up in the discussion.
|
| OTOH, I'm unconvinced that virtual threads make coroutines,
| reactive APIs etc. obsolete. Those are about structured
| concurrency, and with projet Loom, the structured concurrency
| proposal is still in the incubating stage.
| mikmoila wrote:
| SC will be a preview level API in Java 21:
| https://openjdk.org/jeps/453
| zmmmmm wrote:
| Yes from a pure functional perspective, it's really down to
| null handling at this point. But kotlin has a lot of nice
| syntactic sugar still. Same with Groovy.
| Tainnor wrote:
| > it's really down to null handling at this point.
|
| and reified generics, if you care about that sort of thing.
|
| Plus immutability/final by default, the stdlib being a joy
| to use, etc.
| philonoist wrote:
| Elaborated here - https://medium.com/helidon/helidon-nima-
| helidon-on-virtual-t...
| logicchains wrote:
| Great project but the name is very unfortunate for Chinese
| speakers.
| pandemic_region wrote:
| Care to elaborate?
| phyrex wrote:
| "Nima" could be read as "your momma" in mandarin
| twic wrote:
| It's a weird name, because it's a transliteration of the
| Greek word for "thread", which would conventionally be
| "nema" in English (as in nematode worms, nematocysts,
| Treponema, etc).
| logicchains wrote:
| It's the most common Chinese swear word, "your mother" (Ni Ma
| ).
| [deleted]
| pandemic_region wrote:
| I'm not really grokking the code sample on their frontpage. What
| is it doing? Explain it like I am a Java developer with 18 years
| of experience.
| Tainnor wrote:
| It's referencing other variables and functions that aren't
| defined in that snippet. Presumably you can somewhat guess what
| it's trying to do (call a function for each element of the
| response), but it doesn't seem like a very good example. I also
| don't quite understand how this utilises virtual threads
| effectively, isn't the code still adding the elements of the
| list one at a time instead of concurrently?
| mikmoila wrote:
| One at a time, not concurrently.
| Tainnor wrote:
| so what's the point? we can already write blocking code.
| The reason why people want to use reactive frameworks is
| because they don't want to block threads just because some
| I/O is happening.
| mikmoila wrote:
| Nima uses virtual threads where blocking call blocks only
| the current virtual thread but not the underlying
| carrier-thread, which can run another virtual thread
| while waiting i/o to complete.
| Tainnor wrote:
| But in this example, the entire code is still blocking
| because it has to wait for one call of "callRemote" to
| complete while waiting for the next one to be executed.
| So I don't get the point. for (int i = 0;
| i < count; i++) { resp.add(callRemote(client));
| }
| xcv123 wrote:
| That entire section is running on a virtual thread, which
| is scheduled on a platform (OS) thread. While it blocks
| for each callRemote invocation the platform thread is
| free to process other virtual threads.
|
| https://docs.oracle.com/en/java/javase/20/core/virtual-
| threa...
| Tainnor wrote:
| Yeah but if for some reason, no other (or few other)
| request is coming in concurrently, then the CPU will just
| sit by idly. I don't understand why you wouldn't issue
| the "callRemote" calls concurrently, that seems like it's
| missing the point of writing non-blocking code.
| xcv123 wrote:
| Exactly. That's the point. It is beneficial only when you
| have a huge number of requests relative to the number of
| platform threads.
|
| The idea is that you can map a million of these virtual
| threads on to a small number of platform threads, and the
| JVM will schedule the work for you, to achieve maximum
| throughput without needing to write any complicated code.
| Virtual threads are for high throughput on concurrent
| blocking requests.
|
| EDIT: the sequential calls to "callRemote" still process
| in sequence, blocking on each call. Don't get confused
| there. But the overall HTTP request itself is running on
| a virtual thread and does not block other HTTP requests
| while waiting for the callRemote invocations to complete.
| Tainnor wrote:
| > EDIT: the sequential calls to "callRemote" still
| process in sequence, blocking on each call. Don't get
| confused there. But the overall HTTP request itself is
| running on a virtual thread and does not block other HTTP
| requests while waiting for the callRemote invocations to
| complete.
|
| Yeah I got that. But then the use case seems to be much
| narrower, because not every scenario is one where many
| concurrent requests come in at the same time. If you
| write non-blocking code you'll get high throughput no
| matter the number of requests. Ok, maybe non-blocking
| code is harder to write (and I don't think it's actually
| that hard if you use, say, Kotlin coroutines), but
| honestly this seems to me like something that developers
| should learn eventually anyway.
|
| Or maybe it's me being weird. But I remember 10 years ago
| when node.js was being hyped for being "fast" because it
| was "async" and now suddenly using threads and blocking
| operations is again all the rage now.
| xcv123 wrote:
| The difference now is that it's implemented in the JVM
| instead of a library / framework. It is easier, simpler,
| and probably more efficient. You can get higher
| throughput from existing code with minor refactoring.
| Thread thread = Thread.ofVirtual().start(() ->
| System.out.println("Hello")); thread.join();
| mikmoila wrote:
| Also there are some traceability issues involved in using
| asynchronous APIs: "In the asynchronous style, each stage
| of a request might execute on a different thread, and
| every thread runs stages belonging to different requests
| in an interleaved fashion. This has deep implications for
| understanding program behavior: Stack traces provide no
| usable context, debuggers cannot step through request-
| handling logic, and profilers cannot associate an
| operation's cost with its caller. "
|
| https://openjdk.org/jeps/444
| Tainnor wrote:
| I'm not saying that virtual threads aren't a good thing
| (other runtimes than the JVM have had green threads for a
| very long time now), but that it doesn't seem like such a
| paradigm shift. It's still threads (with all the benefits
| and drawbacks), it's just now that they have less
| overhead.
|
| Presumably I could just keep an existing server written
| in Spring MVC or a similar technology and just wait for
| the underlying container to support virtual threads to
| get the same benefits. I believe Jetty already does
| support them. So why would I need a new framework?
| xcv123 wrote:
| It removes the need to write async code in many cases. It
| is a more straightforward and efficient way to accomplish
| what we have already been doing for years on the JVM.
|
| You don't need to switch frameworks
|
| https://spring.io/blog/2022/10/11/embracing-virtual-
| threads
| zmmmmm wrote:
| Because concurrent calls are confusing and create bugs.
|
| The point this train wreck of an example is trying to
| make (and failing) is that it's fine to write sequential
| blocking code in your request handlers. They can take as
| long as they want ... seconds, minutes, days .... because
| the handler threads are now an infinite resource as far
| as the JVM is concerned.
| twic wrote:
| You certainly could make the callRemote calls in
| parallel, and there are easy and safe ways to do that
| (good old CompletableFuture or the new structured
| concurrency stuff). But doing that or not is completely
| independent of using Nima, so I think here they're just
| showing some very simple code, because this is an example
| of using Nima, not an example of writing concurrent code
| in general.
| mikmoila wrote:
| In addition, this implementation detail might help in
| gaining some insight: "The synchronous networking Java
| APIs, when run in a virtual thread, switch the underlying
| native socket into non-blocking mode. When an I/O
| operation invoked from Java code does not complete
| immediately (the native socket returns EAGAIN - "not
| ready" / "would block"), the underlying native socket is
| registered with a JVM-wide event notification mechanism
| (a Poller), and the virtual thread is parked. When the
| underlying I/O operation is ready (an event arrives at
| the Poller), the virtual thread is unparked and the
| underlying socket operation is retried."
|
| https://inside.java/2021/05/10/networking-io-with-
| virtual-th...
| emccue wrote:
| A better example would be var
| remoteResultA = callRemoteA(client); var
| remoteResultB = callRemoteB(f(remoteResultA));
|
| Where handling a request means you have to make two http
| calls and those two calls have some order dependence.
| I.E. the result of one is used to construct the call to
| get the other.
|
| What virtual threads give is the ability to write the
| code like I did above and still get ideal performance, as
| opposed to. callRemoteA(client)
| .then(remoteResultA -> callRemoteB(f(remoteResultA)))
| .then(... your world lives here now ...);
|
| Virtual threads are, from a language semantics point of
| view, the same as regular threads. Virtual threads map
| maybe 10000 "logical threads" to a handful (maybe 8 or
| so) actual operating system threads.
|
| The big difference is that for each regular Thread you
| have exactly one backing Operating System thread.
| Tainnor wrote:
| I agree that this would be a better example that would
| make more sense (and I also think that the reasoning you
| provided should be added as context to that code
| snippet).
| mikko-apo wrote:
| Before virtual threads, Java server would reserve a full
| OS thread while that code is being processed. OS threads
| use lots of memory and there might a limited number
| available. With a single incoming request or low traffic
| that is not a problem, but with many parallel requests
| the server would choke on using too much memory or OS
| threads.
|
| Previously in Java if you wanted to avoid code that
| blocks the OS thread you would need to use callback based
| code or reactive programming:
| https://www.alibabacloud.com/blog/how-java-is-used-for-
| async...
|
| Javascript moved on from callbacks to async/await, which
| colors the async functions and requires the programmer to
| be aware of the issues related to async/await.
|
| For Java Project Loom takes a different approach. For
| example with server applications, the server would start
| running the code in a virtual thread and most of the code
| that looks like blocking will actually work like
| unblocking code. You get to use regular blocking code but
| you get async performance.
|
| When the code needs to wait for a response, JVM can
| unmount the virtual thread from the OS thread and then
| the JVM can run another virtual thread in the OS thread.
| Once the response is returned and the OS thread is free,
| JVM can continue executing the original request.
|
| There are some caveats to virtual threads. There is some
| overhead (but not a lot), some IO calls still block (but
| you don't get an indication about using blocking IO) and
| you might stumble on new kinds of bugs related to
| resource exhaustion (try reserving a million OS file
| handles).
|
| But no more async/await, reactive programming, callbacks.
| Mostly just regular imperative blocking-looking code. I'm
| very excited about that.
| mikmoila wrote:
| "callRemote" simulates a blocking outbound http-call.
___________________________________________________________________
(page generated 2023-08-19 23:00 UTC)