[HN Gopher] Programming language comparison by reimplementing th...
___________________________________________________________________
Programming language comparison by reimplementing the same transit
data app
Author : simonpure
Score : 85 points
Date : 2022-10-23 13:58 UTC (9 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| makapuf wrote:
| Interesting results. Do you think it would be interesting
| /possible to try sqlite as a pure in memory db using :memory db
| file?
| bottlepalm wrote:
| How do you think Node.js would fare here, similar to Deno?
| joshlemer wrote:
| The experience report on Scala I find pretty cathartic. It really
| is absolutely ridiculous the fetishization of extremely complex
| FP and type-level hacking that goes on in the ecosystem, to the
| point where, in the case of the author's hello world web server
| snippet, it's just so complex and laden with concepts you need to
| know that are unrelated to the problem you're trying to solve it
| could be mistaken for a parody or exaggeration, but it's actually
| the library recommended most often to beginners sadly.
| karmakaze wrote:
| This has been my experience as well, both individually and on a
| team. Everyone says it's fine as long as you show restraint and
| stay within bounds you collectively find
| comfortable/manageable. In practice, we all reach a point of
| understanding/comfort that doesn't stay with you on reading
| later, even by the author.
| vvillena wrote:
| I agree with the sentiment that the more FP-heavy parts of the
| Scala ecosystem are not suited to hello-world code _at all_. In
| this particular case, http4s is at its core a low-level
| library. The author was right to go for Play first.
|
| But it turns out there's an alrernative. Tapir is emerging as
| that "pure FP, but approachable" HTTP framework. It allows
| developers to work at a higher level, turning the HTTP backend
| into an implementation detail. Simply define the application
| using Tapir, then choose the better backend for the use case.
| losvedir wrote:
| It's a shame, too, because until that point (when I was still
| just following the "getting started" docs and doing the first
| half of the app, which loads a CSV and parses it and stuff), I
| was actually really enjoying it!
|
| The type-magic style of development is so different from what
| I'm used to, I have periodic crises of conscience where I
| wonder if I've been doing programming wrong all these years, or
| if it's an example of a community barking up the wrong tree.
| This thinking is also what prompts me to look into APL/J/K
| every so often.
|
| The last time I really tried to get into hardcore FP
| programming was several years ago with Haskell, but even then I
| don't recall Yesod (the web framework I tried out, akin to
| http4s here) being _quite_ so overwhelming.
| chowells wrote:
| And Yesod is one of the worst options in Haskell, at least as
| far as making me wonder why it's doing so much magic. It's
| very much trying to build a fully integrated framework
| instead of being composable Haskell. (To be fair, the
| underlying parts written to implement it are composable
| Haskell, and re-used by a lot of other web server projects.)
| hoosieree wrote:
| I dove into the "type driven development" style with Elm (and
| a little Haskell) a few years back. In my opinion the pure
| functional style was more important than the type system.
| Maybe I just didn't try hard enough to think in terms of
| types, who knows. But I have to admit - not having to worry
| about run-time errors is quite nice.
|
| Shortly after Elm, I discovered the array languages via J,
| and felt like I finally found a style of programming that fit
| my brain properly. Immutable by default, rank-polymorphic,
| and extremely powerful. Terseness is a feature - when you can
| write the same program in 1/100 of the code (not an
| exaggeration), you can explore alternative approaches quickly
| and find bugs more easily. On the other hand, there's no
| compiler or fancy type system to find bugs for you, so
| terseness is also kind of necessary.
|
| Today I'd recommend array language newbies to try BQN or K
| first. BQN has some features that other array languages
| surprisingly lack (closures, modules), and only a few
| idiosyncratic design choices (unlike J which is very weird).
| K is pragmatic (it has dictionaries and tables!) and is
| ascii-based, but it has an almost Forth-like minimalism. For
| someone used to 'import xyz' style programming it's a jarring
| transition.
| solidninja wrote:
| I do wonder where the recommendation to use http4s for
| beginners came from. http4s is a very capable library (and if
| you care much about composition it is excellent), but I
| wouldn't describe the documentation as beginner friendly.
|
| A slightly better starting point for scala 3 + type-safe server
| building is tapir e.g.
| https://github.com/softwaremill/tapir/blob/master/examples3/...
| . With that, you get a declarative definition of your endpoints
| (+ error types, auth, etc.) that you can use for both servers
| and clients, which comes very handy when writing integration
| tests of course.
|
| > absolutely ridiculous the fetishization of extremely complex
| FP and type-level hacking that goes on in the ecosystem
|
| An alternative way to look at it is that there is a lot of
| essential domain complexity that gets encoded via the type
| system to let the compiler do the hard work. That "extremely
| complex FP" does not arrive out of nowhere - I really recommend
| at least skimming through the slides from rossabaker, the
| http4s designer, that motivate where the core type signature
| comes from https://rossabaker.github.io/boston-http4s/#2
|
| I suppose one of the "features" that I like about the
| (typelevel) community is that the approach of "worse is better"
| is not taken, and a lot of effort is expended to make things
| correct, modular and orthogonal. This has the drawback of
| increased upfront complexity, that anecdotally pays off the
| moment your compiler does not error and the program runs as
| intended.
| kenhwang wrote:
| The Scala 3 requirement really does cripple the choices
| available. The Scala 3 transition does feel a bit like the
| Python 3 transition where major libraries and frameworks are
| really dragging their feet on transitioning because of the high
| level of effort for very little obvious improvement.
|
| Scala 2 is still getting regular updates so there's no rush to
| use 3 just yet, I'd recommend picking the framework first and
| using the Scala version they support. That makes the
| development experience much better with access to much easier
| to use frameworks like play or finatra.
| vvillena wrote:
| In the long term it will be an improvement. The one thing
| dragging adoption down is the removal of the old macro
| system, which depended completely on the implementation
| details of the old compiler. That said, I agree that Scala 2
| is still very much supported, so there's nothing wrong with
| picking it.
| metaltyphoon wrote:
| You could have used minimal apis in C# and gotten a better
| throughput
| zikohh wrote:
| I would've expected a higher requests per sec for 10VU for Go.
| Throughout the comparisons Go was just right behind rust but here
| it it was off by a high magnitude.
| losvedir wrote:
| Oh, hey, didn't expect to see this here. Thanks for submitting! I
| tried a few days ago but didn't really get any traction.[0]
|
| Since I submitted it, though, I posted it to /r/rust and got a
| lot of feedback. At the time, rust and dotnet were comparable and
| at the top of the list, with ~10k req/sec. Now rust is far and
| away the most performant at ~20k req/sec! I also was able to
| improve Go's performance 30% or so. Still, I want to let the
| other communities chip in and see if I can improve them.
|
| I was actually just in the midst of exploring how to improve
| Elixir's results. I'm finding I can almost double my requests per
| second from simply switching the JSON encoder from Jason to
| Jiffy. That sort of surprised me since Jason is the de-facto
| standard, and I thought was super fast.
|
| [0] https://news.ycombinator.com/item?id=33291604
| pg_bot wrote:
| The elixir implementation seems super slow. The first thing
| that jumps out to me is that we should be using streams instead
| of just reading the file. The second would be the
| `schedule_for_route` which should just be a GenServer lookup
| instead of reading from ets.
| losvedir wrote:
| Yeah, I should benchmark streams. In my experience streams
| are rarely faster, though, and are instead something you
| should reach for if 1) you can't fit the file in memory or 2)
| you might stop the enumeration early.
|
| Regarding GenServer, I'm not so sure about that. I suppose I
| should benchmark it, but intuitively I expect ETS to be
| better here. There's more overhead in getting the data, but
| it allows you to concurrently read it from different
| processes. A GenServer, meanwhile, could respond to a given
| message faster, but now you're serializing the (e.g.) 50
| concurrent virtual users through a single bottleneck. I could
| have multiple copies of the data in several GenServers, I
| suppose, at the expense of much more memory use.
| pg_bot wrote:
| I would guess that there's too much work being done at
| runtime. I would expect all of the data to be cached in
| such a way that it is a read operation without any
| transformation. It's doing multiple lookups and unnecessary
| maps when the data should just be formatted a single time
| and cached. You could even skip the JSON transformation by
| just doing it once if you want to get super fast.
| di4na wrote:
| Are you on OTP 25 ? With the JIT, in theory Jason is faster
| than Jiffy
| di4na wrote:
| Another thing to add, do you log things in the other
| languages ? If not, what you are probably benchmarking there
| is how fast your console is in receiving the text (with a
| lock).
|
| Also use a release, it will help some things here, especially
| on "short" benchmark, and make sure that the JIT is run.
| losvedir wrote:
| Yep, Elixir 1.14, OTP 25. I set the log level to warn so
| Phoenix isn't logging anything that I see. Do you think
| that still impacts performance? I thought Logger compiled
| out log calls below the level threshold.
|
| Good call on running as a release! I'll try that next.
|
| edit: Ah well, good thoughts but didn't pan out. I updated
| the logger config to use `compile_time_purge_matching` just
| to make _sure_ there wouldn 't be any logging impact, and I
| ran the app as a release, but didn't really make any
| difference in the numbers that I saw.
| bhaney wrote:
| > I'm finding I can almost double my requests per second from
| simply switching the JSON encoder from Jason to Jiffy
|
| Personal plug, but have you tried Jsonrs[0]? I typically get
| much better performance out of it (especially in lean mode,
| which seems to be the mode you'd want to use for this
| benchmark) than Jiffy for large JSON encoding workloads.
|
| [0] https://hexdocs.pm/jsonrs/readme.html
| losvedir wrote:
| This is great! Just pushed up a commit that uses it and
| updated the benchmarks[0]. I'm seeing a 1.6X - 2X improvement
| in overall performance. Not bad for a drop-in replacement.
| And since it's based on serde, I trust it, and I feel like
| trying out a different JSON library is within scope for me of
| not just "gaming the benchmarks", as this is actually
| something I'd now consider using at work.
|
| It's not _quite_ as high as I was seeing with `jiffy` (3,800
| req /sec here vs 4,000+ with jiffy), but I'm not confident
| that was a totally fair comparison. `jiffy` doesn't integrate
| as nicely with Phoenix, so I was just calling
| `:jiffy.encode(...)` in the controller and then doing a
| `text(...)` response. I need to double-check if `json(...)`
| is doing more work here.
|
| [0] https://github.com/losvedir/transit-lang-
| cmp/commit/140d693b...
| bhaney wrote:
| One major difference you'll run into here is support for
| encoding protocols. Jason, Poison, and Jsonrs (by default)
| will have protocol impls for Elixir structs that define
| special logic for how to serialize those structs into JSON
| strings. ie, you probably want a DateTime struct to
| serialize as a date string rather than a map containing the
| year, month, etc. Jiffy doesn't support any of that, so you
| end up with differing behavior between it and other
| libraries as soon as you start encoding structs that aren't
| meant to serialize as maps.
|
| Jsonrs supports encoding protocols by default, but lets you
| turn it off for a speed boost if you're okay with more
| Jiffy-like behavior. Plugging it in as Phoenix's JSON
| library won't turn off that protocol handling, so you're
| getting the slower operational mode of Jsonrs (which is
| fine and probably what you want in most cases for
| correctness, but will definitely eat a few ops/sec in a
| benchmark)
| ad404b8a372f2b9 wrote:
| Did you post on the elixir forum? Those results seem very low,
| I feel like there has to be something amiss.
| throwawaymaths wrote:
| What I would love to see is someone injecting an programming
| error into something like this and seeing how long it takes for
| someone experienced in the language to debug it.
| andreygrehov wrote:
| According to the results, Go and Rust pretty much beat every
| other language. While Rust is the clear winner, it has a steep
| learning curve. When it comes to Rust vs Go, I always choose Go,
| because it still beats all the other languages and at the same
| time an extremely simple language to learn / read / write. This
| triplet has a greater weight in my books.
| jph wrote:
| Excellent writeup thank you. And I love that you wrote
| "BurntSushi is a national treasure". I hope he sees your writeup
| because it's true.
|
| (BurntSushi is Andrew Gallant, creator of ripgrep and other
| excellent Rust crates, and writes code that is especially good
| IMHO for learning how to write real-world Rust libraries and
| programs.)
| jamie_ca wrote:
| For a similar project, someone put together a spec for a basic
| clone of Medium split into a frontend design and backend API, and
| now has over 100 different components where you can pair any
| frontend with any backend (plus a few fullstack implementations).
|
| https://github.com/gothinkster/realworld
|
| Not as performance-focused with benchmarks, but a good point of
| comparison for various languages and frameworks implementing
| common behavior.
___________________________________________________________________
(page generated 2022-10-23 23:00 UTC)