[HN Gopher] Ruby 3 JIT can make Rails faster
___________________________________________________________________
Ruby 3 JIT can make Rails faster
Author : rguiscard
Score : 111 points
Date : 2021-05-21 08:01 UTC (15 hours ago)
(HTM) web link (k0kubun.medium.com)
(TXT) w3m dump (k0kubun.medium.com)
| ksec wrote:
| So we now have Ruby MJIT, from CRuby and k0kubun, "MJIT" was
| originally implemented by Vladimir Makarov.
|
| MIR, coming from Vladimir Makarov ( RedHat ) again.
|
| YJIT coming from Shopify, Dr Maxime Chevalier-Boisvert is leading
| the team and Dr Stefan Marr ( I think ) will be joining them
| soon.
|
| TruffleRuby, with Dr Chris Seaton now also working in Shopify.
|
| JRuby, Charles Nutter ( RedHat ).
| an_opabinia wrote:
| Will TruffleRuby win? GraalJS is typically 38x slower than
| Node.
|
| Maybe everyone should be porting to V8.
| lf-non wrote:
| This is surprising. Last I checked performance was better
| than node.
|
| Are you taking into account JVM warmup time ? Also does the
| JVM have adequate memory (thinking of GC thrashing)?
| an_opabinia wrote:
| > Last I checked performance was better than node.
|
| I don't know if there's any real world node application
| that is faster on GraalJS, no matter how much "warming up."
| It is _incredibly_ slow. I understand there 's marketing
| copy out there saying stuff to the contrary, but really, if
| it was that great, people would be using it. Which I'm
| confident people don't, because real world applications use
| tons of the node API that Graal doesn't support, like
| `setTimeout`.
| kaba0 wrote:
| Looking at the issues of GraalJS, it seems to me rather
| that it is a few nodejs devs who doesn't seem to know
| what is a worthwhile benchmark, sometimes laughably so.
| It is well within the same performance range, faster at
| some tasks, slower at others.
| vips7L wrote:
| The readme of TruffleRuby even states that large ruby
| applications take a large time to warmup and they're
| working on it.
| tekknolagi wrote:
| From https://blogs.kent.ac.uk/unikentcomp-
| news/2021/04/16/stefan-... it looks like he will be working on
| GraalRuby/TruffleRuby
| hit8run wrote:
| Thanks at k0kubun and all the others investing their time to Ruby
| optimization. When performance is a metric one needs to care
| about Ruby is a bad choice and will stay a bad choice. I find C#
| with dotnet a very good framework that has amazing performance
| characteristics and also good developer tooling. I coded many
| things with Ruby and Rails privately and as my daily job but
| nowadays it is mostly C# for webdev.
| sosodev wrote:
| JIT doesn't seem like the big optimization that Rails needs.
| Rails has low throughput because you have to run multiple full-
| fat instances of rails to handle requests in parallel or multiple
| beefy threads to handle requests concurrently. Either way you're
| spending a huge amount of ram to add a small amount of
| throughput.
|
| JIT only adds a marginal improvement to this current setup.
|
| A true async backend like Falcon adds a huge amount of concurrent
| throughput if tuned correctly and ractors could provide a big
| parallelism boost if they shared more memory.
| rattray wrote:
| TIL about Falcon, which can power any rack app, including
| Rails: https://socketry.github.io/falcon/index.html
|
| TIL also that in Rails v5+, `config.allow_concurrency` is
| enabled by default, so Falcon works with it out of the box.
| Neat.
|
| Looks like Falcon's
| [benchmarks](https://github.com/socketry/falcon-
| benchmark#results) show 250+ concurrent connections serve with
| an order of magnitude less latency and 2-3x the RPS of puma or
| passenger, and >10x that of unicorn.
|
| Seems worth a look for anyone running Ruby in prod with really
| any concurrent traffic.
| joelbluminator wrote:
| So what is the main advantage of this over say Puma/threads?
| rattray wrote:
| It claims better performance than puma.
| alberth wrote:
| Isn't most of rails performing blocking operations. And as
| such, the numbers you stated above don't hold true.
|
| The better graph to look at is the "Blocking Benchmark",
| which shows all backends performing (equally) poorly.
| sosodev wrote:
| Async Ruby is rapidly maturing. I haven't run Falcon in
| production yet but it's not difficult at this point to
| setup Rails to take advantage of async IO.
| ninkendo wrote:
| Indeed, the postgres drivers I've encountered are all
| blocking.
|
| On this topic, a warning for anyone trying to implement
| async rails with postgres: beware. Connections are per-
| thread in the 'pg' gem (and every other driver I've used),
| so one client's connection may commit/abort another
| client's transaction, and they end up stomping all over
| each other. Which you will only observe under load.
|
| There's an eventmachine-based driver for postgres but it's
| almost completely unmaintained afaict, and doesn't work
| with any new rails versions.
|
| I spent a ton of effort migrating an internal rails app to
| eventmachine, only to realize at the very end that it
| fundamentally doesn't work with typical CRUD/database apps.
|
| (Note, this experience was probably 4 or so years ago at
| this point, things may have matured since then, but I'd
| still say approach async rails with extreme caution.)
| ioquatix wrote:
| Non-blocking Postgres https://github.com/socketry/db-
| postgres
|
| Non-blocking MariaDB https://github.com/socketry/db-
| mariadb
| kayodelycaon wrote:
| > Rails v5+
|
| Rails v4+ :)
|
| Also, you need to use a thread-based app server like puma.
| App servers like unicorn do one process per request rather
| than threading.
| rattray wrote:
| Yes, falcon is a thread-based app server.
| Conlectus wrote:
| I don't follow how an "async backend" would help over threads
| in this case. If you mean using actors everywhere and moving
| that into client code, you effectively have a new framework.
|
| Likewise, async IO has high throughout but often trades that
| off for increased latency jitter, since requests can block each
| other because of the limited thread pool. This is less a
| problem with threads because of preemption (though they of
| course compete for shared resources like database time).
| merb wrote:
| ?!
|
| > This is less a problem with threads because of preemption
| (though they of course compete for shared resources like
| database time).
|
| eh, just because ONE implementation of async is single
| threaded, does not mean that others aren't.
|
| basically you trade of latency because most async
| implementations do SCHEDULE their tasks/promises in a WORK
| STEALING fashion (overscheduling and stuff) and another
| tradeoff is context switching, of course not all task
| implementations try to reduce that. also if you have blocking
| operations it might happen that you have two tasks/promises
| on the same thread, which are lightweight at the beginning
| and at the end the blocking operation might block the other
| promise (thus latency for b is reduced)
|
| with threads you MIGHT have higher latency in p50, but
| probably not in p99. of course if you have 16 threads and
| only 16 requests at any time, the sync engine wins, but most
| often that is only the case in small applications with few
| users.
| viraptor wrote:
| It really depends on your workload. In my case for example an
| app takes ~60% of time in Ruby code. Tweaking how the
| scheduling of the processing works will not affect it as much
| as speeding up the actual Ruby part. For someone else with a
| very thin API service, it may not matter much.
|
| My point is, you can't generalise "what rails needs". Different
| deployments will need different things.
| aardvark179 wrote:
| I don't think Rails is anywhere close to light enough for the
| memory overhead of threads to be the important limiting factor.
| For an async server or something like project loom to pay off
| you need the request processing to be light and the time to be
| dominated by IO which can either be explicitly converted to
| async IO and callbacks or implicitly turned into async IO by
| your language's standard library (as we are doing with Java and
| Loom).
|
| In my experience even with a JIT Rails is spending a lot of
| time in all the layers of request processing, so it has limited
| throughout even when the actual IO is trivial. No async
| framework can save you from processing overhead like that.
|
| Note: I work on TruffleRuby and Project Loom.
| StreamBright wrote:
| I think in the context of high performance it is pointless to
| talk about Rails. Even the memory requirements make it
| impossible scale above a certain (very low) threshold.
|
| In the context of developer productivity it is also pointless
| to talk about high performance because people sacrifice
| performance for developer happiness.
|
| Once you understand that you have these two dimensions most
| frameworks are revolving around it is getting much clearer
| which one to pick for a certain use case.
| aledalgrande wrote:
| This is legend. Rails can be and is used in high throughtput
| scenarios, people who say it can't scale all think they are
| working at Twitter while they just wrote inefficient code.
|
| It is true that spending time optimizing queries will give
| you more benefits than JIT though. CPU is not the bottleneck
| for most Web tasks.
| StreamBright wrote:
| Sorry I used to work for Amazon and we did not allow Rails
| to be exposed to external customers for performance
| reasons. I think what you classify as high performance is
| very different what I classify as high performance.
|
| Btw. I haven't seen Rails or Ruby on this list in the top
| 50% for a while:
|
| https://www.techempower.com/benchmarks/
|
| I would prefer hard data as opposed to "this is a legend".
|
| UPDATE:
|
| Rails is number 399 out of 439, having 0.3% performance of
| the top frameworks in a plain text task.
|
| I know, facts hurt.
| jasonwatkinspdx wrote:
| That's just Amazon's policy, not some sort of fundamental
| constraint of Rails itself.
|
| While it's true Rails will use more ram than other
| options for a given traffic level, many of us have scaled
| it just fine.
|
| The techempower benchmarks aren't particularly useful for
| anything other than forum warrior style arguments. They
| don't really measure a realistic workload.
|
| Your last sentence is entirely unnecessary.
| StreamBright wrote:
| I agree, HN needs less and less facts nowadays and more
| feelings about somebodies favorite tool or framework. The
| quality of comments are going below Reddit levels.
| jasonwatkinspdx wrote:
| You are the one being emotional, and obnoxiously toxic at
| that.
|
| Everything I said is factual. Many of us have built
| successful startups on rails. I'm a very pragmatic person
| in general and abhor the cheering and jeering dynamic
| you're actively courting in this thread.
| davidw wrote:
| Here's some hard data: Amazon is one of the largest
| internet companies out there, and most of us don't need
| to write anything approaching that kind of scale. Not by
| a long shot.
| aledalgrande wrote:
| Oh yeah? Want an example? Apple.
| https://jobs.apple.com/en-ca/details/200226089/ruby-on-
| rails...
| google234123 wrote:
| That job is for writing tools. So... not that performance
| critical.
| aledalgrande wrote:
| That specific job might be for tools. But open Apple
| Music in your browser and tell me what it is using?
| StreamBright wrote:
| No, I want facts like a reproducible performance test
| like Techempower.
| gosukiwi wrote:
| GitHub uses Rails just fine. If it works for GitHub, it
| will most likely work for >90% of use-cases out there.
| StreamBright wrote:
| Did I write anywhere that Github does not use Rails or
| 90% of internet cannot be run on Rails?
| aantix wrote:
| Your biggest constraint is programmer productivity, not
| server performance.
|
| Your concerns are further diminished ~ every two years.
| Please re-evaluate your assumptions on that timeline.
|
| This position was established by the company that owns
| the one of the largest server farms in the world? Are
| they looking to save on a few x-large instances? Seems
| short-sighted to me.
|
| Sub 50ms response times are very achievable with Rails.
|
| Throwing another server at it is a very reasonable
| response. Especially when you want your team to remain
| productive in a cohesive, batteries included framework.
|
| Plenty of real-world Rails examples to demonstrate this.
| mpweiher wrote:
| Yes, it can be used.
|
| Yes, it is crazy inefficient.
|
| At Wunderlist, we launched WL3 with something like 1K rails
| boxes on AWS, running a few services. During the launch, my
| fun was tracking when we reached user/box parity :-)
|
| But it did work.
|
| After a while, we replaced each of the hundred+ rails boxes
| per service with 2 Scala instances per service.
|
| Also worked. No change for users.
|
| The FP folks should be forever grateful to Ruby for making
| their languages seem fast in comparison.
|
| At another company where I consulted, for a web content
| management system, the rails devs were super excited to get
| the performance up to slightly below one second per
| request.
|
| We got much better than that 2 decades earlier, running a
| CGI-Bin (no fast-cgi, so complete restart on every request)
| that completely re-initalised its Oracle DB connection
| every time etc. On hardware that was a small fraction of
| the performance of today's boxes.
| Toutouxc wrote:
| > the rails devs were super excited to get the
| performance up to slightly below one second per request
|
| If that was the case then Rails was their least important
| problem.
|
| At the prototyping stage back-and-forth with the client I
| write VERY rudimentary Rails code, n+1s be damned, but I
| don't remember ever seeing a second long request.
| ativzzz wrote:
| I guess it just depends on what your app does. At my
| previous job we were serving several million requests/day
| off about 15-20 rails instances on heroku
| CyberDildonics wrote:
| It is a giant red flag to hear someone talk about
| requests per day. A day has 86,400 seconds. If you had 3
| million requests in a day, that's only 35 per second on
| average.
|
| If you had 20 instances, that is under 2 requests per
| second per instance.
|
| I don't know what each request does of course, but it
| might be worth asking if each one should really take a
| billion clock cycles to complete.
| maximegarcia wrote:
| At previous job, we ran a rails based 60MEUR/year photos
| album e-commerce with 6 backend devs and a dedicated
| servers fleet on 2 DC that costed less than one dev
| salary.
| nijave wrote:
| I think ORMs are part of the problem. By abstracting away
| SQL, it becomes really easy to write something that
| generates terribly inefficient SQL without realizing. It's
| easy to not realize it in small test environments with
| small test amounts of data, too.
|
| Even a single n+1 query can really hose performance by
| burdening the entire database.
| viraptor wrote:
| This is a popular trope, but in practice it's just as
| easy to do n+1 via your own abstractions where
| "items.each { |i| i.foo }" happens to do it too. Without
| any ORM involved. Everyone will do it sometimes and the
| question is only whether you're going to spot it before
| it causes an outage. (In many cases it will only cause a
| slight slowdown)
|
| At least there are tools to warn you about n+1
| automatically https://github.com/flyerhzm/bullet
| gosukiwi wrote:
| Nowadays in Rails-land there's lots of tools to manage
| that, [the
| latest](https://blog.saeloun.com/2020/02/25/rails-strict-
| loading-mod...) integrated into Rails itself.
|
| ORMs can be bad, but it's not fair to blame them "just
| because".
| whakim wrote:
| I'm not so sure. My experience is that ORMs often
| discourage (or at least lessen the need for) getting a
| good grasp on databases and SQL because they abstract
| away a lot of fundamentals. Like any abstraction, there
| are obviously benefits too, but I do think it's
| reasonable to say that "makes it too easy to write
| sloppy/unperformant queries" is a real cost.
| whakim wrote:
| I don't know what your definition of a "very low" threshold
| is, but it's much lower than mine. If you're Uber or Amazon,
| Rails isn't going to work for you. But that's fine because
| 99.9% of companies in existence don't fall into that
| category. I realize everyone wants to be the next <insert
| huge company here>, but the reality is a) you probably won't
| be; and b) given that you won't be, stuff like fixing poor
| caching strategies and inefficient queries are going to give
| you an order of magnitude more performance/buck than choosing
| <insert framework/language here> over Rails.
| aeontech wrote:
| 100% agreed. I'd also add c) if you _do_ end up getting
| close to hitting those limitations, you'll also have enough
| resources to rebuild the slow parts (and, chances are,
| you're going to end up rebuilding things five times over
| during your growth and scaling phase anyway, no matter what
| framework you had chosen).
| multiplegeorges wrote:
| It's great to see such low-level efforts happening in the Ruby
| community and sponsored by some of the large companies using
| Ruby.
| mchusma wrote:
| This is great to hear. It seems like in 3.1 JIT will make
| probably sense for rails apps. Every little performance
| improvement helps!
| The_rationalist wrote:
| No mention of TruffleRuby?
| open-source-ux wrote:
| I'm very much in favour of seeing performance as a key part of
| the development of any dynamic, interpreted language. (I single
| out dynamic languages because in most typed, compiled languages,
| fast execution speed comes for free.)
|
| However, David Heinemeier Hansson, co-founder of Basecamp and
| creator or Ruby-on-Rails, has this interesting take on the cost
| of running Ruby for their business. Make of it what you will:
|
| _Only 15% of the Basecamp operations budget is spent on Ruby_
| (2019)
|
| https://m.signalvnoise.com/only-15-of-the-basecamp-operation...
| riffraff wrote:
| I mostly agree with the argument, but it's worth noting that a
| faster runtime may itself results in cutting the other 85% of
| costs, i.e. you don't need to complicate your architecture to
| deal with performance issues, nor hire more ops people to
| manage more servers etc..
| dwohnitmok wrote:
| Yeah, that DHH article has one sentence doing a lot of work:
| "Whether we run on Python, PHP, Rust, Go, C++, or whatever,
| we'd still need databases, we'd still need search, we'd still
| need to store files."
|
| It's not clear that's true (as in yes you'd probably still
| need all of those things at a high level, but would you need
| as much of them and would you need them out of process on
| another machine?). If magically your programming language
| gave you enough performance that a single binary + SQLite
| served all your needs and stuff like caches, message buses,
| and the like were completely unnecessary that'd radically
| change your ops budget.
|
| Of course that is truly magical, but then again there are
| quite a lot of papers that demonstrate well-crafted single
| binary services outperforming heterogenous complex clusters
| with orders of magnitude more compute resources.
| multiplegeorges wrote:
| While I agree that the cost of "Ruby" gets dwarfed when running
| a larger business that uses it, I think it's really important
| to the larger ecosystem around Ruby that this low-level
| research/work happens.
|
| It's also really great to see large companies like Shopify
| invest in such low-level things, and not just on gems/changes
| that directly touch their business.
| davidw wrote:
| Sure, that's a really important point, but if you get it mostly
| 'for free', you can - at the margin - expand the kinds of
| businesses where Ruby makes sense.
___________________________________________________________________
(page generated 2021-05-21 23:01 UTC)