[HN Gopher] Ruby 3.3's YJIT: Faster While Using Less Memory
___________________________________________________________________
Ruby 3.3's YJIT: Faster While Using Less Memory
Author : ksec
Score : 161 points
Date : 2023-12-18 18:31 UTC (4 hours ago)
(HTM) web link (railsatscale.com)
(TXT) w3m dump (railsatscale.com)
| apstats wrote:
| I moved from ruby 2.7 to 3.2 for a rails app and was hopeful that
| it would lead to large speedups like shopify claims it did for
| them, but was bummed to find it did basically nothing. Anyone
| else running a large rails app have a similar or different
| experience?
| hartator wrote:
| Yeah, our last benchmark was with 3.1.2 + YJIT, but we actually
| saw a _regression_ in term of performance while RAM usage was
| indeed up: https://serpapi.com/blog/benchmarking-ruby-3-1-yjit-
| ruby-2-7...
| apstats wrote:
| Very interesting blog post! My best guess as to how shopify
| is able to achieve such large increases in speed is they have
| a lot more actual ruby code than most codebases
| (https://shopify.engineering/ruby-yjit-is-production-ready).
| maxime_cb wrote:
| Hope you try again with 3.3. The improvements we've made to
| YJIT since Ruby 3.1 are massive.
| jweir wrote:
| We kept YJIT off for 3.1, but turned it on for 3.2. I am
| working from my memory here, but I recall 3.1 not performing
| well with YJIT.
| jweir wrote:
| We saw memory go up when moving from 3.1 to 3.2, but we also
| removed compiling Ruby with jemalloc and used the defaults. We
| have seen performance increase by about 10% for our web
| applications. Our forking background processes run about the
| same. YJIT appears to perform for repeated requests - ie first
| run will not expose any benefit, ie your tests won't run
| faster.
|
| [edit, the memory increase was small just a few percent IRC]
| adamors wrote:
| I benchmarked a good sized app (ran a black-box QA suite
| multiple times) while monitoring performance and req/response
| times with Prometheus/Grafana with Ruby 3.2.
|
| With YJIT enabled memory usage ballooned and performance dipped
| below non-YJIT Ruby 3.2, IIRC the difference was a good 10%
| degradation. Granted, it's an API only service so no HTML is
| being generated, only JSON and that could be the culprit.
|
| Suffice it to say, we didn't enable YJIT for 3.2. Maybe 3.3 is
| indeed different, but both the faster and less memory claims
| are really suspicious to me.
| apstats wrote:
| Interesting. Do you think html generation is one of the
| things that makes the numbers look so good for 3.2 YJIT? We
| run a mostly json api only app. I think json serialization is
| notoriously slow in ruby so I was hoping yjit would speed it
| up.
| byroot wrote:
| YJIT won't help JSON generation because it's all
| implemented in C (either the stdlib `json`, or `oj` or
| `yajl` etc.
|
| If what is slow is some RUby code like Active Model
| Serializers etc, then maybe it can help a bit there.
|
| But yes, generally YJIT works very well on HTML templates
| because they are compiled as large methods with not a lot
| of branches.
| taf2 wrote:
| Sorry to hear you had that experience we saw 30% speedup moving
| from 2.6 to 3.2 and the 95th in many cases was 40%... Did you
| happen to enable yjit and if so which app server are you
| running?
| JohnBooty wrote:
| Two thoughts.
|
| 1. How did you evaluate performance? Did you let it run for a
| while in production? YJIT's improvements will be most visible
| over time.
|
| 2. If your web app's response times are dominated by database
| queries, then YJIT will do nothing for you. Even re-writing
| your app in assembly language won't help. All languages are
| equally fast when sitting around waiting for the database to
| response. ;-)
|
| That said, if your response times are dominated by db query
| wait times, maybe the async queries in Rails 7.x can offer you
| some very nice improvements. However, you will probably need to
| restructure (not rewrite, exactly, but restructure) your code
| to take advantage. Not the whole app, just the hot spots.
|
| https://www.shakacode.com/blog/rails-7-1-active-record-api-f...
| cntainer wrote:
| Pretty sure it means that Ruby isn't the bottleneck for your
| case.
|
| In most WebApps the first performance bottleneck people tend to
| hit is the DB: missing indexes, n+1 queries, etc.
| mortallywounded wrote:
| I get the need to improve Ruby's speed and memory usage, but
| there's much to be desired. I really enjoy Rails and Ruby in
| general, but the performance is terrible.
|
| I'd like to see how Ruby 3.3 stacks up to other languages and
| runtimes.
| adamors wrote:
| In 2023 if performance is a functional requirement, Ruby is
| still not the answer IMO. For companies like Shopify, there's
| really no other alternative at this point, but their "fast" is
| only fast compared to the performance they've had 1-2 years
| ago.
| avg_dev wrote:
| I used to work with Ruby and Rails and I really found Ruby to be
| a very comfortable, terse, and expressive language, but I found
| it very slow after trying some compiled languages (I think I
| compared service startup and some simple computationally heavy
| stuff like Advent of Code or Project Euler solutions) and I just
| couldn't go back to Ruby after that. Also I found the Rails
| (framework) stacktraces and source code very hard to follow.
|
| However, that was quite some time ago. Does anyone have any more
| recent opinions? How is it to work on a large modern Ruby or
| Rails codebase? Also, I saw some type annotations examples a few
| years back and thought they looked pretty ugly - is that how I
| would feel after some months using them, too?
| culi wrote:
| > Also, I saw some type annotations examples a few years back
|
| You probably saw the third party tool Sorbet. I agree it's ugly
| and extremely limited. Ruby's native typing solution, RBS, is
| written in a separate file. It's a bit better but... it's
| written in a separate file...
|
| Most major companies seem to still primarily rely on Sorbet.
| Yeah I'd say the typing scene in Ruby is still one of it's
| greatest limitations
| jweir wrote:
| Since Ruby 3 I would recommend trying again. It is much faster
| now.
|
| Back in the Ruby 2 days we had tools like Zeus to preload our
| development application because loading was so slow (10-20
| seconds). These days we just load the whole app (2 seconds)
|
| These benchmarks show a 2x speed up. While I haven't see that,
| I have seen at least a 50% increase in performance for our
| applications.
| ecshafer wrote:
| I work at Shopify (not the team that does YJIT) so I am working
| on a very large, modern Rails codebase. (my opinions are my
| own)
|
| I think rails code is the easiest code to follow in any code
| base that I have ever worked on. The enforcement in the
| framework of the code structure, models containing the object
| and schema relationships, focus on message passing, skinny
| controllers. Its very very easy to debug rails code and work in
| code that is relatively foreign due to the consistency across
| the code base. Some of this might be discipline at Shopify, I
| do think we have a very good code review culture. But I have
| been places that rest controllers just led to a spaghetti of
| "Operations" and "Processors" that just change state everywhere
| and a bunch of microservice network calls.
|
| Ruby speed hasn't been an issue. We focus on latency and speed
| quite a bit and work on keeping request times low. The speed of
| ruby running is a good trade off for the speed of development I
| think. And most slow running requests really seem to end up due
| to network io or database calls that I see.
|
| Sorbet uses a kind of type annotation, its very ugly and I do
| not like it. I really would prefer something more built into
| the language. Like def foo(Integer x) -> Integer. The big issue
| is method arguments I think since Ruby already uses :x, x:, x
| =, and a lot of the other symbols that indicate types so its
| hard to add it in an optional fashion.
| graypegg wrote:
| Just curious, what's your opinion on YARD comments for
| typing? I have a little more experience with that, than I do
| Sorbet (or RBS as well) and I found it to be at least
| tolerable. Though something like your `-> Integer` would be
| amazing though if there was somehow a way to make it work
| with the syntax of Ruby.
| ecshafer wrote:
| The RBS syntax I like, I don't like how its a separate file
| though. I haven't used YARD but I think it would have
| similar issues to Sorbet. I much prefer things to be inline
| in programming languages in general. YARD looking at it,
| makes me think of Javadoc, which I always saw the issue
| with it being that it drifted from implementation over time
| as people don't update it. Sorbet does enforce it to be
| updated, and will throw an exception so your build will
| fail. Does YARD do that?
| why-el wrote:
| In my experience most Ruby apps (Rails in particular) develop
| this idea that Ruby is the bottleneck when in reality most of
| the time it's something else entirely (mostly DB or network
| IO). Of course you found compiled languages faster! that's by
| definition. You also tried Ruby on CPU sensitive code bases
| ("Advent of Code or Project Euler solutions"), that's textbook
| "ruby is slower than C". Rails pays dividends in the web and in
| large code bases with many engineers.
| byroot wrote:
| Weird, because if anything I keep hearing people say Ruby
| performance and GVL don't matter because it's all IO anyway.
|
| Which is true to a certain extent only. When you first start
| optimizing a Rails app, it's true that bad queries and N+1 is
| where most time is wasted, but from my experience once you
| clear these out, the IO/CPU ratio really isn't that high,
| generally in the 40-60% IOs range.
| jbverschoor wrote:
| Try Crystal if that's a concern https://crystal-lang.org/
| dcchambers wrote:
| Obviously as an interpreted language, it's never going to be as
| fast as something like C, Rust, or Go. Traditionally the ruby
| maintainers have not designed or optimized for pure speed, but
| that is changing, and the language is definitely faster these
| days compared to a decade ago.
|
| If you like the ruby syntax/language but want the speed of a
| compiled language, it's also worth checking out Crystal[^1].
| It's mostly ruby-like in syntax, style, and developer
| ergonomics.[^2] Although it's an _entirely different language_.
| Also a _tiny_ community.
|
| [1]: https://crystal-lang.org/ [2]: https://crystal-
| lang.org/reference/1.10/crystal_for_rubyists...
| avarun wrote:
| I think Ruby has at this point caught up to modern languages on
| everything except typing. Unfortunately the typing story is so
| poor that it still makes Ruby an incredibly difficult choice to
| justify in 2023.
| jfabre wrote:
| For what use-case? Because I've been developing in ruby for
| over 10 years now and it has been the optimal choice for a ton
| of companies I've worked for. The lack of static typing is a
| feature for a lot of people.
| yokem55 wrote:
| Well, in ruby everything is an object that is a member of some
| class. Those classes have specific properties that only allow
| for specific methods to be applied on them. Ie, integers are
| members of the numeric::integer class, and if you want to use
| that integer as part of a string in output, you do have to
| convert it to a string first using a method defined as such.
|
| So, what exactly do you see as missing in ruby's typing?
| ksec wrote:
| My question is what's next? Without sacrificing (much) memory. We
| know from TruffleRuby that given enough warmups and lots of
| memory it could be multiple times faster than CRuby. And compared
| to Ruby 3.3 YJIT may be 2-3x faster. But it seems the communities
| as a whole cares a lot about memory usage.
|
| We know in order to make JIT more useful we need to move some of
| the Gems from C to Ruby. But other than that do we have anything
| on the table that will significantly speed up Ruby Rails?
| alberth wrote:
| Truffle Ruby is 50-100% faster in most tests.
|
| But uses 5-10x more memory.
|
| Still has way more upside left to gain.
|
| https://railsatscale.com/2023-11-07-yjit-is-the-most-memory-...
| uticus wrote:
| Not to mention the warm-up time - the graph in the referenced
| article is measured in _seconds_ , with max > 20!
|
| Can attest, just pulled TruffleRuby (both native and JVM), it
| is over ten seconds to start a hello world script cold. As
| in, script being run from command line as needed.
|
| These are gains that are only going to make sense for the
| long-running process crowd.
| parentheses wrote:
| It does make sense in the web server context to optimize for
| memory first over compute. This could explain the dynamic you
| called out. Furthermore, the propensity for many popular gems
| to have native portions makes the trade off even more useful.
|
| Regarding JIT being faster, types are the next effective tool
| IMO.
|
| The two drivers I see are:
|
| - JIT has more stack frames to optimize away, allowing more
| optimization - you addressed this one
|
| - JIT has more information to limit possible outcomes,
| therefore finding more opportunities
| schneems wrote:
| > But it seems the communities as a whole cares a lot about
| memory usage.
|
| Matz very much sees Ruby as still a general purpose language
| that still supports scripting as a first tier concern. From
| that vantage bootup and memory are hugely important. Also
| providers like Heroku (I work there) charge for memory and the
| primary way to get parallelism with the GIL/GVL is using
| multiple processes where memory use can multiply rapidly
| (literally).
|
| > speed up Ruby on Rails?
|
| You can look at other communities to get a preview of what
| might come. Ruby now has "object shapes" like node.js which
| will benefit the community more once people start using it,
| enabling warnings and fixing hotspots where instance variables
| are not declared optimally.
|
| Personally, I would like to see rails embrace the power of
| database backed constraints and fallback to using validation
| queries only when the database throws an error on commit. You
| can already saturate a CPU with processes and threads, the
| biggest bottleneck (IMHO) for most rails apps is still the
| database. Counterintuitively, pushing those constraints to the
| database can mean a net decrease in DB computing needed and
| more overall throughput.
|
| Aside from that I think JIT will help do more with fewer
| resources. Which makes sense when the main YJIT sponsor has a
| huge server bill and being able to slash it in half more than
| justifies such long term gambles.
| dalyons wrote:
| I'm curious why you think constraints are such a problem?
| I've worked on quite a few large rails apps, and whilst
| you're not wrong, the extra traffic from checking constraints
| has always been a tiny negligible part of overall db traffic.
| schneems wrote:
| It takes two seconds to add a "validated uniqueness" and
| now for every save or update instead of one SQL command to
| commit the data you've got 2 (multiplied by the number of
| validations). Plus this validation is weak in that it
| cannot protect from actual duplicate data.
|
| These check queries are individually fast but put a
| tremendous load on a running app's database. If you halve
| the number of queries the database has to process you'll
| see more resources to process the slow queries in addition
| to each commit being faster.
|
| It ultimately depends on the read/write patterns of the
| app, but most non-trivial rails apps do a bunch of writing
| AND reading.
|
| My suggestion is less a "this will change the world
| forever" and more a "fairly clear win, we just have to do
| it".
|
| In a prior life I talked Sgrif into implementing it before
| David caused them to drop Active Record maintenance work.
|
| I know the broad strokes but don't have the need or mandate
| to implement it, plus the active record internals are
| pretty gnarly. Tons of modules, very little encapsulation.
| Very hard to know the correct place to put stuff, so stuff
| goes wherever. It's gotten better from some of the work of
| Yehudah, Aaron, and Sage but AFAIK there's no one on core
| who wants to own it right now. Eileen has done a lot of
| work recently, but I don't think it's her primary focus.
| byroot wrote:
| > but AFAIK there's no one on core who wants to own it
| right now.
|
| Active Record is probably the "most owned" part of Rails,
| that's the one with the most core members and committers
| with deep knowledge of it.
|
| As for your feature request, there's
| https://github.com/Shopify/activerecord-
| rescue_from_duplicat..., and we'd like to attempt to
| upstream that capability in Rails next year (but as
| everything it may or may not happen).
| dalyons wrote:
| I guess I've just never seen the combination of lots of
| writes AND lots of validates_uniqueness(or whatever db
| validation) AND the writes are updating the unique fields
| (due to dirty attributes / partial updates). So it's
| never really been an issue.
|
| If that's been a problem for you I totally believe you,
| and everything you say about a fix all makes sense.
| byroot wrote:
| There is still lots of optimizations to make. On the JIT side,
| there is still no inlining, which opens the door to much more
| aggressive optimizations.
|
| But also on memory as well. There was a lot of improvements
| done on the Ruby garbage collector for 3.3, but there's still a
| lot of room for improvement, and a few fine folks at Shopify
| are looking into making the GC swappable so we'd be able to use
| mttk GCs with MRI.
|
| As for the community caring about memory usage, I wonder how
| much of that has to do with the default Heroku dynos. Because
| it's an issue for small scale deployment. Once you scale more,
| you recoup a lot with CoW etc, so it's much less of a worry.
| claudiug wrote:
| can you share on this? :)
| byroot wrote:
| On what, mmTk?
|
| Matt's talk at latest Ruby Kaigi is a good intro:
| https://www.youtube.com/watch?v=chhNDhyPbyc
| uticus wrote:
| Don't forget TruffleRuby's other tricks: parallel code [0] and
| polyglot language interplay [1] [2].
|
| [0] https://www.graalvm.org/latest/reference-manual/ruby/#aim
|
| [1] https://www.graalvm.org/latest/reference-
| manual/ruby/Polyglo...
|
| [2] https://medium.com/graalvm/graalvm-ten-things-12d9111f307d
| dang wrote:
| Recent and related:
|
| _YJIT is the most memory-efficient Ruby JIT_ -
| https://news.ycombinator.com/item?id=38265773 - Nov 2023 (29
| comments)
|
| Also:
|
| _Ruby 3.3 's YJIT Runs Shopify's Production Code 15% Faster_ -
| https://news.ycombinator.com/item?id=37579926 - Sept 2023 (154
| comments)
|
| _Running Ruby 3.2 's YJIT in Production at Discourse_ -
| https://news.ycombinator.com/item?id=35895309 - May 2023 (1
| comment)
|
| _Ruby 3.2's YJIT is Production-Ready_ -
| https://news.ycombinator.com/item?id=34413012 - Jan 2023 (286
| comments)
|
| _Our Experience Porting the YJIT Ruby Compiler to Rust_ -
| https://news.ycombinator.com/item?id=31344065 - May 2022 (89
| comments)
|
| _Ruby YJIT Ported to Rust_ -
| https://news.ycombinator.com/item?id=31094130 - April 2022 (88
| comments)
|
| _Rust YJIT is complete - it passes all the CRuby tests_ -
| https://news.ycombinator.com/item?id=31090084 - April 2022 (1
| comment)
|
| _Ruby: Porting YJIT to Rust_ -
| https://news.ycombinator.com/item?id=29971360 - Jan 2022 (48
| comments)
|
| _Benchmarking CRuby, MJIT, YJIT, JRuby, and TruffleRuby_ -
| https://news.ycombinator.com/item?id=29824076 - Jan 2022 (29
| comments)
|
| _Merge YJIT: an in-process JIT compiler_ -
| https://news.ycombinator.com/item?id=28938446 - Oct 2021 (24
| comments)
|
| _YJIT: Building a New JIT Compiler for CRuby_ -
| https://news.ycombinator.com/item?id=28874283 - Oct 2021 (47
| comments)
|
| _Proposal to Merge YJIT into Ruby_ -
| https://news.ycombinator.com/item?id=28691048 - Sept 2021 (77
| comments)
|
| _YJIT: Building a New JIT Compiler Inside CRuby_ -
| https://news.ycombinator.com/item?id=27371977 - June 2021 (1
| comment)
___________________________________________________________________
(page generated 2023-12-18 23:01 UTC)