[HN Gopher] Fullstaq Ruby: Ruby, optimized for production
___________________________________________________________________
Fullstaq Ruby: Ruby, optimized for production
Author : thunderbong
Score : 218 points
Date : 2021-07-17 13:53 UTC (1 days ago)
(HTM) web link (fullstaqruby.org)
(TXT) w3m dump (fullstaqruby.org)
| sidcool wrote:
| When I read fully open source, bells ring in my mind. I adopted
| Cypress. It advertised open source and community ethos. But when
| delved deep, it many basic features that were paid. They never
| advertised it well in documentation. Once bitten, twice shy
| stepbeek wrote:
| What features are you missing? I've never paid for cypress
| purely because it had what I wanted out of the box. Now I'm
| wondering if I've missed something.
| sidcool wrote:
| Mainly parallel tests run. They have very limited support for
| parallel tests in community version. We need to buy the
| Cypress dashboard license
| FooBarWidget wrote:
| Author of Fullstaq Ruby here. There are absolutely no paid
| features, nor plans to monetize. There are even enough reasons
| to _not_ monetize.
|
| See the FAQ: https://github.com/fullstaq-labs/fullstaq-ruby-
| server-editio...
|
| And the project vision, which is community-based:
| https://www.joyfulbikeshedding.com/blog/2020-05-15-why-fulls...
| mberning wrote:
| Cool project. Not a big fan of the name. Kind of hard to sell to
| a sourcing team. If they called it "Enterprise Ruby" or something
| like that the conversation is going to be much easier.
| sudhirj wrote:
| Phusion Passenger (same team) has lots of enterprise offerings
| with very entierprisey names and prices.
| bdcravens wrote:
| Is it the same team? Obviously it's the same lead developer,
| but I"m unclear the distinction between Phusion and Fullstaq
| sudhirj wrote:
| I'm not entirely sure, but I always thought of Phusion
| Passenger as being a very small team, like Hashicorp.
| pmontra wrote:
| Name taken, see http://www.rubyenterpriseedition.com/
| jmcgough wrote:
| amazing, this hasn't been updated since ruby 1.8
| FooBarWidget wrote:
| Author of both projects here. Yes Ruby Enterprise Edition
| eventually became obsolete because most features found
| their way back to MRI. But that took a few years. During
| those years, Ruby Enterprise Edition made a lot of users
| happy.
| samgranieri wrote:
| can confirm.
| atonse wrote:
| We used REE for a while and it was excellent. Thank you
| for your work on it!
| bdcravens wrote:
| Actually from the same developer. This project is actually
| under a different company; it's not clear why.
| jbverschoor wrote:
| If there are three solutions, you will have to choose which
| one. I think it's an easier sell to the enterprise,
| eventhough all three projects are by the same person
| (passenger, ree, fullstaq)
| bdcravens wrote:
| Fullstaq is a company.
|
| https://fullstaq.com/
| sudhirj wrote:
| For those wondering why on earth this isn't simply a contribution
| to Ruby, the FAQ has a really nice explanation - tldr is that
| this is a variant optimised for the very specific context of
| Linux, x86/64, server environments, with pre compiled and
| packaged binaries, with special memory allocators (jemalloc).
|
| These changes are specific to this context and are unlikely to be
| accepted into the more generalised Ruby core.
|
| Same team as Phusion Passenger, so excellent background.
| [deleted]
| joelbluminator wrote:
| I'm not really buying the FAQ explanation, idk this feels too
| good to be true. Would have loved to see some core team
| discussion on this if there was one.
| FooBarWidget wrote:
| Author of Fullstaq Ruby here. There _have_ been discussions.
| Discussions on jemalloc integration preceded Fullstaq Ruby
| for years. The bug tracker issue on the malloc_trim patch
| went stale. All this can be found on the issue tracker.
|
| I also spoke to Matz during EuRuKo 2019. And this is the
| impression I got.
| joelbluminator wrote:
| Are you referring to this one? https://bugs.ruby-
| lang.org/issues/15667 P.S did Discourse adopt this approach
| ?
|
| Also found a discussion on jemalloc https://bugs.ruby-
| lang.org/issues/14718
| FooBarWidget wrote:
| Yes
| [deleted]
| throwdbaaway wrote:
| My observations after reading https://bugs.ruby-
| lang.org/issues/14718:
|
| - Shyouhei Urabe did a marvelous job of resisting
| pressure from several well-known developers representing
| multiple high profile ruby companies/projects.
|
| - Everyone eventually came to their senses in
| https://bugs.ruby-lang.org/issues/14718#note-89, and
| concluded that the best way forward is to use glibc with
| MALLOC_ARENA_MAX=2.
|
| My personal feeling is that while the "Jemalloc FTW"
| section in http://oldblog.antirez.com/post/everything-
| about-redis-24.ht... might have been correct back in the
| jemalloc 3.x days, a lot has changed on both jemalloc and
| glibc since then. Furthermore, there is also musl in the
| equation these days.
| joelbluminator wrote:
| I tend to be skeptical as well. Why isn't Shopify /
| Github / Heroku adopting this? I'd stay away.
| ksec wrote:
| Dont know about Heroku, but Github, Gitlab, Shopify are
| all using Jemalloc in production. Practically all big
| Rails shop are using it to the point I thought it was
| common knowledge in 2021.
|
| There has been 8 - 10 glibc release since then I am not
| sure if things have improved or not.
| FooBarWidget wrote:
| Many of them _are_ adopting Jemalloc, but they do the
| Jemalloc integration work themselves because they have
| the engineering talent and capacity. Fullstaq Ruby is for
| everybody else: those who can 't, don't want to, or can't
| be bothered to do it themselves. It's for the masses.
| joelbluminator wrote:
| I'm sure Ruby would have adopted it if it would have
| benefitted most users . It's either not benefitting most,
| is gonna break with future changes (like Guilds/Ractors)
| or a combination of the two.
|
| Sam Saffron: "After spending a bit too much time thinking
| about this, I would like to recommend against any
| jemalloc related changes " https://bugs.ruby-
| lang.org/issues/14718#note-89
| FooBarWidget wrote:
| Do you believe copy-on-write friendly garbage collection
| -- which saves ~33% memory, and is essential to
| Passenger's "smart spawning" methode, Unicork's
| preload_app mode, Puma's cluster mode -- is a thing that
| benefits most users?
|
| Besides being the author of Fullstaq Ruby, I am also the
| author of the first copy-on-write-GC-friendliness patch
| for Ruby. Consider the fact that it took ~5 years for the
| copy-on-write-friendly-GC feature to find its way back to
| upstream Ruby.
|
| If you lived in 2008, would you be willing to wait 5
| years in order to save 33% memory?
| joelbluminator wrote:
| You're not really answering to anything I said though.
| Sorry man I get that this is emotional for you, not
| trying to bring you down. You need to see it through my
| eyes; I'm not gonna introduce into my company anything
| that's high risk, what's in it for me? If this goes fine
| I maybe saved some memory, if this goes wrong I'm gonna
| look silly (how do I even justify going this route to
| anyone? it's not like the internet is clear this is the
| way to go; it's quite controversial or even not the way
| to go).
| FooBarWidget wrote:
| I think you're misunderstanding me. I am not selling you
| anything. I make no money off this. I am not invested in
| whether you personally choose to move forward or not. My
| point is different.
|
| Your original statement basically equates quality with
| authority: you believe a project is qualitatively good if
| and only if it's officially blessed by an authority you
| trust. And that belief is fine. Most people either aren't
| able to, or don't care enough to, verify quality
| themselves, so we defer to trusted authorities. Nothing
| wrong with that approach in general.
|
| The thing I'm trying to tell you though, is that in case
| of MRI, there have been multiple past cases when the lack
| of official authority blessing did not in fact mean a
| lack of quality. This isn't hypothetical: there's a
| proven track record. My own track record is proven as
| well.
|
| What is in it for you? Well, reduced memory usage,
| improved performance, faster and lower-hassle
| installation and security patching!
|
| There is always a risk, yes. But all I'm trying to say is
| that the equation "lack of authority blessing == lack of
| quality" is proven not to be true for a Ruby project such
| as this. Whether you want to take the risk of trying it
| out, that's entirely up to you.
| joelbluminator wrote:
| Any source for Shopify / Github / Discourse adopting
| Jemalloc?
| FooBarWidget wrote:
| Source for Github: https://twitter.com/eileencodes/status
| /1289310891914088448
| strzibny wrote:
| You don't need special talent to do that.
|
| I actually have a mention of this in my book on deploying
| Rails (https://deploymentfromscratch.com/). Here's a one
| liner with ruby-install:
|
| $ sudo dnf install -y jemalloc-devel
|
| $ ruby-install 3.0.0 -- --with-jemalloc
|
| Then check you have it:
|
| $ ruby -r rbconfig -e "puts RbConfig::CONFIG['LIBS']"
|
| ... -ljemalloc
| FooBarWidget wrote:
| It's 2021. Installing from source is no longer a low-
| friction, non-scary, default thing to do or something
| that people want to bother with. Users now expect
| binaries and proper package management.
|
| Furthermore, it's not just "dnf install jemalloc-devel".
| The Jemalloc version matters _a lot_. For reasons that
| are not yet clear, significant memory savings are only
| achieved with Jemalloc 3, not with Jemalloc 5. And your
| distribution only ships 1 Jemalloc version. So likely you
| need to compile Jemalloc 3 yourself.
|
| But Jemalloc 3 no longer compiles by default on some
| modern distributions, such as Debian 10. Fullstaq Ruby
| fixes this by patching Jemalloc:
| https://github.com/fullstaq-labs/fullstaq-ruby-server-
| editio...
|
| Another caveat is that LD_PRELOADing Jemalloc is more
| effective than compiling with --with-jemalloc.
|
| Furthermore, _installing_ Ruby is one thing. Keeping it
| security-patched without having to constantly monitor
| things, is a whole other can of worms. Here 's where
| Fullstaq Ruby's second value proposition comes in: we
| supply DEB/RPMs and do the tiny version patching for you,
| you don't have to go through the whole compilation
| process and gem migration process every time there's a
| new Ruby tiny version.
|
| See also this FAQ entry, "Fullstaq Ruby vs LD_PRELOADing
| Jemalloc yourself": https://github.com/fullstaq-
| labs/fullstaq-ruby-server-editio...
| rurban wrote:
| Everybody can do it by themselves. Using jemalloc is trivial.
| Adding the malloc_trim() call to the full GC is trivial. The
| patches are open.
|
| Why MRI ruby likes to keep unused memory pages around is
| only, that they don't like to add that compile-time option
| and probe for jemalloc. It's too trivial to worry
| FooBarWidget wrote:
| Here I explain why it can quickly become not so trivial:
| https://news.ycombinator.com/item?id=27872696
| jashmatthews wrote:
| jemalloc isn't a special allocator. It's basically the FreeBSD
| default.
| actually_a_dog wrote:
| I think this is interesting, but I'm wondering why there isn't a
| corresponding project for Python. It seems like people using
| Python could derive the exact same benefits, so, why doesn't the
| Python version exist of this project?
| karlding wrote:
| _> I 'm wondering why there isn't a corresponding project for
| Python._
|
| But there _are_ various projects for Python.
|
| For example, there's Pyston [0] which was started by Dropbox.
| Or Cinder [1] which was started by Facebook. Or PyPy [2].
|
| [0] https://github.com/pyston/pyston
|
| [1] https://github.com/facebookincubator/cinder
|
| [2] https://www.pypy.org/
| joelbluminator wrote:
| Still, adoption of these solutions seems low (PyPy seems to
| be a JIT as well is it really a corresponding solution?). If
| it magically reduces memory by as much as 30%-50% why isn't
| adoption higher? In other words what are the drawbacks of
| doing this, there must be cons.
| FooBarWidget wrote:
| Author of Fullstaq Ruby here. From what I understand by
| talking to people, the reasons why Fullstaq Ruby doesn't
| have wider adoption are as follows:
|
| It's not a project officially blessed by Ruby core.
| Upstream Ruby is actually "good enough" for most. Most
| people don't feel the pain enough to try something which
| appears as unofficial. Perhaps their thinking are similar
| to yours: I'm hesitant because not everbody else is already
| using this.
|
| I want to make this an official project. But before I do, I
| want to make sure that the project is sufficiently ready:
| healthy, democratized, not reliant on myself:
| https://www.joyfulbikeshedding.com/blog/2020-05-15-why-
| fulls...
|
| I am also not marketing this as hard as I should. I lack
| free time.
|
| Another reason is because of the beta label. The only
| reason why the label is there is because I have a specific,
| ambitious vision on how "final" should look like (i.e. more
| features), not because it's unstable. Feedback from users
| regarding stability has been very good so far. I might be
| shooting myself in the foot marketing-wise by not removing
| the beta label earlier, but I'd rather do a good job than
| to hurry.
| wokwokwok wrote:
| You understand this is mostly ruby using jemalloc and some
| other memory allocation tricks?
|
| Support for doing this in python was added in PEP-445
| (https://www.python.org/dev/peps/pep-0445/) and implementation
| details can be found here from 2013
| (https://bugs.python.org/issue3329). For more info see:
| https://docs.python.org/3/c-api/memory.html
|
| You can see some commentary on using malloc-trim here too ->
| https://www.softwareatscale.dev/p/run-python-servers-more-ef...
|
| ...long story short: basically, you can already, but it doesn't
| really make much difference for most workloads.
|
| Or, if your question is, why doesn't python get optimisation
| love and become fast like the javascript runtimes?
|
| dunno. I guess we just don't have any big FANG companies
| pouring money on that, because they don't have a use for a
| faster python. A faster python does not make numpy, tensorflow
| or pytorch run faster because those are not written in python.
|
| So I guess, there's a lack of value-for-effort in trying to
| 'fix' the problem of 'pure' python being slow.
| jsmeaton wrote:
| FYI Microsoft is funding Guido and a small team to
| specifically work on performance see
| https://m.slashdot.org/story/385526
| karlding wrote:
| This [0] is the announcement from the PSF.
|
| The team also includes core developers Eric Snow and Mark
| Shannon (of the faster-cpython [1] proposal).
|
| [0] https://pyfound.blogspot.com/2021/05/the-2021-python-
| languag...
|
| [1] https://github.com/markshannon/faster-cpython
| snowmaker wrote:
| Wow, I'm really excited about this. Running a large rails
| website, we spent egregious amounts of money getting EC2
| instances with extra ram. If they can fulfill their promise of
| reducing memory usage by 50%, that'll save us tens of thousands
| of dollars.
| rashkov wrote:
| This is neat. There's a good FAQ explaining, for instance, why
| they don't upstream these changes to ruby core.
| https://github.com/fullstaq-labs/fullstaq-ruby-server-editio...
| carl_dr wrote:
| To clarify for casual readers who might not dig into that, it's
| not that Fullstaq is refusing to contribute or that they are
| commercial. They say :
|
| "There will be no paid version. Fullstaq Ruby is fully open
| source. It is also intended to be a community project where
| anyone can contribute. There are no monetization plans."
|
| Thought I would mention that in case the parent's comment
| implied something different. It's more that the core Ruby team
| are reluctant to.
| rapsey wrote:
| Time spent on dealing with optimisations is un under counted
| productivity sink of using languages like Ruby/Python.
| joelbluminator wrote:
| Thank you wise one for showing us the light.
| powerlogic31 wrote:
| ruby on rails! the horror
| stanislavb wrote:
| Has anyone installed fullstaq ruby with ASDF?
| FooBarWidget wrote:
| Anyone who wants asdf support, vote for it here:
| https://github.com/fullstaq-labs/fullstaq-ruby-server-editio...
| augstein wrote:
| Would be a perfect fit, if this was available on Heroku.
|
| Memory bloat is a real problem, even for mid-size Rails apps.
|
| But it might not be in their interest, if customers are suddenly
| able to downgrade to cheaper Dynos with less memory, so we'll
| see.
| zrail wrote:
| You can run whatever you want in Heroku. Fork the official Ruby
| buildoack and teach it to install fullstaq, it's pretty easy to
| work on.
| wlll wrote:
| You can already use jemalloc on Heroku, it's really simple, so
| this comment is about releasing memory back to the OS, the
| other headline feature of Fullstaq.
|
| > Memory bloat is a real problem, even for mid-size Rails apps.
|
| Yes, but the solution isn't releasing the memory back to the
| OS, that's just papering over the cracks. If you're running
| (say) 10 Rails processes on a machine with 10 GB RAM, and
| particular request paths/background jobs whatever pushes the
| process size to 2 GB RAM size, you can release the memory back
| to the OS after you're done, but you still have the underlying
| problem which is the process gets to that size in the first
| place, and if all the processes hit 2GB at once, now you're 10
| GB in debt, either in swap, or OOM errors.
|
| The solution you should be looking for is installing something
| like Scout (or NewRelic, but IMO Scout is better for memory
| stuff), looking to see what is bloating the processes, and in
| 99% of the time fixing it using some the Rails' batch find
| methods (seriously, it's ActiveRecord objects 99% of the time,
| bonus points for in-memory CSV reading or write buffers).
|
| Rails applications don't _have_ to balloon in memory.
| [deleted]
| FooBarWidget wrote:
| > If you're running (say) 10 Rails processes on a machine
| with 10 GB RAM, and particular request paths/background jobs
| whatever pushes the process size to 2 GB RAM size, you can
| release the memory back to the OS after you're done, but you
| still have the underlying problem which is the process gets
| to that size in the first place, and if all the processes hit
| 2GB at once, now you're 10 GB in debt, either in swap, or OOM
| errors.
|
| Fullstaq Ruby author here. I have done research on the memory
| bloat problem[1], and this statement is not true. With the
| way memory allocation in glibc's ptmalloc2 works,
| multithreaded processes can reach bloaty proportions even if
| they don't do much work. I started investigating this whole
| issue because I had a trivial Ruby HTTP proxy server: it had
| no reason to use 1.3 GB of memory, it doesn't do anything
| besides forwarding small requests. That's when I found that
| ~1 GB of that 1.3 GB of memory wasn't even due to anything
| that my app did by itself, but it's just how the memory
| allocator works.
|
| [1] https://www.joyfulbikeshedding.com/blog/2019-03-14-what-
| caus...
| wlll wrote:
| Did you check to see what happened when you used jemalloc?
| FooBarWidget wrote:
| The bloat went away and memory usage became much more
| "normal".
| wlll wrote:
| That's what we see in our app, and I used to see all the
| time in others (I was a performance and scaling
| consultant for years). It's trivial to use jemalloc on
| heroku, so you may as well, and after that it's mostly
| preventing too much loading of AR objects.
| jrochkind1 wrote:
| > It's trivial to use jemalloc on heroku
|
| Can you share the method you are using to do it? Not
| challenging, actually answering. Googling I'm not sure
| which thing I'm finding is the trivial one you
| use/recommend.
| wlll wrote:
| Sure thing! This is what we use:
| https://elements.heroku.com/buildpacks/gaffneyc/heroku-
| build...
| sealjam wrote:
| It does say there's a heroku edition "coming soon"
| rubyn00bie wrote:
| Is this like a fancy new modern version REE (Ruby Enterprise
| Edition)? It's got that vibe.
|
| Edit: yep, exactly that.
| jbotdev wrote:
| Maybe I'm missing something, but it seems the only changes here
| over upstream are jamlloc and/or malloc_trim. I'm not as familiar
| with malloc_trim, but jemalloc is fairly trivial to setup without
| recompiling Ruby (using LD_PRELOAD), and there are even
| buildpacks for environments like Heroku. It seems like more time
| was spent on the marketing page than on the actual optimizations.
| taf2 wrote:
| At least 10 years ago it was normal or at least in my circles
| to always compile your own Ruby with all the production
| settings needed... jemalloc ... we just build our own rpms and
| it's pretty easy then to roll out upgrades too ...
| sprite wrote:
| Thanks I didn't know this was possible. I'm currently running
| Ruby compiled with jemalloc in production on elastic beanstalk
| with a custom AMI. If I can swap back to their AMI and just use
| LD_PRELOAD it will make upgrades a lot easier.
| FooBarWidget wrote:
| Here I explain why it can quickly become not so trivial:
| https://news.ycombinator.com/item?id=27872696
|
| The goal of Fullstaq Ruby is to _democratize_ the fight against
| Ruby memory bloat. Democratization means that as many people
| should be able to reap the benefits as possible. It 's 2021
| now, and expecting users to compile Ruby or Jemalloc from
| source is no longer realistic. Compiling anything is no longer
| a non-scary, low-friction thing to do.
| wgjordan wrote:
| The parent comment noted that using jemalloc for Ruby
| _without_ compiling is already trivially easy, so the
| argument that 'compiling is hard' is irrelevant even if
| true.
|
| The real argument is that installing jemalloc separately and
| running Ruby with an environment variable is too hard.
| FooBarWidget wrote:
| In my comment, as well as in the FAQ, I explain why even
| using Jemalloc _without_ compiling Ruby has its own
| caveats.
|
| The Jemalloc version matters a lot. For reasons that are
| not yet clear, significant memory savings are only achieved
| with Jemalloc 3, not with Jemalloc 5. Your distribution
| only ships one Jemalloc version. So likely you need to
| compile Jemalloc 3 yourself. Here you are already entering
| compilation land.
|
| But Jemalloc 3 no longer compiles by default on some modern
| distributions, such as Debian 10. Fullstaq Ruby fixes this
| by patching Jemalloc for you.
|
| Furthermore, which Ruby binaries are you using? The ones
| provided by the Linux distribution are perpetually
| outdated. Another of Fullstaq Ruby's value proposition is
| that we supply binaries for the latest Ruby version,
| quickly. We packaged Ruby 3.0 on the same day it came out.
|
| "Fullstaq Ruby vs LD_PRELOADing Jemalloc yourself":
| https://github.com/fullstaq-labs/fullstaq-ruby-server-
| editio...
| wgjordan wrote:
| Thanks, these extra details are helpful and seem like the
| more significant motivations underlying this
| distribution.
|
| > For reasons that are not yet clear, significant memory
| savings are only achieved with Jemalloc 3, not with
| Jemalloc 5.
|
| So the _real_ advantage of this package is that it
| bundles a 6+ year-old, unsupported version of Jemalloc,
| because the more recent versions found in current OS
| distributions don 't yield memory savings in practice-
| _for unknown reasons_. This doesn 't instill very much
| confidence.
|
| I would be much more excited by efforts to investigate
| the jemalloc > 3.x changes so Ruby can work optimally
| with current releases packaged in modern Linux
| distributions, rather than double-down on a workaround
| that requires bundling an increasingly-ancient version of
| the software.
|
| I should also add - as mentioned by the jemalloc author
| [1], the addition of the time-based purging feature is
| likely responsible for memory-usage differences between
| jemalloc 3.x and 5.x, so you can reduce `dirty_decay_ms`
| and `muzzy_decay_ms` to get 3.x-like memory usage. I have
| been using this configuration in production since 2018
| for significant memory savings in Ruby using jemalloc
| 5.x.
|
| [1] https://bugs.ruby-lang.org/issues/14718#note-86
| FooBarWidget wrote:
| I think that is an overly cautious take on things. I'll
| explain why.
|
| First, "for unknown reasons" deserves more nuance. The
| vague, high-level reason is clear: Jemalloc 3 behaves
| differently from Jemalloc 5, having different algorithms
| and data structures. What I mean by unknown is not so
| much an indication of incomprehensible arcane magic, and
| that things can collapse at any time.
|
| What I mean is that it's not known in what way the
| algorithms and data structures are different. Consider
| that before I did my 2019 research on why Ruby memory
| bloating occurs[1], Ruby apps suffered from memory bloat
| "for unknown reasons". That didn't mean that before 2019,
| all Ruby apps were houses of cards waiting to fall over.
|
| It's like saying "I don't understand why this Linux
| kernel upgrade made things faster" -- the kernel
| developers know but they have better things to do than to
| answer your questions. And the fact that knowledge about
| a new optimization in the Linux kernel is not widespread,
| does not mean that that kernel version is unstable.
|
| Nobody truly understands every single detail about all
| parts of the stack. Yet I can build reliable, high-
| available web apps just fine without understanding how
| for example how 5G works and why users on 5G can access
| my app faster than on 4G.
|
| The differences between Jemalloc 3 and 5 are not
| explicitly documented anywhere, and to find out requires
| research. I intend on doing that some time in the future,
| but not now. Jemalloc 3 is proven to work, it's proven to
| be stable. The combination of Ruby + Jemalloc is proven
| to work well, not only because we've had several years of
| user feedback now, but also because Github has tested
| this combination for years now even before Fullstaq Ruby.
|
| The pragmatic thing to do is not to prioritize figuring
| out exactly how Jemalloc 5 works. It's to continue the
| packaging work to make Ruby + Jemalloc 3 available to the
| public. Jemalloc 5 can wait.
|
| [1]
| https://www.joyfulbikeshedding.com/blog/2019-03-14-what-
| caus...
| wgjordan wrote:
| Aside- thanks for engaging in detailed discussion here
| and more generally for all of the Ruby-performance
| contributions you've made over many years, I truly
| appreciate them.
|
| First, my lack of confidence in depending on Jemalloc 3
| in production is not only the 'unknown reasons'
| underlying such a frozen dependency, but also due to the
| fact that this particular dependency is over six years
| old and unmaintained. Not only does this lack more recent
| security/bug fixes and features, but also makes it more
| complex to integrate with up-to-date Linux distributions
| (e.g., your need to maintain custom compilation patches
| instead of simply depending on the OS's jemalloc
| package).
|
| > What I mean is that it's not known in what way the
| algorithms and data structures are different. [...] The
| differences between Jemalloc 3 and 5 are not explicitly
| documented anywhere, and to find out requires research.
|
| As I mentioned, the jemalloc developer already
| highlighted the exact differences back in 2018, and he
| even provided a MALLOC_CONF environment variable to use
| that makes memory usage in jemalloc 5 behave like
| jemalloc 3:
|
| > You could verify this by setting dirty decay and muzzy
| decay to 0 in the MALLOC_CONF environment variable (i.e.
| MALLOC_CONF="dirty_decay_ms:0,muzzy_decay_ms:0", unless
| I've typoed something).
|
| There is also a very readable page on performance tuning
| in the jemalloc 5 documentation [1].
|
| In your research, have you ever tried running jemalloc 5
| with this configuration? I did this back in 2018,
| tested/verified against my production workload, and have
| been running Ruby on jemalloc 5 without any issues since.
|
| All that's involved is installing your OS's 'jemalloc'
| package and setting two environment variables (LD_PRELOAD
| and MALLOC_CONF). Simple enough and more confidence-
| inspiring than maintaining a patch against a six-year-old
| frozen dependency if you ask me.
|
| > The pragmatic thing to do is not to prioritize figuring
| out exactly how Jemalloc 5 works. It's to continue the
| packaging work to make Ruby + Jemalloc 3 available to the
| public. Jemalloc 5 can wait.
|
| I disagree about the relative priorities- I spent a day
| tuning Jemalloc 5 for my team's Ruby application back in
| 2018 [2] and it's been a done issue for us since then.
|
| [1]
| https://github.com/jemalloc/jemalloc/blob/dev/TUNING.md
|
| [2] https://github.com/code-dot-org/code-dot-
| org/pull/24676#issu...
| IfOnlyYouKnew wrote:
| I don't know if 2021 is much different than, say, 2012, in
| that I don't mind compiling something as long as it works?
| I'm sure there are intricacies with ruby's configuration,
| specifically. And that you invested time and have a good
| understanding of it.
|
| But I'm not convinced it's a _product_ and not a _service_ ,
| or a _task_ for someone on staff for larger organisations.
|
| As but one thing: to be a product it needs to have some
| universal value for a significant chunk of scenarios
| (possibly with some configuration options). But at that
| point, it's not clear why it shouldn't be a contribution to
| the upstream project. Is there, possibly, an "Enterprise
| Edition" or "Cloud Deployment Pro Plan" in the works?
|
| I'm also (and that's neither here or there but I can't help
| myself) not entirely sure how democracy got involved here.
| While the rule of law and a participatory citizenry do tend
| to lead to obvious benefits, "reaping" evokes a one-way
| process that is sorta antithetical to the cooperative nature
| of democracies? Dunno, maybe it's just me. It's also on some
| buzzword bingo cards, although I can't quite say in what
| specific context...
| FooBarWidget wrote:
| > But I'm not convinced it's a product and not a service,
| or a task for someone on staff for larger organisations.
|
| From my experience with users and the industry at large,
| the amount of people who are comfortable with compilation
| and sysadmin-y tasks are rapidly declining, proportion-
| wise. The industry is heading towards ever-more
| specialization. Many many backend developers nowadays don't
| want to think about infrastructure at all, they just want
| to focus on business logic. There are a huge amount of
| backend developers who have never seen './configure && make
| install'.
|
| In an organization with sufficiently advanced human
| capital, yes there is someone who can take care of that.
| But the existance of such a person can be taken less and
| less for granted nowadays even in very large organizations.
|
| There is also the factor of: _should_ we do this work
| ourselves? Lots and lots of developer tooling nowadays are
| extremely slick. We 've been spoiled. _I_ have been
| spoiled. I can still write C++ but I don 't want to bother
| with './configure && patch && make install' anymore. The
| standard nowadays is higher. Why should I spend a day
| installing a custom-patched Ruby when someone else can do
| that for me and all I have to do is to add an APT repo?
| Especially when I always have better things to do?
|
| > it's not clear why it shouldn't be a contribution to the
| upstream project.
|
| This is explained in the FAQ: https://github.com/fullstaq-
| labs/fullstaq-ruby-server-editio...
|
| There is also a discussion here:
| https://news.ycombinator.com/item?id=27870740
|
| > Is there, possibly, an "Enterprise Edition" or "Cloud
| Deployment Pro Plan" in the works?
|
| There is not. There are absolutely no paid features, nor
| plans to monetize. There are even enough reasons to not
| monetize.
|
| See the FAQ: https://github.com/fullstaq-labs/fullstaq-
| ruby-server-editio...
|
| And the project vision, which is community-based:
| https://www.joyfulbikeshedding.com/blog/2020-05-15-why-
| fulls...
|
| > I'm also (and that's neither here or there but I can't
| help myself) not entirely sure how democracy got involved
| here.
|
| Democratization, not democracy. Democratization is about
| making something available to as many people as possible.
| Which is a different concept from democracy.
|
| Given that so many people are uncomfortable with compiling,
| or with LD_PRELOAD, or with anything outside of "bundle
| install", asking people to "just compile Jemalloc 3, make
| sure to apply this patch, then modify your systemd init
| script to include LD_PRELOAD" is too much to ask. It shuts
| down an entire range of people from benefiting from
| Jemalloc. With democratization, I seek to combat this.
| jbotdev wrote:
| It seems like the main value-add of this project is
| building/packaging pretty much standard Ruby in a consistent
| way across many Linux distributions, which is something
| that's been hard to come by. I would lean into that more than
| anything else.
| faebi wrote:
| Do you have any plans to support ruby-install and therefore
| chruby?
| FooBarWidget wrote:
| No. Only rbenv. Unless there's a lot of user request.
| FooBarWidget wrote:
| Update: if you want asdf support, vote for it here:
| https://github.com/fullstaq-labs/fullstaq-ruby-server-
| editio...
| jbverschoor wrote:
| I recently switched to asdf. Big time rbenv user, then added
| nodenv. I also needed multiple yarn versions. As asdf is
| based off / inspired by rbenv, it was an easy change.
| busterarm wrote:
| I'm using asdf for everything I can. Even using it in
| CI/CD. Wonderful tool.
| jrochkind1 wrote:
| It's possible you can already easily use with chruby by just
| pointing chruby at it? I haven't tried, I am interested if
| anyone wants to report back!
|
| https://github.com/postmodern/chruby#rubies-1
|
| (You don't need to use ruby-install with chruby, you can use
| whatever method of installing rubies you want. The chruby
| README even includes instructions for `ruby-build`. But
| normally whatever method you use, you install to `/opt/rubies`
| or `~/.rubies` which is where chruby will look. But fullstaq
| ruby insists on installing to it's own locations. But possibly
| chruby will be fine with that if you just tell it where that
| is... but you might need to do it for each ruby you install,
| which woudl be annoying. You could probably write a little bash
| shim which just checked the fullstaq install location for which
| rubies it had, and then just added them each individually?)
___________________________________________________________________
(page generated 2021-07-18 23:02 UTC)