[HN Gopher] The original magic Emacs garbage collection hack (2019)
___________________________________________________________________
The original magic Emacs garbage collection hack (2019)
Author : simonpure
Score : 97 points
Date : 2024-01-30 13:54 UTC (9 hours ago)
(HTM) web link (akrl.sdf.org)
(TXT) w3m dump (akrl.sdf.org)
| untilted wrote:
| Of note, this is a post by Andrea Corallo, the legend who brought
| native compilation to emacs lisp a few years back
| swah wrote:
| Is this the default now ?
| G3rn0ti wrote:
| Not yet. I believe, it is discussed for Emacs 30. Meanwhile
| if you are willing to install Emacs 29 as a snap
| (https://snapcraft.io/emacs) you can enjoy the benefits it
| brings w/o needing to compile your Emacs binary. I use it as
| my daily driver and did not have any issues.
| Tomte wrote:
| Yes, since 28.1, last April.
| untilted wrote:
| It's now the default on the development branch: https://git.s
| avannah.gnu.org/cgit/emacs.git/commit/?id=3328c...
|
| It's _not_ the default in any official release, yet. Although
| plenty of distros have either a separate package enabling it
| (or have enabled it in the main package)
| AeroNotix wrote:
| I use:
|
| (setq gc-cons-threshold most-positive-fixnum)
|
| I have 256GB RAM locally so I might as well use it.
| argiopetech wrote:
| Wow, can you share some more specs?
| AeroNotix wrote:
| It's just a pretty old HP Z820 workstation. Not top of the
| line by any stretch of the imagination.
|
| Here's an older geekbench report:
| https://browser.geekbench.com/v5/cpu/3159963
| dilap wrote:
| i use the same trick (who w/ only 16GB of ram) -- so far so
| good.
| sph wrote:
| Having a lot of RAM doesn't mean you have no need for garbage
| collection. By definition, garbage collected programs
| continuously leak memory, and consume all the available
| resources on the machine.
|
| Even setting it to 256GB is a much better idea than just
| turning off GC. I recommend to maybe halve that to leave just
| enough RAM for Chrome.
| dilap wrote:
| i don't know, my experience in practice is that turning off
| emacs GC has resulting in no noticeable degradation and has
| eliminated GC pauses. so why not do it?
|
| maybe emacs just doesn't use enough memory, relative to
| modern systems, for GC to be worth it.
|
| (i do quit emacs at the end of each day, so that is a form of
| GC.)
| sph wrote:
| The memory consumption of GC languages with GC turned will
| never decrease, and basically keep increasing unless
| completely idle. Efficiency has nothing to do with it.
| conses are generated whether you want it or not, for every
| single line of Lisp executed.
| dilap wrote:
| sure, but if you're not generating enough garbage over
| the lifetime of the app for it to matter, why waste time
| collecting? especially is your collector is an old-school
| design that hangs the app for a user-noticeable amount of
| time...
| AeroNotix wrote:
| Not everything in emacs is written in elisp. I can tell
| you _for sure_ that turning off GC in emacs doesn't
| result in a runaway train of resident memory.
|
| You can be incorrect, or you can try it.
| AeroNotix wrote:
| Right, but if I have so much RAM available, it practically
| doesn't matter.
|
| I've been running emacs since I logged in at 9am, edited ~100
| different files. Many LSP inferior-processes running and
| emacs is currently sitting at 1 whole gigabyte resident
| memory.
|
| What's more is that I would say this is a particularly light
| day for me and emacs.
|
| I have literally never, never, not once ever had an issue
| with this much RAM and essentially turning gc off in emacs.
| gwern wrote:
| Do you ever run out of RAM?
| AeroNotix wrote:
| Never. I rarely get above 170GB RES. Most days below 30GB,
| even.
| throwaway81523 wrote:
| It is or was a traditional mark-sweep gc, so a very large
| collection can be slow even if there is not much live data. The
| gc still has to touch all the garbage to return it to the free
| lists. Also, Emacs itself might slow down from the terrible
| cache locality.
| adchari wrote:
| I've done this for a while, increasing the threshold and
| collecting every time I tab away from an emacs window
|
| (setq gc-cons-threshold 100000000) (add-function :after after-
| focus-change-function 'garbage-collect)
| kazinator wrote:
| The refinement in https://news.ycombinator.com/item?id=39191012
| seems useful.
| thom wrote:
| I personally use this (alongside an extremely high threshold):
| (add-function :after after-focus-change-
| function (lambda () (unless (frame-focus-
| state) (garbage-collect))))
|
| So Emacs GCs whenever it loses focus (i.e. I'm looking at some
| docs in a browser or whatever). This works great for me - it's
| rare that I notice it still cleaning up when I switch back.
| celeritascelery wrote:
| Wait... you switch away from Emacs?
| medstrom wrote:
| Humoristic surprise aside, I think this trick just feels neat
| and does not actually solve the problem when it matters most,
| because even if you often switch between apps, there are
| still going to be times when you don't.
|
| Times when you're just staring at compiler output or
| something heavy like that, and now you need a different GC
| trigger to save the day... and if you have that, then you do
| not need the trick.
| thom wrote:
| All I can say is that never happens to me. I moved to this
| after being annoyed by other approaches, especially idle
| ones where I'd pause to think only for the GC to kick in.
| Whereas if Emacs has lost focus, I'm between thoughts
| anyway so my flow is never interrupted.
| _emacsomancer_ wrote:
| cf. https://github.com/emacsmirror/gcmh (the author's [=Andrea
| Corallo's] Emacs package "GCMH - the Garbage Collector Magic
| Hack")
| kazinator wrote:
| > _this is just a polite way to say: "NEVER garbage collect
| unless we are potentially causing the OS to swap pages."_
|
| It's not straightforward to guess where this threshold is because
| it depends on the behavior of other applications. The OS could
| already be swapping before your process even started.
| GMoromisato wrote:
| AITA for pointing out that we've pushed the "barbarian practice
| of memory management" to the user?
| samus wrote:
| Depends who you ask. I personally rarely noticed memory
| management overhead when using Emacs. If it indeed behaves
| janky, then the ancient defaults are to blame. Automated memory
| management is very difficult in the general case, but the
| strategy from TA sounds very reasonable and sane on modern
| machines: UIs typically have infrequent peaks of high activity
| and long breaks in between.
|
| Would be interesting to see how many users are actually using
| it on computers made in the last century, where the default
| strategy might be more appropriate.
| GMoromisato wrote:
| Yeah, I was trying to be funny.
|
| But it's still interesting to me that this thread is filled
| with users talking about their techniques for forcing a GC
| while they're getting coffee or switching tabs.
|
| The fact that in 2024 we're talking about users (expert
| users, to be fair) changing setting in their apps to optimize
| GC pauses makes me think something has gone wrong somewhere.
|
| But maybe I'm just being mean because I mostly use C++.
|
| Goes to show that it's trade-offs all the way down.
| samus wrote:
| Yes, it is appreciated :-)
|
| As I said, Emacs has many ancient defaults that many people
| have to fiddle with to make the editor theirs. Some of them
| is because of a slow drift of user preferences over the
| generations. But memory management strategy is one area
| where objectively a new default strategy would make a lot
| of sense. One which automatically adapts to RAM size, since
| Emacs scales down so well, unlike its competing memory
| guzzlers.
| daitangio wrote:
| Very clever, anyway I put a very high gc-cons-threshold in my
| .emacs and I never had big issues.
|
| The reality is emacs is complex but today computers with 8GBs of
| ram has about 1000x memory of the original emacs target (a
| machine with 8 MB of RAM) and even faster processors.
|
| Now I am in love with vscode, but I think I need to come back to
| emacs :) sometimes
| sfink wrote:
| I work on the Firefox GC, so I guess I'm biased, but this doesn't
| seem like the greatest idea if you use it exactly as proposed. By
| setting a really high threshold for automatic GC scheduling,
| you're guaranteeing that memory usage is going to grow quite a
| bit as long as you're even slightly active (15 seconds is a long
| time to be idle!), which means (1) whatever application you're
| switching to will have had its pages pushed out and will start
| responding more slowly as it faults them back in, and (2) 15
| seconds into using that application, you'll get a system-wide
| stutter as the emacs GC faults all of its massive pile of pages
| back into physical memory in order to scan through the reference
| graph.
|
| Of course, you'll probably blame both of those on whatever
| application you're using, so in a way from emacs's point of view
| this is all free!
|
| I'm not saying the underlying idea is bad. Using idle time for GC
| is good. 1GB just seems like an awfully high threshold for non-
| idle time. It's giving up on the idea of "small enough to be
| unnoticeable" pauses. (Though the right way to do that would be
| to have an incremental GC, but that has costs of its own.)
|
| Personally, I guess I'd want to try a lower non-idle heap
| threshold, plus a shorter idle time threshold (15 seconds seems
| quite long, and a shorter time increases the probability of the
| GC'd pages still being in RAM), plus using focus as an idle
| signal.
|
| And maybe play with using variable heap size thresholds, eg base
| the trigger off of both how long emacs has been idle plus the
| size of the heap just after the previous collection. So you might
| say that the heap threshold is 100% of the previous heap size at
| 15 seconds, 400% at 5 seconds, 1000% when non-idle. Or min(10x
| previous, 20MB) when non-idle, perhaps, where 20MB is adjusted to
| take an amount of time that is barely noticeable? But that's
| adding complexity. I guess I should enable GC logging to see how
| long these collections typically take; I haven't noticed a lot of
| pauses in the first place.
___________________________________________________________________
(page generated 2024-01-30 23:00 UTC)