Post AqpAWLk2PjrPyE35G4 by mmstick@fosstodon.org
 (DIR) More posts by mmstick@fosstodon.org
 (DIR) Post #AqpAWLk2PjrPyE35G4 by mmstick@fosstodon.org
       2025-02-05T15:34:16Z
       
       1 likes, 0 repeats
       
       PSA to #rustlang and #linux developers: there is a long-standing 20y bug in the system allocator (glibc malloc) which causes it to hoard large sbrk buffers in arenas. By default, it uses heuristics to dynamically increase the mmap threshold—the point where it switches from using sbrk to mmap—which causes a memory "leak". Some #libcosmic apps were affected by this bug, causing them to use 10-30x more memory than they need. See the PR below for how to tame malloc in Rust:https://github.com/pop-os/cosmic-bg/pull/73
       
 (DIR) Post #AqpAWMybp36tnhsEbo by BrodieOnLinux@mstdn.social
       2025-02-06T00:00:08Z
       
       0 likes, 0 repeats
       
       @mmstick You mention this is a bug in the system allocator, is there an upstream report I can see about that.
       
 (DIR) Post #AqpAWPgHldEgBkK8xs by mmstick@fosstodon.org
       2025-02-05T15:35:00Z
       
       0 likes, 0 repeats
       
       Alternatively, if you experience this bug in your application, you can switch the global allocator from glibc to jemalloc or mimalloc.
       
 (DIR) Post #AqpCzwAAHMZ7EfVOy0 by mmstick@fosstodon.org
       2025-02-06T00:27:51Z
       
       0 likes, 0 repeats
       
       @BrodieOnLinux There are a handful of related bug reports on their tracker.https://sourceware.org/bugzilla/show_bug.cgi?id=14827https://sourceware.org/bugzilla/show_bug.cgi?id=26969https://sourceware.org/bugzilla/show_bug.cgi?id=30769https://sourceware.org/bugzilla/show_bug.cgi?id=15321You'll find a lot of blog articles on the subject, too.https://www.algolia.com/blog/engineering/when-allocators-are-hoarding-your-precious-memoryhttps://blog.cloudflare.com/the-effect-of-switching-to-tcmalloc-on-rocksdb-memory-use/https://www.linkedin.com/blog/engineering/infrastructure/taming-memory-fragmentation-in-venice-with-jemallochttps://www.softwareatscale.dev/p/run-python-servers-more-efficientlyhttps://codearcana.com/posts/2016/07/11/arena-leak-in-glibc.htmlMany simply result to switching to jemalloc since it handles allocations across multiple threads more efficiently.
       
 (DIR) Post #AqpIz6Q7FyeWU1x4hE by BrodieOnLinux@mstdn.social
       2025-02-06T01:34:59Z
       
       0 likes, 0 repeats
       
       @mmstick Awesome thanks for the links
       
 (DIR) Post #AqxhkEhdzhLXiEuleq by pernia@cum.salon
       2025-02-10T02:50:06.075901Z
       
       0 likes, 1 repeats
       
       @mmstick cc @Suiseiseki defend this