[HN Gopher] Bzip2 crate switches from C to 100% Rust
___________________________________________________________________
Bzip2 crate switches from C to 100% Rust
Author : Bogdanp
Score : 322 points
Date : 2025-06-17 20:06 UTC (1 days ago)
(HTM) web link (trifectatech.org)
(TXT) w3m dump (trifectatech.org)
| dralley wrote:
| How realistic is it for the Trifecta Tech implementation to start
| displacing the "official" implementation used by linux distros,
| which hasn't seen an upstream release since 2019?
|
| Fedora recently swapped the original Adler zlib implementation
| with zlib-ng, so that sort of thing isn't impossible. You just
| need to provide a C ABI compatible with the original one.
| masfuerte wrote:
| They do provide a compatible C ABI. Someone "just" needs to do
| the work to make it happen.
| rlpb wrote:
| > You just need to provide a C ABI compatible with the original
| one.
|
| How does this interact with dynamic linking? Doesn't the
| current Rust toolchain mandate static linking?
| sedatk wrote:
| No. https://doc.rust-
| lang.org/reference/linkage.html#r-link.dyli...
| arcticbull wrote:
| Rust lets you generate dynamic C-linkage libraries.
|
| Use crate-type=["cdylib"]
| nicoburns wrote:
| Dynamic linking works fine if you target the C ABI.
| timeon wrote:
| You can use dynamic linking in Rust with C ABI. Which means
| going through `unsafe` keyword - also known as 'trust me
| bro'. Static linking directly to Rust source means it is
| checked by compiler so there is no need for unsafe.
| bluGill wrote:
| Rust cannot dynamic link to rust. It can dynamic link to C
| and be dynamicly linked by C - if you combine the two you can
| cheat but it is still C that you are dealing with not rust
| even if rust is on both sides.
| mjevans wrote:
| It might help to think of it as two IPC 'servers' written
| in rust that happen to have the C ABI interfaces as their
| communication protocol.
| filmor wrote:
| Rust can absolutely link to Rust libraries dynamically.
| There is no _stable_ ABI, so it has to be the same compiler
| version, but it will still be dynamically linked.
| conradev wrote:
| Rust importing Rust must be statically linked, yes. You can
| statically link Rust into a dynamic library that other
| libraries link to, though!
| alxhill wrote:
| The commenters below are confusing two things - Rust binaries
| can be dynamically linked, but because Rust doesn't have a
| stable ABI you can't do this across compiler versions the way
| you would with C. So in practice, everything is statically
| linked.
| eru wrote:
| Static linking also produces smaller binaries and lets you
| do link-time-optimisation.
| emidln wrote:
| Static linking doesn't produce smaller binaries. You are
| literally adding the symbols from a library into your
| executable rather than simply mentioning them and letting
| the dynamic linker figure out how to map those symbols at
| runtime.
|
| The sum size of a dynamic binary plus the dynamic
| libraries may be larger than one static linked binary,
| but whether that holds for more static binaries (2, 3, or
| 100s) depends on the surface area your application uses
| of those libraries. It's relatively common to see certain
| large libraries only dynamically linked, with the build
| going to great lengths to build certain libraries as
| shared objects with the executables linking them using a
| location-relative RPATH (using the $ORIGIN feature) to
| avoid the extra binary size bloat over large sets of
| binaries.
| IshKebab wrote:
| Static linking _does_ produce smaller binaries when you
| bundle dependencies. You 're conflating two things -
| static vs dynamic linking, and bundled vs shared
| dependencies.
|
| They are often conflated because you _can 't_ have shared
| dependencies with static linking, and bundling
| dynamically linked libraries is uncommon in FOSS Linux
| software. It's very common on Windows or with commercial
| software on Linux though.
| guappa wrote:
| You know how the page cache works? Static linking makes
| it not work. So 3000 processes won't share the same pages
| for the libc but will have to load it 3000 times.
| mandarax8 wrote:
| You can still statically link all your own code but
| dynamically link libc/other system dependencies.
| guappa wrote:
| Not with rust...
| tialaramex wrote:
| I wonder what happens in the minds of people who just
| flatly contradict reality. Are they expecting others to
| go "OK, I guess you must be correct and the universe is
| wrong"? Are they just trying to devalue the entire
| concept of truth?
|
| [In case anybody is confused by your utterance, yes of
| course this works in Rust]
| guappa wrote:
| Can you run ldd on any binary you currently have on your
| machine that is written in rust?
|
| I eagerly await the results!
| tux3 wrote:
| I mean, sure, but what's your point?
|
| Here's nu, a shell in Rust: $ ldd
| ~/.cargo/bin/nu linux-vdso.so.1
| (0x00007f473ba46000) libssl.so.3 =>
| /lib/x86_64-linux-gnu/libssl.so.3 (0x00007f47398f2000)
| libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3
| (0x00007f4739200000) libgcc_s.so.1 =>
| /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f473b9cd000)
| libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
| (0x00007f4739110000) libc.so.6 =>
| /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4738f1a000)
| /lib64/ld-linux-x86-64.so.2 (0x00007f473ba48000)
| libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1
| (0x00007f473b9ab000) libzstd.so.1 =>
| /lib/x86_64-linux-gnu/libzstd.so.1 (0x00007f4738e50000)
|
| And here's the Debian variant of ash, a shell in C:
| $ ldd /bin/sh linux-vdso.so.1
| (0x00007f88ae6b0000) libc.so.6 =>
| /lib/x86_64-linux-gnu/libc.so.6 (0x00007f88ae44b000)
| /lib64/ld-linux-x86-64.so.2 (0x00007f88ae6b2000)
| guappa wrote:
| Well seems I was wrong about linking C libraries from
| rust.
|
| The problem of increased RAM requirements and constant
| rebuilds are still very real, if only slightly less big
| because of dynamically linking C.
| bronson wrote:
| That would have been a good post if you'd stopped at the
| first paragraph.
|
| Your second paragraph is either a meaningless observation
| on the difference between static and dynamic linking or
| also incorrect. Not sure what your intent was.
| guappa wrote:
| Why do facts offend you?
| whytevuhuni wrote:
| I'm genuinely curious now, what made you so convinced
| that it would be completely statically linked?
| IshKebab wrote:
| I think people often talk about Rust only supporting
| static linking so he probably inferred that it _couldn
| 't_ dynamically link with anything.
|
| Also Go does produce fully static binaries on Linux and
| so it's at least reasonable to incorrectly guess that
| Rust does the same.
|
| Definitely shouldn't be so confident though!
| CamouflagedKiwi wrote:
| Go may or may not do that on Linux depending what you
| import. If you call things from `os/user` for example,
| you'll get a dynamically linked binary unless you build
| with `-tags osusergo`. A similar case exists for `net`.
| guappa wrote:
| go by default links libc
| IshKebab wrote:
| Kind of off-topic. But yeah it's a good idea for
| operating systems to guarantee the provision of very
| commonly used libraries (libc for example) so that they
| can be shared.
|
| Mac does this, and Windows pretty much does it too. There
| was an attempt to do this on Linux with the Linux
| Standard Base, but it never really worked and they gave
| up years ago. So on Linux if you want a truly portable
| application you can pretty much only rely on the system
| providing very old versions of glibc.
| guappa wrote:
| The standard library is the whole distro :)
|
| It's hardly a fair comparison with old linux distros when
| osx certainly will not run anything old... remember they
| dropped rosetta, rosetta2, 32bit support, opengl... (list
| continues).
|
| And I don't think you can expect windows xp to run
| binaries for windows 11 either.
|
| So I don't understand why you think this is perfectly
| reasonable to expect on linux, when no other OS has ever
| supported it.
|
| Care to explain?
| guappa wrote:
| Static linking produces huge binaries, it lets you do LTO
| but the amount of optimisation you can actually do is
| limited by your RAM. Static linking also causes the
| entire archive to need constant rebuilds.
| TuxSH wrote:
| You don't _need_ LTO to trim static binaries (though LTO
| will do it), `-ffunction-sections -fdata-sections` in
| compiler flags combined with `--gc-section` (or
| equivalent) in linker flags will do it.
|
| This way you can get small binaries with readable
| assembly.
| connicpu wrote:
| Specifically, the rust dependencies are statically linked.
| It's extremely easy to dynamically link anything that has a
| C ABI from rust.
| quotemstr wrote:
| C++ binaries should be doing the same. Externally, speak C
| ABI. Internally, statically link Rust stdlib or C++ stdlib.
| pjmlp wrote:
| OWL, MFC, Qt, VCL, FireMonkey, AppFramework,
| PowerPlant...
|
| Plenty do not, especially on Apple and Microsoft
| platforms because they always favoured other approaches
| to bare bones UNIX support on their dynamic linkers, and
| C++ compilers.
| pjc50 wrote:
| Exporting a C API from a C++ project to consume in
| another C++ project is really painful. This is how you
| get COM.
|
| (which actually slightly pre-dates C++, I think?)
| Someone wrote:
| > This is how you get COM. (which actually slightly pre-
| dates C++, I think?)
|
| No. C++ is from 1985
| (https://en.wikipedia.org/wiki/C%2B%2B), COM from 1993
| (https://en.wikipedia.org/wiki/Component_Object_Model)
| quotemstr wrote:
| COM is actually good though. Or if you want another
| object system, you can go with GObject, which works fine
| with Rust, C-+, Python, JavaScript, and tons of other
| things.
| pjmlp wrote:
| A culture isse, as in the C++ world, of Apple and Microsoft
| ecosystems, shipping binary C++ libraries is a common
| business, even it is compiler version dependent.
|
| This is why Apple made such a big point of having a better
| ABI approach on Swift, after their experience with C++ and
| Objective-C.
|
| While on Microsoft side, you will notice that all talks
| from Victor Ciura on Rust conferences have dealing with ABI
| as one of the key points Microsoft is dealing with in the
| context of Rust adoption.
| zozbot234 wrote:
| Rust's stable ABI _is_ the C ABI. So you absolutely can
| dynamically link a Rust-written binary and /or a Rust-
| written shared library, but the _interface_ has to be pure
| C. (This also gives you free FFI to most other programming
| languages.) You can use lightweight statically-linked
| wrappers to convert between Rust and C interfaces on either
| side and preserve some practical safety.
| Someone wrote:
| > but the interface has to be pure C. (This also gives
| you free FFI to most other programming languages.)
|
| _Easy_ , not _free_. In many languages, extra work is
| needed to provide a C interface. Strings may have to be
| converted to zero terminated byte arrays, memory that can
| be garbage collected may have to be locked, structs may
| mean having to be converted to C struct layout, etc.
| wmf wrote:
| Ubuntu is using Rust sudo so it's definitely possible.
| egorfine wrote:
| It's not. At least not yet. It's planned for 25.10, but
| thankfully sudo will be packaged and available for a few
| versions after that as promised [1].
|
| [1] https://discourse.ubuntu.com/t/adopting-sudo-rs-by-
| default-i...
| tiffanyh wrote:
| I think that is the goal of uutils.
|
| https://uutils.github.io/
| cocoa19 wrote:
| I hope some are improved too.
|
| The performance boost in tools like ripgrep and tokei is
| insane compared to the tools they replace (grep and cloc
| respectively).
| egorfine wrote:
| I absolutely hate it when people call their tools a
| "replacement" for something that is part of core standards,
| something, that did just fine for _decades_.
|
| ripgrep is an excellent tool. But it's not a grep
| replacement. And should not ever be.
| mprovost wrote:
| The GNU utils were a replacement for the BSD utils which
| were a replacement for the original AT&T utils. Every
| replacement added new functionality and improvements, and
| every time someone complained that they didn't stick
| closer to the thing they replaced. Looking specifically
| at grep, there used to be new versions like egrep and
| fgrep that added functionalities beyond standard grep's,
| but those were eventually pulled into "standard" grep
| (GNU or BSD). If we stuck with standards we'd all still
| be using the Bourne shell. The GNU utilities have been
| around long enough that they feel like the standard now,
| but I'm glad that we're coming into a new phase of
| innovation in command-line utilities. And this didn't
| start with Rust - the new generation of search utilities
| started with ack (Perl) and then ag (C).
| egorfine wrote:
| > we'd all still be using the Bourne shell
|
| Please forgive me my ignorance but what's wrong with
| bash? I'm still using it on all servers and workstations,
| I constantly write scripts for it, some fairly complex.
| It's not an obsolete project and it looks like a
| mainstream shell for me. Am I wrong?
|
| Update: yeah, I realize now that this was about the
| original Bourne Shell, not bash.
| dagw wrote:
| Bash is not Bourne, and that's the point. Bash is the
| Bourne Again Shell, a shell written to improve and
| replace the Bourne shell in the GNU ecosystem. Modern
| bash is a huge improvement over the original Bourne shell
| and I'm convinced you use bash only feature basically
| every day, and would be very annoyed if someone forced
| you to use the actual Bourne shell
| egorfine wrote:
| Ah, right! I do remember the original Bourne Shell,
| though. I wouldn't like to get back to using it. Though I
| might agree provided I get as many years of my age back.
| mprovost wrote:
| Bash isn't the Bourne shell (sh)! It's a replacement
| (Bourne Again Shell). But it's interesting that the
| replacement has become so entrenched that folk assume
| that it was the original.
| egorfine wrote:
| Yeah, got it now lol
| egorfine wrote:
| > which were a replacement for the
|
| You have a point here. I have to agree.
|
| "X but rewritten in Z" is a terrible marketing, though.
| Makes me instantly want to hate the tool and its authors.
| (Love rust. Hate the vibe).
| mprovost wrote:
| Why the hate? It's a genuine question. When you rewrite
| something, you need to justify the effort somehow. The
| GNU coreutils started out as "the BSD utilities, but with
| the GPL!".
| egorfine wrote:
| Because the reimplementation authors skip all the
| complexities of designing the tool in the first place
| while getting right to the fun part (which is coding),
| and then they get to call themselves authors of a well
| known infrastructure tool.
|
| Compare "I have typed a setuid() wrapper in rust" vs "I'm
| the author of sudo-rs".
| scripturial wrote:
| It's a rust crate designed to be a native rust
| replacement for a rust c wrapper crate. It's faster and
| easier to link to in rust projects.
|
| How would you even tell people you made a better rust
| crate without using the word "rust?"
| egorfine wrote:
| Why not spend the efforts to speed up the real zlib? So
| that the whole world actually gets to spin a bit faster.
|
| Rust folks are claiming excellent interoperability with C
| binaries. Why the need for a rewrite then?
| swiftcoder wrote:
| Maybe because the speed up is easier to attain in a
| language where you aren't constantly worrying about
| introducing bugs? Maybe development is easier in a
| language with more modern tooling?
|
| Interoperability runs both ways, everyone currently
| taking a dependency on the C library can swap in the rust
| library in its place and see the same benefits
| throwawaymaths wrote:
| you would say "rewritten FOR rust" instead of "rewritten
| IN rust".
| scripturial wrote:
| It's a rust crate that depended on c and is now literally
| "rewritten in rust"
| throwawaymaths wrote:
| > rust crate
| burntsushi wrote:
| I didn't call ripgrep a replacement. Other people do.
| Because it does actually replace _their_ usage of grep in
| some or all cases, depending on their usage patterns.
|
| https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#
| can...
| dagw wrote:
| _And should not ever be._
|
| Those "core standards" that you talk about didn't spring
| fully formed from the earth. They came about from
| competition and beating out and replacing the old "core
| standards" that lots of people argued very strongly for
| should not ever be replaced. When I was starting out my
| career I was told by experienced people that I should not
| learn to rely on the GNU tool features, since they're far
| from ubiquitous and probably won't be installed on most
| systems I'll be working on.
| egorfine wrote:
| That's right and still true: GNU tools are still not
| ubiquitous on mainstream computers. And I'm not talking
| about that Ultrix box still churning. For example the
| latest macOS carries bsd tar, so.
| jelder wrote:
| I hadn't heard of tokei before, so I tried it on a small
| project of mine.
|
| Tokei _finishes_ before cloc can print its help text. I
| wrote this post in less time than it took `cloc .` to count
| all the files in my project, probably because it doesn't
| know to ignore `target/`.
| arp242 wrote:
| > Tokei _finishes_ before cloc can print its help text.
|
| cloc is a Perl script, so it has the interpreter startup
| time.
| coldpie wrote:
| 1) This is a cool project and I wish them success. It would
| be really cool if these became the default utilities some day
| soon.
|
| 2) I think the MIT license was a mistake. These are often
| cloning GNU utilities, so referencing GNU source in its
| original language and then re-implementing it in Rust would
| be the obvious thing to do. But porting GPL-licensed code to
| an MIT licensed project is not allowed. Instead, the
| utilities must be re-implemented from scratch, which seems
| like a waste of effort. I would be interested in doing the
| work of porting GNU source to Rust, but I'm not interested in
| re-writing them all from scratch, so I haven't contributed to
| this project.
| hueho wrote:
| Plenty of people dislike the perceived bload in GNU utils -
| for them a rewrite from scratch is a feature, not a bug.
| monster_truck wrote:
| What's the point of mentioning you suspect it is happening
| other than as a dig at them?
| coldpie wrote:
| You're 100% right. I removed it.
| deknos wrote:
| i wait until they come to the hard stuff like awk, sed and
| grep.
| GuB-42 wrote:
| ripgrep is one of the best grep replacement you can find,
| maybe even the best, and also one of the most famous Rust
| projects.
|
| I don't know of a sed equivalent, but I guess that would be
| easy to implement as Rust has good regex support (see
| ripgrep), and 90%+ of sed usage is search-and-replace. The
| other commands don't look hard to implement and because they
| are not used as much, optimizing these is less of a priority.
|
| I don't know about awk, it is a full programming language,
| but I guess it is far from an impossible task to implement.
|
| Now the real hard part is making a true, bug-for-bug
| compatible replacement of the GNU version of these tools, but
| while good to have, it is not strictly necessary. For
| example, Busybox is very popular, maybe even more so than GNU
| in terms of number of devices, and it has its own (likely
| simplified) version of grep, sed and awk.
| scns wrote:
| There is sd, not a drop in replacement though.
|
| https://github.com/chmln/sd
| egorfine wrote:
| What would be the point?
| kpcyrd wrote:
| I briefly looked a this and there's already cargo-c
| configuration, which is good, but it's currently namespaced
| differently, so it won't get automatically detected by C
| programs as `libbz2`:
|
| https://github.com/trifectatechfoundation/libbzip2-rs/blob/8...
|
| I'm not familiar enough with the symbols of bzip2 to say
| anything about ABI compatibility.
|
| I have a toy project to explore things like that, but it's
| difficult to set aside the amount of time needed to maintain an
| implementation of the GNU operating system. I would welcome
| pull requests though:
|
| https://github.com/kpcyrd/platypos
| Pesthuf wrote:
| If it hasn't seen an upstream release since 2019, doesn't that
| mean the implementation is just... finished? Maybe there's no
| more bugs to fix and features to add. And in that case, I don't
| see what's wrong with it.
| LinusU wrote:
| Isn't 10-15% faster compression, and 5-10% faster
| decompression, a very nice "feature"?
|
| > [...] doesn't that mean the implementation is just...
| finished?
|
| I don't think that it _necessarily_ means that, e.g. all
| projects that haven't had a release since 2019 aren't
| finished? Probably most of them are simply abandoned?
|
| On the other hand, a finished implementation is certainly a
| _possible_ explanation for why there have been no releases.
|
| In this specific case, there are a handful of open bugs on
| their issue tracker. So that would indicate that the project
| isn't finished.
|
| ref:
| https://sourceware.org/bugzilla/buglist.cgi?product=bzip2
| dale_huevo wrote:
| A lot of this "rewrite X in Rust" stuff feels like burning your
| own house down so you can rebuild and paint it a different color.
|
| Counting CPU cycles as if it's an accomplishment seems irrelevant
| in a world where 50% of modern CPU resources are allocated toward
| UI eye candy.
| 0cf8612b2e1e wrote:
| Every cycle saved is longer battery life. Someone paid the one
| time cost of porting it, and now we can enjoy better
| performance forever.
| dale_huevo wrote:
| They kicked off the article saying that no one uses bzip2
| anymore. A million cycles saved for something no one uses
| (according to them) is still 0% battery life saved.
|
| If modern CPUs are so power efficient and have so many spare
| cycles to allocate to e.g. eye candy no one asked for, then
| no one is counting and the comparison is irrelevant.
| jimktrains2 wrote:
| Isn't bzip used quite a bit, especially for tar files?
| jeffbee wrote:
| If so, only by misguided users. Why would anyone choose
| bz2 in 2025?
| 0x457 wrote:
| To unpack an archive made from the time when bz2 was
| used?
| ben-schaaf wrote:
| Of course no one uses systems, tools and files created
| before 2025!
| jeffbee wrote:
| bzip2 hasn't been the best at anything in at least 20
| years.
| appreciatorBus wrote:
| The same could be said of many things that, nonetheless,
| are still used by many, and will continue to be used by
| many for decades to come. A thing does not need to be
| best to justify someone wanting to make it a bit better.
| MBCook wrote:
| I use plain old zip files almost every day.
|
| "Best" is measured along a lot more axis than just
| performance. And you don't always get to choose what
| format you use. It may be dictated to you by some 3rd
| party you can't influence.
| Twirrim wrote:
| So? If I need to consume a resource compressed using bz2,
| I'm not just going to sit around and wait for them to use
| zstd. I'm going to break out bz2. If I can use a modern
| rewrite that's faster, I'll take every advantage I can
| get.
| kbolino wrote:
| bzip2 is still pretty good if you want to optimize for:
| - better compression ratio than gzip - faster
| compression than many better-than-gzip competitors
| - lower CPU/RAM usage for the same compression ratio/time
|
| This is a niche, but it does crop up sometimes. The
| downside to bzip2 is that it is slow to _de_ compress,
| but for write-heavy workloads, that doesn't matter too
| much.
| Philpax wrote:
| The Wikipedia data dumps [0] are multistream bz2. This
| makes them relatively easy to partially ingest, and I'm
| happy to be able to remove the C dependency from the Rust
| code I have that deals with said dumps.
|
| [0]: https://meta.wikimedia.org/wiki/Data_dump_torrents#E
| nglish_W...
| yuriks wrote:
| It sounds like the main motivation for the conversion was
| to simplify builds and reduce the chance of security
| issues. Old parts of protocols that no one pays much
| attention to anymore does seem to be a common place where
| those pop up. The performance gain looks more like just a
| nice side effect of the rewrite, I imagine they were at
| most targeting performance parity.
| spartanatreyu wrote:
| Exactly, even if we can't remove "that one dependency"
| (https://xkcd.com/2347/), we can reinforce everything
| that uses it.
| Rucadi wrote:
| I personally find a lot more relevant the part about "Enabling
| cross-compilation ", which in my opinion is important and a
| win.
|
| The same about exported symbols and being able to compile to
| wasm easily.
| Terr_ wrote:
| It seems to me like binary file format parsing (and
| construction) is probably a good place for using languages that
| aren't as prone to buffer-overflows and the like. Especially if
| it's for a common format and the code might be used in all
| sorts of security-contexts.
| wahern wrote:
| Buffer overflows are more a library problem, not a language
| problem, though for newer ecosystems like Rust the
| distinction is kind of lost on people. But point being, if
| you rewrote bzip2 using an equivalent to std::Vec, you'd end
| up in the same place. Unfortunately, the norm among C
| developers, especially in the past, was to open code most
| buffer manipulation, so you wind up with 1000 manually
| written overflow checks, some of which are wrong or outright
| missing, as opposed to a single check in a shared
| implementation. Indeed, even that Rust code had an off-by-one
| (in "safe" code), it just wasn't considered a security issue
| because it would result in data corruption, not an overflow.
|
| What Rust-the-language does offer is temporal safety (i.e.
| the borrow checker), and there's no easy way to get that in
| C.
| anonnon wrote:
| > Counting CPU cycles
|
| And that's assuming they aren't lying about the counting:
| https://desuarchive.org/g/thread/104831348/#q104831479
| DaSHacka wrote:
| Rust devs continuing to use misleading benchmarks? I, for
| one, am absolutely shocked. Flabbergasted, even.
| dwattttt wrote:
| Do you have any reason to think their numbers are wrong, or
| is your argument "someone else once lied, maybe they are
| too"?
| jxjnskkzxxhx wrote:
| > lot of this "rewrite X in Rust" stuff feels like
|
| Indeed. You know the react-angular-vue nevermind is churn? It
| appears that the trend of people pushing stuff because it
| benefit their careers is coming to the low level world.
|
| I for one still find it mistifying that Linus torvals let this
| people into the kernel. Linus, who famous banned c++ from the
| kernel not because of c++ in itself, but to ban c++
| _programmer_ culture.
| cornstalks wrote:
| > _Counting CPU cycles as if it 's an accomplishment seems
| irrelevant in a world where 50% of modern CPU resources are
| allocated toward UI eye candy._
|
| That's the kind of attitude that leads to 50% of modern CPU
| resources being allocated toward UI eye candy.
| viraptor wrote:
| Those cycles translate directly to $ saved in a few places.
| Mostly in places far away from having any UI at all.
| tcfhgj wrote:
| > Counting CPU cycles as if it's an accomplishment seems
| irrelevant in a world where 50% of modern CPU resources are
| allocated toward UI eye candy.
|
| Attitude which leads to electron apps replacing native ones,
| and I hate it. I am not buying better cpus and more ram just to
| have it wasted like this
| stevefan1999 wrote:
| You know it is just Wirth's law in action: "Software gets
| slower faster than hardware gets faster." [^1]
|
| In fact Jevons Paradox: _When technological progress
| increases the efficiency with which a resource is used, but
| the rate of consumption of that resource rises due to
| increasing demand_ - essentially, efficiency improvements can
| lead to increased consumption rather than the intended
| conservation. [^2][^3]
|
| [^1]:
| https://www.comp.nus.edu.sg/~damithch/quotes/quote27.htm
|
| [^2]: https://www.greenchoices.org/news/blog-posts/the-
| jevons-para...
|
| [^3]: https://quickonomics.com/terms/jevons-paradox/
| hyperman1 wrote:
| I think it goes deeper. There is a certain level of
| slowness that causes pain to users. When that level is hit,
| market forces cause attention to software efficiency.
|
| Hardware efficiency just gives more room for software to
| bloat. The pain level is a human factor and stays the same.
|
| So time to adapt Wirths law: Software gets slower >exactly
| as much< as hardware gets faster
| bitwize wrote:
| It's a lot like X11 vs. Wayland. The current graphics
| developers, who trend younger, don't want to maintain the
| boomer-written C code in the X server. Too risky and time-
| consuming. So one of the goals of Wayland is to completely
| abolish X so it can be replaced with something more long-term
| maintainable. Turns out, current systems-level developers don't
| want to maintain boomer-written GNU code or any C code at all,
| really, for similar reasons. C is inherently problematic
| because even seasoned developers have trouble avoiding its
| footguns. So an unstated, but important, goal of Rust is to
| abolish all critical C code and replace it with Rust code.
| Ubuntu is on board with this.
| Surac wrote:
| So you say younger programmer have not the required coding
| kung fu to cope with c code? I hope you are wrong. The
| perspective to have rust like things on everydays devices
| realy frightens me. C is like a Lingua franca for computers.
| Nearly any hardware near person can READ it. I am one of this
| Boomers and i am not able to propper READ rust code, because
| the syntax is so academic. The fact that more and more code
| is written in rust, lessens the amount of people that can
| read programs
| zaphar wrote:
| It's more like _no_ programmer of any age has the required
| coding kung fu to cope with C code. It is _inevitable_ that
| they will introduce problematic code. We have decades of
| examples illustrating this. We lived with it because there
| was no truly competitive alternative for so long.
|
| I can read and write C code from the times when there
| weren't any competitive alternatives. I have no problem
| reading or writing Rust code. In fact it communicates more
| to me than C code does or can and I can immediately
| understand more about the code written in Rust than I can
| about code written in C.
| panick21_ wrote:
| Except Wayland was developed by the same people who worked
| for years on X. And they don't dislike X because of C. And
| they didn't write Wayland in Rust.
| bitwize wrote:
| > Except Wayland was developed by the same people who
| worked for years on X.
|
| Yes, and they hated it and "worked hard to kill it" per
| Jordan Petridis. Note that the _maintainers_ of X in the
| Wayland era are not really the same people as the original
| authors of X.
| Scuds wrote:
| you're just an end user, you don't have to maintain the suite.
|
| In OSS every hour of volunteer time is precious Manna from
| heaven, flavored with unicorn tears. So any way to remove Toil
| and introduce automation is gold.
|
| Rust's strict compiler and an appropriate test suite guarantees
| a level of correctness far beyond C. There's less onus on the
| reviewer to ensure everything still works as expected when
| reviewing a pull request.
|
| It's a win-win situation.
| hoseja wrote:
| It's like "adapting" Akallabeth so you can tell your own
| empowering story for modern audiences.
| SpaceNugget wrote:
| Pretty incredible for such a short argument to be so
| inconsistent with itself. Complaining about counting CPU cycles
| and actually measuring performance because... modern software
| development is bad and doesn't care about performance?
| egorfine wrote:
| I fully agree with you on the first statement and I am at loss
| of words at the second...
| firesteelrain wrote:
| Anyone know if this will by default resolve the 11 outstanding
| CVEs?
|
| Ironically there is one CVE reported in the bzip2 crate
|
| [1]
| https://app.opencve.io/cve/?product=bzip2&vendor=bzip2_proje...
| Philpax wrote:
| > The bzip2 crate before 0.4.4
|
| They're releasing 0.6.0 today :>
| tialaramex wrote:
| There's certainly a contrast between the "Oops a huge file
| causes a runtime failure" reported for that crate and a bunch
| of "Oops we have bounds misses" in C. I wonder how hard anybody
| worked on trying to exploit the bounds misses to get code
| execution. It may or may not be impossible to achieve that
| escalation.
| rwaksmunski wrote:
| I use this crate to process 100s of TB of Common Crawl data, I
| appreciate the speedups.
| malux85 wrote:
| Yeah came here to say a 14% speed up in compression is pretty
| good!
| koakuma-chan wrote:
| It's blazingly fast
| aidenn0 wrote:
| bzip2 (particularly parallel implementations thereof) are
| already relatively competitive for compression. The
| decompression time is where it lags behind because lz77 based
| algorithms can be incredibly fast at decompression.
| viraptor wrote:
| What's the reason for using bz2 here? Wouldn't it be faster to
| do a one off conversion to zstd? It beats bzip2 in every metric
| at higher compression levels as far as I know.
| declan_roberts wrote:
| That assumes you're processing the data more than once.
| rwaksmunski wrote:
| Common Crawl delivers the data as bz2. Indeed I store
| intermediate data in zstd with ZFS.
| anon-3988 wrote:
| Is this data available as torrents?
| solarized wrote:
| Do they use any llm to transpile the C to Rust ?
| nightfly wrote:
| Task that requires precision and potentially hard to audit?
| Exactly where I'd use an LLM /s
| CGamesPlay wrote:
| Without commenting on whether an LLM is the right approach, I
| don't think this task is particularly hard to audit. There is
| almost assuredly a huge test suite for bzip2 archives;
| fuzzing file formats is very easy; and you can restrict /
| audit the use of unsafe by the translator.
| MBCook wrote:
| You're right, there is a large existing test suite. It's
| mentioned in an article linked from this one.
|
| https://trifectatech.org/blog/translating-
| bzip2-with-c2rust/
|
| I suspect attempting to debug it would be a nightmare
| though. Given the LLM could hallucinate anything anywhere
| you'd likely waste a ton of time.
|
| I suspect it would be faster to just try and write a new
| implementation based on the spec and debug that against the
| test suite. You'd likely be closer.
|
| In fact, since they used c2rust, they had a perfectly
| working version from the start. From there they just had to
| clean up the Rust code and make sure it didn't break
| anything. Clearly the best of the three options.
| j16sdiz wrote:
| > and you can restrict / audit the use of unsafe by the
| translator.
|
| No. You need to audit for correctness in additional to
| safety.
| Twirrim wrote:
| If you're going to use tools to transpile, don't use something
| that hallucinates. You want it to be precise.
|
| https://github.com/immunant/c2rust reportedly works pretty
| well. Blog post from a few years ago of them transpiling quake3
| to rust: https://immunant.com/blog/2020/01/quake3/. The rust
| produced ain't pretty, but you can then start cleaning it up
| and making it more "rusty"
| dataking wrote:
| They indeed used c2rust for the initial transpile according
| to https://trifectatech.org/blog/translating-
| bzip2-with-c2rust/
| a-dub wrote:
| i'd be curious if they're using the same llvm codegen (with the
| same optimization) backend for the c and rust versions. if so,
| where the speedups are coming from?
|
| (ie, is it some kind of rust auto-simd thing, did they use the
| opportunity to hand optimize other parts or is it making use of
| newer optimized libraries, or... other)
| eru wrote:
| Just speculating: Rust can hand over more hints to the code
| generator. Eg you don't have to worry about aliasing as much as
| with C pointers. See
| https://en.wikipedia.org/wiki/Aliasing_(computing)#Conflicts...
| MBCook wrote:
| This makes a lot of sense to me, though I don't know the
| official answer so I'm just sort of guessing along too.
|
| Linked from the article is another on how they used c2rust to
| do the initial translation.
|
| https://trifectatech.org/blog/translating-bzip2-with-c2rust/
|
| For our purposes, it points out places where the code isn't
| very optimal because the C code has no guarantees on the
| ranges of variables, etc.
|
| It also points out a lot of people just use 'int' even when
| the number will never be very big.
|
| But with the proper type the Rust compiler can decide to do
| something else if it will perform better.
|
| So I suspect your idea that it allows unlocking better
| optimizations though more knowledge is probably the right
| answer.
| Too wrote:
| Ergonomics of using the right data structures and algorithms
| can also play a big role. In C, everything beyond a basic
| array is too much hassle.
| littlestymaar wrote:
| Yeah, that was Brian Cantrill's realization when for the
| sake of learning he rewrote a part of dtrace in Rust and
| was shocked when he saw his naive reimplementation being
| significantly faster than his original code, and the answer
| boiled down to "I used a BTreeMap" in Rust because it's in
| std".
| a-dub wrote:
| hmm.. i wonder how it would compare then with
| clang+linux, clang+stl or hotspot+j2ee.
|
| reminds me a bit of the days when perl programs would
| often outrun native c/c++ for common tasks because
| ultimately they had the most efficient string processing
| libraries baked into the language built-ins.
|
| how is space efficiency? last i checked, because of big
| libraries and ease of adding them to projects, a lot of
| rust binaries tend to be much larger than their
| traditional counterparts. how might this impact overall
| system performance if this trade-off is made en-masse?
| (even if more ram is added to counteract loss of vm page
| cache, does it also start to impact locality and cache
| utilitization?)
|
| i'd be curious how something like redox benchmarks
| against traditional linux for real world workloads and
| interactivity measures.
| bcantrill wrote:
| For whatever it's worth, details of my findings are in
| [0].
|
| [0] https://bcantrill.dtrace.org/2018/09/28/the-relative-
| perform...
| a-dub wrote:
| pretty cool! in isolation looks awesome! i'm still a
| little curious about the impacts increased executable
| image size, especially in a complete system.
|
| if all the binaries are big, does it start to crowd out
| cache space? does static linking make sense for full
| systems?
| WhereIsTheTruth wrote:
| any rewrite, in X, Y, Z language gives you the opportunity to
| speed things up, there is nothing inherent to rust
| adgjlsfhk1 wrote:
| C is honestly a pretty bad language for writing modern high
| performance code. Between C99 and C21, there was a ~20 year gap
| where the language just didn't add features needed to
| idiomatically target lots of the new instructions added
| (without inline asm). Just getting good abstract machine
| instructions for clz/popcnt/clmul/pdep etc helps a lot for
| writing this kind of code.
| zzo38computer wrote:
| Popcount, clz, and ctz are provided as nonstandard functions
| in GCC (and clang might also support them in GNU mode, but I
| don't know for sure). PDEP and PEXT do not seem to be, but I
| think they should be (and PEXT is something that INTERCAL
| already had, anyways) (although PDEP and PEXP can be used
| with -mbmi2 on x86, but are not available for general use).
| The MOR and MXOR of MMIX are also something that I would want
| to be available as built-in functions.
| xvilka wrote:
| I hope they or Prossimo will also look and reimplement in the
| similar fashion the core Internet protocols - BGP, OSPF and RIP,
| other routing implementations, DNS servers, and so on.
| dataking wrote:
| https://www.memorysafety.org/initiative/ this page mentions TLS
| and DNS which goes some way towards your suggestion.
| throw10920 wrote:
| Is that domain actually about memory safety or about Rust?
| nickpsecurity wrote:
| One guy did Ironsides DNS in SPARK Ada which has stronger
| proofs.
| xvilka wrote:
| Nothing against Ada, it's a good language. The only problem
| would be finding contributors in that case.
| everfrustrated wrote:
| Check out
|
| https://nlnet.nl/project/current.html
| https://www.sovereign.tech/programs/fund
|
| There's been good support over the last couple of years to fund
| rewriting critical internet & OS tools into safer languages
| like Rust.
|
| Eg BGP in Rust
| https://www.nlnetlabs.nl/projects/routing/rotonda/
| xvilka wrote:
| Thank you, precisely what I had in mind! Somehow I missed
| this project. As well as Holo[1] (routing)
|
| [1] https://github.com/holo-routing/holo
| broken_broken_ wrote:
| About not having perf on macOS: you can get quite far with dtrace
| for profiling. That's what the original flame graph script in
| Perl mentions using and what the flame graph Rust
| reimplementation also uses. It does not have some metrics like
| cache misses or micro instructions retired but still it can be
| very useful.
| zoobab wrote:
| Lbzip2 had much faster decompressing speed, using all available
| CPU cores.
|
| It's 2025, and most programs like Python are stuck at one CPU
| core.
| guappa wrote:
| Thanks for showing us you have no understanding of python's
| situation.
| tephra wrote:
| I like Rust and have an ambition to learn it as well (I've had a
| few false starts...). One of my issues that I have is that every
| (slight exaggeration) library that I seem to come across is still
| at version 0.x.y. Take this library as an example. 0.1.0 was
| released in 2014 and it still hasn't had a 1.0.0 release, is
| there an aversion to get to 1.0.0 in the rust community?
| liambigelow wrote:
| https://0ver.org/#notable-zerover-projects
| liambigelow wrote:
| Serious answer: For some, they do change semi-often and don't
| feel compelled to declare stability. In other cases, it's a
| stable + widely used 0.x package, and bumping it to 1.0
| usually implies _some_ kind of breaking change. (I don't know
| if that _should_ be the case, but I know that if I see a
| dependency has bumped from 0.x to 1.0 I'm going to be
| cautious and wait to update it until I have more time).
|
| In general: People usually aren't too concerned about it.
| sramsay64 wrote:
| This list's Zig as an entry, despite the Zig project having
| very clear plans[0] for a 1.0 release. That's not 0ver, it's
| just the beta stage of semver.
|
| [0] https://github.com/ziglang/zig/milestone/2
| scripturial wrote:
| Yes, in rust, the package manager has built in rules about when
| to update a package. It won't auto update a major version
| change because it implies a change that breaks something. As
| long as your package is safe to auto update you don't want to
| change the major version number.
| wiz21c wrote:
| FTA:
|
| > Why bother working on this algorithm from the 90s that sees
| very little use today?
|
| What's in use nowadays ? zstd ?
|
| ahh saw this: https://quixdb.github.io/squash-benchmark/
| Aissen wrote:
| Does anyone know if it supports parallel decompression,
| lbzip2-style? (or just iterators doing pre-scanning for the block
| magic that allow doing parallel decompression on top).
|
| _Edit_ : it probably doesn 't.
| agumonkey wrote:
| rust aside, I really enjoy seeing all these different
| implementation benchmarks, very satisfying to read
| nullbyte wrote:
| We should rewrite Rust in Javascript
___________________________________________________________________
(page generated 2025-06-18 23:02 UTC)