[HN Gopher] Rust compiler performance
___________________________________________________________________
Rust compiler performance
Author : mellosouls
Score : 269 points
Date : 2025-06-10 08:24 UTC (3 days ago)
(HTM) web link (kobzol.github.io)
(TXT) w3m dump (kobzol.github.io)
| dboreham wrote:
| I'd vote for filesystem space utilization to be worked on before
| performance.
| stevedonovan wrote:
| Is this not more a Cargo thing? Cargo is obsessed with correct
| builds and eventually the file system fills up with old
| artifacts.
|
| (I know, I have to declare Cargo bankruptcy every few weeks and
| do a full clean & rebuild)
| vlovich123 wrote:
| Correct builds != never running garbage cleanup. I would
| settle for it evicting older variants of a build (I also
| dislike the random hash that's impossible to determine what
| specifically is different between two hashes / which one is
| newer).
| mbrubeck wrote:
| Automatic garbage collection of old build artifacts* is
| coming in Rust 1.88 (currently on the beta channel, will
| become the new stable release in two weeks):
|
| https://github.com/rust-lang/cargo/issues/12633
|
| *EDIT: For now this only deletes old cached downloads, not
| build artifacts. Thanks epage for the correction below.
| vlovich123 wrote:
| That's as an additional flag and not the default?
| mbrubeck wrote:
| In versions earlier than 1.88, garbage collection
| required the unstable -Zgc flag (and a nightly
| toolchain). But in 1.88 and later, automatic garbage
| collection is enabled by default.
| epage wrote:
| That is not for build artifacts but global caches like
| for `.crate` files but its a stepping stone.
| mbrubeck wrote:
| Oops, thanks for the correction.
| epage wrote:
| A little of both. Incremental compilation cache is likely the
| single largest item in the target directory but it gets
| cleaned up on each invocation so it doesn't scale in size
| with time.
|
| I believe the next release will have a cache GC but only for
| global caches (e.g. `.crate` files). I'd like us to at least
| cleanup the layout of the target directory so its easier to
| track stuff before GCing it. Work is underway for this. A
| cheap GC we could add earlier is for artifacts specific to
| older cargo versions.
| dralley wrote:
| The problems are largely related. Cut down the amount of
| intermediate compilation artifacts by half and you'll have sped
| up the compiler substantially. Monomorphization and iterator
| expansion and such is a significant contributor to both issues.
| scripturial wrote:
| One of the reasons I quit rust is literally because having 4-5
| projects checked out that use serde would fill my laptop drive
| with junk in a few weeks.
| ivanjermakov wrote:
| This is one of the only reasons I disliked Haskell. GHC and lib
| files can easily take over 2GB of storage.
| MeetingsBrowser wrote:
| Why not both?
|
| If I had to choose though, I would choose compilation speed.
| Buying an SSD to double my storage is much more cost effective
| than buying a bulkier processor to halve my compilation times.
| estebank wrote:
| > Why not both?
|
| For better and worse, the Rust project is a "show-up-ocracy":
| the work that gets done is the one that volunteers (or paid
| employees from a company donating their time to the project)
| spend time doing. It's hard in open source projects to tell
| people "this is important and you must work on it", but
| rather you have to _convince_ people that it is important and
| have to _hope_ they will have the time, inclination and skill
| to work on it.
|
| FWIW, people were working on the cargo cache GC feature years
| ago[1], but I am not aware of what the current state of that
| is. I wouldn't be surprised if it wasn't turned on because
| there are unresolved questions.
|
| 1: https://blog.rust-lang.org/2023/12/11/cargo-cache-
| cleaning/
| Ygg2 wrote:
| > For better and worse, the Rust project is a "show-up-
| ocracy"
|
| So it's an OSS project?
| windward wrote:
| Linux and GCC are OSS projects but CPU companies pour
| work into their development because it's a strict
| requirement of being able to sell their products.
| kzrdude wrote:
| Performance has been worked on since Rust 1.0 or so, for all
| this time, there has been lots of work on compiler performance.
| There's no "before performance" :)
| estebank wrote:
| There are multiple avenues to improve both first and
| incremental compiles. They "just" require large architectural
| changes that may yield marginal improvements, so it's hard to
| get those projects off the ground. But after the last project
| all-hands I do expect _at least one_ of these to be pursued,
| of not more.
|
| There are cases where cargo recompiles crates
| "unnecessarily", cargo+rustc could invert the compilation
| pyramid to start at the root and then only compile reachable
| items (similar effect to LTO, but can produce a binary if an
| item with a compile error isn't evaluated, improving clean
| compiles), having better communication with linkers and
| having access to incremental linking would be a boon for
| incremental compiles, etc.
| preciousoo wrote:
| What's stopping cargo from storing libraries in one global
| directory(via hash or whatever), to be re-used whenever needed?
| jakkos wrote:
| You can: https://github.com/mozilla/sccache
| preciousoo wrote:
| Oh this is amazing, looks simple to set up too. Thanks!
| kstrauser wrote:
| It's a game-changer, especially on my slower machines
| like Raspberry Pis. They can use every advantage they can
| get.
| epage wrote:
| That work is being tracked in https://github.com/rust-
| lang/cargo/issues/5931
|
| Someone has taken up the work on this though there are some
| foundational steps first.
|
| 1. We need to delineate intermediate and final build
| artifacts so people have a clearer understanding in `target/`
| what has stability guarantees (implemented, awaiting
| stabilization).
|
| 2. We then need to re-organize the target directory from
| being organized by file type to being organized by crate
| instance.
|
| 3. We need to re-do the file locking for `target/` so when we
| share things, one cargo process won't lock out your entire
| system
|
| 4. We can then start exploring moving intermediate artifacts
| into a central location.
|
| There are some caveats to this initial implementation
|
| - To avoid cache poisoning, this will only items with
| immutable source that and an idempotent build, leaving out
| your local source and stuff that depends on build scripts and
| proc-macros. There is work to reduce the reliance on build
| scripts and proc-macros. We may also need a "trust me, this
| is idempotent" flag for some remaining cases.
|
| - A new instance of a crate will be created in the cache if
| any dependency changes versions, reducing reuse. This becomes
| worse when foundation crates release frequently and when
| adding or updating a specific dependency, Cargo prefers to
| keep all existing versions, creating a very unpredictable
| dependency tree. Support for remote caches, especially if you
| can use your project's CI as a cache source, would help a lot
| with this.
| jakkos wrote:
| Not that this isn't a problem, it is, target folders currently
| take up ~100gb on my machine but...
|
| I'd still, by far, prefer a tiny incremental compile speed
| increase over a substantial storage reduction. I can get a
| bigger SSD, I can't get my time back :'(
| kibwen wrote:
| The previous post on the OP's blog is about exactly this:
| https://kobzol.github.io/rust/rustc/2025/06/02/reduce-cargo-...
| vlovich123 wrote:
| I wonder if how much value there is in skipping LLVM in favor of
| having a JIT optimized linked in instead. For release builds it
| would get you a reasonable proxy if it optimized decently while
| still retaining better debugability.
|
| I wonder if the JVM as an initial target might be interesting
| given how mature and robust their JIT is.
| dhruvrajvanshi wrote:
| I would love this in modern languages.
|
| For dev builds, I see JIT compilation as a better deal than
| debug builds because it's capable of eventually reaching peak
| performance. For performance sensitive stuff like games, it
| really matters to keep a nice feedback loop without making the
| game unusable by turning off _all_ optimizations.
|
| AOT static binaries are valuable for deployments.
|
| No idea how expensive it would be to develop for an existing
| language like Rust though.
| zozbot234 wrote:
| The JVM is not a very meaningful target for Rust since it does
| not use C-like flat memory addressing and pointer arithmetic.
| It's as if every single Java object and field is sitting in its
| own tiny memory segment/address space. On the one hand, this
| makes it essentially transparent to GC, which is a key property
| for Java; OTOH, it means that compiling C-like languages to the
| JVM is usually done by reimplementing "memory" as a JVM array
| of byte values.
| bbatha wrote:
| > I wonder if how much value there is in skipping LLVM in favor
| of having a JIT optimized linked in instead. For release builds
| it would get you a reasonable proxy if it optimized decently
| while still retaining better debugability.
|
| Rust is in the process of building out the cranelift backend.
| Cranelift was originally built to be a JIT compiler. The hope
| is that this can become the debug build compiler.
|
| https://github.com/rust-lang/rustc_codegen_cranelift
| actualwitch wrote:
| I recently tried using cranelift on a monorepo with a bunch
| of crates, and it is nothing short of amazing. Nothing broke
| and workspace build time went from a minute and a half to a
| half of a second!
| lsuresh wrote:
| Was this for a release build or a debug build?
| actualwitch wrote:
| Cranelift is only intended for debug builds, there is
| nothing stopping you from using it for release builds but
| -- to the best of my knowledge -- you get noticeably
| degraded runtime performance if you go that way.
| lsuresh wrote:
| LLVM optimizations are the overwhelming majority of the
| compilation bottleneck for us over at Feldera. We blogged about
| some of the challenges we faced here:
| https://www.feldera.com/blog/cutting-down-rust-compile-times...
|
| We almost definitely need to build a JIT in the future to avoid
| this problem.
| jplusequalt wrote:
| Having worked on large scale C++ code-bases and thus used to long
| compilation times, it surprises me that this is the hill many C++
| devs would die on in regards to their dislike of Rust.
| 0cf8612b2e1e wrote:
| It is a quantifiable negative to which you can always point. Of
| course it will be used for justifications.
| logicchains wrote:
| There's a lot of things you can do in C++ to reduce compilation
| time if you care about it, that aren't possible with Rust.
| MeetingsBrowser wrote:
| There are things you can do for Rust if it really is a deal
| breaker.
|
| Dioxus has a hot reload system. Some rust game engines have
| done similar things.
| jplusequalt wrote:
| When in doubt, manually patch your dll's
| kibwen wrote:
| You can absolutely do the same things in Rust, it's just that
| the culture and tooling of Rust encourages much larger
| compilation units than in C or C++, so you don't get the same
| sort of best-case nontrivial embarrassing-parallelism,
| forcing the compiler to do more work to parallelize.
|
| To address the tooling pressure, I would like to see Cargo
| support first-class internal-only crates, thereby
| deconflating the crate as what is today both the unit of
| compilation and the unit of distribution.
| phkahler wrote:
| >> it surprises me that this is the hill many C++ devs would
| die on in regards to their dislike of Rust
|
| I believe people will exaggerate their current issue so it
| sounds like the only thing that matters to them. On another
| project I've had people say "This is the _only thing_ that
| keeps me using commercial alternatives " or the _only thing_
| holding back wider adoption, or the _only thing_ needed for
| blah blah blah. Meanwhile I 've got my own _list_ of high
| priority things needed to bring it to what I 'd consider a
| basic level of completeness.
|
| When it comes to performance it will never be good enough for
| everyone. There is always a bigger project to consume whatever
| resources are available. There are always people who insist on
| doing things in odd ways (maybe valid, but very atypical).
| These requests to improve are often indistinguishable from the
| regular ones.
| pton_xd wrote:
| Makes sense to me! Everyone with enough C++ experience has
| dealt with that nightmare at one point. Never again, if you can
| help it.
| maccard wrote:
| I work on large c++ code bases day in day out - think 30 minute
| compiles on an i9 with 128GB ram and NVMe drives.
|
| Rusts compile times are still ungodly slow. I contributed to a
| "small to medium" open source project [0] a while back, fixing
| a few issues that we came across when using it. Given that the
| project is approximately 3 orders of magnitude smaller than my
| day to day project, a clean build of a few thousand lines of
| rust took close to 10 minutes. Incremental changes to the
| project were still closer to a minute at the time. I've never
| worked on a 5m+ LOC project in rust, but I can only imagine how
| long it would take.
|
| On the flip side, I also submitted some patches to a golang
| program of a similar size [1] and it was faster to clone,
| install dependencies and clean build that project than a single
| file change to the rust project was.
|
| [0] https://github.com/getsentry/symbolicator
|
| [1] https://github.com/buildkite/agent
| jplusequalt wrote:
| Yes, but Go is a higher level language than Rust. It feels
| unfair to compare the two. That's why I brought up C++ (as
| did the article).
| maccard wrote:
| I disagree that it's unfair to compare the two. The
| performance difference between go and rust is far less than
| the difference between go and python, it has a garbage
| collector sure but it's an example of a language designed
| for fast compilation time that achieves an order of
| magnitude faster compile times than rust
| jplusequalt wrote:
| I think you're missing the point of my comparison. Rust
| has a lot more going on than Go, so it feels unfair.
| sagarm wrote:
| Go has a lot less going on than Rust partially because
| compile times were a priority.
| jplusequalt wrote:
| Rust has long compilation times because the borrow
| checker was a priority.
|
| They're very different languages.
| maccard wrote:
| Rust has long compilation times because it doesn't
| prioritise compilation times.
|
| What does [0] have to do with the borrow checker?
|
| [0] https://www.feldera.com/blog/cutting-down-rust-
| compile-times...
| kzrdude wrote:
| The borrow checker is a very small factor in rust compile
| time
| 9d wrote:
| Just curious, are you still able to get instant feedback and
| development conveniences on that 30 minute compile time
| project, like up to date autocomplete and type hints and
| real-time errors/warnings while developing before compiling?
| maccard wrote:
| Yeah - there's a 60-ish second delay in my IDE before this
| info is available but once it's there it's there.
| Fluorescence wrote:
| > clean build of a few thousand lines of rust took close to
| 10 minutes
|
| That doesn't sound likely. I would expect seconds unless
| something very odd is happing.
|
| Is the example symbolicator?
|
| I can't build the optional "symbolicator-crash" crate because
| it's not rust but 300k of C++/C pulled from a git submodule
| that requires dependencies I am not going to install. Your
| complaint might literally be about C++!
|
| For the rest of the workspace, 60k of rust builds in 60
| seconds
|
| - clean debug build on a 6 year old 3900X (which is under
| load because I am working)
|
| - time includes fetching 650 deps over a poor network and
| building them (the real line count of the build is likely
| 100s of thousands or millions of lines of code)
|
| - subsequent release build took 100s
|
| - I use the mold linker which is advised for faster builds
|
| - modern cpus are so much faster than my machine they might
| not even take 10s
| maccard wrote:
| > That doesn't sound likely. I would expect seconds unless
| something very odd is happing.
|
| And yet here we are.
|
| There are plenty of stories like this floating around of
| degenerate cases of small projects. Here's [0] one example
| with numbers and how they solved it. There are enough of
| these issues that by getting bogged down in "well
| technically it's not Rust's fault, it's LLVM's single
| threadedness causing the slowdown here" ignores the point -
| Rust (very fairly) has a rep for being dog slow to compile
| even compared to large C++ projects
|
| > For the rest of the workspace, 60k of rust builds in 60
| seconds
|
| That's... not fast.
|
| https://github.com/buildkite/agent is 40k lines of go
| according to cloc, and running `go build` including pulling
| dependencies takes 40 seconds. Without pulling dependencies
| it's 2 seconds. _That's_ fast.
|
| [0] https://www.feldera.com/blog/cutting-down-rust-compile-
| times...
| afdbcreid wrote:
| What are incremental compile times with the C++ codebase?
|
| Also, does the line of code you count include dependencies
| (admitting, dependencies in Rust are a problem, but it's not
| related to compiler performance)?
| maccard wrote:
| About 15 seconds, because we carve it up into 100 or so
| dlls specifically for this case
| windward wrote:
| 15 seconds plus the months of developer time defining dll
| boundaries and writing pimpls
| maccard wrote:
| No PIMPL's, but yes to dll boundaries. Breaking up the
| code into logical dependency structyures makes sense -
| the DLL's are no different to crates in rust really.
| sapiogram wrote:
| Thanks for actually including the slow repo in your comment.
| My results on a Ryzen 5900X:
|
| * Clean debug build: 1m 22s
|
| * Incremental debug build: 13s
|
| * Clean release build: 1m 51s
|
| * Incremental release build: 24s
|
| Incremental builds were done by changing one line in
| creates/symbolicator/src/cli.rs.
|
| It's not great, but it sounds like your experience was much
| worse for some reason.
| maccard wrote:
| Sorry - my clean build was actually including the
| dependency fetching, which is a large part of it. My
| experience was in 2023 which if we go by article roughly
| scales with compiler performance to 5 minutes or so
| pm215 wrote:
| I do wonder how much the fetch-source-and-build-all-
| dependencies approach contributes to the poor perception
| of compile times -- your very first experience of "how
| long did cargo take to build this project" will be
| terrible because it will include downloading and building
| a ton of dependencies...
| maccard wrote:
| Agreed - I didn't do reliable benchmarks on how long was
| fetch + compile dependencies vs compiling the project
| itself.
| sapiogram wrote:
| My build times also included fetching dependencies, which
| only took a few seconds on my modestly fast 8 MB/s
| network.
| maccard wrote:
| For reference, buildkite-agent [0] is about 40k lines of
| go. Running `go build` including dependencies took 40
| seconds, and running `go clean && go build` took 2 seconds.
| I know Go and Rust aren't comparable, but Rust's attitude
| appears to be "we don't really care" when you compare it to
| Go, considering they both started at _roughly_ the same
| time and Rust's first stable release came long after Go was
| in use.
|
| [0] https://github.com/buildkite/agent
| jason-johnson wrote:
| Can you say what your development environment was like? I was
| having 15 minute build times for a pretty small system.
| Everyone talks about how slow Rust compile times are so I
| thought that's just how it is. Then, by chance, I ended up
| building from a clean install on my work laptop and it took
| about 3 minutes from scratch.
|
| My development environment is VS Code running in a Dev
| container in docker desktop. So after my work laptop was so
| fast, I made some changes to my Mac docker desktop and
| suddenly the mac could build the project from scratch in
| about 2 minutes. Incremental compile was several minutes
| before, instant now.
| maccard wrote:
| Cargo in vscode on windows on a monstrously big machine
| (3990x/128GB RAM/NVMe drive, Gigabit Ethernet)
|
| I think if it's that sensitive to environment issues, that
| solidifies the point that there are major problems that
| lots of people are going to have.
| hinkley wrote:
| 30 minutes versus 60 is really an hour versus two.
|
| Some coworkers and I noticed a long time ago that once you
| try to task switch while doing build/test automation steps,
| it always seems like you remember to come back and check
| about twice as long as the compile was supposed to take. 7+
| turned into 15, 15 into a half hour.
|
| And then one day it hit me that this is just Hofstadter's
| Law. You think you have ten minutes so you start a ten minute
| task and it takes you twenty, or you get in a flow and forget
| to look until your senses tell you you're forgetting
| something.
|
| Cutting 10 minutes off a build really averages 20 minutes in
| saved time per cycle. Which matters a hell of a lot when you
| go from 4 to 5 cycles per 8 hour day.
| bnolsen wrote:
| The answer there was to always write small standalone
| executable unit test sets and simulation for day to day coding.
| Avoiding template heavy pigs like QT or boost helps too.
| akazantsev wrote:
| > Avoiding template heavy pigs like QT
|
| Well, you definitely have no experience with Qt.
| throwaway664786 wrote:
| C++ is one of the fastest languages to compile*, assuming you
| aren't doing silly stuff like abusing templates. It just gets a
| bad rep because actual, real-world, massive projects are
| written in C++. Like, yeah, no wonder Chromium build times
| aren't spectacular, but I assure you that they'd be much, much
| worse if it was written in Rust. Pointing and scoffing at it
| when there's nothing written in Rust that we can even compare
| it to is just intellectually dishonest.
|
| * It's not beating interpreted languages any time soon, but
| that's not really a fair comparison.
| panstromek wrote:
| The original title is:
|
| Why doesn't Rust care more about compiler performance?
| mellosouls wrote:
| (OP) I submitted this some time ago, and am pretty sure I would
| have submitted the title as is, so I'm guessing some manual or
| automatic editing since by the mods before the second chance
| here.
| norir wrote:
| Compiler performance must be considered up front in language
| design. It is nearly impossible to fix once the language reaches
| a certain size without it being a priority. I recently saw here
| the observation that one can often get a 2x performance
| improvement through optimization, but 10x requires redesigning
| the architecture.
|
| Rust can likely never be rearchitected without causing a
| disastrous schism in the community, so it seems probable that
| compilation will always be slow.
| littlestymaar wrote:
| You're conflating _language design_ and _compiler
| architecture_. It 's hard to increment on a compiler to get
| massive performance improvement, and rearchitecture can help,
| but you don't necessarily need to change anything to the
| language itself in that regard.
|
| Roslyn (C#) is the best example of that.
|
| It's a massive endeavor and would need significant fundings to
| happen though.
| dist1ll wrote:
| Language design can have massive impact on compiler
| architecture. A language with strict define-before-use and
| DAG modules has the potential to blow every major compiler
| out of the water in terms of compile times. ASTs, type
| checking, code generation, optimization passes, IR design,
| linking can all be significantly impacted by this language
| design choice.
| formerly_proven wrote:
| No, language design decisions absolutely have a massive
| impact the performance envelope of compilers. Think about
| things like tokenization rules (Zig is designed such that
| every line can be tokenized independently, for example),
| ambiguous grammars (most vexing parse, lexer hack etc.),
| symbol resolution (e.g. explicit imports as in Python, Java
| or Rust versus "just dump eeet" imports as in C#, and also
| things whether symbols can be defined after being referenced)
| and that's before we get to the really big one: type solving.
| littlestymaar wrote:
| This kind of comment is funny because it reveals how
| uninformed people can be while having a strong opinion on a
| topic.
|
| Yes grammar can impact how theoretically fast a compiler
| can be, and yes the type system ads more or less works
| depending on how it's designed, but none of these are what
| makes Rust compiler slow. Parsing and lexing are negligible
| fraction of compile time, and typing isn't particularly
| heavy in most cases (with the exception of niches crates
| who abuse the Turing completeness of the Trait system).
| You're not going to make big gains by changing these.
|
| The massive gains are to be made later in the pipeline (or
| earlier, by having a way to avoid re-compiling pro macros
| and their dependencies before the actual compilation can
| even start).
| lsuresh wrote:
| Hard agree. Practically all the bottlenecks we run into
| with Rust compilation have to do with the LLVM passes.
| The frontend doesn't even come close. (e.g.
| https://www.feldera.com/blog/cutting-down-rust-compile-
| times...)
| uecker wrote:
| While LLVM is known to be slow, not all LLVM-based
| languages are equally slow.
| littlestymaar wrote:
| This isn't an issue with LLVM being slow, but of rustc
| not calling LLVM efficiently, read the linked blog post!
| uecker wrote:
| I guess this was my point.
| edflsafoiewq wrote:
| The point was language design influences compiler
| performance. Rust is heavily designed around "zero-cost
| abstraction", ie. generate tons of IR and let the backend
| sort it out. Spending all the time in LLVM passes is what
| you would expect from this.
| littlestymaar wrote:
| Had you read the linked blog post, you'd have seen that
| here this isn't so much an issue with LLVM having too
| much work, but rustc being currently unable to split the
| work into parallelizable chunks before sending it to
| LLVM, and as such it takes a very long time not because
| LLVM has too much things to do, but because it does it in
| a single-threaded fashion, leaving tons of performance on
| the table.
|
| > Rust is heavily designed around "zero-cost
| abstraction", ie. generate tons of IR and let the backend
| sort it out.
|
| Those two aren't equivalent: Rust is indeed designed
| around zero-cost abstraction, and it currently generates
| tons of IR for the backend to optimize, but it doesn't
| have to, it could run some optimizations in the front-end
| so it generates less IR. In fact there has been ongoing
| work to do exactly this to improve compiler performance.
| But this required rearchitecturing the compiler in depth
| (IIRC Rust's MIR has been introduced for that very
| reason).
| saghm wrote:
| The lexer hack is a C thing, and Ive rarely heard anyone
| complain about C compiler performance. That seems more like
| an argument that the grammar doesn't have that much of an
| impact on compiler performance as other things.
| imtringued wrote:
| Yeah. It's exactly backwards, because good language
| design doesn't make anything except parsing faster. The
| problem is that some languages have hideously awful
| grammars that make things slower than they ought to be.
|
| The preprocessor approach also generates a lot of source
| code that then needs to be parsed over and over again.
| The solution to that isn't language redesign, it's to
| stop using preprocessors.
| uecker wrote:
| The preprocessor does not necessarily create a lot of
| source code in C. I can, if you expand arguments multiple
| times and there is an exponential explosion for nested
| macros, but this also easy to avoid.
| kibwen wrote:
| It's certainly possible to think of language features that
| would preclude trivially-achievable high-performance
| compilation. None of those language features that are present
| in Rust (specifically, monomorphized generics) would have ever
| been considered for omission, regardless of their compile-time
| cost, because that would have compromised Rust's other goals.
| panstromek wrote:
| There are many more mundane examples of language design
| choices in rust that are problematic for compile time.
| Polymorphization (which has big potential to speed up compile
| time) has been blocked on pretty obscure problems with
| TypeId. Procedural macros require double parsing. Ability to
| define items in function bodies prevents skipping parsing
| bodies. Those things are not essential, they could pretty
| easily be tweaked to be less problematic for compile time
| without compromising anything.
| kibwen wrote:
| This is an oversimplification. Automatic polymorphization
| is blocked on several concerns, e.g. dyn safety (and
| redesigning the language to make it possible to paper over
| the difference between dyn and non-dyn safe traits imposes
| costs on the static use case), and/or obscure LLVM
| implementation deficiencies (which was the blocker for the
| last time I proposed a Swift-style ABI to address this).
| Procedural macros don't require double-parsing; many people
| do use syn to parse the token stream, but 1) parsing isn't
| a performance bottleneck, 2) providing a parsed AST rather
| than a token stream freezes the AST, which is something
| that the Rust authors deliberately wanted to avoid, rather
| than being some kind of accident of design, 3) at any point
| in the future the Rust devs could decide to stabilize the
| AST and provide a parsed representation, so this isn't
| anything unfixable that would cause any sort of trauma in
| the community, 4) proc macro expansions are trivially
| cacheable if you know you're not doing arbitrary I/O, which
| is easy to achieve manually today and should absolutely be
| built-in to the compiler (if for no other reason than
| having a sandboxed dev environment), but once again this is
| easy to tack on in future versions. As for allowing item
| definitions in function bodies, I want to reiterate that
| parsing is not a bottleneck.
| zozbot234 wrote:
| AIUI, "Swift-style" ABI mechanisms are heavily dependent
| on alloca (dynamically-sized allocations on the stack)
| which the Rust devs have just proposed backing out of a
| RFC for (i.e. give up on it as an approved feature for
| upcoming versions of Rust) because it's too complex to
| implement, even with existing LLVM support for it.
| kibwen wrote:
| Indeed, an alloca-heavy ABI was what I proposed, and I'm
| aware that the Rust devs have backed away from unsized
| locals, but these are unrelated. I was never totally
| clear on the specific LLVM-related problem with the
| former (it can't be totally insurmountable, because
| Swift), but people more knowledgeable in LLVM than I
| seemed uneasy about the prospect. As for the latter, it's
| because the precise semantics of unsized locals are
| undetermined, and it's not clear how to specify them
| soundly (which carries a lot of weight coming from Ralf
| Jung).
| zozbot234 wrote:
| Why does it have to be soundly specified though? Why not
| just provide an unsafe feature that works the same as
| existing C/LLVM, and "leave no room for a lower-level
| language"?
|
| The safe featureset around it can always come later if
| the issues around how to specify it are worked out.
| kibwen wrote:
| The difficulty is that `unsafe` doesn't mean "yolo", it
| means "there are memory safety invariants here that you,
| the programmer, must manually uphold", so we still need
| to consider what those invariants would be. I'm sure that
| Ralf Jung would be happy to talk more about this if
| anyone has any ideas for how to move forward:
|
| _" With #111374, unsized locals are no longer blatantly
| unsound. However, they still lack an actual operational
| semantics in MIR -- and the way they are represented in
| MIR doesn't lend itself to a sensible semantics; they
| need a from-scratch re-design I think. We are getting
| more and more MIR optimizations and without a semantics,
| the interactions of unsized locals with those
| optimizations are basically unpredictable. [...] If they
| were suggested for addition to rustc today, we'd not
| accept a PR adding them to MIR without giving them
| semantics. Unsized locals are the only part of MIR that
| doesn't even have a proposed semantics that could be
| implemented in Miri. (We used to have a hack, but I
| removed it because it was hideous and affected the entire
| interpreter.) I'm not comfortable having even an unstable
| feature be in such a bad state, with no sign of
| improvement for many years. So I still feel that unsized
| locals should be either re-implemented in a well-designed
| way, or removed -- the current status is very
| unsatisfying and prone to bugs."_
|
| https://github.com/rust-
| lang/rust/issues/48055#issuecomment-...
| zozbot234 wrote:
| > However, they still lack an actual operational
| semantics in MIR -- and the way they are represented in
| MIR doesn't lend itself to a sensible semantics
|
| That's an issue with how the MIR for this feature has
| been defined, not with the feature itself. The claim that
| the implementation should be reworked from the ground up
| is one that I might agree with, but the recent proposal
| to back out of an existing RFC suggests that the devs see
| alloca itself as problematic. And that's bad news if you
| intend to use alloca throughout as a foundation for your
| Swift-like ABI support...
| panstromek wrote:
| Just to clarify - polymorphization is not automatic dyn
| dyspatch or anything related. I'm talking about compile
| time optimization that avoids duplicating generic
| functions.
| kibwen wrote:
| Yes, I bring up dyn because the most straightforward way
| to implement polymorphization would be to conceptually
| replace usages of T: Trait with dyn Trait.
| WhyNotHugo wrote:
| Macros themselves are a terrible hack to work around
| support for proper reflection.
|
| The entire Rust ecosystem would be reshaped in such
| fascinating ways if we had support for reflection. I'd love
| to see this happen one day.
| kibwen wrote:
| _> Macros themselves are a terrible hack to work around
| support for proper reflection._
|
| No, I'm not sure where you got this idea. Macros are a
| disjoint feature from reflection. Macros exist to let you
| implement DSLs and abstract over syntax.
| adrian17 wrote:
| They are disjoint, but the things you can use them for
| overlap a lot. In particular, I'd dare say a majority of
| existing `#[derive]`-style macros might be easier to
| implement in a hypothetical reflection layer instead.
|
| Instead of taking a raw token stream of a struct, parsing
| it with `syn` (duplicating the work the compiler does
| later), generating the proper methods and carefully
| generating trait checks for the compiler to check in a
| later phase (for example, `#[derive(Eq)] struct S(u16)`
| creates an invisible never-called method just to do `let
| _: ::core::cmp::AssertParamIsEq<u16>;` so the compiler
| can show an error 20s after an incorrectly used macro
| finished), just directly iterate fields and check
| `field.type.implements_trait(Eq)` inside the derive macro
| itself.
|
| That said, that's just wishful thinking - with how
| complex trait solving is, supporting injecting custom
| code in the middle of it (checking existing traits and
| adding new trait impls) might make compile time even
| worse, assuming it's even possible at all. It's also not
| a clear perf win if a reflection function were to run on
| each instantiation of a generic type.
| MrJohz wrote:
| If you look at how macros are mostly used, though, a lot
| of that stuff could be replaced directly with reflection.
| Most derive macros, for example, aren't really interested
| in the syntax of the type they're deriving for, they're
| interested in its shape, and the syntax is being used as
| a proxy for that. Similarly, a lot of macros get used to
| express relationships between types that cannot be
| expressed at the type system level, and are therefore
| expressed at a syntactic level - stuff like "this trait
| is derived for all tuples based on a simple pattern".
|
| There are _also_ proc macros just for creating DSLs, but
| Rust is already mostly expressive enough that you don 't
| really need this. There are some exceptions, like sqlx,
| that really do embed a full, existing DSL, but these are
| much rarer and - I suspect - more of a novelty than a
| deeply foundational feature of Rust.
| kibwen wrote:
| I'm not intending to say that reflection is useless, but
| rather to say that judging macros harshly due to not
| being reflection would be incorrect.
| MrJohz wrote:
| But the point is that if you've got reflection (and an
| expressive base language, and a powerful enough type
| system, etc), you probably don't need macros. They're a
| heavy mallet when you almost always need a more precise
| tool. And the result of using macros is almost always
| worse than using that more precise tool - it will be
| harder to debug, it will play worse with tools like LSPs,
| it will be more complicated to read and write, it will be
| slower, etc.
|
| I think macros are a necessarily evil in Rust, and I use
| them myself when writing Rust, but I think it's
| absolutely fair to judge macros harshly for being a worse
| form of many other language features.
| Kranar wrote:
| No disagreement on your point, but this is a different
| argument than claiming that macros are an ugly hack to
| workaround lack of reflection.
|
| Because Rust lacks reflection macros are used to provide
| some kind of ad-hoc reflection support, that much we
| agree... but macros are also used to provide a lot of
| language extensions other than reflection support. Macros
| in general exist to give users some ability to introduce
| new language features and fill in missing gaps, and yes
| reflection is one of those gaps. Variadics are another
| gap, some error handling techniques is yet another, as
| are domain specific languages like compile time regex!
| and SQL query macros.
| MrJohz wrote:
| But the point is that almost all of the common places
| where macros are used in everyday Rust could be replaced
| by reflection. There are exceptions like some of the ones
| you mention, but these are clever hacks rather than
| materially useful. Yes, you can write inline SQL and get
| it type checked, but you can also use a query builder or
| included strings and get the same effects but in a much
| less magical and brittle package.
|
| Macros in Rust are primarily a tool to handle missing
| reflection capabilities, and them enabling other code as
| well is basically just a side effect of that.
| Ygg2 wrote:
| You got it the other way around. Macros can implement
| (nearly) anything. Including some form of opt-in type
| introspection via derive macros.
|
| They weren't a hack to get reflection. They are a way to
| codegen stuff easily.
| swsieber wrote:
| What about crates as the unit of compilation? I am genuinely
| curious because it's not clear to me what trade-offs there
| are around that decision.
| pornel wrote:
| It's a "unit" in the sense of calling `rustc` once, but
| it's not a minimal unit of work. It's not directly
| comparable to what C does.
|
| Rust has incremental compilation within a crate. It also
| splits optimization work into many parallel codegen units.
| The compiler front-end is also becoming parallel within
| crates.
|
| The advantage is that there can be common shared state
| (equivalent of parsing C headers) in RAM, used for the
| entire crate. Otherwise it would need to be collected,
| written out to disk, and reloaded/reparsed by different
| compiler invocations much more often.
| nicoburns wrote:
| > Rust has incremental compilation within a crate. It
| also splits optimization work into many parallel codegen
| units.
|
| Eh, it does, but it's not currently very good at this in
| my experience. Nothing unfixable AFAIK (and the parallel
| frontend can help (but is currently a significant
| regression on small crates)), but currently splitting
| things into smaller crates can often lead to much faster
| compiles.
| littlestymaar wrote:
| Yes the actual implementation is far from what could be,
| but the argument was that it's not a language design
| issue, but an implementation one.
| nicoburns wrote:
| Agreed on that
| kibwen wrote:
| All compilers have compilation units, there's not actually
| much interesting about Rust here other than using the word
| "crate" as a friendlier term for "compilation unit".
|
| What you may be referring to instead is _Cargo 's_ decision
| to re-use the notion of a crate as the unit of _package
| distribution_. I don 't think this was necessarily a bad
| idea (it certainly made things simpler, which matters when
| you're bootstrapping an ecosystem), but it's true that
| prevailing best practices since then have led to Rust's
| ecosystem having comparatively larger compilation units
| (which itself isn't necessarily a bad thing either; larger
| compilation units do tend to produce faster code). I would
| personally like to see Cargo provide a way to decouple the
| unit of distribution from the unit of compilation, which
| would give us free parallelism (which currently today rustc
| needs to tease out via parallel codegen units (and the
| forthcoming parallel frontend)) and also assuage some of
| the perpetual hand-wringing about how many crates are in a
| dependency tree (which is exactly the wrong measure as
| getting upset about how many source files are in your C
| program). This would be a fully backwards-compatible
| change.
| Someone wrote:
| > would have ever been considered for omission, regardless of
| their compile-time cost, because that would have compromised
| Rust's other goals.
|
| That basically says compiler speed isn't a goal at all for
| Rust. I think that's not completely true, but yes, speed of
| generated code definitely ranks very high for rust.
|
| In contrast, Wirth definitely had the speed at which the
| Oberon compiler compiled code as a goal (often quoted as that
| he only added compiler optimizations if they made the
| compiler itself so much faster that it didn't become slower
| because of the added complexity, but I'm not sure he was that
| strict)
|
| http://www.projectoberon.net/wirth/CompilerConstruction/Comp.
| .., section 16.1:
|
| _"It is hardly surprising that certain measures for code
| improvement may yield considerable gains with modest effort,
| whereas others may require large increases in compiler
| complexity and size while yielding only moderate code
| improvements, simply because they apply in rare cases only.
|
| Indeed, there are tremendous differences in the ratio of
| effort to gain. Before the compiler designer decides to
| incorporate sophisticated optimization facilities, or before
| deciding to purchase a highly optimizing, slow and expensive
| compiler, it is worth while clarifying this ratio, and
| whether the promised improvements are truly needed.
|
| Furthermore, we must distinguish between optimizations whose
| effects could also be obtained by a more appropriate
| formulation of the source program, and those where this is
| impossible.
|
| The first kind of optimization mainly serves the untalented
| or sloppy programmer, but merely burdens all the other users
| through the increased size and decreased speed of the
| compiler.
|
| As an extreme example, consider the case of a compiler which
| eliminates a multiplication if one factor has the value 1.
| The situation is completely different for the computation of
| the address of an array element, where the index must be
| multiplied by the size of the elements. Here, the case of a
| size equal to 1 is frequent, and the multiplication cannot be
| eliminated by a clever trick in the source program."_
| kibwen wrote:
| _> That basically says compiler speed isn't a goal at all
| for Rust_
|
| No, it says that language design inherently involves
| difficult trade-offs, and the Rust developers consciously
| decided that some trade-offs were worth the cost. And their
| judgement appears to have been correct, because Rust today
| is more successful than even the most optimistic proponent
| would have dared to believe in 2014; that users are asking
| for something implies that you have succeeded to the point
| of having users at all, which is a good problem to have and
| one that nearly no language ever enjoys.
|
| In the context of Oberon, let's also keep in mind that Rust
| is a bootstrapped compiler, and in the early days the Rust
| developers were by far the most extensive users of the
| language; nobody on Earth was more acutely affected by
| compiler performance than they were. They still chose to
| prefer runtime performance (to be competitive with C++)
| over compiler performance (to be competitive with Go), and
| IMO they chose correctly.
|
| And as for the case of Oberon, its obscurity further
| confirms that prioritizing compiler performance at all cost
| is not a royal road to popularity.
| WhyNotHugo wrote:
| One of the issue why compile times are so awful is that all
| dependencies must be compiled for each project.
|
| 20 different projects use the same dependency? They each need
| to recompile it.
|
| This is an effect of the language not having a proper ABI for
| compiling libraries as dynamically loadable modules, which in
| itself presents many other issues, including making
| distribution of software a complete nightmare.
| kibwen wrote:
| _> This is an effect of the language not having a proper ABI
| for compiling libraries as dynamically loadable modules_
|
| No, this is a design decision of Cargo to default to using
| project-local cached artifacts rather than caching them at
| the user or system level. You can configure Cargo to do so if
| you'd like. The reason it doesn't do this by default is
| because Cargo gives crates great latitude to configure
| themselves via compile-time flags, and any difference in
| flags means you get a different compiled artifact anyway. On
| top of that, there's the question of what `cargo clean`
| should do when you have a global cache rather than a local
| one.
| CrendKing wrote:
| Why can't Cargo have a system like PyPI where library
| author uploads compiled binary (even with their specific
| flags) for each rust version/platform combination, and if
| said binary is missing for certain combination, fallback to
| local compile? Imagine `cargo publish` handle the
| compile+upload task, and crates.io be changed to also host
| binaries.
| cesarb wrote:
| > Why can't Cargo have a system like PyPI where library
| author uploads compiled binary
|
| Unless you have perfect reproducible builds, this is a
| security nightmare. Source code can be reviewed (and
| there are even projects to share databases of already
| reviewed Rust crates; IIRC, both Mozilla and Google have
| public repositories with their lists), but it's much
| harder to review a binary, unless you can reproducibly
| recreate it from the corresponding source code.
| woodruffw wrote:
| I don't think it's that much of a security nightmare: the
| basic trust assumption that people make about the
| packaging ecosystem (that they trust their upstreams)
| remains the same whether they pull source or binaries.
|
| I think the bigger issues are probably stability and
| size: no stable ABI combined with Rust's current release
| cadence means that every package would essentially need
| to be rebuilt every six weeks. That's a lot of churn and
| a lot of extra index space.
| aloha2436 wrote:
| > remains the same whether they pull source or binaries.
|
| I don't think that's exactly true, it's definitely
| _easier_ to sneak something into a binary without people
| noticing than it is to sneak it into rust source, but
| there hasn't been an underhanded rust competition for a
| while so I guess it's hard to be objective about that.
| littlestymaar wrote:
| Pretty much nobody does those two things at the same
| time:
|
| - pulling dependencies with cargo - auditing the source
| code of the dependencies they're building
|
| You are either censoring and vetting everything or you're
| using dependencies from crates.io (ideally after you've
| done your due diligence on the crate), but should
| crates.io be compromised and inject malware in the
| crates' payload, I'm ready to bet nobody would notice for
| a long time.
|
| I fully agree with GP that binary vs source code wouldn't
| change anything in practice.
| ninkendo wrote:
| > Pretty much nobody does those two things at the same
| time: - pulling dependencies with cargo - auditing the
| source code of the dependencies they're building
|
| Your "pretty much" is probably weaseling you out of any
| criticism here, but I fully disagree:
|
| My IDE (rustrover) has "follow symbol" support, like
| every other IDE out there, and I _regularly_ drill into
| code I'm calling in external crates. Like, just as often
| as my own code. I can't imagine any other way of working:
| it's important to read code you're calling to understand
| it, regardless of whether it's code made by someone else
| in the company, or someone else in the world.
|
| My IDE's search function shows all code from all crates
| in my dependencies. With everything equal regardless of
| whether it's in my repo or not. It just subtly shades the
| external dependencies a slightly different color. I
| regularly look at a trait I need from another crate, and
| find implementations across my workspace and
| dependencies, including other crates and impls within the
| defining crate. Yes, this info is available on docs.rs
| but it's 1000x easier to stay within my IDE, and the code
| itself is available right there inline, which is way more
| valuable than docs alone.
|
| I think it's insane to not read code you depend on.
|
| Does this mean I'm "vetting" all the code I depend on? Of
| course not. But I'm regularly reading large chunks of it.
| And I suspect a large chunk of people work the way I do;
| There are a lot of eyeballs on public crates due to them
| being distributed as source, and this absolutely has a
| tangible impact on supply chain attacks.
| littlestymaar wrote:
| You answer your own argument here:
|
| > Does this mean I'm "vetting" all the code I depend on?
| Of course not.
|
| Inspecting public facing parts of the code is one thing,
| finding nasty stuff obfuscated in a macro definition or
| in a _Default_ or _Debug_ implementation of a private
| type that nobody is ever going to check outside of
| auditors is a totally different thing.
|
| > My IDE (rustrover) has "follow symbol" support
|
| I don't know exactly how it works for RustRover, since I
| know Jetbrain has reimplemented some stuff on their own,
| but if it evaluates proc macros (like rust-analyzer)
| does, then by the time you step into the code it's too
| late, proc macros aren't sandboxed in any ways and your
| computer could be compromised already.
| ninkendo wrote:
| The point of my argument is not to say I'm vetting
| anything, but to say that there are tons of eyeballs on
| crates today, because of the fact that they are
| distributed as source and not a binary. It's not a silver
| bullet but every little bit helps, every additional
| eyeball makes hiding things harder.
|
| The original claim is that "pretty much no one" reads any
| of their dependencies, in order to support a claim that
| they should be distributed _as binaries_ , meaning "if
| there was no source available at all in your IDE, it
| wouldn't make a difference", which is just a flatly wrong
| claim IMO.
|
| A disagreement may be arising here about the definition
| of "audit" vs "reading" source code, but I'd argue it
| doesn't matter for my point, which is that additional
| eyeballs matter for finding issues in dependencies, and
| seeing the source of your crates instead of a binary blob
| is essential for this.
| littlestymaar wrote:
| > The original claim is that "pretty much no one" reads
| any of their dependencies,
|
| No the claim is that very few people read the
| dependencies[1] _enough to catch a malicious piece of
| code_. And I stand by it. "Many eyeballs" is a much
| weaker guarantee when people are just doing "go to
| definition" from their code (for instance you 're never
| gonna land on a build.rs file this way, yet they are
| likely the most critical piece of code when it comes to
| supply chain security).
|
| [1] (on their machines, that is if you do that on github
| it doesn't count since you have no way to tell it's the
| same code)
| ninkendo wrote:
| > No the claim is that very few people read the
| dependencies[1] enough to catch a malicious piece of
| code.
|
| You're shifting around between reading enough to catch
| _any_ issue (which I could easily do if a vulnerability
| was right there staring at me when I follow symbol) to
| catching _all_ issues (like your comment about build.rs.)
| Please stick with one and avoid moving goal posts around.
|
| There exists a category of dependency issues that I could
| easily spot in my everyday reading of my dependencies'
| source code. It's not all of them. Your claim is that I
| would spot zero of them, which is overly broad.
|
| You're also trying to turn this into a black-or-white
| issue, as if to say that if it isn't _perfect_ (ie. I
| don't regularly look at build.rs), it isn't worth
| _anything_ , which is antithetical to good security. The
| more eyeballs the better, and the more opportunities to
| spot something awry, the better.
| littlestymaar wrote:
| I'm not moving the goal post, a supply chain attack is an
| _adversarial situation_ it is not about spotting an issue
| occurring at random, it is about spotting an issue
| specially crafted to avoid detection. So in practice you
| are either able to spot _every kind of issues_ , or _none
| of the relevant ones_ because if there 's one kind that
| reliably slips through, then you can be certain that the
| attacker will focus on this kind and ignore the trivial
| to spot ones.
|
| If anything, having access to the source code gives you
| an illusion of security, which is probably the worse
| place to be in.
|
| The worse ecosystem when it comes to supply chain attacks
| is arguably the npm one, yet there anyone can see the
| source and there are almost two orders of magnitude more
| eyeballs.
| ninkendo wrote:
| In such an environment I'm doomed anyway, even if I'm
| vetting code. I don't understand why the goal has to be
| "the ability to spot attacks specifically designed to
| prevent you from detecting." For what you're describing,
| there seems to be no hope at all.
|
| It's like if someone says "don't pipe curl into bash to
| install software", ok that may or may not be good advice.
| But then someone else says "yeah, I download the script
| first and give it a cursory glance to see what it's
| doing", wouldn't you agree they're marginally better off
| than the people who just do it blindly?
|
| If not, maybe we just aren't coming from any mutual
| shared experience. It seems flatly obvious to me that
| being able to read the code I'm running puts me in a
| better spot. Maybe we just fundamentally disagree.
| littlestymaar wrote:
| > It's like if someone says "don't pipe curl into bash to
| install software", ok that may or may not be good advice.
| But then someone else says "yeah, I download the script
| first and give it a cursory glance to see what it's
| doing", wouldn't you agree they're marginally better off
| than the people who just do it blindly?
|
| I don't agree with your comparison, in this case it's
| more like downloading, then running it without having
| read it and then every once in a while look at a snippet
| containing a feature that interest you.
|
| The comparison to "download the script and read it before
| you run it" would be to download the crate's repo, read
| it and then vendor the code you've read to use as a
| dependency, which is what I'd consider proper vetting (in
| this case the attacker would need to be much more
| sophisticated to avoid detection, it's still possible but
| in this case at least you've actually gained something),
| but it's a lot more work.
| littlestymaar wrote:
| No stable ABI doesn't mean the ABI changes at every
| release though.
| hobofan wrote:
| It might as well. If there is no definition of an ABI,
| nobody is going to build the tooling and infrastructure
| to detect ABI compatibility between releases and leverage
| that for the off-chance that e.g. 2 out of 10 successive
| Rust releases are ABI compatible.
| littlestymaar wrote:
| Why wouldn't they do exactly that if they decided to
| publish binary crates...
|
| Nobody does that right now because there's no need for
| that, but it doesn't mean that it's impossible in any
| way.
|
| Stable ABI is a massive commitment that has long lasting
| implications, but you don't need _that_ to be able to
| have binary dependencies.
| zozbot234 wrote:
| You can have binary dependencies with a stable ABI;
| they're called C-compatible shared libs, provided by your
| system package manager. And Cargo can host *-sys packages
| that define Rust bindings to these shared libs. Yes, you
| give up on memory safety across modules, but that's what
| things like the WASM Components proposals are for. It's a
| whole other issue that has very little to do with
| ensuring safety within a single build.
| fc417fc802 wrote:
| If you have reproducible builds it's no different.
| Without those binaries are a nightmare in that you can't
| easily link a given binary back to a given source
| snapshot. Deciding to trust my upstream is all well and
| good but if it's literally impossible to audit them
| that's not a good situation to be in.
| woodruffw wrote:
| I think it's already probably a mistake to think that a
| source distribution consistently references a unique
| upstream source repository state; I don't believe the
| crate distribution layout guarantees this.
|
| (I agree that source is easier to review and establish
| trust in; the observation is that _once_ you read the
| upstream source you're in the same state regarding
| distributors, since build and source distributions both
| modify the source layout.)
| nicoburns wrote:
| > Unless you have perfect reproducible builds
|
| Or a trusted build server doing the builds. There is a
| build-bot building almost every Rust crate already for
| docs.rs.
| epage wrote:
| It runs counter to Cargos curreat model where the top-
| level workspace has complete control over compilation,
| including dependencies and compiler flags. I've been
| floating an idea of "opaque dependencies" that are like
| python depending on C libraries or a C++ library
| dependening on a dynamic library.
| littlestymaar wrote:
| That would work for debug builds (and that's something
| that I would appreciate) but not for release, as most of
| the time you want to compile for the exact CPU you're
| targeting not just for say "x86 Linux" to make sure your
| code is optimized properly using SIMD instructions.
| surajrmal wrote:
| A trustworthy distributed cache would also work very well
| for this in practice. Cargo works with sccache. Using
| bazel + rbe can work even better.
| goodpoint wrote:
| That's solved with sccache but even with that compilation
| time is still garbage
| eddd-ddde wrote:
| Dependencies must compile with the right features enabled.
| You can't possibly share the 2^n versions of every binary.
| ABI stability doesn't fix this.
| surajrmal wrote:
| If you use bazel to compile rust, it doesn't suffer from this
| problem. In fact you can get distributed caching as well.
| fngjdflmdflg wrote:
| This was a big reason for dart canceling its previous macros
| attempt (as I understand it). Fast compilation is integral for
| Flutter development - which accounts for a late percentage of
| dart usage - so after IIRC more than two years of developing it
| they still ended up not going through with that iteration of
| macros because it would make hot reload too slow. That degree
| of level-headedness and consideration is worthy of respect IMO.
| krzat wrote:
| Dart is a meh language but their focus on hot reload single
| handedly made it worth it's existence.
| pjmlp wrote:
| Not only language.
|
| Many of complaints towards Rust, or C++, are in reality tooling
| complaints.
|
| As shown on other ecosystems, the availability of interpreters
| or image based tooling are great ways to overcome slow
| optimizating compilers.
|
| C++ already had a go at this back in the early 90's with
| Energize C++ and Visual Age for C++ v4, both based on Common
| Lisp and Smalltalk from their respective owners.
|
| They failed on the market due to the hardware requirements for
| 90's budgets.
|
| Now slowly coming back with tooling like Visual C++ hot reload
| improvements, debugging optimised builds, Live++, Jupiter
| notebooks.
|
| Rational Software started their business selling Ada Machines,
| the same development experience as Lisp Machines, but with Ada,
| lovely inspired on Xerox PARC experience with Mesa and
| Mesa/Cedar.
|
| Haskell and OCaml, besides the slow compilers, have bytecode
| interpreters and REPLs.
|
| D has the super fast dms, with ldc and gdc, for the optimised
| builds suffering from longer compile times.
|
| So while Rust cannot be archited in a different way, there is
| certainly plenty of room for interpreters, REPLs, not compiling
| always from source and many other tooling improvements, within
| the same language.
| hinkley wrote:
| I had a coworker who was using Rational back then, and found
| out one of its killer features was caching of pre compiled
| headers. Whoever changed them had to pay the piper of
| compilation, but everyone else got a copy shipped to them
| over the local network.
| pjmlp wrote:
| Yes, you are most likely talking about ClearMake, the build
| tool used by ClearCase.
|
| It may have required dedicated infra team, but it had
| features that many folks only got to discover with git.
|
| Better save those view description configurations safely.
| Fiahil wrote:
| At some point, the community is also responsible for the
| demanding expectation of a "not slow" compiler.
|
| What's "slow"? What's "fast"? It depends. It depends on the
| program, the programmer, his or her hardware, the day of the
| week, the hour of the day, the season, what he or she had for
| lunch, ...
|
| It's a never ending quest.
|
| I, for exemple, am perfectly happy with the current benchmark
| of the rust compiler. I find a x2 improvement absolutly
| excellent.
| muth02446 wrote:
| The key to unlocking a 10x improvement to compilation speeds
| will like be multithreading. I vaguely remember that LLVM
| struggled with this and I am not sure where it stands today. On
| the frontend side language (not compiler) design will affect
| how well things can be parallelized, e.g. forward declatations
| probably help, mandatory interprocedural anaylyses probably
| hurt.
|
| Having said that, we are in a bad shape when golang compiling
| 40kLOC in 2s is a celebrated achievement. Assuming this is
| single threaded on a 2GHz machine, we 2s * 2GHz / 40kLOC = 100k
| [cycles] / LOC
|
| That seems like a lot of compute and I do not see how this
| cannot be improved substantially.
|
| Shameless plug: the Cwerg language (http://cwerg.org) is very
| focussed on compilation speeds.
| hinkley wrote:
| If the application works poorly for the developers it will
| eventually work poorly for everyone.
|
| Being surrounded by suck slowly creeps into the quality of your
| work.
|
| Computer programming is the only skilled labor I know of where
| people eschew quality tools and think they won't output slop by
| doing so.
| felipeccastro wrote:
| It is ironic how "rewrite it in Rust" is the solution to make
| any program fast, except the Rust compiler.
| jmyeet wrote:
| I'm a big fan of Rust but there are definitely warts that are
| going to be difficult to cure [1]. This is 5 years old now but I
| believe it's still largely relevant.
|
| It is a weird hill to die on for C/C++ devs though, given header
| files and templates creating massive compile-time issues that
| really can't be solved.
|
| Google is known for having infrastructure for compiling large
| projects. They use Blaze (open-sourced at Bazel) to define
| hermetic builds then use large systems to cache object graphs
| (for compilation units) and caching compiled objects because
| Google uses some significant monoliths that would take a
| significant amount of time to compile from scratch.
|
| I wonder what this kind of infrastructure can do for a large Rust
| project.
|
| [1]:https://www.pingcap.com/blog/rust-compilation-model-
| calamity...
| 1718627440 wrote:
| I think there is a massive difference in compile times between
| idiomatic C and C++, so its problematic to be lumping them
| together. But there is also some selection bias since large
| projects tend to migrate from C to C++.
| cozzyd wrote:
| C++ compile times are why I often stick to C, which compiles
| nearly instantly.
| teleforce wrote:
| You can always upgrade to Dlang with modern features and
| conveniences that has comparable compilation time to C for
| the most parts (excluding CTFE).
| daxfohl wrote:
| Maybe these features already exist, but I'd like a way to: 1)
| Type check without necessarily building the whole thing. 2) Run a
| unit test, only building the dependencies of that test. Do these
| exist or are they remotely feasible?
| lpapez wrote:
| You don't even need to ask AI to get an answer to the first
| question, the first hit on both Google and Bing will tell you
| how to do it - it takes 2 seconds!
| panstromek wrote:
| cargo check exists for option 1. For 2.) it depends on the
| project structure. Either way, they don't help as much as you
| would hope for.
| kibwen wrote:
| cargo check absolutely helps as much as I'd hope for, and
| more. It's the basis of my entire workflow and it's like two
| seconds on my codebase.
| Ar-Curunir wrote:
| What? cargo check absolutely helps.
| jtrueb wrote:
| A true champion
|
| > when I started contributing to Rust back in 2021, my primary
| interest was compiler performance. So I started doing some
| optimization work. Then I noticed that the compiler benchmark
| suite could use some maintenance, so I started working on that.
| Then I noticed that we don't compile the compiler itself with as
| many optimizations as we could, so I started working on adding
| support for LTO/PGO/BOLT, which further led to improving our CI
| infrastructure. Then I noticed that we wait quite a long time for
| our CI workflows, and started optimizing them. Then I started
| running the Rust Annual Survey, then our GSoC program, then
| improving our bots, then...
| agumonkey wrote:
| Talk about proper `continuous improvement`
| c-cube wrote:
| I'm worried this person is going to experience a Yak overflow,
| honestly.
| mrec wrote:
| Coincidentally, I discovered this glorious page literally
| five minutes ago:
|
| https://github.com/SerenityOS/yaksplained?tab=readme-ov-
| file...
| culebron21 wrote:
| What a wonderful term. I love it!
| jamesmunns wrote:
| Kobzol is an absolutely wonderful person to work with. I also
| work in the Rust project, and any time I've interacted with
| him, he's been great.
| jadbox wrote:
| Not related to the article, but after years of using Rust, it
| still is a pain in the ass. While it may be a good choice for OS
| development, high frequency trading, medical devices, vehicle
| firmware, finance software, or working on device drivers, it
| feels way overkill for most other general domains. On the other
| hand, I learned Zig and Go both over a weekend and find they run
| almost as fast and don't suffer from memory issues (as much as
| say Java or C++).
| sfvisser wrote:
| This comment would have been more useful with some
| qualification of why that's the case. The language, tooling,
| library ecosystem? Something else?
| skrtskrt wrote:
| For me the hangup is that async is Still Hard. Just a
| ridiculous amount of internal implementation details exposed
| in order to just write, like, an http middleware.
|
| We looked at proposing Rust as the second blessed language in
| addition to Go where I work, and the conclusion was
| basically... why?
|
| We have skilled Go engineers that can drop down to manual
| memory management and squeeze lots of extra performance out
| of it. And it's still dead simple when you don't need to do
| that or the task is suitable for a junior engineer. And
| channels are simply one of the best concurrency primitives
| out there, and baked into the language unlike Rust where
| everything is library making independent decisions. (to be
| fair I haven't tried Elixir/Erlang message passing, I
| understand people like that too).
| akazantsev wrote:
| For Go, it's a design decision. From the start, they strived
| to make compilation as fast as possible.
|
| https://en.wikipedia.org/wiki/Go_(programming_language)#Desi.
| ..
| bobbylarrybobby wrote:
| Not to be that guy who comes to Rust's defense whenever Go is
| mentioned, but... Rust protects from a much larger class of
| errors than just memory safety. For instance, it is impossible
| to invalidate an iterator while iterating over it, refer to an
| unset or invalid value, inadvertently merely shallow copy a
| variable, or forget to lock/unlock a mutex.
| codr7 wrote:
| If only these were common problems that were difficult to
| otherwise avoid.
| 90s_dev wrote:
| Rust feels like wearing a giant bubble just to go outside
| safely.
|
| C++ feels like driving a car. Dangerous but doable and
| often necessary and usually safe.
|
| (Forth feels like being drunk?)
| windward wrote:
| Any non-trivial C++ program includes unchecked int casts
| dvt wrote:
| I like Rust, but I think this post is unfairly downvoted.
| Rustaceans often annoyingly point out that "you can't use
| super-common-footgun X with Rust!" which, while true, they
| also omit the compromises made are immense (frankly,
| compiler performance is one of them).
| Ar-Curunir wrote:
| The parent did not mention any of these compromises,
| beyond claiming that they are uncommon (which is untrue
| for many domains)
| bmitc wrote:
| They are uncommon in many languages though that don't
| require Rust's type system, such as functional languages
| that simply pass values around and nothing else.
| 90s_dev wrote:
| Could you elaborate on the memory issues in all four languages
| that you ran into?
| adrian17 wrote:
| > On this benchmark, the compiler is almost twice as fast than it
| was three years ago.
|
| I think the cause of the public perception issue could be the
| variant of Wirth's law: the size of an average codebase (and its
| dependencies) might be growing faster than the compiler's
| improvements in compiling it?
| IshKebab wrote:
| Yeah definitely when you include dependencies. Also I've
| noticed that when your dependency tree gets above a certain
| size you end up pulling in _every alternative_ crate for a
| certain task, because e.g. one of your dependencies uses
| miniz_oxide and another uses zlib-rs (or whatever).
|
| On the other hand the compile to for _most_ dependencies doesn
| 't matter hugely because they are easy to do in parallel. It's
| always the last few crates and linking that take half the time.
| littlestymaar wrote:
| In my experience working on medium-sized Rust projects (hundreds
| of thousands of LoCs, not millions), incremental compilation and
| mold pretty much solved the problem in practice. I still
| occasionally code on my 13 years old laptop when traveling and
| compilation time is fine even there (for cargo check and debug
| build, that is, I barely ever compile in release mode locally).
|
| What's painful is compiling from scratch, and particularly the
| fact that every other week I need to run _cargo clean_ and do a
| full rebuild to get things working. IMHO this is a much bigger
| annoyance than raw compiler speed.
| sureglymop wrote:
| Yes but are these the default? I want this to work pleasantly
| out of the box so we don't scare new users as quickly.
| littlestymaar wrote:
| Incremental compilation has been the default for a few years
| now. Mold isn't and won't be anytime soon but AFAIK there's a
| PR to use lld by default and that should happen sooner than
| later. It's not exactly as fast as mold but it will still be
| a linking speed boost for everyone by default.
| kristoff_it wrote:
| > Speaking of DoD, an additional thing to consider is the
| maintainability of the compiler codebase. Imagine that we swung
| our magic wand again, and rewrote everything over the night using
| DoD, SIMD vectorization, hand-rolled assembly, etc. It would
| (possibly) be way faster, yay! However, we do not only care about
| immediate performance, but also about our ability to make long-
| term improvements to it.
|
| This is an unfortunate hyperbole from the author. There's a lot
| of distance between DoD and "hand-rolled assembly" and thinking
| that it's fair to put them in the same bucket to justify the
| argument of maintainability is just going to hurt the Rust
| project's ability to make a better compiler for its users.
|
| You know what helps a lot making software maintainable? A Faster
| development loop. Zig has invested years into this and both users
| and the core team itself have started enjoying the fruits of that
| labor.
|
| https://ziglang.org/devlog/2025/#2025-06-08
|
| Of course everybody is free to choose their own priorities, but I
| find the reasoning flawed and I think that it would ultimately be
| in the Rust project's best interest to prioritize compiler
| performance more.
| 90s_dev wrote:
| Yeah but it's _Zig._ Rust is for when you want to write C but
| have it be _easier_. Zig is when you want it to be _harder_
| than C, but with more control over execution and allocation as
| a trade off.
| AndyKelley wrote:
| For anyone who wants to form their own opinion about whether
| this style of programming is easier or harder than it would
| be in other languages:
|
| https://github.com/ziglang/zig/blob/0.14.1/lib/std/zig/token.
| ..
| Rusky wrote:
| The tokenizer is not really a good demonstration of the
| differences between these styles. A more representative
| comparison would be the later stages that build, traverse,
| and manipulate tree and graph data structures.
| AndyKelley wrote:
| OK, parser then:
|
| https://github.com/rust-
| lang/rust/tree/1.87.0/compiler/rustc... (main logic seems
| to be in expr.rs)
|
| vs
|
| https://github.com/ziglang/zig/blob/0.14.1/lib/std/zig/Pa
| rse...
|
| Again, for those who wish to form their own opinions.
| kobzol wrote:
| I think a reasonable comparison would have to be DoD Rust
| parser vs current Rust parser. Comparing across languages
| isn't very useful, because Zig has very different syntax
| rules, and doesn't provide diagnostics near the same
| level as Rust does. The Rust compiler (and also its
| parser) spends an incredible amount of effort on
| diagnostics, to the point of actually trying to parse
| syntax from other languages (e.g. Python), just to warn
| people not to use Python syntax in Rust. Not to mention
| that it needs to deal with decl and proc macros,
| intertwine that with name resolution, etc. etc. This all
| of course hurts parsing performance quite a lot, and IMO
| would make it both much harder to write the whole thing
| in DoD, and also the DoD performance benefits would be
| not so big, because of all the heterogeneous
| functionality the Rust frontend does. Those are of course
| deliberate decisions of Rust that favor other things than
| compilation performance.
| mlugg wrote:
| [edited to correct formatting]
|
| Your points here don't really make sense. There are many
| ways you can apply DoD to a codebase, but by far the main
| one (both easiest and most important) is to optimize the
| in-memory layout of long-lived objects. I won't claim to
| be familiar with the Rust compiler pipeline, but for most
| compilers, that means you'd have a nice compact
| representation for a `Token` and `AstNode` (or whatever
| you call those concepts), but the code between them --
| i.e. the parser -- isn't really affected. In other words,
| all the fancy features you describe -- macros intertwined
| with name resolution, parsing syntax from other
| languages, high-quality diagnostics -- don't care about
| DoD! Our approach in the Zig compiler has evolved over
| time, but we're slowly converging towards a style where
| all of the access to the memory-efficient dense
| representation is abstracted behind functions. So, you
| write your actual processing (e.g. your parser with all
| the features you mention) just the same; the only real
| difference is that when your parser wants to, for
| instance, get a token (as input) or emit an AST node (as
| output), it calls functions to do that, and those
| functions pull out the bytes you need into a lovely
| `struct` or (in Rust terms) `enum` or whatever the case
| may be.
|
| Our typical style in Zig, or at least what we tend to do
| when writing DoD structures nowadays, is to have the
| function[s] for "reading" that long-lived data (e.g.
| getting a single token out from a memory-efficient packed
| representation of "all the tokens") in the implementation
| of the DoD type, and the functions for "writing" it in
| the one place that generates that thing. For instance,
| the parser has functions to deal with writing a
| "completed" AST node to the efficient representation it's
| building, and the AST type itself has functions (used by
| the next phase of the compiler pipeline, in our case a
| phase called AstGen) to extract data about a single AST
| node from that efficient representation. That way, barely
| any code has to actually be aware of the optimized
| representation being used behind the scenes. As mentioned
| above, what you end up with is that the actual processing
| phases look more-or-less identical to how they would
| without DoD.
|
| FWIW, I don't think the parser is our best code here:
| it's one of the oldest "DoD-ified" things in the Zig
| codebase so has some outdated patterns and questionable
| naming. Personally, I'm partial to `ZonGen`[0] as a
| fairly good example of a "processing" phase (although I'm
| admittedly biased!). It inputs an AST and outputs a
| simple tree IR for a subset of Zig which is analagous to
| JSON. Then, for an example of code _consuming_ that
| generated IR, take a look at `print_zoir`[1], which just
| dumps the tree to stdout (or whatever) for debugging
| purposes. The interesting logic is in
| `PrintZon.renderNode` in that file: note how it calls
| `node.get`, and then just has a nice convenient tagged
| union (`enum` in Rust terms) value to work with.
|
| [0]: https://github.com/ziglang/zig/blob/dd75e7bcb1fe142f
| 4d60dc2d...
|
| [1]: https://github.com/ziglang/zig/blob/dd75e7bcb1fe142f
| 4d60dc2d...
| sestep wrote:
| All great points. In case it's relevant, I will also note
| that the rust-analyzer parser is considerably closer to
| DoD than the rustc parser is: https://github.com/rust-
| lang/rust-analyzer/tree/a642aa8023be...
| dgb23 wrote:
| I had almost the exact opposite experience.
| moooo99 wrote:
| I share this impression. I've never worked much with low
| level languages with the exception of a few university
| assignments. I did last years advent of code in Zig and was
| quite productive and never really struggled with the
| language that much. Meanwhile, rust makes things a lot more
| complicated on my current project.
|
| The main benefit of rust over zig seems to be the maturity.
| Rust has had a decade+ to stabilize and build an ecosystem
| while Zig is still a very new language
| littlestymaar wrote:
| Zig is C with a better syntax, easy things are easy to
| do, but for hard things you're on your own with no borrow
| checker friend to cover your back.
|
| Also Rust makes you think more upfront (like string
| handling), which makes it more complicated for _advent of
| code_ kind of stuff but vastly reduces bugs later on.
| Rusky wrote:
| "Hand-rolled assembly" was one item in a list that also
| included DoD. You're reading way more into that sentence than
| they wrote- the claim is that DoD itself also impacts the
| maintainability of the codebase.
| deadfa11 wrote:
| I was working on a zig project recently that uses some complex
| comptime type construction. I had bumped to the latest dev
| version from 0.13, and I couldn't believe how much improvement
| there has been in this area. I am very appreciative of really
| fast iteration cycles.
| Animats wrote:
| This isn't a huge problem. My big Rust project compiles in about
| a minute in release mode. Failed compiles with errors only take a
| few seconds. That's where most of the debugging takes place. Once
| it compiles, it usually works the first time.
| afdbcreid wrote:
| But how big are your big projects?
| Animats wrote:
| About 40,000 lines of my own code, plus a hundred or so
| crates from crates.io.
| johnfn wrote:
| A minute is pretty bad. I understand it may work for your use
| case, but there are plenty of use cases out there where errors
| typically don't fail the compile and a minute iteration time is
| a deal killer. For instance: UI work - good luck catching an
| incorrect color with a compile error. Vite can compile 40,000
| loc and display it on your screen in probably a couple of
| milliseconds.
| kalaksi wrote:
| Compile what language?
| FridgeSeal wrote:
| Probably js given they mentioned Vite; not exactly sure I'd
| call it "compiling" in nearly t he r same order of
| magnitude of complexity though...
| shmerl wrote:
| Consider how long it takes to compile the Linux kernel for
| example. So one minute is very good.
| dominicrose wrote:
| Different programming languages have different qualities. For
| some tasks I like Ruby because it doesn't get in my way. But
| Ruby is built in C and so are JS VMs and web browsers etc
| (C/C++/Rust). A good LLM can convert Ruby code to Rust for a
| 10-100x performance boost, only multiplying the number of
| lines of code by 2. That makes Ruby a good programming
| language and Rust a good target language.
| synthos wrote:
| Regarding AVX: could rust be compiled with different symbols that
| target different x64 instruction sets, then at runtime choose the
| symbol set that is the more performant for that architecture?
| kobzol wrote:
| I'm not sure how that works. You either let the compiler
| compile your whole program with AVX (which duplicates the
| binary) or you manually use AVX with runtime detection on
| selected places (which requires writing manual vectorization).
| johnfn wrote:
| > First, let me assure you - yes, we (as in, the Rust Project)
| absolutely do care about the performance of our beloved compiler,
| and we put in a lot of effort to improve it.
|
| I'm probably being ungrateful here, but here goes anyway. Yes,
| Rust cares about performance of the compiler, but it would likely
| be more accurate to say that compiler performance is, like, 15th
| on the list of things they care about, and they'll happily trade
| off slower compile times for one of the other things.
|
| I find posts about Rust like this one, where they say "ah, of
| course we care about perf, look, we got the compile times on a
| somewhat nontrivial project to go from 1m15s to 1m09s" somewhat
| underwhelming - I think they miss the point. For me, I basically
| only care if compile times are virtually instantaneous. e.g. Vite
| scales to a million lines and can hot-swap my code changes in
| instantaneously. This is where the productivity benefits come in.
|
| Don't just trust me on it. Remember this post[1]?
|
| > "I feels like some people realize how much more polish could
| their games have if their compile times were 0.5s instead of 30s.
| Things like GUI are inherently tweak-y, and anyone but users of
| godot-rust are going to be at the mercy of restarting their game
| multiple times in order to make things look good. "
|
| [1]: https://loglog.games/blog/leaving-rust-gamedev/#compile-
| time...
| kalaksi wrote:
| Isn't Vite for javascript though, which is, of course, a
| scripting language?
|
| Btw, I've used QML and Dioxus with rust (not for games). Both
| make hot reloading the GUI parts possible without recompiling
| since that part is basically not rust (Dioxus in a bit more
| limited manner).
| kobzol wrote:
| You have a fair point, I agree that while compiler performance
| is a priority, is is one of many priorities, and not currently
| super high on the list for many Rust Project developers. I wish
| it was different, but the only thing we can do is just do the
| work to make it faster :) Or support the people that work on
| it.
| kunley wrote:
| The article is fine and has a lot of good points, but tries to
| avoid the main issue like a plague. So I will speak it here:
|
| The slowness comes mainly from LLVM.
| kobzol wrote:
| For many use-cases yes, but there are crates bottlenecked on
| different things than the codegen backend.
|
| But I don't think that's the point. We could get rid of LLVM
| and use other backends, same as we could do other improvements.
| The point is that there are also other priorities and we don't
| have enough manpower to make progress faster.
| kunley wrote:
| Fair reply, thank you.
| panstromek wrote:
| This is somewhat true but also a bit misleading. Lot of the
| problems comes from how rust interacts with it, and how are
| rust projects structured. This ultimately shows up as time in
| LLVM, but LLVM is not entirely responsible for it.
| juliangmp wrote:
| >[...] this will depend on who you ask, e.g. some C++ developers
| don't mind Rust's compilation times at all, as they are used to
| the same (or worse) build times
|
| Yeah pretty much. C++ is a lot worse when you consider the
| practical time spent vs compilation benchmarks. In most C++
| projects I've seen/worked on, there were one or sometimes more
| code generators in the toolchain which slowed things down a lot.
|
| And it looks even more dire when you want to add clang-tidy in
| the mix. It can take like 5 solid minutes to lint even small
| projects.
|
| When I work in Rust, the overall speed of the toolchain (and the
| language server) is an absolute blessing!
| carlmr wrote:
| >And it looks even more dire when you want to add clang-tidy in
| the mix. It can take like 5 solid minutes to lint even small
| projects.
|
| And running all tests with sanitizers, just to get some runtime
| checks of what Rust excludes at compile time.
|
| I love Rust for the fast compile times.
| bnolsen wrote:
| Compilation speed makes go nice. Zig should end up being king
| here depending on comptime use (ie: lack of operators can be
| overcome by using comptime to parse formulae strings for things
| like geometric algebra).
| ruuda wrote:
| The Rust ecosystem is getting slower faster than the compiler is
| getting faster. Libraries grow to add features, they add
| dependencies. Individually the growth is not so bad, and
| justified by features or wider platform support. But they add up,
| and especially dependencies adding dependencies act as a
| multiplier.
|
| I started writing a post about this many years ago, but never
| finished it. I took a few slow-changing projects of mine that had
| a pinned Rust compiler, and then updated both the compiler and
| dependencies to the latest versions. Invariably, everything got
| slower to compile, even though the compiler update in isolation
| made things faster!
| simonask wrote:
| I don't know, these things ebb and flow.
|
| There's a bit of pushback against high-dependency project
| structures and compile times recently, and even niche crates
| like `unsynn` have garnered some attention as an alternative to
| the relatively heavy `syn` crate.
| eddd-ddde wrote:
| When a platform has good support and is easy to onboard this is
| the inevitable result. It's just like the JavaScript ecosystem.
|
| But this is not a downside. Just like I can start a new website
| project and not use a single dependency, I can start a new rust
| project and not install a single dependency.
|
| To me the real value is in the tools and core language feature.
| I could probably implement my own minimal ad-hoc async IO
| framework if I wanted to, and shape it to my needs. No
| dependencies.
| baalimago wrote:
| Seems to me that Rust has hit bedrock.
|
| If there's no tangible solution to this design flaw today, what
| will happen to it in 20 years? My expectation is that the amount
| of dependencies will increase, as will the complexity of the Rust
| ecosystem at large, which will make the compilation times even
| worse.
| zozbot234 wrote:
| It's OK; 20 years ought to be enough time to rewrite LLVM in
| Rust.
| kobzol wrote:
| I don't think we hit a bedrock. As I wrote, we have a lot of
| ideas for massive improvements. But we need more people to work
| on them.
| baalimago wrote:
| >But we need more people to work on them.
|
| That's my point: I don't see how there could be people
| dedicated to work on an issue as grand as this in Rust's
| current organizational form. Especially considering all the
| gotchas, and continuous development of 'more fun' things (why
| work on open source if it's no fun?). That's why it's 'the
| bedrock'.
|
| To do something like that, Rust would need to be forked and
| later on rewritten with optimizations. But by then it
| wouldn't be "Rust" anymore, it would be a sibling language
| with rusty syntax. Rust++, perhaps.
| diggan wrote:
| > I don't see how there could be people dedicated to work
| on an issue as grand as this in Rust's current
| organizational form
|
| You're getting half ways there of giving actionable
| feedback, what exactly is the problem with the current
| organization structure that would prevent any "grand"
| issues like these? Is there a specific point in time when
| you felt like Rust stopped being able to work on these
| grand issues, or it has always been like this according to
| you?
|
| > why work on open source if it's no fun
|
| It's always fun to _someone_ out there, I 'm sure :) There
| are loads of thankless tasks that seemingly get done even
| without having a sexy name like "AI for X". With a language
| as large as Rust, I'm sure there might even be two whole
| people who are salivating at the ideas of speeding up the
| current compiler.
| baalimago wrote:
| >what exactly is the problem with the current
| organization structure that would prevent any "grand"
| issues like these?
|
| Well, it's summarized quite well here:
|
| >"Performing large cross-cutting changes is also tricky
| because it will necessarily conflict with a lot of other
| changes being done to the compiler in the meantime. You
| can try to perform the modifications outside the main
| compiler tree, but that is almost doomed to fail, given
| how fast the compiler changes8. Alternatively, you try to
| land the changes in a single massive PR, which might lead
| to endless rebases (and might make it harder to find a
| reviewer). Or, you will need to do the migration
| incrementally, which might require maintaining two
| separate implementations of the same thing for a long
| time, which can be exhausting." - OP
|
| A rigid organizational form (such as a company) can say:
| "Okay, we'll make an investment here and feature freeze
| until the refactors are done". I have a hard time seeing
| how the open source rust community who are doing this out
| of passion would get on board on such a journey. But
| maybe, who knows! The 'compilation people' would need to
| not only refactor to speed up the compilation times,
| they'd also need to encourage and/or perform refactors on
| all features being developed 'on main'. That, to me,
| sounds tedious and boring. Sort of like a job. Maybe
| something for the rust foundation to figure out.
| diggan wrote:
| I understand that quoted part as "it's tricky" rather
| than "It's impossible because no one wants to do it",
| just like many collaboration-efforts in FOSS. But you're
| right that it's probably for the foundation to figure
| out, a lone compiler-optimization geek isn't gonna be
| able to come up with a project-wide solution and force it
| through.
|
| Haven't the Rust team already implemented "grand
| features" that took many years to get across the finish
| line? For example, GATs didn't look particularly fun,
| exciting or sexy, but somehow after being thought about
| and developed for like 5-6 years eventually landed in
| stable.
|
| Edit: Also just remembered a lot of the work that "The
| Rust Async Working Group" has done, a lot which required
| a large collaborations between multiple groups within
| Rust. Seems to have worked out in the end too.
| zozbot234 wrote:
| This is a concern for any fast-moving project, i.e. it's
| a _good_ problem to have! You can work on your
| modifications on a side branch and then forward port them
| to the current state of main before proposing them for
| merge, it will probably be less work overall.
| baalimago wrote:
| Why haven't Rust been forked by some bigger company, who have the
| time and resources to specialize it into something which fits
| better into a professional market? Yes I'm saying low compilation
| time -> high development RTT is a requirement for the
| professional market.
| WJW wrote:
| Maybe it has been, but said bigger company hasn't published
| their work?
|
| I disagree fast compile times are "required" for the
| professional market btw. They are nice, sure, but there's
| plenty of professional development out there in languages that
| are slow to compile.
| bob1029 wrote:
| > bigger company, who have the time and resources to specialize
| it into something which fits better into a professional market
|
| Welcome to the central thesis for using Microsoft's stack.
|
| If I'm getting paid money based upon the direct outcome of my
| work (I.e., freelance / consulting / 1099), I am taking zero
| chances with the tooling. $500 for a perpetual license of VS
| the cheapest option by miles if you value your time and sanity.
|
| Iteration time is nice, but the debugger experience is the most
| important thing once you are working on problems people are
| actually willing to pay money to solve. Just because it's
| "possible" doesn't mean it is ergonomic or accessible. I don't
| have to exit my IDE if I want to attach to prod and run a
| cheeky snippet of LINQ on a collection in break mode to
| investigate a runtime curiosity.
| kjuulh wrote:
| Could Rust be faster, yes. But honestly, for our use-case
| shipping; tools, services, libraries and what have you in
| production, it is plenty fast. That said, Rust definitely falls
| off a cliff once you get to a very large workspace (I'd say plus
| 100k lines of code it begins to snowball), but you can design
| yourself out of that, unless you build truly massive apps.
|
| Incremental builds doesn't disrupt my feedback loop much, only
| when paired with building for multiple targets at once. I.e.
| Leptos where a wasm and native build is run. Incremental builds
| do however, eat up a lot of space, a comical amount even. I had a
| 28GB target/ folder yesterday from working a few hours on a
| leptos app.
|
| One recommendation is to definitely upgrade your CI workers, Rust
| definitely benefits from larger workers than the default GitHub
| actions runners as an example.
|
| Compilling a fairly simple app, though including DuckDB which
| needs to be compiled, took 28 minutes on default runners. but on
| a 32x machine, we're down to around 3 minutes. Which is fast
| enough that it doesn't disrupt our feedback loop.
| crohr wrote:
| What kind of CI runners do you use then? Do you self-host?
| kjuulh wrote:
| You can rent bigger runners from github. They're still not as
| fast as third party ones, but it takes 5 minutes to set up
| and is still pay as you go. I just see a lot of people use
| the default ones, which are very small.
| lrvick wrote:
| If anyone wants to feel better about compile times for their rust
| programs, try full source bootstrapping the rust compiler itself.
| Took about 2 days on 64 cores until very recently (thanks to
| mrustc 0.74). Now only 7 hours!
| superkuh wrote:
| The biggest problem with the Rust compiler is not it's speed in
| compiling. It's that rustc from 3 months ago can't compile most
| Rust code written today. And don't tell me that cargo versioning
| fixes this, it doesn't. The very improvements we are celebrating
| here, which are very real and appreciated, are part of this
| problem. Rust is young and Rust changes very, very fast. I think
| it'll be a great language in a decade when it's no longer just
| used by bleeding edge types and has a target that stands still
| for more than a few months.
| ModernMech wrote:
| The biggest thing that's happened in recent time to improve Rust
| compiler performance was the introduction the Apple M-series
| chips. On my x86 machine, it'll take maybe 10 minutes for a fresh
| build of my project, but on my Apple machine that's down to less
| than a minute, even on the lower end Mac Mini. For incremental
| builds it only takes a few seconds. I'm fine with this amount of
| compilation time for what it buys me, and I don't feel it slows
| me down any because (and I know this sounds like cope) it gives
| me a minute to breathe and collect my thoughts. Sometimes I find
| that debugging a problem while actively coding in an interactive
| REPL is different from debugging offline.
|
| I'm not sure why but the way I would explain it is when you're
| debugging in an interactive REPL you're always get fast
| incremental result, but you may be going down an unproductive
| rabbit hole and spinning your tires. When I hit that compile
| button, I'm able to take a step back and maybe see the problem
| from another angle. Still, I prefer a short development loop, but
| I do think you lose something from it.
___________________________________________________________________
(page generated 2025-06-13 23:01 UTC)