https://lwn.net/SubscriberLink/952029/412bfd44912e90b2/ LWN.net Logo LWN .net News from the source LWN * Content + Weekly Edition + Archives + Search + Kernel + Security + Events calendar + Unread comments + ------------------------------------------------------------- + LWN FAQ + Write for us User: [ ] Password: [ ] [Log in] | [Subscribe] | [Register] Subscribe / Log in / New account Committing to Rust for kernel code [LWN subscriber-only content] Welcome to LWN.net The following subscription-only content has been made available to you by an LWN subscriber. Thousands of subscribers depend on LWN for the best news from the Linux and free software communities. If you enjoy this article, please consider subscribing to LWN. Thank you for visiting LWN.net! By Jonathan Corbet November 22, 2023 --------------------------------------------------------------------- Maintainers Summit Rust has been a prominent topic at the Kernel Maintainers Summit for the last couple of years, and the 2023 meeting continued that tradition. As Rust-for-Linux developer Miguel Ojeda noted at the beginning of the session dedicated to the topic, the level of interest in using Rust for kernel development has increased significantly over the last year. But Rust was explicitly added to Linux as an experiment; is the kernel community now ready to say that the experiment has succeeded? The Rust-for-Linux project has added a full-time engineer in the last year, Ojeda said, and a student developer as well. Various companies have joined in to support this work. There is also work underway to get the Coccinelle tool working with Rust code. A priority at the moment is bringing in more reviewers for the code that is being posted. On the toolchain front, work on gccrs, the GCC-based Rust compiler, has slowed significantly. The GCC code generator for rustc is showing better progress; it can compile kernel code now and has been merged into the compiler. This GCC-based backend will enable the expansion of Rust support to architectures that are not supported by the LLVM-based rustc. Meanwhile, the Rust project itself is increasing its involvement in this work; this is good, since the kernel has some unique requirements and will need guarantees that language changes won't break kernel code in the future. [Miguel Ojeda] Within the kernel, work is proceeding in a number of subsystems. The Rust implementation of Android's binder is working well and its performance is on a par with the C implementation. The amount of unsafe code that was needed to get there was pleasingly small. Filesystem bindings are the subject of work by Wedson Almeida Filho, who is targeting read-only support for now. The object there is to make it possible to implement a filesystem in 100% safe Rust. In general, he is finding an increasing number of maintainers who are open to the idea of using Rust. That leads to an issue the Rust developers have run up against, though. It would be good to have some reference drivers in the kernel as an example of how drivers can be written and to make it possible to compare Rust and C drivers. The best way to do that often seems to be to merge a Rust driver that duplicates the functionality of an existing C driver -- but that sort of duplicate functionality is not welcomed by maintainers. Perhaps, he said, it would be good to allow a few duplicate drivers that are not meant for actual use, but only as examples for other developers to use. There are some other challenges; upstreaming the block-layer abstractions has run into some resistance. Virtual filesystem layer maintainer Christian Brauner said that he is not opposed to merging those abstractions, but he would rather not do that and see filesystems built on it right away. He would prefer to see an implementation of something relatively simple, along the lines of the binder driver, to show that things work as expected. A driver soon? Dave Airlie, the maintainer of the DRM (graphics) subsystem, said that, if he has his way, there will be a Rust DRM driver merged within the next couple of releases. Christoph Hellwig shot back that Airlie was willing to "make everybody's life hell" so that he could play with his favorite toy. Merging Rust, Hellwig said, would force others to deal with a second language, new toolchains, and "wrappers with weird semantics". Dan Williams said that the current situation "is what success looks like", and that the kernel community was already committed to Rust. Airlie continued that a lot of the Rust work is currently blocked in a sort of chicken-and-egg problem. Abstractions cannot be merged until there is a user for them, but the code needing those abstractions is blocked waiting for code to land in multiple subsystems. As a result, developers working on Rust are dragging around large stacks of patches that they need to get their code to work. Breaking that roadblock will require letting in some abstractions without immediate users. Ojeda agreed that this problem has been slowing progress, but said he has tried not to put pressure on maintainers to merge code quickly. In the case of networking, ironically, the Rust developers had to ask the networking maintainers to slow down merging Rust code. The conversation took several directions from there. Greg Kroah-Hartman said that merging the binder driver would be a good next step; it is self-contained, has a single user that is committed to its maintenance, and doesn't touch the rest of the kernel. Kees Cook disputed the description of Rust as a "toy", saying that there is a lot of pressure to not use C for new code; Hellwig responded that the developers would have to rewrite everything in Rust, otherwise the resulting dual-language code base would be worse than what exists now. Dave Chinner worried that maintainers lack the expertise to properly review the abstractions that are being merged. Airlie replied that maintainers merge a lot of C APIs now without really understanding how they work. A lot of mistakes have been made in the process, but "we're still here". When things turn out to be broken, they can be fixed, and that will happen more quickly if the code goes upstream. Ted Ts'o expressed concern about the burden that adding Rust code will place on maintainers. The Rust developers are setting higher standards than have been set in the past, he said. Getting good abstractions merged is one thing, but who is responsible for reviewing drivers, and how will tree-wide changes be handled? The Rust effort, he said, is getting to a point where it is impacting a growing part of the community. Williams pointed out that the previous session had discussed how hard it is to get kernel subsystems to move to new APIs; now, he said, there is talk of moving to a whole new language. Hellwig said that the real problem is that the Rust bindings tend to work differently than the C APIs they provide abstractions for; the new APIs may well be better, but they are still completely new APIs. What should be done, he said, is to first fix the C APIs so that they are directly usable by Rust code. He proposed that, for each subsystem that is considering bringing in Rust code, a year or two should first be spent on cleaning up its APIs along those lines. Ojeda said that this kind of API improvement has already happened in some subsystems. Linus Torvalds said that he was seeing a divide between the filesystem and driver maintainers. Developers on the filesystem side tend to be more conservative, while the driver world "is the wild west". Driver authors tend not to understand concurrency, he said, and a lot of the code there is broken and unfixable. So it is unsurprising that there is interest in bringing in a language that better supports the writing of correct and safe code. Brauner said that Rust can help with a lot of problems, since the compiler can keep a lot of bugs from making it into the kernel. But he worried about whether there would be maintainer and development support for it a few years from now. Airlie again mentioned developers with out-of-tree code needed by Rust code; Cook answered that the people shepherding that code are maintainers, and that bringing it in would bring the maintainers with it. Airlie added that those maintainers are the sort of younger developers that the kernel community would like to attract. Chinner said that he would like to see a reimplementation of the ext2 filesystem in Rust. It is a complete filesystem that makes wide use of the kernel's APIs, but it is still small enough to read and understand. If the Rust APIs can support an ext2 implementation, they will be enough to implement others as well. Meanwhile, the ext2 implementation would be good reference for maintainers, who could compare it to the C version. Confidence Ts'o asked when the community would feel enough confidence that it could have modules where the only implementation is in Rust. Binder could be a good start, he said, perhaps followed by a driver that sees wider use. Airlie said that he is considering a virtual graphics driver that reimplements an existing C driver. There is also the driver for Apple M1 GPUs. He is feeling a fair amount of pressure to get it upstream and is wondering if there is any reason why he should keep it out. After that, he would love to see a rewrite of the Nouveau driver for NVIDIA GPUs. Arnd Bergmann said those drivers could be OK, but that it will be quite a bit longer before something like a keyboard driver could be merged; the toolchain just isn't ready, he said, for a driver what would be widely used. That led to a question about the frequent version upgrades being seen in the kernel, which moved to Rust 1.73.0 for 6.7. That upgrade process will eventually stop and a minimum Rust version will be set once the important features that the kernel depends on have stabilized. He said that he has been working to get the kernel code into the Rust continuous-integration tests to help ensure that it continues working as the compiler and language evolve. Bergmann said that he didn't plan to look seriously at the language until it could be compiled with GCC. Torvalds answered that, while he used to find problems in the LLVM Clang compiler, now he's more likely to find problems with GCC instead; he now builds with Clang. Ojeda said that he is working on finding developer resources for gccrs; the project is currently sitting on over 800 out-of-tree patches and still has a lot of work to do on top of that. GCC support will be a while, he said. Ts'o complained that the language still isn't entirely stable. This could be a particular problem for the confidential-computing community; they are concerned about security and, as a consequence, about backports to long-term-support kernels. But if those kernels are on different Rust versions, those backports will be problematic. Ojeda said that, while it is a "crazy idea", backporting the version upgrades could be considered. He doesn't think that the change rate will be high enough to be a problem, though. At the conclusion, Torvalds pointed out that there have been problems over the years with GCC changes breaking the kernel; the same will surely happen with Rust, but it will be the same thing in the end. The session, well over time, was brought to a halt at this point. Whether the kernel community has truly concluded that it is committed to Rust remains to be seen; there will almost certainly be pull requests adding significant Rust code in the near future. Index entries for this article Kernel Development tools/Rust Conference Kernel Maintainers Summit/2023 [Send a free link] ----------------------------------------- (Log in to post comments) Committing to Rust for kernel code Posted Nov 22, 2023 15:32 UTC (Wed) by Paf (subscriber, #91811) [Link ] Certainly fascinating to watch this go by...! [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 15:53 UTC (Wed) by tux3 (subscriber, #101245) [ Link] This level of tension between maintainers who aren't too thrilled to bear the extra cost and others who deem it worth it, doesn't seem like it is going away on its own. Has there been much discussion about possible forms of analgesia that I might have missed? Perhaps people willing to answer questions, resources, or other commitments from the Rust side that may help? [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 16:01 UTC (Wed) by corbet (editor, #1) [Link] The Rust-for-Linux folks have been clear from the beginning that they understand the costs and that they plan to be there to help maintainers through this process. [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 16:22 UTC (Wed) by willy (subscriber, #9762) [ Link] Agreed with Dave that ext2-in-rust would be a very worthwhile project. I've reviewed the most recent batch of VFS abstractions and they suffer from "we're just implementing toy filesystems" syndrome. That's not to say that I want anyone to disappear for a year and come back with rext2. I'd like to see somebody working on it publicly with regular postings about what they're finding. In particular, I'd like to see such a filesystem either following the existing C s_op / f_op / i_op / a_ops abstractions, or an explanation of why they are bad and a proposal for what a better set of methods would be. I don't think we've necessarily got the perfect abstractions, but they are here and work well enough that we have ~80 filesystems. This is a great chance to fix them up if there's a better model. Also, rext2 shouldn't be a "port" of C ext2. It should be a reimplementation of the filesystem, using modern infrastructure (ie not buffer heads). [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 22:50 UTC (Wed) by amiculas (guest, #167443) [ Link] Just so I can be less ignorant, what's the modern alternative to buffer heads? [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 22:55 UTC (Wed) by willy (subscriber, #9762) [ Link] Usually, it's iomap. Ritesh has done an excellent patch set this week that uses iomap for data. For directories, we're not really sure what the right abstraction is -- should we use the page cache raw; should we have a thinner version of buffer heads; should we use XFS buffer cache; should we use iomap? All TBD. [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 16:39 UTC (Wed) by b7j0c (subscriber, #27559) [ Link] Even though I am enjoying coding in Rust myself and advocate it to others, I still feel like I am in the uninformed minority that is against introducing a major tool-based segmentation into the kernel. Given that the C will always be part of the kernel, isn't the best-case scenario here splitting kernel dev into "Rust side" and "C side"? Maybe some better informed folks here can clarify... [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 18:26 UTC (Wed) by pizza (subscriber, #46) [Link] > Given that the C will always be part of the kernel, A primary goal of this Rust-in-Linux effort is to change that via a kernel-of-thesius approach. First make it so that, if you have a certian common platform (Android) or hardware (Apple), you need to use Rust. Then you incrementally push it it deeper into core layers, effectively making it a requirement for every other platform. Once it's required period, then incrementally replace or drop the rest (obsolete filesystems, device drivers, etc) until it's all Rust. So they may or may not be explicitly stating it, but making Linux Rust-First is the only logical conclusion. (Now if I want to go more conspiratorly-minded, once can also consider the Rust-on-Linux efforts to be an extension of the decades-long effort to purge dependencies on copyleft tooling, notably GCC. Linux is copyleft in name only, due to an utter lack of enforcement) [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 18:51 UTC (Wed) by Wol (subscriber, #4433) [Link] > (Now if I want to go more conspiratorly-minded, once can also consider the Rust-on-Linux efforts to be an extension of the decades-long effort to purge dependencies on copyleft tooling, notably GCC. Linux is copyleft in name only, due to an utter lack of enforcement) In which case, wouldn't it make sense for all kernel Rust code to be liberally licenced? I'm thinking about the same thing with Scarlet. Not that I probably will, but re-writing it in Rust might be an idea, and requiring Rust code to be MPL, or Apache, or MIT, or something like that might not be a bad idea. Personally, I'd probably go down the MPL route, although the GPL isn't particularly appropriate for Scarlet and can actually be ignored for the most part ... Cheers, Wol [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 19:03 UTC (Wed) by josh (subscriber, #17465) [ Link] Absolutely not. Copyleft is a good thing, let's not weaken it. As a Rust developer, it makes me sad that rustc is permissively licensed. Just about the only *good* thing about it being permissively licensed is that that makes a good argument that there's no reason to need a second one. :) [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 19:15 UTC (Wed) by Wol (subscriber, #4433) [Link] > Absolutely not. Copyleft is a good thing, let's not weaken it. But *W*H*Y* is it a good thing? It makes large swathes of endeavour economically unviable. Open Source that respects the four freedoms is fine too! To my mind, Copyleft falls fairly seriously foul of the law of unintended consequences. Nature unfortunately is full of "pick two, pick any two" scenarios, and why is copyleft any different? But anyways, I was merely responding to pizza's comment about moving away from copyleft stuff, and if it was a conspiracy theory, the easy way to carry out that conspiracy would be to make sure all kernel Rust code was liberally licenced. Cheers, Wol [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 21:03 UTC (Wed) by gdamjan (subscriber, #33634) [ Link] Isn't there FreeBSD for people wanting a BSD licensed kernel? Leave our kernel GPL2 [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 22:52 UTC (Wed) by Karellen (subscriber, #67644) [Link] But *W*H*Y* is it a good thing? Because ensuring that all users have the same freedoms is a good thing. Because allowing one set of users (3rd-party devs) to take the freedoms they have been given, away from other end-users, so that those end-users cannot make use of those freedoms themselves (or pay others to do so on their behalf), is - you know - unjust. Because code is not scarce. It can be infinitely copied, and me sharing code I have received with you does not decrease the utility that that code has for me. Me giving you the same rights I was given, does not diminish mine. The claim that "I am not free unless I have the freedom to take away other people's freedoms" is, IMHO, absurd. It makes large swathes of endeavour economically unviable. Maybe this being an issue is a problem with economics, not with code? [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 7:44 UTC (Thu) by Wol (subscriber, #4433) [Link] > The claim that "I am not free unless I have the freedom to take away other people's freedoms" is, IMHO, absurd. Yes it is absurd. Not least because it's circular, and applies *EQUALLY* to GPL vs MIT. GPL takes away (in considerable part) developers' freedom to make a living. Even its pre-amble says it takes away freedoms, iirc. If we, as developers, *choose* to pass on our MIT freedoms to our customers, then everyone gets the best of both worlds. The problem is, this is a social choice, a *freedom*, and, as I keep saying, this falls squarely into nature's "pick any two" conundrum. How do you square the circle? I don't think you can ... Cheers, Wol [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 8:21 UTC (Thu) by ssmith32 (subscriber, #72404) [ Link] What's "the freedom to make a living"? Based on your past comments, I'm pretty sure my knee-jerk reading of that as an entitled "right to make a living" is most certainly off - that is, it's a genuine question.. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 13:14 UTC (Thu) by Wol (subscriber, #4433) [Link] I think it's all down to the problem that freely available source code makes it hard to earn money from writing that source. Sure, some companies (eg Red Hat) make money supporting that source. While it's hard to put a finger on it, I just get the impression that more people make a living from OSS code that isn't Copyleft. And OSS makes it easier to sell the resulting software, even if you do "live by the spirit" and release the corresponding source. Maybe I am a bit naive, but I believe most people agree with "live by the spirit", and if you do your best only to associate with others like that, OSS is much better than copyleft. OSS came before copyleft (in practice), and looks like outliving copyleft. The difficulty is probably down to the fact that the software industry makes money from selling software (and the GPL interferes with this), while the world in general saves money by sharing software, and OSS helps this massively. Two completely opposing financial incentives, two completely mismatched mindsets. Cheers, Wol [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 13:18 UTC (Thu) by Wol (subscriber, #4433) [Link] Just to add, I said the GPL takes away developer freedom. Because it quite openly seeks to ensure *end*user* freedom. So software producers naturally favour OSS? Yep, people whos' business is software production prefer closed licences, but if software is a byproduct, OSS looks the best bet. Cheers, Wol [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 17:42 UTC (Thu) by ballombe (subscriber, #9523) [ Link] > Just to add, I said the GPL takes away developer freedom. Because it quite openly seeks to ensure *end*user* freedom. Every developer is using far more code than what they contributed, so globally, the GPL is still a win for them. Yes, you can say it is the "prisoner dilemma" all over and again and use MIT license to get an edge, but the "prisoner dilemma" terminology calls it "being selfish". I will not go that far because release code under the MIT license is very generous but you see the point I making. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 18:16 UTC (Thu) by Cyberax ( supporter , # 52523) [Link] > Every developer is using far more code than what they contributed, so globally, the GPL is still a win for them. Why? It's vice-versa. MIT is more advantageous for individual developers. > Yes, you can say it is the "prisoner dilemma" all over and again and use MIT license to get an edge, but the "prisoner dilemma" terminology calls it "being selfish". I think you might be confusing the "selfish" vs "selfless" choices here. GPL is the "selfish" choice, while MIT is a "selfless" choice that results in a better outcome if everyone cooperates. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 19:41 UTC (Thu) by pbonzini (subscriber, #60935) [Link] > MIT is more advantageous for individual developers. Only if they are not interested in using any of Linux, GCC, glibc, GMP, GNOME, KDE, git, etc., free equivalents of which might or might not exist in a world without copyleft. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 8:27 UTC (Thu) by LtWorf (subscriber, #124958) [ Link] > GPL takes away (in considerable part) developers' freedom to make a living. Au contraire :) MIT takes away that possibility by letting for example google or apple use that code. But gpl3 or agpl would force most companies to reimplement, and consequently hire someone to re-do the work. The way I see it, with GPL the community is getting an advantage, with MIT some stakeholders are getting an advantage. Developers who use GPL can also double license if they want. What you mean is that GPL is taking away the freedom to make a living off of someone else's work without contributing anything. Arguably also getting on a drakkar and raiding england is a way to make a living that current pesky laws have forbidden. Perhaps not all ways of making a living are equally good? [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 19:01 UTC (Wed) by Phantom_Hoover (subscriber, # 167627) [Link] > A primary goal of this Rust-in-Linux effort is to change that via a kernel-of-thesius approach. Is this something supported by statements people have actually made, or something you've invented to support your grand conspiracy theory of a purge of copyleft software? Why would this conspiracy need to undertake the expense of rewriting the kernel in Rust to break its dependency on GCC when the same thing could be and has been achieved much more easily just by building it with Clang? [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 19:26 UTC (Wed) by pizza (subscriber, #46) [Link] >> A primary goal of this Rust-in-Linux effort is to change that via a kernel-of-thesius approach. > Is this something supported by statements people have actually made, or something you've invented As I said in the message you responded to, it's the only logical goal for all of this effort, especially in light of the article's numerous cited examples of the fundamental conflicts between the C and Rust camps in the Kernel. (This is also an opinion shared by numerous other kernel developers, fueling much of the skepticism in the article's citations!) Mind you, I'm not saying this goal is inherently a bad thing, but it doesn't make a whole lot of sense to go through all of this effort and the immense ongoing maintenance burden of having two separate camps, bifrucating hardware support based on tooling and subtly different APIs unless the goal is to eliminate C from the highly complex "core" subsystems like MM, VFS, and so forth. [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 19:28 UTC (Wed) by pbonzini (subscriber, #60935) [Link] Conversion of Linux to 100% Rust is not going to happen. It is simply impossible, also given the hard to test nature of hardware-dependent kernel code. On the other hand, requiring it for most distro builds is definitely going to happen. My bet is 2 years before distro builds use it and 3-5 years before all of us run Rust code. But it may be earlier too. [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 19:39 UTC (Wed) by pizza (subscriber, #46) [Link] > Conversion of Linux to 100% Rust is not going to happen. It is simply impossible, also given the hard to test nature of hardware-dependent kernel code. FWIW I generally agree, but that doesn't mean that C will remain the "primary/core" language. Instead it will eventually be relegated to the long tail of "legacy" device drivers. > On the other hand, requiring it for most distro builds is definitely going to happen. My bet is 2 years before distro builds use it and 3-5 years before all of us run Rust code. But it may be earlier too. I'll be surprised if it takes even a year, probably starting with Fedora wanting to run out of the box on modern Apple hardware. (That's going to be a fun change proposal discussion!) Meanwhile the next LTS/EL cycle is all but guaranteed to require Rust. As I said, "Rust is optional" makes no logical sense, giving you the worst of both worlds. Either commit to it as a core feature, or drop it entirely. [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 19:48 UTC (Wed) by Conan_Kudo (subscriber, # 103240) [Link] Fedora Asahi kernel maintainer here. I will be the first to say that given how Rust-for-Linux is governed and developing, I don't want to turn Rust on in the Fedora kernel yet. Having the kernel build break every six weeks is basically madness, and the lack of priority to eliminate usage of nightly/unstable Rust features and the difficulty getting them to care about tracking newer Rust compiler versions has soured me significantly on the effort. With the lack of a proper specification; lack of stabilized interfaces; lack of stability for even compiling the code; and the lack of interest from kernel developers to review the Rust abstraction patches needed to bring in Rust-based drivers, I do not think enabling Rust in a distribution kernel would be reasonable to consider for a few years. [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 20:13 UTC (Wed) by pizza (subscriber, #46) [Link] > Having the kernel build break every six weeks is basically madness, and the lack of priority to eliminate usage of nightly/unstable Rust features and the difficulty getting them to care about tracking newer Rust compiler versions has soured me significantly on the effort. [...] the lack of a proper specification; lack of stabilized interfaces; lack of stability for even compiling the code; and the lack of interest from kernel developers to review the Rust abstraction patches [...] This reminds me of the ultimately-unsuccessful Van Jacobson Network Channels effort, only with many more moving parts. ....Much like the VJNC, RoL has to commit to building out enough of the new approach to determine if it will be viable in the long run. But unlike VJNC, this adds a major cultural clash into the mix, and that might be what ultimately dooms this whole experiment.. (Thanks for your reply, btw) [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 0:00 UTC (Thu) by ojeda (subscriber, #143370) [ Link] > Having the kernel build break every six weeks is basically madness What do you mean by "break"? > the lack of priority to eliminate usage of nightly/unstable Rust features This is false. It is, in fact, a priority for us: https:// rust-for-linux.com/unstable-features. > the difficulty getting them to care about tracking newer Rust compiler versions has soured me significantly on the effort. This is also false. We are, in fact, tracking the latest version, as anybody can see in the kernel Git log. See also https:// rust-for-linux.com/rust-version-policy. > lack of a proper specification I am not sure what you mean by this (and other points you make in this second paragraph), but if you are talking about a Rust language specification, then that is not really on us. In any case, the Rust project has decided to start working on an official specification for the Rust language. Ferrocene has one too. > the lack of interest from kernel developers to review the Rust abstraction patches needed to bring in Rust-based drivers Different subsystems have different timelines, requirements, bandwidth and so on. You cannot expect every kernel maintainer to be on board or have time to work on Rust this early on. Having said that, as I mentioned in the summit, during this last year we have seen an increased interest from kernel maintainers, and in fact some of them have already been very helpful and supportive. > I do not think enabling Rust in a distribution kernel would be reasonable to consider for a few years. A few major distributions told me it should be possible for them to support Rust for Linux in the future. Ubuntu, for instance, provides packages with the Rust toolchain needed for the kernel. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 0:25 UTC (Thu) by Conan_Kudo (subscriber, # 103240) [Link] >> Having the kernel build break every six weeks is basically madness > > What do you mean by "break"? It fails to compile because something has broken in rustc in some way, or the unstable feature has changed and broken something, or both. It happens without fail. >> the lack of priority to eliminate usage of nightly/unstable Rust features > > This is false. It is, in fact, a priority for us: https:// rust-for-linux.com/unstable-features. You still allow *new* unstable features to be used. That list will keep growing. And in fact, I know it will once the DRM stuff is incorporated into mainline RFL (whenever that happens). It's only a priority once you start saying "no" to adding more of them. That can't happen for a while yet since the various subsystems don't yet have Rust abstractions in mainline. >> lack of a proper specification > > I am not sure what you mean by this (and other points you make in this second paragraph), but if you are talking about a Rust language specification, then that is not really on us. In any case, the Rust project has decided to start working on an official specification for the Rust language. Ferrocene has one too. I mean that you do not have a specification of how you want to handle Rust *in* Linux itself. This makes it really hard for people to figure out how to grade it and grok it when patches are being submitted to build new abstractions or kernel modules. >> the lack of interest from kernel developers to review the Rust abstraction patches needed to bring in Rust-based drivers > > Different subsystems have different timelines, requirements, bandwidth and so on. You cannot expect every kernel maintainer to be on board or have time to work on Rust this early on. Having said that, as I mentioned in the summit, during this last year we have seen an increased interest from kernel maintainers, and in fact some of them have already been very helpful and supportive. No, that's true. But I *can* expect the ones that *are* already doing Rust in other places to do more than nothing in the kernel. *My* patch load keeps growing because *nothing* has made it upstream in a year. It's extremely frustrating. >> I do not think enabling Rust in a distribution kernel would be reasonable to consider for a few years. > > A few major distributions told me it should be possible for them to support Rust for Linux in the future. Ubuntu, for instance, provides packages with the Rust toolchain needed for the kernel. I am shocked. I was also shocked when you said you reached out to all the major distributions, because I have not seen you reach out to Fedora *at all*. Insofar as Ubuntu shipping it, it's pretty easy when there's nothing to ship. The Fedora Asahi Remix kernel has one of the most significant Rust-based kernel modules (the Asahi DRM driver), and it can't even build on Rust 1.73 because something is broken in rustc itself (though nobody cares because it's broken somewhere related to a nightly/unstable feature). My next rebase will be the next time I try with Rust 1.74, and I'm dreading it. I have recommended to upstream Fedora to *not* enable it because it's so risky to enable right now. Since we track upstream Rust on their compilers, Linux Rust code is busted all the time. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 14:01 UTC (Thu) by wtarreau (subscriber, #51152) [Link] > I'll be surprised if it takes even a year, probably starting with Fedora wanting to run out of the box on modern Apple hardware. (That's going to be a fun change proposal discussion!) Meanwhile the next LTS/EL cycle is all but guaranteed to require Rust. > As I said, "Rust is optional" makes no logical sense, giving you the worst of both worlds. Either commit to it as a core feature, or drop it entirely. In a world where specs and compilers are distinct and evolve at their own rhythm, it would be true. But with a fast moving target which evolves to address the limitations making its use hard to impossible in the kernel for various purposes, and forcing periodic major upgrades, it will quickly become a huge mess. Gcc that didn't evolve that fast has already been quite a mess. And it did change only every few years. I was part of those who had to work on making late kernel 2.4 build with gcc-3.4 because that's what users had on their distros... In the end we only addressed two archs and the most visible stuff IIRC. I can't imagine what it will look like in just one year with the current pace. Users will be forced to perform major updates from 6.6 to 7.1 just because "ah that old driver you're using, no, we're not maintaining it anymore, it required and old version of Rust that was not convenient for us, why don't you upgrade your LTS instead?". So that would basically mean no more long term maintenance and even more breakage in field. Sure, for plenty of use cases, nobody cares, machines are short-lived, deployed by automated scripts, and totally untuned. But for other use cases where it takes time to validate a version and adapt to it, it's another story. Another thing is to observe how long rarely used drivers will be maintained. With a language that constantly evolves you can have change of behaviors or guarantees over time, possibly with nobody touching the driver for a while to adapt to the new version as long as it builds fine. Testing driver for hardware is very hard so that doesn't happen often. And possibly that some drivers will not build anymore after a few toolchain upgrades, and lacking a maintainer they will simply be dropped from the tree. Time will tell, anyway, that train has already started. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 18:29 UTC (Thu) by mathstuf (subscriber, #69389) [Link] > Users will be forced to perform major updates from 6.6 to 7.1 just because "ah that old driver you're using, no, we're not maintaining it anymore, it required and old version of Rust that was not convenient for us, why don't you upgrade your LTS instead?". Who is offering LTS kernels but not supporting the drivers they ship with it? IIUC, the kernel community isn't going to say "hey, you want to maintain this driver but we don't want you to, so we're ripping it out" is it? > But for other use cases where it takes time to validate a version and adapt to it, it's another story. Again, why would maintenance on the stable branch of the driver not be a suitable solution here? Presumably with the attitude the developers have in your scenario, they're not backporting anything anyways and not worried about conflicts. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 13:43 UTC (Thu) by wtarreau (subscriber, #51152) [Link] Well, the thing is that it will take more than 20 years and every 10 years new languages appear with their new fans who want to use them for everything and fix all of the world's remaining problems. Thus the migration from Rust to FooBar will start before C to Rust has completed and Rust has finally reached a stable state or gets two compilers to agree on what is safe and what is not. And no, they won't do C to FooBar because by then the remaining C will still be C precisely because it's too difficult to turn into languages where the compiler knows better than the developer. So in 10 years we'll have C, Rust and FooBar with even more maintenance fragmentation. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 14:09 UTC (Thu) by farnz (subscriber, #17727) [ Link] The Linux kernel started over 30 years ago in C and assembly. If every 10 years, the kernel starts moving to a new language, but doesn't complete the migration before moving onto the next shiny, what are the 2 to 3 languages that it's using beyond C and assembly, and isn't yet finished migrating to? [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 16:18 UTC (Thu) by wtarreau (subscriber, #51152) [Link] Till now nobody claimed to replace the one in place. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 16:25 UTC (Thu) by farnz (subscriber, #17727) [ Link] There are several languages I've worked with that have claimed to be C replacements in the 30 years of the kernel using C. None of them got adopted, even with a group of people pushing for people to replace C with their language of choice. What makes you think that, assuming the kernel does adopt Rust, the next language to come along will be adopted, when C++, D, and others were not? [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 15:41 UTC (Thu) by beagnach (guest, #32987) [Link ] > every 10 years new languages appear with their new fans who want to use them for everything and fix all of the world's remaining problems. In the 30+ years of Linux kernel development this is the first time a new language has been introduced, so I don't think your point stands. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 20:26 UTC (Thu) by sfeam (subscriber, #2841) [ Link] Well, there was BPF... [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 14:41 UTC (Thu) by linusw (subscriber, #40300) [ Link] I would rather say the approach is BOTH/AND not EITHER/OR. Everybody will be required to be bilingual and it is not just going one way; e.g in the event that a reviewer request a feature in a newly submitted "leaf driver" (for something like phylib) to rather be moved to the core, then that means that the developer submitting that Rust code has to go and reimplement that feature in the core using C. People grossly overestimate how hard Rust is, it's a trivial language to learn. All it does is deliver what Algol 68 and PASCAL delivered, but C omitted when it comes to spatial memory safety, then some temporal safety is added by nudging immutable variables and pure functions where possible inspired by misc functional languages. The rest is syntax. Gordon was right saying the language contains "nothing new" really. In the device tree bindings that almost all driver maintaines have to deal with, we have to use YAML which is a *WAY* harder language to learn than Rust. Yet we just learn it, so I don't see the problem here. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 19:39 UTC (Thu) by cozzyd (subscriber, #110972) [ Link] I don't think many people will agree that YAML is a more difficult language to learn than rust. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 19:48 UTC (Thu) by pbonzini (subscriber, #60935) [Link] Rust is trivial to learn and insanely hard to master, let's not kid ourselves. Most maintainers in the kernel could not write (and could barely read) the code for some of the bindings. See for example the workqueue bindings, https://lwn.net/Articles/932271/ (might not be the last version, the point still stands). The way it works is to trust that the people who contributed them, or someone else, will be around to help. Fortunately the Linux APIs are relatively stable, and changes are less frequent than people think, but I think it's reasonable at least for now to have even experienced C programmers ask for help, because introducing technical debt would be worse. [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 19:05 UTC (Wed) by josh (subscriber, #17465) [ Link] > the language still isn't entirely stable Clarification: Rust provides a stable version, but the Linux kernel uses nightly Rust because it wants features that aren't stable yet. This will get fixed once all the remaining nightly features used in the kernel are either stabilized or replaced with some other mechanism. [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 21:06 UTC (Wed) by Conan_Kudo (subscriber, # 103240) [Link] That assumes that they won't just consume more Rust nightly stuff. I don't have confidence in that. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 1:54 UTC (Thu) by josh (subscriber, #17465) [Link ] I'm fairly confident in that, given that they've specifically said they want to get to stable Rust. [Reply to this comment] Committing to Rust for kernel code Posted Nov 22, 2023 23:19 UTC (Wed) by ojeda (subscriber, #143370) [ Link] > GCC support will be a while To add some context on the two GCC approaches: * rustc_codegen_gcc is already able to compile the kernel without source code changes (modulo unrelated alloc changes). Though still experimental, it looks like it is the first approach we will be able to use to build through GCC. See https:// rust-for-linux.com/rustc_codegen_gcc for the latest status and kernel-specific instructions from Antoni Boucher (the rustc_codegen_gcc lead developer) who kindly maintains that page in our website. * GCC Rust (gccrs) is working on upstreaming all their work so far, to be released in GCC 14. They have done a lot of work, but upstreaming takes a long time and it is still a long way from compiling the kernel. However, building Rust for Linux is one of their explicit objectives. See this very recent talk from Arthur Cohen (co-lead developer of GCC Rust) or the slides in our Rust for Linux talk at LPC that Arthur generously provided. Both approaches should allow the Rust side of the kernel to access GCC-only architectures. However, due to the nature of each approach, GCC Rust will take longer to be useful for the kernel purposes, which is why I think more resources would help them significantly. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 15:02 UTC (Thu) by rahulsundaram (subscriber, # 21946) [Link] > GCC Rust will take longer to be useful for the kernel purposes, which is why I think more resources would help them significantly. What if any do you see as an advantage to the second approach? [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 17:09 UTC (Thu) by atnot (subscriber, #124910) [ Link] So far I've seen a few: - A second implementation to validate against. With their ambition to be bug-compatible and to share the borrow checker with rustc (probably the least rigorously specified part of rust) that does feel a bit doubtful. - Having a fully copyleft implementation. See previous for limitations. This doesn't seem to be an issue for languages like Python, but some people do seem to care. - Another path for bootstrapping. mrustc already covers this mostly, but it's an option. - Possibly better cross-language builds. I haven't really seen any examples of this from other gcc frontends, but it's plausible. - Force of habit of compiling native code with gcc. - Desire to keep GCC relevant over LLVM, especially as a flagship for GNU. - Vague ambitions to dethrone the rustc developers because you don't like them or they won't let you say racial slurs - All of the people who reflexively claimed they didn't want to use rust because it doesn't have a GCC frontend will have to find a new excuse. Perhaps they'll suddenly become really big fans of ISO. Although all of these are mostly just reasons for it existing somewhere in the aether, not for anyone actually using it as their primary rust compiler. I don't think any of these reasons are incredibly convincing on their own, but between them they do fund a few positions. Is that enough? I don't know. But to be a bit cynically, does it really matter when it seems to be most important as a symbolic gesture for everyone involved anyway? [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 17:30 UTC (Thu) by atnot (subscriber, #124910) [ Link] Oh, to be clear, this is not me saying they shouldn't do it. I don't think it hurts anyone and it does seem like it could come in handy sometimes. I'm just not convinced it's something that will change the trajectory of rust in any sense or that there's actually an alleged army of fence-sitters out there who would just *love* to use rust if it weren't for rustc. [Reply to this comment] Committing to Rust for kernel code Posted Nov 23, 2023 15:54 UTC (Thu) by atnot (subscriber, #124910) [ Link] > Williams pointed out that the previous session had discussed how hard it is to get kernel subsystems to move to new APIs; now, he said, there is talk of moving to a whole new language I think it's worth considering why those moves are so hard instead of just taking it as a given in such a context. It's generally not that there's a lack of people with the ability to do such rote transformations. Often they can even be done automatically. The part that's difficult is building the confidence that those changes are correct, which usually requires intensive auditing of the surrounding code, manual testing and extensive domain knowledge. Those resources are scarce. There's a common definition of Legacy Code as "Code that is too scary to change and too valuable to delete". Arguably this applies to a lot of the kernel today. Short of stopping development or going on a deletion spree, there is really no alternative to finding ways of getting more confidence in changes. That is going to involve a bit of churn, no matter the method you choose. But that investment really does pay off quickly. [Reply to this comment] Copyright (c) 2023, Eklektix, Inc. Comments and public postings are copyrighted by their creators. Linux is a registered trademark of Linus Torvalds