https://lwn.net/SubscriberLink/889924/2b330ed9ea4a9e23/ LWN.net Logo LWN .net News from the source LWN * Content + Weekly Edition + Archives + Search + Kernel + Security + Distributions + Events calendar + Unread comments + ------------------------------------------------------------- + LWN FAQ + Write for us User: [ ] Password: [ ] [Log in] | [Subscribe] | [Register] Subscribe / Log in / New account Rustaceans at the border [LWN subscriber-only content] Welcome to LWN.net The following subscription-only content has been made available to you by an LWN subscriber. Thousands of subscribers depend on LWN for the best news from the Linux and free software communities. If you enjoy this article, please consider subscribing to LWN. Thank you for visiting LWN.net! By Jonathan Corbet April 14, 2022 Support for developing in the Rust language is headed toward the kernel, though just when it will land in the mainline is yet to be determined. The Rust patches are progressing, though, and beginning to attract attention from beyond the kernel community. When two languages -- and two different development communities -- come together, the result can be a sort of cultural clash. Some early signs of that are appearing with regard to Rust in the kernel; if the resulting impedance mismatches can be worked out, the result could be a better development environment for everybody involved. The latest round of Rust patches was posted by Miguel Ojeda on March 17. This time around, Rust support has moved forward to version 1.59.0 of the Rust language, which has stabilized a couple of important (for the kernel) features. The patches add a new module abstracting access to hardware random-number generators. A CString type has been added for C strings. The spinlock implementation has been improved. All told, the patch series, which can be found in the linux-next repository, adds over 35,000 lines of code and documentation; it is a considerable body of work. There has been no public discussion on just when these patches might be deemed ready to go into the mainline kernel. Rust support is still considered "experimental" even by its developers; that is likely to remain the case for some time (even after this work is merged into the mainline) until the language proves itself for kernel development. Clearly, though, some developers are beginning to play with it -- and they are not all traditional kernel developers. Recently, Nandor Kracser asked on the Rust-for-Linux mailing list about the possibility of including Rust modules from the crates.io repository into the kernel build. This request, seemingly, is not just for small stuff: Currently I'm experimenting with different crates which I would like to use in my module, serialization libraries, math libraries. etc, even complex ones, are really hard to pull in as a direct source library (copy the code to the module), and if they have a transitive dependency that complicates things even more. Shortly thereafter, Chris Suter showed up with a similar request. Rust developers working with kernel modules, it seems, want more functionality than the current kernel crate provides to them. This should not be entirely surprising. Like many newer languages, Rust is closely tied to a language-specific package-management system and associated central repository; in this case, the Cargo package manager and crates.io. Developers in such languages quickly become accustomed to pulling in new modules (and any dependencies they may have) with a simple command, and to having the build system make dependencies magically appear when building a new program. For these developers, the idea of working in an environment where complex libraries are not obtainable with a few keystrokes starts to have a distinct lack of appeal after a while. The kernel does not work in this way, though. To those of us who didn't grow up with that kind of development environment, it looks like a recipe for bloat, bugs, and security problems. Depending on central repositories opens up a project to problems like the famous leftpad incident or, worse, the deliberate insertion of malicious software. A lack of attention to API compatibility leads to a thicket of version requirements and dependency-resolution problems so complex that machine-learning systems are emerging to deal with them. Plus it all just looks so undisciplined and messy. At least some of the criticisms of this mode of development are valid, but it's also not hard to detect a bit of Stockholm syndrome as well. For many of us, for much of our careers, building a new program from source was likely to involve a lengthy cycle of "try to build, figure out which dependency it wants now, install the dependency" iterations -- and recursive iterations at that when the dependencies turn out to have missing dependencies of their own. This exercise helped us to understand our systems better and must somehow have helped us to build better moral character, so we can't understand why Kids These Days just don't want to live that way. The kernel community seems more than usually likely to have developers who are resistant to newer methods of development. The kernel has to stand alone, and its developers keep a firm grip on its dependencies. The kernel repository contains all of the code needed to build a working kernel; developers can be expected to install a limited set of tools to do the build, but the idea of installing external libraries to build into the kernel would not go over well. So when developers see a shopping list like the one posted by Suter: Like I said, I'm interested in futures. Why it's useful: async Rust is arguably more common and easier to use than other forms of multi-threaded processing. Other crates that I'd like: anyhow, bincode, byteorder, log, once_cell, pin-project, rand, serde, slab, static_assertions, uuid plus some more esoteric ones. The first temptation will be to either run and hide or to respond in a way that may not be compliant with anybody's code of conduct. There are some good reasons for this. As Greg Kroah-Hartman pointed out, code that has been written to be useful in user space almost certainly does not work within the constraints imposed on kernel code. "Async Rust" knows nothing about kernel threads or how context switching is done in the kernel, for example. Kernel code must be extremely careful in how it allocates memory, must not use floating-point arithmetic, cannot store large data structures on the stack, and cannot use unbounded recursion, among many other rules. Most user-space code, which was not written with these rules in mind, will fare poorly in this environment. For this reason, Kroah-Hartman said that any functionality desired by Rust programs must be specially written and provided in the dedicated kernel crate. The Rust-for-Linux developers understand this situation and are not envisioning adding the ability to pull in modules with a tool like Cargo. So it is interesting that a long-time kernel developer, Kent Overstreet, was the one to argue for a different approach. "The world is changing", he said, and perhaps it is time for the kernel community to change with it as well. There are numerous situations where it can be beneficial to run code in both user and kernel space, he said, and the fact that doing so is currently painful is a problem for developers on both sides: The solution to problems like these are to stop thinking that kernelspace and userspace _have_ to be completely different beasts - they really don't have to be! and start encouraging different thinking and tooling improvements that will make our lives easier. It is true that the boundary between kernel and user space has become more porous over the years. Various subsystems provide hooks that allow formerly kernel-specific tasks to be carried out in user space instead, while user space can use BPF to run code inside the kernel. But the two environments are still quite different, and code meant to run on one side generally cannot run on the other. There has not been a lot of effort put into thinking about how to reduce that divide; perhaps it really is time for that to change. The Rust language might just be the environment in which this transformation could happen. As Overstreet put it: Rust's conditional compilation & generics story is _much_ better than what we've had in the past in C and C++, meaning writing Rust code that works in both userspace and kernelspace is much saner than in the past. If an initiative like this were to work, it could greatly reduce the barrier to entry for future kernel developers while making a lot of useful code available to the kernel community. It would be a different kernel project than the one we know now, but it might be a more fun and more productive one. Interesting things tend to happen when immigrants show up in a new land. They can often create a backlash among those who are already there -- the new people dress funny and their cooking smells weird, after all, and some of them even have a crab as their mascot. But they can also bring energy and ideas that shake up their new home and make it richer for everybody involved. It may just be that we will see something like that happen if and when a crowd of Rust developers descends upon the kernel community. The end result could be difficult to recognize -- and perhaps better than anything we had before. Index entries for this article Kernel Development tools/Rust [Send a free link] ----------------------------------------- (Log in to post comments) Rustaceans at the border Posted Apr 14, 2022 19:24 UTC (Thu) by rvolgers (subscriber, #63218) [Link] > "Async Rust" knows nothing about kernel threads or how context switching is done in the kernel, for example. Kernel code must be extremely careful in how it allocates memory, must not use floating-point arithmetic, cannot store large data structures on the stack, and cannot use unbounded recursion, The Rust response to this would probably be "can't we just... solve that?" Various embedded Rust folks have been doing work on things like static stack usage analysis. Tight stack requirements are not unique to the Linux kernel by any means, it is clearly desirable to have insight and control over that. And as for memory allocation, again, people write Rust code which has no support for memory allocation at all, using libraries which are on crates.io. And they're not always super custom libraries, sometimes it's literally just a matter of enabling the "no_std" feature flag, and you can use a bog-standard Rust library (with a somewhat more limited API, of course). More complex things like interrupt context seem more likely to remain "here be dragons" territory (just like signal handlers are, to some degree, in userspace). But Rust is hardly unique in that; not all kernel C functionality is safe to use there either. TLDR: Some of the arguments being used smack of prejudice against Rust ecosystem libraries as being by definition userland-focused. Many are, but many are not, and some adapt well to either use case. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 19:47 UTC (Thu) by atnot (subscriber, #124910) [ Link] > sometimes it's literally just a matter of enabling the "no_std" feature flag, and you can use a bog-standard Rust library To add to that, some of the requested libraries like pin_project and serde to my knowledge do not even really exist at runtime at all. There are a lot of utility crates like that which contain a bunch of fancy macro and type definitions, at most a few lines of which will actually end up in the final binary. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 22:13 UTC (Thu) by farnz (subscriber, #17727) [ Link] On top of that, this is a case where a perfect solution isn't needed; if automation can permit 70% of the crates that would work well in kernel space into the kernel, and provide a way for a human to audit and certify the remaining 30% of those that would work as accepted for kernel use, that's a good outcome. Which, in turn, means that a conservative approximation is good enough - rejecting 90% of crates as "not kernel compatible for $reason", accepting 7% as "kernel compatible", and leaving 3% as "not sure - needs a human involved" would work just fine as a solution. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 22:33 UTC (Thu) by ssokolow (guest, #94568) [Link ] sometimes it's literally just a matter of enabling the "no_std" feature flag, and you can use a bog-standard Rust library (with a somewhat more limited API, of course). Rust feature flags are additive, so the convention is to declare an on-by-default feature flag named std which toggles the no_std attribute. (no_std being equivalent to GCC's -nostdlib.) [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 0:05 UTC (Fri) by tialaramex (subscriber, #21167) [Link] Note that -nostdlib is getting rid of almost everything, whereas Rust's no_std only gets rid of stuff from std itself (much of what you think of as Rust's standard library was only re-exported from std and actually lives in core or alloc). So e.g. suppose you've got a slice (maybe an array, but however you got it, some contiguous memory) of Things and you'd like to sort them. In C without the standard library you're out of luck, code it yourself, but in Rust lacking std only means you don't have a nice stable allocating merge sort, you do still get a perfectly usable (albeit not always trivially what you needed) in-place unstable sort. https://rust-for-linux.github.io/docs/core/primitive.slice.html# method.sort_unstable When C was invented such things would be too heavy, but today Rust's compiler and linker are certainly smart enough that if you never actually perform sort_unstable the code to implement it is omitted from your binary so the "price" of Rust's more comprehensive library is only that the documentation is a little larger and for that price you avoid the unsettlingly common (even in Linux) discovery that six people have re-implemented some useful idea, meaning the kernel carries not one but six copies of the code, and worse at least one of them is actually faulty. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 19:24 UTC (Thu) by atnot (subscriber, #124910) [ Link] I think Kent Overstreet hinted at another good point in the message: > We already have large swaths of code being imported from userspace into the kernel as-is - e.g. zstd [...] Presumably, this is in reference to this event last November: > Hi Linus, > > I am sending you a pull request to add myself as the maintainer of zstd and > update the zstd version in the kernel, which is now 4 years out of date, > to the latest zstd release. [...] (https://lore.kernel.org/all/20211109013058.22224-1-nickrt...) It sounds like it would already be pretty desirable in general to create a more formal, more automated version of the process of making vetted external dependencies available to the kernel, not just for Rust, but also for C. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 19:39 UTC (Thu) by Gaelan (subscriber, #145108) [ Link] Most user-space code, which was not written with these rules in mind, will fare poorly in this environment. It's worth nothing that user-space Rust will fare better than user-space C in this regard. Rust's standard library is split into three pieces: core, alloc, and std. core runs pretty much anywhere; alloc contains the interfaces that require malloc() (or something similar); and std contains the interfaces that require an operating system. Kernelspace Rust obviously can't use std (and uses a custom fork of alloc), but there's a large body of "no_std" Rust code on crates.io which should uses only core and should, in principle, run fine in kernelspace. This already happens in practice: no_std crates are extensively used by people writing Rust for devices like microcontrollers, which (due to their small memory) are also likely to be sensitive to concerns about stack space, recursion, etc. While there will obviously be crates that would never be useful in kernel space, and it's certainly wise to be suspicious of introducing dependencies on third-part package maintainers, there's still a lot of code out there which could be very useful in the kernel. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 19:59 UTC (Thu) by mb (subscriber, #50428) [Link] I think the question is more like: Should we depend our kernel build on external sources such as crates.io? Yes, we should allow no_std crates to be used from kernel code. (after careful review for things like floating point, etc..) But how should we integrate the crate? Should we only declare the version in the kernel's Cargo.toml and fetch the latest compatible SemVer at kernel build time? That would result in kernels with the same version having different content. Should we pin a specific version via Cargo.lock or similar? Should we copy the crate into the kernel tree to be independent of crates.io? [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 20:29 UTC (Thu) by djc (subscriber, #56880) [Link ] Keeping the Cargo.lock file in the Git repository is pretty common for Rust applications, we do that at work. I don't see the point in Rust for Linux forbidding the use of crates.io crates. Yes, you'd probably want to enforce that only no_std crates are pulled in (I don't know if that's currently supported), but otherwise it seems like a great way of sharing functionality with other parts of the ecosystem. I can see how pulling in an async runtime would be a bigger deal, but at the same time being able to write async code instead of callback hell seems like it might be important for making code easier to understand the same way it does in userspace. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 22:48 UTC (Thu) by ssokolow (guest, #94568) [Link ] Keeping the Cargo.lock file in the Git repository is pretty common for Rust applications, we do that at work. I think it'd be more likely they'd want to use cargo vendor (which automatically vendors the specified dependencies) so the kernel repository can use third-party crates but rely on crates.io only as a means of standardizing/automating the rote steps in their existing workflow for manually OKing updates to vendored dependencies. I can see how pulling in an async runtime would be a bigger deal, but at the same time being able to write async code instead of callback hell seems like it might be important for making code easier to understand the same way it does in userspace. They'd almost certainly want to write their own executor which is just an API adapter for the relevant kernel machinery. That's the whole point of Rust's async system being designed as it is. To avoid tying everyone to a single executor. (Yeah, most people use tokio because the APIs provided beyond the executor itself are still working toward standardization, but those are generally std things like userspace I/O anyway, so that's not relevant to kernel or embedded use-cases.) [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 22:42 UTC (Thu) by josh (subscriber, #17465) [ Link] I absolutely *don't* think we should pull in third-party crates over the network. I do think we should have a means of "vendoring" unmodified upstream crates that work in the kernel, without having to rewrite them. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 0:32 UTC (Fri) by IanKelling (subscriber, #89418) [Link] > I absolutely *don't* think we should pull in third-party crates over the network. I strongly agree. For multiple reasons, including the fact that when I go to https:// crates.io I get a blank page which says I should enable javascript. Please don't make kernel development require remote code execution on my machine. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 0:55 UTC (Fri) by mjg59 (subscriber, #23239) [ Link] I just want to be clear on this - you'll download kernel source code from the internet and type "make", resulting in a great deal of code running on your system, but you're not ok with javascript running inside a strongly sandboxed environment? [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 8:39 UTC (Fri) by tux3 (subscriber, #101245) [ Link] Presumably a possible defense for the more general version of this position is that the review process is very different, between web pages in general and the kernel. While I personally run most JavaSscript without a second though, some websites that are not crates.io sometimes run undesirable JS - cryptominers, supercookies, what have you. Sometimes there is third-party JS code that has clever ways of escaping the sandboxes. I am told Firefox's content process lost connection to the X11 server just a month ago. Although the sandbox is improving as well, hardness may sometimes be a relative thing. In contrast, I will trust Debian to update many millions of lines of code straight from the internet into a running system, much of it which could enter my $HOME. If the kernel were to pull in more external libraries, like it did with zstd, there's more than a couple git remotes that I'd feel I could pull from without being gifted any crypto miner or other code that might try to poke at a sandbox. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 8:49 UTC (Fri) by mjg59 (subscriber, #23239) [ Link] I have no reason to believe that kernel code gets significantly more review than crates.io does - I agree that the web as a whole doesn't meet that bar, but frankly most software I pull from Debian has also had much less review than the kernel does (https://lists.debian.org/ debian-devel/2003/02/msg00771.html is an example of this not going well) so it feels like what's missing is a way to express what level of trust I place in any provider of code I end up executing rather than just to assert that websites that use Javascript are unacceptable . [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 9:08 UTC (Fri) by tux3 (subscriber, #101245) [ Link] I agree with that. As for crates.io, I have respect for the work they do and I'm happy to run their code (though I have not read it). This may be getting off-topic, but now I'm curious if you have anything specific in mind when you write about expressing levels of trust -- would that look like further sandboxing? [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 9:27 UTC (Fri) by mjg59 (subscriber, #23239) [ Link] Great question! I spent a while looking into whether it was feasible to apply different LSM profiles (SELinux/Apparmor/whatever) to dpkg depending on where the package was downloaded from, and unfortunately the architecture doesn't make that terribly possible. From the web perspective, I think that probably comes down to extension-level handling at the moment? In an ideal universe we'd have infrastructure to tie any given piece of javascript back to an upstream repo and make a trust decision based on things like licensing and review assurances, but that feels like kind of a lot of work. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 13:51 UTC (Fri) by IanKelling (subscriber, # 89418) [Link] Hah good point. I do generally try to also run make within a sandbox too though. Websites should not requiring people to download and run a program to simply read a page of text. Crates.io is almost entirely used for text that could be simple html. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 6:41 UTC (Fri) by qyliss (subscriber, #131684) [ Link] It's not a solution to any of the broader points being discussed, but I recommend https://lib.rs/, an alternative frontend to crates.io, which does not require JavaScript (and also has a much nicer interface). [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 8:23 UTC (Fri) by marcH (subscriber, #57642) [ Link] > "vendoring" unmodified upstream Problem: everyone knows "copy/paste/diverge" sounds bad. Solution: keep inventing new synonyms to disguise it. Like when $BIGCORP changes name after some scandal. The real solution is of course some form of branching / forking == copy/paste under (version) control. Probably what the author of a git tool written in rust had in mind :-) [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 16:14 UTC (Fri) by pj (subscriber, #4506) [Link] I think it's pretty common these days to 'vendor' code via git submodules, which seems to be what you're advocating for. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 17:35 UTC (Fri) by smurf (subscriber, #17840) [ Link] "git submodule" doesn't vendor anything. The code is still pulled from a remote repository, it's just pinned to a specific version. Which is exactly what you want to avoid spurious external changes that introduce more-or-less-subtle security problems. "Real" vendoring, aka copy/paste/ignore, disconnects the copy entirely from its source. (Thanks but no thanks.) [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 17:12 UTC (Fri) by mathstuf (subscriber, #69389) [Link] What we do is we have a separate git history for the import that is subset from upstream (remove test suites, test data, docs, sources we don't care about, etc.) and commit that on the respective tracking branch. One can even transform the import to do things like sqlite's amalgamate to make a smaller set of sources. This then gets `-Xsubtree` merged into the main tree and is checked that: - changes to this directory only come from such a merge - the merge came from the "right" branch (tracked by its root commit) - the merge did not modify the subtree in the merge commit (yay) We also check that *all* such changes come in through this mechanism and we track our patches in a fork of upstream that we tag for each import for posterity. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 15:01 UTC (Fri) by cesarb (subscriber, #6266) [ Link] > Should we copy the crate into the kernel tree to be independent of crates.io? As mentioned in a comment above, there's precedent with things like zstd, which is also fully copied into the kernel tree instead of being kept as an external package. So I'd expect every crate included in the kernel, with the exception of the ones which come with the compiler (i.e. the "core" crate), to be vendored directly into the kernel tree. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 20:09 UTC (Thu) by mb (subscriber, #50428) [Link] One (IMO) major problem with letting each kernel module choose to fetch random crates from crates.io is that it will result in multiple incompatible versions of the same crate being built into the kernel. E.g. the module wants foo-1.0 and a dependency of the module wants foo-2.0. And another module requires foo-3.0. Therefore, three versions are included. The kernel as a whole must agree on which crate versions to use. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 22:23 UTC (Thu) by nybble41 (subscriber, #55106) [Link] To an extent you'll see that happen anyway when Rust-based modules are compiled separately from the kernel. Rust code compiled for the kernel will be specialized based on how it's used in the kernel, and the same Rust code compiled for a loadable module will be specialized for the module. In Rust that can even affect things like structure layout, unless the structure has been declared #[repr(C)]. In general this isn't a problem so long as you don't try to cross kernel/module boundaries through the (unstable) Rust ABI. The module should expose C entry points and only share #[repr(C)] data with outside code--even other code written in Rust. Even if a module happens to pull in a different version of some dependency than the kernel, the dependency will be constrained to that module and will not affect the rest of the system. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 19:36 UTC (Fri) by bartoc (subscriber, #124262) [ Link] You actually don't have to agree on what version to use, since the build system makes sure that you see the definitions you were expecting and everything is mangled such that you can't screw it up. It is not good to have multiple versions of stuff for other reasons, like code auditing, and code size, and compile time [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 21:49 UTC (Thu) by tialaramex (subscriber, # 21167) [Link] I don't foresee people whose mascot is a penguin having too much trouble welcoming people whose mascot is a crab. As a C programmer I will say that Rust's standard library (not all of which, admittedly, the kernel gets, see other comments about the stack of core -> alloc -> std of which the kernel will receive core and a somewhat custom alloc) has much more than I was used to from the C standard library and in many cases also utility libraries like Glib. Programs where in C I'd have reached for some third party utility libraries at least, often in Rust the standard library is more than enough. Yet it's still a very hospitable place to do the sort of low-level programming Linux is all about. For example, Rust explicitly has a char type which reflects Unicode scalar values, but it pragmatically offers predicates like is_ascii_hexdigit() both on the char type and on u8 (the unsigned byte). If you actually have bytes (say from a character device), and the only thing you care about is whether those bytes are ASCII hex digits, with no interest in whether they might be part of an Emoji or a Korean word, or anything fancy like that, then Rust doesn't waste your time. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 22:32 UTC (Thu) by NHO (subscriber, #104320) [ Link] As a Gentoo user, I build kernel sometimes and customize the configuration to my hardware. Sometimes, I make a mistake and end up with no network. If I can't rebuild my system in that situation, because kernel needs network for build, then that? There is a local mirror of Gentoo repo, source included, in Russia. But our internet censorship organization could get some garbage in their heads because some ancient judge from Nowhere, Ural mountains added crates.io to the list of sites to be blocked. Debian prevents network access when building packages, no? What I'm saying is that any solution that breaks kernel build in airgapped sandbox is bad. [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 22:55 UTC (Thu) by ssokolow (guest, #94568) [Link ] I remember them saying they had no plans to use Cargo for the build system but, if they decide to use it for dependency handling, I imagine they'd use cargo vendor to automate what they already do with C libraries they use. (i.e. Keep a copy in the kernel repository and manually pull in updates from upstream every now and then.) This cargo subcommand will vendor all crates.io and git dependencies for a project into the specified directory at . After this command completes the vendor directory specified by will contain all remote sources from dependencies specified. Additional manifests beyond the default one can be specified with the -s option. The cargo vendor command will also print out the configuration necessary to use the vendored sources, which you will need to add to .cargo/config.toml [Reply to this comment] Rustaceans at the border Posted Apr 14, 2022 23:13 UTC (Thu) by gerdesj (subscriber, #5446) [ Link] I've run Gentoo on 10s of systems for well over over a decade. There are plenty of other, far more straightforward, ways to break a Gentoo box! For starters, emerge will pull down the dependencies and build them first and work up the chain to the final result. Unless you go mad with eclean, you'll have all the bits on site. I do understand where you are coming from though. It all sounds a bit ... devops. Before you know it we'll be piping curled configs through configure/make. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 5:19 UTC (Fri) by q_q_p_p (subscriber, #131113) [ Link] If they want kernel in rust, they should write kernel in rust like google does with fuchsia. Rust is such a cancer... attaching to popular projects and destroying them - why don't they do this with llvm, huh ? because they would have to deal with at least dependency hell, proper compiler boostrapping procedure, etc. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 5:45 UTC (Fri) by Cyberax ( supporter , #52523) [Link] > If they want kernel in rust, they should write kernel in rust like google does with fuchsia. Fuchsia is written mainly in C++. > Rust is such a cancer... attaching to popular projects and destroying them - why don't they do this with llvm, huh ? because they would have to deal with at least dependency hell, proper compiler boostrapping procedure, etc. Cargolift. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 5:47 UTC (Fri) by Cyberax ( supporter , #52523) [Link] Sorry, Cranelift: https://github.com/bytecodealliance/wasmtime/tree/ main/cr... [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 6:41 UTC (Fri) by q_q_p_p (subscriber, #131113) [ Link] LLVM: https://github.com/llvm/llvm-project I can use the same argument about the kernel, just use redox (cranelift) if you don't like C: https://gitlab.redox-os.org/redox-os /redox and leave kernel(LLVM) alone ? I was wrong about fuchsia though... [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 6:44 UTC (Fri) by Cyberax ( supporter , #52523) [Link] The decision to use Rust will be done by actual Linux developers. And nobody is forcing you to use the mainline Linux. If you so dislike Rust, then you are totally free to fork Linux and maintain your own Rust-less fork. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 10:04 UTC (Fri) by mpr22 (subscriber, #60784) [ Link] > attaching to popular projects and destroying them This is the kind of claim that needs hard citations to be anything other than a content-free "look at me see how much I hate Rust" signal flag. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 11:30 UTC (Fri) by fazalmajid (guest, #158016) [ Link] Totally agree. The Rust project's cavalier attitude to language and compiler stability, the absurd compiler bootstrapping and limited platform support makes it throughly unsuitable for the Linux kernel., and that's also why I won't touch it despite the many benefits. Not to mention their belief that "curl something}bash" is acceptable practice. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 12:51 UTC (Fri) by mpr22 (subscriber, #60784) [ Link] > The Rust project's cavalier attitude to language and compiler stability I'm curious: What's "cavalier" about their attitude to language and compiler stability at this time? For this question, I'm only interested in claims backed by concrete examples, relating to officially designated stable features of the compiler and language, postdating version 1.0 of the toolchain. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 18:14 UTC (Fri) by nix (subscriber, #2304) [Link] Oh yeah, the system in which things are tested under nightly feature guards, getting stabilized and available to ordinary code only when ready, and only then broken if unsound or after an edition change (and the old versions continue to work under older editions), and the way that feature additions are validated against all of crates.io to make sure they don't break things before going in... it's just so totally cavalier! Nothing like the elegant and refined C world, where, say, the introduction into POSIX of getline() and its consequent promotion into the _POSIX_C_SOURCE glibc feature flag (formerly it was only visible under _GNU_SOURCE) could not possibly break dozens of unrelated packages that just happened to use that as an identifier with no way to fix it other than to change all of them. (That's not to say Rust can't improve. My personal if-wishes-were-unicorns perfect-world wish: if Rust grew the same amazing system Swift has to allow generic types with a stable ABI without heaps of monomorphization code bloat, that would be fabulous. It is an incredibly intricate system though, so I can see why it hasn't happened, at least not yet: it certainly can't be used unchanged but would need some redesign, and I'm not even sure this is possible. I know others have mused about this before, because Rust's lack of a nice way to do the shared library stable-ABI thing is probably its biggest remaining downside compared to other, lesser languages like C. C++, of course, has the exact same flaw, but unlike Rust it's much harder to apply the same cunning Swift tricks to it.) [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 19:05 UTC (Fri) by LoganDark (guest, #158019) [ Link] At least C++ has the benefit of "well USUALLY if you use the same C++ compiler it's compatible with pretty much all C++ libraries compiled by that compiler". Sometimes it's even compatible with other compilers. With Rust it's "the ABI is 100% undefined and changes basically every commit, you can't use generic types at all because they're not present at all in the library", etc. At least Rust has the ability to declare C functions. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 22:06 UTC (Fri) by Cyberax ( supporter , # 52523) [Link] I see you've never used binary-only C++ libs. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 6:47 UTC (Fri) by qyliss (subscriber, #131684) [ Link] I'd expect one of the big problems here to be the build system. Right now, Rust for Linux integrates Rust into the kernel build system, rather than using Cargo. I think the attraction of using lots of third-party crates would quickly fade if developers had to write Makefiles for any dependency they wanted to add (very few Rust libraries build without a build.rs file doing something fancy somewhere in their dependency graph), but Cargo really does not like co-operating with other build systems. This has made life difficult for any sort of generic build or meta-build system to add Rust support -- Meson, gn, Soong, Bazel, Nix, etc. It comes up over and over when big established projects try to adopt Rust, but no progress has been made on it yet as far as I'm aware. I think it would require some very fundamental changes to how Cargo works. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 11:46 UTC (Fri) by LoganDark (guest, #158019) [ Link] If this were my project (and maybe 0.1% the size), I'd migrate the rest of the existing codebase to use Cargo and Rust build scripts, rather than the other way around (trying to get Cargo to fit into makefiles/something else), since other parts of the kernel are probably going to eventually be rewritten in Rust as well. That doesn't exactly sound feasible here, but it's a nice thought. imho, it would be an almost-all-around win for Cargo to become more interoperable with existing build systems, but it could also encourage laziness or tricky setups. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 7:31 UTC (Fri) by tlamp (subscriber, #108540) [ Link] > "Async Rust" knows nothing about kernel threads or how context switching is done in the kernel, for example. Hmm, but it doesn't know that stuff for the user space either. Async is basically sugaring to wrap handling a pinned future type, each runtime, and future type (with a poll funciton, and a pending and ready state), can map that to their actual, underlying execution model of choice. I mean, that's why the Rust ecosystem can have multiple async runtime libaries like tokio, async-std or smol. https://rust-lang.github.io/async-book/02_execution/01_chapter.html [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 8:30 UTC (Fri) by seanyoung (subscriber, #28711) [Link] Jonathan Corbet, I must say I am disappointed with the reporting on rust. Rust is an entirely new language with many interesting properties, memory safety for example and just how expressive the language is. Instead we get an article which almost uniquely focusses on the ability to pull in external crates, which is always going to be controversial for a kernel. A lot of us kernel hackers will need to learn how to use rust in the kernel. How about something a bit more in-depth? [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 8:35 UTC (Fri) by beagnach (subscriber, #32987) [ Link] > A lot of us kernel hackers will need to learn how to use rust in the kernel. How about something a bit more in-depth? Let me google that for you... https://www.google.com/search?q=site%3Alwn.net%20rust&... [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 8:46 UTC (Fri) by seanyoung (subscriber, #28711) [Link] Nothing in depth to see there. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 13:29 UTC (Fri) by pbonzini (subscriber, #60935) [Link] https://lwn.net/Articles/869428/ might have something for you. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 10:39 UTC (Fri) by Karellen (subscriber, #67644) [Link] Um, that's the subject that the kernel mailing list threads he was reporting on were about? As in, that's a subject that kernel developers are currently discussing, which is why it's worth summarising now. What? [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 12:02 UTC (Fri) by kay (subscriber, #1362) [Link] I'm in contrast are completely happy with the lwn covering of rust. If I want to learn developing in Rust I wouldn't expect lwn as place to go Ym2C [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 13:11 UTC (Fri) by xav (subscriber, #18536) [Link ] Seconded; we see lots of Python technical articles but I don't think there's much chance to have some parts of the kernel written in it anytime soon ... it'd be nice to have at least some example of where Rust shines for kernel dev (some obvious ones: getting rid of void*, and getting rid of "if (myptr != NULL)"). [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 16:47 UTC (Fri) by thoughtpolice (subscriber, # 87455) [Link] That's because it's the topic of the mailing list discussion. You can quite literally find dozens of articles about using Rust for things like kernel/bare metal programming if that's what you want. It's actually so easy I'm not sure why you're bothering to complain here. But regardless of complaints, it's not what the topic of the discussion is about, though. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 19:32 UTC (Fri) by bartoc (subscriber, #124262) [ Link] At the very least they probably don't wanna allow multiple versions of packages, which is very common in rust projects (but people don't realize it, because they are hidden behind an up-to-date dependency). Cargo doesn't really like being the passenger in the build process, it likes to drive things itself, and that's probably not gunna fly in the kernel either, which means they need to interact with the rust compiler itself, and that interface is not well documented. At a higher level I think this is the sort of thing that could cause rust to stabilize their ABI unintentionally, without the core rust developers having a choice in the matter. [Reply to this comment] Rustaceans at the border Posted Apr 15, 2022 20:20 UTC (Fri) by shemminger (subscriber, #5739) [Link] Seems like getting a secure language with the insecurity of an external package system would be a bad tradeoff. [Reply to this comment] Copyright (c) 2022, Eklektix, Inc. Comments and public postings are copyrighted by their creators. Linux is a registered trademark of Linus Torvalds