[HN Gopher] A pair of Linux kernel modules using Rust
___________________________________________________________________
A pair of Linux kernel modules using Rust
Author : chmaynard
Score : 288 points
Date : 2022-09-13 09:12 UTC (13 hours ago)
(HTM) web link (lwn.net)
(TXT) w3m dump (lwn.net)
| lofaszvanitt wrote:
| Grsec in the kernel eta?
| rs_rs_rs_rs_rs wrote:
| I can only see that happening if spender goes insane.
| JoshTriplett wrote:
| *goes sane
|
| More seriously: many different security improvements are
| filtering into the kernel idea-by-idea, insofar as folks
| working on kernel security do the actual work of making them
| fit in with the kernel, and/or coming up with better
| alternatives.
| geofft wrote:
| https://kernsec.org/wiki/index.php/Kernel_Self_Protection_Pr...
| is basically trying to upstream grsec's good ideas in upstream-
| friendly ways.
|
| (Oddly, grsec itself has expressed opposition to Rust in the
| kernel, I think on the grounds that their custom GCC plugins
| can't cross the language boundary. Now that there's work on
| supporting Rust in GCC itself, I'm not sure if that objection
| still applies.)
| ncmncm wrote:
| UncleMeat wrote:
| I use C++ on a daily basis. I am extremely proficient in it.
|
| It is an observable and downright undeniable fact that
| developers, even expert developers, cannot consistently write
| correct C++ code. This is true even if you have a greenfield
| project that uses all of the modern best practices and you
| don't have the keyword "new" anywhere in your codebase. People
| fuck up lifetimes. People fuck up iterators. People fuck up
| integer arithmetic. People fuck up concurrency. Using a subset
| of the language and relying on modern tooling like sanitizers
| will not save you.
| ncmncm wrote:
| This does not match my experience.
|
| It has been literally years since I shipped a memory usage
| bug. It just doesn't come up. There is no temptation to make
| memory usage bugs, because they would be extra work to code.
|
| I understand that there is a sub-literature devoted to making
| up stories about how the C++ experience is exactly identical
| to C's, by people insisting they are active, competent C++
| coders, but there are almost always clues that this is false.
| ncmncm wrote:
| Spiteful downvotes notwithstanding, there _really is_ nothing
| Linus can do to stop you coding your eBPF fragments in C++.
| galangalalgol wrote:
| eBPF doesn't prevent UB, just keeps that code from hurting
| the rest of the kernel right?
|
| I have written c++ for 28 years and I can only recall letting
| something rust would have protected me from make it out of
| test once. That I know of...
|
| A coworker found some UB in a rather large c++98 codebase
| recently. It wasn't exactly idiomatic modern c++. But the
| same person writing the same code in c++20 would just have
| had explicit and auto in more places. It was a use after free
| on the stack that wasn't terribly subtle but I only found it
| because I knew something was there. Cppcheck and clang-tidy
| were no help. Debug builds didn't exhibit the UB with g++ or
| clang, clang didn't exhibit it with -O3 (optimized out), but
| g++ did. Asan would have caught it, if they used asan (they
| don't) and if they had unit test coverage for that branch
| (also no).
|
| So experienced c++ developers with years of experience (no
| novices on that codebase) made an error that can't be caught
| by static analysis, and requires 100% branch coverage for
| asan to catch. It eventually popped up in production.
|
| I find modern c++ much more pleasant to write than rust. I
| like that it is easy to interface with the rest of the world.
|
| I hate spending eons to get unit tests to find that last path
| theough the code in a meaningful way. I hate waiting for
| pipelines to run unit tests and coverage to find a mistake.
|
| Modern C++ built with static analysis (including
| modernization checks) takes as long as a rust builds, but
| still doesn't catch as much stuff.
|
| I'm not using rust yet, but I'm planning to start. I'm tired
| of this stuff, it's been almost 30 years.
| j-krieger wrote:
| You're also missing that there is a chance of UB in your
| code base for which your compiler just assumes your
| original intention and will then go on to correct it
| fyvhbhn wrote:
| Could you share an example, please?
| galangalalgol wrote:
| Well the use after free on the stack was getting the aame
| memory over and over so it just happened to work in the
| case above. Until it didn't.
| [deleted]
| speed_spread wrote:
| Going from C++ to Rust, keep in mind that Rust does not
| mainly solve a technical problem, but a human one. It's not
| so much about making programs that run fast and don't fail
| than about making such code writable and maintainable by a
| wide variety of people over a long period of time. The
| initial Rust learning curve is steep but the competence
| curve is actually much flatter than C++, IMO. It's worth
| reiterating this because this is not as apparent to a
| seasoned systems programmer.
| galangalalgol wrote:
| I do find that surprising. I have never used rust for
| more than toy programs, and I don't use a fraction of
| what is available, how can a language with so many
| features have a flat competency curve? I can easily write
| something that doesn't use traits or iterators, panics
| all the time, and calls clone all over the place to
| satiate the BC and call it a day right? I could create
| very unidiomatic code that was still correct, but not
| performant or easy to read/use. I suppose the fact I was
| able to make that list without ever having used the
| language in anger maybe makes your point... But with all
| those features surely I'm missing as many pitfalls as I
| know about?
| speed_spread wrote:
| I meant "flatter" as in "linearly increasing" competence.
| Once one knows the basics, it's easy to write "correct"
| programs in Rust that will not exhibit unexpected
| behavior. Of course, learning more of the language, one
| will write more expressive and performant constructs. The
| progression is made easier by the language and lib having
| relatively few pitfalls. Also, much care has been given
| to the developer's cognitive load through proper tooling
| (cargo) and informative error messages from the compiler.
|
| "Fighting the borrow checker" has become a Rust meme but
| it's actually a good part of the Rust experience to have
| a "conversation" with the compiler. One is forced to
| reflect upon the implications of the app or lib's model,
| often leading to a better design or at least an
| understanding of the limits of the code. This is possibly
| similar to what LISP people describe when talking about
| their REPL-based workflow.
| sebzim4500 wrote:
| >I can easily write something that doesn't use traits or
| iterators
|
| Do you mean that you aren't using them at all or you
| aren't defining your own? Because I would expect any
| nontrivial rust program (outside of some obscure embedded
| context maybe) to make use of the traits that are in the
| standard library/your dependencies. Similarly, you can't
| even write a for loop without using iterators.
|
| >panics all the time
|
| That might be fine, depending on your usecase. Presumably
| if you were going to put it into production you would go
| through and handle the errors. Crucially, by grepping for
| `unwrap` and `expect` you can find the panics very
| easily.
|
| >calls clone all over the place to satiate the BC
|
| If it's not a performance issue then this might be fine.
|
| >I could create very unidiomatic code that was still
| correct, but not performant or easy to read/use
|
| This is true in every language. The value proposition of
| rust is that if you do write idiomatic code then it will
| likely be reasonably performant and easy to maintain.
|
| >But with all those features surely I'm missing as many
| pitfalls as I know about?
|
| Maybe, but not every program will need to use every
| feature. Quite a few of the more advanced features are
| mainly useful for libraries, for example.
| couchand wrote:
| > Crucially, by grepping for `unwrap` and `expect` you
| can find the panics very easily.
|
| I write Rust code up and down the stack, and this is the
| thing that ticks me off more than anything else. You can
| get most of them looking for this stuff (and a handful of
| others), but there are just so many places in vanilla
| Rust that exhibit the Wrong Behavior By Default, e.g.
| a[i] = x + 1;
|
| Which looks so innocent but can panic in two places! Ugh.
| tialaramex wrote:
| If x's Add doesn't have panic on overflow then you get
| rid of the panic risk on the right hand side.
|
| For example if x is Wrapping<u8> then it can't panic.
| Having chosen Wrapping<u8> you will presumably not be
| startled when 255 + 1 == 0
|
| Or if x is Saturating<i16> again, no panic, but 32767 + 1
| == 32767
|
| Wrapping and Saturating really exist (well Saturating
| isn't stable, but it's in nightly) and if you're in a
| space where that's what you want, you can express it
| cleanly in Rust. Most programmers just mean the integers
| they learned in school which don't have a "maximum" or
| "minimum" they go on forever, and so overflow means the
| programmer's model of the world mapping those integers to
| a type like usize was wrong and this code is faulty.
|
| [Edited to add: This doesn't quite work because in the
| end neither Wrapping nor Saturating implement Add over
| the normal integers, since it's not certain what you
| intend if you wrote that, better to have an error asking
| you to explain. AddAssign works though]
|
| Now, there aren't any built in types whose IndexMut can't
| panic AFAIK, but we could build one, and if a was such a
| type then a[i] doesn't panic either.
|
| For example we can imagine a is a (hypothetical)
| PythonDict while x is a Saturating<i16> and now our code
| just compiles to add 1 to x, saturating if necessary, and
| then shove the result in our PythonDict under the value
| of i.
| couchand wrote:
| Yes, Wrapping and Saturating very neatly provide behavior
| you might want in some context. I use them a lot in
| embedded contexts.
|
| Since indexing is nearly always fallible, it should be
| fallible!
| throwaway894345 wrote:
| One can conceive of tooling that can find these panics
| for you so you don't need to grep, but I'm not aware of
| anything that's been written at this time.
| sophacles wrote:
| Sure, you wrote 2 overflows - a buffer one and an integer
| one. Would you rather it silently breaks?
| couchand wrote:
| No, I'd rather it not compile!
| pitaj wrote:
| There are two clippy lints that cover these possible
| panics:
|
| - indexing_slicing
|
| - integer_arithmetic
|
| Enabling those will throw an error on usage in your
| codebase.
|
| Also it's worth noting that arithmetic is only checked
| against overflow when building in debug mode by default.
| couchand wrote:
| > Also it's worth noting that arithmetic is only checked
| against overflow when building in debug mode by default.
|
| Yes, the default behavior in release mode is even worse!
| [deleted]
| mustache_kimono wrote:
| I agree with your parent's assessment. And I think it may
| be because Rust's multi-paradigm approach allows you to
| use what you need when you need it.
|
| > I can easily write something that doesn't use traits or
| iterators, panics all the time, and calls clone all over
| the place to satiate the BC and call it a day right?
|
| You're basically not going to write your own traits until
| you've spent the time required to get something working.
| You'll copy and paste and modify to create another
| function.
|
| If you're unfamiliar with iterators, then you'll write
| for loops. I was unfamiliar with iterators until I said,
| "I'm going to rewrite every for loop in this personal
| project as an iterator." Rust forces you to use lots of
| patterns re: memory, but forcing you locally to implement
| as a for loop or a iterator is not one of them.
|
| I think a fear of clone() is wrongheaded. Most of the
| time it just doesn't matter.
|
| When you think, ugh, this isn't the best I can do, it's
| time to learn iterators/traits, then you can modify the
| code to work then.
|
| > But with all those features surely I'm missing as many
| pitfalls as I know about?
|
| I guess my question would be: compared to what? I think
| the pitfalls are much, much, much more explicit in Rust,
| than C, like this won't even compile because this code
| has a condition which might be a null which I have not
| handled.
|
| For app development, Rust really feels like a smallish
| language once you get over the hump.
| bitcharmer wrote:
| I downvoted you for being hilariously incorrect; in
| particular:
|
| > Coding C++, you need only choose known-correct primitives
| to get the same level of assurance.
| seabrookmx wrote:
| Correct. And eBPF is cool, but it's not a replacement for in-
| tree drivers. It also goes through a strict verification
| process in the kernel before being run which among other
| things, imposes limits on looping.
| throwaway894345 wrote:
| I didn't downvote you out of spite, but because of the
| factual errors in your post, notably:
|
| > Coding C++, you need only choose known-correct primitives
| to get the same level of assurance.
|
| C++ has no such primitives that offer the same level of
| assurance as Rust does. Not every disagreement is spiteful.
| delfinom wrote:
| I too pretend C++ does not have the same exact memory safety
| issues as C because I am also an elite coder.
|
| Also, exceptions, just what the kernel needs! An extremely
| expensive overhead to just error codes because we have the
| processing power in 2022 amirite?
| planede wrote:
| Error handling with exceptions mostly perform better on the
| happy path than error codes and explicit checking. Granted,
| on the sad path they perform much worse.
|
| Of course returning an error code, then not checking it
| performs better than exceptions, but that's comparing apples
| to oranges. The same applies for comparing the same program
| compiled with and without -fno-exceptions.
| playdmc2 wrote:
| exceptions opens the door to your control flow being rug
| pulled in exchange for... an error message? realistically
| all you can do is wrap chunks of code in a `try { } catch
| (const std::exception &e)` block and hope they've
| implemented `std::exception::what`. an enum based approach
| gives you both better error messages and certainty of what
| errors can occur and when.
| ncmncm wrote:
| Trolling is unwelcome here.
| biorach wrote:
| making valid points in a sarcastic manner, while maybe
| annoying to the respondee, is not trolling
| ncmncm wrote:
| Diggsey wrote:
| > Coding it in Rust does not, in fact, guarantee it is correct.
|
| Of course.
|
| > Coding C++, you need only choose known-correct primitives to
| get the same level of assurance.
|
| C++ does not even have the means to express "known-correct"
| primitives. The compiler will not tell you if you accidentally
| access a non-thread-safe type from another thread. It will not
| tell you if you accidentally resize a vector while having an
| outstanding reference to one of its elements. These are not
| "bad primitives" you can blame - getting this stuff right
| requires higher-level reasoning which the C++ compiler cannot
| do for you, and the Rust compiler can.
|
| > is exactly as safe as Rust eBPF, but likely less annoying to
| code.
|
| Personally I find C++ far more annoying to code. I mean header
| files... In 2022? Text based macros? No standardised way to
| manage dependencies?
| ncmncm wrote:
| davidatbu wrote:
| What about this part "the compiler will not tell you if you
| accidentally access a non-thread-safe type from another
| thread"?
| ncmncm wrote:
| seabrookmx wrote:
| From hackernews guidelines:
|
| > Comments should get more thoughtful and substantive,
| not less, as a topic gets more divisive.
|
| > Please don't post shallow dismissals
|
| If you have a real rebuttal, add it. Otherwise don't
| bother commenting.
| sophacles wrote:
| Why not provide some sort of useful statement of how a
| lack of compiler checked memory access provides compiler
| checked memory safety?
|
| To quote you: Trolling is unwelcome here.
| davidatbu wrote:
| Thanks! Could you point me to the C++ primitives that
| prevent cross-thread access of non-thread-safe data? I'd
| honestly love to learn about this.
|
| If it's not asking too much, could you also point me to
| the C++ primitives that prevent dangling pointers as
| well?
|
| EDIT: Also, the primitives that prevents the mutation of
| what a const reference refers to? (Much in the way that
| shared references and exclusive references can't co-exist
| in Rust)?
|
| I'd hope the C++ primitives for the above all perform
| their job at compile time of course.
| fizzynut wrote:
| By definition you can't safely access "non-thread-safe
| data" from another thread unless it was already safe to
| access despite being "non-thread-safe", but there are
| also many ways to make something thread safe.
|
| You can prevent dangling pointers using strong/weak
| pointers or many other ways depending on the context,
| most examples are quite contrived though and the only
| "dangling pointers" I see are usually those relating to
| resources/handles to things from the OS where the
| lifetimes need to be managed more carefully, i.e.
| audio/video/gpu/etc, how would a rust program handle a
| handle to the currently playing audio device being
| unplugged and invalidating your handle? Every OS also has
| its own quirks, so how would it work cross platform?
|
| Constexpr primatives/functions have compile time
| guarantees and can't be changed.
| davidatbu wrote:
| About the threading stuff, I think I wasn't clear
| initially. Maybe a rephrasing/simplification of my
| question will help: Does C++ have a compile-time way of
| ensuring, say, that mutex-protected data isn't accessed
| without acquiring a lock on the mutex?
|
| > You can prevent dangling pointers using strong/weak
| pointers
|
| I presume you're talking about
| unique_ptr/shared_ptr/weak_ptr right? If one avoids raw
| pointers and uses those, I think you cannot have a shared
| reference without ref counting right? (because shared_ptr
| uses ref-counting). Not that this invalidates your point,
| just thought it was useful to point out.
|
| > how would a rust program handle a handle to the
| currently playing audio device being unplugged and
| invalidating your handle?
|
| I've never had to deal with that situation, but I presume
| some mechanism similar to how the mutex in the stdlib has
| a .lock() method that returns an Option<MutexGuard<_>>
| (which forces the user to acknowledge/handle the case of
| a poisoned mutex).
|
| > Constexpr primatives/functions have compile time
| guarantees and can't be changed.
|
| Consider the c++ code here: int a = 4;
| const int &b = a; a += 1;
|
| Such a thing would be disallowed in Rust, which (though
| hard to see from this example) is the exact mechanism
| that prevents a Vec being modified while it is being
| iterated on. Does C++ have primitives that provide such
| guarantees?
| nyanpasu64 wrote:
| Practically all valid Rust can be rewritten as valid C++.
| To make C++ reject most invalid thread-based code that
| Rust rejects, you can mark cross-thread-shared references
| as const&, atomic fields as mutable, and write a Mutex<T>
| class which wraps a mutable T, and only allows accessing
| a T& through a Mutex const& by locking the mutex. So
| implementing Rust-style threading in C++ is viable, but
| extra work to write a Mutex<T>, and the compiler lacks
| borrow checking to prevent you from making lifetime and
| overlapping &/&mut mistakes and committing data races.
| (http://www.drdobbs.com/cpp/volatile-the-multithreaded-
| progra... proposes using volatile& for this purpose, but
| many regard this as actually a bad idea.) Though in
| practice most C++ codebases sprinkle mutexes ad-hoc,
| relying on at best comments describing what data is
| protected by what mutex, with predictably disastrous
| results.
|
| I feel Rust is a better language than C++ at implementing
| correct multithreaded code, but is so constraining that
| it makes correct single-threaded code hard to implement
| when it doesn't fit neatly into tree-shaped ownership
| (eg. graphs, GUI trees, and extending existing program
| architectures).
| davidatbu wrote:
| Learnt a lot from this. Thanks!
| fizzynut wrote:
| Like I said, use constexpr. constexpr int
| a = 4; auto& b = a; a+= 1; // compile
| error b+= 1; // compile error
|
| Trying to make b constexpr with "a" as an int is also a
| compile error.
| davidatbu wrote:
| That would unnecessarily constraint `a` to be known at
| compile time though? I imagine that would be incredibly
| limiting.
|
| E.g., if you're implementing an iterator that takes a
| reference to the length of the container being iterated,
| and you require that that reference is constexpr, that
| means the container will have a fixed, compile-time-
| known, size.
|
| Since you didn't reply to the other parts of the comment,
| I'll presume that C++ doesn't have the mutex I'm talking
| about (apart from this attempt[0], which I just learnt
| about and commented on in a different reply), and that
| it's not possible to have non-ref-counted shared
| references while statically avoiding dangling pointers.
|
| [0] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
| fizzynut wrote:
| A lot of constants are known at compile time and
| constexpr solves a lot of problems. However, you do not
| need constexpr to solve your example- this is still a
| compile error:
|
| void iter(std::vector<int>& vec) {
| auto& vec_size = vec.size(); vec_size +=
| 10; //compile error
|
| }
|
| There are many ways of forcing compile errors for mutexes
| that people have built, but they all come with their own
| trade-offs.
|
| I've not encountered a problem where I specifically
| needed "non-ref-counted shared references while
| statically avoiding dangling pointers", so I'm not an
| expert on it.
| davidatbu wrote:
| What I'm referring to with regards containers is:
| preventing containers being modified while they are being
| iterated on (which is usually bad, because you will
| probably invalidate the iterator when modifying the
| container). That is what is prevented by Rust's
| references rules, and to my knowledge, isn't possible to
| protect against in C++ (statically).
|
| Thanks for the replies about the other stuff!
| tialaramex wrote:
| > how would a rust program handle a handle to the
| currently playing audio device being unplugged and
| invalidating your handle?
|
| You would either represent this as all the operations
| having a Result which can be Unplugged or whatever, or
| you might decide (as the designer of that library) that
| you'll eat the error and silently ignore operations when
| unplugged, offering an unplugged() predicate so that
| callers can check whether they got unplugged if they
| care.
|
| If you mean, what if the system is allowed to just
| invalidate handles we've got for some reason, without
| telling us about that, and then after invalidation they
| just mustn't be used, then you're screwed, regardless of
| programming language, that's a pretty bad design and
| there's nothing to be done about it.
|
| A more common design (e.g. for OS file handles) has an
| explicit call where you give back the handle (close in
| the case of file handles), promising you won't use it
| again. If the handle is meanwhile broken for some reason
| anyway (e.g. user pulled out the USB stick with the file
| on it) then the OS will tell you about the problem, but
| _you_ still need to acknowledge that by closing the
| broken handle, you don 't just suddenly find there's a
| different file behind your existing handle now. It's no
| trouble to represent this properly in Rust.
|
| Let's explain a bit more about why Rust cares about
| thread safety. Rust types can have marker traits named
| Send and Sync. Send means "You can safely give this type
| to another thread" and Sync means "You can safely give
| _references_ to this type to another thread ".
|
| For example lets say I have a Goose, which is three
| 32-bit signed integers named x, y, z, plus three booleans
| flapping, honking and hissing. All six of those elements
| are Send, so Goose is Send. I can give a Goose to another
| thread no problem.
|
| But the type Rc<Goose> isn't Send. Rc is a reference
| counted smart pointer, like shared_ptr<Goose> from C++
| except it's not for threaded software. It's a little bit
| faster, especially on some CPUs, but you can't safely use
| it across threads. Rust has another reference counted
| smart point Arc, and Arc<Goose> is Send for the same
| reason shared_ptr<Goose> is thread safe in C++ -- it uses
| atomic integers for reference counting.
|
| Because Rust tracks the marker traits for me, I don't
| need to carefully read documentation for a new type I'm
| using FunkyTractor to check whether it's thread safe. If
| it's Send then I can give it to another thread, and if it
| isn't then that program doesn't compile and I go "Aww"
| and use say a Mutex to wrap my FunkyTractor so that
| multiple threads can access it safely, then it will
| compile.
|
| Only the people actually innovating tricky thread safety
| stuff (e.g. building your own spinlock) need to care
| about this, and make decisions like "Should this type I'm
| creating say it is Send?" for everybody else it's
| automatic.
| NavinF wrote:
| Ignoring GP's trolling, closest would be
| https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
| davidatbu wrote:
| Thank you! IIUC correctly, the TLDR of that is:
| An extension to C++ that adds (pretty much) a type system
| on top of C++ to enable static analysis and prevention of
| race conditions. Widely used at Google.
|
| Super interesting! That's exactly what Rust is doing too,
| except I think Rust's version (which pretty much relies
| on two "traits", `Send` and `Sync`) is much simpler, much
| more ergonomic, and much more complete.
| jdright wrote:
| He can't because he doesn't understand what you're
| asking.
| ncmncm wrote:
| Trolling is unwelcome here.
| davidatbu wrote:
| Tbh, I'm also a Rust beginner and know even less about
| C++, so I'm sincerely trying to learn here.
| Filligree wrote:
| C++ doesn't provide any of the checks being talked about
| here. There's nothing much to learn; they just don't
| exist.
| davidatbu wrote:
| Thank you! Still waiting on you @ncmncm, if you think
| differently.
| ncmncm wrote:
| If you can see it at all, it will be an atomic type.
| davidatbu wrote:
| I skimmed docs for std::atomic, and I believe what you're
| suggesting is a super-brief version of what was outlined
| here[0].
|
| If so, thanks!
|
| Through I don't yet "see it" that C++ has safety
| primitives that "equal" that of Rust's, I've learnt a lot
| about C++'s safety features from other folks' replies in
| this thread. So again, thanks!
|
| [0] https://news.ycombinator.com/reply?id=32824168
| tialaramex wrote:
| > Hint: use of std::vector would not appear in kernel code,
| in any case.
|
| They didn't ask for std::vector, just for a vector, which
| in this context is any extensible array type. So that seems
| like a disadvantage straight away. Rust for Linux provides
| Vec like your userspace Rust code.
|
| Of course, the alloc::Vec in Rust for Linux doesn't have
| yolo-allocation, if you want space for 615 items in your
| Vec you will need Vec::try_with_capacity(615) rather than
| Vec::with_capacity(615) because you must decide what to do
| if the allocation fails, and if you want to push an item
| you'll need to try_push(item) not just push_item) for the
| same reason as the Vec might be full and unable to grow.
|
| Rust's usual promises hold up though, if I try to
| Vec::truncate() a Vec I need to provide a mutable reference
| to the Vec (&mut self) and Rust won't give me one of those
| while I still have outstanding references, even immutable
| ones, to that Vec, my code won't compile.
| j-krieger wrote:
| The module system in CPP is dead on arrival. Almost no
| libraries and build systems support them. Modules still
| produce bad output on gcc.
| mathstuf wrote:
| It is being worked on (at the build system level). Even
| so, I don't think Linux would support modules because
| it'd require some...unfortunate behaviors to support in
| `make`, namely at least one recursive make level.
|
| Though if `gmake` is assumed, there is `libcody` support
| that's in a patch that can do it without recursive `make`
| at the expense of having an unknown number of outstanding
| compilation processes open waiting to figure out what
| their topological order is at build time. I do not know
| what the behavior of such a solution is in the presence
| of import cycles or unsatisfied imports, but that's also
| why I prefer the "scanning" solution.
|
| FD: CMake developer working on C++ modules support.
| throwaway894345 wrote:
| > Personally I find C++ far more annoying to code. I mean
| header files... In 2022? Text based macros? No standardised
| way to manage dependencies?
|
| Yep. This is basically why I switched to Rust from C++, but I
| stayed for the coherent language design, the rapid pace of
| improvement, and the borrow checker.
| Kukumber wrote:
| galangalalgol wrote:
| You forgot to insult vi or emacs and denegrate some brace
| style.
| iExploder wrote:
| Typical spaces over tabs enjoyer...
| azernik wrote:
| Religions generally don't phrase their actions as experiments,
| as this developer is doing.
|
| Engineers test new techniques to see what works. Not all of
| them play out. The Linux kernel community is very good at
| sorting out the junk, and this is part of that process.
| sophacles wrote:
| What junk am I subscribing to by thinking compiler checking of
| my memory access is neat, and by thinking it has uses in the
| kernel (which manages memory as one of its jobs IIRC)?
| Kukumber wrote:
| Repeating what i said elsewhere:
|
| My issue is people who will "trust" security only because of
| a language used, wich is a flawed and dangerous mindset
|
| I don't want to depend on that kind of project
| sophacles wrote:
| Im confused. You are going to change an OS because you
| think "trusting something based on use of a particular
| language" is a problem that might come up. Isn't that just
| you basing your trust level on the use of a particular
| language?
| adgjlsfhk1 wrote:
| It's not so much trusting security because of language use
| as distrusting security because of language use. With the
| tools C gives you, it is impossible for multiple humans to
| write a large secure program. Rust doesn't guarantee
| security, but it gives you a fighting chance.
| sebzim4500 wrote:
| Isn't choosing an OS based on what language it is written in
| basically doing what 'rust evangelists' are accused of?
|
| Programming languages are tools, why would you stop using
| something because of a change which does not make the product
| worse in any way?
| Kukumber wrote:
| My issue is people who will "trust" security only because of
| a language used, wich is a flawed and dangerous mindset
|
| I don't want to depend on that kind of project
| playdmc2 wrote:
| thank you fellow brogrammer, i would only trust Real
| Code(tm) written by Real Coders(tm) in a Real Programming
| Language(tm) such as c, as god intended
| stoplying1 wrote:
| Build up that absurd strawman so you can be angry at Rust
| continuing to eat the software world, yeah! Rah!
|
| You won't use Linux, because it might have a bit of Rust in
| it, because some microcosm of Linux users will like that it
| has Rust and trust it more? That about right? Okay, you
| have fun.
| Kukumber wrote:
| I am still free to do what i want? right?
|
| I never said they shouldn't do it, or you shouldn't use
| it, they do what ever they want too, hence why i move
| away, to let them do what the f they want!
| davidatbu wrote:
| Question: The author of the 9P server module "created an executor
| that can run async code in a kernel workqueue"; is this made a
| lot easier by the fact that Rust doesn't include an async
| executor as part of the core/stdlib?
| JoshTriplett wrote:
| The work on Rust in the Linux kernel doesn't include the
| standard library, and even if it did, having an executor around
| doesn't prevent you from creating and using a different one.
| davidatbu wrote:
| Thank you!
| estebank wrote:
| I think they are pointing out that because the Rust design
| has decoupled the functionality from the implementation,
| these kind of replacements are even _possible_ in the first
| place.
|
| Of course, if you use an alternative executor there are large
| swaths of the (non-std) async ecosystem that are not
| available to you (thinking of anything like relying on a
| tokio::timer::Delay as an example). But as you point out Rust
| in the Linux kernel already doesn't rely on the std, so they
| wouldn't even begin to imagine to use an off the shelf crate
| and expect it to work unmodified for their use case. I just
| think it is worth pointing out that this _is_ a constraint we
| currently have.
| davidatbu wrote:
| You're better at expressing my thoughts than I am :)
| littlestymaar wrote:
| estebank reads other people's mind, that's why he's so
| good at writing helpful compiler errors.
| estebank wrote:
| Thank you, you're too kind. I don't know how much that
| experience of "make the compiler try to talk to a human
| in a way they can understand it" translates to other
| parts of life, though :)
| JoshTriplett wrote:
| Right, yes: if we had a design that built a single executor
| into the language or we had a runtime that included it, you
| wouldn't be able to replace that as easily. Because
| executors are, at most, a library mechanism, you can always
| replace them or run more than one.
| colonwqbang wrote:
| It doesn't _in rust_ but in many other languages there would
| have been a single blessed async library and trying to use
| anything else would have been very painful. The fact that
| rust doesn 't particularly force the use of it's standard
| runtime is key to why rust is useful for kernel work.
| brundolf wrote:
| Yeah, but I think what they're getting at is that Rust
| could provide a standard executor in the library _and also_
| be modular enough that you can use whichever one you want.
| The questions are orthogonal
| [deleted]
| nu11ptr wrote:
| I'm a huge fan of Rust (I write in it for a living), but I still
| can't help wonder how they are going to tolerate the compile
| times. A couple of drivers...no problem, but what happens once
| they get a large number of drivers in Rust? Compile times will
| definitely start going up considerably. I wonder how much this
| has been considered, or possibly they have no plans to do
| anything this extensive?
| playdmc2 wrote:
| increased compile times are a reasonable tradeoff for
| correctness, no? hardware can always be thrown at the problem.
| as soon as someone does a c oopsie you're going to lose
| whatever time you saved to debugging
| CraigJPerry wrote:
| It might not be a big hurdle. OS developers aren't as sensitive
| to compile times as app devs. An hour of futzing to bring up on
| a hardware dev board isn't unheard of (although 5-10 mins would
| be more typical).
|
| There was a big regression in compile times in the linux 2.2 to
| 2.4 time frame (the kernel grew a lot of functionality) but i
| don't recall it getting much more than a little grumbling.
|
| That was in the time of the Linux from Scratch book and Gentoo
| stage 0 builds. Some people don't mind overnight compiles if it
| means they can specify just exactly the right combo of compiler
| flags to eeek out the max performance of their cpu.
| yencabulator wrote:
| Kernel compiles are a classic benchmark by which Linus has
| judged performance-related patches. Kernel developers do
| care(, mostly about incremental builds).
| guenthert wrote:
| I worked on a (legacy now) larg-ish code base (mostly C++ '03
| with some self-imposed restrictions, but extensive usage of
| templates) for an 'embedded' system (enterprise NAS). A full
| build took north of 6h on the reasonable beefy desk-side
| workstations available to us (we considered distributed
| builds, but never got to test it afair). Every once in a
| while there was some grumbling about the build times, but it
| never really improved (every three years or so we got better
| build machines, but by that time the code base had grown as
| well), partly, I suspect, due to https://xkcd.com/303/ and
| partly because we learned to work-around that by finding the
| Makefile in the tree which (likely) gave us a proper build in
| the shortest possible time for the files we were actually
| working on (not being helped by the most complex Makefile
| tree I've ever seen).
|
| I very much doubt that such large build times would go over
| well in the Linux kernel community.
| kibwen wrote:
| Given the disposition of kernel developers and the bespoke
| needs of drivers, I assume that Rust-written drivers will be
| neither pulling in dependencies willy-nilly nor will they be
| using generics all that heavily (but feel free to correct me if
| I assume wrongly). In that instance I don't think the Rust
| compilatiom process would impose any undue burden, especially
| since Rust can only be used for optional components of the
| kernel since it doesn't compile for all the targets that the
| kernel supports.
| Matthias247 wrote:
| If you watch the actual talk and check code at
| https://github.com/Rust-for-Linux you will see that it's
| actually pretty heavy on generics and macros. Eg for the
| trait that defines file system operations
| (https://github.com/Rust-for-
| Linux/linux/blob/459035ab65c0ebb...) and other bindings to C
| code. I was actually surprised of seeing those, since they
| certainly don't help with telling people a story that Rust is
| an easy language. But it's probably the best one can do when
| having to interface with predefined APIs.
|
| But You are certainly right in that external dependencies
| won't be an issue.
| kibwen wrote:
| The trait there doesn't look concerning to me (e.g. none of
| its methods are generic (its methods do use associated
| types, but those are pretty trivial to resolve or generate
| code for; there's only ever one set of associated types per
| _impl_ , whereas with generic functions there's the
| potential for combinatorial code explosion depending on
| _usage_ )).
|
| However, I do see a custom `vtable` macro in there, and
| indeed macros can be a concern for compilation speed if
| they're not implemented with compiler-friendliness in mind.
| But other than that, I'd say that file shouldn't represent
| a problem for the compiler.
| loeg wrote:
| A big source of compile time used to be proc macros -- the
| kernel could just disallow those?
| sebzim4500 wrote:
| The Asahi GPU driver which is currently being developed uses
| a proc macro to deal with the different versions of firmware
| that the driver must support (i.e. fields are added/removed
| in firmware updates, and the driver must support both).
|
| That probably will never compile especially quickly. I'm not
| sure how sensitive kernel devs are to clean build times.
| Presumably doing a non-incremental build is relatively rare?
| kibwen wrote:
| _> That probably will never compile especially quickly._
|
| Can you link the code? Proc macros are not inherently slow
| to compile, but most proc macros pull in the `syn` and
| `quote` crates for convenience, which are fairly
| heavyweight (and then most people experience proc macros
| most commonly via Serde, which is doing a ton of work on
| its own). In simple cases you can forego most of that work,
| and some libraries like
| https://github.com/Manishearth/absolution exist as
| lightweight alternatives in these cases. Depending on what
| the Asahi driver needs it could also benefit from something
| like this (although without seeing the code I'm not sure
| why it would need proc macros when it sounds like
| conditional compilation via `cfg` would suffice there).
| Groxx wrote:
| When tightly controlled, Rust compilation times are quite good.
| Without attention it can bloat tremendously with proc macros
| and excessive generics (turn them into dyns!), but it's totally
| manageable. Depending on your corners of the ecosystem, your
| default experience could be a few orders of magnitude better or
| worse than someone else's.
| nu11ptr wrote:
| I haven't done much experimentation, but I've heard the same.
| Currently, for new projects I try to design with as many
| crates as possible in order to a) keep them small b) make as
| many 'spokes' (aka not dependent on other crates) as
| possible. This helps a ton and something I wish I'd have
| known before starting some of my large projects. I also wish
| I had used more trait objects and less generics - the
| overhead rarely matters in practice.
| Gwypaas wrote:
| For the professional work I have done in Rust the compile times
| are workable when using cargo workspaces and incremental
| compilation. Making sure to when possible work directly in a
| member crate compiling only that for the iteration cycle. That
| requires a DAG structure for all dependencies so does not work
| everywhere.
|
| Getting CI up to speed is a completely different (awful) topic.
| opless wrote:
| The 9P file protocol, he said, comes from the Plan 9 operating
| system. The kernel has a 9P client, but no 9P server. There is a
| 9P server in QEMU that can be used to export host filesystems
| into a guest. The protocol is simple, Almeida said, defining a
| set of only ten operations. His 9P server implementation works
| now, in a read-only mode, and required just over * _1,000*_ lines
| of code.
|
| ... I wonder how many lines of code the Plan9 (C) server takes?
| (much less I suspect)
| yencabulator wrote:
| This Linux 9P server is a 9P-to-Linux-VFS translator. No such
| software component exists in Plan9.
|
| You can think of the Plan9 kernel as a 9P multiplexer. 9P in,
| 9P out. The closest matching thing would be e.g. a 9P-to-
| ext4-in-a-file translator -- but it's always "9P server talking
| to $FOO" (or "9P server that makes files up on the fly").
|
| Most of the complexity of this Linux 9P server would be in
| translating between the unrelated worlds of 9P and Linux VFS. A
| more apt comparison is the size of this 9P server vs an in-
| kernel NFS server -- they both translate something unrelated to
| Linux VFS.
| [deleted]
| tialaramex wrote:
| From previous example drivers, I'd expect that it's not
| dissimilar overall and mostly the cause will be style
| preferences e.g. maybe the C programmers loves one-line for
| loops and the Rust programmer chooses default Rust style which
| doesn't do that, or contrariwise the C programmer finds it
| helpful to write out complicated variable types across multiple
| lines and the Rust programmer was comfortable letting them be
| inferred instead.
|
| If the C ends up hand rolling something Rust just has built-in,
| that adds up pretty quickly. For example Rust's arrays, strings
| etc. know how big they are, C's do not and so you need extra
| code to re-calculate or to explicitly store and retrieve those
| sizes. On the other hand it may be tempting to express a Rust
| type as having methods defined over it, rather than as you must
| in C using only free functions throughout, there's also a Rust
| discipline of always deriving common traits e.g. Clone, Debug
| and Eq, when they're appropriate, as a convenience, that's one
| line extra in typical style.
| ainar-g wrote:
| I did not expect Plan 9, the Linux kernel, and Rust to intersect
| like that! Would adding this module mean that I could finally
| mount a 9P filesystem on my Linux machine without the need for
| third-party software and FUSE?
| remexre wrote:
| You already could, actually! zgrep CONFIG_9P_FS /proc/config.gz
| to see if your kernel's configured for it, modprobe 9p if it's
| built as a module. The article's talking about doing an in-
| kernel 9P _server_ instead; I think this is just so that it can
| act as an abstraction of an SMB server, because you ought to be
| able to do a 9P server in userspace with no loss of
| capabilities...
| ainar-g wrote:
| /proc/config.gz seems to be unavailable to Debian 11, at
| least by default, but "modinfo 9p" does show the module.
| Thanks for the clarification!
| rascul wrote:
| > /proc/config.gz seems to be unavailable to Debian 11
|
| For clarification:
|
| > /proc/config.gz isn't available in Debian, because the
| config is provided in /boot/config-*, no need for the in-
| memory variant
|
| https://wiki.debian.org/KernelFAQ#line-46
| MisterTea wrote:
| > I think this is just so that it can act as an abstraction
| of an SMB server, because you ought to be able to do a 9P
| server in userspace with no loss of capabilities...
|
| Correct, 9p file servers can live in user-space. At some
| point you have to mount the fs which on lunix/unix is done
| in-kernel.
|
| In plan 9 the kernel is just a VFS that lets you build
| arbitrary file trees from collections of mounts and binds.
| On-disk file systems like FAT and EXT are served via user-
| space programs which open the disk partition (served as a
| file by sd(3)) and translates the 9p messages to EXT, FAT,
| etc operations. File servers can also provide synthetic
| files, e.g. an arduino can serve its io bits and/or internal
| variables over 9p (ninepea on github written by echoine.)
|
| Since its a VFS all the way down and data moves via two way
| pipes mapped to fd's you can boot and pull root from ANY 9p
| server. Don't care if its CWFS, HJFS, FAT, EXT, SMB, SSHFS,
| NFS, etc. Kernel don't care as long as it sees files through
| a 9p connection over whatever 2-way medium you are using
| including rs232. You cant touch that level of flexibility
| with other operating systems.
| zppln wrote:
| This was a pretty weak article? All it basically says is that it
| can be done and adds some speculation that it might make things
| better. But not much testing have been carried out and the
| comparisons to existing code that have been made show the Rust
| implementation to perform worse.
| Karellen wrote:
| Linux Weekly News focusses heavily on work going on in the
| Linux kernel, and often looks at features and patch sets which
| aren't quite ready for mainline yet, but could be an
| interesting shift in direction for the kernel in future.
|
| The inclusion of Rust in the kernel is one of these ongoing
| bits of work which isn't ready for mainline yet, but has still
| generated a lot of interesting discussion in the past on
| whether Rust is suitable at all, or whether it's suitable in
| it's default form, or if it is suitable then what is the best
| way to start introducing it, or, or, etc...
|
| And this is another step along that (long, convuluted) path.
| The proposed 9P implementation specifically - to me at least.
|
| If you mostly care about the Linux kernel insofar as what
| features it can offer you right now then, sure, this might not
| seem that interesting. And that's entirely fair - there are
| plenty of other technologies out there that I have that level
| of interest in.
|
| But for the people who take a deep interest in the upcoming
| work and future direction of Linux, I think it's quite
| interesting.
|
| I'd agree that the article probably is a bit too niche, or
| forward-looking, for a generic tech website/magazine, and
| doesn't have a huge target audience. But for a focussed site
| like LWN, I think it fits in pretty well, and matches the
| audience that LWN targets. It's definitely the kind of article
| that I keep my subscription for.
| kibwen wrote:
| _> show the Rust implementation to perform worse_
|
| Here in the Linux Plumbers Conference talk regarding the NVMe
| driver (the same one discussed in the OP), the benchmarks show
| nearly identical performance to the existing implementation. At
| the end of the talk the co-author of the NVMe spec (Matthew
| Wilcox) made a point to stand up and comment with how
| unexpectedly impressed he is with the performance: _" I was not
| expecting to see these performance numbers, they are amazing._"
| (at this timestamp: https://youtu.be/Xw9pKeJ-4Bw?t=9486 ).
|
| EDIT: here's the timestamp of the benchmarks themselves:
| https://youtu.be/Xw9pKeJ-4Bw?t=8626
| motiejus wrote:
| Folks, there is a good reason this is a subscriber-only content.
| It takes time and effort (and money!) to keep us up to date. For
| such a long time.
|
| This is the 5'th SubscriberLink over the last week on HN:
| https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
|
| Please, folks, subscribe, and let Corbet put the subscriber links
| here himself. The LWN weekly editions are definitely worth every
| penny.
|
| Edit: I am paying LWN since 2012 and intend to keep paying for as
| long as there is LWN.
| Bancakes wrote:
| If LWN disappears tomorrow, I'll get my news elsewhere. Thank
| you for your concerns.
| penguin_booze wrote:
| I recommend browsing their index pages, and pause and ponder
| where else you'd find such high quality, curated,
| approachable content, in such concentration. You could
| suggest the kernel's own documentation, but that's more
| developer-facing.
| sophacles wrote:
| Where else do you get in depth kernel reporting?
| Bancakes wrote:
| Like the other poster said, in the source code.
| sophacles wrote:
| That's not reporting.
| jbirer wrote:
| Hacker News is not the place to post paywalled content, whether
| you believe the reasons are valid or not.
| cycomanic wrote:
| It should be pointed out that Corbet himself has come out in
| support of subscriber links posted here and at other places.
| That's why they provided that capability. That said it
| shouldn't be abused and I suspect if it ends up becoming too
| much they might remove that ability.
|
| I agree with one of the other posters, we should support
| quality journalism, and lwn definitely is.
| Aissen wrote:
| Rules are a bit more subtle:
| https://news.ycombinator.com/newsfaq.html (see: "Are paywalls
| ok?")
| davidatbu wrote:
| In fact, the only thing the rules call out as not ok with
| regards to pay walls is complaining about them.
| yamtaddle wrote:
| ... or posting paywalled links that _don 't_ have work-
| arounds.
| wartijn_ wrote:
| It is according to the faq[0]
|
| ---
|
| Are paywalls ok?
|
| It's ok to post stories from sites with paywalls that have
| workarounds.
|
| In comments, it's ok to ask how to read an article and to
| help other users do so. But please don't post complaints
| about paywalls. Those are off topic. More here.
|
| ---
|
| [0] https://news.ycombinator.com/newsfaq.html
| o_1 wrote:
| i really hope substack takes off, the strategy behind pay
| content is much more responsible than javascript "walls"
| mhss wrote:
| Thanks for the reminder. I was a subscriber for a few years and
| then I stopped when I hit a long time crunch with no time to
| read much news. I just subscribed now that I feel I have some
| leisure time again for this and HN in general.
| mccorrinall wrote:
| I only found out about LEB via links shared on HN. I haven't
| subscribed yet but will consider it once I landed a job. I
| don't think that sharing their most interesting articles is
| hurting them, more like the opposite.
| jabiko wrote:
| > _This is the 5 'th SubscriberLink over the last week on HN_
|
| Its the first post to gain any meaningful traction
|
| > Please, folks, subscribe, and let Corbet put the subscriber
| links here himself. The LWN weekly editions are definitely
| worth every penny.
|
| According to corbet its totally fine if every now and then a
| subscriber link pops up on HN:
| https://news.ycombinator.com/item?id=1966033
|
| > _As long as the posting of subscriber links in places like
| this is occasional, I believe it serves as good marketing for
| LWN - indeed, every now and then, I even do it myself. We just
| hope that people realize that we run nine feature articles
| every week, all of which are instantly accessible to LWN
| subscribers._
|
| The LWN FAQ also considers it no problem:
| https://lwn.net/op/FAQ.lwn#slinks
|
| > _Where is it appropriate to post a subscriber link?_
|
| > _Almost anywhere. Private mail, messages to project mailing
| lists, and blog entries are all appropriate. As long as people
| do not use subscriber links as a way to defeat our attempts to
| gain subscribers, we are happy to see them shared._
|
| I think posting this on HN does not defeat the purpose of
| gaining subscribers, but supports them. At least for me it
| worked. After reading some LWN articles on HN I decided to
| become a paying subscriber.
|
| Telling people they are not allowed to post such links here
| actually IMHO hurts LWN.
| jwilk wrote:
| Another, more recent statement from corbet:
|
| https://news.ycombinator.com/item?id=31852477
| capitol_ wrote:
| But they didn't tell them that its not allowed, they just
| encouraged people to subscribe to this excellent publication.
| yencabulator wrote:
| "and let Corbet put the subscriber links here himself"
| smcl wrote:
| Is "SubscriberLink" a way to share paywalled LWN stuff with
| non-subscribers? Damn, I had no idea. I just presumed they un-
| paywalled stuff every now and again (like after N days or so).
| gnastygnorc wrote:
| I believe all paywalled content becomes free to read two
| weeks after initial publication
| smcl wrote:
| Ah ok. I should still probably sub, I tend to read LWN when
| it pops up here and never really gave it a moments thought
| before :O
| wufocaculura wrote:
| Highly recommend.
|
| Just to show support for corbet and his team. He could
| keep the articles paywalled forever, but is making them
| free after a week. Becoming a paying member is our way to
| say thank you for his fair attitude (and keeping ads at
| minimum level).
|
| I work at totaly different field in IT, most articles are
| just "wow, that's interesting" for me and - I still keep
| LWN's subscriptionm and it is the only one I have.
| _wolfie_ wrote:
| In my humble opinion it is definitely worth it. All
| people over at LWN do tremendous job and should really be
| supported.
| steveklabnik wrote:
| I haven't even used Linux for a few years now. Still pay
| for LWN. It's so worth supporting.
| sophacles wrote:
| I can honestly say the resources at LWN have helped my
| career immensely. The in depth series on topics like
| namespaces and io_uring directly allowed me to tackle
| some hard projects - without those articles I would have
| steered clear of "magic i don't understand" and the
| relevant projects would have suffered.
|
| It's definitely worth subscribing if you have the means
| (and the low cost of subscriptions suggests a lot of
| readers here have the means).
| couchand wrote:
| I'd agree that often with new features and subsytems
| recent LWN articles are the best documentation around.
| jwilk wrote:
| It's (usually?) less than that.
|
| https://lwn.net/op/FAQ.lwn
|
| > _Must I subscribe to read the Weekly Edition?_
|
| > _No, the Weekly Edition becomes freely available to all
| readers one week after its publication._
| [deleted]
| thejammahimself wrote:
| Indeed. They even provide a mailing list you can subscribe
| to which will notify you whenever an article goes free. I
| think you need to have an account with them firstly though.
| voxadam wrote:
| > I just presumed they un-paywalled stuff every now and again
| (like after N days or so).
|
| All of LWN's paywalled articles become free for everyone to
| view after a week. During the week that articles are
| paywalled any subscriber can share currently paywalled
| content by creating a SubscriberLink.
|
| LWN produces some of the best highly technical Linux and
| Linux adjacent content on the web.
| ArrayBoundCheck wrote:
| So one person said creating the rust interface is very difficult
| to do correctly and the other said rust bare metal isn't complete
| enough for his use case
|
| I called both of these (and compile times) when people first
| tried to get rust on the linux kernel. Maybe it's better to
| acknowledge these problems before you start instead of beating
| your head? I'm no psychic and I know I wasn't the only one who
| called these
| dogleash wrote:
| > Maybe it's better to acknowledge these problems before you
| start instead of beating your head?
|
| What's unacknowledged? They're doing exploratory work. That
| often involves developing under known and expected limitations
| to understand the problem space and have something to test with
| when progress is made towards addressing limitations.
| steeleduncan wrote:
| How do these Rust kernel modules handle out of bounds access in
| an array?
|
| In a C module it would be undefined behaviour leading to kernel
| panic or weird bugs, in a Rust userspace binary it would be a
| panic that terminates the process with an error message, what
| happens here? Would it be a kernel panic as well? or is there a
| possibility of handling it?
| EugeneOZ wrote:
| You can catch the panic. It will panic, but I don't know if the
| driver will catch it. I hope so :)
| kzrdude wrote:
| Can you, in kernel context?
| EugeneOZ wrote:
| After comments below, I'm not so sure. I was talking about
| regular Rust, I didn't know that Linux Rust is patched.
| Sorry.
| ncmncm wrote:
| This not actually substantively different from throwing an
| exception.
| davidatbu wrote:
| I believe Rust in linux was made so that it never panics.
| Here's a patch that removes panicking allocations for
| example: https://lore.kernel.org/lkml/20210704202756.29107-
| 1-ojeda@ke... (but I think _all_ other instances of
| panicking were removed as well).
|
| EDIT: Look at replies. "Linus considers panics acceptable
| in some cases".
| tialaramex wrote:
| Linus considers it acceptable to panic for some
| programming mistakes, since after all the C code also
| blows up if you make some programming mistakes.
|
| One I ran into (in the sense of read about, not
| experienced) was if I flatten a Vec of arrays, it's
| theoretically possible that the flattened structure has
| too many items in it to represent as a machine word
| integer. If this happens the flatten operation will
| panic.
|
| This can't happen in a language like C (or C++) because
| their smallest type has size 1, so all the arrays can't
| possibly be bigger in total size than the amount of
| memory, that's nonsense. But Rust has two smaller sizes
| than this. The relevant one here is the Zero Size Type,
| Rust has no problem with the idea of an array of empty
| tuples, such an array could have say, a billion empty
| tuples in it, yet on a 32-bit system it just needs 4
| bytes (to remember how many empty tuples are in it).
|
| We can see that flattening a Vec of arrays of empty
| tuples is a pretty wild thing to choose to do, and
| nevertheless even if we do it, it only panics when the
| total amount of empty tuples won't fit in the integers of
| our native word size. But the function could be asked to
| do this, and so it might panic.
|
| [ You might be wondering how can there be _two_ sizes
| smaller than C 's single byte types in Rust. The answer
| is the Never type ! and its pseudonym Infallible. The
| Never type is Empty, no values of this type are possible,
| so not only does Rust never need to store this type, it
| doesn't even need to emit machine code to handle this
| type - the code could never run. This makes sense in
| Generic programming, we can write Generic error handling
| code, but it evaporates when the error type was
| Infallible ]
| davidatbu wrote:
| This is exactly the kind of stuff I come on HN for. Thank
| you!
| ncmncm wrote:
| I presume it will still Oops...
| davidatbu wrote:
| I'm not sure what you mean by "it will still Oops".
| dogleash wrote:
| https://en.wikipedia.org/wiki/Linux_kernel_oops
| davidatbu wrote:
| Gotcha. Thanks!
| EugeneOZ wrote:
| Indeed, but it's much better than undefined behavior :)
| hiimkeks wrote:
| You can also use .get(idx), which gives you either Some(data)
| or None in case of out-of-bounds access.
| dcomp wrote:
| From what I understand a rust panic will just call BUG(). There
| is no support for unwinding as such.
|
| Most likely you would have to use .get() which returns an
| Option rather than [] array index which panics.
| ogoffart wrote:
| Exactly. A rust panic will call the panic_handler,
| implemented there: https://github.com/Rust-for-
| Linux/linux/blob/459035ab65c0ebb...
|
| So accessing an array out of bound will have a runtime check
| that will call the panic handler, and that panic handler
| calls BUG() which means kernel panic.
| tialaramex wrote:
| The Rust for Linux implementation converts a Rust panic into a
| Linux kernel BUG macro call. I believe this will expand to an
| invalid CPU instruction (at least on popular architectures),
| and if you're a kernel thread you die immediately with the
| kernel reporting the state where this happened. Obviously in
| some cases this is fatal and the report might only be seen via
| say, a serial port hooked up to a device in a test lab or
| whatever.
|
| So, it's not a kernel panic, but it's extremely bad, which
| seems appropriate because your code is definitely wrong. If
| you're not _sure_ whether the index is correct you can use
| get() or get_mut() to have an Option, which will be None if
| your index was wrong (or of course you could ask about the
| length of the array since Rust remembers how long arrays are).
| megous wrote:
| BUG() will panic the kernel.
|
| https://elixir.bootlin.com/linux/latest/source/include/asm-g.
| ..
|
| I guess that's quite drastic for a checked out of bound
| access, when there's no actual memory safety issue and the
| compiler can simply return an error from the function, or do
| something else less drastic.
| sealeck wrote:
| In which case you can call the get method (which returns an
| Option - i.e. either the value or null) rather than
| indexing and return an error value.
| couchand wrote:
| In Rust code, if you're not able to locally reason that an
| array index is valid, it should be written with .get() and
| the None case handled appropriately.
|
| It's impossible to claim there's "no actual memory safety
| issue" when a program's invariants have been broken: all
| bets are off at that point.
| AshamedCaptain wrote:
| > It's impossible to claim there's "no actual memory
| safety issue" when a program's invariants have been
| broken: all bets are off at that point.
|
| When the underlying _runtime's_ invariants have been
| broken, not when the program invariants have been broken.
| i.e. you can recover from almost everything save for a VM
| error in a VM language like Java, since there's no way
| for the program to mess up the VM's data structures in a
| way that they cannot be brought back to a defined state.
| couchand wrote:
| let innocent_var = operation_that_panics_but_returns_some
| thing_random_instead(); unsafe {
| do_something_assuming_validity(innocent_var); }
| megous wrote:
| Why allow indexed access at all if the compiler is
| emitting a conditional check anyway?
| connicpu wrote:
| Because in the common case you assume that, if your code
| is correct, all of your indexing will be in bounds, but
| for memory safety reasons we need a bug to be reported if
| memory safety would have been violated. So we allow
| direct indexing with a panic on out of bounds because
| it's the most ergonomic for that common case
| megous wrote:
| The bug is not just reported here, the whole computer
| shuts down and all your unsaved work gets lost. That's
| not very ergonomic either.
| vgel wrote:
| Some people have argued that indexed access is a wart,
| but it would be quite heavyweight to _always_ have to
| unwrap an option when accessing a known-good index:
| let foo = [0_u8, 1, 2]; foo[0].unwrap(); //
| really?
|
| Instead, indexing on arrays / vecs is (essentially) sugar
| for .get(index).unwrap(), if you don't want the unwrap
| behavior use get. This is very similar to Python, though
| Python throws an exception which obviously isn't
| available to Rust.
| throwaway894345 wrote:
| Would a kernel module be written as a normal Linux-targeting
| Rust program, or would it be more like a bare metal target
| with its own (user-provided) panic handler?
| loeg wrote:
| More like the latter. Kernel modules don't run in
| userspace.
| nicoburns wrote:
| Worth noting that one doesn't need to use raw array accessing
| in Rust nearly so much as in C because you have things like
| iterators and for..in loops that will ensure correct access.
|
| But I would assume it would be a kernel panic. It definitely
| won't be UB.
| pipeline_peak wrote:
| > and unlike in C++, we can write it and actually trust that it's
| correct
|
| Some of these people are so obnoxious...
| playdmc2 wrote:
| obnoxious about what? c++ is so brittle that a gentle breeze
| could cause a cacophony of soundness issues
| pipeline_peak wrote:
| > obnoxious about what?
|
| Vaguely descriptive attacks that serve no purpose other than
| religious wars. When I see statements like that I see another
| person who thinks it's simply fashionable to hate popular
| languages because they heard someone like Torvalds, Thompson,
| or Stallman say it. If you have a fundamental explanation
| like they do, then it's at least reasonable, disregarding the
| irrelevant context of a Linux kernel forum thread. But I
| rarely see that, instead I find mutter like "cpp is bloated,
| unstable, overcomplicated crap with ugly syntax, that's why I
| use rust, or nim, or any other pretty new thing that was
| conceived 35+ years later".
|
| For instance:
|
| > c++ is so brittle that a gentle breeze could cause a
| cacophony of soundness issues
|
| Okay, how so, anything about C++ or just C in general?
| tialaramex wrote:
| There's a design principle here, rather than just some sort
| of hubris.
|
| In languages as powerful as Rust and C++ you can express
| programs which fall into three categories, two of these
| aren't very interesting, the programs which are obviously
| valid, and the programs which are obviously nonsense. We
| know what to do with these programs, the former should
| result in emitting correct machine code, the latter should
| cause a diagnostic (error message). The problem is the last
| group, programs whose validity is difficult to discern. The
| bigger and more complicated your software the more likely
| it may end up in this last group.
|
| In C++ the third category are treated as valid. In the ISO
| standard this is achieved by having clauses which declare
| that in certain cases the program is "Ill-formed, no
| diagnostic required" which means the standard can't tell
| you what happens, but you won't necessarily get an error
| message, your compiler may spit out a program that does...
| something. Maybe it does what you expected, and maybe it
| doesn't, the standard asks nothing more.
|
| In Rust these are treated as invalid. If you try hard
| enough (or cheat and Google for one) you can write Rust
| programs which you can reason through why they should work
| but the compiler says no. You get an error message
| explaining why the compiler won't accept this program.
|
| Now, if Rust's compiler doesn't like your program, you can
| rewrite it so that it's valid. Once you do that, which is
| often very easy - you're definitely in that first category
| of correct programs, hooray. The program might well do
| something you didn't intend, the compiler isn't a mind
| reader and has no idea that you meant to write "Fnord" in
| that text output and "Ford" is a typo, but it's definitely
| a valid Rust program.
|
| In the C++ case we can't tell. Maybe our program is the
| ravings of a lunatic, the compiler isn't obliged to mention
| that and we are in blissful ignorance. This also provides
| little impetus for the standard's authors or compiler
| vendors to reduce the size of the third category of
| programs, after all they _seem_ to compile just fine.
|
| Obviously it'd be excellent in theory to completely
| eliminate the third category. Unfortunately Rice's Theorem
| says we cannot do that.
| pipeline_peak wrote:
| >In the C++ case we can't tell
|
| Inherently, or in the compiler implementations you've
| seen?
| tialaramex wrote:
| Inherently. Because of Rice's theorem you can't just
| ensure all the valid programs are correctly identified
| (basically you need to solve the halting problem to pull
| that off). But the ISO document doesn't allow you to do
| what Rust does and just reject some of the valid programs
| because you aren't sure.
|
| Now of course you _could_ build a compiler which rejects
| valid programs anyway and is thus not complying with the
| ISO document. Arguably some modes of popular C++
| compilers are exactly that today. But now we 're straying
| from the ISO C++ standard. And I'm pretty sure I didn't
| see a modern C++ compiler which reliably puts all the "No
| diagnostic required" stuff into the "We generate a
| diagnostic and reject the program" category even with a
| flag but if you know of one I'm interested.
| zacmps wrote:
| I'm not sure if there's a formal proof, but I believe its
| inherently undecidable.
|
| There's some discussion here
| https://stackoverflow.com/questions/7237963/a-c-
| implementati...
___________________________________________________________________
(page generated 2022-09-13 23:02 UTC)