https://lwn.net/SubscriberLink/885941/01fdc39df2ecc25f/ LWN.net Logo LWN .net News from the source LWN * Content + Weekly Edition + Archives + Search + Kernel + Security + Distributions + Events calendar + Unread comments + ------------------------------------------------------------- + LWN FAQ + Write for us User: [ ] Password: [ ] [Log in] | [Subscribe] | [Register] Subscribe / Log in / New account Moving the kernel to modern C [LWN subscriber-only content] Welcome to LWN.net Free trial subscription The following subscription-only Try LWN for free for 0 content has been made available to month: no payment or you by an LWN subscriber. Thousands credit card required. of subscribers depend on LWN for Activate your trial the best news from the Linux and subscription now and see free software communities. If you why thousands of readers enjoy this article, please consider subscribe to LWN.net. accepting the trial offer on the right. Thank you for visiting LWN.net! By Jonathan Corbet February 24, 2022 Despite its generally fast-moving nature, the kernel project relies on a number of old tools. While critics like to focus on the community's extensive use of email, a possibly more significant anachronism is the use of the 1989 version of the C language standard for kernel code -- a standard that was codified before the kernel project even began over 30 years ago. It is looking like that longstanding practice could be coming to an end as soon as the 5.18 kernel, which can be expected in May of this year. Linked-list concerns The discussion started with this patch series from Jakob Koschel, who is trying to prevent speculative-execution vulnerabilities tied to the kernel's linked-list primitives. The kernel makes extensive use of doubly-linked lists defined by struct list_head: struct list_head { struct list_head *next, *prev; }; This structure is normally embedded into some other structure; in this way, linked lists can be made with any structure type of interest. Along with the type, the kernel provides a vast array of functions and macros that can be used to traverse and manipulate linked lists. One of those is list_for_each_entry(), which is a macro masquerading as a sort of control structure. To see how this macro is used, imagine that the kernel included a structure like this: struct foo { int fooness; struct list_head list; }; The list member can be used to create a doubly-linked list of foo structures; a separate list_head structure is usually declared as the beginning of such a list; assume we have one called foo_list. Traversing this list is possible with code like: struct foo *iterator; list_for_each_entry(iterator, &foo_list, list) { do_something_with(iterator); } /* Should not use iterator here */ The list parameter tells the macro what the name of the list_head structure is within the foo structure. This loop will be executed once for each element in the list, with iterator pointing to that element. Koschel included a patch fixing a bug in the USB subsystem where the iterator passed to this macro was used after the exit from the macro, which is a dangerous thing to do. Depending on what happens within the list, the contents of that iterator could be something surprising, even in the absence of speculative execution. Koschel fixed the problem by reworking the code in question to stop using the iterator after the loop. The plot twists Linus Torvalds didn't much like the patch and didn't see how it related to speculative-execution vulnerabilities. After Koschel explained the situation further, though, Torvalds agreed that "this is just a regular bug, plain and simple" and said it should be fixed independently of the larger series. But then he wandered into the real source of the problem: that the iterator passed to the list-traversal macros must be declared in a scope outside of the loop itself: The whole reason this kind of non-speculative bug can happen is that we historically didn't have C99-style "declare variables in loops". So list_for_each_entry() - and all the other ones - fundamentally always leaks the last HEAD entry out of the loop, simply because we couldn't declare the iterator variable in the loop itself. If it were possible to write a list-traversal macro that could declare its own iterator, then that iterator would not be visible outside of the loop and this kind of problem would not arise. But, since the kernel is stuck on the C89 standard, declaring variables within the loop is not possible. Torvalds said that perhaps the time had come to look to moving to the C99 standard -- it is still over 20 years old, but is at least recent enough to allow block-level variable declarations. As he noted, this move hasn't been done in the past "because we had some odd problem with some ancient gcc versions that broke documented initializers". But, in the meantime, the kernel has moved its minimum GCC requirement to version 5.1, so perhaps those bugs are no longer relevant. Arnd Bergmann, who tends to keep a close eye on cross-architecture compiler issues, agreed that it should be possible for the kernel to move forward. Indeed, he suggested that it would be possible to go as far as the C11 standard (from 2011) while the change was being made, though he wasn't sure that C11 would bring anything new that would be useful to the kernel. It might even be possible to move to C17 or even the yet-unfinished C2x version of the language. That, however, has a downside in that it "would break gcc-5/6/7 support", and the kernel still supports those versions currently. Raising the minimum GCC version to 8.x would likely be more of a jump than the user community would be willing to accept at this point. Moving to C11 would not require changing the minimum GCC version, though, and thus might be more readily doable. Torvalds was in favor of that idea: "I really would love to finally move forward on this, considering that it's been brewing for many many years". After Bergmann confirmed that it should be possible to do so, Torvalds declared: "Ok, somebody please remind me, and let's just try this early in the 5.18 merge window". The 5.18 merge window is less than one month away, so this is a change that could happen in the near future. It is worth keeping in mind, though, that a lot of things can happen between the merge window and the 5.18 release. Moving to a new version of the language standard could reveal any number of surprises in obscure places in the kernel; it would not take many of those to cause the change to be reverted for now. But, if all goes well, the shift to C11 will happen in the next kernel release. Converting all of the users of list_for_each_entry() and variants (of which there are well over 15,000 in the kernel) to a new version that doesn't expose the internal iterator seems likely to take a little longer, though. Index entries for this article Kernel Build system Kernel GCC [Send a free link] Did you like this article? Please accept our trial subscription offer to be able to see more content like it and to participate in the discussion. ----------------------------------------- (Log in to post comments) Moving the kernel to modern C Posted Feb 24, 2022 15:12 UTC (Thu) by ballombe (subscriber, #9523) [ Link] Note that, if for some reason you need to stay with c89, you can always add a block around the for() statement to hold the loop variable. [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 15:51 UTC (Thu) by smurf (subscriber, #17840) [ Link] You'd need to do that to each caller, which is a *lot* of code churn. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 3:08 UTC (Fri) by kmeyer (subscriber, #50720) [ Link] You would just do it once, in the macro definition. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 6:24 UTC (Fri) by NYKevin (subscriber, #129325) [ Link] Based on the code shown in this article, I'm not clear on how you would actually do that. It *looks* like the macro expands to while (something) or for(something), and doesn't even have the "regular" set of braces, so you can't edit it to include an outer set of braces without modifying the call sites. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 7:08 UTC (Fri) by josh (subscriber, #17465) [Link ] I don't think you could with Linux's list macros. You could with Sparse's, which have pairs of macros invoked at the start and end of each loop (and which use two levels of braces). But C99 makes this easy. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 11:33 UTC (Fri) by 4m1rk (guest, #157085) [Link] The macro just creates the `for` expression. If you need to put the for expression inside a block then the macro needs to accept the body of the for too. Do C macros allow passing a code block? [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 16:22 UTC (Thu) by pbonzini ( supporter , # 60935) [Link] Is "documented initializers" Linus's typo for "designated initializers"? [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 17:36 UTC (Thu) by iabervon (subscriber, #722) [ Link] Instead of leaking a not-necessarily-valid pointer, couldn't the macro set it to NULL at the end? Actually, I'm surprised there isn't a standard trick for doing an assignment that will be an error unless the compiler eliminates it as dead code. [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 17:59 UTC (Thu) by Paf (subscriber, #91811) [Link ] Cost. Pretty significant cost in some cases for something that shouldn't even be necessary. [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 20:09 UTC (Thu) by nybble41 (subscriber, #55106) [Link] It should be cost-free in any case where the iteration variable isn't accessed after the loop, since the compiler would eliminate the dead store. The code change is also fairly trivial: just edit the condition from "&pos->member != (head)" to "(&pos->member != (head)) || ((pos = NULL))". Unfortunately this alone doesn't handle loops which exit early due to "break" or "goto". The "goto" case is unavoidable, but the "break" case can be dealt with by wrapping the macro in a second, trivial loop as shown in this example[0]. Note that the generated code (for gcc 5.1 with -O2) is *identical* between the version with the extra loop (traverse1) and the original version which does not set the iterator to NULL after the loop (traverse2). The initialization of the iterator to the flag state (-1), the condition for the outer loop, and the store of NULL to the iterator after the loop are all successfully eliminated. [0] https://godbolt.org/z/4obYManzc [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 20:48 UTC (Thu) by iabervon (subscriber, #722) [ Link] It might work to have: extern unsigned long list_iterator_live_after_loop; and "|| ((pos = (void *) list_iterator_live_after_loop), 0)" I didn't try changing the kernel macro that way, but my little test code doesn't link if the iterator is used after the loop, but does link and work if it's not used. As I recall, the kernel is already using that sort of trick to use compiler optimization to remove an error message only if the compiler can disprove it. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 22:20 UTC (Fri) by NYKevin (subscriber, #129325) [Link] Unfortunately, this is probably UB: https://en.cppreference.com/w/c/ language/extern > The entire program may have zero or one external definition of every identifier with external linkage. > > If an identifier with external linkage is used in any expression other than a non-VLA, (since C99) sizeof, or _Alignof (since C11), there must be one and only one external definition for that identifier somewhere in the entire program. There's no exception for short-circuit operators. If you use it at compile time, for anything other than sizeof, then it has to exist (have storage allocated somewhere). [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 1:19 UTC (Fri) by ianloic (subscriber, #54050) [ Link] It's kind of fascinating how small the cost is, even when using the pointer afterwards: https://godbolt.org/z/cbv4fqan3 [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 20:26 UTC (Thu) by iabervon (subscriber, #722) [ Link] Oh, I meant to imply that the compiler would eliminate all of those writes except for ones that expose bugs, but then I got side-tracked by wondering if you could make the kernel not even link unless the compiler eliminated the write. Anyway, it wouldn't affect the generated code unless the compiler can't tell the code is correct. [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 22:12 UTC (Thu) by Paf (subscriber, #91811) [Link ] Very good point. God I'd sure love to get to a newer C standard though... [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 8:54 UTC (Fri) by ncm (subscriber, #165) [Link] The smarter move would be to start compiling the kernel with a C++ compiler. A weenier step would be to accept source files with a ".cc" suffix and build those with a C++ compiler. There would, in any case, be no need to step outside Gcc, where in fact that was done long ago, with no disruption, but with massive benefits. Anybody spooked about C++ should understand that Gcc and Clang are both coded in C++, whatever the language you compile on them. Similarly, anybody spooked by C++ "hidden code" should understand that Rust does literally all of the things they are spooked by; and all of its power comes from that. Staying on ancient EOL'd language Standards does nobody any good. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 9:02 UTC (Fri) by mpr22 (subscriber, #60784) [ Link] I suspect the people most likely to be spooked (for whatever value of "spooked") by any "hidden code" aspect of C++ are equally likely to be similarly spooked by the similar aspects of Rust. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 9:33 UTC (Fri) by ncm (subscriber, #165) [Link] If they understood programming better, they would be more spooked by their C compiler failing to emit such code where, without, the code they wrote is buggy. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 9:30 UTC (Fri) by Wol (subscriber, #4433) [Link] A weenier step accepting .cc files? The problem with that is "all of C++" is a security / close-to-the-metal nightmare, and the definition of what is the acceptable subset varies with who you talk to. A further problem is the size / speed of the code. Yes C++ is *mostly* pretty good, but I suspect the compiler devs will barf on that word "mostly". To what extent does kernel C currently drop out of C into assembler, and to what extent will C++ make that worse? No I don't actually know the answers, I'm just predicting the devs' reactions. Cheers, Wol [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 21:24 UTC (Fri) by ncm (subscriber, #165) [Link] You demonstrate you know neither the answers, nor the questions, and are simply making things up as you go along. Just stop. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 14:52 UTC (Fri) by jd (subscriber, #26381) [Link] If we aren't going to go with ancient, then move to D. It's much, much newer and doesn't carry anything like the risks or overheads of C++. (Not that I'd recommend it, for other reasons, but it helps illustrate the age of the standard shouldn't matter as much as the quality of the result.) Is there markup for any of the static checkers beloved by kernel developers that could be used to improve the quality of the results? (And when was the last time Coverity checked the kernel?) There must be plenty that could be done to improve the kernel code without a drastic change of language. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 18:04 UTC (Fri) by davej (subscriber, #354) [Link ] > And when was the last time Coverity checked the kernel 99% of the time, the answer to this question is the same as "when did Linus last cut an -rc/final". I usually kick off a run the same day, failing that the following morning. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 21:33 UTC (Fri) by ncm (subscriber, #165) [Link] I doubt Coverity works on D code. It does work on C++. "Risks and overheads" is wholesale speculation, unwelcome here. [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 21:11 UTC (Thu) by abatters ( supporter , # 6932) [Link] It would break code that does this: list_for_each_entry(iterator, &foo_list, list) { if (do_something_with(iterator)) { break; } } if (list_entry_is_head(iterator, &foo_list, list)) { // iteration finished } else { do_something_else_with(iterator); } All this "compare to head" nonsense is why I prefer regular NULL-terminated linked lists to the kernel's circular linked lists. Insert/delete may take more instructions but iteration is much easier. [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 21:55 UTC (Thu) by nybble41 (subscriber, #55106) [Link] Yes, and you also have macros like for_each_list_entry_continue() which depend on the value being left in the iterator. All of these would also break if the macro was changed to declare the iterator inside the `for` statement, C99-style. One way to work around the problem in your example would be to move the condition inside the loop, like this: list_for_each_entry(iterator, &foo_list, list) { // ... if (do_something_with(iterator)) { do_something_else_with(iterator); break; } // ... if (&iterator->list == &foo_list) { // this is the last entry; iteration finished } } The compiler should be smart enough to avoid checking the end condition twice in each iteration. Of course this becomes much less convenient if there is more than one break statement. [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 17:46 UTC (Thu) by flussence (subscriber, #85566) [Link] My thought is that it's silly to require compatibility with standards old enough that even software implementing their newer versions are falling out of long-term support. There are other cases where arguably there's a risk of causing a flag day, but moving off of C89 isn't one of them. [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 18:47 UTC (Thu) by adobriyan (subscriber, #30858) [Link] Yay! Don't forget this one too: # warn about C99 declaration after statement KBUILD_CFLAGS += -Wdeclaration-after-statement [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 19:37 UTC (Thu) by zuzzurro (subscriber, #61118) [Link] If the plan is to make this move at the beginning of the next cycle, shouldn't the -next kernel adopt it right now? [Reply to this comment] Moving the kernel to modern C Posted Feb 24, 2022 20:30 UTC (Thu) by marcH (subscriber, #57642) [ Link] Yes please, finally! Combined declarations and initializations like every other programming language. More 'const' and fewer "this variable 'may' be used uninitialized" guessing/silliness. No more reverse Christmas trees. In even more advanced languages 'const' is the default but let's not get carried away; too much maths that could scare hardware engineers emotionally attached to their registers. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 9:29 UTC (Fri) by geert (subscriber, #98403) [ Link] > In even more advanced languages 'const' is the default "const" is the default for /var/iables?!? [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 9:37 UTC (Fri) by ncm (subscriber, #165) [Link] Let us not confuse the name with the thing named. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 9:39 UTC (Fri) by wtarreau (subscriber, #51152) [ Link] > Combined declarations and initializations like every other programming language. Please no! That's the most horrible thing I hate in modern C. Normally when reviewing code and looking for a variable, you just need to glance at the top of each upper level opening brace and nothing more. With those insane declarations after statement, you work like in bash: you have to read *ALL* lines above where you are, hoping you didn't miss the right one. This serves absolutely no purpose, and only has for effect to complicate code reviews and ease introduction of new bugs. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 11:14 UTC (Fri) by Wol (subscriber, #4433) [Link] You're confusing "use" and "initialise". Don't allow mixing USE and declaration. But DO allow the *compiler* to set the initial value. Cheers, Wol [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 15:13 UTC (Fri) by wtarreau (subscriber, #51152) [Link] I'm not sure we're speaking about the same thing. I'm speaking about not making this monstrosity possible, where I'd say "good luck" for figuring the type of "i" depending on the line you're reading, and its bounds: #include #include int blah(long x, int j) { long i = x ? x : -1; int k = i; for (int i = 1; i < j; i++) { k += i * 2; char i = (k & 1) ? 'O' : 'E'; int pid = getpid(); printf("i=%d j=%d pid=%d\n", i, j, pid); } return k; } PS: sorry for the formatting, I didn't find how to make a code block. [Reply to this comment] Moving the kernel to modern C Posted Feb 25, 2022 16:16 UTC (Fri) by farnz (subscriber, #17727) [ Link] Making a code block on LWN needs two tags in HTML formatting: 
to indicate that formatting matters, and  to indicate that you
want monospaced fonts. Below is 
 followed by your code (with
indentation added by my brain), followed by 
 - I've also
had to escape special characters with HTML escapes (but it's a simple
matter to write code to do this for you).


#include 
#include 

int blah(long x, int j)
{
    long i = x ? x : -1;
    int k = i;

    for (int i = 1; i < j; i++) {
        k += i * 2;
        char i = (k & 1) ? 'O' : 'E';
        int pid = getpid();
        printf("i=%d j=%d pid=%d\n", i, j, pid);
    }
    return k;
}


[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 16:51 UTC (Fri) by Wol (subscriber, #4433) [Link]

I thought that was allowed in ancient C ...

Unless you mean actually declaring inside the "if" statement ... but
I thought declaring after a { was permitted anywhere. I dunno, it's
ages since I've programmed C in anger.

But it would be nice to say you can ONLY declare after a {, but that
includes things like "int i = 1". You shouldn't be able to do things
like "int i; i=1; int j; j=2;", though.

Cheers,
Wol
[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 18:44 UTC (Fri) by nybble41 (subscriber, #55106)
[Link]

In C89 and GNU89 declarations must occur before statements within
each block. C89 additional requires initializers to be compiler-time
constants. However, it's not as if this equivalent GNU89 code is any
easier to follow:

#include 
#include 

int blah(long x, int j)
{
    long i = x ? x : -1;
    int k = i;
    {
        int i;
        for (i = 1; i < j; i++) {
            k += i * 2;
            {
                char i = (k & 1) ? 'O' : 'E';
                {
                    int pid = getpid();
                    printf("i=%d j=%d pid=%d\n", i, j, pid);
                }
            }
        }
    }
    return k;
}

The real lessons here are "use meaningful names" and "avoid
shadowing".

[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 11:28 UTC (Fri) by jem (subscriber, #24231) [Link
]

You can add this to the list of things you hate about Rust, too. In
Rust you can even do this:

fn main() {
    let i = 0;
    println!("{}", i);    // Prints 0.

    let i = i+1;
    println!("{}", i);    // Prints 1.
}

Note that the i variables are immutable ("const") and there are two
of them. The second let introduces a new variable which is
initialized with the value i+1, where i refers to the first variable.

[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 19:30 UTC (Fri) by ballombe (subscriber, #9523) [
Link]

> Please no! That's the most horrible thing I hate in modern C.

I am glad to see I am not alone.

Usually, when it is used, it is a sign the function is too large and
should be split, which would resolve the scoping issue.
Of course since C17 still does not support gnu89 nested functions,
sometime splitting the function require passing an inordinate amount
of parameters.
[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 19:53 UTC (Fri) by marcH (subscriber, #57642) [
Link]

> > Please no! That's the most horrible thing I hate in modern C.

You meant: in _any_ vaguely modern language. What rock have you been
living under?

> Usually, when it is used, it is a sign the function is too large
and should be split, which would resolve the scoping issue.

Exactly. If you can't find a variable declaration then the function
is simply too long.

Of course with https://en.wikipedia.org/wiki/Type_inference (1958)
you don't even need to declarations _at all_ but again, let's not get
carried away and scare ancient species...

[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 20:53 UTC (Fri) by mpr22 (subscriber, #60784) [
Link]

If I can't find the variable declaration, I'm using generic or weakly
language-aware editing tools instead of strongly language-aware
editing tools.
[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 20:31 UTC (Fri) by abatters ( supporter , #
6932) [Link]

> gnu89 nested functions

Which require your entire program to have an executable stack.
[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 20:55 UTC (Fri) by pbonzini ( supporter , #
60935) [Link]

They only do if they are used as function pointers.
[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 22:12 UTC (Fri) by ncm (subscriber, #165) [Link]

Of course in C++ and Rust you have function literals. In C++,


  auto f = [](auto a, auto b) { return a + b; };
  assert(f(3, 4) == 7);
  assert(f(3.25, 3.75) == 7.0);
  assert(f("3"s, "4"s) == "34");


Rust lambdas might be less versatile.
[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 4:31 UTC (Fri) by pabs (subscriber, #43278) [Link
]

I wonder what the implications of this are for Bootstrappable Builds.
I guess they will just need a longer GCC versions bridge before
getting to the step of building Linux.

https://bootstrappable.org/
[Reply to this comment]
Moving the kernel to modern C

Posted Feb 25, 2022 10:49 UTC (Fri) by georgm (subscriber, #19574) [
Link]

As written in the article, the minimum supported gcc version (5.1)
already supports C11, so there shouldn't be any change here.
[Reply to this comment]

                  Copyright (c) 2022, Eklektix, Inc.
   Comments and public postings are copyrighted by their creators.
          Linux is a registered trademark of Linus Torvalds