[HN Gopher] Extreme include discipline for C++ code
___________________________________________________________________
Extreme include discipline for C++ code
Author : todsacerdoti
Score : 48 points
Date : 2022-05-13 09:08 UTC (13 hours ago)
(HTM) web link (blog.kowalczyk.info)
(TXT) w3m dump (blog.kowalczyk.info)
| mc4ndr3 wrote:
| #pragma once
| account42 wrote:
| Not standardizing #pragma once because of edge cases with with
| network and other esoteric filesystems when pretty much every
| C++ compiler implements #pragma once is really peak C++
| committee bs. And it's not even like the C++ standard doesn't
| leave other edge case behavior undefined or implementation
| defined...
| ncmncm wrote:
| Fortunately there is no need to standardize it. Everybody
| implements it, and all the same way.
| krapp wrote:
| It may be widely supported but it's also nonstandard. You also
| have to use include guards anyway, so it's best to avoid pragma
| once altogether.
|
| And as the article mentions, that doesn't always solve the
| problem.
| jcelerier wrote:
| > It may be widely supported but it's also nonstandard.
|
| by that logic no one should use python, ruby or rust because
| they don't have an iso or ecma standard either
| steveklabnik wrote:
| (Ruby does have an ISO standard but it's weird and nobody
| uses it.)
| jcelerier wrote:
| oh, TIL ! why wouldn't anyone use it ?
| steveklabnik wrote:
| For reference: https://www.iso.org/standard/59579.html
|
| This work happened in 2009. It was originally based on
| Ruby 1.8.7, the final draft was completed in 2010, and
| didn't cover the full standard library at the time. Took
| two more years to get through ISO.
|
| If you don't have an encyclopedic remembering of Ruby
| version releases, when that draft was complete, the
| current released version of Ruby was 1.9.2. Ruby
| underwent huge changes between 1.8.7 and 1.9.2; many
| other languages would have characterized it as a major
| version change at the time; Ruby didn't follow semver
| back then.
|
| So basically, it was already out of date well before it
| was final, and never was updated. So it's really
| irrelevant today.
|
| Why did this even happen? Well, supposedly there are some
| requirements for Japan government work that require a
| spec, and so a spec was produced. That's my recollection,
| anyway.
| krapp wrote:
| I wasn't making a general statement about software
| standards but one specific to C/C++ preprocessor
| directives.
| jcelerier wrote:
| but "standards" is a general statement in itself ; you
| can't dissociate the word from the whole ISO process in
| this context
| account42 wrote:
| Name one C++ compiler that keeps up with C++ standards but
| does not provide #pragma once.
| krapp wrote:
| Every compiler must support include guards, but only the
| ones that choose to support pragma once. It should be
| obvious, between the two, which the more stable option.
| ncmncm wrote:
| Except, _all_ choose to. And, must, if they want to be
| compatible with code that all must compile.
| tlb wrote:
| I've been living on the edge using only #pragma once for
| several years and haven't regretted it. According to this
| support table, it's pretty widely supported these days:
| https://en.wikipedia.org/wiki/Pragma_once#Portability
| casualrandomcom wrote:
| Only to underline this common fallacy: "Name an economically
| successful communist country."
|
| The purpose of communism has never been that of being
| "economically successful".
|
| If you want to dismiss communism as a failure, first off, you
| should not measure money, and not even wealth, but simply
| happiness. Second, you should measure equality, and, finally, you
| should measure the happiness of the least happy, not that of the
| most happy.
| lordnacho wrote:
| So that's what the C in C++ stands for!
| casualrandomcom wrote:
| mmmmh, if your criterion is the happiness of the least
| happy... I guess C++ is a total failure!
| cassepipe wrote:
| While I agree with you that economic success does not have to
| be the utmost goal of a state because that's a goal that is
| easily too narrowly defined ("Growth"),
|
| 1. The doctrine those states called "Marxism" or "Marxism-
| Something" very much emphasized well-being as material well-
| being stemming from economic success.
|
| 2. Self called Socialist states, generally referred to as
| Communist states as they were dominated by an all powerful
| "communist" party, have not been in general successful in terms
| of general happiness, although they argued that they at least
| brought an industrialized level of economic prosperity whereas
| capitalism would have left those places stranded in misery.
| Wether that is true or it was worth the price is a question for
| historians to debate on.
|
| To summarize : Humans don't live by bread alone, and I am not
| even sure the bread was great there.
| woodrowbarlow wrote:
| so this discipline speeds up compilation, but makes the code
| "annoying" to write and requires "eternal vigilance".
|
| fast compiles are nice but to trade away your code's hackability
| seems infinitely counter-productive to me. won't that have an
| immense negative impact on total development time?
| kevindong wrote:
| I think the author is asserting that trading away the
| hackability is worth the faster compile times because faster
| compiles lets you iterate faster and therefore increases
| hackability (i.e. put in minutes of effort once and get minutes
| of benefits per compile in the future).
| yolo69420 wrote:
| if you work on a non-trivial c++ code base, being vigilant
| about compile times can literally make the difference between
| compiling your codebase from scratch in a minute vs. compiling
| it in an hour or more. 0% exaggeration here. With similar
| relative differences for incremental compiles. Nothing kills a
| code-base's hackability like waiting around to be able to
| actually run code you wrote.
|
| Include code style is also just the tip of the iceberg. This
| also includes not using many c++ features that blow compile
| times up unless the gain from using them is so big that you eat
| the compile time (like containers would be an obvious example).
| ncmncm wrote:
| The good news is that abandoning this "eternal vigilance" has
| no effect on compilation time, but speeds up development by the
| exact degree that you don't waste that time and attention
| anymore.
| jcelerier wrote:
| > Don't get me wrong: the price of minimizing compilation times
| is eternal vigilance.
|
| I don't follow any such rules for my own codebase and here are
| how my compilation times look like for three representative
| files: https://streamable.com/pboot1
|
| - first and second one have tons of Qt / std / boost:: / etc...
| stuff
|
| - third one includes opencv.hpp (which includes most opencv
| libraries)
|
| I have a hard time seeing the benefit of putting more effort in
| this considering that the edit -> build -> run cycles takes
| around a second or two in all cases, and I don't even use all the
| clang PCH options available in recent clang versions.
|
| So no,
|
| > the price of minimizing compilation times is
|
| using your damn tools correctly.
| ncmncm wrote:
| This.
|
| For decades, Bloomberg development, and people so unfortunate
| as to follow them (because of silly book from the early '90s)
| adhered to an idiotic convention that wrapped every #include
| directive in an #ifdef block. Of course, all compilers, also
| for decades, recognized when they had already seen an include
| file that itself was wrapped in its own #ifdef block, and
| skipped it. Later this same behavior was simplified to _#pragma
| once_ , obviating the need to invent unique preprocessor names
| for all headers, which on occasion collided or were misspelled,
| with amusing results.
|
| There are no remaining C++ compilers in use, outside of sad
| pre-Standard backwaters, that do not implement _#pragma once_.
|
| And anyway, the proper solution if you are worried about
| compilation time is to switch to C++20 modules.
| brundolf wrote:
| > and bang! You just included and parsed bar.h twice
|
| I'm sure I'm missing something, but I'm curious why the compiler
| can't parse the header just once and keep it in memory (with
| further dependencies forming a graph of ASTs referencing each
| other), instead of naively combining all the raw text every time
| before it can parse the entire combined file?
| mehrdada wrote:
| You can do some tricks but the basic issue is that the
| "environment" in which you interpret the preprocessor
| directives for a ".h" file may change across different contexts
| resulting in a potentially distinct AST: simplest example is
| the common #ifndef guard (evaluates the true branch in the
| first load, false in subsequent ones).
| brundolf wrote:
| Ach. Fun
| db48x wrote:
| Plus the default is to launch a different instance of the
| compiler for every .c/.cpp file, so that state has to be
| written out to disk and reread by the next one.
|
| If I ever did another C or C++ project (which would
| probably only be at gun-point), I would go the opposite
| route and have only a single compilation unit. I would have
| a single primary source file that included all of the
| others, and none of the others would include anything. Then
| I could run the compiler a single time on just that one
| file and it would be as fast and as simple as possible.
| Well, I don't know of any C or C++ compilers that can
| spread their work over multiple cpus (since we usually just
| run multiple instances simultaneously with make -j), but
| other than that it would be as fast as possible.
| jcelerier wrote:
| If you use cmake it can do exactly this for you
| automatically : just pass -DCMAKE_UNITY_BUILD=1
| grogers wrote:
| I seriously doubt the compiler deduplicating header files that
| specify "pragma once" is what is slow about parsing all those
| header files. I really think the main issue is that all the
| different translation units have to reparse the web of headers,
| which this doesn't solve since you still need to manually figure
| out the correct order and include them all in each source file.
| Seems like a lot of headache for no gain.
|
| Holding things by pointer on the other hand actually does let you
| prevent the includes since you can just forward declare in the
| header, but it's an indirection :-/
| DavidVoid wrote:
| > Holding things by pointer on the other hand actually does let
| you prevent the includes since you can just forward declare in
| the header, but it's an indirection :-/
|
| And it means you now need to deal with raw pointers, which
| often isn't optimal or preferred (over say references for
| example).
|
| At least we'll get modules soon. That should reduce the need
| for some of these workarounds that are really mainly used to
| reduce compile times, and not to improve the code in any other
| meaningful way.
| lfowles wrote:
| Sure, but you can also forward declare references!
| jll29 wrote:
| There are issues with type safety when using raw pointers
| (e.g. void _) over alternatives.
|
| Reminds me of the good ol' days when Modula-2 had _.def
| (definition) and *.mod2 (implementation) files, and you could
| compile the definition files alone to get binaries of the
| interface files that you could write and compile against in a
| typesafe way before the interfaces were even implemented
|
| And as far as I recall the decoupling also made things
| compile fast (e.g. with Applications Systems Heidelberg's
| Modula-2 on the Atari ST 520+).
| dzdt wrote:
| The gain is mentioned in the article: without such discipline
| sometimes when you #include button.h (what you expect to be
| innocuous header file for a desired button functionality) you
| accidently inlcude nuclearpowerplant.h (a giant monstorisity of
| recursively included functionality none of which you actually
| need).
|
| It may be possible to avoid that worst-case situation without
| such strict discipline, but with the no-recursive-include rule
| for sure it won't happen.
| TrianguloY wrote:
| If including button.h ends including nuclearpowerplant.h it's
| because it's a dependency, so with the article method you
| will end up including nuclearpowerplant too, otherwise it
| won't compile.
|
| Of course this assumes that headers only include what they
| really need. If a header includes something it shouldn't
| (maybe an old dependency that was forgotten) the issue is old
| dependencies, and the fix is to remove it from the header
| once. With the article method you will need to remove it from
| every file that includes button (except maybe it is also used
| by another dependency and compile will fail).
|
| Edit: I've realized the problem. If A includes B because it
| requires it, and C requires A and B, C will probably only
| include A and it will compile. Later if A is refactored and
| no longer requires B, removing it will break C. The solution
| is to always include what you use, even if it it's already
| included as dependency, as explained by another comment.
| account42 wrote:
| I think TFAs approach actually makes things worse over time as
| now you need even more discipline to clean out no longer needed
| #inlcues from _all_ source files using a header when that
| header removes a dependency.
|
| IME the best tools to solve the duplicated parsing (and
| compilation for templates) is to use IWYU [0] to cut down on
| unneeded includes and unity/jumbo builds to combine multiple
| translation units so common headers only need to be parsed
| once.
|
| [0] https://include-what-you-use.org/
| flohofwoe wrote:
| IME the main problem of allowing to include headers in headers
| is that this quickly grows out of control, and after a while
| headers will include stuff that's not actually needed (usually,
| no longer needed).
|
| Then you get quickly into a situation where a header includes
| another header that's no longer needed by itself, but is used
| by another header in the same 'include tree', which in turn
| _doesn 't_ include the required header itself. Gets very messy
| very quickly.
|
| It's much easier to notice and fix such situations in the top-
| level implementation file, and another advantage is that the
| "dependency complexity" of an implementation file is visible at
| a glance by looking at the include list at the top of the file.
|
| TL;DR: It's not primarily for compilation speed, but for
| 'header hygiene' (but of course this will also eventually help
| with compilation speed as the project grows).
|
| Another well known (at least among game devs) proponent of the
| the same idea is Our Machinery (granted, the whole idea makes a
| lot more sense in C than in C++, because in C declaration and
| implementation is usually much stricter separated into header
| and source files than in C++)
|
| https://ourmachinery.com/post/physical-design/
| account42 wrote:
| The tool for deal with this is https://include-what-you-
| use.org/
|
| The idea is that each file (source or header) should include
| exactly those headers from which it uses things. In practice,
| it gets a bit more complicated has you don't want to include
| internal implementation headers and sometimes the same thing
| does not even have a canonical public header but IWUY does
| allow you to configure all that to your liking.
| marco_craveiro wrote:
| Great point. And this will soon be available directly from
| Clang tools such as clangd, which will make life much
| easier.
| tadeegan wrote:
| > It's a known problem so we mitigate with #ifdef guards, #pragma
| one etc. but in my experience those band-aids don't solve the
| problem.
|
| Errr, why?
| deutschepost wrote:
| The "problem" at this part in the text is that the headers
| still have to get parsed multiple times by the preprocessor.
| With his approach this would not be necessary.
|
| That said, I don't think that is that big of a problem compared
| to final compile time reduction.
|
| EDIT: Disregard that, the preprocessor remembers the states of
| include guards. So there is literally no point.[1]
|
| [1] https://gcc.gnu.org/onlinedocs/cpp/Once-Only-Headers.html
| ur-whale wrote:
| Yeah, came here to say the same thing.
|
| Plus:
|
| > I don't think I've ever seen any C++ code bases that follows
| this rule.
|
| It's not particularly useful to do this given that the moment
| you include anything from /usr/include and STL that rule will
| get broken from under you a 100 times.
| cassepipe wrote:
| See also the almost unknown Makeheaders program that is part of
| the Fossil project that will by default generate one .h file per
| .c with the same name as the C file. Only headers that needs be
| modified are touched so that when a header file is updated only
| the corresponding .c file is recompiled with no need for
| gcc/Makefile wizardry with -M flags and -include directives
|
| Of course you then find the same function prototypes repeated
| over and over in many headers but I don't mind the boilerplate
| when it is automated and I don't have to actually write any of
| it.
|
| The only problem is that it is bare bones and only supports
| outputting the header file in the same directory than the c
| files, that's mildly annoying but that could easily be modified
| and the program is contained in a single c file.
|
| https://fossil-scm.org/home/doc/trunk/tools/makeheaders.html
|
| https://www.hwaci.com/sw/mkhdr/
|
| https://fossil-scm.org/home/file/tools/makeheaders.c
| duped wrote:
| I don't believe performance related blog posts without
| benchmarks, and neither should you!
| JoeAltmaier wrote:
| This seems questionable. The author suggests 'faulting in' header
| files to trace dependencies, in pursuit of the iffy goal of
| including exactly what your compilation unit needs and no more.
|
| So you're manually doing what the compiler and code pattern used
| to do for you. And creating dependencies in your source code to
| files that may become obsolete (include bar.h because foo.h needs
| it; then foo.h changes and now bar.h is no longer needed yet you
| still include it)
|
| I have a different code pattern: header files include everything
| they need and no more; I include just the API's I use in my
| source files. I prefer this pattern by leaps and bounds over the
| OP's recommended technique. Because the source and tools manage
| themselves.
| Shadonototra wrote:
| one of the reason i moved away from C/C++ and went full Zig/D
| deutschepost wrote:
| This discussion is the reason I hope that C++20 modules gain
| traction. It is nice to improve myself a a developer (and have a
| look at different viewpoints), but in the end I don't want to
| have a tradeoff between code performance and readability.
|
| I would convert my entire codebase to modules in a second if the
| CMake Support was nicer.
| cassepipe wrote:
| I really suggest to check out the fast and friendly Xmake that
| do support C20++ modules
|
| https://xmake.io/#/ https://tboox.org/2021/10/30/xmake-
| update-v2.5.9/
| ncmncm wrote:
| There is no danger of C++20 modules failing to "gain traction".
| They are already in the Standard. All that is left is to define
| Standard module names for the Standard library components.
|
| Most likely that will be, simply, "import std" in almost all
| cases, with no reason for finer granularity.
|
| Of course we will need to update our non-Standard libraries to
| get the benefits there. That will happen fast once more
| compilers finish implementing C++20 features.
| deutschepost wrote:
| For me at least, it seems like development on the build
| system side is very slow. The relevant ticket in the CMake
| Repository[1] is more than three years old for a critical
| functionality. There are efforts to make an open interface
| between Compiler and Build System[2] but these are non-
| standard and for now it seems like there is not "one good
| way" to do modules. For example it is possible to do modules
| with the CMake MSVC Generator[3] but I can't use Ninja when
| working with MSVC.
|
| [1]https://gitlab.kitware.com/cmake/cmake/-/issues/18355
|
| [2]https://www.open-
| std.org/jtc1/sc22/wg21/docs/papers/2021/p16...
|
| [3]https://devblogs.microsoft.com/cppblog/cpp20-modules-in-
| cmak...
| duped wrote:
| One compiler implements module support to the point it's
| useful, and that compiler targets one operating system.
|
| I don't think it's hard to fathom that C++ modules might not
| gain traction when there is little indication of adoption in
| compilers, let alone code based that could benefit them.
| 29athrowaway wrote:
| C++ now supports modules.
| Dwedit wrote:
| If a header file can't include another header file, then how do
| you #include <windows.h>?
| svnpenn wrote:
| No mention of include what you use?
|
| https://github.com/include-what-you-use/include-what-you-use
| inetknght wrote:
| I'm currently on a code-formatting-and-linting spree on my
| team. include-what-you-use is definitely something I'm adding.
|
| I'm currently having to fight against the tide of the very-
| broken way that the codebase was built ... very much _not_ as a
| modern or even old school project let alone a Docker-based
| project. So tons of things get included everywhere and some
| things are even compiled multiple times. It's a bit of a
| nightmare.
|
| Putting tooling into CI will help prevent problems from showing
| up ... but the tooling wasn't there from the start so most of
| the project needs to be refactored so that CI doesn't
| immediately turn red. And that's the biggest headache tbqh
| account42 wrote:
| Adding this kind of thing to existing large codebases is
| always challenging. If you can, add it to CI but only fail if
| there are _new_ warnings. Then things will get better over
| time until fixing the remaining cases becomes feasible.
| inetknght wrote:
| > _If you can, add it to CI but only fail if there are new
| warnings._
|
| That's a great idea. But I'm not sure how to easily do
| that. Getting CI to fail if clang-format reports a warning
| is easy enough. But... you suggest that I should store all
| of the existing warnings somewhere and only report new
| warnings? That's a lot (!) more effort unless you know an
| easy way
| floor_ wrote:
| This person needs extreme ssl cert discipline.
|
| "blog.kowalczyk.info uses security technology that is outdated
| and vulnerable to attack. An attacker could easily reveal
| information which you thought to be safe. The website
| administrator will need to fix the server first before you can
| visit the site.
|
| Error code: NS_ERROR_NET_INADEQUATE_SECURITY"
| kjksf wrote:
| Tell that to cloudflare, snarky person
|
| I don't know what tool you're using to determine this, but
| it'll say the same for half of the internet as my site is
| hosted on render.com and proxied via cloudflare.
|
| Also, everything on this website is open source
| https://github.com/kjk/blog
|
| You can read all the "secrets" you want without even visiting
| the site.
| floor_ wrote:
| My tool is Firefox ver. 100.00. I straight up can't visit the
| site.
| ncmncm wrote:
| Oddly, mine is 100 too. I see a flat file with only
| 1995-grade HTML formatting.
| Kranar wrote:
| >Many ideas seem great on paper but fail in practice.
|
| Yes, including the one posted by the author, sounds great on
| paper but in practice provides next to no benefit. Notice how the
| author has provided absolutely zero evidence to justify their
| claim.
|
| >You just included and parsed bar.h twice.
|
| No you didn't, because header files have header guards of the
| form #ifndef HEADER_GUARD/#define HEADER_GUARD or #pragma once
| which means that header files are only parsed the first time they
| are included.
|
| >This makes me either a madman or a genius.
|
| It makes you neither, you're just following something because
| someone with a big name who you respect said it, so you to do it
| without actually verifying for yourself whether it's true.
|
| >Name an economically successful communist country.
|
| China.
| phkahler wrote:
| >> No you didn't, because header files have header guards of
| the form #ifndef HEADER_GUARD/#define HEADER_GUARD or #pragma
| once which means that header files are only parsed the first
| time they are included.
|
| The header file still as to be read from disk ignoring
| everything until #end and then continue. This is probably not
| much of an issue these days with SSDs and even HDDs with
| caches.
|
| I think the rule can still be a good thing by making you aware
| of what the dependencies actually are, which can be an
| indicator of excess complexity or poorly defined interfaces.
| Kranar wrote:
| Loading the same file from disk twice (which will be cached
| in memory) and parsing a file are not the same things.
| Furthermore almost every C++ compiler I'm aware of, including
| icc, gcc, clang, msvc, has optimizations to special case
| header guards.
| ncmncm wrote:
| _Every_ C++ compiler still in use will not even try to open
| a header-guarded or _#pragma once_ file it has already
| seen.
|
| Anybody trying to optimize on top of that is engaged in
| foolish superstition.
|
| (If you are still using Sun's pre-Standard compiler, you
| are not reading this. I think even Bloomberg has abandoned
| that.)
| kevin_thibedeau wrote:
| It doesn't have to be read twice if the preprocessor
| recognizes and keeps track of include guards. GCC does this.
| db48x wrote:
| > Notice how the author has provided absolutely zero evidence
| to justify their claim.
|
| That is true and unfortunate, but it is easy to find evidence
| that cleaning up header files will decrease compilation times:
| https://lore.kernel.org/lkml/YdIfz+LMewetSaEB@gmail.com/T/
|
| >> Name an economically successful communist country. > China.
|
| Except that China's success has mostly come by instituting a
| "Special Economic Zone" where a lot of the normal communist
| rules don't apply, and then gradually relaxing the rules
| elsewhere as well. Just the fact that China allows individuals
| to start and run businesses is a huge break from communism.
| Kranar wrote:
| The article isn't about cleaning up header files. I don't
| think anyone criticizing the article is claiming that
| reducing the amount of code in header files will speed up
| compile times. The burden that the author should substantiate
| with evidence instead of claiming that he's a genius or a
| madman is whether moving includes out of header files and
| into source files as a strategy to avoid including the same
| header file multiple times has an effect on compile times.
|
| Most people criticizing this article, including myself, argue
| that it does not (along with reasons why, such as header
| guards and the optimizations that compilers include to
| recognize them).
___________________________________________________________________
(page generated 2022-05-13 23:03 UTC)