[HN Gopher] Experience converting a mathematical software packag...
___________________________________________________________________
Experience converting a mathematical software package to C++20
modules [PDF]
Author : vblanco
Score : 87 points
Date : 2025-07-01 13:46 UTC (9 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| trostaft wrote:
| Oh, it's Wolfgang. In computational math, he has a focus on
| research software that few others are able to do, he (the deal.ii
| team more generally) got an award for it last SIAMCSE. Generally
| a great writer, looking forward to reading this.
| Asooka wrote:
| I would like to see a comparison between modules and precompiled
| headers. I have a suspicion that using precompiled headers could
| provide the same build time gains with much less work.
| pjmlp wrote:
| As per Office team, modules are much faster, especially if you
| also make use of C++ standard library as module, available
| since C++23.
|
| See VC++ devblogs and CppCon/C++Now talks from the team.
|
| Pre-compiled headers have only worked well on Windows, and OS/2
| back in the day.
|
| For whatever reason UNIX compilers never had a great
| implementation of it.
|
| With exception of clang header maps, which is anyway one of the
| first approaches to C++ modules.
| fpoling wrote:
| This has been puzzling me for over 3 decades. My first
| experience with C++ was Borland C++ for DOS. It had
| precompiled headers and it worked extremely well.
|
| Then around 1995 I got access to HP-UX and native compiler
| there and GCC. Nobody heard about precompiled headers and
| people thought the only way to speed up compilation was to
| get access to computer with more CPUs and rely on make -j.
|
| And then there was no interest to implement precompiled
| headers from free and proprietary vendors.
|
| The only innovation was unity builds when one includes
| multiple C++ sources into super-source. But then Google
| killed support for it in Chromium claiming that with their
| build farm unity builds made things slower and supporting
| them in Chromium build system was unbearable burden for
| Google.
| dataflow wrote:
| Precompiled headers are generally better for system/3rd-party
| headers. Module are better than PCHs for headers you own,
| although in some cases you may be better off not using them at
| all. (I say these because the benefit depends on the frequency
| with which you need to recompile them, and the relative
| coupling etc.) Depending on how heavy each one is in your
| codebase, and how often you modify global build settings, you
| may have a different experience. And neither is a substitute
| for keeping headers lightweight and decoupled.
| w4rh4wk5 wrote:
| From my experience, compile times ain't an issue if you pay a
| little attention. Precompiled header, thoughtful forward
| declarations, and not abusing templates get you a long way.
|
| We are commonly working with games that come with a custom
| engine and tooling. Compiling everything from scratch (around
| 1M lines of modern C++ code) takes about 30-40 seconds on my
| desktop. Rebuilding 1 source file + linking comes in typically
| under 2 seconds (w/o LTO). We might get this even lower by
| introducing unity builds, but there's no need for that right
| now.
| ttoinou wrote:
| 40 seconds for 1M lines seems super fast, do you have a fast
| computer and/or did you spend a lot of time optimizing the
| compilation pipeline ?
| vblanco wrote:
| The modern cryengine compiles very fast. Their trick is
| that they have architected everything to go through
| interfaces that are on very thin headers, and thus their
| headers end very light and they dont compile the class
| properties over and over. But its a shame we need to do
| tricks like this for compile speed as they harm runtime
| performance.
| ttoinou wrote:
| Why does it ruin runtime performance ? The code should be
| almost the same
| vblanco wrote:
| Because you now need to go through virtual calls on
| functions that dont really need to be virtual, which
| means the possible cache miss from loading the virtual
| function from vtable, and then the impossibility of them
| being inlined. For example they have a ITexture interface
| with a function like virtual GetSize(). If it wasnt all
| through virtuals, that size would just be a vec2 in the
| class and then its a simple load that gets inlined.
| ttoinou wrote:
| Ah yes this kind of interface ok indeed this doesn't seem
| like a useful layer when running the program. Maybe the
| compilers could optimize this though
| w4rh4wk5 wrote:
| We didn't create this code base ourselves, we are just
| working with it. I'd assume the original developers payed
| attention to compile times during development and
| introduced forward declarations whenever things got out of
| hand.
|
| My computer is fast, AMD Ryzen 9 7950X, code is stored on
| an NVMe SSD. But there certainly are projects with fewer
| lines of code that take substantially longer to compile.
| npalli wrote:
| Thanks to author for doing some solid work in providing data
| points for modules. For those like me looking for the headline
| metric, here it is in the conclusion While the
| evidence shown above is pretty clear that building a software
| package as a module provides the claimed benefits in terms of
| compile time (a reduction by around 10%, see Section 5.1.1) and
| perhaps better code structure (Section 5.1.4), the data shown in
| Section 5.1.2 also make clear that the effect on compile time of
| downstream projects is at best unclear.
|
| So, alas, underwhelming in this iteration and perhaps speaks to
| 'module-fication' of existing source code (deal.II, dates from
| the '90s I believe), rather than doing it from scratch. More work
| might be needed in structuring the source code into modules as I
| have known good speedup with just pch, forward decls etc. (more
| than 10%). Good data point and rich analysis, nevertheless.
| Someone wrote:
| It wouldn't surprise me if they could do better if they gave up
| on doing most of the work programmatically.
|
| One part of me agrees with (both from the paper)
|
| > For example, putting a specific piece of code into the right
| place in each file (or adding necessary header files, as
| mentioned in Section 5.2) might take 20-30 seconds per file -
| but doing this for all 1051 files of deal.II then will take
| approximately a full day of (extremely boring) work. Similarly,
| individually annotating every class or function we want to
| export from a module is not feasible for a project of this
| size, even if from a conceptual perspective it would perhaps be
| the right thing to do.
|
| and
|
| > Given the size and scope of the library, it is clear that a
| whole-sale rewrite - or even just substantial modifications to
| each of its 652 header and 399 implementation files - is not
| feasible
|
| but another part knows that spending a few days doing such
| 'boring' copy-paste work like that often has unexpected
| benefits; you get to know the code better and may discover
| better ways to organize the code.
|
| Maybe, this project is too large for it, as checking that you
| didn't mess up things by building the code and running the test
| suite simply takes too long, but even if it seems to be, isn't
| that a good reason to try and get compile times down, so that
| working on the project becomes more enjoyable?
| isatty wrote:
| The code block styling is less than ideal.
___________________________________________________________________
(page generated 2025-07-01 23:00 UTC)