Post Apjgk1HoWLPqxIvZvk by ljs@social.kernel.org
(DIR) More posts by ljs@social.kernel.org
(DIR) Post #ApjgeZJD73hQjkyvBI by arnd@society.oftrolls.com
2024-07-23T13:20:57Z
1 likes, 0 repeats
I had a feeling that kernel compilation got slower recently and tried to find the slowest file across randconfig builds. It turned out to be arch/x86/xen/setup.c, which takes 15 seconds to preprocess on a reasonably fast Apple M1 Ultra.This all comes from one line "extra_pages = min3(EXTRA_MEM_RATIO * min(max_pfn, PFN_DOWN(MAXMEM)), extra_pages, max_pages - max_pfn);" that expands to 47MB of preprocessor output after commits 80fcac55385c..867046cc70277.
(DIR) Post #ApjgehYkRAsiKGJRVg by arnd@society.oftrolls.com
2024-07-23T18:35:55Z
0 likes, 0 repeats
If anyone wonders what other files are bad, this is the list of all files with over five second compile times: https://pastebin.com/raw/fXJe7y9P.The worst case for this particular file was 44 seconds, three times slower than in defconfig.The median compile time across all files is 0.26s.
(DIR) Post #Apjgk1HoWLPqxIvZvk by ljs@social.kernel.org
2024-07-23T16:27:55.806092Z
0 likes, 0 repeats
@arnd we going to revert that series then?I feel there is pedantry at play...
(DIR) Post #Apjgk2pWmTyhk9i17Q by osandov@fosstodon.org
2024-07-23T16:44:52Z
0 likes, 0 repeats
@ljs @arnd eh I don't think it's fair to call this pedantry. This looks like a legitimate quality of life improvement for a common annoyance with min/max, but given the consequences, it's fair to say that it's not worth it.
(DIR) Post #Apjgk3bjtEXi9gGWGW by ljs@social.kernel.org
2024-07-23T16:49:22.242705Z
0 likes, 0 repeats
@osandov @arnd Regardless, this is an unacceptable build regression.Chatted to @arnd on IRC who suggests he is tied up with stuff to dig into this more so I'll have a wee look :)EDIT: Think that'll be more likely a report to list + leave others to solve as this seems quite involved and I have a ton of things to do :>)
(DIR) Post #Apjgk4bmAaANG5I3xg by ljs@social.kernel.org
2024-07-23T22:08:31.191138Z
1 likes, 0 repeats
@osandov @arnd For those following, posted https://lore.kernel.org/linux-mm/c83c17bb-be75-4c67-979d-54eee38774c6@lucifer.local/T/#u summarising situation + showing some results on my local box (ignore cringe hostname of said box)
(DIR) Post #ApjhGw3WzJiYI2wUWu by KeyJ@mastodon.gamedev.place
2024-07-23T15:38:10Z
0 likes, 0 repeats
@arnd But ... how? Even if it expanded to 47 kilobytes, that would be excessive, but 47 *megabytes*? 🤯
(DIR) Post #ApjhGxWdWaaqqbZFXE by arnd@society.oftrolls.com
2024-07-23T16:04:03Z
0 likes, 0 repeats
@KeyJ it nests min() multiple levels deep with the use of min3(), and each one expands its argument 20 times times now (up from 6 back in linux-6.6). This gets 8000 expansions for each of the arguments, plus a lot of extra bits with each expansion. PFN_DOWN(MAXMEM) contributes a bit to the initial size as well.See https://pastebin.com/MmfWH7TM for the first few pages of it.
(DIR) Post #ApjhGyHQic1XBjScTI by dirksteins@social.cologne
2024-07-23T18:05:20Z
0 likes, 0 repeats
@arnd @KeyJ and this is effective and efficient code? Doesn’t look like it to me - but I’m not a kernel developer.
(DIR) Post #ApjhGzCVIPg42kACQq by mansr@society.oftrolls.com
2024-07-23T18:27:44Z
0 likes, 0 repeats
@dirksteins @arnd @KeyJ It's a huge constant expression that gets evaluated by the compiler. The final code is (probably) fine.
(DIR) Post #ApjhGzyMQTxURAYQ1g by dirksteins@social.cologne
2024-07-23T18:46:28Z
0 likes, 0 repeats
@mansr @arnd @KeyJ probably. But I still ask myself if a 47MB expression is useful and if the compiler is able to process this correctly (it probably is, but is it efficient?).
(DIR) Post #ApjhH0n3O0VYyOGu2a by mansr@society.oftrolls.com
2024-07-23T18:53:11Z
1 likes, 0 repeats
@dirksteins @arnd @KeyJ There's no reason to doubt the correctness, but as has been noted, it is somewhat slow.Now as shocking as this might be, it is quite tame compared to the boost C++ library.