I3 instances and NVMe: booya! Or how you can build FreeBSD from the source in under 11 minutes vs. 12+ hours on a desktop. reddit / BSD ---------------------------------------------------------------------- dcorbe · 2017-09-05 14:10 It shouldn't be taking 12 hours to run a buildworld. ---------------------------------------------------------------------- phessler · 2017-09-05 16:14 I can build world in 25 minutes on my openbsd laptop. why is this taking you 12 hours? ---------------------------------------------------------------------- BumpitySnook · 2017-09-05 16:35 Fricking Clang. ---------------------------------------------------------------------- stopczyk · 2017-09-06 14:39 The article is somewhat confusing. First of all with today's ram amounts storage is not the bottleneck, especially so if you just unpacked the source tree - it's all cached in ram. The entire 11.1 world + src is not that big in the first place - unpacked on tmpfs fits in 1.7GB. With buildworld finished the complete usage goes up to only 5.5GB. The build process itself is definitely not super parallel, but it should still see benefits from 64 hardware threads. A quick test with 40 hardware threads with user/sys/idle times collected every 10 seconds over -j 40 buildworld shown 78.1413 4.91304 17.8478 (i.e. about 18% of the total time completely wasted). However, the most time consuming component (clang) utilizes extra cores very nicely and accounts for about 40% of the total build time. It should also be noted that system time is extremely high. * there are some scalability issues, but on bare metal with otherwise equivalent configuration they would not show up * this is presumably on the default kernel. using one with VM_NUMA_ALLOC will make things faster due to reduced contention * there is an optimisation which did not make it to 11.1, but is present in stable/11 which results in significantly fewer IPIs being sent. it's not a big deal on bare metal, but it significantly affects performance on ec2. tl;dr * these days storage is not a big deal for this workload * the freebsd kernel itself can build this faster as it is * the kernel can be improved and there are people working on that * the build process itself probably is not as fast as it should be * the build process itself should benefit from 64 hardware threads, but once more it wont utilize them to their full extent all in all, even with the hardware as is I suspect the build process could go down to about 9 minutes or even less. for interested parties this is what i got on an old system after buildworld on tmpfs: 821.86 real 22204.57 user 1672.78 sys ---------------------------------------------------------------------- stopczyk · 2017-09-06 14:39 the 12h note apparently is a remark about 'an old pc desktop', not anything recent. still way too long though.