Benchmarks: FreeBSD 13 vs. NetBSD 9.2 vs. OpenBSD 7 vs. DragonFlyBSD 6 vs. Linux reddit / BSD ---------------------------------------------------------------------- [deleted] · 2021-12-10 23:02 [deleted] ---------------------------------------------------------------------- DarthRevanG4 · 2021-12-11 07:51 I don’t know anything about it; but I do know BSDs often need a lot of different configuration for whatever situation they’re being used in. Linux generally comes that way already. He mentioned “out of the box state” more than once during that article which makes me think the BSDs weren’t given the best chance due to not being configured in the most efficient way for those tasks. It would be interesting to see Mac OS on those benchmarks as it’s the only BSD that really comes configured any certain way and for workstation/desktop use. Of course he’d have to hackintosh it onto that i9 system and the 1080ti used wouldn’t work on the latest version. ---------------------------------------------------------------------- qci · 2021-12-11 07:56 Do they have `atime` enabled? Or why is zstd so bad on BSDs? Edit: I also found that FreeBSD does not perform well on NVMe. You need to set the block alignment/size unusually high to get reasonable performance. With the default 4k it was slower than SATA SSDs. ---------------------------------------------------------------------- JavaLangObject · 2021-12-11 12:21 Even if he is right (which he isn't) I wont let go of such an organised ---------------------------------------------------------------------- grahamperrin · 2021-12-12 10:19 > … (I think there was even a BSDCan presentation about how wrong his results often are.) Link please. finds only one thing: * [Benchmarking FreeBSD](https://www.bsdcan.org/2013/schedule/attachme nts/242_BenchmarkingFreeBSD.pdf) ([Ivan Voras](https://people.freebsd.org/~ivoras/), 2013-05-20) ---------------------------------------------------------------------- grahamperrin · 2021-12-12 11:00 > Do they have atime enabled? Probably enabled by default. Here: % zfs get atime | grep default Transcend atime on default Transcend/VirtualBox atime on default august/poudriere/ports/default atime off inherited from august % zfs get atime august NAME PROPERTY VALUE SOURCE august atime off received % uname -aKU FreeBSD mowa219-gjp4-8570p-freebsd 14.0-CURRENT FreeBSD 14.0-CURRENT #116 main-n251146-d109559ddbf: Mon Nov 29 14:34:59 GMT 2021 root@mowa219-gjp4-8570p- freebsd:/usr/obj/usr/src/amd64.amd64/sys/GENERIC-NODEBUG amd64 1400043 1400043 % > Or why is zstd so bad on BSDs? … My result yesterday, booted from a pool with `atime` `off` for all relevant datasets: * Results were tarnished slightly by me sometimes making heavy use of Firefox during the most CPU-intensive tests. Tarnished but not useless. Simplified comparison: * **[20.3 for FreeBSD 13.0-RELEASE](https://openbenchmarking.org/embed.php?i=2112039-TJ- BSDLINUX675&sha=bf9c27c769a9&p=2)** with an SSD, 32 GB memory and a thirty-six core CPU from around 2019 * **[27.7 for me](https://openbenchmarking.org/result/2112119-TJ- ZSTD6248729#result-b72640d8b85a943206c9f7a260db873277a444e7)** with FreeBSD 14.0-CURRENT on a circa 2013 HP EliteBook 8570p (7,200 RPM HDD, 16 GB memory and a four-core CPU from around 2012). ---- **Why so much better for me, with the vastly inferior hardware?** First guess: L2ARC – two low-end USB thumb drives. Probed today: with the cache devices currently at `da0` and `da1` . % zfs-stats -L ------------------------------------------------------------------ ------ ZFS Subsystem Report Sun Dec 12 10:59:31 2021 ------------------------------------------------------------------ ------ L2 ARC Summary: (HEALTHY) Low Memory Aborts: 1.48 k Free on Write: 10.32 k R/W Clashes: 39 Bad Checksums: 0 IO Errors: 0 L2 ARC Size: (Adaptive) 28.78 GiB Decompressed Data Size: 69.85 GiB Compression Factor: 2.43 Header Size: 0.11% 76.43 MiB L2 ARC Breakdown: 8.89 m Hit Ratio: 33.51% 2.98 m Miss Ratio: 66.49% 5.91 m Feeds: 111.39 k L2 ARC Writes: Writes Sent: 100.00% 17.54 k ------------------------------------------------------------------ ------ % zpool iostat -v capacity operations bandwidth pool alloc free read write read write --------------------- ----- ----- ----- ----- ----- ----- Transcend 371G 92.9G 1 0 62.9K 43.9K gpt/Transcend 371G 92.9G 1 0 62.9K 43.9K cache - - - - - - gpt/cache-transcend 14.4G 99.8M 0 0 3.69K 5.84K --------------------- ----- ----- ----- ----- ----- ----- august 258G 654G 21 24 2.80M 567K ada0p3.eli 258G 654G 21 24 2.80M 567K cache - - - - - - gpt/cache-august 7.01G 21.8G 13 0 613K 26.0K gpt/duracell 7.44G 7.98G 12 0 594K 28.4K --------------------- ----- ----- ----- ----- ----- ----- % uptime 10:59a.m. up 1 day, 7:36, 7 users, load averages: 1.15, 1.03, 1.04 % I might re-run the set of tests with the cache devices offline. ---------------------------------------------------------------------- qci · 2021-12-12 11:17 If you get 27 MB/s it's still slow. There is something hugely different. I cannot tell what. Maybe even the zstd binary is a lot better on Linux. Edit: 1) I looked at the test suite. What is done there is compiling zstd by hand and starting it with `-T $NUM_CPU_CORES` which translates to `-T` which is number of CPUs not cores (SMT/HT). I cannot really tell, if they handle this environment variable correctly. 2) I noticed that there is not enough difference when running zstd with 1 or 8 threads. 1 thread needs 13s compressing an ISO and 8 threads need 11s. Weird... Edit 2: 1) Is verified wrong. The suite detected 8 cores correctly. Edit 3: I noticed that the zstd that Phoronix manually build is substantially slower (around 1/3rd speed). But the default pkg build is still very slow, in my opinion. ---------------------------------------------------------------------- grahamperrin · 2021-12-12 13:17 Thanks. Certainly slow, by comparison, still I wonder why my vastly inferior hardware gets superior results. Cross-reference where /u/masterblaster0 quotes an observation by Allan Jude. ---------------------------------------------------------------------- grahamperrin · 2021-12-12 13:20 ### L2ARC > … re-run the set of tests with the cache devices offline. * offline for the second run * online for the third * no significant difference. ---------------------------------------------------------------------- qci · 2021-12-12 13:27 Ok, I cannot answer there with my observations, because I'm banned. They are wrong that zstd benchmark only uses 1 core. I verified with top that the -T switch works and top shows the correct and plausible values. And the resulting runtimes do not differ much. This seems to be interesting: https://lists.freebsd.org/archives/freebsd- current/2021-December/001181.html ---------------------------------------------------------------------- grahamperrin · 2021-12-12 14:49 Thanks. ### Zstandard compression test results See what's linked from the foot of the opening post at ---------------------------------------------------------------------- qci · 2021-12-12 15:29 [I found some further information](https://www.reddit.com/r/freebsd_se lfhosted/comments/req0t2/benchmarks_by_phoronix_display_erroneous/ho8z fmo/?utm_source=reddit&utm_medium=web2x&context=3) about the `zstd` benchmark function that might have caused displaying of skewed results. ---------------------------------------------------------------------- grahamperrin · 2021-12-13 07:21 Yep, that's probably the c90 that I quoted. Thanks.