8x slower SCP uploads to OpenBSD (vs FreeBSD) reddit / BSD ---------------------------------------------------------------------- ZoDalek · 2018-06-30 22:07 I have a identically configured VPSes running FreeBSD and OpenBSD. Both installations use the default file system, automatically recommended disk layouts and built in SSH daemons. Using WinSCP, uploading a 100MB file to the OpenBSD host takes about 8x longer than uploading to the FreeBSD host. Download speeds are the same, as are upload and download speeds for a batch of 10KB files. Why could this be? I’m going to try some more things to narrow my focus, but I’m wondering if anyone here experienced the same. I’m also open for recommendations for other file sharing protocols, as long as there’s a good GUI Windows client. ---------------------------------------------------------------------- avatar4d · 2018-06-30 23:11 [Try the tunables listed in this article](https://calomel.org/network_performance.html) ---------------------------------------------------------------------- Great_Wall · 2018-07-01 03:36 Have you tried mounting the filesystem with [softdep](https://man.openbsd.org/mount)? ---------------------------------------------------------------------- ZoDalek · 2018-07-01 05:54 Thanks, that looks like a very comprehensive article. Will go through it later today and report back. ---------------------------------------------------------------------- ZoDalek · 2018-07-01 05:55 From first sight I’d think that’d be most useful when dealing with large number of files (rather than large ones) but I’ll give it a try. ---------------------------------------------------------------------- meeu · 2018-07-01 07:18 wtf i thought it was just fallout fanart until i tried to scroll wheel ---------------------------------------------------------------------- phessler · 2018-07-01 08:18 No. That entire site is full of stupid shit. Never use anything from there. ---------------------------------------------------------------------- Great_Wall · 2018-07-01 09:33 I was thinking the same thing, but then I figured it was possible that when uploading a single file to the box, things might still get bogged down due to the FTP server constantly updating the file's metadata as it's appending to it. If softdep works, let us know, since I'm curious to see if my guess was right. ---------------------------------------------------------------------- ChiefDetektor · 2018-07-01 09:33 OpenBSD disabled Hyperthreading on it's latest releases. You might check out if this is the case. :) ---------------------------------------------------------------------- skimpark · 2018-07-01 10:50 I took a lot of calomel’s advises along last two years, especially about tcp tuning, and my boxes literally flew to the moon. can’t figure out why you are writing this ---------------------------------------------------------------------- avatar4d · 2018-07-01 13:32 I can certainly respect your opinion given your involvement in the project. Is there a better resource available for tuning advice or are you saying the default is suitable for all situations? I've been using BSD for 20 years doubt that would be the case, but I'm interested in your legitimate guidance. That being said, I'm trying to help while your comments are less than helpful and not very professional. As a monthly donor to the project I'm disappointed. ---------------------------------------------------------------------- rysto32 · 2018-07-01 14:19 Does FreeBSD still ship with the High Performance Networking patches to OpenSSH? They used to, but I'm not sure whether they didn't get dropped due to the increased maintenance costs. ---------------------------------------------------------------------- daemonpenguin · 2018-07-01 15:24 I'd be interested in knowing if this is a network/service or file system issue. Maybe the system is slow writing large amounts of data to the disk for some reason and it's not a problem with OpenSSH? If you sign into both systems and use dd to write a large file to the hard drive, does one complete faster than the other? Something like "time dd if=/dev/zero of=myfile count=1000000 bs=1000" on each machine should give an indication if data writing big files to disk is slower on one than the other. ---------------------------------------------------------------------- ZoDalek · 2018-07-01 16:04 I tested `time dd if=/dev/zero of=big bs=1M count=1K` which took 24.5s on both OSes (±0.1s). I also tried copying files between volumes earlier with similar results, so I don’t think it’s the file system or such. Perhaps I’ll try another network protocol next. ---------------------------------------------------------------------- rkoberlin · 2018-07-01 16:35 Phessler has contributed enough to the project. He's allowed to be abrasive with the amount of stupid questions he (and other contributers) have to deal with. ---------------------------------------------------------------------- tidux · 2018-07-02 00:58 Did you add `conv=fsync` to make sure the files were actually getting written through to disk? ---------------------------------------------------------------------- phessler · 2018-07-02 07:16 That site has caused a lot of problems for users, and has not fixed the bad information even after years of them being asked to. ---------------------------------------------------------------------- ZoDalek · 2018-07-02 07:25 There’s no fsync conversion in OpenBSD. The manual doesn’t mention anything similar except the various sync options that are used for padding to the block size. Copying files between directories or disks also has similar performance between the operating systems so I don’t think the culprit is to be found there. The next thing I will try is compare local VMs to rule out the VPS provider. ---------------------------------------------------------------------- bee_man_john · 2018-07-03 10:52 however at the same time a lot of openbsd defaults are stuck at 1999 PC level, and are really, really crippling to preformance, esp network speed and memory, and there is relatively little documentation about how to make things not suck, can you blame people for using that site? ---------------------------------------------------------------------- BumpitySnook · 2018-07-11 03:35 They got dropped in r294325 (2016). (See also r294335.) ---------------------------------------------------------------------- unixisawesome · 2018-07-12 23:56 and there is relatively little documentation The documentation is called man pages, and there's one for every program in the system. I blame people for being lazy and seeking out 3rd party info when it's all there straight from the source. If the program has arguments and tunable features do you think the programmer writing the manual page is going to hide them from you because OpenBSD is some sort of elite clique? Get real man. ---------------------------------------------------------------------- bee_man_john · 2018-07-12 23:59 Im well aware about which manpages to look in to fix the issues I stated. It is entirely not obvious what the problem even *is* however, you just get shitty tcp preformance, and randomly crashing browsers. ---------------------------------------------------------------------- [deleted] · 2018-07-13 00:21 [deleted] ---------------------------------------------------------------------- unixisawesome · 2018-07-13 00:47 My internet speed with OpenBSD esp. using ethernet has always been on par with whatever the ISP was providing at the moment. Bittorrent clients worked well and fast too, no different than Arch Linux or Windows 10 in terms of speed. This complaint is really questionable, I mean how do we really know it's the tcp performance? Maybe it's hardware or the ISP? This guy has a pretty strange setup to begin with he could have just misconfigured it. Millions of dollars have been spent running though code just to discover a bug was caused by typing i==2 as opposed to i=2. The latest Firefox 61 on OpenBSD is really stable and I haven't experienced a crash yet. The crashing is not OpenBSD's fault, it's in engineering the conversion layer. Other OS's let the web browser do unsafe things with memory, whereas OpenBSD is REALLY strict about memory usage. If the program even slightly violates memory rules the kernel kills it immediately, assuming it may be an attacker exploiting memory (better safe than sorry). But in Firefox at least, the port- maintainer has worked most of this out by now. And in the past it was usually only questionable websites like porn and pirating and adware that crashed the browser (and sometimes plugins), well in a security minded operating system this already is counter-intuitive. ---------------------------------------------------------------------- SnoweCat7 · 2018-07-15 22:48 Me too. I think I roughly doubled the speed of samba on my FreeBSD NAS with the help of those guides, but with the qualifier I also read carefully, put some effort into researching the options elsewhere, and did not implement all the tweaks where my investigations suggested it might be useless or even detrimental in some cases. ---------------------------------------------------------------------- mdickers47 · 2018-10-10 18:14 FWIW, I have a very similar observation, of 3MB/s throughput going to an OpenBSD machine and 13MB/s going to FreeBSD across the same links. This is very consistent and reproducible running netcat of /dev/zero to /dev/null, so ssh and local disk speeds are out of the picture. From pcaps, the most obvious difference is that OpenBSD is offering a TCP window around 14k that never increases, and FreeBSD starts at 64k (and also never increases). This appears to only happen when the initiator of the TCP connection is Linux.