High CPU usage by sshd and sftp-server when transferring files reddit / selfhosted ---------------------------------------------------------------------- Big-Finding2976 · 2023-11-29 03:00 I'm using a SFF Lenovo M700 (i5-6400T, 8GB RAM, 256GB SSD, 16TB USB HDD, Debian 11) for my server. At the moment, I'm just copying all the files from the drives on my Windows PC to the server using WinSCP to make sure that I have a backup. Speeds are around 40MB/s with large files, which is probably as good as I can expect transferring from/to spinning disk, but sshd on the M700 is using 35-50% CPU, and sftp-server is using 15-20%, so about 70% in total. That only seems to happen with large files, when transferring lots of small files they're using about 10% and 5% respectively, although it varies and can be double or half that. If it's going to use this much CPU whenever someone (or my sync or backup software) is transferring large files, I'm concerned that it won't have the capacity to run the other services that I need (Adguard, Home Assistant (probably as a VM in Proxmox), Jellyfin, Tailscale, Crowdsec, etc.). The 16TB USB HDD is encrypted with Veracrypt, but I don't think that's the issue as I see separate processes in top for kcryptd, and they generally add up to less than 10%. Is there anything I can do to reduce the CPU usage when transferring files to/from other PC's on the LAN to the server? Once it is deployed, the users won't be using WinSCP to transfer files, they'll probably use Filebrowser or STFPGo, and I'll set up automated syncs and pull backups, so will it use less CPU to transfer files using those methods rather than WinSCP? ---------------------------------------------------------------------- p_235615 · 2023-11-29 03:56 Do you have compression activated on SSH ? ---------------------------------------------------------------------- adamshand · 2023-11-29 04:06 By default SSH compresses and encrypts all data. This takes CPU. You can disable compression on the client or server, and you can choose a less computationally expensive encryption algorithm. It doesn't solve your problem, but in general, I recommend learning to use rsync for moving files around. ---------------------------------------------------------------------- Big-Finding2976 · 2023-11-29 09:35 It doesn't look like it. I believe it defaults to off, and I've checked ssh\_config and sshd\_config and it's not enabled in those, and in WinSCP compression isn't enabled under the SSH settings. ---------------------------------------------------------------------- Big-Finding2976 · 2023-11-29 09:44 From what I've read, it defaults to encrypt but not to compress, and I haven't enabled it on the server or in WinSCP. I'm sure rsync is great and I'll end up using it for some things, but for what I'm doing at the moment, digging through all the folders on my PC's HDDs and working out what I want to copy to the server HDD, I need a decent dual-pane explorer GUI so I can find stuff and drag and drop folders and files. The only GUI rsync client I've found for Windows is Acrosync, which costs $100 and doesn't have even dual-pane or drag and drop. ---------------------------------------------------------------------- Amazing_Shake_8043 · 2023-11-29 10:07 Have you considering experimenting with SMB ? ---------------------------------------------------------------------- Big-Finding2976 · 2023-11-29 11:05 Yeah, I'll try that. I'll need to find some software that shows the transfer speed though, as Windows Explorer isn't very helpful in that regard. I know it exists, as I've used it in the past, so it shouldn't be hard to find it, or something similar. ---------------------------------------------------------------------- FrontDry1622 · 2023-11-29 14:31 Windows Explorer shows transfer speed in the transfer Window. You could also open task manager and look at the network card statistics as it will show bandwidth used. ---------------------------------------------------------------------- Big-Finding2976 · 2023-11-29 15:18 I know Explorer attempts to show transfer speed, but my recollection is that it doesn't work very well and the speed shown often freezes or bears no relation to reality. Maybe it's better with Windows 11 though, I haven't really spent much time using that yet, so I'll try it. Using task manager is a good tip, thanks. ---------------------------------------------------------------------- adamshand · 2023-11-29 20:15 It might vary between distros but on Debian: > Compression > Specifies whether compression is enabled after the user has authenticated successfully. The argument must be yes, > delayed (a legacy synonym for yes) or no. The default is yes. ---------------------------------------------------------------------- adamshand · 2023-11-29 20:17 Not sure if it'll be helpful, but if you're looking for a better SFTP client you might like Filestash. https://www.filestash.app/ ---------------------------------------------------------------------- Big-Finding2976 · 2023-12-01 08:38 I tried setting 'Compression no' in /etc/ssh/sshd\_config and restarting sshd but it didn't make any difference unfortunately. I read that setting 'Ciphers arcfour' might help, but sshd wouldn't run at all with that, so it must be an old setting. ---------------------------------------------------------------------- Big-Finding2976 · 2023-12-01 09:07 I'm having trouble getting Samba to work on Debian. It's a darn sight more complicated than sharing a folder from Windows. I've setup a couple of read-only folders with guest access, so I don't need to enter a username and password to access them, and they work fine, but I've got another one that I want to be writable and user- restricted, and it won't let me login to that. This is the shares section of my smb.conf [https://pastebin.com/TDMd4rKc](https://pastebin.com/TDMd4rKc) and I've done: sudo groupadd sambashare sudo smbpasswd -a main sudo smbpasswd -e main sudo usermod -aG main sambashare sudo chown main:sambashare /mnt/16TB/Musicsoft sudo chmod 2775 /mnt/16TB/Musicsoft and restarted smbd.service, but when I try to access it on my Windows PC, it asks me for the username and password and then says that they're wrong. ---------------------------------------------------------------------- adamshand · 2023-12-01 19:35 Try blowfish, it's widely supported and almost as fast as arcfour. Try this in your `~/.ssh/config` like this: ``` Host * Ciphers blowfish-cbc Compression no ```