Help with transferring media library from 4tb HDD to 14tb HDD reddit / selfhosted ---------------------------------------------------------------------- Crazy-Hall8823 · 2023-10-18 03:01 Hey guys I recently purchased a 14tb ironwolf pro to replace my current 4tb hard drive that is used for my media server. I have tried to copy my media folder across in an SSH session but it got to about 20% and then I had a broken pipe. I have just set it off to copy again but I'm wondering if this will happen again? Is there a set time out on SSH sessions? Could there be an easier way to get this 4tb of data across to my new hard drive? I don't have access to my hard drive over SMB because the 4tb drive is having some issues in OMV. I also don't have a Linux desktop where I can read and copy over in ext4 format. Any assistance would be greatly appreciated ---------------------------------------------------------------------- Crazy-Hall8823 · 2023-10-18 03:04 Perhaps hooking up a screen and logging directly into the pi to do the transfer? ---------------------------------------------------------------------- RuaidhriM · 2023-10-18 03:06 What OSs? Maybe rsync? That will pick up where it left off so you can at least keep hammering away until all done. ---------------------------------------------------------------------- geo38 · 2023-10-18 03:08 Two suggestions 1) After connecting via ssh, run 'screen' (sudo apt install -y screen) That will create a virtual session. If your ssh connection breaks, that virtual session still exists. https://linuxize.com/post/how-to-use-linux-screen/ 2) Use some sort of copy command that can deal with restarting a partial copy rsync -av /mnt/4tb/* /mnt/14tb/ https://www.geeksforgeeks.org/rsync-command-in-linux-with-examples/# https://www.digitalocean.com/community/tutorials/how-to-use-rsync-to- sync-local-and-remote-directories ---------------------------------------------------------------------- Crazy-Hall8823 · 2023-10-18 03:22 Dude awesome reply thank you! I'll wait for this session to break and give screen a crack and then rsync if screen doesn't get the job done. ---------------------------------------------------------------------- Crazy-Hall8823 · 2023-10-18 03:24 I'm ssh in from a macbook. My only linux os stuff are headless servers. Nothing windows. That's good to know I've heard of rsync I was gonna try to set that up to pull my xbox clips from one drove and delete them but I never got around to trying it. The other guy mentioned rsync too. Think I'll try screen then rsyc if that doesn't work. ---------------------------------------------------------------------- adamshand · 2023-10-18 05:03 Learn rsync, it is your only friend for these sorts of jobs. :-) ---------------------------------------------------------------------- CrispyBegs · 2023-10-18 06:59 i did this a while back, but just connected to smb. Dragged all the media from one drive to the other in my mac's finder, left it to run overnight and it was done. ---------------------------------------------------------------------- Crazy-Hall8823 · 2023-10-18 18:17 Yea thats what I've done in the past but I can access my 4tb drive over SMB. The drive is throwing errors in OMV when I try to mount it and update omv. So I can't do it that way ---------------------------------------------------------------------- Crazy-Hall8823 · 2023-10-18 18:18 That is becoming more and more evident. It seems like an excellent solution. ---------------------------------------------------------------------- Low-Chapter5294 · 2023-10-19 03:38 tmux is a very good alternative to screen, if you come across that. ---------------------------------------------------------------------- jhf2442 · 2023-10-21 14:59 don't rsync a remote mounted directory as rsync may have to read the file to determine which portions to transfer, thus generating huge network traffic for nothing. install rsync on both hosts and then do rsync -av /mnt/4tb/ user@remote:/mnt/14tb/ ---------------------------------------------------------------------- geo38 · 2023-10-21 15:55 I must have missed where OP had two hosts. I believe OP used SSH to remotely connect to his pi to do the copy between two drives attached to the pi. His complaint is that his ssh session to the pi broke which stopped the copy process running in that ssh session. I could be wrong. Your advice is certainly sound when rsync'ing between two separate hosts. ---------------------------------------------------------------------- jhf2442 · 2023-10-21 17:16 deduced it from OP mentioning ssh, but now also unsure about it. anyhow, we agree on the technical topic 😉