How to setup *arr apps without worries on Hard-linking in NAS storage reddit / selfhosted ---------------------------------------------------------------------- Illadan · 2023-10-07 13:16 I have a main NAS running on OMV which contains all my finalized & cleaned/metadata-ed media & a different system for managing the torrent downloads. I want to use the \*arr apps for media management & downloads, but i am confused by all the hard-links stuff. My systems is split into 3machines: * App Server which contains all my docker images (contains torrent client; plan to keep the \*app apps as well) * Staging NAS: this a small NAS which is used for staging downloads & is temporary storage. I use this place to first download stuff & start cleaning the media & add metadata. * Torrent downloads will be done to this system via an SMB share folder. * Media NAS: This is the main & final machine to which I move all my media & which will be managed by the \*arr apps * All the apps will be running in the app server machine via docker. Both the downloads folder & media folder will be available to the app server via SMB shares. **NOTE: Both NAS devices run on <2GB RAM & so cannot be used to host the apps.** I want to create a semi-automated setup for the \*arr apps, torrent client & both NASes & I do not want the hard-linking process/steps if it can be avoided as both folders will not be in the same physical system. How can i set-up my app eco-system to ensure smooth running of the entire media management ? ---------------------------------------------------------------------- mandonovski · 2023-10-07 13:27 Maybee Fileflows can help. ---------------------------------------------------------------------- Illadan · 2023-10-07 16:22 I am unaware of this tool; does it help me in moving files without hardlock needed ? ---------------------------------------------------------------------- ElevenNotes · 2023-10-07 16:37 Why do you need to hardlinking? You download to the staging NAS and unpack to the media NAS. ---------------------------------------------------------------------- ______-_-_________ · 2023-10-07 16:40 I ignored all the stuff about hard and soft links and setup *arr services as I saw fit. Mine all run in containers. Each got their own config folder mapped to a local folder. I learned the hard way that databases should be local. The downloads and media folders are nfs shares that are also referenced by Transmission and Plex, respectively. Only issues I've had are when I saturate Transmission with too much. I probably need to give the VM Transmission is running on a few more cores, but I won't. Transmission is on a VLAN with a VPN gateway. Plex and *arr are on the regular LAN. ---------------------------------------------------------------------- GrumpyCat79 · 2023-10-07 18:44 Hard links only work within the same drive, so in your case it won't do anything and files should br copied instead. You can also disable it under System > Media Management (under Advanced) if you are worried ---------------------------------------------------------------------- Wick3d68 · 2023-10-07 18:46 I have the same setup, and I don't understand the problem. Why don't you use samba shares to mount all Nas on the server? ---------------------------------------------------------------------- ethereal_g · 2023-10-07 23:23 If you can eliminate the staging nas, and download to the media nas that would be ideal. By keeping the download directories and media directories on the same file system you’ll be able to benefit from hard links. Otherwise you’ll be dealing with a second copy of every downloaded file and that may impact performance. ---------------------------------------------------------------------- daedric · 2023-10-07 23:26 Ok... i can answer this one. But first... # Hard Link VS Soft Link A soft link is like a shortcut, it's a pointer to another thing, and that thing can be another file, a folder, a /dev etc. Softlinks are very versatile, but come with a problem for us... 1. You instructed *arr to download something to /incoming. After it's done, your torrent app moves it to /complete. 2. Now.. the *arr must make sure it's available on /media/tvshows/TVShow Name/S03/etc etc. 3. You can softlink it then, saves the space, it's MUCH faster... etc. But then you want to delete it from the torrent APP... and now your softlink points to something that doesn't exist. *Enter hardlink* Hardlinks work quite lower... When you store a file in a partition, it must be present on a inode. So, the index will have /complete/tvshownameS03E01.mp4 is on inode 12345. **What if... you tell the index that /media/tvshows/TVShow Name/S03/E01.mp4 is also on inode 12345 ?** Now you have the same file... on two different dirs. You can delete one of them, the other remains... safe and sound. **The issue is.. hardlinks only work on the same filesystem.** # Because you're using actually different devices for the storage of files... hardlinking is NOT a solution for you. You must instruct the arrs (they will know that hardlink is not available...) that you cannot use hardlinks and the files must be copied from Stagin NAS -> Media NAS. **You must make sure that the arr's see the Staging download dir EXACTLY like the downloading app does.** If it's deluge, and it's downloading to /torrents/complete, the arr should see it exactly like this. Otherwise, you must configure dir mapping. After this, the ARR will monitor the download dir , and when something is done, copy it to the defined dir in Media NAS. Obviously... it must have access to it as well. ---------------------------------------------------------------------- Illadan · 2023-10-08 08:50 Thanks u/daedric, that was like a ELI5 understanding & made me understand inodes a bit better. Also thanks to u/______-_-_________ & others for the suggestion to go ahead with separate folders. - I have just finished setting up the stack & its running good for now - no issues other than needing to tweak the setting to better perform in Rpi4. u/ethereal_g, due to some home network & family issues, I have to keep the staging NAS separate as it involves ALL downloads from ALL Family members. The amount of malware/spyware/viruses/junk getting downloaded becoz my folks don't listen to may warnings..... the staging NAS is entirely secured & protected from the rets of my network; only qbit is given slightly better access, that too is monitored. ---------------------------------------------------------------------- Illadan · 2023-10-08 08:51 made sure to disable them; Thx.