Integrating samba, veracrypt and snapraid in ezjail reddit / freebsd ---------------------------------------------------------------------- RyzenRaider · 2020-09-26 11:55 Hi guys, I get that this question involves a fair bit of 3rd party software, but hoping to get some opening guidance about how to set this up. Been interested in FreeBSD for a long time and tried using it as a desktop but found the learning curve a bit too steep and went to Linux instead. Now setting up my first server to replace a cheap QNAP NAS and Raspberry Pi, running some services and hosting several hard drives. Below is the setup. 1. 7 hard drives connected, each of them wholly encrypted via Veracrypt (full disk, not a file container). The file system on each disk is ext4. 1 disk is new and currently just a spare. 2. I was using Snapraid to generate parity (if you don't know about Snapraid, it's most similar to Unraid, except parity isn't maintained in realtime). The disks are not striped, and parity is stored on dedicated parity disks. In case anyone wonders why I don't use ZFS or other raid, I just want to avoid striping. Creates more problems than its worth in my use case. 3. I'm intending to share by SMB, as I know all my other devices on the network are working well with that protocol. To start, this is where I'm at: 1. Installed FreeBSD RELEASE 12.1. 2. Installed veracrypt and ezjail from pkg in the base system. 3. Setup an ezjail for the samba share, and installed samba within the jail. 4. Mounted the Veracrypt disks into `/usr/jails/samba/media` in the samba jail. Currently the disks are mounted read-only so I can't do anything stupid. 5. Configured samba to share the /media folder, also configured to read only. I am now able to access all of my files in a read-only share, so this setup 'works', although I don't know if this is ideal or recommended. I also haven't yet setup Snapraid. 1. Should/can I run veracrypt, snapraid and samba in 3 separate jails, yet each of them can access the data on those drives (veracrypt to mount/unmount, snapraid to check and update parity, and samba to share)? Or do these all need to be in one jail and share the same mount points? 2. Side question: It seems like ext4 support is fairly new and maybe incomplete? One of the drives is new to the array and currently empty. Would it be recommended to switch each drive over to UFS for better support/stability/reliability? Or is ext4 pretty solid? Advice, suggestions or questions to clarify welcome. Thanks in advance :) ---------------------------------------------------------------------- spanctimony · 2020-09-26 18:34 What’s the problem with striping? This seems like an insane configuration. ---------------------------------------------------------------------- bawdyanarchist · 2020-09-26 19:38 You might think about using pefs instead of Veracrypt. ---------------------------------------------------------------------- kraileth · 2020-09-26 21:36 Hm. Your setup sounds a bit strange to me. Here are a few thoughts: 1) Do you need to be able to read the encrypted drives from any other system? If so, Veracrypt might be the best solution. If not you might want to go with FreeBSD's GELI encryption instead. It's very easy to use and it's native to FreeBSD - if you ever have e.g. a problem with your machine, any live medium will let you decrypt GELI and repair things or get your data off the drives. Veracrypt being third-party would be less to my liking (unless you have to use it or already know it perfectly well). 2) Why ezjail? It's probably the oldest jail management utility out there. It had its time and is still in use by many for the simple reason that they know it. But if you're starting fresh consider using another tool as several are available that are much more versatile and provide more modern features. A pretty popular one is `iocage`. If you're looking for a comprehensive solution also give `cbsd` a shot; it does jails, bhyve and more while providing nice dialog-based menus and great freedom with advanced settings (for people who want to touch those). 3) You're on 12.1 so EXT4 rw support is available. Things *should* be fine, however support for journaling as well as some advanced features of that fs are missing. Especially with journaling support missing, I wouldn't really recommend using EXT4 for your SMB shares. For copying data over from or to a Linux system - sure. But not for regular use. 4) What is the issue with striping? And even when you really want to avoid that, ZFS might still be your friend. FreeBSD has the GEOM storage framework that gives you great flexibility. You could actually concat (not stripe if you really don't want that) your drives together and use that storage provider for your zpool. Be sure to create two, though, and have ZFS create a mirror to get redundancy for your data. ---------------------------------------------------------------------- RyzenRaider · 2020-09-27 04:29 About striping: 1. Striping increases read/write speeds, except that I'm bound by gigabit ethernet, so one hard drive is enough to saturate NAS bandwidth over my network. So I don't benefit here. 2. Striping requires all disks to be spinning, creating a larger power draw and more noise. Unstriped, I can watch a movie with only 1 disk spinning. Living in a small apartment, noise is an issue for me. 3. My disks are also not matched - 2 are shucked, 4 are WD Red 4TB and another is a WD Red 8TB. This is primarily due to repurposing drives from an old NAS and other enclosures I've owned. Striping these would likely have horrible performance. 4. Striping is difficult to expand, without buying a batch of matching disks. I don't have that kind of money, I'll be buying disks one a time to expand a few TB as needed, and the drive may not match the ones I already have. So there's lots of drawbacks in my use case, and it's primary benefit is nullified. I switched from a QNAP NAS to using SnapRAID for that very reason. I was running out of space and realised I need to spend over $1k to expand it, and I didn't have that kind of money. For this reason, any hardware RAID 5/6 or ZFS RAIDZ setup is effectively ruled out for me. ---------------------------------------------------------------------- spanctimony · 2020-09-27 04:52 None of this is a valid reason to not use ZFS. ---------------------------------------------------------------------- RyzenRaider · 2020-09-27 05:06 Thanks for the reply. 1. Encryption - The intention is to share the drives on my home network once they are unlocked and mounted. I've dabbled with GELI - RobboNuggie actually posted a Youtube tutorial after I had suggested it, and I was able to recreate it in a VM. But you nail the last point, I've been using VC for about 5-10 years and have become very comfortable with it. I like it's mode - if it's unmounted, the entire disk - including the file system - is secured. Mounted, I can see all. I also like that VeraCrypt is portable across OSs (it's unlikely that this will realistically be needed, but it's nice to know that it's there). 2. Why ezjail - because it was in the FreeBSD handbook lol. I had noticed it hadn't been updated for a while. I knew iocage was around, but thought for some reason it could only be used with FreeNAS and ZFS volumes. My boot disk on this server is ZFS, so I'll actually look into iocage. Thanks for this suggestion. 3. ext4 on BSD - Ok your position sounds like mine. It works-ish, but have trust issues. In this case, perhaps I should use the spare disk to hop a disk at a time over to something more FreeBSD friendly (like UFS). I am keeping the whole array on read only for the time being so things like journalling shouldn't be a problem. 4. I replied to another comment about striping. The tl;dr is that gigabit networklng prevents me from taking advantage of the throughput, all disks must spin for even minor activity (more power draw, more noise), my disks aren't matched and expansion options are limited and costly. Also, if the server fails, I could literally take a disk out of the server, connect it to my desktop, mount it (via VeraCrypt) and see/recover the contents. Striping has its benefits, but I don't think it's ideal for home user network shares. 5. About GEOM framework - JBOD would be fine for the content itself, but mirroring would be inefficient (50% storage for redundancy, but only supports 1 disk failure). For comparison, I'm currently using 4 disks for data and 2 for parity. A 5th data disk will be rolled out, once the server config is finalised. It's probably worth mentioning that due to my experience with VeraCrypt and SnapRAID over time, I've become very comfortable with them. I know how they work, their scopes and limitations. I've also already written scripts on Linux to check for the drives, unlock and mount them, then sync or scrub parity with SnapRAID and output a report that I can peruse occasionally. The script streamlines a lot of that hassle using different tools. I'm thinking this is a plan to move forward (but again, open to feedback and suggestion): 1. Investigate iocage. Is it better for my needs and can I understand it. Choose the better between ezjail and iocage for jail management. 2. Setup a jail with veracrypt, snapraid and samba. Update my veracrypt/snapraid script to match the new environment and have samba share the mount points. This one jail is then responsible for unlocking, validating/scrubbing and sharing my storage pool. 3. Any other services I setup won't need access to these disks, as they'll be doing other things. Bit torrent client for sure, then others as interests expand (DNS sinkhole, Searx instance, syncthing, etc). So these can be in separate jails and isolated from storage management. Does this sound realistic? ---------------------------------------------------------------------- RyzenRaider · 2020-09-27 05:28 My current setup is 4 data disks, 2 parity disks. My understanding is that the equivalent RAIDZ2 does stripe data and parity across all 6 disks. Is there a RAIDZ2 unstriped variant that I'm not aware of? If not, then I think the cost issue alone (needing to spend over $1k to add more storage) is a valid reason to exclude it. I can't use something that I can't afford. And non-striped parity has benefits which are useful to me at my budget, which ZFS lacks (to my knowledge): 1. I can also lose 'n-parity' + 1 disks and still retain the data on the working disks, rather than lose everything. Theoretically, I could lose 3 of my 6 drives right now and still have upto 75% of my data intact (albeit no further fault protection). 2. I can add a storage disk or a parity disk at my leisure, and have it apply instantly. No resilvering process needed in most cases. 3. Let's say I have an extra drive lying around and decide to add it to the array as a parity disk. When I eventually run low on free space, I can just reassign the parity disk as another data disk (think of going from 10 data/4 parity to 11 data/3 parity, as an example). Again, this change can be instantaneous. ZFS mirroring can address some of those issues, but then you lose at least half of raw storage to the mirror. I'd be losing capacity. I know ZFS is amazing, I spent a lot of time reading about it and how it protects data. It's a true gold standard. But striping data introduces more problems than it solves for me, at the budget I'm working with. I need per-disk flexibility, and I don't see how ZFS can offer me that. I hope that clarifies some of my choices. ---------------------------------------------------------------------- RyzenRaider · 2020-09-27 05:34 I wasn't aware of this. Reading through the website now, thanks :) ---------------------------------------------------------------------- TowelGroundbreaking · 2021-01-30 19:13 Could you tell me how you installed VeraCrypt on your Freenas system? I have a jail and I need to decrypt a veracrypt file container inside the jail. The file container is on my storage pool and is mounted as read-only in the jail. ---------------------------------------------------------------------- RyzenRaider · 2021-01-31 01:47 I wasn't using FreeNAS, I was just using FreeBSD. Also, I wasn't able to make it work in the end. I ended up installing Ubuntu on the server. I needed to get it going, so had to stick to what I know.