Disk partitioning/ZFS question for a new server reddit / freebsd ---------------------------------------------------------------------- ladder_filter · 2021-04-27 13:53 Hello everyone! I'm a linux admin that's been working on learning freebsd in my spare time. I have a server that I want to install freebsd 13 on, but I have a question about ZFS: this box is an oddball - it has 3 disks, 2 are the same size, the 3rd one is much bigger. with ZFS, can you install on one disk and then just add the other disks later? I've read that the installer does something called "root on ZFS", does that allow me to add disks later and the OS takes care of "automagically" adding space? am I understanding this correctly? thank you all! ---------------------------------------------------------------------- freebsd_guy · 2021-04-27 14:19 It depends how you want to organise the storage. If you just want a stripe, then yes, you can install on one disk, add the others later and get the space of all three. Not very advisable though as there’s no redundancy and the loss of any disk loses everything. If you install to one disk, then want to make a mirror, ZFS will only accept the disk if it’s same size or bigger than the existing disk (obviously you can’t mirror to a smaller disk). You could make a raidz out of all three, allowing you to lose one disk, but each disk will be limited in size to that of the smallest one, so your usable space will be around the size of both smaller disks put together. ---------------------------------------------------------------------- urglecom · 2021-04-27 14:21 You will be able to add the other disks to the pool later, yes; but this will mean that you have an increased chance of things breaking (as one disk failure will take down the whole pool). To avoid that you need to add disks in a raid or mirror; it's been a while but I think you can do it at install time either way. If you use a raidz1 then you can lose one disk without data loss. However, the space of a 3 disk raidz1 pool will be twice the size of the smallest disk. For example, if you have a 4TB disk and 2 3TB disks, then you can pretend it's an old-stool RAID5 array made of the 2 3TB disks, and the first 3TB of the 4TB disk. This gives you 6TB of usable data, 3TB used as redundant sore, and 1TB wasted. ---------------------------------------------------------------------- ladder_filter · 2021-04-27 14:24 > Not very advisable though as there’s no redundancy and the loss of any disk loses everything. yup, makes perfect sense. the good news is this box is just for learning. I will probably start with the stripe on one disk approach, and on the next rebuild (once I get a little more experience) start playing with organizing the storage a little more appropriately. > You could make a raidz out of all three, allowing you to lose one disk, but each disk will be limited in size to that of the smallest one, so your usable space will be around the size of both smaller disks put together. again, makes perfect sense. thank you so much for replying! ---------------------------------------------------------------------- ladder_filter · 2021-04-27 14:24 > However, the space of a 3 disk raidz1 pool will be twice the size of the smallest disk. For example, if you have a 4TB disk and 2 3TB disks, then you can pretend it's an old-stool RAID5 array made of the 2 3TB disks, and the first 3TB of the 4TB disk. This gives you 6TB of usable data, 3TB used as redundant sore, and 1TB wasted. thanks for writing all of this out, makes perfect sense. thank you! ---------------------------------------------------------------------- [deleted] · 2021-04-27 18:35 one disk failure will take down the whole pool ? can you explain this as this seems totally opposite to what ZFS does. ---------------------------------------------------------------------- urglecom · 2021-04-27 20:23 ZFS can be configured in a way that is akin to old-school "striping" or "raid-0". It makes sense in some situations, but not in the general case. ---------------------------------------------------------------------- [deleted] · 2021-04-27 21:45 That sounds like a disaster. ---------------------------------------------------------------------- Lucretius_5102 · 2021-04-27 22:53 That’s what RAID-0 is, yes. ---------------------------------------------------------------------- mirror176 · 2021-04-27 23:11 With 2x+ the disks you can RAID (but why?) or mirror the stripe so you have redundancy on disk failure. Theres a ZFS command that will lock the state of the pool so you could undo changes to it in case you use the wrong command and add the new disk as an additional striped disk. ZFS will not automatically add disks.; you will likely also want to preemptively tweak things before adding such as portioning the disk even if to use slightly less than 100% so you know the next disk will fit properly even if a few sectors short. Such tweaking thoughts go along the lines of vfs.zfs.min\_auto\_ashift=12 being a better future proof default even if not 'needed' for current disks. ---------------------------------------------------------------------- urglecom · 2021-04-28 06:51 Well, yes. That's why it only makes sense in some situations; those where the disaster-ness of the setup are outweighed by the extra space.