(Over)Thinking the perfect data storage. Share yours plz
reddit / selfhosted

----------------------------------------------------------------------
KotahLab · 2023-12-20 23:20

Hey all, I have an Intel nuc with Ubuntu - docker compose setup. It
runs a dozen of services via SWAG, including Home assistant, Jellyfin,
Nextcloud, Mealie and a few more.

So far I have just a 3TB USB HDD plugged into it to store large
amounts of data. I know it is not great, but so far it does the job.

On some point I want to upgrade it to something more robust and fault
tolerant. I mostly care about photos and videos.

What would you do (have already done)? USB Raid? Network Raid? Is it
possible to have HDD+SSD (reliability + speed) in the same RAID array?

Thanks :-)

----------------------------------------------------------------------
NikStalwart · 2023-12-21 00:56

> Perfect data storage

My perfect data storage would be multiple ceph nodes for high
availability and redundancy with tape library backup.

But I can't justify that for my infrastructure, so I content myself
with  normal spinning rust RAID (but I'll chuck in some NVMe disks for
cache when I upgrade my workstation from 2x TB to 2x 4TB).

I don't like the idea of USB storage  —  certainly not for something
that is meant to be always online. USB has bandwidth issues, has
longevity issues, too. If you want to store a lot of data, you need to
consider building a proper NAS box with multiple drives.

For more on this, talk to /r/datahoarder.

----------------------------------------------------------------------
KD_done · 2023-12-21 01:30

Okay.. what I got running is super overcomplicated and ridiculous, but
when it works it works like a treat. I'm sure that I gained a couple
of gray hairs.

I did this because I could, and it does have some benefits.. but..
sjeez, every single time something gets screwed up, I doubt myself, my
being, and curse every storage god alive (and dead).

NAS (today a synology, tomorrow something else if I the else is better
than what I have). I have a 200Gb volume that I offer to a ceph layer,
complimenten by 2 hardware servers (elswhere) and 5 VPS servers.. each
offering 200Gb.

The reason I chose for this is a VM running backuppc, that runs on the
NAS and only has to track one single environment. (and db servers dump
their dumps directly into the ceph storage too).

USB storage is used for travel, quick builds, clones and restores..

I agree with u/NikStalwart .. r/datahorder is way!

----------------------------------------------------------------------
SenritsuJumpsuit · 2023-12-21 06:49

You look cute with gray hairs so keep going :p

----------------------------------------------------------------------
ninjersteve · 2023-12-21 13:59

I’ll plug ZFS with however many drives you need in a raid-z3 pool. Or
just a 2 or 3 way mirror pool if the size of one disk is enough.

----------------------------------------------------------------------
KD_done · 2023-12-21 15:45

well, I think so too!! .. but not everybody agrees :/

----------------------------------------------------------------------
kmisterk · 2023-12-22 18:01

## Post Removed

Hey, KotahLab!

Thanks for your submission on selfhosted.

Your post has been removed due to a violation of the purpose of the
subreddit: **Self Hosting**

## Not Self-Hosted

When it comes to posts regarding applications in this subreddit, they
must feature a self-hosted tool, or a tool that *can be* self-hosted,
or some kind of related information, help request, or otherwise
related to a tool that is something that one can self-host.

If you believe this post *is in fact* relevant to /r/selfhosted, feel
free to [message the mods](https://www.reddit.com/message/compose/?to=
/r/selfhosted&subject=Removed For Not Being Self-
Hosted&message=[Removed
Post](https://www.reddit.com/r/selfhosted/comments/18n89ii/-/\)) with
your concerns.