[HN Gopher] Thoughts on "The Theory and Craft of Digital Preserv...
       ___________________________________________________________________
        
       Thoughts on "The Theory and Craft of Digital Preservation"
        
       Author : todsacerdoti
       Score  : 39 points
       Date   : 2021-07-19 07:11 UTC (1 days ago)
        
 (HTM) web link (ajroach42.com)
 (TXT) w3m dump (ajroach42.com)
        
       | mftb wrote:
       | Long article, so I had to bookmark it for later, but I'm glad
       | this made it to the front page of HN. Digital preservation is
       | important to me.
        
         | rambambram wrote:
         | Also bookmarked, but also read 80% of it. The author is talking
         | terabytes of data, but I see some overlap with my own humble
         | 'backup flow' (as I call it).
         | 
         | In my case, I'm doing a couple of gigabytes max. I made a
         | distinction between my essential files on one big USB stick
         | (what I need to get going again in case of a broken computer),
         | and the bigger files like audio and pictures on another big USB
         | stick. I also use rsync (mostly) and a mysqldump per database,
         | all nicely in a bash script that I can quickly run but also
         | quickly change. The USB stick is FAT formatted and the contents
         | are not zipped or otherwise compressed. Just plain files as I
         | saw them on my origin computer, easily accessible with other
         | computers than my Ubuntu machines. And the most important part,
         | I can quickly verify the success of the backup with my own
         | eyes. Is that important file also backed up? Yes, it is!
         | 
         | I also decided on and implemented a 'backup flow' wherein I: -
         | set up new computers with the least amount of files copied from
         | the previous machine (this is done manually, and usually I take
         | one week in which I gradually transfer the files I need); -
         | just keep all files from the old computers on my backup USB
         | stick, so I have access when I need to and can forget it for
         | the rest; - just backup the new computer to it's own 'root
         | folder' on the USB stick, so old and new computers can have
         | their backups on the same USB stick (gives me more oversight
         | and makes it more portable for me).
         | 
         | Just my two cents. I'm very curious how other HN'ers handle
         | their backups!?
        
         | state_less wrote:
         | I'm starting to wonder if I've had a hobby of collecting
         | information and knowledge this whole time, and just now
         | realized what I've been up to.
         | 
         | I'm sorry if this sounds like it's coming out of left field,
         | but I really love you all, for all that you've shared with me.
         | Thank you.
        
       | adultSwim wrote:
       | I think preservation should be verifiable. ZFS, AWS S3, etc are
       | great and should be used, but not relied upon to prevent data
       | corruption. That should be a separate step on top of the storage
       | layer.
       | 
       | We can also do better than hashing by using error correcting
       | codes. If a file doesn't match a stored hash, you have to check
       | other backups to see if the file went bad or if the hash went
       | bad. ECC can do this without checking backups and has the ability
       | to "heal" a damaged file (see Reed-Solomon, which is used in RAID
       | 5/6).
       | 
       | Parchive/par2 is an off the shelf replacement/supplement for
       | checksums.
        
       ___________________________________________________________________
       (page generated 2021-07-20 23:03 UTC)