[HN Gopher] Users reporting artifacts appearing in old images st...
___________________________________________________________________
Users reporting artifacts appearing in old images stored in Google
Photos
Author : duiker101
Score : 458 points
Date : 2022-09-25 12:15 UTC (10 hours ago)
(HTM) web link (support.google.com)
(TXT) w3m dump (support.google.com)
| 2OEH8eoCRo0 wrote:
| These artifacts aren't present when I download the original
| image. This gave me quite the scare though.
| varispeed wrote:
| Photos probably worn off from all that AI training /s
| uwagar wrote:
| ur photos are spam to the provider if u arent paying for storage.
| so they may compress them, reduce the quality over time and most
| may not notice.
| orangepurple wrote:
| RAID-Z2 is the long term solution for anybody capable enough of
| managing such a setup
| lostlogin wrote:
| At least one user on that thread is reporting it's fixed. It'll
| be interesting to see if all do.
| 12ian34 wrote:
| I've been long considering switching my Google Photos library
| fully over to ente [0], of course alongside my existing monthly
| backups to local HDDs and an annual backup to a cloud server via
| borgbackup.
|
| With every passing week, every Google fuck up like this and with
| ente's great pace of development, I draw myself nearer and nearer
| to the precipice of full switchover. I will soon be having my
| last Google Takeout.
|
| [0]: https://ente.io/
| DoingIsLearning wrote:
| Could you write a little bit about the workflow you described
| with borg?
| 12ian34 wrote:
| This great Hetzner page[0] explains it really well - its
| pretty much exactly what I'm doing. My Google Photos are
| extracted from the Google takeout.zip and those folders are
| backed up into a borg archive.
|
| [0]: https://community.hetzner.com/tutorials/install-and-
| configur...
| eliaspro wrote:
| The download button that's front & center on the homepage leads
| to a Google Play Store 404.
| vishnumohandas wrote:
| Hey, sorry, please check out our auto-updating APK[1] built
| straight from source, or our listing on F-Droid[2].
|
| It looks like Google has unpublished our app, will figure out
| a fix.
|
| [1]: https://ente.io/download/apk
|
| [2]: https://f-droid.org/packages/io.ente.photos.fdroid/
| Scene_Cast2 wrote:
| I remember trying ente, but the mobile app had a notoriously
| low amount of features and low FPS (felt like 30fps on my
| modern, 120Hz phone - stuck out like a sore thumb). Has there
| been any progress on that front?
| TingPing wrote:
| I think flutter is pretty bad, some animations feel off, but
| core functionality like scrolling through hundreds of
| pictures functions just fine.
|
| I switched to the service completely.
| Apocryphon wrote:
| Does Google Photos or ente actually use Flutter?
|
| edit: RTFA, guess ente does - https://github.com/ente-io
| vishnumohandas wrote:
| Hey! One of the founders of ente here. We have shipped a
| bunch of performance improvements and features[1] over the
| last few months. But we are far from where we want to be.
|
| If you happen to revisit and find something lacking, please
| write to vishnu[at]ente.io, we'd love to make ente work
| better for you.
|
| [1]: https://roadmap.ente.io/top-released/
| renewiltord wrote:
| I have my images synced to icloud and Google photos. It'll be
| interesting to get them out later and compare.
| fy20 wrote:
| Has there been any progress on self hosted alternatives to Google
| Photos in the past couple of years? Last time I checked there was
| great systems for individuals, but the main reason why I use
| Google Photos is so I can have shared albums with family &
| friends where anyone can add their photos.
| choppaface wrote:
| A little buggy but easy to run and very feature complete:
| https://github.com/LibrePhotos/librephotos
| jfoster wrote:
| I had noticed one of the images in my Google Photos account
| seemingly not showing up. It was very noticeable because it was
| one of the "People" images in my Photos account. It's a bit
| concerning that a photo storage service seemingly can't maintain
| the integrity of their data. Makes me wonder about what other
| data issues there are, lurking in the depths of Google Drive...
| insightcheck wrote:
| I personally found a corrupted file stored in online storage
| only, though it wasn't with Google, but with Microsoft's
| OneDrive.
|
| I tried to open a Word document that I hadn't viewed for
| several years, but got an error that the file was damaged. It
| looks like this happens sometimes, after searching for the
| error online. Luckily I was able to recover an earlier version
| of the file (via version history), but it was alarming that it
| happened in the first place (also I'm not sure if important
| edits were made in the latest corrupted version).
|
| It's unfortunate how many cloud services default to storing on
| the cloud only instead of also keeping a local copy, and don't
| even provide the option to opt out. Even if you choose the
| "keep offline" option for several services, for unknown
| reasons, this doesn't seem reliable (Maybe I'm 'using it
| wrong'? But in practice, I've found myself having to download
| files that I was sure I set to keep offline.)
| londons_explore wrote:
| Google is pretty proud of their record of not losing user
| data. As far as I know, they have never permanently lost even
| one byte of consumer data.
|
| The same isn't true for GCP - there they lost some writes to
| customers persistent disks - but only ~50 megabytes
| worldwide, which is still rather good when you consider they
| store millions of terabytes.
| KennyBlanken wrote:
| > As far as I know, they have never permanently lost even
| one byte of consumer data.
|
| Back in the aughts Google was infamous for disappearing
| mail, if not entire accounts just disappearing into thin
| air.
|
| https://techcrunch.com/2006/12/28/gmail-disaster-reports-
| of-...
|
| https://www.theinternetpatrol.com/has-your-gmail-email-
| disap...
|
| Years later:
|
| http://www.cnn.com/2011/TECH/web/03/01/gmail.lost.found/ind
| e...
| ocdtrekkie wrote:
| It really depends what you consider lost. Thousands of
| users (at least) have been permanently and eternally cut
| off from their Google data after some AI system decided to
| axe someone's access to their own account. The loss is
| significantly over a byte. Or 50 MB.
| jfoster wrote:
| > Google is pretty proud of their record of not losing user
| data.
|
| As far as I know, they just haven't admitted to losing
| consumer data. Until they define that and put even a little
| bit of effort into checking whether it might hold true,
| they don't really have anything to be proud of.
| KennyBlanken wrote:
| There have been multiple mass-deletion events and Google
| has admitted to at least one of them - the infamous "only
| 0.02% of our users" incident, for example - but only
| because so many people were affected, the press picked up
| on it and google's PR firms couldn't keep it quiet.
| dmw_ng wrote:
| They definitely had permanent data loss in App Engine
| around a decade ago, along with leaving any app active
| during the outage with copies of every database table to
| manually merge
| perlgeek wrote:
| > As far as I know, they have never permanently lost even
| one byte of consumer data.
|
| It seems that every other day you learn about people being
| permanently locked out of of their Google account. Every
| byte of data stored in there is permanently lost to the
| person losing their account.
| londons_explore wrote:
| Perhaps I should add... "except by policy".
| saltcured wrote:
| What if the policy is to lock accounts when data
| integrity errors are detected...
|
| "We have always had better storage durability than
| Europa."
|
| Note, I am not arguing that this is necessarily what
| happens. The thread full of contrary views on cloud
| storage just tickled this cynical take loose from
| wherever it was lodged.
| hedora wrote:
| It seems more likely that the consumer team isn't set up to
| detect data corruption events or process the resulting
| support tickets:
|
| https://duckduckgo.com/?q=google+drive+data+corruption
| crazygringo wrote:
| I'm mostly curious in what compression algorithm results in
| artifacts like these, and assume somebody here on HN recognizes
| them? But we're 225 comments in (so far) and not a single answer.
|
| They're absolutely nothing like block-based JPEG, that's for
| sure. When I inspect the images Google Photos serves me in my
| browser, it is serving up JPEG (from the response headers). But
| is this an artifact that shows up in AVIF or WebP? I wonder if
| mobile clients are getting a different encoding.
|
| The only objective thing I notice is that the artifact lines tend
| to track a line of constant _brightness_ , so you seem them
| appearing perpendicular to gradients of light and shade. And that
| each artifact line is black/dark on one side and white/light on
| the other -- and that the white edge seems to be in the direction
| of darkening, while the black is in the direction of lightening.
|
| Someone here who works with modern image codecs must be able to
| hypothesize what part of encoding/decoding must be bugging out
| here?
| a_e_k wrote:
| Looks to my eye like some 8-bit underflow artifacts that I've
| seen. And the way the line follows the gradient reminds me of
| when I've broken a gradient-domain Poisson solver to
| reconstruct an image from gradients. Maybe they're
| decorrelating the image data via gradients to compress better
| and then compressing those along with a sparse set of primal
| pixels?
|
| (See, e.g.,
| http://graphics.cs.cmu.edu/courses/15-463/2019_fall/lectures...
| for a nice overview of gradient-domain image processing.)
| viksit wrote:
| psa: noticed today that going to the original photo shows one
| that's not corrupted. only the resized algorithm seems to do this
| on lower dpi photos.
| llaolleh wrote:
| Not your server not your photos?
| carvking wrote:
| Looks like someone is doing some unauthorized time travelling.
| bilsbie wrote:
| Seems like the back to the future theory of time travel. Photos
| start fading.
| sitkack wrote:
| My first thought was that Google as recompressed everyone's
| images to save space and corrupted them in the the process.
| Hopefully everyone gets their images restored.
| cm2187 wrote:
| I would have rather thought it would be some sort of image
| enhancement algo gone wrong.
| IshKebab wrote:
| Yeah it's definitely some overflowing brightness enhancement
| thing. You can see that in the before/after photo that
| someone posted.
| vanderZwan wrote:
| I was thinking something similar: it has a distinct
| posterization look that I tend to get when I mess around
| with images and reduce the bit-width of my input a little
| too much and my integers overflow. But those tend to result
| in "zebra stripes" like patterns, so you're probably right
| that the issue is more one of not using saturating integers
| while amplifying a value above the bit-width of its
| container.
| [deleted]
| Shorel wrote:
| Every time Google tells me my 15 GB of storage is almost full,
| I download every photo and delete it from their servers.
| ReactiveJelly wrote:
| In the thread they say that the original files are available in
| Google Takeout, so it'll probably get fixed eventually.
| nullc wrote:
| Very weird looking artifacts-- they don't seem to be structured
| in any way that makes sense relative to the format of jpeg.
| scarface74 wrote:
| For context, I know this is a display issue and not a storage
| issue and you can get your originals out via Takeout.
|
| That being said my photos are the only digital files I really
| care about not losing. My photos and videos from my phone get
| backed up to iCloud, Google Photos, OneDrive and Amazon's photo
| storage that comes with Prime.
|
| My videos get backed up to all of the above except Amazon's
| storage.
| ghaff wrote:
| You should back them up locally as well even though that has
| its own failure points. I consider anything other than a
| dedicated local and/or cloud file backup system a nice
| supplementary place to store photos but not primary.
| scarface74 wrote:
| At one point I had them synced to my Windows computer via
| iTunes and that was backed up to Backblaze.
|
| But, I don't have a personal computer anymore and soon won't
| have a personal residence. My wife and I are going to be
| digital nomads traveling across the country for a few years.
| count wrote:
| 1TB thumb drives are tiny and can hold a TON of pictures.
| Cyph0n wrote:
| Same here. I lost some photos years back and decided to never
| make that same mistake.
|
| My backup flow is: phone, iCloud, local ZFS array served by
| PhotoPrism (synced using PhotoSync), Backblaze.
| natch wrote:
| Thanks for sharing. I would add a par2 in there somewhere.
| Not that my system is as good as yours yet.
| Cyph0n wrote:
| The local ZFS array is the "source of truth". ZFS has
| built-in end-to-end checksumming. I also have a monthly
| scrub that should correct any untouched corrupt blocks.
|
| Details on ZFS checksums:
| https://openzfs.github.io/openzfs-
| docs/Basic%20Concepts/Chec...
| woliveirajr wrote:
| Old problem. If you have only one copy, you have none...
| thriftwy wrote:
| Looks like fractal encoding going bad.
| epinephrinios wrote:
| That's why I use Dropbox. They added a lot of new Photos
| features.
| aceazzameen wrote:
| I use Dropbox and Google Photos. Photos for sharing and Dropbox
| + local backups of the Dropbox folder for a more permanent
| storage solution.
| nharada wrote:
| Seems unlikely Dropbox would be immune from some sort of issue
| like this. Probably best to have your photos in storage
| somewhere else (ideally both locally and off-site).
| agumonkey wrote:
| Impressive compression glitch. Bitrot evolved.
| philips wrote:
| I keep intending to launch a service that downloads cloud data
| and sends it to you on an SD card.
|
| I hope to fix up my prototype and launch it next year.
|
| https://www.clonecamel.com/
|
| If anyone knows of a quality source of bulk SD cards or flash
| drives in high capacity please let me know.
| ravel-bar-foo wrote:
| The susceptibility of the SD card to data corruption is a big
| showstopper. The only digital files I've had corrupted were
| stored on my phone's external SD cards (after 3 to 7 years).
| When I looked into it, it seemed to be an issue related to lack
| of radiation hardening. So if you do make this service, please
| include some error detection/error correction metadata.
| philips wrote:
| That is good feedback
|
| The idea would be to subscribe to have this data shipped once
| or more per year so hopefully some of the data would survive.
|
| Any advice on error correction metadata and tools?
| amelius wrote:
| This is an indication that Google underestimates the value of
| data stored by users.
|
| E.g. if this was about storing monetary data, this would never
| have happened.
| tmpz22 wrote:
| > E.g. if this was about storing monetary data, this would
| never have happened.
|
| Let's hope none of the images were NFT "assets" then haha
| johnywalks wrote:
| > This is an indication that Google underestimates the value of
| data stored by users.
|
| That becomes apparent if you ever had any issue with a google
| product. There's no way to resolve issues outside of canned
| answers from "AI" systems and public forums.
| ztqejq wrote:
| Or maybe you overestimate the data of users?
| amelius wrote:
| No doubt the average value of data is not very high. That
| doesn't mean it also holds for the most valuable data stored.
| Jabbles wrote:
| This story from last week seems to indicate that that is not
| true, though it hardly makes Google look perfect!
|
| https://twitter.com/samwcyo/status/1569897392560050178
|
| Discussed in https://news.ycombinator.com/item?id=32835190
| type-r wrote:
| most users aren't paying for Google Photos. sounds to me like
| they're accurately estimating the value.
| anotheraccount9 wrote:
| Do you have any suggestions for a secure/private option to
| store photos? I'm not sure what would be best. (Dang, I really
| want to move away from Goo now)
| woleium wrote:
| Amazon photos is pretty good as a like for like alternative
| khimaros wrote:
| Photoprism
| layer8 wrote:
| Managed Nextcloud hosting in the EU is an option.
| 29083011397778 wrote:
| I stood up a pair of QNAP NAS for both my parents and I, and
| encrypt-then-backup their photos/documents to my NAS, and
| vice-versa. RAID 5 or 6 keeps hardware failure at bay, and
| keeping them tucked away in our respective homes makes fire
| or flood irrelevent (for this).
|
| QNAP or Asustor has standalone apps my parents can use as
| well. Without getting your parents involved, you could either
| pay for remote hosting for your off-site, or trade RAID space
| with a friend.
| hedora wrote:
| Definitely use client side encryption, if only because it
| defeats attempts to "repair" image data or rewrite exit
| headers, etc.
|
| Another good test: upload your photo collection, then
| download it and binary diff the two sets of files.
| SkyPuncher wrote:
| Dropbox, Box, a Microsoft service.
| slazaro wrote:
| Your own devices, with a proper backup solution (which you
| should already have with or without your photos in it).
| Nobody can mess with your data then.
|
| Personally, I use Syncthing to sync between devices,
| including a NAS as a target, which regularly backs up to
| Backblaze.
| lostlogin wrote:
| This sounds a bit like what I do. I r-sync two different
| libraries to NAS then upload them to Backblaze B2. Post
| r-sync success result to a muted Slack channel and failure
| to an unmuted alert channel.
|
| rsync -av /Source /Destination rsync -h
| kijin wrote:
| As a starter, look for a "dumb" service that doesn't treat
| photos differently from any other type of binary blob. No
| thumbnails. No grouping by date or location. No tagging other
| people. The storage service should only concern itself with
| safekeeping the binaries you upload, not analyzing and
| processing them. Use your own machine to do the latter.
| lostlogin wrote:
| Backblaze B2.
| victorbstan wrote:
| iCloud
| celsoazevedo wrote:
| iCloud may be secure, but private? No end-to-end encryption
| and Apple scans the user's content. It's no better than
| Google Photos in that regard.
| deepersprout wrote:
| I'm using Nextcloud. Works great.
| yawz wrote:
| Do you self-host it? Do you access it with Android and iOS?
| milosmns wrote:
| It's naive and/or biased to think that. Everyone makes
| mistakes, everywhere, all the time.
| amelius wrote:
| That's not the point. I doubt that the engineers who worked
| on the compression algorithms would store their own precious
| photos using the same techniques.
|
| Also, when was the last time you heard a banker say that
| everyone makes mistakes all the time.
| mgraczyk wrote:
| Hi, I work on the processing algorithms at Google like the
| one here (not sure about this particular issue) and I use
| Google photos for all my old photos. People at Google read
| Hacker News and fix things, they just don't comment on
| specifics for things like this.
| ouid wrote:
| That's a fairly serious accusation to level. Google's bad
| behavior here is not that their compression algorithm ruined
| users photos. Google's bad behavior is running a compression
| algorithm at all.
| mgraczyk wrote:
| Every photo sharing and storing site compresses photos for
| view. The originals are still there (read further down in
| the issue).
| LionRoar wrote:
| I find the idea of old photo/data degradation strangely
| comforting. Even digital data seems to be unable to escape the
| law of entropy (don't mention the black holes). On another note;
| if corrupting data/photos was a wide occurrence now wouldn't we
| have heard about it in a big way?
| choppaface wrote:
| One of the best ways to deal with Google customer service is to
| simply send in a support case and immediately dispute the credit
| card charge. I've done this with Youtube TV several times (wrong
| recording, live the doesn't work, their location check locked me
| out) and got free months of service.
|
| Don't bother getting all frustrated in a big support thread. The
| engineers who caused these bugs DO NOT care about you. They are
| way way too busy solving dynamic programming problems.
| Chargebacks are the most effective tool here.
| OrangeMonkey wrote:
| I hope to god noone does this.
|
| You are flirting with having your entire google account
| disabled. As this may end up un-personing you, I would suggest
| strongly taking another route.
| esprehn wrote:
| What have you done a chargeback for with Google? There's
| stories of folks doing a chargeback and then having their
| account closed afterwards meaning you lose Gmail etc.
| nathanaldensr wrote:
| You know how the mantra in the crypto community is "not your
| wallet, not your coins?" This applies to this situation. Once you
| trust Google with your bits, don't be surprised when they modify
| them without telling you.
| jonnydubowsky wrote:
| Us this the beginning of some Back To The Future moment in this
| timeline?
| TazeTSchnitzel wrote:
| Could it be some sort of clipping/banding issue when
| decompressing? The way it follows curves would be unusual for
| photos corruption.
| gbasin wrote:
| Nothing a little Stable Diffusion can't fix. Soon, digital
| memories will be just like real ones -- slightly different every
| time we look back, but with no way of knowing what's changed
| brador wrote:
| > with no way of knowing what's changed
|
| You can just hash them locally or use AI to identify
| modifications.
| ReactiveJelly wrote:
| The promise of the cloud was that you don't need local
| storage, because Someone Else will remember things for you,
| and you just ask them what you remember. This is also the
| problem of the cloud.
| rsync wrote:
| "The promise of the cloud was that you don't need local
| storage, because Someone Else will remember things for you
| ..."
|
| Agreed. And that promise has been fulfilled - although it
| costs _more than nothing_ and you need to do your homework.
|
| You _know_ these providers are proven bad actors. You
| _know_ their business model(s) are antagonistic to your
| goals. You also know there are stable, time-tested
| alternatives.
|
| _What did you think was going to happen here_ ?
| sgregnt wrote:
| I would say at least partially aligned with your goals
| otherwise why would you use them in the first place?
| Godel_unicode wrote:
| They're unaligned with rsync.net's goal of getting people
| to pay for services.
|
| Edit: to pay a minimum of $12/month??
| johnnymorgan wrote:
| Exactly..what is proof of existence for $400 Alex
| gbasin wrote:
| ok and then what? :D you still can't recover original
| emptyparadise wrote:
| That's already what browsing most websites feels like.
| craftuser wrote:
| How can your Photos library be real if our eyes aren't real? /s
| rsync wrote:
| "Soon, digital memories will be just like real ones -- slightly
| different every time we look back, but with no way of knowing
| what's changed ..."
|
| Well, let's see:
|
| ssh user@rsync.net sha256 photos/IMG_1234.HEIC
|
| Yep, checks out.
| low_tech_love wrote:
| Expecting general users to use checksums is exactly the
| answer I expect from HN.
| bitexploder wrote:
| A solution around archive fidelity isn't far from this idea
| though. And how hard is it to sling a find command to
| collect all your photo hashes?
| pdntspa wrote:
| Extraordinarily difficult, judging by GP's cynicism.
| Personally it is appalling just how little people are
| motivated to be nomically tech-literate in this extremely
| tech-driven society.
| Godel_unicode wrote:
| Tech is the only industry that asks this of it's users,
| it's just laziness on the developers part. People needing
| to know about hashing in order to save photos would be
| like people needing to know what refrigerant their A/C
| uses in order to keep cool.
|
| Can you? Sure. Are their weird edge cases where it's
| mildly helpful to know as opposed to having to ask
| someone? Debatably. Should you need to in order to not
| have things break for no reason? Absolutely not.
| colinsane wrote:
| why do you assume GP is talking about general users? as you
| point out, this is HN: it's reasonable to think that most
| comments here are written for that mostly-tech-literate HN
| audience, unless they state otherwise, no?
| Kuinox wrote:
| Sadly, you stored only the MD5 back then.
| throwawayallday wrote:
| ZFS defaults to fletcher4 for checksum. It has never had
| MD5 support.
| userbinator wrote:
| MD5 is weak against malice but still much stronger than
| something like CRC if your goal is to detect random
| corruption.
| charcircuit wrote:
| No, it's not. MD5 can still protect you from someone
| modifying the file without changing the hash.
| rsync wrote:
| We don't store any checksums - we just store your files.
|
| The checksum commands:
|
| (md5, sha1, sha224, sha256, sha384, sha512, sha512t256,
| rmd160, skein256, skein512, skein1024, cksum)
|
| ... are available to run as you see fit.
| derefr wrote:
| I think you misinterpreted the comment you're responding
| to; they meant that "sadly, you [the user] never thought
| to store a high-quality hash (e.g. SHA256) for integrity
| comparison with your stored data; and the hash you _did_
| decide to archive for integrity-comparison was a low-
| quality one (e.g. MD5) that can be trivially preimage-
| attacked such that a cloud provider could silently
| replace your data with a different one -- with the same
| low-quality hash -- without your knowledge. "
|
| Nothing about rsync.net per se, other than the general
| idea that any data you put on "somebody else's server"
| can't be trusted to stay the same if you don't have a
| _high-quality_ integrity-comparison content-hash of that
| data kept somewhere.
| userbinator wrote:
| MD5 still has no known preimage attacks. Neither does
| MD4, for which collisions are absolutely trivial to
| generate.
| godelski wrote:
| > Nothing a little Stable Diffusion can't fix.
|
| As someone who works in generative modeling, that's probably
| what is creating these issues in the first place. Kinda like
| how you can see certain GAN type of patterns on Netflix when
| there is upscaling or if you don't turn off the fancy features
| on your TVs. Of course, finding how these images are corrupted
| leads to better models so they will be solved. This isn't the
| first time Google has edited photos and made them worse and it
| won't be the last. But the trend is that the photos get better.
| morley wrote:
| > slightly different every time we look back, but with no way
| of knowing what's changed
|
| This is already the case with different image compression
| algorithms improving on one another. Users couldn't and
| shouldn't tell the difference between JPG, or slightly smaller
| / resized JPG, or WebP, or whatever. I personally don't think
| image hosts should have to alert users when making undetectable
| changes to their images like this.
| soulofmischief wrote:
| I'm sorry, but JPEG compression is extremely noticable
| especially with non-photographic material such as web
| graphics. The compression is lossy by nature, and predictably
| introduces noticeable artifacts. WebP on the other hand has
| support for lossless encoding.
|
| It's simply not the same thing. If I opened my file browser
| one day and saw that all of my FLACs had been converted to
| MP3, I would go on a murder spree.
| zitsarethecure wrote:
| Once a year I select about 100 of my favourite pictures from my
| phone and have them printed and bound to a book. I suspect this
| is likely to be the only way my descendants will be able to see
| them.
| hef19898 wrote:
| That's why I, and every other enthusiast photographer I know,
| keep my archive locally, meaning on my laptops hardrive and an
| external one as backup. You cannot trust any cloud provider to
| properly maintain your data, and external storage is becoming
| cheaper every year.
| iLoveOncall wrote:
| > You cannot trust any cloud provider to properly maintain
| your data
|
| I trust any cloud provider a lot more than my lazy self.
|
| > and external storage is becoming cheaper every year
|
| Sure, but electricity isn't. At the current price in the UK
| it seems like running a small NAS would be 10 GBP per month,
| more than most online storage solutions.
|
| That is without counting the initial investment of buying the
| NAS, the cost of maintaining it (if I have to spend an hour a
| month on maintenance, that's $1000+ a year of time wasted),
| the footprint of the NAS, the noise, the heat, etc.
| [deleted]
| pixl97 wrote:
| Why not both.
|
| A backup in one place isn't a backup. The 3, 2, 1 rule and
| all.
| iLoveOncall wrote:
| Of course, but then my solution will be an external
| drive, not a NAS.
| ghaff wrote:
| Yeah. I have a NAS for other purposes but redundant
| external USB drives is the easiest solution. You can even
| periodically make a clone and store it somewhere else.
| ocdtrekkie wrote:
| It sounds like you know... very little on the topic. My NAS
| requires no maintenance, it makes no sound while running,
| and costs maybe pennies to run.
|
| Meanwhile, there hasn't been a cloud provider without
| massive reliability or data loss issues. My NAS has better
| uptime than Amazon, Microsoft, or Google.
| SECProto wrote:
| Their estimate of PS10/month seems reasonable to me -
| that's basically 1 or 2 kWh of electricity per day (can't
| find current UK electricity rates). Is your NAS under 40W
| power draw?
|
| > costs maybe pennies to run.
|
| Pennies per hour, per day, or per month?
| tpxl wrote:
| My RPi 4b runs on a 18W power supply, which would put it
| at 432Wh per day if running at max load (It doesn't, it
| idles at <3W most of the time). Any arm NAS is going to
| have a similar power draw, and you really should pay
| attention to the power draw when choosing a NAS.
| iLoveOncall wrote:
| > My NAS requires no maintenance
|
| So you took it out of the shipping box, plugged it to
| power, and pressed the power button, and everything was
| up and running?
|
| And since then you haven't had to update any software on
| it, reboot it, unplug and replug an ethernet cable, shut
| it down and opened it to add storage, etc.?
|
| > it makes no sound while running
|
| Looking at a basic NAS on Synology shows that it's around
| 20 dB, which is admittedly very little, but is not 0.
|
| > and costs maybe pennies to run
|
| Again, looking at the same NAS and using my actual price
| per kWH, it's around $10 a month.
|
| > there hasn't been a cloud provider without massive
| reliability or data loss issues
|
| My searches haven't turned up any article about a data
| loss from any of the 3 big cloud providers. Even the
| story here, while scary, is just about compressed files,
| not originals.
|
| ---
|
| I'm not denying that a NAS has advantages (the fact that
| it's a lot more than just a data backup), but in the
| context of backing up files, it's not at all convenient
| compared to using online solutions.
| tagawa wrote:
| It's not big three but Canon announced cloud image/video
| loss a couple of years ago:
| https://www.canonrumors.com/canon-updates-on-the-status-
| of-i...
| Finnucane wrote:
| Now imagine you are old like me and got your first camera in
| 1972, and still have all the film.
| emkoemko wrote:
| what happens when your house floods, burns down etc? you need
| to keep another backup off site somewhere, i keep it at my
| parents house just in case.
| flatiron wrote:
| As an enthusiast photographer with a lot of inner circle
| friends: we encrypt and backup to cloud. 0 degradation.
| dv35z wrote:
| Can you share some specifics on the setup? Trying to do
| something similar. Haven't tried it yet, but MacOS can
| create encrypted "sparse images", which can be mounted and
| contain files. I'd like find to a better way, especially
| something that could stand the test of _some_ time (10
| years seems reasonable).
| newZWhoDis wrote:
| > You cannot trust any cloud provider to properly maintain
| your data, and external storage is becoming cheaper every
| year.
|
| Exactly, and setting up a NAS has never been easier.
| reaperducer wrote:
| For 99% of people, a NAS is overkill. Plug-and-play USB
| drives get the job done at a fraction of the complexity.
| pronoiac wrote:
| One benefit of a NAS: my data's robust against any single
| drive dying, which is something that's happened to me,
| friends, and family.
| nobodyandproud wrote:
| A toaster box NAS with hotswap drives, RAID, and support
| for Android and iOS is a reality.
| newZWhoDis wrote:
| I love having a NAS that replaces Time Machine, Dropbox,
| iCloud, Netflix, Spotify, Disney plus, Hulu, runs home
| automation, has my security system on it, etc etc.
|
| People use so many cloud services and having it all
| private and self-hosted is absolutely amazing. It's
| really not that hard to set up either, and there's little
| to maintenance outside replacing a failed drive.
| haidev wrote:
| Mind sharing your setup, I'd love to replicate it. Thanks
| type-r wrote:
| I vastly prefer content be cloud based. Ever since
| Spotify came out, I've never downloaded music illegally
| again -- it's just so much better and more convenient.
| Can't imagine going back to what.cd days and having to
| download and maintain a music collection manually.
| newZWhoDis wrote:
| Sure, but then Netflix removed stargate on a whim and one
| of my favorite shows went poof.
|
| That started me down the path to owning all my own media.
|
| It also didn't help that iCloud Photos is a privacy
| nightmare.
| criddell wrote:
| People here likely know how to do all that safely, but
| lots of less technically inclined people end up with
| their data encrypted and have to choose between starting
| over or paying a ransom. I have a QNAP device so I follow
| their subreddit and there's a steady stream of people who
| have lost everything to ransomware because their NAS was
| visible from the internet. I follow the standard advice
| from that subreddit - keep it behind a firewall with UPnP
| turned off and no ports forwarded to the NAS and never
| enable MyQNAPCloud (or whatever it's called).
| newZWhoDis wrote:
| Seems like more of a QNAP problem than anything
| sockaddr wrote:
| Yup. QNAP software quality (and hardware) is complete
| garbage. I've off and on tried to incorporate them into
| my "system" and they always let me down. You're putting
| everything at risk using one of their boxes.
| madiator wrote:
| What will you do if there is a fire and you lose your laptop
| and the external hard disk?
| brycewray wrote:
| That's why to use the 3-2-1 method[0].
|
| [0]: https://www.backblaze.com/blog/the-3-2-1-backup-
| strategy/
| iforgotpassword wrote:
| A photo book doesn't sound much better for that either.
| [deleted]
| Pxtl wrote:
| I do both, personally. Have everything on my HD and on the
| Google cloud. That seems like an appropriate level of
| redundancy.
| shaky-carrousel wrote:
| Have a backup somewhere else, of course.
| retrac wrote:
| For the really important digital stuff (almost 30 years of
| family digital photography and video and some paperwork) I
| have a rotating backup schedule. Two of the drives in the
| rotation stay at my mother's place. Nothing fancy and it's
| mostly when I can bother to remember. About once a year or
| so. Still, some peace of mind.
| hef19898 wrote:
| Law of propability: I had zero dataloses due to fire, some
| data loss due to loosing a thumb drive backup and some data
| losses due to cloud storage shennenigans.
| madiator wrote:
| Hmm. I highly recommend a three way redundancy as others
| have mentioned. Put a copy in the cloud or store your
| harddisks somewhere else.
| hef19898 wrote:
| You are right, a cloud storage would make things a tad
| saver. Personally, I made a point in not putting anything
| in the cloud, especially none of the big providers, since
| I value my privacy and just don't trust any o those
| companies.
| capableweb wrote:
| Create a new backup from the cloud provider one. Vice-versa
| if the cloud provider backup gets lost. Chances of loosing
| both at the same are minuscule.
| 2OEH8eoCRo0 wrote:
| I like that. Convenience of phone camera + curation, quality,
| and resiliency of physical media.
| TigeriusKirk wrote:
| I'd like to do this myself. What service do you use for it?
| ThrowawayTestr wrote:
| Check your local independent print shop. They're not
| expensive.
| djbusby wrote:
| rsync.net is my off-site/cloud of choice. And they keep your
| bits on ZFS.
| ipaddr wrote:
| What service do you use?
| ilaksh wrote:
| After a few hundred years, most of your descendants may live in
| a computer.
| LightG wrote:
| They're going to be p!ssed at degradation issues like this
| one.
| Shorel wrote:
| After a few hundred thousand years, sure.
|
| Nowadays it's only available inside the world of Black Mirror
| and similar fiction.
| bavell wrote:
| Even after a few millennia, I doubt it. The human experience
| is inseparable from the human body.
| Sharlin wrote:
| That's a pretty bold statement. The null hypothesis is that
| there's nothing in the body that cannot in principle be
| emulated _in silico_ to whatever precision necessary.
| jodrellblank wrote:
| Why is that the null hypothesis? For one thing, the only
| experience we have is existing as bodies in a real world,
| so that should be the default with any other claims being
| supported by evidence. For another thing, simulating a
| bridge falling down is not a bridge falling down, and
| it's not proven that simulated thinking is thinking (or
| even, possible).
| [deleted]
| Traubenfuchs wrote:
| What makes you say that? You could literally be a brain in
| a jar or be hooked up to god knows what "deep diving" /
| "full VR" experience humanity will come up with.
| implements wrote:
| Sad news for people hoping for a life after death, though!
| (ie nearly everyone)
|
| I think it'll be possible to create inorganic 'brains' that
| have human-like subjective experience, because to think
| otherwise reduces to some form of vitalism, but I doubt
| we'll be able to move specific individual consciousness
| across technologies.
| Semaphor wrote:
| > (ie nearly everyone)
|
| Are you sure about that? That does not sound right to me.
| implements wrote:
| Well, I'm assuming the majority of people currently alive
| (and even more of those who have ever lived) are
| religious, and the majority of religions followed hold
| with the Heaven & Hell / Cycle of Reincarnation until
| Enlightenment model - and that experience in the
| afterlife was still recognisably 'human'.
|
| Though, saying that, OP's point about needing a human
| body to have human experiences may be correct if souls
| get a new human body in the afterlife - I'd just assumed
| liberated souls, like ghosts, had no physical form.
| Semaphor wrote:
| There's a vast 49% gulf between nearly everyone and the
| majority.
| KronisLV wrote:
| > The human experience is inseparable from the human body.
|
| I'd say that the human body is just one very sophisticated
| I/O device.
|
| The fact that we can't replicate it now doesn't mean that
| it wouldn't be possible after a millenia of engineering,
| even though this replacement device might as well be a
| partially organic one in the end.
|
| That said, there are no guarantees that we don't fall into
| a sort of "dark age" of no more growth/research into any
| new directions, or don't wipe ourselves out altogether.
| londons_explore wrote:
| > printed and bound to a book. I suspect this is likely to be
| the only way my descendants will be able to see them.
|
| My experience is that when someone dies, all their paperwork
| gets left in a drawer for years until the next house move and
| then it's all thrown out.
| renewiltord wrote:
| The point is for someone to discover them years later and
| post them on Reddit.
| rasz wrote:
| Sometimes a good friend will dig into the garbage and rescue
| them https://www.youtube.com/watch?v=hwu1427MUIE
| reaperducer wrote:
| _My experience is that when someone dies, all their paperwork
| gets left in a drawer for years until the next house move and
| then it 's all thrown out._
|
| My experience has been the opposite, with relatives eagerly
| snatching up the old photographs.
|
| My boss is young and one of those people for whom digital is
| the only way. Then one day she got a box of old photographs
| and was absolutely amazed that she could turn them over and
| read where they were taken, on what occasion, and who was in
| the photos.
|
| She now prints out all of her photos and writes notes on the
| back of them and keeps them in a box, just like people have
| done for the last century.
| jvm___ wrote:
| I was on the subway in Toronto in 2017, two college age
| girls had an inch thick stack of 4x6's that they were going
| through. It was a strange throwback to the pre-digital era.
| josefx wrote:
| My family has/had a handful of generations worth of records.
| The problem is finding out who last had what. Some of it has
| been backed up digitally, but finding a thumb drive is a lot
| harder than spotting a man sized pile of boxes.
| justinzollars wrote:
| flakeoil wrote:
| But at least people will notice the photo book and at least
| have one look through. And it might even be a great moment
| when family members together can look at it and discuss
| memories. And quite likely it will be kept by a descendant.
| And it can be found one day by their descendants.
|
| The digital archive or cloud storage will likely in many
| cases not even be discovered. And if it is discovered, it
| will probably be such a load of images, good and bad, that
| no-one will have the time to really look it through and
| appreciate it.
|
| But for sure, both options combined is probably best.
| dazzawazza wrote:
| Don't worry, the images will be used to train an AI which
| will feed derivations of them into a replicant to keep them
| sane.
| debaserab2 wrote:
| My gmail account has a deadman's switch to grant full
| access to my next of kin.
|
| If I had a gigabytes of photos of my grandparents or great
| grandparents I would spend hours pouring over them looking
| at what their life was like before I knew them. That would
| be such a neat keepsake.
| hezralig wrote:
| I hate to tell you this but many people have been locked
| out of their Google Accounts despite knowing the correct
| username/password combination. Their security policies
| and ML algo locks accounts and requires 2FA to be used
| from time to time. How confident are you that in your
| passing, your next of kin will be able to get into that
| account if 2FA or backup codes are inaccessible?
|
| Given their is no real customer service team that handles
| their free tier accounts, I am personally reluctant to
| assume my Gdrive data will be salvageable.
| debaserab2 wrote:
| According to the Google Inactive Account support page,
| they aren't actually assuming the account, they're just
| getting access to the data. So, presumably, they'd just
| download the data and that would be it.
|
| https://support.google.com/accounts/answer/3036546?hl=en
|
| (That's not to say there aren't other concerning reasons
| why that data may not be accessible.)
| vineyardmike wrote:
| I imagine someone may one day do the same for me, but
| sadly most of my photos are clogged with mundane
| screenshots and repeint pictures and other junk.
| noneeeed wrote:
| My wife does a photo book every year, combining the best of
| both our photos. It's a bit of a chore, but I'm so glad she
| does it, they are so much nicer to browse through. The kids
| love looking at them all from time to time.
| Forge36 wrote:
| I've started picking my favorites and printing on metal/canvas.
|
| My home backup is going to be it's own problem after my death.
| rasengan0 wrote:
| This is normal. I found out that Google Photos has limited
| reliability. My 2007 photo was garbaged into a lower resolution.
| I have since recovered from the original drive. Back then Google
| Photos gave an option to store in lower res but still sufficient
| high quality or original quality. Don't trust either. Lesson:
| backup to your media you own or print it.
| mwambua wrote:
| Yikes, this is kind of scary. Google Photos is the only place to
| find some of my older photos. What's the easiest way to download
| my entire library for backup on a physical drive?
| iforgotpassword wrote:
| Takeout
| willcipriano wrote:
| I do takeout once a year for such an eventually.
|
| https://takeout.google.com/
| mwambua wrote:
| Ah, awesome! Thanks!
| buro9 wrote:
| I used this not too long ago:
| https://github.com/gilesknap/gphotos-sync and it stores photos
| and videos in a YYYY/MM directory structure.
|
| It works _extremely_ well, and you can re-run it any time to
| sync new files locally too.
|
| I used it like that for a few months before I finally installed
| syncthing on my phone and stopped using Google Photos
| altogether.
|
| Now what I do is take photos on my phone, have them sync to a
| NAS. And on the NAS I used a modified version of this
| https://forum.syncthing.net/t/android-photo-sync-with-exifto...
| to build up a YYYY/MM folder organisation and move files older
| than 30 days from the syncthing folder into my archive. My
| archive is then in my Plex so it's still accessible to me.
|
| In essence: 1) Take photo (implicit sync to NAS), 2) wait 30
| days, 3) archive photo into long term directory naming
| convention, whilst making available to Plex and deleting the
| version from my phone (by deleting the syncthing version it
| will delete the one on the phone after 30 days too).
| exhilaration wrote:
| Do you know if gphotos-sync downloads the photos in their
| original quality/version?
| buro9 wrote:
| It does.
|
| The only thing that it gets wrong is that it cannot restore
| the file created date to video. But in the grand scheme of
| things, that's a nit.
| exhilaration wrote:
| Unfortunately it looks like it's subject to the same
| limitation of the Google Photos API: _Photo download
| compresses the photos even if you ask for the original
| file_ https://gilesknap.github.io/gphotos-
| sync/main/explanations/g...
|
| I've been putting off a Google photos back up for years,
| and this was my excuse - waiting for this issue to be
| resolved. But I'm being a fool, 60% compression is better
| than losing everything.
| cj wrote:
| Google Photos used to have the option to sync all photos into
| Google Drive but that feature was removed. I'm guessing the raw
| data is still available via Google Takeout
| squeaky-clean wrote:
| Only tangentially related but I just checked the Google Photos
| iOS app to see if any of mine are corrupted, and it won't even
| allow me to view my cloud photos without giving it full
| permission to all current photos on my device?! Isn't this
| against the app store policy? You can't deny me access to one
| feature just because I refuse to grant permissions to a different
| feature.
|
| But because it's Google I'm sure they can break whatever rules
| they want as long as it doesn't involve trying to avoid Apple's
| 30% fees.
| Macuyiko wrote:
| Each passing month I am more and more convinced that I should
| _urgently_ get a) my mails out of my personal (non paying)
| Gmail account and b) get my associated photos out of Google
| Photos, mainly because of baby pictures which I cannot fathom
| to lose.
|
| But then I think about the convenience of my 2TB Google Drive
| and I can't do it.
|
| And I think, too, of the Polaroid pictures my parents took and
| gave me in a box once I got married and are now fainted.
| Mayhaps algorithms deciding that it's time to let go of your
| own data is just the modern form of yellowing.
| lozenge wrote:
| You can put those on a service like AWS Glacier for next to
| nothing, the only challenge is keeping your billing details
| current.
| igorkraw wrote:
| Smugmug pro my friend. Not even paid to shill, but it's
| reasonably priced for complete photo -freedom for me
| pdntspa wrote:
| Mirror them to a external 2TB hard drive?
| cobertos wrote:
| a) was very freeing. I didn't realize there were services so
| much better than Gmail. Even ones I pay a small amount for
| and get as many custom domains as I want. b) did this a long
| time ago. Glad I did. I can sync any file I want too, not
| just photos. Really been liking Seafile + AutoSync (android)
| for this.
| metters wrote:
| > I should urgently get a) my mails out of my personal (non
| paying) Gmail account and b) get my associated photos out of
| Google Photos, mainly because of baby pictures which I cannot
| fathom to lose.
|
| I highly recommend it. It feels like pure freedom, putting a
| price tag on my personal data. The migration is something
| done in two steps: set up your alternative service, but also
| keep the old one (gmail/gdrive/calendar/...) running. Then
| start to use the new email service, while keeping the old
| one.
|
| There are enough affordable options and you do not have to
| migrate everything at once. I started with my emails - gmail
| is still running and forwards all incoming emails, so I wont
| miss anything. I already updated all my accounts/logins to
| the new email address.
|
| The next step was copying all my files from Dropbox, GDrive,
| and so on to {service I am using}. The data in the other
| clouds are still there, but I am going to delete it all soon.
|
| There are things that are not perfect - like accepting
| appointments, which I still want to show up in my Google
| calendar. To those mails I still have to respond in google
| mail. But I am getting there, one step after another. The win
| so far is much bigger than the loss/pain.
| Arnavion wrote:
| I take periodic backups of my Google data using Takeout,
| primarily for the GMail backup. When I took one a few months
| ago, I noticed that the Drive part of the backup had some
| mangled filenames that the backup from a year ago didn't
| have. I don't remember exactly what it was but it was
| something like long-filename.pdf got truncated to long-
| file~01.pdf, like back in the 8.3 DOS days. Luckily I already
| have a local copy of the Drive contents so I didn't need to
| rely on it, but anyone else should be worried.
| choppaface wrote:
| This is an engagement hack. Get the user to give max
| permissions at the start so that any future call to action will
| have less friction and/or not require a modal.
|
| It's less about Google wanting to scan your data today and more
| about Google's stingy strategy for photos plus Google's lazy
| engineering (too many other interesting games to solve). Google
| Photos was born out of Google Plus, and is under heavy revenue
| efficiency pressure.
| sixothree wrote:
| And it will spend its spare time analyzing every single photo
| in your library.
| smoldesu wrote:
| Google doesn't scan photos that don't exist on their servers.
| choppaface wrote:
| Which is why it's obnoxious to ask permissions for all
| photos.
| junon wrote:
| Fun fact, Xiaomi does by default on their stock OS.
| coffeeblack wrote:
| Who would allow the CCP to scan their device?
| midislack wrote:
| What difference does it make anyway at this point?
| Everybody does it.
| junon wrote:
| There wasn't anything to deny. There's no way to opt out.
| soulofmischief wrote:
| ... currently.
|
| Trusting a dubious central authority up until you get
| burned is a bad heuristic.
| smoldesu wrote:
| I agree, which is why I don't pay for cloud storage.
| bbarnett wrote:
| Especially when Google has said "we won't do thing X",
| then been caught already doing it, or just silently
| starting to do it.
|
| Just location tracking. The lies. Over, and over, and
| over, and over again the lies.
| maxbond wrote:
| Institutions of that nature are incapable of following
| through on nearly any promise made to the public at
| large; every promise is a lie, whether they break it or
| not, because they can't be said to have the intention to
| uphold it.
|
| I see no reason not to believe that Page and Brin
| sincerely desired Google not to be evil; but it's the
| institution decides to do evil, not any individual, so
| they did not have (or could not maintain, as the
| organization scaled) the power to guarantee this. No one
| does.
| sgregnt wrote:
| What's the difference? Potentially apple itself can
| fecide to scan all your photos on the phone....
| soulofmischief wrote:
| I don't use an iPhone, and I use an aftermarket Android
| ROM.
| pmlnr wrote:
| How do you know? With full access, it's basically
| impossible to tell.
| balaji1 wrote:
| > water stain with massive discoloration
|
| Maybe they store physical photos and the albums got soaked
| in water. Sadly it happened to all our 90s photo albums
| during a flood. Some of the photos were stained really bad.
| talkinghead wrote:
| bit rot
| eis wrote:
| That can be ruled out. Bit rot does not appear suddenly in a
| widespread manner and does not cause very similar visual
| effects in a variety of images. Also the original images are
| still available, just new modified versions were shown. To me
| it looks like a new compression algo gone wrong.
| [deleted]
| trustingtrust wrote:
| A bad bit wouldn't even be noticeable in an image. Changing one
| bit would make literally no difference. Artifacts look like
| something the compression algorithm has messed up. Also since
| the images are actually intact according to Google this is
| likely just the compression algorithm used to browse the
| images.
|
| Update: my limited knowledge on this is proof I'm wrong.
| rainbowzootsuit wrote:
| I've encountered older digital photos where a small
| corruption affects all of the remaining parts of the image as
| it is presumably recorded in some linear fashion as shown in
| this Wikipedia entry's "Visual example."
|
| https://en.wikipedia.org/wiki/Data_degradation
| can16358p wrote:
| A bad bit might have no visible effect at all (e.g. Affecting
| part of EXIF metadata, or a totally black pixel turning again
| very close to total black) or can corrupt the whole image
| (flipping a bit in the file header or cosine quantization
| table).
|
| It totally depends.
| mahathu wrote:
| As a total layman when it comes to storage: Don't they use
| forward correcting codes to make sure a single bad bit can
| reliably be detected and flipped without affecting the data
| at all? Or are these used only for data transmission?
| d110af5ccf wrote:
| The vast majority of file formats do not do such things.
| Such a concern is (typically) out of scope for that
| layer.
|
| Even most data transmission components don't use forward
| correcting codes. They typically just have error
| detection.
| Spacemolte wrote:
| Correction: "bit rot as depicted in a movie"
| [deleted]
| Forge36 wrote:
| Last post suggests Google has already started fixing the issue.
| Another suggests the original isn't lost and is available via
| takeout
| WirelessGigabit wrote:
| Even the original isn't in takeout. They separate the photo
| from the exif data.
| bumblebritches5 wrote:
___________________________________________________________________
(page generated 2022-09-25 23:01 UTC)