[HN Gopher] New 5D Storage to Offer 10,000x the Density of Blu-ray
___________________________________________________________________
New 5D Storage to Offer 10,000x the Density of Blu-ray
Author : WithinReason
Score : 107 points
Date : 2021-11-02 11:54 UTC (1 days ago)
(HTM) web link (www.tomshardware.com)
(TXT) w3m dump (www.tomshardware.com)
| kingcharles wrote:
| To quote a comment from Reddit: "I can't wait to never hear about
| this again."
| dividuum wrote:
| Yep. Still waiting for my tesa drive.
| louissan wrote:
| Leto II approves of this technology :-)
| MisterTea wrote:
| The other day I was talking with a friend about the high cost and
| perceived fragility of tape drives for long term personal
| archival while bemoaning the loss of higher density optical
| storage technology. It's not easy to backup or shuffle around
| terabytes of personal video, audio and pictures. Hard drives
| aren't exactly the right media for safe deposit boxes.
|
| In the 90's I had an Iomega Ditto 250MB drive that held 120MB per
| tape for $15-20 per tape and my hard drive was 540MB. This was
| when hard disc storage cost $1000/GB. The tape drive had a
| quarter the capacity of the hard disc and was cheap by
| comparison. It was practical. Later when CD burners came out they
| offered 650 then 700MB when hard drives were less than 10GB. Then
| they became too small and DVD came out but shortly was also
| passed up as 4.7GB wasn't much storage. Bluray was DoA with too
| little storage to be of real interest with a paltry 25-50GB.
|
| I'd love to see CHEAP >=1TB optical discs with a write speed of
| 100MB/s allowing us to burn a disc in ~10 minutes. I'd happily
| pay $1 a disc in bulk and upward of $1000 for a drive. I'd also
| happily pay $10+ for an archival disc that has a guaranteed shelf
| life of 50+ years. Bring back the 100 disc juke boxes while we're
| at it ;-) Cheap storage for everyone!
| ksec wrote:
| I have been talking and asking about this for years. And was
| told about Archival Disc [1] on HN.
|
| 500GB per disc right now, and only came out in 2019. 1TB per
| disc in the future but I suspect not in the next two years.
|
| I wish they have a more consumer, or prosumer friendly version
| though. The current drive goes for about $9K [2]
|
| There are Archival BluRay disc that offer 50 + year storage,
| but they are only in ~50GB per disc.
|
| [1] https://panasonic.net/cns/archiver/optical_technology/
|
| [2]
| https://www.bhphotovideo.com/c/product/1549175-REG/sony_odsd...
| Hamuko wrote:
| I have very little hope for the price for these things to
| come down to prosumer levels. The best I can hope for is for
| local libraries to have one available for use (although you
| need like 8 hours to fill the 5.5 TB maximum and they're so
| expensive you don't want to lend them out). However, I have
| very little hope for that as well since most consumers are
| just told to use the cloud.
| ksec wrote:
| >since most consumers are just told to use the cloud.
|
| Yes and I dont like it this trend at all. But it seems we
| dont have any other way to fight it.
| MayeulC wrote:
| Some have theorized that this is what's behind Amazon Glacier,
| and that they want to keep it to themselves:
| https://storagemojo.com/2014/04/25/amazons-glacier-secret-bd...
|
| I also crave for such cheap storage, perfect for
| archival/backups, especially when combined with modern
| snapshotting.
|
| HVDs never really materialized, despite being promising, and
| actually on the market at some point, IIRC:
| https://en.wikipedia.org/wiki/Holographic_Versatile_Disc
|
| A lot of other medium were developped, never commercialized:
| https://en.wikipedia.org/wiki/3D_optical_data_storage
|
| A new optical storage medium is long overdue. This time, it
| will probably be without support from the media industry.
| jcadam wrote:
| I'd take an on-prem solution for cheap archival storage any
| day over Glacier.
| manquer wrote:
| With egress costs on s3 that may not be viable if you
| generate data on the cloud
| zitterbewegung wrote:
| Why are tape drives fragile? I use them for my long term
| backups of personal files I thought they were robust ?
| jl6 wrote:
| What scale are you working at and how have you made it cost-
| effective versus a stack of external HDDs?
| adrian_b wrote:
| I am not that poster, but I also use tapes.
|
| One may save about $14 per TB by using tapes instead of
| external HDDs (which strangely have become cheaper than
| internal HDDs).
|
| In order to recover the cost of the tape drive, you need to
| write about 200 TB.
|
| I have about 50 LTO-7 tapes, i.e. about 300 TB of
| compressed archives, so I have recovered much more than I
| have paid on a LTO-7 tape drive, 5 years ago.
|
| Nevertheless, even if would have written only 100 TB, I
| would still have bought the tape drive, just for the peace
| of mind, because after the optical discs became too small I
| have used HDDs for archival storage, but after a few years
| in cold storage they become unreliable. If I had not stored
| duplicates for each HDD, I would have lost a lot of data.
|
| Of course, someone who needs to archive just 20 TB or even
| 40 TB of raw data, can buy just 2 or 4 HDDs and store each
| file on 2 HDDs and that should be OK.
|
| For 100 TB or more of raw data (i.e. 200 TB, when counting
| the redundancy, because also for tapes you should store at
| least 2 copies, preferably not in the same location), tapes
| become cheaper.
| zitterbewegung wrote:
| My brother got a cheap LTO tape drive on eBay and he bought
| a few LTO tapes so I think I bought one off of him for a
| few dollars more.
|
| It's only a few hundred gigabytes and the tape holds about
| one terabytes .
|
| Note this is my long term cold storage I have backups made
| on my MacBook Air (hard disk) and my sold MacBook Pro (ssd
| which I will use on my new 2021 MacBook Pro eventually).
|
| I also have backups using blu ray discs and I have one long
| term storage blu ray that I have been meaning to use
| adrian_b wrote:
| They are not fragile in an absolute sense, the poster meant
| that no matter what tape drive you have, it will break down
| many years before the tapes written with it would become
| unreadable.
|
| After the tape drive becomes unusable, you may no longer be
| able to buy another compatible tape drive, because they might
| be discontinued.
|
| For example I am using LTO-7 tapes. For the moment, I do not
| have to worry, because the current LTO-8 tape drives are able
| to read my LTO-7 tapes and the same should be true for the
| future LTO-9 tape drives.
|
| However, when LTO-9 tape drives will be the norm, I will have
| to buy a LTO-9 tape drive and many LTO-9 tapes and copy all
| my archives, because if I would delay this I would risk to no
| longer find even LTO-9 tape drives. In that case my tape
| collection will become unreadable, even if the tapes
| themselves are guaranteed for at least 30 years.
|
| I have already lost many years ago some archives that I had
| on QIC tape cartridges, because after my tape drive broke I
| was not able to find any other compatible tape drive.
| blue1 wrote:
| iomega stuff was generally rather precarious. LTO drives are
| solid.
| mixmastamyk wrote:
| Optical discs aren't torn apart when the reader malfunctions?
| dragontamer wrote:
| https://forums.tomshardware.com/threads/my-cd-shattered-
| insi...
|
| I've had a CD-ROM back in the day shatter on me like this.
| I'm pretty sure its a common enough story that everyone
| 'knows' it could happen... but was rare enough that not
| everyone personally came across it... but I never got to
| the bottom of it (why it happened, or what conditions led
| to it). It only ever happened to me once after all.
|
| It sounds like x56 speed drives were just too fast for some
| CD-ROMs. We went from x1 speed in the late 80s to x56 speed
| by the late 90s, so it makes sense that some CD-ROMs just
| weren't designed for the speed increase and maybe shatter.
|
| Alternatively: maybe some physical damage existed. We beat
| up CD-ROMs back in the day: storing them in backpacks
| without protectors and whatever. They were very reliable in
| terms of reading, but maybe those kinds of mistakes would
| lead to fractures that would literally blow up on you
| later.
|
| Even then: the most common protector were paper-sleeves,
| because dust / scratches were what we were most worried
| about.
|
| ---------
|
| We used to carry around stronger plastic cases in the early
| 90s. But by 00s, it was all paper-sleeves. The plastic-
| cases would themselves shatter when carried around, but the
| CD-ROMs / DVD-ROMs were stronger, lol.
| mixmastamyk wrote:
| Never seen it but have seen many ruined tapes over the
| years.
| iypx wrote:
| I remember my x52 drive would make at least 2 or 3 "test"
| spins every time I'd insert a disk, just to see how fast
| it can spin it. It felt like an airplane taking off every
| time I'd put a new scratch-less disk.
|
| I had one year warranty on that drive. This "feature"
| stopped working exactly one week after my warranty
| expired, it would just go full speed regardless of how
| beaten the disks were.
|
| That's how I got two of my heavily scratched disks
| shatter.
|
| A few more weeks down the road, the function that's
| supposed to slow down a disk before a full stop, also
| stopped working... so now even if a disk was spinning at
| x52 inside and you'd press the eject button, it would
| instantly stop the motor and eject it!
|
| That's when I finally threw it away and got myself a
| "normal" x46 or x44 (don't remember) drive...
| zitterbewegung wrote:
| Optical disks degrade due to bit rot when the substrate of
| the disc starts to peel off
| willis936 wrote:
| In my opinion, it's best to adopt industry practices for home
| life here. Schofield's 2nd Law of computing comes to mind: if
| your data doesn't exist in two places, then it doesn't exist
| anywhere.
|
| Offloading reliable backups to companies seems reasonable. It's
| a lot more money than archiving to physical media, but it's
| also a lot more reliable. NAS systems also make it easy to have
| reliable, hot on-site data that keeps a copy of the data backed
| up to a cloud service. Many of these cloud backup services are
| also zero-trust.
| kmeisthax wrote:
| The problem is that much of the demand for cheap, shelf-stable
| backup media has gone away over the last decade or two. Most
| backup jobs either go to cloud or to disk, not tape. And
| because of that, tape drives are more expensive than ever,
| which takes away much of the benefit of the cheap media.
|
| LTO, for example, is relegated to archival use-cases nowadays,
| such as video production. You'd think that would make sense,
| because the media's shelf stable... but it's actually really
| awful for archival. Why? Because new drives can't read old
| tapes past two generations, and old drives stop getting made.
| So every time a new tape drive generation comes out, a movie
| studio can't just upgrade drives in their libraries and use the
| new tapes. Nor can they just hold onto their old drives forever
| - what happens when they start breaking, and they can't read
| their old tapes anymore? (Or worse, an old/failing drive eats a
| tape?)
|
| Every movie studio with an archival program is on a constant
| upgrade treadmill, including costly media migrations. That is,
| they're copying data from old tapes onto new, so they can
| maintain compatibility with new drives. This entirely defeats
| the purpose of shelf-stable media; if I can't buy newly
| manufactured drives for old tape formats then who cares if the
| tapes themselves can sit in a box for 20 years?
|
| Now, let's say you're _not_ a movie studio with a massive
| digital archival problem. You just want to backup 5 terabytes.
| Unfortunately, 5 terabytes is so little that tape vendors have
| forgotten how to count that low. It 's far cheaper (not to
| mention, more performant) to just buy a bunch of disk drives
| and migrate data between them. Or just pay Amazon to do it.
| Which is what everyone wound up doing.
| ghostly_s wrote:
| > This entirely defeats the purpose of shelf-stable media; if
| I can't buy newly manufactured drives for old tape formats
| then who cares if the tapes themselves can sit in a box for
| 20 years?
|
| Why can't you buy new drives for old tape formats? What new
| features are these new formats providing?
| throwaway81523 wrote:
| The new formats have higher storage density than the old
| ones. First generation LTO (LTO-1) stored 100GB on a tape:
| how much would you pay for a drive like that today?
| Unsurprisingly, they don't make them any more. LTO drives
| are read-compatible with tapes from two generations
| earlier, so you can read LTO-1 tapes on LTO-3 drives, but
| LTO-3's capacity was 400GB, so no one buys those any more
| either. The current generation (LTO-8) has 12TB per tape,
| so you can see the upgrade treadmill chugging along.
|
| https://en.wikipedia.org/wiki/Linear_Tape-Open#Generations
| MisterTea wrote:
| Tape drive backwards compatibility is an issue for sure but
| optical drives have had decent backwards compatibility so
| far. DVD drives can read CD's. BR drives can read CD, DVD and
| BR. I think backwards compatibility is more feasible for
| optical as the larger pits on older media can still be read
| by narrower wavelengths. They'll just look like giant craters
| flashing by instead of nice neat pits.
| kmeisthax wrote:
| AFAIK I believe that they actually do use different lasers
| for different disc types, but the cost to put two or three
| laser diodes in one drive is basically nil these days.
|
| The big problem with optical storage is the same that tape
| struggles with - it's primary market shrank.
|
| This was a bit of a self-inflicted wound. CDs and DVDs were
| basically ubiquitous, drives came down in price
| dramatically, and the storage densities and pricing was
| competitive with competing technologies. BD is where that
| all started breaking down. BD had a format war with HD-DVD,
| and both formats relied upon new and then-expensive
| technologies such as blue lasers and quad-layer stacking.
| PC manufacturers (notably Apple) balked at the cost of
| either format. Microsoft backed HD-DVD, but wisely decided
| not to bleed money by not sticking an HD-DVD drive in the
| Xbox 360. This created an odd situation where the Xbox 360
| sold better as a game console, but Blu-Ray outsold HD-DVD
| because Sony had let the PlayStation division bleed out for
| a couple of years.
|
| Of course, Microsoft still wanted the 360 to be able to
| play HD movies, and their response was to partner with
| Netflix to make streaming a reliable (arguably better)
| substitute for movies on optical media. In other words,
| Sony killed HD-DVD, so Microsoft killed the whole home
| video market by pushing digital distribution for
| everything.
|
| If you're wondering, there _are_ successor formats to Blu-
| Ray. Interestingly enough, the whole quad-layer stacking
| thing was rebranded as BD-XL and UHD-BD (which the PS4
| doesn 't even support), and there's also a proper successor
| format for data storage called Archival Disc. Except the
| latter is basically an LTO competitor - you can't buy bare
| AD drives and discs and use them like Blu-Rays. The
| technology only exists in archival systems like Sony's ODA
| system (Gen 2 and Gen 3; Gen 1 used Blu-Rays), which has
| discs stacked in cartridges that have to be loaded into
| special, very expensive readers and media libraries.
| memco wrote:
| To add: LTO speeds are currently in the 300MB/s range: if you
| want > TB size storage you likely don't want ~100MB/s for
| regular use. I work on systems where we are regularly writing
| multiple tapes of 5+ TB and if you get only 100MB/s you're
| waiting hours and if something fails or gets stuck you can be
| thrown off schedule by more than a day very quickly. Even at
| the current 300MB/s speeds it's very slow. If personal
| archival or even small-scale archival is your interest
| getting network storage is probably easier, faster and
| cheaper. What's more, is that you have to keep some kind of
| inventory of your tapes so you know where to find data if you
| ever need it. Depending on the size of your collection you
| can be looking for even more expensive equipment to manage
| the archive of your archives.
| kmeisthax wrote:
| Not to mention that 300MB/s is considered best-case
| conditions for sequentially stored data. Anything even
| remotely random is going to have far slower archival times,
| unless you have archival software that does parallel I/O.
| (In my experience, not many archivers do this.)
|
| Most production tape installations include dedicated disk
| storage to act as a buffer for the drives. You archive to
| disk first, so that your drives aren't tied up, and then
| write the now-sequential data to tape.
| mixmastamyk wrote:
| I burn 100GB backups via blu-ray. One or two is enough for my
| non-movie files. They started stagnating a few years ago when
| streaming ascended however. Shame as I'd like higher capacity
| as well.
|
| Cheap USB flash drives are already moving into this space. I
| just got a compact 256GB one for $35 or so, just a matter of
| time. SD cards would probably be more convenient, but they
| aren't sadly aren't read in devices like stereos any longer.
| rbanffy wrote:
| That much music (even at lossless compression that's a
| lifetime of nonstop music) is completely impractical to
| navigate using a common stereo UI
| mixmastamyk wrote:
| Not really, it's about five clicks to start an album, which
| I'm pretty adept with, and plays for an hour. I have
| several hundred in Letter/Artist/Year,Album order. Less
| work than futzing with the radio constantly, which is
| generally terrible (unless you love tired top-40).
| tomc1985 wrote:
| Nooooooooooo, you're supposed to back it up to the cloud and
| pay some startup jerk $XX/mo for the privilege! [/s]
| GhettoComputers wrote:
| Says whom? Most cloud storage have huge free tiers. I have
| unlimited storage on telegram.
| IceWreck wrote:
| Telegram upload speed is limited to 3MB/s.
|
| Doesn't matter how high your connection speed is, it is
| capped from telegram's end.
| tomc1985 wrote:
| So will they intake 10TB?
| GhettoComputers wrote:
| Never tried my upload is low speed and I don't have that
| much data.
| Hamuko wrote:
| I bet you actually have "unlimited" storage.
| GhettoComputers wrote:
| Can you name something that is not "unlimited"?
|
| I don't have a limit, I never hit the cap. By that logic
| nothing is unlimited (texts, calls, data, etc), but
| realistically it doesn't matter because I don't use it
| much. I don't care to test it, I use it sparingly and
| never had a issue with limits.
| Hamuko wrote:
| I assume that my S3 bucket is as close to unlimited as it
| gets.
| GhettoComputers wrote:
| Its also "unlimited" isn't it? There is no such thing as
| unlimted. As a subsciption you will also lose it if you
| stop paying.
| [deleted]
| ok123456 wrote:
| I remember a very similar research project from the 90s. A 3D
| Holographic cube that was supposed to have a, at the time, insane
| density.
|
| https://en.wikipedia.org/wiki/Holographic_Data_Storage_Syste...
| jijji wrote:
| The thpical lifetime of a CD or DVD or BlueRay is about 3-5 years
| depending on the handling by the user. I'd like to see them
| outline the claim that this media can last "billions" of years.
| It sounds highly suspect, as does all the other claims in the
| article.
| terafo wrote:
| It is very durable. It withstood 1000 degree heat for 2 hours
| in referenced paper, there are no moving parts on the storage
| itself, it doesn't require additional energy to sustain
| records, records themselves don't deteriorate over time, and
| glass by nature is quite resistant to mechanical damage(outer
| few mm obviously shouldn't be used). If stored correctly, I see
| no reason for it not lasting a few million years.
| gjm11 wrote:
| In case anyone is wondering what they mean by "5D": they mean
| it's a 3D array of marks, each of which has both size and
| orientation. The 3D array is three 2D layers.
|
| According to this definition, a DVD or Blu-Ray disc is "4D"
| because it can have multiple layers each of which stores
| 1-dimensional data (pit versus no pit). And a book is at least
| "7D" because it contains a 3D arrangement of marks, and marks can
| vary in (at least) what letter they are, how large, how bold, and
| how slanted. (It would be easy to add to that list.)
|
| I would describe this thing as 2.5D storage: it's basically
| 2-dimensional but with multiple layers.
|
| If it's genuinely 10000x denser than Blu-ray, great! If it's
| likely to last longer, also great! But there's no need for this
| "five-dimensional" bullshit.
|
| (It looks to me as if what they have at the moment is not
| anything like 10000x denser than Blu-ray. I'm not sure it's even
| 1x denser than Blu-ray at present.)
| chmod775 wrote:
| A better definition then would be: Assuming each dimension is
| infinite, how many dimensions do you need to uniquely address a
| bit.
|
| The infinite requirement is there so the answer can't be
| reduced to "1 dimension" every time (if you had a 10x10 matrix
| you could just turn it into an array of 100 elements).
| clairity wrote:
| what you're dancing around is linear independence, which is
| the idea that two functions can't be described as a function
| of each other, which also means they can be used to specify
| every other function in the intervening function space. the
| least number of linearly independent coordinate functions
| (usually transformed down to unit vectors) is the
| dimensionality of the system. note that you can have linearly
| _dependent_ functions /coordinate systems, which this '5d'
| moniker is akin to.
| chmod775 wrote:
| > what you're dancing around is linear independence
|
| Thanks! That was a "ohhhhhh of course!" moment for me.
|
| At least that's more succinct and I don't see a way to
| cheat it.
| chriswarbo wrote:
| Slight nit-pick, but any number of dimensions can be reduced
| to one, using a pairing function
| https://en.wikipedia.org/wiki/Pairing_function or a space-
| filling curve https://en.wikipedia.org/wiki/Space-
| filling_curve
| chmod775 wrote:
| Right. I don't immediately see how either of those would
| work for uncountable infinite sets, but it may be possible.
|
| Clearly some other limitations are required to make
| "dimensions" a useful metric in this context.
| ClumsyPilot wrote:
| I think once you are talking about "uncountable infinite
| sets" and different kinds of infinities, the definition
| has become very niche.
| chriswarbo wrote:
| > I don't immediately see how either of those would work
| for uncountable infinite sets, but it may be possible.
|
| We can think of an N-dimensional space as a grid of
| (hyper)cube "cells":
|
| - Cantor's pairing function traces diagonal lines through
| the top-right quadrant of a (countable) 2D grid
|
| - We can extend Cantor pairing to the whole (countable)
| 2D plane by joining the diagonals of each quandrant (they
| meet at the axes). The resulting line spirals out from
| the origin, in a diamond shape.
|
| - We can extend the 2D plane to higher dimensions, again
| by joining diagonals when they meet at the axes. This
| results in a diamond-shaped "shell" spiralling out from
| the origin.
|
| The above gives us a 1D line which visits every cell one
| after another. Next we associate each unit-interval on
| our line with the cells we're traversing, i.e. the
| interval between 0 and 1 is associated with the 1st cell;
| the interval between 1 and 2 is associated with the 2nd
| cell; and so on. (We could also jump between positive and
| negative, like [0, 1), [-1, 0), [1, 2), [-2, -1), ... to
| include the negative half of our 1D line as well)
|
| Finally, we use a space-filling curve, like a Z-curve, to
| map each interval to the volume of its associated grid
| cell. This works for uncountable intervals and cell
| volumes.
|
| The result would be a line which starts at the origin,
| and gradually spirals outwards. From a distance, the line
| appears to form N-dimensional diamond-shaped 'shells'. If
| we look closer, we see each 'shell' has jagged edges,
| since it's formed of N-dimensional grid cells. If we look
| even closer, we see that the line traces a z-curve
| through each grid cell, filling its volume.
|
| The above works for any pairing function and space-
| filling curve, although I've written about cantor pairing
| at http://chriswarbo.net/projects/procedural/cantor.html
| and z-curves at
| http://chriswarbo.net/projects/procedural/z.html :)
| guerrilla wrote:
| > But there's no need for this "five-dimensional" bullshit.
|
| Sure there is. It's going to sell great with idiots. I still
| remember morons super hyped about Opteron trying to claim it
| was an optical CPU.
| MAGZine wrote:
| I think Dimension in the data sense of the word, not Dimension
| in the spatial sense of the word.
|
| Multi-dimension data is a thing, and it goes a lot higher than
| 4, 5, or 7.
| gruez wrote:
| >I think Dimension in the data sense of the word
|
| Where else is that used? At least to me (and the common
| layperson), the only usage seems to be from people trying to
| hype up storage mediums.
|
| also, if we're using that definition, does that mean I have a
| 7D SSD? The NAND cells are arranged on both the planar axis,
| and stacked (3d NAND). That's 3 dimensions. Each NAND cell
| also encodes data using various charge levels, which
| currently tops out at 4 bits. In total that makes for 3 + 4 =
| 7 dimensions.
| lelandbatey wrote:
| It's mostly a mathematical term which also applies to
| pieces of data. See here:
| https://en.wikipedia.org/wiki/Dimension
| In physics and mathematics, the dimension of a mathematical
| space (or object) is informally defined as the minimum
| number of coordinates needed to specify any point within
| it.
| c5e3ebe93d2c wrote:
| In my physics courses we would routinely talk about 6
| dimensional spaces when discussing position-momentum space,
| also we would sometimes refer to two dimensional objects
| like a moebius strips as "1-dimensional".
|
| > also, if we're using that definition, does that mean I
| have a 7D SSD?
|
| Maybe. You can use whatever definitions you want if you
| feel it's helpful. However, based on your description, if
| you're using a single bit as a dimension, you should figure
| out how many bits the the dimensional information encodes.
| As it stands, it's not clear what you're counting as a
| 'dimension' in this context. Also, normally we think of
| degrees of freedom as being independent of each other
| (hence the claim to 5d in this example). The charge
| information would normally be thought of as a single degree
| of freedom, so you'd have a 4d nand disk.
|
| One reason it is helpful to think of "a dimension" as an
| independent degree of freedom, is that it becomes a
| parameter you can focus on improving. So if you say "Well
| what can we change, or make more precise?", the answer for
| the nand case is "Well, we can't easily add another spatial
| degree of freedom, but we can improve our stacking to
| improve how much we can pack into the vertical dimesion
| (but we're limited by Job's obsession with thinness). We
| can improve our measurements of spatial resolution, which
| will affect 3 of our dimensions, but not the charge. We can
| also improve our charge resolution. Let's figure out which
| one is cheapest to scale."
|
| Now granted, sometimes tweaking one parameter affects the
| other, so they aren't strictly independent. You can pack a
| lot of charge on a NAND, but as the other dimensions get
| smaller, you start having increased difficulty with
| leaking.
|
| Anyhow, I think there's more here than you're giving them
| credit for, even if they are guilty of tooting their own
| horn a bit (shocking).
|
| But yes, the article presented here would be greatly
| improved if it gave examples of how to classify existing
| technology using the researcher's dimensional
| classification scheme. It's a common scientific writing
| tactic that I'm surprised was not used here.
| emteycz wrote:
| It's a math/information science term that was applied in
| physics (its most well known usage), not the other way
| around...
| Rygian wrote:
| Which reduces to one single dimension anyway, on a storage
| medium: my bytes are stored from offset 0 to offset N.
| 4ad wrote:
| The fact that spaces of different dimensionality can be put
| into a bijective map does not mean that dimensionality is
| not real or useful.
| babypuncher wrote:
| Yes, the dimensions are an abstraction to make it easier
| for our human brains to organize certain collections of
| data. I don't think it makes them any less valid, otherwise
| we would refer to every data type as an array of bits and
| nothing more.
| klodolph wrote:
| That's a bit reductive.
| zardo wrote:
| Maybe for some. For most uses though, people just want to
| write data to and read it from the storage media. They
| don't want to think about the coordinates their data was
| written to on the DVD.
| arcticbull wrote:
| Isn't this normally referred to as "degrees of freedom"
| rather than dimensions?
| mvanaltvorst wrote:
| Statisticians and mathematicians call them dimensions,
| machine learning engineers call them "features".
| Statisticians and mathematicians usually interpret
| observations as vectors in n-dimensional space, which leads
| to some interesting geometric results. Hence the term
| dimension, we do mathematics with them as if they were
| actual points in some high-dimensional space. Degrees of
| freedom are something a bit more nuanced that are of use
| when you start to do specific statistical tests, and they
| do not have to be exactly equal to your amount of
| dimensions.
| panda-giddiness wrote:
| No, these are separate concepts. As an example: a simple
| linear regression through 7 data points has 2 dimensions (x
| and y) but 5 degrees of freedom (number of data points -
| number of estimated parameters).
| kmeisthax wrote:
| Counting size and orientation as extra dimensions would be
| justifiable if you could overlap dots of different sizes or
| orientations, even a little.
| rbanffy wrote:
| You still need 5 values to retrieve a single bit of data,
| even if only two of those numbers have more than 3 bits.
| AtlasBarfed wrote:
| They mentioned 500TB for a single glass disk, isn't that
| 100,000x more than a 5GB blu-ray, or are they comparing it with
| the 10-layer blu-rays that never really hit the market?
|
| Is this the same UK group that did all the holographic disc
| research that is basically vaporware in the mass market (I
| don't want to call their research vaporware, just that it never
| hit the consumer market)?
|
| Edit: Whoops. 4.7 geebees is for a regular DVD, not a Blu-ray.
| Dual layer blu-ray is 50GB
| Snitch-Thursday wrote:
| Whether it be this or project Silica, I want my digital permanent
| storage. Write it in glass, treat it nicely, and yes, you may end
| up with digital bitrot or whatever, but I want media that stores
| digital data that cannot physically degrade for hundreds of
| years, kind of like paper does.
|
| Paperbak being forked and made mainstream, I can accept that.
| Provided I can also do an automated book printing.
|
| I will also accept holographic stick gum storage a la isolinear
| chips.
|
| But there has to be more than just tape and blu rays.
| ajuc wrote:
| > The new approach can achieve write speeds of 1,000,000 voxels
| per second, the equivalent of 230 kilobytes of data per second
|
| > 500 TB of data on a small disc
|
| So, it would take 70 years to fill it. By the time you can fill
| it we will have RAM with higher density :)
| terafo wrote:
| > `The researchers say that with improvements to the writing
| techniques, particularly by taking advantage of parallelism,
| they can design a system that can fill that same 500 TB in a
| mere 60 days.`
|
| It's possible to get it up to 100 megs per second, which is
| write speed of a good hard drive. And I suspect it could be
| scaled up even more. Progress in this field is quite good, 8
| years ago they put 300kb on it, now it's 5gb.
| tmikaeld wrote:
| They do mention parallell writes (multiple lasers at the same
| time) which would give writing speeds of 96MB/s (500 TB in 60
| days).
|
| Which isn't half bad, certainly on pair with standard mechanical
| hard-drives.
| ChuckMcM wrote:
| Pretty neat, this is worrisome (from the paper): _A book was
| recorded in 4-bit voxels with nearly 100% readout accuracy._
|
| Kind of like "this new car hardly kills anyone." :-) Given that
| data recovery is like the most significant aspect of any data
| archival solution.
|
| Not a mention of readout speed either, there are many read-only
| datasets that could benefit from something like this of course,
| but if it is way slower than LTO tape I'm not sure exactly where
| it fits in.
| throwawayswede wrote:
| Who comes up with garbage naming techniques such is 5D? What the
| fuck do they expect people to think when they hear that?
| GhettoComputers wrote:
| It's magical storage that goes into the multiverse. Technology
| is magic!
| rdevsrex wrote:
| Yeah, my first thought even before reading the article, was "What
| about seek times?". Maybe it can be improved, but thinking of web
| apps specifically, I'm not sure what kind of use case you would
| have for something that is this slow to access.
| azinman2 wrote:
| This is for archival purposes. Seek times don't matter for that
| use case.
| Hamuko wrote:
| Seek times always matter even if they're not the top
| priority. And especially if we're talking about a storage
| medium that takes in a whopping 500 TB.
| azinman2 wrote:
| The GP is talking about something interactive like a web
| app, where a seek time likely would matter. This isn't
| intended to be used as such.
| EastOfTruth wrote:
| I don't care about seek times if we can have media with orders
| of magnitude more storage capacity and life expectancy. (of
| course if you can also have good seek times, it would be icing
| on the cake)
| GhettoComputers wrote:
| What kind of data are you trying to store? How much of it? I
| can't imagine my life being more than 100 GB and probably
| much smaller, probably under 20GB.
| EastOfTruth wrote:
| Scihub, libgen, datasets, etc... as much data as possible.
|
| My personal data is probably less than 1TB at this point,
| but a way to store it long term would be nice.
| Grustaf wrote:
| > promising incommensurate storage in a palm-sized device
|
| Incommensurate with what, one might ask.
| jbarrs wrote:
| I remember when I first heard about 5D storage back in 2013
| [1][2]. The concept isn't at all new; in fact, there's a 5D
| storage device floating around in a Tesla in space, and Musk has
| another in his personal library. Nonetheless, it's great to see
| that it hasn't just dropped off the radar and that developments
| are still being made. I hope that this technology can one day be
| as widely available as CDs are; its potential usefulness to
| archivers/data hoarders really excites me, and (so long as the
| price is right) who can really sniff at extremely high-capacity,
| long-lasting storage?
|
| [1] https://www.orc.soton.ac.uk/news/4282
|
| [2] https://doi.org/10.1364%2FCLEO_SI.2013.CTh5D.9
| mattowen_uk wrote:
| This is all well-and-good, but how do we read all that data in
| 500 years time, or even 20 years time?
|
| People were busily archiving data to CDs and DVDs two decades
| ago, thinking 'yeah this digital storage media will last for
| centuries!'... Loads of bit-rot combined with complete lack of
| DVD drives still in common [computer] usage, has totally shown
| that the more complex the storage media the shorter it's useful
| lifetime is. Compare this to books, which can still be read a
| millennium later.
| TacticalCoder wrote:
| > This is all well-and-good, but how do we read all that data
| in 500 years time, or even 20 years time?
|
| I know it won't last due to bit-rot but all my CDs burned in
| the nineties are still readable.
|
| > Loads of bit-rot combined with complete lack of DVD drives
| still in common [computer] usage...
|
| They are still very common: both in desktop and as USB-
| connected readers/burners. You can find countless models on
| Amazon. I don't think they're going away anytime soon.
|
| > People were busily archiving data to CDs and DVDs two decades
| ago, ...
|
| That's precisely why they're not going away anytime soon.
|
| There are even companies like FB would, at some point, were
| burning insane amount of data to Blu-Ray discs.
|
| > Compare this to books, which can still be read a millennium
| later.
|
| Having a book from 1575 here I do agree (not quite a millenium
| but still). However the solution to disc bit-rot is simple:
| once every ten years or so, read your discs and burn them on
| the new fancy tech of the day (say CDs to DVDs, DVDs to Blu-
| Ray, now Blu-Ray to these 5D thinggies, rinse and repeat).
|
| It's not as if new mediums had less capacity than older ones.
| nomel wrote:
| > I know it won't last due to bit-rot but all my CDs burned
| in the nineties are still readable.
|
| Were these the more expensive archival disks? I lost all ~50
| of my old CDs and DVDs to rot, within 10 years. They were
| middle range Memorex.
| 0des wrote:
| Let's not forget actual rot too. I peered into a stack of CD's
| not long ago to find that a lot of them have quite literally
| rotted while sitting in a dark and dry closet. No dust, no
| humidity, no smoking, no abuse, just a spindle of CD's
| recognizable only by their sharpie'd identifiers covered in
| pits of rotted foil.
| throwaway946513 wrote:
| Temperature? I've yet to experience rot on decades old discs
| at home. Most sit in a basement that doesn't exceed 68*F
| 0des wrote:
| A constant 72F, this is an indoor closet, top shelf, no
| moisture incidents.
| cogman10 wrote:
| We are mostly talking about burnt discs which achieve their
| burning via chemical reactions with light. Those disks rot.
|
| Any standard Blu-ray/DVD/CD you have has no rotting as it's
| simply pressed.
| adrian_b wrote:
| While the pressed discs should have a longer life, there
| are some where the metallic mirror oxidizes in time and
| it becomes partially transparent, which results in
| reading errors.
|
| The oxidation speed depends on the quality of the lacquer
| that protects the mirror (on the label side), so it
| varies greatly between manufacturers. As an end-user, you
| cannot predict the lifetime, you can just hope for the
| best.
|
| There have been archival CDs, with gold used for the
| mirror instead of aluminum. Those were immune to
| oxidation, but they have been discontinued, due to high
| cost.
| MivLives wrote:
| It's a well known problem:
| https://en.wikipedia.org/wiki/Disc_rot
| TacticalCoder wrote:
| Yup it's a well-known problem, but several of us still
| have CDs decades old that still do read totally fine.
| That doesn't mean we rely on them as backups: mine have
| long since been copied to DVDs, then more recent DVDs but
| also backup'ed on harddrive / servers etc.
| mixmastamyk wrote:
| I still have music CDs from 1985 and CD-R from 1995 in
| perfect condition. None of them have gone bad.
| rbanffy wrote:
| CDs and DVDs are pressed. Recordable media is not and,
| with the wrong conditions, will rot.
|
| DVD-RWs will get corrupted if you look funny at them.
| throwaway946513 wrote:
| The linked article also explains:
|
| > DVDs are more resistant to rot > DVDs and CDs use
| aluminum layers for the reflective 'data encoded' layer >
| Discs using a gold layer are extremely resistant to
| corrosion and thus disc rot > Blu-ray discs use a silver-
| alloy for their reflective layer.
|
| Though not linked, or stated, it is quite likely that the
| BR discs are significantly less likely to be damaged due
| to disc rot for similar reasons as gold-layered discs
| are, though not necessarily as resistant as gold.
|
| Disc rot though appears to stem due to the damage of the
| lacquer or plastic layer on discs, which if the disc then
| remains undamaged, it should not be as susceptible to
| rot.
| vel0city wrote:
| Regular CD's and DVD's have an organic layer that will rot
| away. There are alternative styles of discs which use a
| ceramic layer instead of organic which are rated to last a
| thousand years. These discs can still be burned and read in
| most standard burners and readers.
|
| https://www.mdisc.com/
| Sosh101 wrote:
| Same - maybe something oxidised.
| vel0city wrote:
| Optical drives aren't common in most consumer devices but that
| isn't really a hurdle if you're trying to recover archived data
| on optical media. If you're only worried about DVDs a USB DVD
| drive can easily be had for ~$20. Blu-Ray drives aren't very
| expensive, I included one in my last desktop build for the
| purpose of data archival.
|
| Many consumers figured optical disks would last a long time but
| many knew about disc rot from the get go. I picked optical
| media that marketed itself as archive quality. My oldest discs
| are reaching 20 years old now and I've lost less than 10% of
| the discs I've burned that were burned on good discs.
|
| These days I burn data to M-DISCs which use a ceramic layer
| instead of organic. They're rated to 1000 years and the Blu-
| Rays are quite durable. I've had one bouncing around my desk
| and outside for several years. The one on my desk has zero
| errors while the one outside has a few bit errors.
| kipchak wrote:
| I think right now the most permanent way to store digital data
| is have enough redundant copies on enough impermanent media one
| is likely to survive. For example my CD copy of 1998's HARDWAR
| might be lost due to disk rot, but there's enough copies of
| copies of copies floating around on various places it's still
| easily accessible.
| MisterTea wrote:
| > This is all well-and-good, but how do we read all that data
| in 500 years time, or even 20 years time?
|
| If the data is important enough, we'll find a way. Perhaps by
| then camera phones will have sensors with quantum resolution
| that can capture the optical pits in one snap and use AI or
| whatever to reconstruct the bits from the image on whatever
| magical storage exists.
| GhettoComputers wrote:
| What kind of data is even worth storing? If it's text based
| it's not hard and it's not very large. If it's media it's
| probably some uncompressed videos and pictures that are overly
| large, phones that take 16mb photos that aren't going to have
| more useful information over compressing it to 400kb. I am sick
| of my phone taking large photos that don't have any clear
| benefit to me aside from clogging my phone with unreasonably
| large files that don't look better, Apple wants to sell me
| iCloud by not allowing me to take lower resolution images.
| wazoox wrote:
| Some guys I know propose a storage on 35mm-film solution that
| looks pretty foolproof for a couple of centuries at least.
|
| https://digifilm-corp.com/preserve-your-works-for-eternity
|
| To read back the data, you need something to digitize the film
| (a flatbed scanner would work, I've tried, or probably even a
| smartphone with a a DIY magnifier), then something to decode
| the data (the necessary source code is provided in clear on the
| first images of each roll).
| nomel wrote:
| I'm having trouble finding the data density. How many lbs of
| film/GB?
| louwrentius wrote:
| Archiving data is a process, not a one-time effort.
|
| You can't just store data on a medium and hope if that medium
| will last.
|
| You have to periodically check for defects and from time to
| time, transfer data to fresh media.
|
| At some point, you may even need to convert the data to keep it
| readable...
| rbanffy wrote:
| It all depends on how technology changes. We can still read
| floppies written in the 70s with essentially a motor, something
| magnetic and a wire going around it (and a computer reading the
| currents). I'm pretty sure we'll know how to cobble together an
| optical microscope and camera device to read at least one bit
| out of every volume cell. Let's just hope the data format is
| simple enough.
| bufferoverflow wrote:
| There's no lack of DVD drives, they are widely available on
| Amazon, BestBuy, eBay.
| handrous wrote:
| They're also pretty cheap ($15-30), powered fully over USB--
| no extra power cable like in the old days--and they're tiny.
| tiffanyh wrote:
| Off topic: if I wanted to digital store files to pass down to my
| children that might not be access for _decades_ , what's the best
| medium to store such data?
|
| Floppy/ZipDrive/CD-ROM/Blueray/USB-A are either obsolete or are
| on their way out. What's a future proof storage option? (and if
| you said Cloud Storage {X} - just look at how many services have
| folded in just the last few years alone). Secondly, what's the
| best file formats also that will be future proof?
| GhettoComputers wrote:
| What kind of files and how much?
| zamadatix wrote:
| As far as formats stick to either very popular or very simple.
| E.g. for video in 100 years time there is still going to be a
| way to be able to decode h.264 videos just because of the shear
| amount of content from this era that was encoded in it.
| Alternatively use a format that is so simple it doesn't matter
| if there is a decoder in 1,000,000 years as it's trivial to
| make. On the simplicity side take PPM or bitmap images for
| example, one can get a decoder up and running from scratch in a
| matter of minutes.
|
| The harder half is the storage of the digital information over
| time. Interfaces, filesystems, formats, and devices are not
| made to last decades as there is no need for the vast majority
| of data to sit for decades without being transferred to newer,
| cheaper, and smaller media. Any solution you come up with to
| avoid copying will therefore be esoteric and one off, not to
| mention likely complicated for any significant amount of data,
| making it hard to guarantee easy enough readability even if it
| is physically fine in 100 years.
| mixmastamyk wrote:
| No such thing. Other poster had a good idea, duplicate
| everything every ten years. I suspect flash drives will have
| the best longevity in the medium term however.
| IndexCardBox wrote:
| I'd hedge my bets and store it multiple ways in multiple
| formats. Including physical (print off pictures, diaries, etc.
| on acid free paper with acid free inks)
___________________________________________________________________
(page generated 2021-11-03 23:01 UTC)