[HN Gopher] Putting 5,998,794 books on IPFS
___________________________________________________________________
Putting 5,998,794 books on IPFS
Author : NKosmatos
Score : 114 points
Date : 2022-11-19 22:00 UTC (59 minutes ago)
(HTM) web link (annas-blog.org)
(TXT) w3m dump (annas-blog.org)
| wrs wrote:
| Considering the goals of IPFS, I'm surprised it has trouble at
| this scale. I mean, it's supposed to be "interplanetary" but this
| data would fit on two hard drives. The post talks about
| performance problems with advertising "hundreds" of content IDs.
| Is it a design problem or just an early implementation problem?
| russellbeattie wrote:
| A few months ago I decided on a whim that I should download and
| de-drm my Audible audiobooks. I have a lot of them from a decade+
| of monthly credits being used. I hadn't really considered how
| much space it would take up so I just set off the download
| process and did other stuff. 650 audiobooks took up nearly 2TB of
| space before I stopped it, only noticing when my Mac gave me a
| warning about not having enough space left on the drive. Whoops.
| (Audible keeps track of your total listening time... It adds up
| to over 9 months solid. Crazy, but I can think of worse ways to
| spend that time.)
|
| Anyways, if 6 million ebooks takes up 31TB, I wonder how much
| space the equivalent amount of audiobooks would use? And how much
| space a decent library of movies has? I'm kinda thinking that guy
| on reddit who somehow got a hold of a used Netflix edge server
| was on to something.
| halpmeh wrote:
| Isn't IPFS a terrible technology to host copyrighted material on?
| As far as I know, it has no privacy features, so you can see
| exactly who is hosting which content.
| Gigachad wrote:
| I've been interested in IPFS for a while now but I'm still unable
| to come up with a good explanation of how it's different to
| regular torrents. Does anyone have any examples of stuff you can
| do with IPFS that you can't with torrents?
|
| Seems like over the last 5 years they haven't really done
| anything but add crypto buzzwords to the project site.
| rudolph9 wrote:
| The content is the address, no central issuer of the tracker
| file needed just the CID content ID. Also since the CID is a
| hash of the data it can be used to validate the data :wink:
| nodja wrote:
| That is exactly how bittorrent works as well tho. Bittorrent
| hasn't needed any trackers for public torrents since 2005 and
| can work solely off magnet links using the DHT network. Just
| like how IPFS does it.
| teraflop wrote:
| That's basically the same as torrents with magnet links.
|
| As other commenters pointed out, the biggest _real_
| difference is that IPFS objects can cross-reference each
| other by hash, which allows you to do partial updates.
| layer8 wrote:
| An "update" means a new CID though, and this basically
| works like a persistent data structure [0]?
|
| [0] https://en.wikipedia.org/wiki/Persistent_data_structure
| bakugo wrote:
| That's quite literally how bittorrent with DHT works. To
| download a public torrent you only need its content hash, the
| metadata and peers are then fetched via DHT assuming at least
| one reachable DHT node is aware of the torrent.
| [deleted]
| georgyo wrote:
| The biggest thing is that in IPFS each chunk has a CID, and
| that CID is globally unique and sharable from other CIDs.
|
| You can think of each CID as a mangent link, and each CID can
| point to more CIDs.
|
| Where as in a torrent the peer tracking is done at the whole
| torrent level, instead of the chunks inside the torrent.
|
| Torrent V2[0], which has poor adoption thus far, should also
| allow for a similar ability. To my knowledge this is not being
| taken advantage of yet.
|
| [0]: https://blog.libtorrent.org/2020/09/bittorrent-v2/
| [deleted]
| typingmonkey wrote:
| You cannot "put stuff" on ipfs. Either you seed it or someone
| else does. If noone seeds it, it is gone. I bet most of it will
| be offline in 3 months.
| tshaddox wrote:
| That's clear from the article, which is about the nontrivial
| task of making tens of terabytes of data available on IPFS.
| Xeoncross wrote:
| Isn't that true of anything shared on a network?
| tinalumfoil wrote:
| This is very nit picky. The entire web works by severs
| "seeding" data to the client, so by your logic you can't "put
| stuff" on the web. The difference with torrents and IPFS is you
| can have multiple servers seeding the same content, and not be
| dependent on any one.
|
| I'm not making any predictions about how long Z library stays
| up, but the illegal seeding of movies and tv shows has remained
| very strong until today.
| typingmonkey wrote:
| Fair point. From my experience people often think ipfs works
| equal to sth like a shared drive or filecoin, so I had to
| point that out.
| imhoguy wrote:
| This post really starts to show the right direction:
|
| Z-Library - "the desktop app", with built in Tor (for seeders
| safety), IPFS (for p2p distribution), IPNS (to download updated
| indexes), with local search engine (no SPOF & convenience),
| optional at-rest file encryption, and some random pining
| algorithm to let users donate 1-10GB of disk to host chunks of
| the library.
|
| It needs to be dead easy to let anyone use and contribute.
| roenxi wrote:
| Between the IPFS & the XMR address at the bottom of the blob post
| - this is using important technology that didn't exist a few
| years ago - this blog post wasn't technically feasible in 2012.
|
| It has become substantially cheaper to do this sort of
| crowdfunded guerrilla knowledge sharing.
| gw67 wrote:
| How to search for a book? Is there a search engine?
___________________________________________________________________
(page generated 2022-11-19 23:00 UTC)