[HN Gopher] I Moved My Blog from IPFS to a Server
___________________________________________________________________
I Moved My Blog from IPFS to a Server
Author : neiman
Score : 119 points
Date : 2024-01-31 20:07 UTC (2 hours ago)
(HTM) web link (neimanslab.org)
(TXT) w3m dump (neimanslab.org)
| schmichael wrote:
| > I pinned content from my own server and played forever with
| settings and definitions, but couldn't get the content to be
| available everywhere.
|
| > The blog you're reading now is built with Jekyll and is hosted
| on my own 10$ server.
|
| > don't get me wrong, I'm still an IPFS fanboy.
|
| ...how could you still be a fanboy? When IPFS cannot fulfill even
| the most basic function of globally serving static content, why
| does it deserve anyone's interest? It's not even new or cutting
| edge at this point. After 8 years of development how can the most
| basic functionality still not work for even an expert?
| neiman wrote:
| It worked fine before for many years when it was slightly less
| popular.
|
| They're having growing pains due to scalability problems and
| some libraries, like Helia (the JS library of IPFS), being new.
| I guess I'm also quite stubborn in wanting to do it my way,
| without the aid of any services, and for the content I pin to
| be available in all places, including Helia in the browser.
| p4bl0 wrote:
| The official IPFS client in Go has always been very, very
| hungry for resources. At some point it crashed because it
| needed to many file descriptors if it ran for too long. Even
| for a simple static site with infrequent update it needed
| some maintenance. But even if one was willing to put the
| effort in, it was not actually rewarded, because if your
| server were you pin your website is offline, the truth is
| that your site is offline too, so what's the point?
| withinboredom wrote:
| [wrong thread]
| TheRealPomax wrote:
| What on earth are you talking about? IPFS has nothing to do
| with "coins", it's a distributed data management system. You
| can use IPFS for finance in the same way you can use a
| database for finance. You can also use IPFS for literally
| anything else that falls in the "data that you want other
| people to be able to access" category.
| withinboredom wrote:
| oh shit, I replied to the wrong comment.
| heipei wrote:
| It's working fabulously for hosting phishing websites frontend
| by the popular IPFS gateway providers like Cloudflare, so at
| least there's that...
| lindig wrote:
| Filecoin, which is based on IPFS, creates a market for unused
| storage. I think that idea is great but for adoption it needs to
| be as simple as Dropbox to store files. But visit
| https://filecoin.io/ and the dropbox-like app that you could be
| willing to try is nowhere to be found. So maybe it is an
| enterprise solution? That isn't spelled out either. So I am not
| surprised that this has little traction and the article further
| confirms the impression.
| pierat wrote:
| Here's your $.10/day for that 1GB with bandwidth... but running
| the filecoin stack will cost you a $50/mo server.
|
| That fucker's a PIG on cpu and ram.
| kkielhofner wrote:
| IPFS is as well.
|
| Clearly much more going on but take a machine that can serve
| 10k req/s with [insert 100 things here] without flinching and
| watch it maybe, just maybe, do 10 with IPFS.
|
| I'm not kidding.
| diggan wrote:
| > to be as simple as Dropbox to store files. But visit
| https://filecoin.io/ and the dropbox-like app that you could be
| willing to try is nowhere to be found
|
| I agree with this fully. But as said elsewhere, it's kind of
| far away from that, and also slightly misdirected.
|
| Imagine asking someone to get started with web development by
| sending them to https://www.ietf.org/rfc/rfc793.txt (the TCP
| specification). Filecoin is just the protocol, and won't ever
| solve that particular problem, as it's not focused on solving
| that particular problem, it's up to client implementations to
| solve.
|
| But the ecosystem is for sure missing an easy to use / end-user
| application like Dropbox for storing files in a decentralized
| and secure way.
| poorman wrote:
| That flagship app you are looking for seems to be
| https://nft.storage/ (by Protocol Labs).
| nickstinemates wrote:
| this is what storj.io does.
| ahmedfromtunis wrote:
| This is, in my opinion, is the first and only "solution" to a
| real problem built using the blockchain.
|
| Distributed file storage, if done correctly, can be a
| transformative technology. And it can be even more
| revolutionary implemented at the OS level.
| p4bl0 wrote:
| I'm surprised by the beginning of the post talking about
| pioneering in 2019. Maybe it is the case for ENS (I never cared
| for it), but regarding IPFS, my website was available over IPFS 3
| years before that in 2016 [1]. Granted, I was never IPFS only. I
| also started publishing a series of article about censorship-
| resistant internet (Tor, I2P, IPFS, and ZeroNet) in 2600 Magazine
| - _The Hacker Quarterly_ - back in 2017 [2].
|
| Anyway, I came to the same conclusion as the author, but several
| years ago: in the end, nothing is actually decentralized, and
| maintaining this illusion of decentralization is actually costly,
| for no real purpose (other than the initial enjoyment of playing
| with a new tech, that is).
|
| So I stopped maintaining it a few years ago. That decision was
| also because of the growing involvement of some of these projects
| with blockchain tech that I never wanted to be a part of. This is
| also why I cancelled my article series in 2600 before publishing
| those on IPFS and ZeroNet.
|
| [1] See for example this archive of my HN profile page from 2016
| with the link to it:
| https://web.archive.org/web/20161122210110/https://news.ycom...
|
| [2] https://pablo.rauzy.name/outreach.html#2600
| neiman wrote:
| > Maybe it is the case for ENS
|
| Oh yeah, I was referring strictly to IPFS+ENS websites. I have
| been working with it for several years so my mind goes for this
| use-case automatically.
| r3trohack3r wrote:
| > Anyway, I came to the same conclusion as the author, but
| several years ago: in the end, nothing is actually
| decentralized, and maintaining this illusion of
| decentralization is actually costly, for no real purpose (other
| than the initial enjoyment of playing with a new tech, that
| is).
|
| Do you have any writing (blog posts, HN comments, etc.) where
| you explore this thought more? I'm in the thick of building p2p
| software, very interested in what you came to know during that
| time.
| pphysch wrote:
| True P2P networks don't scale, because every node has to
| store an (accurate if partial) representation of the whole
| network. Growing the network is easy, but growing every
| single node is virtually impossible (unless you control them
| all...). Any attempt to tackle that tends to increase
| centralization, e.g. in the form of routing authorities.
|
| And if you try to tackle centralization directly (like
| banning routers or something), you will often create an anti-
| centralization regulator, which is, you guessed it, another
| form of centralization.
|
| So your decentralized P2P network is either small and works
| good, medium and works not so good, or large and not actually
| decentralized.
|
| The best P2P networks know their limits and don't try to
| scale infinitely. For human-oriented network, Dunbar's Number
| (N=~150) is a decent rule of thumb; any P2P network larger
| than that almost certainly has some form of centralization
| (like trusted bootstrapping/coordination server addresses
| that are hard-coded in every client install, etc.)
| KMag wrote:
| > True P2P networks don't scale, because every node has to
| store an (accurate if partial) representation of the whole
| network
|
| Former LimeWire dev here... which P2P networks use a fully
| meshed topology? LimeWire and other Gnutella clients just
| have a random mesh with a fixed number of (ultra)peers. If
| the network gets too large, then your constrained broadcast
| queries hit their hop count before reaching the edge of the
| network, but that seems fine.
|
| Last I checked, Freenet used a variation on a random mesh.
|
| Kademlia's routing tables take O(log(N)) space and traffic
| per-peer to maintain (so O(N log(N)) for global total
| network space and traffic). Same for Chord (though, twice
| as much traffic due to not using a symmetric distance
| metric like XOR).
|
| There are plenty of "True" (non-centralized) P2P networks
| that aren't fully meshed.
| sanity wrote:
| Creator of Freenet here. Freenet[1] relies on peers self-
| organizing into a small world network[2].
|
| Small world networks have the advantage of being able to
| find data in log N time where N is the network size,
| they're also completely decentralized, self-healing, and
| distribute load evenly across peers. The principle is
| similar to DHTs like Kademlia but more flexible and
| intuitive IMO, while having similar scaling
| characteristics.
|
| It's surprisingly common for people to confuse small
| world networks with "scale free networks", but scale free
| networks rely on a subset of highly connected peers which
| do a disproportionate amount of the work - which isn't
| truly decentralized.
|
| The new Freenet design incorporates learning into the
| routing algorithm. When a peer is deciding to route a
| message, it predicts the response probability and time
| for each neighboring peer based on past performance and
| chooses the best. With conventional "greedy routing"
| peers just choose the neighbor with a location closest to
| the data being retrieved. The new approach is adaptive to
| actual network performance.
|
| [1] Both the original Freenet from 1999 and the new
| sequel we're currently building - see
| https://freenet.org/ for more. We hope to launch the
| network in the next few weeks.
|
| [2] https://en.wikipedia.org/wiki/Small-world_network
| p4bl0 wrote:
| The main thing is that "true" (in the sense of absolute)
| decentralization does not actually work. It doesn't work
| technically, and it doesn't work socially/politically. We
| always need some form of collective control over what's going
| on. We need moderation tools, we need to me able to make
| errors and fix them, etc. Centralized systems tend to be
| authoritarians, but the pursue of absolute decentralization
| always end up being very individualistic (and some kind of
| wrongly placed elitism). There are middle grounds: federated
| systems for example, like Mastodon or emails, actually work.
|
| That is not to say that all p2p software is bad, especially
| since we call p2p a lot of things that are not entirely p2p.
| For example, BitTorrent is a p2p software, but its actual
| usage by humans relies on multiple more-or-less centralized
| point, trackers and torrent search engine.
| Almondsetat wrote:
| Your software cannot be more decentralized than your
| hardware.
|
| For example, true p2p can only happen if you meet with
| someone and use a cable, bluetooth or local wifi. Anything
| over the internet needs to pass through routers and *poof*
| decentralization's gone and you now need to trust servers to
| a varying level of degrees
| colordrops wrote:
| "varying" is a pretty wide range here. If you mean "trust"
| as in trust to maintain connectivity, yes, but beyond that
| there are established ways to create secure channels over
| untrusted networks. Could you provide specifics about what
| you mean if anything beyond basic connectivity?
| Almondsetat wrote:
| Sure: what is the process to start downloading a torrent?
| what is the process to message someone on Jami? what is
| the process to call someone on (old) Skype or Jitsi?
| Answer this and you realize you can only get as
| decentralized as your hardware infrastructure
| int_19h wrote:
| I may be missing something, but name resolution has been touted
| as one of the more legitimate and sensible uses for blockchain
| for a very long time. Could you clarify what your issues with
| it in IPFS context are?
| edent wrote:
| It isn't. Unless you want a long incomprehensible string.
|
| Someone is always going to want a short, unique, and
| memorable name. And when two people share the same name
| (McDonald, Nissan, etc) there needs to be a way to
| disambiguate them.
|
| If people die and are unable to release a desirable name,
| that just makes the whole system less desirable.
|
| I know one of the canonical hard problems in Computer Science
| is "naming things" and this is a prime example!
| bombcar wrote:
| And if you want a long incomprehensible string we already
| have that .onion sites work without a blockchain, too.
| ShamelessC wrote:
| > but name resolution has been touted as one of the more
| legitimate and sensible uses for blockchain for a very long
| time.
|
| Blockchain enthusiasts have a history of talking out of their
| ass and being susceptible to the lies of others.
| p4bl0 wrote:
| Well, I do not actually believe that blockchains can do name
| resolution correctly. First and foremost, the essential thing
| to understand about blockchains is that the only thing that
| is guaranteed by writing an information on a blockchain is
| that this information is written on this blockchain. And
| that's it. If the writing itself is not performative, in that
| it's mere existence performs what it describes, then nothing
| has been done. It works for crypto-assets because what makes
| a transaction happen is that it is written on the blockchain
| where that crypto-asset lives and where people look to see
| what happened with that crypto-asset.
|
| But for _any_ other usage, it cannot work, blockchain are
| useless. Someone or something somewhere has to make sure
| either that what 's written on the blockchain corresponds to
| the real world, or to make the real worlds corresponds to
| what's written on the blockchain. Either way you need to have
| a kind of central authority, or at least trusted third
| parties. And that means you don't actually need a blockchain
| in the first place. We have better, more secured, more
| efficient, less costly alternatives to using a blockchain.
|
| Back to name resolutions.
|
| Virtually no one is going to actually host locally the
| blockchain where all names are stored. That would be way too
| big and could only get bigger and bigger, as a blockchain
| stores transactions (i.e., diffs) rather than current state.
| So in practice people and their devices would ask resolvers,
| just like they currently do with DNS. These resolvers would
| need to keep a database of the state of all names up-to-date
| because querying a blockchain is way too inefficient, running
| such a resolvers would be a lot more costly than running a
| DNS servers so there would be less of them. Here we just lost
| decentralization which was the point of the system. But
| that's just a technical problem. There is more: what if
| someone gets a name and we as a society (i.e., justice,
| whatever) decides that they should not be in control of it?
| Either we are able to enforce this decision and it means the
| system is not actually decentralized (so, we don't need a
| blockchain), or we can't, and that's a problem. What if a
| private key is lost, the associated names are gone forever?
| What if your private key is leaked by mistake and someone
| hostile take control of your name?
|
| Using a blockchain for names resolution doesn't actually
| work, not for a human society.
| DEDLINE wrote:
| When evaluating use-cases where blockchain technology is
| leveraged to disintermediate, I came to your same conclusions.
| Technically novel? Yes, sure. But, for what?
| hanniabu wrote:
| For incentive alignment, consensus, trustless, etc
| chaxor wrote:
| I never fully understood the use of ipfs/iroh for websites, but
| I _really_ like the idea for data caching in data science
| packages.
|
| It makes me more sense to me that someone would be much more
| willing to serve large databases and neural network weights
| that _they actually use everyday_ , rather than 'that one guys
| website they went to that one time'.
|
| I'm very surprised it's not as popular, if not _more_ popular
| to just have @iroh-memoize decorators everywhere in people 's
| database ETL code.
|
| _That 's_ a better use case (sense the user has a vested
| interest in keeping the data up) than helping people host we
| sites.
| nonrandomstring wrote:
| Well done to the author for writing this up.
|
| Having tried fringe technologies over the years, spun up a server
| and run them for a few months, struggled and seen all the rough
| edges and loose threads, I often come to the point of feeling -
| this technology is good, but it's not ready yet. Time to let more
| determined people carry the torch.
|
| The upside is:
|
| - you tried, and so contributed to the ecosystem
|
| - you told people what needs improving
|
| Just quitting and not writing about your experience seems a waste
| for everyone, so good to know why web hosting on IPFS is still
| rough.
| yieldcrv wrote:
| I host all the static files of my Netlify and Vercel servers on
| IPFS
|
| it is simple enough and free even on hosted solutions, and it
| keeps my Netlify and Vercel free during spikes in traffic
|
| but the availability issue is perplexing, just like OP
| encountered
|
| some people just randomly wont be able to resolve some assets on
| your site, sometimes! the gateways go up and down, their cache of
| your file comes and goes. browsers dont natively resolve ipfs://
| uris. its very weird.
| indigodaddy wrote:
| " and it keeps my Netlify and Vercel free during spikes in
| traffic" -- how exactly would this help re: potentially
| breaching outbound transfer limits?
| yieldcrv wrote:
| if static assets arent being requested from their server then
| it would not contribute to your bandwidth meter
| indigodaddy wrote:
| You still have to transfer that data through the network
| pipe to the end user no? How the server accesses them seems
| irrelevant to me.
| MenhirMike wrote:
| > the more readers it has, the faster it is to use it since some
| readers help to spread the content (Scalable).
|
| In other words: Once a few big websites are established, no small
| website will ever be able to gain traction again because the big
| websites are simply easier to reach and thus more attractive to
| use. And just like an unpopular old torrent, eventually you run
| out of seeders and your site is lost forever.
|
| One can argue about the value of low traffic websites, but I got
| to wonder: Who in their right mind thinks "Yeah, I want to make a
| website and then have others decide if it's allowed to live".
| Then again, maybe that kind of "survival of the fittest" is
| appealing to some folks.
|
| As far as I am concerned, it sounds like a stupid idea. (Which
| the author goes into more detail, so that's a good write up)
| lelandbatey wrote:
| It's not up to others alone; you get a say too because you can
| seed your own content and that can be fast. In the worst case
| of no interest, then it's approximately the same as you hosting
| your own website in the world of today. This doesn't exonerate
| the shortfall of the "old torrent" pattern though, as you say.
| fodkodrasz wrote:
| This is a false dilemma. Why would you not "seed" (pin) your
| own site, and be at others' mercy? You pin it, and when others
| also do so, the readers get faster and more redundant service.
| kimixa wrote:
| For "unpopular" sites having a single origin somewhat removes
| the advantages of IPFS, it's not decentralized, not
| censorship resilient, and still costs the publisher for
| ongoing infrastructure to host it. Yet still had the
| disadvantages and complexity of IPFS vs a static http server.
|
| So if you're not going to be publishing something that will
| always have multiple copies floating around, why use IPFS?
| fodkodrasz wrote:
| 1. to give a chance to avoid being slashdotted. 2. to allow
| anybody who finds it valuable to archive it, or parts of
| it?
|
| The complexity of IPFS is another thing, which should be
| solved. However popular or unpopular your site might be,
| you must host is somewhere somehow, if you wish to be sure
| it sticks around. It is simple as that.
| cle wrote:
| It helps to use more specific terms than "decentralized"
| and "censorship resilient", there are a lot of attack
| vectors for both. IPFS certainly does address some of the
| attack vectors, but not all. For example if the
| "centralized" thing you're worried about is DNS and
| certificate authorities, then you can avoid those
| authorities entirely with IPFS. Replication is one aspect
| of centralization, and IPFS doesn't completely fail at it,
| it's just more expensive (you can guarantee two node
| replication, you will have to run two nodes though). And
| there are other aspects not addressed by IPFS at all like
| its reliance on IP address routing infrastructure.
| p4bl0 wrote:
| If you need to pin your content anyway, it's actually faster
| and less expensive to host a normal website then. And if you
| want to get it to readers faster, there are a lot a cheap or
| free CDN available, but that's generally not even an issue
| with the kind of website we're talking about here when
| they're served normally, over the web.
| fodkodrasz wrote:
| Yes, that is the state of affairs now. I can use cloudfront
| for my site, but cannot use it to pin my ipfs site (should
| I have one) as far as I know.
|
| You are fighting a strawman. If you don't take of your
| site, but expect others to take care of it (pin it), then
| it is not your site. You must ensure it has at least one
| pinned version. Others might o might not pin it, it depends
| on the popularity, or the accessibility of the stack, which
| is lacking right now according to the article.
| koito17 wrote:
| > it's quite an inconvenience to run your own IPFS node. But even
| if you do run your own node, the fact you access a website
| doesn't mean you pin it. Not at all.
|
| This has always been my major UX gripe with IPFS. The fact that
| `ipfs add` in the command line does little but generate a hash
| and you need to actually pin things in order to "seed" them, so
| to speak. So "adding a file to IPFS", in the sense of "adding a
| file to the network", requires the user to know that (1) the
| "add" in `ipfs add` does not add a file to the network, and (2)
| you must pin everything you want to replicate manually. I
| remember as recently as 2021 having to manually pin each file in
| a directory since pinning the directory does not recursively pin
| files. Doing this by hand for small folders is okay, but large
| folders? Not so much.
|
| More importantly, the BitTorrent duplication problems that IPFS
| has solved are also solved in BitTorrent v2, and BitTorrent v2
| IMO solves these problems in a much better way (you can create
| "hybrid torrents" which allows a great deal of backwards
| compatibility with existing torrent software).
|
| This isn't a UX issue, but another thing that makes it hard for
| me to recommend IPFS to friends is the increasing association
| with "Web3" and cryptocurrency. I don't have any strong opinions
| on "Web3", but to many people, it's an instant deal-breaker.
| hot_gril wrote:
| IPFS provides nice stable links to media, and there are
| HTTP->IPFS gateways if needed. That seems useful for embedding
| content on multiple apps/sites. Yeah it happens to fit NFTs
| particularly well, then again we all know what BitTorrent is
| known for. And yes I agree IPFS has some UI problems.
|
| Would BitTorrent also be suitable for hosting embeddable
| content? I haven't seen that yet. A magnet URL is mainly a file
| hash and doesn't seem to encode a particular peer server, kinda
| like IPFS. But every time I've torrented Ubuntu, it's taken
| half a minute just to find the peers.
| mvdtnz wrote:
| > IPFS provides nice stable links to media
|
| Anyone who has tried to torrent an old movie or lesser-known
| television show knows this is simply not true.
| hot_gril wrote:
| I mean it's not like HTTP where all URLs are tied to a
| particular webserver. If someone different starts seeding,
| you'll get the same data again at the same URL, with built-
| in checksumming.
| adamzochowski wrote:
| That is something I wanted to know, does IPFS guarantee
| that same two files have same two IPFS URLs / hash links?
|
| Otherwise, someone sharing same data again, because it
| will be in different IPFS folder won't be actually
| discoverable as same data.
| hot_gril wrote:
| Yes, a file's hash is only based on its contents. The way
| I understand it, a file doesn't really live in a
| directory, it's more like a directory (which is a kind of
| file itself) references files. So the same file can be in
| two directories, yet it'll have the same URL/hash.
| flashm wrote:
| 'ipfs add' pins the file by default, not sure if that's recent
| behaviour though.
| b_fiive wrote:
| Totally biased founder here, but I work on
| https://github.com/n0-computer/iroh, a thing that started off as
| an IPFS implementation in rust, but we broke out & ended up doing
| our own thing. We're not at the point where the iroh implements
| "the full IPFS experience" (some parts border on impossible to do
| while keeping a decentralized promise), but we're getting closer
| to the "p2p website hosting" use case each week.
| hot_gril wrote:
| Is it named after the Avatar character?
| joshspankit wrote:
| Yes, and that makes me happy every time I see it.
| b_fiive wrote:
| I can neither confirm nor deny, but oh boy does uncle iroh
| seem cool
| axegon_ wrote:
| I see where the author is coming from but I find something else
| strange: Considering that the blog is in practice a collection of
| static files, I don't see the benefit of paying for a server at
| all. Host it on github, if github gets killed off for whatever
| reason, switch to something else and move on. Seems like an
| unnecessary overhead to me.
| hot_gril wrote:
| Same. IPFS seems far more useful for hosting static content
| that might be embedded in _multiple_ websites.
| neiman wrote:
| I get told that a lot! xD
|
| My original aim was to write an IPFS blogging engine for my
| personal use, so I needed some dynamic loading from IPFS there.
|
| Now I switched to Jekyll, and it would be easier to host the
| blog on Github indeed, but I'm kind of playing a quixotic game
| of trying to minimize the presence of Google/Microsoft/Amazon
| and other big-tech in my life.
| ChrisArchitect wrote:
| Aside: visited this curious as to what the Nieman Journalism Lab
| (https://www.niemanlab.org/) was doing with IPFS if anything. Not
| the nicest near-collision naming move.
| shp0ngle wrote:
| Yeah this is exactly my experience with IPFS. Nobody actually
| uses IPFS directly, and even those few that do never actually pin
| anything because it's an extra step.
|
| (Also I heard it's computationally costly, but I am not sure if
| it's true, I can't imagine why it would be the case actually.)
|
| As a result it's actually more centralised than web, there are
| like 3 pinning services that everyone uses. At which point I
| don't get the extra hoops.
| filebase wrote:
| Did you try IPNS Names? - https://filebase.com/blog/unveiling-
| names-a-dive-into-ipns-o...
| neiman wrote:
| Oh yes, I even had a free IPNS pinning service (for community
| members mostly) built with a friend.
|
| https://dwebservices.xyz/
| mbgerring wrote:
| Did they ever address the issue with IPFS where running a node
| necessarily required you to distribute an unknowable amount of
| pieces of material you may not want to be hosting (like CSAM, for
| example)?
| alucart wrote:
| I'm exploring a similar project, having a "decentralized" website
| (hosted on github) which saves users' data in the blockchain
| itself and then provides that same data to other users through
| public APIs and/or from actual blockchain clients.
|
| Wonder if there is actual use or need for such thing.
| tempaccount1234 wrote:
| As a user I'd stay away from sharing IPFS because of legal
| reasons. Just like torrenting, by actively distributing content I
| take up legal responsibility (at least in Europe where I'm
| located) - that's risk is tolerable for a legal torrent because
| the file doesn't change over time. For a changing web site, I'd
| constantly have to monitor the site for changes or trust the site
| 100% - which is not happening as soon as the site is somewhat
| controversial...
| wyck wrote:
| I build a blog in IPFS, its basically reliant on several
| centralized services to actually work in browsers (DNS , GitHub,
| Fleek, etc). I wrote about how I build it here, the experaince
| was underwhelming. https://labcabin.eth.limo/writing/how-site-
| built.html
___________________________________________________________________
(page generated 2024-01-31 23:00 UTC)