[HN Gopher] Show HN: Arxiv.org on IPFS
___________________________________________________________________
Show HN: Arxiv.org on IPFS
Author : hugoroussel
Score : 221 points
Date : 2021-09-07 07:35 UTC (15 hours ago)
(HTM) web link (www.xirva.org)
(TXT) w3m dump (www.xirva.org)
| brylie wrote:
| I get an error on several articles to the effect of:
|
| invalid ipfs path: invalid path "/ipfs/undefined/2107.00648.pdf":
| invalid CID: expected 1 as the cid version number, got:
| 31965309853
| imback wrote:
| Slightly OT, but how is IPFS these days? Every time I've tried
| using it the past few years it seemed very WIP. Cool sounding
| protocol though.
| jstanley wrote:
| I got turned off it because things that I'd pinned, and _knew_
| I 'd pinned, would eventually become mysteriously inaccessible
| unless accessed through the node that I pinned it on, and I
| couldn't work out why. It's a great idea, but I found it too
| flakey to satisfy me.
| Taek wrote:
| It's still very slow - retrieval times >30s for files that
| aren't cached on cloudflare or ipfs.io. Also, those two
| providers each have multiple periods of downtime every year.
|
| If the files are cached and the services are up, it's plenty
| fast for static data but dynamic data (IPNS) is still very
| slow.
|
| We've built a competitor called Skynet that is much faster
| (less than 200ms for files that aren't in the cache) and scales
| better. It's currently hosting tens of millions of files across
| 200+ TB of data.
|
| We really like the vision that IPFS had and we think
| decentralized data is the future of the Internet. We're proud
| to have put in the legwork to make it practical.
|
| https://docs.siasky.net/
| pimterry wrote:
| Do you have any documentation that compares Skynet to IPFS
| anywhere? From an architectural standpoint and/or features.
| Taek wrote:
| Closest we got at the moment is this:
| https://skynet.guide/discover/storage-chains-
| compared.html#f...
|
| I'm writing a more direct comparison this week, we just
| recently (less than 30 days ago) hit full feature parity
| with ipfs, any webapp or file deployed on IPFS should work
| natively on Skynet now as well. The link/identifier will be
| different but you shouldn't need to change any code
| menmob wrote:
| Are you able to "pin" files yourself like IPFS? Or are you
| required to pay hosts to store it?
| Taek wrote:
| You can always run a host yourself and pin it to your own
| host.
|
| The main reason we chose a host-based architecture instead
| of a pin based architecture is that we saw on IPFS that
| having people pin their own data resulted in really poor
| uptimes, a lot of file rot, and it also substantially
| reduced scalability and increased fetch times. And after
| all of those tradeoffs, the vast majority of accessible
| content on IPFS is hosted via a pinning service anyway.
| a1369209993 wrote:
| > You can always run a host yourself and pin it to your
| own host.
|
| (It sounds like it, but) to clarify, can you do this
| completely for free, with no cooperation from a third
| party (eg, you don't need to pay a existing host to vouch
| for you)?
| Taek wrote:
| No need to pay an existing host or get any sort of
| external party to enable you, it's a permissionless
| network
| menmob wrote:
| Makes sense. I've just been looking at creating a
| distributed YT archive on IPFS, but as you said, load
| times are absolutely terrible, especially for big files
| like video. I've been following sia since 2016-ish, and
| skynet looks awesome, just worried about maturity. I will
| try hosting my own files as you described, thanks!
| amelius wrote:
| Why do you need a blockchain to implement Skynet?
| Taek wrote:
| The blockchain gets us a decentralized marketplace for
| decentralized storage providers. Anyone can join as a
| provider and get paid, and the blockchain can act as a
| decentralized escrow that holds the payment until proof is
| provided that the storage contract was properly fulfilled.
|
| 98% of our technology is off-chain. Only a little tiny
| sliver (the file contract open and close) is actually
| posted to the blockchain.
| woile wrote:
| Wow looks super nice, feels so easy to navigate and read, I was
| thinking it would be nice to have a RSS feed for some topics to
| read as news. Great work!
| eecc wrote:
| Ah cool... I also took a stab at something similar several years
| ago: https://github.com/ecausarano/heron
|
| Also at the time I was considering IPFS.
|
| But I guess the real trick is implementing a WOT to implement
| peer review and filter out the inevitable junk that will be
| published
| hugoroussel wrote:
| Exactly the problem that I have. Many edge cases too.
| westurner wrote:
| "Help compare Comment and Annotation services: moderation,
| spam, notifications, configurability" executablebooks/meta#102
| https://github.com/executablebooks/meta/discussions/102 :
|
| > jupyter-comment supports a number of commenting services
| [...]. In helping users decide which commenting and annotation
| services to include on their pages and commit to maintaining,
| could we discuss criteria for assessment and current features
| of services?
|
| > Possible features for comparison:
|
| > * Content author can delete / hide
|
| > * Content author can report / block
|
| > * Comments / annotations are screened by spam-fighting
| service
|
| > * Content / author can label as e.g. toxic
|
| > * Content author receives notification of new comments
|
| > * Content author can require approval before user-contributed
| content is publicly-visible
|
| > * Content author may allow comments for a limited amount of
| time (probably more relevant to BlogPostings)
|
| > * Content author may simultaneously denounce censorship in
| all it's forms while allowing previously-published works to
| languish
|
| #ForScience
| westurner wrote:
| FWIW, archiving repo2docker-compatible git repos with a DOI
| attached to a git tag, is possible with JupyterLite:
|
| > _JupyterLite is a JupyterLab distribution that runs
| entirely in the browser built from the ground-up using
| JupyterLab components and extensions_
|
| With JupyterLite, you can build a static archive of a
| repo2docker-like environment so that the ScholarlyArticle
| notebook or computer modern latex css, its SoftwareRelease
| dependencies, and possibly also the Datasets can be run in a
| browser tab with WASM. HTML + JS + WASM
| kloch wrote:
| What do you mean by junk? Spam and Abuse, or scientific papers
| that the reviewer simply doesn't like?
| sebmellen wrote:
| Hey Hugo, do you have an email I could reach you at? I've been
| thinking/working on these problems for 3 years now and would love
| to find some smart people to partner with to further develop the
| ideas. I don't have much to show publicly right now, but
| https://intpub.org/ (soon to be scipub.app) is the start.
| [deleted]
| eecc wrote:
| Also keep me in the loop, plz :)
| sebmellen wrote:
| For sure! Will do.
| betwixthewires wrote:
| Oh wow I'm going to be watching your github repo, what you're
| doing looks interesting.
|
| Do you have any plans to implement some search functionality to
| help find documents, or is it about permissionless publishing
| only?
| MayeulC wrote:
| Doesn't seem to work (I don't have a local gateway on this
| machine):
|
| https://www.xirva.org/list/eess.IV/2011/2011.00052
|
| Brings me to https://ipfs.io/ipfs/undefined/2011.00052.pdf
|
| which says: invalid ipfs path: invalid path
| "/ipfs/undefined/2011.00052.pdf": invalid CID: expected 1 as the
| cid version number, got: 31965309853
| hugoroussel wrote:
| Not all articles are uploaded yet, it is still WIP.
| fnord77 wrote:
| is search still a WIP?
| hugoroussel wrote:
| Everything is still WIP, very difficult to do rich indexing
| on IPFS files. A new project idea ?
| hevalon wrote:
| Just a heads up comment, it seems some of the IPFS links are
| broken. For example I just visited
| https://www.xirva.org/categories/cs.AI and all links point to an
| "undefined" path example
| https://ipfs.io/ipfs/undefined/2107.00082.pdf
|
| But overall, great idea!
| hugoroussel wrote:
| Not all articles are uploaded yet, it is still WIP.
| hevalon wrote:
| that's fine. Maybe then for a better user experience have
| grey-out or removed the IPFS download button for the articles
| that haven't been uploaded yet.
| meroje wrote:
| links ? everything is a fake js button
| T3RMINATED wrote:
| ipfs resolve -r /ipfs/bafybeiamnft57u4lniylffuafzxouf4ocaw5uylahf
| qoawj6rjnvwthuuu/list/cs.CV/2111/: no link named "list" under
| bafybeiamnft57u4lniylffuafzxouf4ocaw5uylahfqoawj6rjnvwthuuu
| leavenotracks wrote:
| Search bar doesn't seem to be working...
| hugoroussel wrote:
| yes
| T-A wrote:
| From the repo [1]:
|
| "research publishing platform that is community based,
| transparent and _censorship resistant_ " (my emphasis)
|
| "Community members moderate the platform and _can increase or
| decrease the visibility of the uploaded files_ " (again, my
| emphasis)
|
| [1] https://github.com/hugoroussel/xirva
| prvc wrote:
| >censorship _resistant_
|
| My emphasis.
| Jhsto wrote:
| How does the NFT thing work? I do not have Polygon tokens but I
| wonder what have you envisioned as the functionality of this to
| be?
| hugoroussel wrote:
| Since it was for a ETHGlobal hackathon I thought it would be
| fun to experiment with the feature where we mint an NFT where
| the metadata points to the IPFS link. You could then do
| whatever you do with an NFT.
| Jhsto wrote:
| I wonder how much information do you get from arxiv regarding
| the dependency graph via citations. Could there be a way that
| I, as I upload my own manuscript, to tip the authors of the
| people I citated and conversely someday have the possibility
| of also generating revenue personally as such? It would be
| much nicer, I think, if the fees that are currently paid to
| journals to instead go to the authors that also contributed
| to my work.
| hugoroussel wrote:
| There are for sure interesting ideas related to new forms
| of scientific funding. The issue I have and why the project
| is currently on standby is how to combat spam/hoax articles
| flexd wrote:
| Hmm, my Firefox here on Ubuntu does not trust that CA for some
| reason. And clicking "Accept the risk and continue" seems to just
| land me back at the "Warning: Potential Security Risk Ahead"
| page.
| hugoroussel wrote:
| I think only www.xirva.org has valid certificate, maybe Firefox
| strips the www for some reason.
| hugoroussel wrote:
| Build this during an hackathon where I felt like that arXiv was
| in grand need of a small face lift. Currently not all articles
| are uploaded. The repository is here :
| http://github.com/hugoroussel/xirva
| stevejpurves wrote:
| does the upload here also submit to arxiv, or will uploading to
| xivra mean they diverge?
| hugoroussel wrote:
| It will diverge.
| ithinkso wrote:
| Not to be confused with viXra, which is a sci-fi alternative to
| arXiv
| kloch wrote:
| viXra is full of nonsense but then again so is arXiv (see:
| 750GeV debacle). viXra desperately needed a voting system and
| if it did it would likely have been much more useful and become
| a viable alternative to arXiv.
|
| arXiv does not accept papers from authors with no institutional
| affiliation and viXra was the only (and ugly) alternative.
| There is an opportunity there to fix both sites.
| detaro wrote:
| What was the 750GeV debacle? That thing were some people said
| LHC had found a new particle maybe?
| kloch wrote:
| A spurious signal at LHC (that disappeared in later runs)
| that spurred a cottage industry of arXiv submissions trying
| to explain it. Authors, mostly grad students, quickly
| submitted hundreds of low quality articles trying to get in
| on the "discovery". Later articles would cite over 400
| previous articles and it became one giant circle jerk.
| There were even blog posts complaining about the blatant
| "ambulance chasing".
|
| https://motls.blogspot.com/2016/06/ambulance-chasing-is-
| just...
|
| http://resonaances.blogspot.com/2016/06/game-of-
| thrones-750-...
| hugoroussel wrote:
| I nearly called the project this name without knowing it
| existed. Thanks for the share.
| aerique wrote:
| Cool.
|
| Now the same for Sci-Hub?
| onhn wrote:
| "You can upload your research and publish it on the open web.
| Members of the community will be able to vote on your research to
| raise its visibility."
|
| Oh dear.
| Taek wrote:
| How would you set it up? The decentralized world doesn't really
| have a great system for curation at this point (unless you can
| point to a counterexample!), and so I'm in favor of any sort of
| playing around with decentralized voting/curation until we find
| something that seems to be working well.
| onhn wrote:
| The standard and most effective form of curation in science
| is the reference list at the end of a paper.
|
| But usually you just read everything that is relevant to your
| research interests from the daily arxiv posting.
| SkyMarshal wrote:
| Start from the objective of first do no harm. Voting systems
| may eventually be gamed to distort results, so eliminate the
| voting system. Instead rely on ad-hoc personal networks to
| disseminate signal about quality papers out-of-band. Don't
| assume you have to systematize everything.
| jamescampbell wrote:
| Voting (as was bore out in many examples including digg.com
| and elsewhere) becomes a mob rule situation and variation of
| tyranny of the commons without a novelty algorithm in
| addition to total votes. If you just go by totals, it will be
| easily gamified and rendered useless as a metric.
| [deleted]
| baby wrote:
| Sounds amazing to me
| f0e4c2f7 wrote:
| A perfect system? No, but think about how people must have felt
| about Wikipedia on launch.
|
| Love this idea.
| gambler wrote:
| Using Wikipedia as an example of a seemingly naive idea that
| was ultimately proven to work is a pretty bad argument that
| completely ignores how Wikipedia operates at the moment.
|
| It's routinely used for propagating smears:
|
| https://odysee.com/@AlisonMorrow:6/how-wikipedia-decides-
| if-...
|
| Even one of its co-founders says it's failing as an accurate
| source of information:
|
| https://odysee.com/@TimcastIRL:8/former-founder-of-
| wikipedia...
|
| Just like Jaron Lanier predicted in 2006:
|
| https://www.edge.org/conversation/jaron_lanier-digital-
| maois...
|
| I never understood why so many technologists vehemently
| defend a website that was _obviously_ prone to a form of
| "regulatory capture" and groupthink.
| bigphishy wrote:
| In all my experience using wikipedia it has been successful
| at providing facts and accurate references.
|
| I don't mean to attack the speaker here, but that former
| cofounder of wikipedia you just cited... isn't he an
| extremist neo-conservative? Why did he leave wikipedia in
| the first place? What are his proposed solutions?
| NmAmDa wrote:
| Actually I don't think science has democratic nature. Yes we
| do somehow do that as a theory would still need to be
| accepted widely. But in reality one person can have the
| correct idea while all others disagree. Still this person is
| doing it right.
| wongarsu wrote:
| Scientific consensus is democratic in nature (even though
| votes are not distributed evenly). The ideal is that
| through reproducible experiments and application of the
| scientific method the scientific consensus moves to
| increasingly accurate models of reality over time. But
| obviously the speed at which that happens varies, and some
| right ideas took annoyingly long to get accepted into
| scientific consensus.
| kloch wrote:
| Sure the right answer will eventually prevail but the
| process is much worse than we like to admit. Many
| breakthrough advances were outright rejected by
| contemporary peers when first proposed.
|
| "Fermi first submitted his "tentative" theory of beta
| decay to the prestigious science journal Nature, which
| rejected it "because it contained speculations too remote
| from reality to be of interest to the reader." Nature
| later admitted the rejection to be one of the great
| editorial blunders in its history. ... Fermi found the
| initial rejection of the paper so troubling that he
| decided to take some time off from theoretical physics,
| and do only experimental physics"
| https://en.wikipedia.org/wiki/Fermi%27s_interaction
| lmohseni wrote:
| I believe science is a democratic process. If someone has
| the correct idea but communicates it poorly, so poorly that
| others in the field disagree, then this person is doing it
| wrong. (Thinking specifically of
| https://en.m.wikipedia.org/wiki/Shinichi_Mochizuki )
| hugoroussel wrote:
| All and all it is a non trivial problem. You have at the
| very least have to attach some kind of form of reputation
| system into the verification process. Even with that you
| will still have the "misunderstood genius" issue, or the
| "excellent reputation professor" that everyone trust
| without (enough) verification.
| elcritch wrote:
| But at least there's be a system for other researchers to
| record "failed to replicate" that could give a channel to
| critique reputable professors that's not controlled by
| the same professors (as they often can in journals).
| nextaccountic wrote:
| The participation ought to be democratic in the sense of
| being open to everyone to participate. But, you can't do
| a vote and use it to decide who is right. Deep down we
| know that being right or wrong is independent from the
| scientific consensus. Mochizuki may be interacting with
| the scientific community in the wrong way, but it has no
| bearing on whether his theory is correct.
|
| The consensus itself has some democratic features, but
| it's weighed by prestige and adherence to the current
| paradigm. I think Kuhn described its mechanism pretty
| well. It's far easier to convince people of a wrong
| result if you follow the established paradigm, than
| convince people of something right if you go against it.
| What really saves science from being pure dogma is that
| there are paradigm shifts, revolutions in which the
| scientific consensus change.
| rglover wrote:
| https://www.youtube.com/watch?v=GA0l1JXhLaI&t=19s
___________________________________________________________________
(page generated 2021-09-07 23:01 UTC)