[HN Gopher] Nebulus: An IPFS-Less IPFS
___________________________________________________________________
Nebulus: An IPFS-Less IPFS
Author : skogard
Score : 104 points
Date : 2021-07-19 14:56 UTC (8 hours ago)
(HTM) web link (nebulus.dev)
(TXT) w3m dump (nebulus.dev)
| chejazi wrote:
| Reminds me of Camlistore, one of the earlier content-addressable
| personal storage systems, by Brad Fitzpatrick. Now called Perkeep
| apparently https://perkeep.org/
| adkadskhj wrote:
| This looks cool! I'm writing an immutable data store, with my own
| hashes as IDs; I wonder how viable it would be to use IPFS CID's
| as my IDs?
|
| I expect the trouble there would be how the data is chunked. If
| IPFS disagrees about the chunking we would have different CIDs.
|
| .. also i'd need to implement all of this myself to get it in
| Rust. Hmm.
| Jayschwa wrote:
| https://github.com/ipfs-rust
|
| Maybe some of the building blocks you need can be found in
| there somewhere.
| handrous wrote:
| Not implementing the core of the protocol in something
| portable-library-friendly has been a real drag on IPFS. It's
| made it, in practice, more of a program than a protocol.
|
| The counter-argument is that they'd not have made so much
| progress in the first place, in the short list of languages
| that qualify as portable-library-friendly.
| CGamesPlay wrote:
| What I really want is a private, secure IPFS. I do want the
| network functionality, but I want it to exist only between my
| nodes. I know I can pin encrypted blobs, but I want something
| that handles this transparently. Is this possible somewhere?
| mtnGoat wrote:
| run IPFS node in --offline so it doesnt connect to unwanted
| peers and then stick it behind a firewall so access is
| controlled?
| handrous wrote:
| Yeah. You'll need to run your own bootstrap node(s) and create
| a custom key for the swarm. IIRC that's about all it takes.
|
| A bootstrap node is just any node that's (more or less)
| permanently reachable on the public Internet. Same as lots of
| tech like this, IPFS needs them for initial peer discovery and
| (in many cases, and I'm guessing in IPFS' case as well) NAT-
| breaking, plus as a route of last resort.
| yusefnapora wrote:
| You can run a private network by configuring all your nodes
| with a pre-shared "swarm key" and running your own bootstrap
| nodes.
|
| There's some documentation here: https://github.com/ipfs/go-
| ipfs/blob/release-v0.9.0/docs/exp...
| qbasic_forever wrote:
| What's the use-case for this kind of usage of IPFS? Is it just a
| generic content addressable store for local use? Like perhaps use
| it to compile and install packages that you symlink into your
| system path a bit like DIY nix?
| skogard wrote:
| I wrote a dedicated section in the documentation just for this,
| it can be pretty powerful: https://docs.nebulus.dev/#/?id=use-
| cases
| argvargc wrote:
| Looks great, thank you for presenting it so thoughtfully too.
|
| Do I read correctly to mean Nebulus could potentially be used
| as a replacement for p2p file xfer, such as per wormhole? Any
| caveats with such a usecase?
|
| (Also, just FYI there's a one-letter typo with the first sub-
| title under API)
| chalst wrote:
| Any application using content addressing can make use of at
| least some IPFS technologies. Sharing the contents of
| /nix/store on a local network or VPN is something I've been
| thinking about recently.
| capableweb wrote:
| This entire project can be replaced in two different ways with
| just IPFS. You don't have to run IPFS to be able to add files or
| retrieve them (if you've added them on the same computer), so
| simply add the files without running the daemon and you're
| already "Offline and Private" and "IPFS without the Network".
|
| If that's not enough, you can also run your IPFS daemon with
| `--offline` and it won't connect to any network, giving you
| another way of using IPFS offline and privately.
|
| If you want to go one step further, when adding files with the
| IPFS CLI, you can add the flag `--only-hash` and IPFS will only
| hash the content without actually writing anything to disk,
| making it even more "offline" and "private".
|
| All in all, this seems to be a project whose author missed to
| read the documentation for IPFS as everything mentioned is
| already supported in go-ipfs, the main IPFS implementation.
|
| Edit: the title "IPFS-less IPFS" is fun too, since nebulus seems
| to include ipfs-core as a dependency, not really IPFS-less then
| I'd say :)
| https://github.com/skogard/nebulus/blob/480d43dc22ccd949c6ae...
| cocktailpeanuts wrote:
| I see the "ipfs less" terminology as similar to the term
| "serverless". Just like how serverless still has servers but
| the point is that you don't have to maintain your own server,
| ipfs less does make sense in that you don't have to run ipfs
| node to use it. But also just like how many people hate the
| term serverless, ipfs less will be hated by many as well. I
| guess at the end of the day what's important is whether
| something provides value, and I think this is pretty cool
| technology in that sense. Just my two cents...
| skogard wrote:
| Hi there, just want to clarify that Nebulus is not a
| replacement for IPFS or competes with IPFS. In fact it
| internally uses all the features you just mentioned. In fact
| it's even powered by JS-IPFS and makes use of the "--only-hash"
| features and other features you mentioned.
|
| I needed a programmatic way to do all these things (as well as
| many other user friendly features that the IPFS raw protocol
| will never provide) and the existing IPFS libraries don't
| provide a way to do so, so I built it and have been using it
| from another project, and it was so useful that I decided to
| extract it out of that project into a standalone one. Just hope
| others find it useful as well. Thank you.
| IncRnd wrote:
| So, it isn't an IPFS-less IPFS but a layer on top of IPFS?
| sigg3 wrote:
| It's full on IPFS with a bit of IPFS on top.
| whyrusleeping wrote:
| This is really great stuff, thank you for making this!
| 1vuio0pswjnm7 wrote:
| Any ideas on the future of reduced-size IPFS utilities. For
| example, the go-ipfs Linux packages are somehere around 24MB.
| Is this a problem anyone is working on.
| nynx wrote:
| Two things:
|
| 1) I wonder why this wasn't written in a language that can
| compile to wasm, so it could be used on both native and js
| without reimplementing.
|
| 2) Can someone contrast hypercore and IPFS?
| pfraze wrote:
| Hypercore distributes signed log-based structures addressed by
| pubkey. It embeds indexes in each log entry to make this
| perform well; for instance Hyperbee is a b-tree over a log.
| (See https://github.com/hypercore-protocol/p2p-indexing-and-
| searc... for how that works.)
|
| IPFS distributes content-addressed blobs which interlink to
| form DAGs. Mutability is supported by pubkey (or similar)
| pointers to recent hashes.
|
| I work with Hypercore. It's often a matter of preference
| between them.
| skogard wrote:
| I mentioned this in above thread, but I am thinking of ways
| to mash up Nebulus with Hypercore (Because the content
| addressable aspect has been unbundled from the networking
| aspect of IPFS, I think there will be some interesting ways
| to deliver IPFS over Hypercore). Will keep the hypercore
| community posted when I make some progress, and thank you for
| all your work with Hypercore. I use Hypercore for several of
| my projects.
| Taek wrote:
| What's the performance like for hypercore? How long does it
| take for updates and new data to propagate the network?
| What type of throughput do you get for uploads and
| downloads?
| skogard wrote:
| > 1) I wonder why this wasn't written in a language that can
| compile to wasm, so it could be used on both native and js
| without reimplementing.
|
| The most straightforward answer is, I extracted this out of an
| existing project I've been working on: https://rarepress.org/ I
| had to build a system that makes use of IPFS without having to
| publish everything to the public network, and this was where
| the idea came from. But the "IPFS-Less IPFS" turned out to be
| much more powerful idea than I originally thought, so decided
| to turn it into its own open source project.
|
| That said, I think it shouldn't be difficult to implement this
| using Go at some point if this approach becomes popular. For
| now I wanted to keep the initiative minimal so I can move fast.
|
| > 2) Can someone contrast hypercore and IPFS?
|
| This is a great question in this context. Hypercore is more
| focused on the networking aspect, but IPFS is not just a
| networking protocol but also a way to represent files using
| immutable identifiers (content addressable file system). What I
| just did with Nebulus is unbundle the "file system" from the
| "network" so we can use IPFS in more flexible manner.
|
| Speaking of Hypercore, because Nebulus unbundles the network
| aspect of IPFS from the content addressable file system aspect,
| it is possible to use ANY network transport to replicate IPFS
| files, which includes Hypercore.
| thesausageking wrote:
| Very confused by the name. I thought this was related to the
| Nebulous Labs (the startup behind the Sia storage network) and
| their work around IPFS: https://blog.sia.tech/skynet-
| summer-2021-update-86ed8db21eae
| dennis-tra wrote:
| A couple of days ago I've released an IPFS DHT crawler and
| named it Nebula [0] if I knew that this was such a crowded
| place...
|
| [0] https://github.com/dennis-tra/nebula-crawler
___________________________________________________________________
(page generated 2021-07-19 23:01 UTC)