Post APRELvXfL23TZdctdI by feld@bikeshed.party
(DIR) More posts by feld@bikeshed.party
(DIR) Post #APQo2V0j9vKsyiIMQi by feld@bikeshed.party
2022-11-09T15:28:09.712386Z
34 likes, 24 repeats
the fediverse needs IPFS for media. There's literally no point in tens of thousands of instances having full copies of every single thing that gets shared when we could all just have a warm cache and multiple copies floating around for backup in case the originating instance goes away
(DIR) Post #APQo6PB2n3Ne2mz16m by hj@shigusegubu.club
2022-11-09T15:29:10.666370Z
3 likes, 0 repeats
@feld IPFS for fediverse database wen
(DIR) Post #APQoDv2IjnHAscTcB6 by newt@stereophonic.space
2022-11-09T15:30:29.550055Z
4 likes, 0 repeats
@hj @feld IPFS for fediblock lists when
(DIR) Post #APQoIdpg7yH4yJpAgK by tk@bbs.kawa-kun.com
2022-11-09T15:31:02.630539Z
2 likes, 0 repeats
@feld Warm and floating around? Are you saying you feel the compulsion to take a bath?
(DIR) Post #APQotIEjNqIShx1GAC by feld@bikeshed.party
2022-11-09T15:37:39.168758Z
0 likes, 0 repeats
no, IPFS is a terrible idea for FediblocklistsInstead we should be using DNSRBL for that which we know scales incredibly well for email anti-spam.https://www.corpit.ru/mjt/rbldnsd.htmlhttps://www.rbl-dns.comI made a test MRF for it here:https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3278For every post that comes in it sends a DNS query to the DNSRBL serverif it responds with 127.0.0.1 or no response (server issue), the post is allowed throughYou setup classifications for other things127.0.0.2 == reported abuse127.0.0.3 == adult contentetc etcThen you share access to the DNSRBL server with whoever you want (or run your own) and it's the DNSRBL config that gets synchronized
(DIR) Post #APQoylJkgkk0LgyYYS by newt@stereophonic.space
2022-11-09T15:38:58.167105Z
1 likes, 0 repeats
@feld @hj honestly, IPFS is just a terrible idea in general
(DIR) Post #APQpE68zXSCXHiFtJY by r000t@ligma.pro
2022-11-09T15:41:38Z
1 likes, 0 repeats
@feld At one point I wanted to start something like https://jortage.com/ but for cool instances
(DIR) Post #APQrSIf9zStA4dCYzY by frssoft@udongein.xyz
2022-11-09T16:02:56.522620Z
1 likes, 0 repeats
@feld What if include attachment as IPFS link...
(DIR) Post #APQrSJExqLWprfx9nc by feld@bikeshed.party
2022-11-09T16:06:17.717781Z
3 likes, 1 repeats
IPFS links should be included in the activity as an alternate source of the content. It's very easy to do and everyone can do it without even running IPFS -- they just need the function to generate the hash.Then if any server on the Fediverse automatically adds the media to their IPFS node during post ingestion it would make the IPFS link work for everyone else.
(DIR) Post #APQvbyC9LOLJJ2UynY by neet@minidisc.tokyo
2022-11-09T16:50:26.394Z
0 likes, 0 repeats
@feld@bikeshed.partyso the server sends a hash and the client has a webIPFS node to retrieve? :cs_grad:
(DIR) Post #APQvbye9fGkChtbLRw by feld@bikeshed.party
2022-11-09T16:52:58.059651Z
0 likes, 0 repeats
If we completely removed the current way media is federated -- yes, they could use a web IPFS gateway or their own local IPFS node for retrieval and they'd be able to retrieve the data from every IPFS node that already has a copy (kinda like a torrent, but i don't think IPFS fetches parts the same way)
(DIR) Post #APQxx59OsV0GrtPK6q by neet@minidisc.tokyo
2022-11-09T17:07:33.633Z
1 likes, 0 repeats
@feld@bikeshed.party i'm sold.it's also a good way for users to contribute with their bandwidth
(DIR) Post #APQxx5cT8QFuK30XQ0 by feld@bikeshed.party
2022-11-09T17:19:08.890826Z
0 likes, 0 repeats
it would also prevent small instances from getting absolutely crushed if they post something that is shared widely. It could really reduce the bandwidth requirements of running an instance by a ton.
(DIR) Post #APQys8cmdP8A9HMET2 by GNUxeava@mk.absturztau.be
2022-11-09T15:30:56.013Z
0 likes, 0 repeats
@hj@shigusegubu.club @feld@bikeshed.party i've been advocating for ipfs over blockchain unironically
(DIR) Post #APR3DyD2RFD629yu2q by ignaloidas@not.acu.lt
2022-11-09T18:18:37.184Z
1 likes, 0 repeats
@feld@bikeshed.party I'd say sure, but every time I tried to use IPFS it has been slow as balls
(DIR) Post #APR9MdCMWcyFCWB78C by feld@bikeshed.party
2022-11-09T19:27:09.438311Z
0 likes, 0 repeats
@ignaloidas we use IPFS @ work a lot (lol, NFTs) and sometimes download at multiple gigabytes per second
(DIR) Post #APRAhXemQLng370O2K by neglesaks@mstdn.io
2022-11-09T19:42:24Z
0 likes, 0 repeats
@feld That, or simply have a fast cache of the 10.000 most used pictures/ vids with the relevant associated hashes.
(DIR) Post #APRB2hZ8mGrArYSPOi by sullybiker@sully.site
2022-11-09T19:46:12Z
0 likes, 0 repeats
@feld Masto (at least my version) does not handle this well either. It's very inefficient in terms of space.
(DIR) Post #APRBHoAOWAoLg95NPk by alex@gleasonator.com
2022-11-09T19:48:53.989969Z
2 likes, 0 repeats
@feld This, 100%
(DIR) Post #APRBPSQlDRXqaAO8jQ by selea@social.linux.pizza
2022-11-09T19:50:14Z
0 likes, 0 repeats
@feld "Posts are probably satire.":mortyheh:
(DIR) Post #APRBTUjWb36Gsl5egi by feld@bikeshed.party
2022-11-09T19:50:45.119854Z
0 likes, 0 repeats
@selea some people just don't appreciate character work
(DIR) Post #APRBWjH6zGnqqqLIo4 by feld@bikeshed.party
2022-11-09T19:50:48.070641Z
0 likes, 0 repeats
@sullybiker it's literally why I joined the Pleroma projectToo messy of a codebase with too many dependenciesToo bloated resource requirementsI tried to run it, but couldn't stomach it
(DIR) Post #APRBc1689KuXZljAEC by sullybiker@sully.site
2022-11-09T19:52:35Z
0 likes, 0 repeats
@feld I was a bit alarmed to discover there is no clean up of avatars. They just stay there forever. The solution was 'buy more storage'. I am of course made of money.
(DIR) Post #APRBk6QKETkaPU1rlI by teknomunk@pl.polaris-1.work
2022-11-09T19:53:24.930876Z
2 likes, 0 repeats
@feld Or at least some content addressed media system.Has IPFS fixed the privacy issue that lets people check if a particular IP address is hoating fragments?Honestly, content addressed media could be done with the existing HTTP servers that make up the fediverse if a standard existed for it.
(DIR) Post #APRBkAFpzPjoI79XyC by feld@bikeshed.party
2022-11-09T19:53:51.952127Z
0 likes, 0 repeats
@sullybiker well don't worry, Pleroma still doesn't clean up user uploads properly when the post is deleted. I've been meaning to work on that but something else always comes up.
(DIR) Post #APRDAz4ZlptPNvEoTo by pee_zombie@schelling.pt
2022-11-09T20:10:08Z
0 likes, 0 repeats
@feld so many ways to integrate other decentralized utility projects into Mastodon, too bad a. the community is pathologicaly allergic to anything that even smells of "blockchain" and b. it's written in ruby
(DIR) Post #APRDBYPMOYJYyWEPEe by feld@bikeshed.party
2022-11-09T20:09:50.797817Z
1 likes, 0 repeats
@teknomunk > privacy issueHow would you possibly get around that?There's a DHT swarmYou tell the DHT swarm what you're hostingThey retrieve those parts you're hosting by your IP addressI don't get what you could possibly do to tell the world you have something but not let anyone know you have it
(DIR) Post #APRDIymIkZzOmsVDrE by feld@bikeshed.party
2022-11-09T20:11:16.950918Z
0 likes, 0 repeats
@pee_zombie there's a branch out there where you login to your Pleroma instance with your ETH wallet 😶🌫️
(DIR) Post #APRDwKXnl4CDaUU60u by pee_zombie@schelling.pt
2022-11-09T20:18:41Z
0 likes, 0 repeats
@feld see now that's actually fuckin A, and something I'd love to have on schelling as well! I just don't know ruby nor want to learn. mb worthwhile to look into pleroma? altho tbh I just wanna rewrite the whole thing in Java for legibility
(DIR) Post #APRELvXfL23TZdctdI by feld@bikeshed.party
2022-11-09T20:22:56.868662Z
0 likes, 0 repeats
@pee_zombie it would give us actual nomadic identities on the fediverse:the ability to post directly on any instance we're viewing by signing a message with your wallet to prove your identity.The server could then send a signed payload to your server for anything needed (like ensuring the post is correctly stored on your own instance too), plus automatically pull your avatar/bio/etc from your ENS profile or your originating server etc etcplus it's inherently 2FA: something you have (fake internet money) and something you are (a crypto douche)I would use the shit out of this
(DIR) Post #APRFUYExRW11rhYHvk by AJ02m2RByuKqNy0eES.tassoman@orwell.fun
2022-11-09T19:58:52.585489Z
0 likes, 0 repeats
maybe you can save post ID inside metadata (exif?) or filename then do some recurring action finding and purging orphans? You can call this function HerodJob 🤔 😆
(DIR) Post #APRFUZ68FoYAWcQkoS by feld@bikeshed.party
2022-11-09T20:35:41.912670Z
0 likes, 0 repeats
@tassoman @sullybiker nah we literally just need to walk the database and create a table linking them to the post(s) they're used with and only delete the media when the post is deleted and nothing in the database claims ownership of it
(DIR) Post #APRGsNcKgrMHp4PabY by ciaby@social.brokenbydesign.org
2022-11-09T20:49:05.953910Z
0 likes, 0 repeats
@feld @enkiv2 true. Is IPFS immutable?Also, how do you guarantee at least X copies at all time? Who pays for that?
(DIR) Post #APRGsOWLKcA4cmcJuK by feld@bikeshed.party
2022-11-09T20:51:08.068367Z
0 likes, 0 repeats
@ciaby @enkiv2 originating server would pin it, everyone else would have it in their cache if they've received it recentlyyou could easily create a little community of fediverse servers that agree to automatically pin each others' media as a backup strategylots of options are possible this type of architecture including paid pinning services that exist out thereIPFS is not immutable except in the sense that an object at an address cannot change its contents or it will not match the hash
(DIR) Post #APRHHVzaglhP2NHAA4 by enkiv2@eldritch.cafe
2022-11-09T20:55:03Z
1 likes, 0 repeats
@ciaby @feld In practice, there is no guarantee, & people who want to enforce one will make agreements with each other to pin each other's content.However, if all or many clients serve their cache (pinning for however long), the number of available copies scales with the popularity, and as something grows in popularity, the load on the origin shrinks rather than growing. It's antifragile against getting slashdotted.
(DIR) Post #APRHPPfg4gv5kVjB9U by ciaby@social.brokenbydesign.org
2022-11-09T20:56:28.531915Z
0 likes, 0 repeats
@enkiv2 @feld sounds good enough. What’s in the way of implementing it?
(DIR) Post #APRHPQIJl1pPgLo2Nc by feld@bikeshed.party
2022-11-09T20:57:21.715425Z
0 likes, 0 repeats
@ciaby @enkiv2 The Mastodon Project wouldn't support it for 5 years after the rest of the fediverse adopted it
(DIR) Post #APRHfPLR3Y9nMzt1KC by rimugu@liberdon.com
2022-11-09T21:00:27Z
0 likes, 0 repeats
@feld Like masto.host on steroids?
(DIR) Post #APRHzuWMqx6jqxmCK8 by Moon@shitposter.club
2022-11-09T21:03:59.887425Z
1 likes, 0 repeats
@feld @enkiv2 @ciaby in terms of implementation i would do it if i could set a whitelist of whose servers i peer for
(DIR) Post #APRKIT65Er3uD4SW12 by jnv@mastodon.social
2022-11-09T21:06:17Z
0 likes, 0 repeats
@newt @feld @hj Please elaborate. The idea of a distributed content addressable file system or IPFS' handling of it?
(DIR) Post #APRKITd3GHQvrJsqP2 by newt@stereophonic.space
2022-11-09T21:29:52.504903Z
1 likes, 0 repeats
@jnv @feld @hj >Please elaborate.No.
(DIR) Post #APRLmY7ZDxR8Qu3oR6 by raucao@kosmos.social
2022-11-09T21:43:25Z
0 likes, 0 repeats
@feld I thought about this back in 2017. Still think this is the way to go. And pinning could even be done by separate, specialized non-profit orgs, in addition to the source servers...https://kosmos.social/@raucao/49864
(DIR) Post #APRLmYZDZ9YRoeztXE by feld@bikeshed.party
2022-11-09T21:46:08.145937Z
0 likes, 0 repeats
@raucao biggest issue right now is that ipfs/kubo sucks at scaling. pretty sure every large organization that has launched it is using their own custom storage backend.I appear to have fixed my scaling issue at work doing the same. (AFAIK I'm the only person on the planet using IPFS/kubo with Badger2)
(DIR) Post #APRM058HnbWya5qjXU by raucao@kosmos.social
2022-11-09T21:46:46Z
0 likes, 0 repeats
@feld I guess one of the main issues is that go-ipfs (nay, kubo) doesn't scale that well. So the largest instances would run into issues trying to use it as is.There are a couple of companies out there solving this, and also adding S3-compatible APIs to their hosted offerings. But not sure how much of the real solutions are FOSS.
(DIR) Post #APRM05TCXqGFcxdR8i by feld@bikeshed.party
2022-11-09T21:48:41.623536Z
0 likes, 0 repeats
@raucao yep, I made this PR https://github.com/ipfs/kubo/pull/9230
(DIR) Post #APROJHvsdKLzgpKhPs by silverpill@mitra.social
2022-11-09T22:12:36.187280Z
2 likes, 0 repeats
@feld @pee_zombie I implemented wallet-signed activities in my project: https://mitra.social/objects/01843a3b-d887-4131-967e-1bfc1e71de72.I don't think it makes sense to sign every activity, because this would result in a bad user experience. Having the ability to sign Move() to indicate a primary account should be enough.
(DIR) Post #APRObHB0BPxmYbz1Hs by feld@bikeshed.party
2022-11-09T22:17:44.948783Z
0 likes, 0 repeats
@silverpill @pee_zombie agreed, signing every activity is overkill
(DIR) Post #APRPHOYh7vKFcC9JDM by MidwitNavigator@poa.st
2022-11-09T22:25:46.442809Z
0 likes, 0 repeats
@feld Perhaps a system with an antifungal blockchain or some such.
(DIR) Post #APRPldM6g0LgC7gEPQ by feld@bikeshed.party
2022-11-09T22:30:50.522304Z
0 likes, 0 repeats
@MidwitNavigator Tinactin Chain will sooth your fungal feet. Buy the dip!!!
(DIR) Post #APRSjVeMecdLZztmaG by Czarnobajewka@101010.pl
2022-11-09T23:04:26Z
0 likes, 0 repeats
@feld files could be pinned by relays, additionally to source
(DIR) Post #APRX6OwU9vQLFkXu3k by y3mz@mastodon.tech
2022-11-09T23:53:25.594910Z
0 likes, 0 repeats
Has anyone worked towards interop on this? I see a PR for an IPFS-Uploader on Akkoma. 🤔
(DIR) Post #APTgS0vtVxAuaH2TXE by moomin@poa.st
2022-11-11T00:47:36.055842Z
0 likes, 0 repeats
@feld good media policy starts with proper tagging. if media isn't tagged properly, ie. searchable, then there is no point. hydrusnetwork.github.io/hydrus/
(DIR) Post #APTgX6Xi5HajQm9ci8 by balrog_booger@kabuki.club
2022-11-11T00:48:30.084248Z
0 likes, 0 repeats
The built in proxying methods in (at least) Pleroma servers does a lot to mitigate the privacy concerns, and those concerns are already an issue as far as connecting to other, untrusted instances is concerned
(DIR) Post #APTgXFXgcjv7LJDEQK by tk@bbs.kawa-kun.com
2022-11-11T00:48:20.688107Z
1 likes, 0 repeats
@feld Pleroma already does the warm cache thing without IPFS.
(DIR) Post #APTgeuupCJfJgcguw4 by Jain@blob.cat
2022-11-11T00:49:54.961384Z
1 likes, 0 repeats
@tk @feld beside of that, i doupt that ipfs would sync faster than the media proxy loaded the file once
(DIR) Post #APTgiWF4sHMuK0sjxo by helios@monstergirl.space
2022-11-11T00:50:33.802001Z
0 likes, 0 repeats
@feld the idea has been floated around. just needs some omega autistic to actually implement, and make it easily accessible to less tech-savvy people, and someone to convince the extremely disagreeable admins of various instances to agree to use it....
(DIR) Post #APTj2Xf1VgZuQAh2Yq by moomin@poa.st
2022-11-11T01:16:36.591200Z
0 likes, 0 repeats
@feld the most important part of any possible IPFS fediverse media implementation is getting a reliable clearnet ipfs gateway together with a decent amount of multilocation nodes. as well as having a number of people who will have the responsibility of going through stuff and approving files so illegal shit does not get uploaded. (you are responsible for everything on there) to do this you'll have to make a decent admin backend so files can be gone through easily and fast. a good way would be to have certain accounts build up trust, and all new or low activity accounts have to get their uploads approved first before they get access to the ipfs higher speed, and will have to be happy with locally hosted video while their uploads are approved for sync on IPFS. the largest bandwidth usage is webms and videos. if you can do this just for videos it would be worth it. then you just have the ipfs video hash streamed from your ipfs gateway. if you don't have your own ipfs gateway, files go missing (public ipfs gateways delete files to clear storage) and you get shit speed (they automatically make your shit slow, because they want speed for their own shit). what IPFS actually is, is a way to sync contextless hashes, instead of direct files. You're still selfhosted essentially (no one is going to become a peer unless they are interested in your files). Plus it also introduces a bunch of issues regarding dissident stuff. Anyone can see who is sharing what especially on a public gateway. Hence why you need a private self owned gateway. Only your own people on your instance should know about it. Some things don't need to be decentralized. What IPFS actually brings to the table is being able to quickly scale and seed dissident stuff as hashes on a VPS without ever having to know what is inside the hashes, nor having access unless you have the private key to access. You could load up a bunch of VPSs just syncing to your gateway and you instantly have the speed. correct me if im talking out my ass.
(DIR) Post #APTobuxVNbakaL1ejo by feld@bikeshed.party
2022-11-11T02:18:33.843804Z
1 likes, 0 repeats
@moomin I wrote an Elixir IPFS gateway frontend that sits in front of the raw IPFS and can ban media correctly by generating all possible supported hashes that IPFS can handle and putting them in a blacklist database it checks on new connections