Post B5MUMXIn8qwVAA0WMC by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
(DIR) More posts by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
(DIR) Post #B5MQncvruugOcvwp96 by mcc@mastodon.social
3 likes, 0 repeats
Bluesky is down today. "Hah", I think, "since I use a self-hosted PDS for posting and Blacksky for viewing posts, I can go on using the service just fine".Blacksky can't show me my own posts. I can make them and they show up on my PDS, but my profile shows none more recent than last night.I wonder if Blacksky is coincidentally having server problems, or if Blacksky has a still-undisclosed dependency on Bluesky services. (It *does* have disclosed use of Bluesky moderation; maybe that's it.)
(DIR) Post #B5MQndA34C1dKua9FQ by mcc@mastodon.social
2 likes, 0 repeats
There's a bit in Terry Jones' "Starship Titanic" where an alien gets caught in a traffic jam on Earth, and explodes "your transportation system is so poorly designed! when more people use it, it goes *slower*! you should design it so it goes *faster*— to accommodate the extra load!".It's funny because the former seems like the obvious, natural way transportation works, and the latter seems to require magic alien future technology.
(DIR) Post #B5MQndQ06smm8O2t72 by IngaLovinde@embracing.space
0 likes, 0 repeats
@mcc well you probably know that the transit system Utopia designed goes *way* slower than The Six-Hive Transport System, but with 100% less fatal traffic accidents
(DIR) Post #B5MQndX5gWSOUNMYAC by mcc@mastodon.social
2 likes, 2 repeats
P2P is a world where naturally the more people use it, the faster and more resilient the network becomes. Load gets distributed. Working nodes talk to each other and ignore nonworking nodes. That's how the primitive, BitTorrent era systems worked.Bluesky somehow applied superfancy alien future technology to invent P2P traffic jams. When one node goes down, the others go down because they depended on it. Because it's a mesh of interoperating microservices by different providers, not federation.
(DIR) Post #B5MQnds0QlBfXF9FlQ by mcc@mastodon.social
1 likes, 0 repeats
This appears to be the explanation:https://bsky.app/profile/did:plc:w4xbfzo7kqfes5zb7r6qv3rw/post/3mjmztqnuwc2gIn Bluesky, the PDS talks to the relay talks to the appview goes to the client. Blacksky set up all four last year. But they only deployed their PDS and client, at first. They used Bluesky's relay and appview. This wasn't clearly disclosed. Then there was a censorship scare, and they switched to their own appview. But apparently they're still using Bluesky's relay. This wasn't clearly disclosed. Now relay death kills Blacksky.
(DIR) Post #B5MQne91PUnYO16qHo by mcc@mastodon.social
1 likes, 0 repeats
Now, interestingly, this means that Blacksky users can continue talking to Blacksky users. I can read Rudy's posts on Blacksky. Because that bypasses the relay. But¹ to read my *own* posts, *on a self-hosted PDS*, Bluesky is apparently required, because Blacksky relies on Bluesky's "relay" to scrape my PDS before it gets added to the Blacksky appview database.¹ (if I'm taking Rudy's posts correctly, hardly a guarantee)
(DIR) Post #B5MQnePgPY7rDgu9Fw by mcc@mastodon.social
1 likes, 0 repeats
And it's extremely relatable why Rudy took this shortcut of "build out our own stuff, but rely on Bluesky's components until we're forced to drop it": *Because standing up your own Bluesky stack is nightmarish!* It is a borderline miracle that a team his size made this work at all; I'm not sure a third team could replicate to the extent Blacksky has (and even on non-outage days, there are still large technical problems with Blacksky which cannot conveniently be fit in this thread).
(DIR) Post #B5MQnfCFUyyReJcvxI by mcc@mastodon.social
1 likes, 0 repeats
Because this is the other "we used future alien technology to make it worse" thing about Bluesky.In the "natural", Hobbesian form of P2P, the more nodes you add the less work per node you need to do, because of work sharing.But Bluesky's "federation" is like blockchain. When you create a second "instance", that instance must duplicate *literally all the work* of the first instance. It must scrape all the posts itself. It must archive all the posts itself. It must CSAM-scan the posts itself.
(DIR) Post #B5MQngHFTsZF06yRO4 by mcc@mastodon.social
1 likes, 2 repeats
This is why I believe Bluesky was never meant to be federated. To create a Bluesky "instance", like Blacksky is heroically attempting, you have to perfectly duplicate every server Bluesky runs. But Bluesky is a business operating at a loss by burning unlimited-for-now VC cash. That has always implied only a business with unlimited VC cash can create an instance. Blacksky is succeeding. Except on days where they aren't.
(DIR) Post #B5MQnhF9t8UPzv0Hlg by mcc@mastodon.social
2 likes, 1 repeats
TLDR1. My definition of "P2P" or "Federated" is that if server A goes down, servers B and C can still talk to each other.2. Bluesky/"Atmosphere" fails at this because Blacksky (B) requires Bluesky (A) to talk to me (C).3. In order for Blacksky to avert this, they have to do something unreasonable and expensive.4. Blacksky someday *will* do this, but will depend heavily on massively overworking Rudy and a few other people. This may someday fail.5. ActivityPub has problems, but not these
(DIR) Post #B5MQnijKMSDSbm7tQm by mcc@mastodon.social
0 likes, 0 repeats
(And *how* does ActivityPub avert these problems? Well, ActivityPub has the "instance" abstraction. The federate-or-defederate relationships serve as a basic web of trust so some work, like moderation, doesn't have to be fully duplicated. Data is shared between instances only when a follow-relationship requires it, reducing work. Instances can still get too big and maintainers overworked, but you can fix that problem with more, smaller instances. As above, *there ARE no small Bluesky instances*)
(DIR) Post #B5MRLbssJxENDnkgVM by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social 2 is totally false.I was able today to log in on BlackSky with my EuroSky PDS and keep talking with anybody currently not using BlueSky AppView, including people using BlueSky PDS but not BlueSky AppView.
(DIR) Post #B5MRUnP8rE8BYeDzIO by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social https://blacksky.community/profile/did:plc:z5wqufpi3akdylu2sqyzryqr/post/3mjmersqlu22aThis post was posted using BlackSky during BlueSky trouble, using a OAuth authentication from my EuroSky PDS, to somebody who create his account with EuroSky during the BlueSky shutdown.
(DIR) Post #B5MSD14NPclkgHb7c8 by mcc@mastodon.social
0 likes, 0 repeats
@aeris I don't know what to tell you. It working on your machine does not cause it to work on my machine. In my own testing, right now, with the loop of "can blacksky view a post made using blacksky and hosted on a fully independent PDS", it takes about an hour for my own posts to show up in my own feed. Because Bluesky is made of many parts, it's possible the Bluesky relay was not having problems at the start of the Bluesky outage (when you did your test) but is now.
(DIR) Post #B5MSD1FijRqLFSuBIO by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social It would probably be the same here on the Fediverse. I already report to Mastodon they have huge problem with down instance, and if a huge one stop responding, I expect many delay all over other instances, because down instance will fill all background job with dangling requesthttps://github.com/mastodon/mastodon/issues/12445
(DIR) Post #B5MSYcxHkXIkGKaNbE by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social Even on fully (technically) distributed system, a huge sub-system will generate trouble everywhere in case of trouble. See how when Amazon or Cloudflare on trouble they can impact other sub-system, even if Internet is fully distributed. Such generated trouble is from concentrated node more than not decentralized protocol.
(DIR) Post #B5MSsKDT88k9w6Y2qG by mcc@mastodon.social
0 likes, 0 repeats
@aeris You have just put your finger on the difference between decentralized and distributed!!! Amazon is distributed but not decentralized. Bluesky is distributed but not decentralized. Fedi is both.
(DIR) Post #B5MSsKPsO0fUYaLxBI by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social Fedi is not both. And worse, by design, Fediverse is LESS distributed, because a down instance will stuck ALL other instance with dangling job, because it's a push protocol and not a pull one.
(DIR) Post #B5MSyrIn1ydyHt9Vrc by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social Each message from my instance will generate a job to mastodon.social, because at least one ppl there follow me. If mastodon.social vanish, my own instance will start to hang for ALL message I post.
(DIR) Post #B5MT8Rm4nT0y0jQHyq by mcc@mastodon.social
0 likes, 0 repeats
@aeris I do not believe this is true and if it does it indicates some kind of really weird problem with your instance specifically.
(DIR) Post #B5MT8Rwi9vWOXiOmYa by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social No, it's the trouble with the push design of ActivityPub.
(DIR) Post #B5MTBqCaQqzydlRQSO by mcc@mastodon.social
0 likes, 0 repeats
@aeris I don't think you know what you're talking about, and if mastodon.social goes down firefish.imirhill.fr can continue to talk to digipres.club.
(DIR) Post #B5MTBqPLfPCtHLPcLg by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social No. Or with huge delay. Because each of my message will generate a background job to mastodon.social, leading to queue overflow over time and more and more lag even for digipres.club delivery.
(DIR) Post #B5MTFVSjU1nFTZk1dg by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social Each of your message generate a background job on a queue to be submitted to every instance with at least one ppl following you. If a huge one is down, all other instances will start to fill background queue with tons of dangling query, delaying more and more request for still live instance.
(DIR) Post #B5MTOZQM7hp4Mwx3xY by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social Currently i have around 600 "delayed" job because of down instance polluting all delivery. This was reported to Mastodon years ago. Nothing change.
(DIR) Post #B5MTY5CFMb6XW2t7se by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social For tiny instance, it's not really a trouble, because few message and so queue don't fill.For huge instance, pretty all message from all instances will generate a dangling request in queue. When queue filled, delay all message for any other instance even the one alive.
(DIR) Post #B5MTkQp0mrf83nuO0m by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social And it's worst for huge still alive instance. Hundred of message per second. Hundred of job per second for down instance. Hundred of dead job filling queue because timeout, competing resources for alive job. At a point, all workers process only dead job…
(DIR) Post #B5MTyVhlQdaPG8EQPw by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social I don't know exactly what would be the effect of a 10 hour downtime like bluesky for a mastodon.social downtime for example. I expect at least delay growing over time even from no mastodon.social communication.
(DIR) Post #B5MU2aQpnPN94tFyee by mcc@mastodon.social
0 likes, 0 repeats
@aeris If this problem is real I can imagine multiple ways to mitigate it. This is a software engineering problem.
(DIR) Post #B5MU2acX5ujJfAjJtA by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social No, it's a design trouble. ActivityPub use push when ATProto use pull.
(DIR) Post #B5MU6ISuXLhAVvzsp6 by kunev@blewsky.social
0 likes, 0 repeats
@mcc@mastodon.social they're allowed to succeed so they can be paraded around thet "see, it's all super distributed and decentralized".The moment VCs realize they need RoI a bunch of " improvements" likely mostly "for security", probably " for safety", definitely "for the children" will add to the already insane architectural costs, a bunch of operafional burden that makes it impposible for other "instances" to exist.
(DIR) Post #B5MU6IkdTRsDOuI2S0 by khm@hj.9fs.net
0 likes, 0 repeats
that's the Signal playbook. "sure we can federate, but we won't, for reasons"CC: @mcc@mastodon.social
(DIR) Post #B5MUGEL4jXRac14K8G by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social So by design a down instance pollute everything. You can mitigate that with software yes, but background task scheduling is a hard field.Pull troubles is simpler to mitigate, because only require throttling output request on down instance after restart after a downtime to avoid hammering other instance to fill the gap.
(DIR) Post #B5MUMXIn8qwVAA0WMC by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social A down ATP instance is really down. No more pull or effect in the network.A down AP instance is not really down, all other instances try to communicate with it.
(DIR) Post #B5MUmtiUQaOQD2b7Gi by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social And it's a vector attack in theory. You can bootstrap thousands of instance, just subscribing to as many account as possible, and then just shutdown your instance.Any content from subscribed account will generate a background job to your down instance, then hiting timeout each time.You can just flood instance like that to continue to overflow queue with dangling content.
(DIR) Post #B5MV2TMECKZ95jbygS by jeromechoo@masto.ai
0 likes, 0 repeats
@aeris @mcc failed deliveries happen all the time. mastodon.social is just another instance. Since Mastodon and misskey operate on shared inboxes, the failed deliveries won’t scale sender failures by recipient instance size.https://seb.jambor.dev/posts/understanding-activitypub/#:~:text=To%20combat%20this,our%20followers%20internally.
(DIR) Post #B5MV2TZhOFLDlVujgG by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@jeromechoo@masto.ai @mcc@mastodon.social Yes, I know that. Trouble is not one content send to many, but many content sent to one.Each post of one instance is sent only once to mastodon.social, but EACH post.
(DIR) Post #B5MVCJpOU2LaQpwspk by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai So a huge instance sent dozen of post per second (many content generated, but delivered only one) to another huge instance, with one background job per content to deliver.
(DIR) Post #B5MVKAlK4bSQdEn0bI by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai The trouble scale not to the down instance size, but to the alive instance size. The more it is active with many content generated, the fastest the background job queue fill with dangling content.
(DIR) Post #B5MVuvmsiDqWu22vw0 by jeromechoo@masto.ai
0 likes, 0 repeats
@aeris @mcc this post you made probably just failed to deliver to a few instances. One of them could be mastodon.social if it was down. How does that affect delivery to masto.ai? mastodon.social being down is no different to your server than any other server being down. One request each.
(DIR) Post #B5MVuvya0jChUJWHAW by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@jeromechoo@masto.ai @mcc@mastodon.social It affect deliver to masto.ai because EACH of my post generate a dangling request, hiting timeout. After a while, my worker consume more time to dangling request taking 2-3s (hiting timeout) than trying to send content to masto.ai.
(DIR) Post #B5MW61Aj267emvUAym by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai Each post is a dangling request which will consume 3s of CPU time and so 10× consumption of 300ms for alive server, and planned for reschedule. After a while, all workers are just stuck with full of 3s waiting process, with starvation for alive requests.
(DIR) Post #B5MWHVpRONkV3O0kIC by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai After a while, you have 43 minutes latency for EVERY DELIVERY, even alive server. I experience that on my own Mastodon instance…
(DIR) Post #B5MWXqKyuQI7IGYDxI by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai At the end any workers just have 7% of "luck" (3 out of 42) to hit a down request, consuming resource for nothing for 2-3s, having no more time to schedule alive server, with 13.000 pending request because starvation, with many many alive request in those 13.000. Perhaps the 13.000th will be a alive one, but it will be delivered in only 43 minutes in average.
(DIR) Post #B5MWchcQkg67hY1uRU by jeromechoo@masto.ai
0 likes, 0 repeats
@aeris @mcc like I have already said — every post you’ve made just now has failed to deliver to several instances. Your instance is running just fine is it not?If mastodon.social goes down. It would add ONE more failed delivery to the queue of thousands your instance is already managing.
(DIR) Post #B5MWchmM9m2OCKfpui by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@jeromechoo@masto.ai @mcc@mastodon.social No, it not running fine. I ALREADY reported 43 minutes latency to deliver ANY MESSAGE on Mastodon. This "bug" (in fact bad design) is known since ages.https://github.com/mastodon/mastodon/issues/12445
(DIR) Post #B5MWkoYaq8tzXJ6RnM by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai And my new instance (migrating from Mastodon to Misskey exactly for this reason) is ALREADY filled with 600 dangling requests. At this point it doesn't generate any noticable delay, but only because the overall death rate is low. If a huge instance goes down, it would not be the same at all.
(DIR) Post #B5MWoN7cmhdzbfBWwS by jeromechoo@masto.ai
0 likes, 0 repeats
@aeris @mcc and yet, despite the 43 min latency you’re reporting, we’ve been having a perfectly synchronous conversation for the last 15 minutes.
(DIR) Post #B5MWoNIG9A9Q8eA1WC by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@jeromechoo@masto.ai @mcc@mastodon.social Yes, because I stop using Mastodon exactly for this reason. And now using Misskey which know less this trouble for a single user instance.
(DIR) Post #B5MX0nxgowoOK7Syzw by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai But I still have it, lag is just not noticable because rate error is low. Would NOT be the same with a huge instance down like mastodon.social. Rate error will raise, delayed background job too, and at a point workers would not be able to dispatch requests in time to live instance because of starvation, generating noticable lags growing over time gradually with downtime duration.
(DIR) Post #B5MX3yU49jAZpI7bfs by galactus@col.social
0 likes, 0 repeats
@mcc in fairness, I don’t think they ever pretended that federation was a goal, their main claim was the protocol prevented user lock in by providing a credible exit (“if we become evil, some other big player can create their own appview and steal our users”)
(DIR) Post #B5MXclsmzzb6sbHJB2 by jeromechoo@masto.ai
0 likes, 0 repeats
@aeris @mcc 🤷 you figured it out for a second there and looped back around again.
(DIR) Post #B5MXcm5YEXo1WBFV4K by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@jeromechoo@masto.ai @mcc@mastodon.social I figured nothing. I just say it's not that obvious at all than a down instance would not generate effect of the overall network, including communication between two still alive instances.
(DIR) Post #B5MXnWamBmVadsyqCe by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai We ALREADY hit this trouble on Mastodon (43 minutes delay for everything including com between 2 alive instances). We know this is a design trouble because based on push communication.All good when error rate is low, but down instance can generate by design resource starvation for content delivery in any still alive instance.
(DIR) Post #B5MXvn6MgtzwA7hIWW by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai You can mitigate it (smarter scheduling to try to reorganize jobs to priorize supposed alive instance, more workers to keep delay low, etc), but it's a design flaw compare to pull mode like ATProto.
(DIR) Post #B5MY4E1qSc8jzYbZaa by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai And the more you communicate, the more you delayed job grow. For example now we exchange a lot, each of my message is multiple dead jobs to dead instance.
(DIR) Post #B5MYBwUPYT0duZe8Tg by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@mcc@mastodon.social @jeromechoo@masto.ai And each post is 1 more job to each down instance.
(DIR) Post #B5MZcluN6UwErp980O by mcc@mastodon.social
0 likes, 0 repeats
@galactus okay well, i want to have a system where i can escape from the clutches of big player A without needing big player B to need to help me.why should i trust big player B more than big player A, after all? they're big, so probably eventually they'll act the same way.
(DIR) Post #B5MaQzndjJq0QhwiqO by galactus@col.social
0 likes, 0 repeats
@mcc I agree, but I think it is honest to acknowledge that having big indices of the whole network is nice, and we don't yet know how to do that without the kind of $$$ that only big players have.
(DIR) Post #B5MafONz7EvAjjoLKq by mcc@mastodon.social
0 likes, 0 repeats
@galactus Having big indices of the whole network is nice. Having big indices of the whole network is harmful. I kinda liked the Mastodon compromise better. There are things I simply don't say on bluesky because I can't say them without them going in a big index and a lot of LLM training sets.
(DIR) Post #B5MirhOL4trkn3M7lY by chiraag@mastodon.online
0 likes, 0 repeats
@khm @kunev @mcc That's not really fair considering the costs that Session paid (they got rid of PFS, for example) and Session is...about to die in < 90 days.Signal, meanwhile, literally has the gold standard of encryption and is designed so that we largely don't have to trust the server.Federation is nice because of the properties it brings, but it also has trade-offs. Sometimes, those trade-offs don't always make sense or the benefits are overridden by the costs.1/2
(DIR) Post #B5MirhfM3dTddpJiHw by chiraag@mastodon.online
0 likes, 0 repeats
@khm @kunev @mcc Signal, for all of its flaws (and I definitely count centralized among those flaws, along with requiring a phone number), is a good app. It largely works, protects my information, doesn't collect shit, is open source, and takes on the big guys with way less funding. Maybe let's not shit on the one messaging app to take on vastly shittier alternatives and actually *succeed* (to some extent) with normies?2/2
(DIR) Post #B5MirhsTGry8IVSBjU by khm@hj.9fs.net
0 likes, 0 repeats
their bad decisions render any advantages to their software moot. They built a protocol that supports pseudonymous use and federation, and then arbitrarily denied both of those things. Now we're stuck with it because of a combination of networking effects and simps who will make excuses for it.Just like BlueSky!CC: @kunev@blewsky.social @mcc@mastodon.social
(DIR) Post #B5MwjBQzPQmciaQg7s by fleeky@prsm.space
0 likes, 0 repeats
@galactus @mcc i am super confused by this stance, for the internet at large imagine if every server needed to keep a backup copy of the entire internet? sure that would definitely be the most resilient possible internet but who could actually afford that? the whole point of the new wave of federation / decentralization was smol = better and bsky outages just prove that.
(DIR) Post #B5N7QSXagNd9lan7Dc by fleeky@prsm.space
0 likes, 0 repeats
@galactus @mcc i am aware of atproto designs that can handle not the whole firehose but as i watch the brilliant people working on this i am reminded of people squeezing blood from a stone. the original design is as you said big indice or as some others have said big heap .. it just seems top heavy and expensive for normal ass internet weirdos to be able to run?
(DIR) Post #B5N7QSl3sIPERN5sDQ by galactus@col.social
0 likes, 0 repeats
@fleeky @mcc Sure, but that's kind of my original point, no? They're not claiming everyone can or should run a full index. They're saying the architecture makes it possible for others to do so, and that this is decentralized ("credible exit") enough and delivers better (arguably) UX (discoverability, consistent timelines, etc, with decent latency).Again, this isn’t necessarily my view, I'm just trying to represent their philosophy as accurately as I can.
(DIR) Post #B5OND2UK1laFcqqRjE by light@noc.social
0 likes, 0 repeats
@mccWhat's stopping people from making a big index of fediverse posts and training an LLM on them?@galactus
(DIR) Post #B5OND2t8XVQuroSGPI by mcc@mastodon.social
0 likes, 0 repeats
@light @galactus You could build such an index. You can't get everything. You get sent only posts from accounts you "follow". You can follow a lot of accounts, but people can detect you're doing this and ban/block you. You can scrape instance feeds, but these don't always exist and don't contain everything. You can't easily get followers only posts. You can't get mentions-only posts at all. You can't even easily make a map of all instances on the fediverse. Scraping the fediverse is all friction
(DIR) Post #B5OND3JMxyPuBAjDIO by mcc@mastodon.social
0 likes, 0 repeats
@light @galactus By contrast, Bluesky optimized for the "build a big index" case, which means they optimized for the "everything ever said goes in the LLM trainer"/"everything ever said goes in a big FBI database of potential terrorist speech" case. The whole thing is *designed* around funneling everything to a firehose, and tapping the firehose for malicious purpose looks just like normal use. And you cannot, by design, hide *anything* from the firehose. Rudy/Blacksky did it…by breaking ATProto
(DIR) Post #B5OND3WUBCuOpqrgjw by galactus@col.social
0 likes, 0 repeats
@mcc @light yes! it's even in the original paper "as Bluesky grows, there are likely to be multiple professionally run indexers for various purposes. For example, a company that performs sentiment analysis on social media activity about brands could easily create a whole-network index that provides insights to their clients."
(DIR) Post #B5ONQggdsg7K437EO0 by galactus@col.social
0 likes, 0 repeats
@mcc @light I was told yesterday in bsky that the "whole network index" was a myth, when it's actually *the point*.
(DIR) Post #B5ONaGLEgdwFFffWUK by mcc@mastodon.social
0 likes, 0 repeats
@galactus @light From a user perspective, it's highly desirable! It is a major selling point of Bluesky over Mastodon. Unless you've ever been internet harassed or surveilled, in which case maybe it sets off all your alarm bells.
(DIR) Post #B5ObEq2cfqV6xadndY by fleeky@prsm.space
0 likes, 0 repeats
@galactus @mcc at the very minimum it is a naive argument and also the argument then suddenly vanishes if you work on smol atproto ? so basically atproto only works if you are a giant community or big corporation that can "afford to be" decentralized on atproto ? side note i am all for smol atproto and may try to contirbute to that ..
(DIR) Post #B5ObeAGkbWfUw3KFd2 by fleeky@prsm.space
0 likes, 0 repeats
@galactus @mcc i also feel the need to constantly point out the genesis of atproto , pfrazee was working on dat/hypercore/beaker browser .. he had to throw his hands up and say decentralization doesn't work in any practical sense (its hard for sure) he was then broke and got a job at bluesky to essentially port ctzn (his live coded social media experiment) to bsky ,, he basically took ideas that were maybe 3/4's of the way done and thats what we have now.
(DIR) Post #B5QgG6tGP3nRapboMS by khm@hj.9fs.net
0 likes, 0 repeats
Yes, I'm familiar with the Gospel of Signal Apologetics, and the Weird Nerd Oath to repeat it unto the heat death of the universe.You're allowed to use account name pseudonyms after you give them a phone number, rendering the feature moot. I don't trust Signal, and I don't want them to have my phone number.You can't easily ensure people are running an up-to-date (i.e. secure) client no matter what you do, because the client can lie to you. That whole idiotic theory just completely ignores that third-party clients work fine.Besides, it would be trivial to use certificate expiration to ensure federated servers are playing the same tricks that Signal does with clients. I swear, it's like your brains shut down when someone tells you it's for securityCC: @chiraag@mastodon.online @kunev@blewsky.social @mcc@mastodon.social
(DIR) Post #B5Ses1SNF6n09WtxQW by khm@hj.9fs.net
0 likes, 0 repeats
I don't use any of them, because none of them keep the promises they make. In fact, I don't use "apps" at all, or a smartphone. If I need to securely pass information to someone I'll go and talk to them.CC: @chiraag@mastodon.online @kunev@blewsky.social @mcc@mastodon.social
(DIR) Post #B6rHlGc3Zv4MIkVKpk by ellenor2000@mastodon.top
0 likes, 0 repeats
@aeris @mcc @jeromechoo Your server can walk and chew gum at the same time. Depending on how it's designed, maybe it delivers messages in limited parallel, so a dangling job only has a minor effect. #lang_en
(DIR) Post #B6rHlGvuO6wtIJnBmC by AYz3YbdyXoXnWNtcau.aeris@firefish.imirhil.fr
0 likes, 0 repeats
@jeromechoo@masto.ai @ellenor2000@mastodon.top @mcc@mastodon.social Queue theory. Even 10% failure rate leads to 50% workers dangling for nothing, polluting every working jobs.