Post APlNUwq6mxrUTRISno by nicopmat@genomic.social
(DIR) More posts by nicopmat@genomic.social
(DIR) Post #APlLCVp2IemJYnsPa4 by skepteis@skepteis.social
2022-11-19T13:14:15Z
0 likes, 0 repeats
Question to anyone who's up-to-date with web development (I haven't touched it seriously in 5-10 years)Given the limited financial resources of Admins of #Mastodon instances, isn't the Ruby implementation a bit of a problem? As I said, I'm out of the loop, but would some alternatives (golang?) have the potential to substantially lower costs?
(DIR) Post #APlLohgkd5AvMqDEQa by phreeza@social.tchncs.de
2022-11-19T13:18:34Z
0 likes, 0 repeats
@skepteis completely out of the loop, too, but cursory googling suggest there are jit compilers available, so maybe it's not quite as bad as it used to be?
(DIR) Post #APlLxP8Mz4X2QaHOoC by skepteis@skepteis.social
2022-11-19T13:22:44Z
0 likes, 0 repeats
@phreeza I run my own instance and for what it does, I feel it eats up quite a lot of resources...
(DIR) Post #APlLzUsEcArwoBjLhw by mwfc@chaos.social
2022-11-19T13:23:06Z
0 likes, 0 repeats
@skepteis Yeah pleroma shows nicely that elixir is the weapon of choice ;)In all seriousnes tho, I do not know. I doubt it is an issue.We probably have entered a new phase with 10k+ Follower accounts being normal. And them entering in this way (due to migration tools) instead of growing and being more predictable load. Böhmerman is at 120k.So now the distribution of toots needs to be optimized. At the same time a lot of instances pop up.
(DIR) Post #APlMBs2MJtho7hMlTU by skepteis@skepteis.social
2022-11-19T13:25:20Z
0 likes, 0 repeats
@boynux Node might be an option as that seems to be where all the devs are recently
(DIR) Post #APlN0GMw82OM4B1JGi by skepteis@skepteis.social
2022-11-19T13:34:26Z
0 likes, 0 repeats
@boynux Ok, makes sense.
(DIR) Post #APlNER7kwDXFTdg68u by mwfc@chaos.social
2022-11-19T13:26:58Z
0 likes, 0 repeats
@skepteis So maybe the remote fediverse instance is slow and blocks your workers. so the question is how do you scale the distribution and get this problem off your shelf.Is the language performance a big cost factor? Are there more skilled golang devs who are cheaper and helping to make a switch viable? Or maybe we just need to do the engineering to fix the issues that might be unrelated or neglectable penalities compared to access to devs.How do friendica (php) instaces do currently?
(DIR) Post #APlNERa7EmDitawkLY by skepteis@skepteis.social
2022-11-19T13:37:00Z
0 likes, 0 repeats
@mwfc Hm, yeah, true. Would be interesting to benchmark the existing solutions...
(DIR) Post #APlNUwq6mxrUTRISno by nicopmat@genomic.social
2022-11-19T13:39:59Z
0 likes, 0 repeats
@skepteis @phreeza A bit unrelated question: What insights do you actually have as an admin about the users (yourself)? Is it basically the same as is available public (followers etc) or does the platform provide more stats?
(DIR) Post #APlNhojIFiZH5OFHyC by tfluehmann@swiss-talk.net
2022-11-19T13:42:18Z
0 likes, 0 repeats
@skepteis @phreeza what does that mean? can you provide some stats?
(DIR) Post #APlNn5Q7ckWwIcn5zE by nicopmat@genomic.social
2022-11-19T13:41:26Z
0 likes, 0 repeats
@skepteis @phreeza. Just asking since it could in theory be problematic if users do not know the admin & think DM's etc are 'private'.
(DIR) Post #APlNn5wNgoKnufsrGi by skepteis@skepteis.social
2022-11-19T13:43:15Z
0 likes, 0 repeats
@nicopmat @phreeza Nothing is encrypted, AFAIK, so anything goes. What you get in the Mastodon UI are some stats and the whole blocking/reporting interface.
(DIR) Post #APlOT7hbWuxDXV0r2W by skepteis@skepteis.social
2022-11-19T13:50:53Z
0 likes, 0 repeats
@tfluehmann @phreeza ~1.2 gigs and ~20% of a digital ocean intel (2nd get Xeon scalable) for my single user instance. Looking at the resources on https://masto.host/pricing/, scaling seems to be a bit problematic.
(DIR) Post #APlOyBLC3HQHnjL9VI by mwfc@chaos.social
2022-11-19T13:56:27Z
0 likes, 0 repeats
@skepteis @tfluehmann @phreeza Have you compared your instance with 1200 followers (how many servers do you federate with) and your incomeing queue to other smallish instances?I would not be suprised that initial setup is quite expensive for Mastodon, but additional users are less of an issue (sure, more overhead than optimized languages) as long as you are in within a certain bounds.
(DIR) Post #APlPGVtWN9sClA8bj6 by nicopmat@genomic.social
2022-11-19T13:59:46Z
0 likes, 0 repeats
@skepteis @phreeza Thanks.. That's unfortunate. Sure, on the birdsite they could also have spied in the past (apparently they work on End-to-End encryption for DMs now), but with the huge number of mastodon instances popping up, it is only a question of time until some will abuse their power. 😏
(DIR) Post #APlPUqEpX2YJuOwAE4 by skepteis@skepteis.social
2022-11-19T14:02:24Z
0 likes, 0 repeats
@nicopmat Yeah, also security will be an issue. The problem will be a combination of this and a lot of people not understanding what they should and should not share here (it is after all public).
(DIR) Post #APlPaU3wtPoQs0XPrE by Mina@swiss-talk.net
2022-11-19T14:03:24Z
0 likes, 0 repeats
@skepteis Oh god!I didn't know, Ruby was even still a thing.
(DIR) Post #APlPcaa1dGiLf8Xru4 by paolog@mastodon.social
2022-11-19T14:03:47Z
0 likes, 0 repeats
@skepteis Discourse is another messaging-heavy platform built on Ruby, and this 9-years old post about their rationale for choosing Ruby is stlll interesting: https://blog.codinghorror.com/why-ruby/. Being built on ActivityPub, Mastodon can interoperate with other implementations. https://github.com/superseriousbusiness/gotosocial looks like an interesting initiative.
(DIR) Post #APlQ5fJ18pQ8iVSFDE by Toasterson@chaos.social
2022-11-19T14:09:02Z
0 likes, 0 repeats
@skepteis Most of it is with sidekiq so most of the message processing could be offloaded to a rust process or cron managment system that can test different implementations. Based on your OS threads and processes also have different costs associated depending if you add CGROUPS(Docker) and RBAC et al to the mix. All in all the webserver itself only serves requests to you so its no problem. the queues need optimizations
(DIR) Post #APlQ6ZQ2VuJOkVAdvs by skepteis@skepteis.social
2022-11-19T14:09:14Z
0 likes, 0 repeats
@mwfc Yeah, that's probably the case, but again, based on the suggested scaling on the hoster's website, it is definitely not too good.
(DIR) Post #APlQYSaRVcyVfSKrh2 by mwfc@chaos.social
2022-11-19T14:14:14Z
0 likes, 0 repeats
@skepteis iirc most had to adapt their pricing. So thei calculations calculation is currently off.I recall https://mastodon.ar.al/@aral/109315412776960105 And some discussions about it.So the question is, as soon as you have heavy users, how much more are a couple of hundred heavy users? Because you already have to connect to basicly every other fediverse server for every post. If you batch your outgoing messages it should already reduce the cost.Lets give them a few months to find optima
(DIR) Post #APlQZttJTfx4dkqgxk by skepteis@skepteis.social
2022-11-19T14:14:32Z
0 likes, 0 repeats
@Toasterson Yeah, but sidekiq is also a Ruby implementation. Would be interesting to experiment with alternatives at some point
(DIR) Post #APlQnZgtNXso2nYkKm by Toasterson@chaos.social
2022-11-19T14:16:58Z
0 likes, 0 repeats
@skepteis Well migrating parts of the processing or the whole processing into rust with https://github.com/danielpclark/rutie#using-rust-in-ruby or https://github.com/matsadler/magnus will probably already help in the hot channels.
(DIR) Post #APlR5aK9lMOlDkVrTU by skepteis@skepteis.social
2022-11-19T14:20:13Z
0 likes, 0 repeats
@mwfc Yeah... Thanks for the link to the post. Will be really interesting to see how this develops. Would be cool to use something like BOINC (volunteer computing), but I assume it's technically not really feasible.
(DIR) Post #APlRQQhuiH5NTZZ4sa by mwfc@chaos.social
2022-11-19T14:23:59Z
0 likes, 0 repeats
@skepteis The issue is distribution of messages iirc. Not compute.
(DIR) Post #APlVVn1tXwUHyJipqC by DNPFred@mstdn.social
2022-11-19T15:02:59Z
0 likes, 0 repeats
@skepteis this is a very interesting conversation
(DIR) Post #APlYZYXvdXfW4cC2fA by skepteis@skepteis.social
2022-11-19T15:44:05Z
0 likes, 0 repeats
@mwfc Might be offloaded as well?
(DIR) Post #APlZ4XGIShzP4DiaNk by mwfc@chaos.social
2022-11-19T15:49:40Z
0 likes, 0 repeats
@skepteis Hardly. Different rules on different servers makes it a policy mess.I would need a trusted entity that relays the messages. If I take kiwifarms as an example again, I would outright not cooperate with them. If I have to use my capacity to work with blocked servers due to various stuff I am not comfortable with it.And ofc if MY messages get routed through kiwifarms I consider them compromised. I would probably add the sending server that allowed this routing to a killfile.