Post AVdvNZLPxJ88Fm97XE by rolle@mementomori.social
(DIR) More posts by rolle@mementomori.social
(DIR) Post #AVdsTd7sVbbFrC0mae by rolle@mementomori.social
2023-05-14T10:01:46Z
0 likes, 0 repeats
Is there any tutorial on how to separate Sidekiq and PostgreSQL to another servers?I have currently them on the same machine but it would give me more peace of mind in the next big event to have them separated for performance. #MastoAdmin #SysAdmin #Servers
(DIR) Post #AVdsrirCqtyMa7hZ1U by lari@suomi.social
2023-05-14T10:06:05Z
0 likes, 0 repeats
@rolle At least PostgreSQL should be very straightforward. Just copy the database elsewhere and change the address on Mastodon config. I'm planning to do it as well, because currently it seems to run out of memory too often when on the same server with the web service.Probably Sidekiq shouldn't be much different, though I don't have experience on working with it.
(DIR) Post #AVdugZf4tulYmiqB2u by anders@mastodon.cyborch.com
2023-05-14T10:26:30Z
0 likes, 0 repeats
@rolle I’m running postgres on a separate instance and mastodon and sidekiq on the same instance but with two replicas. Works perfectly. I can share my helm charts if you’re interested.
(DIR) Post #AVduuLPKh3bSdC2f4K by rolle@mementomori.social
2023-05-14T10:28:58Z
0 likes, 0 repeats
@anders I'm interested in your setup! What means "two replicas"? The other works as a some kind of load balancer? Any tutorials + configs to share?
(DIR) Post #AVdvF4ZMUc9MUPcJbU by anders@mastodon.cyborch.com
2023-05-14T10:32:47Z
0 likes, 0 repeats
@rolle I have everything running on Kubernetes so two replicas literally means setting replica count to 2 and making the load balancer forward requests to a healthy instance. Gotta go enjoy the sunshine, but I’ll share my setup a little later.
(DIR) Post #AVdvNZLPxJ88Fm97XE by rolle@mementomori.social
2023-05-14T10:34:21Z
0 likes, 0 repeats
@anders Oh. Kubernetes is a stranger to me, but would like to learn more about it. Have a blast in the sun!
(DIR) Post #AVdvYPuPNSPUAtgYN6 by ileoo@mastodontti.fi
2023-05-14T10:36:11Z
0 likes, 0 repeats
@rolle There is a lot written down https://hazelweakly.me/blog/scaling-mastodon/ by @hazelweakly , might be little bit deep dive if you don't yet want to tune all the knobs (soon you will ;).
(DIR) Post #AVdwsHWqLNd9vjhPMm by rolle@mementomori.social
2023-05-14T10:51:10Z
0 likes, 0 repeats
@ileoo I have read that, but it's not actually a tutorial, just thoughts. What I'd need is to point out which file to edit, what to do etc. I'm familiar with nginx and mariadb but that's about it. Need to dig deeper to sidekiq and psql.@hazelweakly
(DIR) Post #AVdxV5GNse4HBQVEkS by ileoo@mastodontti.fi
2023-05-14T10:57:56Z
0 likes, 0 repeats
@rolle Then I would start in something like https://www.eigenmagic.com/2022/12/11/better-mastodon-sidekiq-scaling-with-systemd-environmentfile/ as most likely you want latency to be more under control in queues than try to scale up database unnecessary.
(DIR) Post #AVdxydPbzjzXUYuTTc by rolle@mementomori.social
2023-05-14T11:03:06Z
0 likes, 0 repeats
@ileoo This is exactly what I'm currently reading. :blob_smile_sweat:
(DIR) Post #AVdzIo5a6kKSLriv7Q by hazelweakly@hachyderm.io
2023-05-14T11:18:10Z
0 likes, 0 repeats
@rolle @ileoo I need to write a tutorial at some point that goes over this all in detail :)Because you're right, it's just a collection of thoughts on how to scale things and now I can make it actually coherent
(DIR) Post #AVe2Kt1t0h8S7Eg7oO by rolle@mementomori.social
2023-05-14T11:52:12Z
0 likes, 0 repeats
I have successfully increased the mastodon-sidekiq processes and optimized them and postgresql. But I have all of these on the same machine. Postgresql should be rather easy to move, but how about sidekiq? Should it be even moved? #MastoAdmin
(DIR) Post #AVe2gV7uYq9wLxnrF2 by rolle@mementomori.social
2023-05-14T11:56:20Z
0 likes, 0 repeats
Well, I guess I'll try with this boosted setup for a while now and tune up before the next #Eurovision. :blob_smile_sweat2: This is fine.
(DIR) Post #AVeAOyIw8gP1Ge72Se by anders@mastodon.cyborch.com
2023-05-14T13:22:28Z
0 likes, 0 repeats
@rolle I actually documented my own switch from classic hosting to Kubernetes a while ago. The quoted configuration files are a bit out of date, but it might be a place to start:https://blog.cyborch.com/how-i-switched-to-kubernetes/
(DIR) Post #AVeATtAIfrigmBPwqO by anders@mastodon.cyborch.com
2023-05-14T13:23:14Z
0 likes, 0 repeats
@rolle it comes with a bit of a learning curve, but I think it’s worth it in the long run.
(DIR) Post #AVeAtkTtaErvN9ujRI by rolle@mementomori.social
2023-05-14T13:28:07Z
0 likes, 0 repeats
@anders Thanks. I dislike dockers and kubernetes might just be too extensive for me, but gotta look at it more thoroughly when I have a moment.
(DIR) Post #AVeBBl5O4dxySc2Qoi by anders@mastodon.cyborch.com
2023-05-14T13:31:18Z
0 likes, 0 repeats
@rolle interesting. I’m curious though - how are you hosting services? One machine per service? Everything installed on the same machine? Something else entirely?
(DIR) Post #AVeBPd1CrhcRUghEhM by ikkeT@fosstodon.org
2023-05-14T13:33:48Z
0 likes, 0 repeats
@rolle @anders Haha, this is super nice to follow. Kinda slow motion on how learning scaling takes it naturally to use kube, and knowing already how much frustration it causes along the way for (mainly) css guy learning it 😁 And sooner or later all Dude"s hosting will be on kube. Welcome to my world 😁
(DIR) Post #AVeBZnON9QU6YyNdGy by rolle@mementomori.social
2023-05-14T13:36:04Z
0 likes, 0 repeats
@anders Generally caching (redis) and files (front) on one server (per purpose) and database on another.I'm running WordPress websites for living so we have multiple instances on one server and when the servers get "full" enough, we create another server pair. S3/CDN for media when needed. WordPress is not very resource intensive so this is fine. Deploying is rather straightforward with Capistrano.I'm always amazed some use 1 docker per WordPress instance. In my world it's like running a docker for 1 html file - it's overkill.Mastodon is of course completely different infrastructure, but I still like the way it's hosted originally, the classic way.
(DIR) Post #AVeBkKkbylH6v2o760 by rolle@mementomori.social
2023-05-14T13:37:57Z
0 likes, 0 repeats
@ikkeT @anders Well I have already dumped Ansible, Puppet, Docker setups in the past and concentrated mostly on the things that I feel safe with, which is bash and capistrano... I like it simple. So there is no big worry :blob_smile_sweat: But I like to learn, that's for sure.
(DIR) Post #AVeCRZ3Og0i7jOCxSC by ikkeT@fosstodon.org
2023-05-14T13:45:44Z
0 likes, 0 repeats
@rolle @anders Docker is good to dump anyhow in benefit of #podman. Which is very good stepping stone to #kube. Or step down stone from kube if one wants to go back to smaller. But I wish you don't end up setting up and maintain kube just for #mastodon... hopefully #systemd+podman would do for simplicity. Unless someone is running kube for you. Anyhow, learning all this geeky stuff is super nice 😁
(DIR) Post #AVeDld4NYUhFvpnaNM by mcg@social.lol
2023-05-14T14:00:35Z
0 likes, 0 repeats
@rolle Sidekiq will need to connect to Redis and Postgres. If you move it, you’ll need to correctly configure the hosts and make sure connectivity is allowed.
(DIR) Post #AVeE8WNaayCxqKhSpE by rolle@mementomori.social
2023-05-14T14:04:20Z
0 likes, 0 repeats
@mcg Not moving it for now, but I wish there was some tutorials for this.
(DIR) Post #AVeI3mZRMx9GQaEtCi by mcg@social.lol
2023-05-14T14:48:16Z
0 likes, 0 repeats
@rolle Seems like most docs are single machine or large scale.