Post APOzf2qAXWkG5D1Fq4 by w8emv@hachyderm.io
(DIR) More posts by w8emv@hachyderm.io
(DIR) Post #APOzf2qAXWkG5D1Fq4 by w8emv@hachyderm.io
2022-11-08T18:16:01Z
0 likes, 1 repeats
I'm working on a blog post about the technical bits of Mastodon servers scaling up - mostly just an overview of what people have done at various scales.So far examples from:@dzombak @rey @M0YNG@tek @Taniwha @ariadne At small scale, the answer seems to be "spend more money on VPS"; at a larger scale, "tune @sidekiq ". I have a blog post from @tek linked so far. If you have done a writeup on what you have done I'm happy to share it.trying to catch a moment in time.
(DIR) Post #APP7gNRsChvbv9nkRM by M0YNG@mastodon.radio
2022-11-08T19:00:18Z
0 likes, 0 repeats
@w8emv @dzombak @rey @tek @Taniwha @ariadne @sidekiq I think the nature of the situation has changed. Before sticking more CPU at it was enough, but my sidekiq has gone from 50k things a day to 1m today.I'd very much like to know how to optimize it!Disk space is also suddenly a thing to worry about.
(DIR) Post #APP7gO4Vt2pvqzsbfU by tek@freeradical.zone
2022-11-08T19:59:04Z
0 likes, 0 repeats
@Taniwha @dzombak @rey @ariadne @sidekiq @M0YNG @w8emv If you haven’t seen it, here’s how I got through the flood of queued tasks: https://blog.freeradical.zone/post/surviving-thriving-through-2022-11-05-meltdown/
(DIR) Post #APP8ydHIBo7IBJ7Yn2 by M0YNG@mastodon.radio
2022-11-08T20:13:32Z
0 likes, 0 repeats
@tek thanks, interesting read. Currently I can't work out why I have a pull queue of 10k but not all of my (50) sidekiq workers are busy.
(DIR) Post #APP950ysyUgnQtYS4u by tek@freeradical.zone
2022-11-08T20:14:44Z
0 likes, 0 repeats
@M0YNG What’s the CPU on the “bundle” process look like?
(DIR) Post #APP9N6aYrduA2Mnrw8 by M0YNG@mastodon.radio
2022-11-08T20:16:44Z
0 likes, 0 repeats
@tek basically 100% with overall CPU usage around 50%. I need a second process not just more workers don't I?
(DIR) Post #APP9i1mRw74LCpLB2G by tek@freeradical.zone
2022-11-08T20:21:49Z
0 likes, 0 repeats
@M0YNG That’s exactly it.
(DIR) Post #APPAEEddfNbULnZq3U by M0YNG@mastodon.radio
2022-11-08T20:27:31Z
0 likes, 0 repeats
@tek I'd normally just YOLO this ... but I'm extra cautious now.I'm not using docker, I'm using mastodon etc. directly on the OS with systemd.do I just copy the service file, change the ExecStart to point to a copy of the config file without the schedule and start the service?
(DIR) Post #APPAV91dmvVRrirnxw by tek@freeradical.zone
2022-11-08T20:30:15Z
0 likes, 0 repeats
@M0YNG That sounds right.
(DIR) Post #APPCLVEuXesJm1bVzM by M0YNG@mastodon.radio
2022-11-08T20:51:18Z
0 likes, 0 repeats
@tek well that's got it moving! thanks a lot for the help and support! :alex_heart_eyes:
(DIR) Post #APPCtQSQpMFwmx6VX6 by tek@freeradical.zone
2022-11-08T20:57:22Z
0 likes, 0 repeats
@M0YNG Awesome! QRO