Posts by ferh@mastodon.host
(DIR) Post #9sEeC9pOwz43lcnpZ2 by ferh@mastodon.host
2020-02-20T11:59:39Z
0 likes, 1 repeats
https://relay.mastodon.host has been migrated to it's new home!
(DIR) Post #9sEeCTqUWdotqnSz2W by ferh@mastodon.host
2020-02-20T16:34:08Z
0 likes, 1 repeats
peertube.mastodon.host migrated
(DIR) Post #9sGt4ribloHFCdWfsO by ferh@mastodon.host
2020-02-21T18:52:16Z
0 likes, 0 repeats
@gled wait what the fuck :'
(DIR) Post #9sGt7GHpOiO9yj8tJw by ferh@mastodon.host
2020-02-21T18:57:16Z
0 likes, 0 repeats
@gled yeah, AMD Epyc + NVMe is like a rocket ! love them :)
(DIR) Post #9sGth6JlwEDXUD8oqG by ferh@mastodon.host
2020-02-21T19:02:43Z
1 likes, 0 repeats
@gled I definitely love how it never goes beyond than a load average of 3. :D :D thoses boxes are so fast
(DIR) Post #9xi05jZ3ldYuARGMmO by ferh@mastodon.host
2020-08-02T10:14:44Z
0 likes, 1 repeats
due to an incident on Scaleway Object Storage, media uploads/medias may have issues - https://status.scaleway.com/incident/933
(DIR) Post #9xnwwHq05Z6QVEZwmG by ferh@mastodon.host
2020-08-05T07:07:29Z
1 likes, 1 repeats
🛎️ Syncing of media files to object storage went too fast and rate limited the storage bucked during the night (CEST night), resulting in Mastodon's backlog processing/retry queue having a damn huge latency (4 hours right now).Stopped the sync (which was mostly complete) so rate limiting would stop and Mastodon could process more without retrying.Gonna add temporarily more workers for the queue so it processes the backlog more rapidly
(DIR) Post #9xnyVwfONbONDlBTO4 by ferh@mastodon.host
2020-08-05T07:25:08Z
1 likes, 1 repeats
interestingly even after stopping the sync we're still getting rate limited by object storage (while it shouldn't exceed 250 req/s, as there's just 50 workers)…
(DIR) Post #9xnzbhYZZT6839IqVE by ferh@mastodon.host
2020-08-05T07:36:38Z
0 likes, 1 repeats
@trinsec Weird (20 hours) because Sidekiq is saying (Latency: 4 hours). But cool. Not really solved tho, still getting a lot of rate limiting errors from S3. still on it..
(DIR) Post #9xo7LvPV5x1y14311s by ferh@mastodon.host
2020-08-05T08:46:23Z
1 likes, 1 repeats
New update: Scaleway Object Storage is just incredibly broken. Even without any requests it's still limiting or even returning error 400.
(DIR) Post #9xo7eS5iKKv2aQcwD2 by ferh@mastodon.host
2020-08-05T09:07:29Z
1 likes, 1 repeats
Amazing! Three hours after opening the ticket at Scaleway they responded. We never hit the rate limit, it was a technical issue on their side. Should be fixed now and the processing queue should process everything now
(DIR) Post #9xoH7t0iDgdg3pOiPo by ferh@mastodon.host
2020-08-05T10:53:25Z
0 likes, 1 repeats
@trinsec yes quite sarcastic. I wish they were monitoring their services a bit more.... always annoying to stumble upon an issue on their side, while they don't notice it, and act after some hours...
(DIR) Post #9xopPGIIImuhdc4uzw by ferh@mastodon.host
2020-08-05T17:16:23Z
0 likes, 1 repeats
@trinsec yeah :( the queue is going down very slowly. For some reason I can get Sidekiq to use all his threads nor all the DB connections.
(DIR) Post #9xoz5ZsKmxcY7OBjxw by ferh@mastodon.host
2020-08-05T19:06:15Z
0 likes, 1 repeats
@panina @panina see my posts for the status. sorry :)
(DIR) Post #9xozBqyU7wg4t73e3k by ferh@mastodon.host
2020-08-05T19:06:33Z
0 likes, 1 repeats
another update: the queue is still processing :(
(DIR) Post #9xp5CL3csYqBuh4qK8 by ferh@mastodon.host
2020-08-05T20:14:37Z
0 likes, 1 repeats
@eqe haha :-) It's not ODS I'm using, but "Object Storage". Either way -- ODS had a shit ton of issues since sept, and yeah, you can see it right ? :))) @trinsec
(DIR) Post #9xq3QKvSA0J0oBlFx2 by ferh@mastodon.host
2020-08-06T07:29:45Z
0 likes, 1 repeats
@trinsec yes! \o/
(DIR) Post #9ygDnqd71O04tc9dbs by ferh@mastodon.host
2020-08-31T11:31:00Z
0 likes, 0 repeats
@popey Why not try migrating to Pleroma ? It's more lightweight. A lot more.