Post AQXjWJUpWdxCXFJ2Y4 by robs@aus.social
(DIR) More posts by robs@aus.social
(DIR) Post #AQVV8Yd3p77wqP0wOu by SDF@mastodon.sdf.org
2022-12-11T19:42:17Z
0 likes, 1 repeats
Hello Folks, we were able to resolve the pg_toast issue and are working on a write up now which should be helpful to other instances that inevitably will run into this issue.We've also updated our "about" which specifically identifies 0500 to 0700 UTC as our daily maintenance window. It does not mean that the instance will always be taken down during that time, it is just a window we've identified as a low usage period. Please see https://mastodon.sdf.org/about for updates.Thank you!
(DIR) Post #AQVVN5uasLVi7h9AJs by FinchHaven@mastodon.sdf.org
2022-12-11T19:44:53Z
0 likes, 0 repeats
@SDF Bravo!Thanks for the updates!Take the rest of the weekend off!hahaha 😜
(DIR) Post #AQVVOeiuvqo2NZQi5Q by ParadeGrotesque@mastodon.sdf.org
2022-12-11T19:45:02Z
0 likes, 0 repeats
@SDF Mad respect to you guys! Good to know this database is getting back under control! 👍
(DIR) Post #AQVVS8sHHudYl9TEfI by Rabon@mastodon.sdf.org
2022-12-11T19:45:48Z
0 likes, 0 repeats
@SDF When will our pre-migration followers/follows be restored?I lost ⅔ of each that have yet to be restored. Please help.
(DIR) Post #AQVVVToBubMcYQ9OXB by Ricardus@mastodon.sdf.org
2022-12-11T19:46:27Z
0 likes, 0 repeats
@ParadeGrotesque Is it? My account is basically broken beyond repair.
(DIR) Post #AQVVcXVCZqQU2MdPkG by msquebanh@mastodon.sdf.org
2022-12-11T19:47:41Z
0 likes, 0 repeats
@SDF Thank you for all the hard work 👏
(DIR) Post #AQVVlpU35lkkPaWcuu by sarae@mastodon.sdf.org
2022-12-11T19:49:22Z
0 likes, 0 repeats
@SDF Thanks much. Does this mean that the current state of play is "as fixed as it's going to get"? I've been holding off on doing any work until your part was complete.
(DIR) Post #AQVVtmOmRhTFA0qseO by ParadeGrotesque@mastodon.sdf.org
2022-12-11T19:50:50Z
0 likes, 0 repeats
@Ricardus I suspect I should have said "it's slowly" getting back under control... Feel free to contact SDF if your account is still messed up on a couple of days.
(DIR) Post #AQVWCUaaswAjQZIh72 by Aphelion@mastodon.sdf.org
2022-12-11T19:54:12Z
0 likes, 0 repeats
@SDF yayyy!
(DIR) Post #AQVWUuGCBCPaG0XWro by codebalion@mastodon.sdf.org
2022-12-11T19:57:30Z
0 likes, 0 repeats
@SDF Cheers! Looking forward to reading the write-up. Thanks for all the hard work you've all been doing.
(DIR) Post #AQVWhrAH03nqQAwdcW by wasootch@mastodon.sdf.org
2022-12-11T19:59:51Z
0 likes, 0 repeats
@SDF Stuff is still not resolved for me. I'm still missing everything from Nov 20 until that one update, still can't favorite posts. The only thing working for me now is "Explore" has finally updated to new posts.
(DIR) Post #AQVXwXsd581kuQGBmK by HunterZ@mastodon.sdf.org
2022-12-11T20:13:45Z
0 likes, 0 repeats
@ParadeGrotesque @Ricardus this is the one part that's been consistently unclear/confusing about the situation: what, if anything, can we expect will be recovered of the stuff that's gone missing since Nov 20? I've seen no change - everything from posts to account settings changes between then and a few days ago has been lost.I've made peace with the idea that some things may be lost permanently, but it would be really really nice to actually get explicit closure instead of being left hanging.
(DIR) Post #AQVZjtRNoCT7vCbupE by Ricardus@mastodon.sdf.org
2022-12-11T20:33:53Z
0 likes, 0 repeats
@HunterZ @ParadeGrotesque I can't even see my posts on my profile page anymore. It just gives me errors. It says I have like 1 follower and I'm following 2 people. It seems like my account is basically GONE after cultivating an account with cool followers and follows for 5 fucking years. It's beginning to make me angry.
(DIR) Post #AQVZmVS1STEoO69iJU by Ricardus@mastodon.sdf.org
2022-12-11T20:34:21Z
0 likes, 0 repeats
@ParadeGrotesque I have. Crickets. Pretty soon and I'm going to have to restart somewhere else.
(DIR) Post #AQVZrMilLMTelBIphA by HunterZ@mastodon.sdf.org
2022-12-11T20:35:14Z
0 likes, 0 repeats
@Ricardus @ParadeGrotesque oh yeah that's awful - if I try to view your profile, my screen fills up with an endless cascade of "500" errors 😬
(DIR) Post #AQVZwjCLCZPDv3h8tM by Ricardus@mastodon.sdf.org
2022-12-11T20:36:12Z
0 likes, 0 repeats
@HunterZ My situation seems much worse. It's unfuckingbelievable frustrating and disheartening to think 5 years of time building this presence is gone.
(DIR) Post #AQVa76lgQKjfusAwka by mstine@mastodon.sdf.org
2022-12-11T20:38:04Z
0 likes, 0 repeats
@Ricardus @ParadeGrotesque seems like your account was on the long tail of the failure distribution. Please be patient. Up until 3 months ago there were < 5K accounts to support. Now there are nearly 70K. I expect the volunteers have a few thousand requests just like yours to sift through.
(DIR) Post #AQVaQdQzCCjcjMo17Q by msquebanh@mastodon.sdf.org
2022-12-11T20:41:37Z
0 likes, 0 repeats
@Ricardus @HunterZ @ParadeGrotesque If it helps - when I click on your follows & followers - they seem to all show up. It just seems the counts on your profile are off. I can see your toots on my timeline but your page still shows up blank except for 500 error.
(DIR) Post #AQVb2nuCiDwUD8fl8y by mmiasma@mastodon.sdf.org
2022-12-11T20:48:10Z
0 likes, 0 repeats
@SDF Hi guys, some follow-up questions for you please:- What exactly does resolving the problems with pg_toast mean for your user base?- Some people are still having significant access problems (please see @Ricardus as an example) What is the best way to bring these examples to your attention so that they can be corrected?- Several secondary sources have tooted that our lost follows and toots will be resurrected. Is that still your belief as well?Thanks in advance for any clarifications.
(DIR) Post #AQVc6cBILRsUxe31EG by Tengis65@mastodon.sdf.org
2022-12-11T21:00:22Z
0 likes, 0 repeats
Well done you lot 👏👏
(DIR) Post #AQVfzTWCYQWpspg4XI by phf@social.sdf.org
2022-12-11T19:49:53Z
0 likes, 0 repeats
@ParadeGrotesque @SDF Mad respect yes, but I am still rather confused why I am seeing 2d old posts on profiles when I look from social.sdf.org but I see 1d old posts on the same profiles when I look from mastodon.sdf.org. 😕 Is there a 24h+ synchronization gap?
(DIR) Post #AQVgbnacNVq58a19I8 by ratamacue@mastodon.sdf.org
2022-12-11T21:50:49Z
0 likes, 0 repeats
@Ricardus @HunterZ @ParadeGrotesque but when I tap on the "2 followers" in Tusky, I see a LOT more in the list. It's seems like something summary data needs to be reconstructed and its possible some of it can still be recovered.
(DIR) Post #AQVi4CfiofJYzWVruq by Speak4Trees2@mastodon.sdf.org
2022-12-11T22:06:55Z
0 likes, 0 repeats
@SDF Thanks for the repair work. Will we be getting back the followers and posts we made at end of Nov. into early Dec? If so I'll just wait 'till that happens. If not I'll start rebuilding my network of contacts.
(DIR) Post #AQVoX7zLuJbyaZs21w by moonwalkerwiz@mastodon.sdf.org
2022-12-11T23:19:30Z
0 likes, 0 repeats
@SDF Thank you guys for your hard work. Would really appreciate a definitive announcement whether there are still efforts on your side to restore the posts and follows/followers that were lost or if this is it, and we should try rebuilding our profiles now. I think a definitive announcement like that would at least stop people from asking the same question repeatedly to you guys, and we can actually move on.
(DIR) Post #AQW7ICwV6XQOjs8TbM by brogers@social.sdf.org
2022-12-11T21:31:36Z
0 likes, 0 repeats
@wasootch I don't know about the problem with favorites. Please send me a separate message describing that.As for the rest, I wish I had a better answer. Unfortunately, @SDF couldn't recover some of the data. At this point, if it hasn't been recovered, you should begin rebuilding it yourself. If you had recently downloaded any archives or exports of your data, you could use those. If you use a mobile app, you may see some of your posts or other data that the instance itself is now missing.
(DIR) Post #AQW7IDT79HVqN1OWR6 by wasootch@mastodon.sdf.org
2022-12-12T00:31:10Z
0 likes, 0 repeats
@brogers @SDF Oh wow. Really. That's super disappointing. I have nothing that I backed up or downloaded so I guess it's all gone then. For the favorites thing, I click on the star and then it gives a 500 error. It's done that since this happened.
(DIR) Post #AQW7IE6olfGuM9yEK0 by brogers@social.sdf.org
2022-12-12T01:38:04Z
0 likes, 0 repeats
@wasootch @SDF Can you see your list of "Favourites"? I don't have access to the instance's error logs, but maybe we can at least determine the shape of the problem.
(DIR) Post #AQW7IEcMsMVbw0jQUy by wasootch@mastodon.sdf.org
2022-12-12T02:07:01Z
0 likes, 0 repeats
@brogers @SDF Sometimes when I follow people I get that 500 error too. I have trouble with following certain accounts too. Tried so many times.
(DIR) Post #AQW7IF6r30tZSYzm1A by SDF@mastodon.sdf.org
2022-12-12T02:49:17Z
0 likes, 0 repeats
@wasootch @brogers Shannon, looking at your profile page it seems to look normal. There was a timeline rebuild which completed that may have given you timeouts or a blank page. We'll keep an eye on it.
(DIR) Post #AQWABu4AfmLcJuPPoe by wasootch@mastodon.sdf.org
2022-12-12T03:22:18Z
0 likes, 0 repeats
@SDF @brogers But I'm missing like a ton of things. Lists, the tags I was following. I had something like 500 followers and I was following that many or more. And I still have this favorite issue and if you scroll back all my posts after Nov 20 are missing until that one update where things broke.
(DIR) Post #AQXiGgjHHIi60I37M8 by wasootch@mastodon.sdf.org
2022-12-12T03:24:00Z
0 likes, 0 repeats
@SDF @brogers All the posts are missing. See screen shot.
(DIR) Post #AQXiGh71qzi1BxA5NQ by brogers@social.sdf.org
2022-12-12T03:49:50Z
0 likes, 0 repeats
@wasootch Sadly, the lost data is lost. But hopefully the @SDF admins can do something about the "500" errors you're getting when you try to favorite toots and follow accounts.
(DIR) Post #AQXiGhl5S3kfCBu4oa by wasootch@mastodon.sdf.org
2022-12-12T21:17:44Z
0 likes, 0 repeats
@brogers @SDF Still having issues with 500 errors when I favourite things. It worked for a while last night, so I thought it was good to go. Then this morning it's broken again. Is it maybe that I need to clear cache or something. It happens on my phone too.
(DIR) Post #AQXiGisDJ32weaFHYu by SDF@mastodon.sdf.org
2022-12-12T21:18:50Z
0 likes, 0 repeats
@wasootch @brogers can you post the specific link?
(DIR) Post #AQXj3jEakk4bxk53Ds by SDF@mastodon.sdf.org
2022-12-12T21:27:42Z
0 likes, 0 repeats
@wasootch @brogers here is an idea, logout, close the app, open the app and login again.
(DIR) Post #AQXj96833jhaPsa1ku by wasootch@mastodon.sdf.org
2022-12-12T21:28:40Z
0 likes, 0 repeats
@SDF @brogers I did that before on the desktop. Do you think the app is the problem on the phone? Do they communicate back and forth?
(DIR) Post #AQXjBNwbnCvQFWJtdg by wasootch@mastodon.sdf.org
2022-12-12T21:29:06Z
0 likes, 0 repeats
@SDF @brogers There aren't any links with it. Just shows 500 error whenever I click on the little star on any post.
(DIR) Post #AQXjFwIg048XxcoVBg by SDF@mastodon.sdf.org
2022-12-12T21:29:55Z
0 likes, 0 repeats
@wasootch @brogers What would help is to also know what device you are using, the OS version, the application and its version as well.
(DIR) Post #AQXjOD11jjpo7RV9Oq by wasootch@mastodon.sdf.org
2022-12-12T21:31:24Z
0 likes, 0 repeats
@SDF @brogers On desktop I'm using an iMac, MacOS Monterey, version 12.6 with Chrome (Version 107.0.5304.121). On my iPhone I'm using Metatext.
(DIR) Post #AQXjWJUpWdxCXFJ2Y4 by robs@aus.social
2022-12-12T19:10:43Z
0 likes, 0 repeats
@Ricardus I'm so sorry for you this happened, this is terrible
(DIR) Post #AQXjZlpocVtZIIloy8 by SDF@mastodon.sdf.org
2022-12-12T21:33:30Z
0 likes, 0 repeats
@wasootch @brogers It could be a Metatext issue. Can you try with the web browser on the iPhone and see if that works?
(DIR) Post #AQXmSNJnfmGN5qxcWW by philsplace@mastodon.sdf.org
2022-12-12T22:05:33Z
0 likes, 0 repeats
@SDF @wasootch @brogers I had to delete Metatext and reinstall it after my account disappeared.I recreated my account with the exact same info… makes me think the auth token was tied to the user id instead of the user name. Just guessing tho.(Same with my other two iOS mastodon clients, toot! and the official mastodon app) #FWIW #HTH
(DIR) Post #AQXmaogDpJxKGqTPaC by SDF@mastodon.sdf.org
2022-12-12T22:06:59Z
0 likes, 0 repeats
@philsplace @wasootch @brogers This is a great tip. Some Apps have a setting that will allow you to do this without removing the app and reinstalling it but as philspace points out, its likely just a token that needs to be cleared.
(DIR) Post #AQXpzdFZl8CVvLXCG8 by wasootch@mastodon.sdf.org
2022-12-12T22:45:23Z
0 likes, 0 repeats
@SDF @philsplace @brogers Oh, that's interesting. I'll give it a try. I have to get off Metatext anyway and use a different one I think... with the developer being sick now. Hopefully that will help.
(DIR) Post #AQXzNKidD1dOzHV97o by wasootch@mastodon.sdf.org
2022-12-13T00:30:32Z
0 likes, 0 repeats
@SDF @philsplace @brogers I deleted the app and it's still a no go. Deleted Mastodon cookies. Still a problem with favouriting posts. Deleted cache for Mastodon using Chrome developer tools. Still getting the 500 error. I can't think what else I can do on my end except try another browser. I'll try Firefox.
(DIR) Post #AQY1QYe65Mv2xS2uBc by wasootch@mastodon.sdf.org
2022-12-13T00:32:52Z
0 likes, 0 repeats
@SDF @philsplace @brogers Ok, just tried my account on Firefox... hadn't opened it on there before. I still get the 500 error when favouriting a post. There it says "500 Internal Server Error". So I've exhausted all I can see to do for me.
(DIR) Post #AQY1QZG1oLGCr5nCJE by brogers@social.sdf.org
2022-12-13T00:41:14Z
0 likes, 0 repeats
@wasootch @SDF @philsplace Thanks, Shannon. I think that effectively rules out any cause on your side. I suspect that some portion of the database related to your favorites is still corrupted. At this point, it sounds like only the admins can fix it.
(DIR) Post #AQY1QZlZv2UuQwYOUC by wasootch@mastodon.sdf.org
2022-12-13T00:47:27Z
0 likes, 0 repeats
@brogers @SDF @philsplace Yeah, I think so. For some reason I just got booted out and had to reset my password. Anyway, same problem after logging back in with a new password.
(DIR) Post #AQY1QaCWIs33mV9uTo by SDF@mastodon.sdf.org
2022-12-13T00:53:27Z
0 likes, 0 repeats
@wasootch @brogers @philsplace Did you try clearing your browser cache too?
(DIR) Post #AQY1Tde1xLM3KYU36G by wasootch@mastodon.sdf.org
2022-12-13T00:54:05Z
0 likes, 0 repeats
@SDF @brogers @philsplace I tried an entirely different browser. But yes. I cleared it for SDF using the Chrome developer tools.
(DIR) Post #AQYWzx4QMemJQLiWw4 by rajat@mastodon.sdf.org
2022-12-13T06:47:12Z
0 likes, 0 repeats
@SDF I lost my content between 20 Nov - 9 Dec. Was it due to this issue?
(DIR) Post #AQZFRiUCZsdaHcp9Vo by Ricardus@mastodon.sdf.org
2022-12-13T15:05:21Z
0 likes, 0 repeats
@robs Yeah. It actually made me a little nauseous when I thought my 5 year masto presence might be gone.