Post AXKfUmPTcp7XydESUC by nickzoic@aus.social
 (DIR) More posts by nickzoic@aus.social
 (DIR) Post #AXKeZhQVTs8I0zUoFs by mjg59@nondeterministic.computer
       2023-07-03T23:08:13Z
       
       0 likes, 0 repeats
       
       Hitting Twitter with a Googlebot user agent now returns "Unverified web crawlers are not allowed"
       
 (DIR) Post #AXKerCHU1AbQdCcQKG by Viss@mastodon.social
       2023-07-03T23:10:20Z
       
       0 likes, 0 repeats
       
       @mjg59 i wonder what would happen if you figured out the user agent that twitter itself uses to do things like fetch cards and whatnot and used that against it :D
       
 (DIR) Post #AXKf5T7qBfk0vZuifI by dango_@mas.to
       2023-07-03T23:11:15Z
       
       0 likes, 0 repeats
       
       @mjg59 bold strategy, ngl
       
 (DIR) Post #AXKfUmPTcp7XydESUC by nickzoic@aus.social
       2023-07-03T23:14:32Z
       
       0 likes, 0 repeats
       
       @mjg59 surprisingly polite response, for Twitter.
       
 (DIR) Post #AXKfqqV760TH2GOHCK by sam@queernerds.social
       2023-07-03T23:19:25Z
       
       0 likes, 0 repeats
       
       @mjg59 Google pay $8
       
 (DIR) Post #AXKg7NyeOo8RUh5q2i by mjg59@nondeterministic.computer
       2023-07-03T23:22:22Z
       
       0 likes, 0 repeats
       
       Although there's still material being indexed, so it's possible that they're still allowing it from specific IP ranges
       
 (DIR) Post #AXKhyR0jwz6uClvj6m by FritzAdalis@infosec.exchange
       2023-07-03T23:46:59Z
       
       0 likes, 0 repeats
       
       @mjg59What if you use Spider Man's user agent?
       
 (DIR) Post #AXKiOEpjqf0Bxz5X60 by natanbc@uwu.social
       2023-07-03T23:51:08Z
       
       0 likes, 0 repeats
       
       @mjg59 wonder if GCP ranges bypass that
       
 (DIR) Post #AXLPpQonh1eoz6rXzU by flameeyes@mastodon.social
       2023-07-04T07:58:38Z
       
       0 likes, 0 repeats
       
       @mjg59 so they caught up with me circa 2010.https://flameeyes.blog/2010/05/01/so-you-think-you-can-crawl/