Post AaIeuiqJ29OwDBHfUG by fzorb@akko.layer08.com
(DIR) More posts by fzorb@akko.layer08.com
(DIR) Post #AaIbsltwOJIfFwMWoK by Salastil@pleroma.salastil.com
2023-09-30T17:07:41.277714Z
3 likes, 1 repeats
@p You seem like a clever guy that may know of this web technique. I'm getting my Nitter instanced scraped by a botnet that appears to be 100k IP large, they get fed in as fast as I ban them, but I don't believe they assign more than 100 IP to scraping at a time as to not DDOS the site, one IP never doing a scrape under 7 seconds so rate limiting wont nab them. Banning doesn't seem to be working, what I want to do is something I read about 10+ years ago, a sticky trap. I want to ensnare the bot into a perpetually open http request so that it never completes its loop, I figure that it the botnet notices when its banned and starts getting 403'd, but if the loop never completes it may just seize up that load balancer.Happen to know what this technique is called or of any reading material?
(DIR) Post #AaIcOq50OJNxTwpR3o by graf@poa.st
2023-09-30T17:13:28.134448Z
4 likes, 0 repeats
@Salastil @p it's literally called a honeypot trap but a better solution might be to restrict or ban the url they are trying to reach. for example, some guy was scraping poast nitter for furry art with an entire /24. I return 402 whenever somebody tries to fetch that user, the guy scraping fucks off to the next instance
(DIR) Post #AaIeuiqJ29OwDBHfUG by fzorb@akko.layer08.com
2023-09-30T17:41:36.613556Z
2 likes, 0 repeats
@graf @p @Salastil should've replaced the page for the /24 with one imitating an account with a bunch of junk data
(DIR) Post #AaIewxmCTthz6hQqsy by graf@poa.st
2023-09-30T17:42:03.484192Z
4 likes, 0 repeats
@fzorb @p @Salastil not hard to do in hindsight. if i run into this again, I will
(DIR) Post #AaIfXvuGOe1a1QpefA by Salastil@pleroma.salastil.com
2023-09-30T17:48:44.643287Z
1 likes, 0 repeats
In my case they're not scraping a single account, they're scraping half of twitter via my instance, such simple regex options have never worked for me. One of the approaches brought up was to ban anything that wasn't containing a referrer from the site.
(DIR) Post #AaIjyldn4ViXoqB6jQ by PunishedD@poa.st
2023-09-30T18:38:25.394822Z
1 likes, 0 repeats
@graf @fzorb @p @Salastil Years ago I did this by writing a custom “webserver”. If you can detect which are the bad actors, you route them to a simple thread with a socket outputting Math.random() data every second. Probably have to wrap that in boilerplate X/nitter post data if it’s a scraper.
(DIR) Post #AaJ3CmB5lmoqfU9F5M by Salastil@pleroma.salastil.com
2023-09-30T22:13:50.860845Z
2 likes, 1 repeats
Welp, after autobanning anything that connected to the site for 8 hours the botnet is only increasing in speed. The access log moves so fast I cannot even begin to read it any longer.
(DIR) Post #AaJ3TgEB95zX8XeVOa by jeff@misinformation.wikileaks2.org
2023-09-30T22:16:48.435457Z
3 likes, 0 repeats
@Salastil @p fingerprint the botnet client's TLS and tarpit those connections
(DIR) Post #AaJ4ldOAXULTUzRyfg by p@freespeechextremist.com
2023-09-30T22:31:21.726800Z
4 likes, 1 repeats
@Salastil > I'm getting my Nitter instanced scraped by a botnet that appears to be 100k IP large, they get fed in as fast as I ban them, but I don't believe they assign more than 100 IP to scraping at a time as to not DDOS the site,WEIRD> one IP never doing a scrape under 7 seconds so rate limiting wont nab them.Out of curiosity, what UAs are they using? Tried SSL fingerprinting? You know why they'd be hitting your server, like did you check if DiscordBot or something is in your referrers, or someone linked to it from somewhere, or...?> something I read about 10+ years ago, a sticky trap. I want to ensnare the bot into a perpetually open http request so that it never completes its loop,Ah, okay, so you can do this pretty easily with nginx: you can forward to different backends conditionally with one of the (really badly documented) `if` directives. Set up a little script, listen/accept on one end, and then make a connection on the other to the actual upstream. So if your Nitter instance is running on localhost:4444, you have this script listen on localhost:4445. Have it relay all of the traffic upstream and then get the entire response (to avoid jamming up the real server), but trickle the response a few bytes at a time. Some clients time out if you take too long to get the headers to them, so maybe send the headers back faster, but like delay a couple of seconds, then send the headers, then trickle the rest at a few bytes per second.Another way to do this is to use iptables. True story: `-m statistic --mode random --probability 0.5 -j DROP` does more or less what you would expect. This is what I did when Pawoo was flooding FSE with massive numbers of deletes, like as a kind of dopey rate-limiting ability. (Unintentional on their part: a few accounts with really long post history deleted themselves, and this causes Mastodon to send one delete per activity since the beginning of time to every server it has heard of...except the ones that it has blocked.)> I figure that it the botnet notices when its banned and starts getting 403'd,Basically zero of the scrapers that hit FSE do this. Boardreader.com didn't even notice when I started actively poisoning their data until about a week after I started including the phone number of the guy that was ignoring my emails in the data.
(DIR) Post #AaJ4qjGI8Bo1A4C9GS by p@freespeechextremist.com
2023-09-30T22:32:16.742441Z
1 likes, 0 repeats
@graf @Salastil > better solution might be to restrict or ban the url they are trying to reach.If their target's the server, that might not work. It is best to do it by behavior rather than source, though, absolutely.
(DIR) Post #AaJ5vyeYQxMQIabtNw by Salastil@pleroma.salastil.com
2023-09-30T22:44:25.758517Z
1 likes, 0 repeats
>Out of curiosity, what UAs are they using? Tried SSL fingerprinting? You know why they'd be hitting your server, like did you check if DiscordBot or something is in your referrers, or someone linked to it from somewhere, or...?Nitter is a Twitter proxy and there is only a few left after Elon's antics trying to make it a walled garden. The current design of Nitter requires us to make a large number of "guest accounts" that are created during an onboarding process using an old Android version of the Twitter account. These guest accounts give us access to most API features that used to exist because the walled garden. Each one has about 499 requests out of them before getting rate limited and only last 30.5 days before expiring. As to why? Nitter is effectively the only way to scrape content from Twitter, the guest_account stuff can only be created 1 per IP per day, so a lot have to be generated via proxy service. All of the basic stuff like obvious bot user agents have been handled, these botnets never have a single IP make a request more than once every 7-11 seconds and always with a legitimate User Agent. Sometimes it looks like desktop windows Chrome sessions, sometimes Iphones, its all over the place no real pattern, same with the stuff being searched for. I think I may come up with a way of 403ing anything that doesn't have a referrer to specific endpoints. In theory they should hit the root page, search from there and get referred to another page.
(DIR) Post #AaJ6GlRKktgdOrL2Ei by p@freespeechextremist.com
2023-09-30T22:48:11.616050Z
3 likes, 2 repeats
@Salastil @fzorb @graf > they're scraping half of twitter via my instance,Ha, it sounds a lot like what Boardreader was doing to FSE. They actually recorded browser sessions and played them back, big army of residential US proxies. I actually ended up writing a script that watched the logs and waited until some client had a suspiciously high proportion of requests hitting TWKN (watching behavior instead of source) and they would fire off a few hundred requests and then hop IPs. If I killed an IP, another one would arrive really quickly. Since they'd recorded browser sessions, it was hard to tell until they had already gotten some of the data already, but by the time they had hit several hundred requests for TWKN after the initial burst, it was too late to detect them. Maybe check if you see `devtools.boardreader.com` in your logs anywhere, ha.They weren't executing JavaScript (they couldn't) but I didn't wanna break all the clients by doing something like that. Nitter, on the other hand, is basically *just* a web UI, so you could go that route. Tack on some JS that adds a hash of the IP address plus a nonce to every link, this precludes a lot of proxy use and non-JS-executing scrapers because they'd have to know which place they're exiting from then do a hash.> One of the approaches brought up was to ban anything that wasn't containing a referrer from the site.That works sometimes, but they will pretty often spoof it or start spoofing it.
(DIR) Post #AaJ6zjVInfRSQWdZUe by Salastil@pleroma.salastil.com
2023-09-30T22:56:17.590528Z
1 likes, 0 repeats
>That works sometimes, but they will pretty often spoof it or start spoofing it.They already are spoofing to a degree, but they fuck up and will use a referrer from the wrong site sometimes and I'll see referrer from nitter.poast.org or one of the other instances, and this isn't how nitter operates. I just think banning isn't a viable strategy at this point, I've banned about 120k IP today and the botnet doesn't seem to have slowed a bit. I've been dealing with this guy since August and have managed to get him to fuck off multiple times, but this time he seems hellbent on scraping my instance until the instance no longer functions.
(DIR) Post #AaJBzBoqXT3pj5L5do by p@freespeechextremist.com
2023-09-30T23:52:14.213550Z
1 likes, 0 repeats
@Salastil @fzorb @graf ...They've all got the same user-agent.
(DIR) Post #AaJC2WhZM1EfMSWhXM by kirby@lab.nyanide.com
2023-09-30T23:52:45.923761Z
1 likes, 0 repeats
@Salastil @fzorb @p @graf now get a dark hoodie and an rgb keyboard and make it green. Also make your terminal colors green on black
(DIR) Post #AaJCOAcK92QtEoOSg4 by Salastil@pleroma.salastil.com
2023-09-30T23:56:45.242962Z
1 likes, 0 repeats
This batch right there does since its at the tail end of an 8 hour session of banning everything that connected to the site. The guy does indeed rotate user agents on his bots, I've seen him masquerade as Netscape Navigator 5 at one point, which was impressive since the browser was never released.
(DIR) Post #AaJCcjpK26NQUcQ5o0 by p@freespeechextremist.com
2023-09-30T23:59:23.712808Z
2 likes, 0 repeats
@jeff @Salastil If they're using headless Chrome, that randomizes some bits of the handshake, it's hard to fingerprint.
(DIR) Post #AaJCgkL84FB1F5CzoW by jeff@misinformation.wikileaks2.org
2023-10-01T00:00:04.118147Z
2 likes, 0 repeats
@p @Salastil it's probably not that, too much overhead
(DIR) Post #AaJCjvZokGyoxgb0Yi by Salastil@pleroma.salastil.com
2023-10-01T00:00:40.901779Z
2 likes, 3 repeats
@kirby @fzorb @p @graf I already have a RGB keyboard and I use tiling window manger i3wm btw did you know I use Arch ? :archlinux:
(DIR) Post #AaJClBAgFbDdcJR2Cu by EllisDee@wolfgirl.bar
2023-10-01T00:00:54.091464Z
2 likes, 0 repeats
@jeff @p @Salastil too much futa head
(DIR) Post #AaJCoC8h6lk5UIwaZc by jeff@misinformation.wikileaks2.org
2023-10-01T00:01:23.035176Z
2 likes, 0 repeats
@EllisDee @p @Salastil tru
(DIR) Post #AaJD8np2iHO89RZwNE by jeff@misinformation.wikileaks2.org
2023-10-01T00:05:03.830529Z
1 likes, 0 repeats
@Salastil @p nginx has 444 error code which is better than 403 here as 444 hard closed the connection immediately.
(DIR) Post #AaJDO7TFwUQzQygxJA by jeff@misinformation.wikileaks2.org
2023-10-01T00:07:50.312338Z
1 likes, 0 repeats
@Salastil @p you should have teach requests return static content for bots that is basically something that poisons their dataset. like a markov chain poster with a bunch of fake followers that are smurfed. that us a good tarpit idea
(DIR) Post #AaJDTJU6heRS8rZBL6 by jeff@misinformation.wikileaks2.org
2023-10-01T00:08:50.425342Z
2 likes, 0 repeats
@Salastil @p every post ending with "and the pee is stored in the balls too"
(DIR) Post #AaJDUrXzXoag4i7diq by graf@poa.st
2023-10-01T00:09:09.153549Z
1 likes, 0 repeats
@Salastil @fzorb @p map $http_user_agent $baduseragents { default 0; "~Trident/[1-7]\." 1; "~Chrome/(([1-9]{1})|([0-7]{1}[0-9]{1})|(7[0-9]{1}))\." 1; "~YaBrowser/(([1-9]{1})|([1-9]{1}[0-8]{1}))\." 1; "~Firefox/(([1-9]{1})|([0-7]{1}[0-9]{1})|(8[0-9]{1}))\." 1; "~EdgA?/(([1-9]{1})|([0-7]{1}[0-9]{1})|(8[0-6]{1}))\." 1; "~Version/(([1-9]{1})|([1-9]{1}[0-1]{1}))\." 1; }
(DIR) Post #AaJEPESrWe9hMtbOSG by jeff@misinformation.wikileaks2.org
2023-10-01T00:19:15.005585Z
1 likes, 0 repeats
@Salastil @p oh oh, idea, map every user to some random fedi user and gave it serve that as the tarpit. lots of ganee words in there tbh
(DIR) Post #AaJEXCKRLmwd3pSdua by Salastil@poa.st
2023-10-01T00:20:47.914472Z
1 likes, 0 repeats
@graf @Salastil @fzorb @p Doesn't this cover almost every single permutation of Chrome Firefox or uh Edge?
(DIR) Post #AaJEZIZV6uEf0BVVR2 by Salastil@pleroma.salastil.com
2023-10-01T00:21:10.365520Z
1 likes, 0 repeats
@graf @fzorb @p Doesn't this cover every permutation of Chrome Firefox or Edge?
(DIR) Post #AaJXR9qhYTe4lINtZY by p@freespeechextremist.com
2023-10-01T03:52:36.466663Z
1 likes, 0 repeats
@Salastil > Nitter is a Twitter proxyYep. I was running one for a while. (Killed it when it stopped working for a while after the Twitter lockdown, didn't miss it, never turned it back on.)> the guest_account stuff can only be created 1 per IP per day, so a lot have to be generated via proxy service.Ah, okay, so there is a reason they'd do that without targeting you specifically.> All of the basic stuff like obvious bot user agents have been handled, these botnets never have a single IP make a request more than once every 7-11 seconds and always with a legitimate User Agent.Yep, saw the logs.> I think I may come up with a way of 403ing anything that doesn't have a referrer to specific endpoints. In theory they should hit the root page, search from there and get referred to another page.Sounds reasonable.
(DIR) Post #AaJYAWeDdY60VBB99s by Lyx@cum.salon
2023-10-01T04:00:48.112889Z
0 likes, 0 repeats
@p @fzorb @Salastil @graf ha u said nonce thats a naughty brittish word
(DIR) Post #AaJYHBAjseubvLYw2y by Lyx@cum.salon
2023-10-01T04:02:00.330595Z
1 likes, 0 repeats
@jeff @p @Salastil im learning so much right now wowser
(DIR) Post #AaJdt5alZkFRONdIHY by p@freespeechextremist.com
2023-10-01T05:04:53.120931Z
0 likes, 0 repeats
@Salastil @fzorb @graf > but they fuck up and will use a referrer from the wrong site sometimes and I'll see referrer from nitter.poast.orgDang.> I've been dealing with this guy since August and have managed to get him to fuck off multiple times, but this time he seems hellbent on scraping my instance until the instance no longer functions.If he's mass-scraping Twitter through public Nitter instances, then he has to be able to account for them going down. Maybe him switching when one disappears is how you get the bad referrers, like he retries the same req against different ones but didn't bother to update the referer.
(DIR) Post #AaJgDDFozt6or8YkGO by p@freespeechextremist.com
2023-10-01T05:30:55.951839Z
0 likes, 0 repeats
@Salastil @fzorb @graf He must have hacked jwz's Gibson.
(DIR) Post #AaJgRwMo4ABWpBqiPI by miscbrains@misc.brainsoap.net
2023-10-01T05:33:29.128Z
2 likes, 1 repeats
@Salastil@pleroma.salastil.com @kirby@lab.nyanide.com @fzorb@akko.layer08.com @p@freespeechextremist.com @graf@poa.st Amber on black > green on black.
(DIR) Post #AaJgTiOMzknO3tQW7E by graf@poa.st
2023-10-01T05:33:53.473693Z
3 likes, 0 repeats
@miscbrains @Salastil @p @kirby @fzorb new vegas enjoyer spotted
(DIR) Post #AaJghvo1v3nPibCwvQ by LordMordred@poa.st
2023-10-01T05:36:28.539094Z
0 likes, 0 repeats
@graf @kirby Ah yes the P i s s f o n t
(DIR) Post #AaJhqh0SPfPoNXlJrc by miscbrains@misc.brainsoap.net
2023-10-01T05:49:15.352Z
1 likes, 0 repeats
@Salastil@pleroma.salastil.com @kirby@lab.nyanide.com @fzorb@akko.layer08.com @p@freespeechextremist.com @graf@poa.st The first computer i got to play with was a dual 5.25 floppy amber screened laptop type thing. Played zork, and some kinda ascii dungeon crawler thing on it. Shame modern web has a bit of a fit when you try to cram in low ascii characters.
(DIR) Post #AaJlSfhpzmvxDEr2LA by p@freespeechextremist.com
2023-10-01T06:29:45.090755Z
1 likes, 1 repeats
@Lyx @Salastil @fzorb @graf It's actually the same sense of the word that became a slang term in the UK.
(DIR) Post #AaJmjlxg5hTXUTyvsO by p@freespeechextremist.com
2023-10-01T06:44:02.835502Z
1 likes, 0 repeats
@miscbrains @graf @Salastil @kirby @fzorb Accurate. It's inviting, it's pelasant.
(DIR) Post #AaJmnwfYDprvrGUc52 by p@freespeechextremist.com
2023-10-01T06:44:48.082862Z
2 likes, 0 repeats
@graf @miscbrains @Salastil @kirby @fzorb I am a Hercules Monochrome CGA video card with matching monitor enjoyer.
(DIR) Post #AaJmuyAn1sKIY7SCfI by sysrq@lab.nyanide.com
2023-10-01T06:46:03.445735Z
1 likes, 0 repeats
@p @fzorb @kirby @miscbrains @Salastil @graf kino
(DIR) Post #AaJoHkUi9VHijxCKQq by p@freespeechextremist.com
2023-10-01T07:01:23.563995Z
1 likes, 0 repeats
@sysrq @Salastil @fzorb @graf @kirby @miscbrains Yeah, I wish I still had it.
(DIR) Post #AaJoR17XhnPIubmReK by miscbrains@misc.brainsoap.net
2023-10-01T07:03:03.044Z
1 likes, 0 repeats
@p@freespeechextremist.com @sysrq@lab.nyanide.com @Salastil@pleroma.salastil.com @fzorb@akko.layer08.com @graf@poa.st @kirby@lab.nyanide.com Damn shame. The 80s era monochrome displays are such a vibe.
(DIR) Post #AaJp6zzOIoXlZ6zBpY by p@freespeechextremist.com
2023-10-01T07:10:39.295538Z
0 likes, 0 repeats
@miscbrains @graf @Salastil @kirby @sysrq @fzorb Impossible to get from internets; Mr. Rogers has suggested making friends with people that work at telcos because office closures result in mounds of old hardware.
(DIR) Post #AaK1ziLwm2VIFEOR5U by laurel@freespeechextremist.com
2023-10-01T09:35:00.205388Z
1 likes, 0 repeats
@Salastil @p What if you start returning garbage data. Use a template to make a page on the fly, fill it with a lot of gamer words and gore, and serve it back.8d8c8a8d30352a60ecf926840a078b42154c5c00d8f9385a7b83b4d85c27153c.png
(DIR) Post #AaK2o5tFXNm5iszd4q by Salastil@pleroma.salastil.com
2023-10-01T09:44:05.109771Z
2 likes, 0 repeats
The problem with garbage data is that it could ensnare normal users, it is difficult to identify the bot with 100% accuracy. Secondly I don't want to get into a legal gray zone, Nitter is a proxy of Twitter accounts, its a very simple front end and lightweight, but for the most part its 100% faithful to what is on the selected user's timeline is. If I start getting into the rat race of poisoning the feed I could in theory run into libel lawsuits from some demented Twitter user that thinks their reputation is being ruined because the feed is full of gore and gamer words. Third, if I start getting into the habit of "curating" the timelines I wouldn't be able to hide behind Section 203 as effectively when some cretin starts browsing child porn accounts and hashtags, most of the VPS hosting companies are aware of Nitter and when you get a nastygram from the gubmint they are more willing to play ball since you're just repacking public data, if Nitter gets a reputation for being all over the place that trust is eroded. Sorry I just woke up but I think my thoughts on the matter are clear enough.
(DIR) Post #AaK3jOQ1JyWZh7dxIW by laurel@freespeechextremist.com
2023-10-01T09:54:27.829983Z
1 likes, 0 repeats
@Salastil @p You don't have to identify all of them with 100% certainty, just some of them.You don't have to retain the original usernames or images on the scraped pages, those can also be replaced.>I wouldn't be able to hide behind Section 203 as effectivelyIs there a single similar case where this happened?
(DIR) Post #AaK4RSgonrGfgMVwmG by Salastil@pleroma.salastil.com
2023-10-01T10:02:25.211734Z
1 likes, 0 repeats
>Is there a single similar case where this happened?Legal fees still cripple people even if you win the case, I have no intention of handing insane people ammunition to grind me down for no reason. Were this a Pleroma instance I was running and I was the HNIC I'd consider well poisoning because ultimately its _MY_ domain and the users would have to abide by some sort of EULA or be briefed that I was doing such things in advance to their data. With Nitter its just meant to be a replication of existing data. >You don't have to identify all of them with 100% certainty, just some of them. Therein lies the problem, I _can't_ identify between a random user that just has a bookmark of @realgronalddrumpf and just lands at his timeline and a bot that just lands at the timeline. This requires getting into invasive practice like fingerprinting or using CAPTCHA programs. I'm supposed to be offering a privacy frontend, subjecting the users to this stuff defeats the purpose.
(DIR) Post #AaK6jOkIPAr6ZToywa by laurel@freespeechextremist.com
2023-10-01T10:28:04.794835Z
1 likes, 0 repeats
@Salastil @p >Legal fees still cripple peopleI didn't say a lost case, I said any case for someone being litigated for something similar.>I _can't_ identify between a random userBut you have been doing it for a couple of days now with hundreds of thousands of IPs.Nobody said anything about CAPTCHA. You'd use the same way to identify them as you'd use to perform the sticky attack.You've been using fingerprinting btw, if at a rudimentary level.You can always use some less offensive content if you don't want to use Fuck niggers, faggots and Trannies.
(DIR) Post #AaKe0Djht8M5puAuno by graf@poa.st
2023-10-01T16:40:51.705029Z
4 likes, 0 repeats
@miscbrains @Salastil @p @kirby @fzorb my gf the other day "i miss when you used to play fallout"
(DIR) Post #AaKe606nT5XhJBjPUm by tyler@1611.social
2023-10-01T16:41:55.266007Z
1 likes, 0 repeats
I'm building a half-timber cottage and I might use an old audio oscilloscope I have as a monitor for a small terminal.
(DIR) Post #AaKe8Fb4qzM5vLy3UW by tyler@1611.social
2023-10-01T16:42:19.254064Z
1 likes, 0 repeats
I switched fnv to German and now it's impossible
(DIR) Post #AaKeL3D6DbxLGur7Ls by RaHoWaJoe@poa.st
2023-10-01T16:44:38.764257Z
1 likes, 0 repeats
@graf @miscbrains @Salastil @p @kirby @fzorb I miss your Doritos
(DIR) Post #AaKeWk0PRlaVBVUijA by graf@poa.st
2023-10-01T16:46:44.249991Z
4 likes, 0 repeats
@RaHoWaJoe @miscbrains @Salastil @p @kirby @fzorb :graf_1::graf_2::graf_3::graf_4::graf_5: :graf_6::graf_7::graf_8: :graf_9:
(DIR) Post #AaKfOP8bFVz66Lujtg by TeaTootler@poa.st
2023-10-01T16:56:27.197208Z
1 likes, 0 repeats
@graf @miscbrains @Salastil @p @kirby @fzorb Me too
(DIR) Post #AaL0cDBqZ5zS8crG6K by p@freespeechextremist.com
2023-10-01T20:54:14.363432Z
0 likes, 0 repeats
@Salastil @laurel > if I start getting into the habit of "curating" the timelines I wouldn't be able to hide behind Section 203 as effectivelyI don't think feeding garbage data to bots counts as curating: it's not exercising editorial control over the posts. It's like putting an alarm on the back door because legitimate customers are supposed to come in through the front. I get what you mean about poisoned data, though, so there are a lot of options.Have you tried associating cursors with IPs?
(DIR) Post #AaLIIHur5CG6B8Ykue by Salastil@pleroma.salastil.com
2023-10-02T00:12:19.218693Z
2 likes, 0 repeats
I managed to smack down 90% of the bots by 403ing anything that makes a request to a specific endpoint without a referrer from the site itself. In normal cases the site should operate with them going to the root page / -> search -> then either to the timeline of an account or to a reply. This is a bit draconian in that it prevents people with a bookmark from just showing up to the timeline with_replies but I set up a 403 explaining why. I doubt the guy with the botnet is really investigating why his bots are getting 302'd to an error page, its just not getting data. Now its back down to the baseline bots again.
(DIR) Post #AaLcwtEbkbMoaRR8Ge by Lyx@cum.salon
2023-10-02T04:03:47.167885Z
1 likes, 0 repeats
@p @fzorb @Salastil @graf i always thought a nonce was a knob end or something like a girly faggot. Whats the dog there sounds like an interesting story and now i dont know what it means ha
(DIR) Post #AaLd2iUiaxkn0wkazw by p@freespeechextremist.com
2023-10-02T04:04:50.598749Z
0 likes, 0 repeats
@Salastil @laurel I think in general, just hitting the root isn't going to help; like RSS readers, for instance.I'm assuming you've got a reasonable robots.txt in place, so all the bot traffic you're getting is unwelcome.