[HN Gopher] Ask HN: How to stop an AWS bot sending 2B requests/m...
___________________________________________________________________
Ask HN: How to stop an AWS bot sending 2B requests/month?
I have been struggling with a bot- 'Mozilla/5.0 (compatible;
crawler)' coming from AWS Singapore - and sending an absurd number
of requests to a domain of mine, averaging over 700 requests/second
for several months now. Thankfully, CloudFlare is able to handle
the traffic with a simple WAF rule and 444 response to reduce the
outbound traffic. I've submitted several complaints to AWS to get
this traffic to stop, their typical followup is: We have engaged
with our customer, and based on this engagement have determined
that the reported activity does not require further action from AWS
at this time. I've tried various 4XX responses to see if the bot
will back off, I've tried 30X redirects (which it follows) to no
avail. The traffic is hitting numbers that require me to re-
negotiate my contract with CloudFlare and is otherwise a nuisance
when reviewing analytics/logs. I've considered redirecting the
entirety of the traffic to aws abuse report page, but at this
scall, it's essentially a small DDoS network and sending it
anywhere could be considered abuse in itself. Are there others
that have similar experience?
Author : lgats
Score : 197 points
Date : 2025-10-17 05:28 UTC (17 hours ago)
| giardini wrote:
| Hire a lawyer and have him send the bill for his services to them
| immediately with a note on the consequences of ignoring his
| notices. Bill them aggressively.
| Animats wrote:
| Yes. Computer Fraud and Abuse Act to start.
|
| The first demand letter from a lawyer will usually stop this.
| The great thing about suing big companies is that they have to
| show up. You have no contractual agreement which prevents
| suing; this is entirely from the outside.
| SoftTalker wrote:
| Threatening to sue is one thing. Actually doing it will cost
| you time and money. And even if you get a judgement how are
| you going to collect from some rando in Singapore?
| tracker1 wrote:
| AWS isn't some rando in Singapore.
| SoftTalker wrote:
| AWS isn't doing this. The rando renting the AWS instance
| in Singapore is.
| Animats wrote:
| There are ways. You sue AWS and "Does 1-50". Then AWS's
| lawyers become eager to tell you who misused their
| service so you can sue the other party. Talk to a lawyer.
| tempestn wrote:
| That's not how lawyers or bills work, unfortunately in this
| case, but fortunately in general.
| bigfatkitten wrote:
| Do you receive, or expect to receive any legitimate traffic from
| AWS Singapore? If not, why not blackhole the whole thing?
| caprock wrote:
| Agreed. You should be able to set the waf to just drop the
| packets and not even bother with the overhead of a response. I
| think cloud flare waf calls this "block".
| marginalia_nu wrote:
| Yeah, this is the way. Dropping the packets makes the
| requests cheaper to respond to than to make.
|
| The problem with DDoS-attacks is generally the asymmetry,
| where it requires more resources to deal with the request
| than to make it. Cute attempts to get back at the attacker
| with various tarpits generally magnifies this and makes it
| hit even harder.
| firecall wrote:
| Yep, I did this for a while.
|
| The TikTok Byte Dance / Byte Spider bots were making millions
| of image requests from my site.
|
| Over and over again and they would not stop.
|
| I eventually got Cloudinary to block all the relevant user
| agents, and initially just totally blocked Singapore.
|
| It's very abusive on the part of these bot running AI scraping
| companies!
|
| If I hadn't been using the kind and generous Cloudinary, I
| could have been stuck with some seriously expensive hosting
| bills!
|
| Nowadays I just block all AI bots with Cloudflare and be done
| with it!
| lozenge wrote:
| Here's the IP address ranges-
| https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-work...
| Jean-Papoulos wrote:
| You don't even need to send a response. Just block the traffic
| and move on
| MrThoughtful wrote:
| If it follows redirects, have you tried redirecting it to its own
| domain?
| lgats wrote:
| I've tried localhost redirects, doesn't impact the speed of
| their requests, all ports are closed on the suspect machines
| Scotrix wrote:
| Just find a Hoster with low traffic egress cost, reverse proxy
| normal traffic to Cloudflare and reply with 2GB files for the
| bot, they annoy you/cost you money, make them pay.
| tgsovlerkhgsel wrote:
| Isn't ingress free at AWS? You'd have to find a way to generate
| absurd amounts of _egress_ traffic - absurd enough to be
| noticed compared to billions of HTTP requests. 2B requests at 1
| KB /request is 2 TB/month so they're likely paying a double-
| digit dollar amount just for the traffic they're sending to you
| (wtf - where does that money come from?).
|
| But since AWS considers this fine, I'd absolutely take the
| "redirecting the entirety of the traffic to aws abuse report
| page" approach. If they consider it abuse - great, they can go
| turn it off then. The bot could behave differently but at least
| curl won't add a referer header or similar when it is
| redirected, so the obvious target would be their instance
| hosting the bot, not you.
|
| Actually, I would find the biggest file I can that is hosted by
| Amazon itself (not another AWS customer) and redirect them to
| it. I bet they're hosting linux images somewhere. Besides being
| more annoying (and thus hopefully attention-getting) for
| Amazon, it should keep the bot busy for longer, reducing the
| amount of traffic hitting you.
|
| If the bot doesn't eat files over a certain size, try to find
| something smaller or something that doesn't report the size in
| response to a HEAD request.
| ndriscoll wrote:
| If it's making outbound requests it might be going through a
| NAT gateway, in which case response traffic will be
| expensive.
| hylaride wrote:
| I'd be surprised to see a mass-scraping bot behind a NAT
| gateway. They're probably using public lambdas where they
| can't even control the egress IPs (unless something has
| changed in the last 6 months since I last looked) and
| sending results to a queue or bucket somewhere.
|
| What I'd do is block the AWS AP range at the edge (unless
| there's something else there that needs access to your
| site) - you can get regularly updated JSON formatted lists
| around the internet, or have something match its
| fingerprint to send it heaps of garbage, like the zip-bombs
| others have suggested. It could be a recursive "you're
| abusing my site - go away" or what-have-you. You could also
| do some-kind of grey-listing, where you limit the speed to
| a crawl so that each connection just consumes crawler
| resources and gets little content. If they are tracking
| this, they'll see the performance issues and maybe adjust.
| 2000swebgeek wrote:
| block the IPs or setup an WAF on AWS if you cannot be on
| Cloudflare.
| re-thc wrote:
| AWS WAF isn't free. Definitely cheaper but all the hits still
| cost.
| shishcat wrote:
| if it follows redirect, redirct him to a 10gb gzip bomb
| nake89 wrote:
| I was just going to post the same thing. Happy somebody else
| thought of the same thing :D
| sixtyj wrote:
| You nasty ones ;)
| cantor_S_drug wrote:
| https://zadzmo.org/code/nepenthes/
|
| This is a tarpit intended to catch web crawlers. Specifically,
| it targets crawlers that scrape data for LLMs - but really,
| like the plants it is named after, it'll eat just about
| anything that finds it's way inside.
|
| It works by generating an endless sequences of pages, each of
| which with dozens of links, that simply go back into a the
| tarpit. Pages are randomly generated, but in a deterministic
| way, causing them to appear to be flat files that never change.
| Intentional delay is added to prevent crawlers from bogging
| down your server, in addition to wasting their time. Lastly,
| Markov-babble is added to the pages, to give the crawlers
| something to scrape up and train their LLMs on, hopefully
| accelerating model collapse.
|
| https://news.ycombinator.com/item?id=42725147
|
| Is this a good solution??
| iberator wrote:
| Best tarpit ever.
| brunkerhart wrote:
| Write to aws abuse team
| lgats wrote:
| "[AWS has] engaged with our customer, and based on this
| engagement have determined that the reported activity does not
| require further action from AWS at this time."
| swiftcoder wrote:
| Making the obviously-abusive bot prohibitively expensive is one
| way to go, if you control the terminating server.
|
| gzip bomb is good if the bot happens to be vulnerable, but even
| just slowing down their connection rate is often sufficient -
| waiting just 10 seconds before responding with your 404 is going
| to consume ~7,000 ports on their box, which should be enough to
| crash most linux processes (nginx + mod-http-echo is a really
| easy way to set this up)
| Orochikaku wrote:
| Thinking along the same lines a PoW check like like anubis[1]
| may work for OP as well.
|
| [1] https://github.com/TecharoHQ/anubis
| hshdhdhehd wrote:
| Avoid if you dont have to. It is not really good traffic
| friendly. Especially if current blocking works.
| CaptainOfCoit wrote:
| > Especially if current blocking works.
|
| The submission and the context is when current blocking
| doesn't work...
| lagosfractal42 wrote:
| This kind of reasoning assumes the bot continues to be non-
| stealthy
| swiftcoder wrote:
| I mean, forcing them to spend engineering effort the make
| their bot stealthy (or to be able to maintains 10's of
| thousands of open ports), is still driving up their costs, so
| I'd count it as a win. The OP doesn't say _why_ the bot is
| hitting their endpoints, but I doubt the bot is a profit
| centre for the operator.
| lagosfractal42 wrote:
| You risk flagging real users as bots, which drives down
| your profits and reputation
| swiftcoder wrote:
| In this case I don't think they do - unless the
| legitimate users are _also_ hitting your site at 700 RPS
| (in which case, the added load from the bot is going to
| be negligible)
| heavyset_go wrote:
| If going stealth means not blatantly DDoS'ing the OP then
| that's a better outcome than what's currently happening
| somat wrote:
| xkcd 810 comes to mind. https://xkcd.com/810/
|
| "what if we make the bots go stealthy and indistinguishable
| from actual human requests?"
|
| "Mission Accomplished"
| HPsquared wrote:
| This has pretty much happened now in the internet at large,
| and it's kinda sad.
| lucastech wrote:
| Yeah, there are some botnets I've been seeing that are much
| more stealthy, using 900-3000 IP's with rotating user agents
| to send enormous amounts of traffic.
|
| I've resorted to blocking entire AS routes to prevent it
| (fortunately I am mostly hosting US sites with US only
| residential audiences). I'm not sure who's behind it, but one
| of the later data centers is oxylabs, so they're probably
| involved somehow.
|
| https://wxp.io/blog/the-bots-that-keep-on-giving
| mkj wrote:
| AWS customers have to pay for outbound traffic. Is there a way
| to get them to send you (or cloudflare) huge volumes of
| traffic?
| horseradish7k wrote:
| yeah, could use a free worker
| _pdp_ wrote:
| A KB zip file can expand to giga / petabytes through
| recursive nesting - though it depends on their
| implementation.
| sim7c00 wrote:
| thats traffic in the other direction
| swiftcoder wrote:
| The main joy of a zip bomb is that it doesn't consume
| much bandwidth - the transferred compressed file is
| relatively small, and it only becomes huge when the
| client tries to decompress it in memory afterwards
| crazygringo wrote:
| It's still going in the wrong direction.
| dns_snek wrote:
| It doesn't matter either way. OP was thinking about ways
| to consume someone's bandwidth. A zip bomb doesn't
| consume bandwidth, it consumes computing resources of its
| recipient when they try to unpack it.
| crazygringo wrote:
| I know. I was pointing out that it doesn't matter what it
| consumes if it's going the wrong way to begin with.
| gildas wrote:
| Great idea, some people have already implemented it for the
| same type of need, it would seem (see the list of user agents
| in the source code). Implementation seems simple.
|
| https://github.com/0x48piraj/gz-bomb/blob/master/gz-bomb-ser...
| CWuestefeld wrote:
| We've been a similar situation. One thing we considered doing
| is to give them bad data.
|
| It was pretty clear in our case that they were scraping our
| site to get our pricing data. Our master catalog had several
| million SKUs, priced dynamically based on availability,
| customer contracts, and other factors. And we tried to add some
| value to the product pages, with relevant recommendations for
| cross-sells, alternate choices, etc. This was pretty compute-
| intensive, and the volume of the scraping could amount to a DoS
| at times. Like, they could bury us in bursts of requests so
| quickly that our infrastructure couldn't spin up new virtual
| servers, and once we were buried, it was difficult to dig back
| out from under the load. We learned a lot during this period,
| including some very counterintuitive stuff about how some
| approaches to queuing and prioritizing that appeared sounded
| great on paper, actually could have unintended effects that
| made such situations worse.
|
| One strategy we talked about was that, rather than blocking the
| bad guys, we'd tag the incoming traffic. We couldn't do this
| perfect accuracy, but the inaccuracy was such that we could at
| least ensure that it wasn't affecting real customers (because
| we could always know when it was a real, logged-in user). We
| realized that we could at least cache the data in the
| borderline cases so we wouldn't have to recalculate (it was a
| particularly stupid bot that was attacking us, re-requesting
| the same stuff many times over); from that it was a small step
| to see that we could at the same time add a random fudge factor
| into any numbers, hoping to get to a state where the data did
| our attacker more harm than good.
|
| We wound up doing what the OP is now doing, working with
| CloudFlare to identify and mitigate "attacks" as rapidly as
| possible. But there's no doubt that it cost us a LOT, in terms
| of developer time, payments to CF, and customer
| dissatisfaction.
|
| By the way, this was all the more frustrating because we had
| circumstantial evidence that the attacker was a service
| contracted by one of our competitors. And if they'd come
| straight to us to talk about it, we'd have been much happier
| (and I think they would have been as well) to offer an API
| through which they could get the catalog data easily and in a
| way where we don't have to spend all the compute on the value-
| added stuff we were doing for humans. But of course they'd
| never come to us, or even admit it if asked, so we were stuck.
| And while this was going, there was also a case in the courts
| that was discussed many times here on HN. It was a question
| about blocking access to public sites, and the consensus here
| was something like "if you're going to have a site on the web,
| then it's up to you to ensure that you can support any
| requests, and if you can't find a way to withstand DoS-level
| traffic, it's your own fault for having a bad design". So it's
| interesting today to see that attitudes have changed.
| gwbas1c wrote:
| > rather than blocking the bad guys, we'd tag the incoming
| traffic
|
| > had circumstantial evidence that the attacker was a service
| contracted by one of our competitors
|
| > we'd have been much happier ... to offer an API through
| which they could get the catalog data easily
|
| Why not feed them bad data?
| snvzz wrote:
| Null-route the entirety of AWS ip space.
| JCM9 wrote:
| Have ChatGPT write you a sternly worded cease and desist letter
| and send it to Amazon legal via registered mail.
|
| AWS has become rather large and bloated and does stupid things
| sometimes, but they do still respond when you get their lawyers
| involved.
| molszanski wrote:
| Maybe add this IP to a blacklist? https://iplists.firehol.org/ It
| would be easier to pressure AWS when it is there
| reisse wrote:
| What kind of content do you serve? 700 RPS is not a big number at
| all, for sure not enough to qualify as a DoS. I'm not surprised
| AWS did not take any action.
| marginalia_nu wrote:
| FWIW, a HN hug of death, which fairly regularly knocks sites
| offline tends to peak at a few dozen RP.
| reisse wrote:
| On the other hand, I've only seen complaint letters from AWS
| for doing tens of thousands of RPS on rate-limited endpoints
| for multiple days. Even then, AWS wasn't the initiator of
| inquiry (it was their customer being polled), and it wasn't a
| "cease and desist" kind of letter, it was "please explain
| what you're doing and prove you're not violating our ToS".
| hsbauauvhabzb wrote:
| Why would aws care if you're consuming one of their
| customers resources when the customer is the one that pays?
| Hizonner wrote:
| > 700 RPS is not a big number at all, for sure not enough to
| qualify as a DoS.
|
| That depends on what's serving the requests. And if you're
| _making_ the requests, it is _your job_ to know that
| beforehand.
| pingoo101010 wrote:
| Take a look at https://github.com/pingooio/pingoo
|
| It's a reverse-proxy / load balancer with built-in firewall and
| automatic HTTPS. You will be able to easily block the annoying
| bots with rules (https://pingoo.io/docs/rules)
| neya wrote:
| I had this issue on one of my personal sites. It was a blog I
| used to write maybe 7-8 years ago. All of a sudden, I see insane
| traffic spikes in analytics. I thought some article went viral,
| but realized it was too robotic to be true. And so I narrowed it
| down to some developer trying to test their bot/crawler on my
| site. I tried asking nicely, several times, over several months.
|
| I was so pissed off that I setup a redirect rule for it to send
| them over to random porn sites. That actually stopped it.
| sim7c00 wrote:
| this is the best approach honestly. redirect them to some place
| that undermines their efforts. either back to themselves, their
| own provider, or nasty crap that no one want to find in their
| crawler logs.
| throwaway422432 wrote:
| Goatse?
|
| Wouldn't recommend Googling it. You either know or just take
| a guess.
| Rendello wrote:
| I googled a lot of shock sites after seeing them referenced
| and not knowing what they were. Luckily Google and
| Wikipedia tended to shield my innocent eyes while
| explaining what I _should_ be seeing.
|
| The first goatse I _actually_ saw was in ASCII form,
| funnily enough.
| antonymoose wrote:
| I use the ASCII form to reply to spammers, since it will
| not trip up on an attachment filter or anything most
| usually. I get mixed results from them, but the results
| are usually funny.
| nosrepa wrote:
| The Jason Scott method.
| specialist wrote:
| Maybe someone will publish a "nastylist" for redirecting
| bots.
|
| Decades later, I'm still traumatized by goatse, so it'll have
| to be someone with more fortitude than me.
| znpy wrote:
| > I've tried 30X redirects (which it follows) to no avail
|
| Make it follow redirects to some kind of illegal website. Be
| creative, I guess.
|
| The reasoning being that if you can get AWS to trigger security
| measures on their side, maybe AWS will shut down their whole
| account.
| _pdp_ wrote:
| As others have suggested you can try to fight back depending on
| the capabilities of your infrastructure. All crawlers will have
| some kind of queuing system. If you manage to cause for the
| queues to fill up then the crawler wont be able to send as many
| requests. For example, you can allow the crawler to open the
| socket but you only send the data very slowly causing the queues
| to get filled quickly with busy workers.
|
| Depending on how the crawler is designed this may or may not
| work. If they are using SQS with Lambda then that will obviously
| not work but it will fire back nevertheless because the
| serverless functions will be running for longer (5 - 15 minutes).
|
| Another technique that comes to mind is to try to force the
| client to upgrade the connection (i.e. websocket). See what will
| happen. Mostly it will fail but even if it gets stalled for 30
| seconds that is a win.
| stevoski wrote:
| > Thankfully, CloudFlare is able to handle the traffic with a
| simple WAF rule and 444 response to reduce the outbound traffic.
|
| This is from your own post, and is almost the best answer I know
| of.
|
| I recommending you configure a Cloudflare WAF rule to block the
| bot - and then move on with your life.
|
| Simply block the bot and move on with your life.
| burnte wrote:
| > The traffic is hitting numbers that require me to re-
| negotiate my contract with CloudFlare and is otherwise a
| nuisance when reviewing analytics/logs.
|
| It's having negative financial repercussions now. It's not
| ignorable anymore.
| hamburgererror wrote:
| There might be some ideas to dig here:
| https://news.ycombinator.com/item?id=41923635
| theginger wrote:
| If it follows the redirect I would redirect it to random binary
| files hosted by Amazon, then see if it continues to not require
| any further action
| nurettin wrote:
| What kind of website is this that makes it so lucrative to run so
| many requests?
| sim7c00 wrote:
| if they have some service up on the machines the bot connect from
| then u can redirect them to themselves.
|
| otherwise, maybe redirect to aws customer portal or something -_-
| maybe they will stop it if it hit themselves...
| hyperknot wrote:
| Use a simple block rule, not a WAF rule, those are free.
| ahazred8ta wrote:
| Silly suggestion: feed them bogus DNS info. See if you can figure
| out where their DNS requests are coming from.
| lgats wrote:
| they're using google dns, unfortunately.
| locusm wrote:
| I am dealing with a similar situation and kinda screwed up as I
| managed to get Google Ads suspended due to blocking Singapore. I
| see a mix of traffic from AWS, Tencent and Huawei cloud at the
| moment. Currently Im just scanning server logs and blocking ip
| ranges.
| crazygringo wrote:
| > _I managed to get Google Ads suspended due to blocking
| Singapore_
|
| How did that happen, why? I feel like a lot of people here
| would not want to make the same mistake, so details would be
| very welcome.
|
| As long as pages weren't being served and so there was never
| any case of requesting ads but never showing them, I don't
| understand why Ads would care?
| bcwhite wrote:
| I redirect such traffic to a subdomain with an IP address that
| isn't assigned (or legally assignable). The bots just wait for a
| response to connection requests but never gets them. This seems
| to typically cost 10s waiting. The traffic doesn't come to my
| servers and it doesn't risk legitimate users who might hit it by
| mistake.
| lgats wrote:
| I've attempted a few of these, redirecting to invalid domains
| or
| https://en.wikipedia.org/wiki/Black_hole_(networking)#:~:tex...
| bcwhite wrote:
| An idea I had was a custom kernel that replied ACK (or SYN+ACK)
| to every TCP packet. All connections would appear to stay open
| forever, eating all incoming traffic, and never replying, all
| while using zero resources of the device. Bots might wait minutes
| (or even forever) per connection.
| fabioyy wrote:
| no need to mess with the kernel, block on the local machine
| firewall outgoing RST packet ,create a program that reads raw
| socket for incoming SYN and answer the syn/ack). but anyway,
| this technique will not differentiate legitimate connections.
| xena wrote:
| Main author of Anubis here. Have CloudFlare return a HTTP 200
| response instead of a rejection at non-200. That makes the bots
| stop hammering until they get a 200 response.
| andrewmcwatters wrote:
| I've also gotten good results just dropping the connection if
| it hits the application layer, and you can't get CloudFlare to
| return the desired behavior first.
|
| Not ideal, but it seems to work against primitive bots.
| Ameo wrote:
| I thought you quit the orange site for good
| Rothnargoth wrote:
| Blocking before the traffic reaches the application servers (what
| you're doing) is the most effective and cost/time efficient.
|
| It sounds like the bot operator is spending enough on AWS to
| withstand the current level of abuse reports.
|
| If you really wanted to retaliate, you could try getting a
| warrant to force AWS to disclose the owners of that AWS instance.
| jedberg wrote:
| Tell cloudflare it's abusive, and they will block it outside your
| account so it doesn't count against you.
| reconnecting wrote:
| tirreno(1) guy here.
|
| I'd suggest taking a look into patterns and IP rotation (if any)
| and perhaps blocking IP CIDR at the web server level, if the
| range is short.
|
| Why simple _deny from 12.123.0.0 /16_ (Apache) is not working for
| you?
|
| 1. https://github.com/tirrenotechnologies/tirreno
| AdamJacobMuller wrote:
| > I've tried 30X redirects (which it follows)
|
| 301 response to a selection of very large files hosted by
| companies you don't like.
|
| When their AWS instances start downloading 70000 windows ISOs in
| parallel, they might notice.
|
| Hard to do with cloudflare but you can also tar pit them. Accept
| the request and send a response, one character at a time (make
| sure you uncork and flush buffers/etc), with a 30 second delay
| between characters.
|
| 700 requests/second with say 10Kb headers/response. Sure is a
| shame your server is so slow.
| notatoad wrote:
| >301 response to a selection of very large files hosted by
| companies you don't like.
|
| i suggest amazon
| lgats wrote:
| unfortunately, it seems AWS even has firewalls that will
| quickly start failing these requests after a few thousand,
| then they're back up to their high-concurrency rate
| gruez wrote:
| >When their AWS instances start downloading 70000 windows ISOs
| in parallel, they might notice.
|
| Inbound traffic is free for AWS
| jacquesm wrote:
| It's free, but it's not infinite.
| gitgud wrote:
| > _Accept the request and send a response, one character at a
| time_
|
| Sounds like the opposite of the [1] Slow Loris DDOS attack.
| Instead of attacking with slow connections, you're defending
| with slow connections
|
| [1] https://www.cloudflare.com/en-au/learning/ddos/ddos-
| attack-t...
| n_u wrote:
| Dumb question but just cuz I didn't see it mentioned have you
| tried using a Disallow: / in your robots.txt? Or Crawl-delay: 10?
| That would be the first thing I would try.
|
| Sometimes these crawlers are just poorly written not malicious.
| Sometimes it's both.
|
| I would try a zip bomb next. I know there's one that is 10 MB
| over the network and unzips to ~200TB.
| pknerd wrote:
| It's for crawlers not custom scrapers
| n_u wrote:
| Respecting robots.txt is a convention not enforced by
| anything so yes the bot is certainly free to ignore it.
|
| But I'm not sure I understand your distinction. A scraper is
| a crawler regardless of whether it is "custom"or an off the
| shelf solution.
|
| The author also said the bot identifed itself as a crawler
|
| > Mozilla/5.0 (compatible; crawler)
| pknerd wrote:
| Redirect it to Trump's website. He will take care of it
| g-mork wrote:
| CloudFlare page rule or similar to a custom internal URL with the
| max request timeout jacked up as high as possible (or whatever)
| set, stick a little async web server behind it that hangs every
| request after the first byte for say.. 1 hour. Give the aync web
| server a good chunk of RAM to waste. Most providers don't bill
| for time, only bytes, and most bots have some timeout tolerance,
| especially when the status headers and body are already being
| sent
|
| Similarly, you can also try delivering one byte every 10 seconds
| or 30 seconds or whatever keeps the client on the other end
| hanging around for without hitting an internal timeout.
| for char in itertools.repeat(b"FUCKOFF"): await
| resp.send(char) await resp.flush() await
| asyncio.sleep(10) # etc
|
| In the SMTP years we called this tarpitting IIRC
| yabones wrote:
| Return a 200 with the EICAR test string in the body. Nothing like
| some data poisoning for some vindictive fun
|
| https://en.wikipedia.org/wiki/EICAR_test_file
| tetha wrote:
| Heh, I was wondering if you could do something like SSRF
| exploits, just the other way around. You know, redirect the bot
| to <cloud-provider-metadata-api>/shutdown.
|
| Even funnier, include the EICAR test string in the redirect ot
| the cloud provider metadata. Maybe we could trip some automated
| compromise detection.
| kachapopopow wrote:
| redirect it to the client ip, not abuse since you're just an
| innocent redirect to client-ip service and the (most probable)
| timeout should consider the service dead after a couple of days
| or even better they just overload their own servers if there is a
| page on the client ip or even better is that it causes automatic
| abuse trigger to kick in and shut down the service.
| lgats wrote:
| I've tried sending a redirect to http://localhost or
| http://127.0.0.1 to no avail
| eek2121 wrote:
| That isn't the address you should be using. Use whatever
| public addresses they are hitting you from.
| scrps wrote:
| Singapore's comms regulator bans porn (even possessing it), serve
| up some softcore to the bot, e-mail the regulator and AWS.
| CaptainOfCoit wrote:
| To be honest, I'd give that a try too. When someone is
| bothering you across the internet, the best way to reply is to
| use their local law system against them, not many other parties
| will care otherwise.
| 2OEH8eoCRo0 wrote:
| IANAL- sue them for DDoSing and disrupting your service.
|
| > The traffic is hitting numbers that require me to re-negotiate
| my contract with CloudFlare and is otherwise a nuisance when
| reviewing analytics/logs.
|
| So you're able to show financial hardship
| Bender wrote:
| _' Mozilla/5.0 (compatible; crawler)'_
|
| Assuming one trusts the user-agent _in this case_ one could
| reduce the traffic reply to them and avoid touching the disk or
| any applications in Nginx with something like:
| if ($http_user_agent ~ (crawler|some-other-bot) ) { return 200
| '\n\n\n\nBot quota exceeded, check back in 2150 years.\n\n\n\n';
| }
|
| There are other variables to look for to see if something is a
| bot but such things should be very well tested.
| _$http_accept_language, $http_sec_fetch_mode, etc..._
|
| I don't use CF but maybe they have a way to block the entire ASN
| for AWS on your account assuming one does not need inbound
| connections from them. I just blackhole their CIDR blocks [1] but
| that won't help someone using a CDN.
|
| [1] - https://ip-ranges.amazonaws.com/ip-ranges.json
| lloydatkinson wrote:
| I blocked the entirety of Singapore via Cloudflare for my
| personal site. I was seeing persistent _weird_ traffic patterns
| and sometimes very odd if a little creepy. Not anymore though,
| the whole country is blocked.
| geraldcombs wrote:
| I ran into a similar situation a couple of years ago. It wasn't
| at the scale you describe, but it was an absurd number of
| requests for a ~80 MB software installer. I ended up redirecting
| the offending requests to a file named "please-stop.txt" that
| contained a short note explaining what was happening and asking
| them to stop. A short time later they did.
| jeroenhd wrote:
| So far I've been able to get away with just blocking the data
| centers/countries that cause problems for my servers. Singapore
| and China are common causes for trouble.
|
| As for trying to get them to stop, maybe redirect the bot to
| random IP:port combinations in a network that's less friendly to
| being scanned? I believe certain parts of DoD IP space tends to
| not look kindly upon attempts to scan them.
|
| Depending on your setup, you could try to poison the bot's DNS
| for your domain. Send them the IP address of their local police
| force maybe.
|
| My guess is that this is yet another AI scraper. There are others
| complaining about this bot online but all they seem to come up
| with is blocking the ASN in Cloudflare.
|
| If there's no technical solution, if consider consulting with a
| legal professional to see if you can get Amazon to take action.
| Lawyers are expensive, but so is a Cloudflare bill when they
| decide you need to be on the "enterprise" tier.
| 1a527dd5 wrote:
| We've seen tons of illegitimate traffic emanating from SG. So
| much so, that it is a part of the standard WAF country block
| (along with CN).
| leros wrote:
| That's interesting. I've been getting 1k requests per second
| from Meta bots from SG.They slowed down after a month of 429
| responses.
| lucastech wrote:
| Meta Ireland is just as bad, I've noticed a lot of Tencent
| from SG.
| lucastech wrote:
| I wrote about this a few weeks ago, because it really is quite
| insane.
|
| I wish AWS would curtail abuse from their networks. My hope is to
| build some tools to automate detection and reporting of this sort
| of abuse, so we can force it into AWS's court.
|
| https://wxp.io/blog/abuse-from-amazon-ip-networks-never-end
| pickle-wizard wrote:
| Do you have any legitimate traffic coming from AWS? My thought is
| to just drop all traffic from their ASN. Once they can't contact
| you for a while they'll move along and you could unblock.
| nijave wrote:
| Have you tried redirecting the bot in a loop? That should allow
| it to keep making a ton of requests and hopefully generate
| traffic they'll have to pay for.
|
| Another idea is replying with large cookies and seeing if the bot
| saves them and replies with them (once again, to eat traffic)
|
| The idea is to increase their egress to the point someone notices
| (the bill)
| jimrandomh wrote:
| In addition to whatever other mitigations you do, you should put
| a deny rule for the bot's user-agent in robots.txt, and use a
| status code of 429 (Too Many Requests), even if the bot doesn't
| respect these. This will strengthen your case if you need to
| convince a third party (AWS, or a court, or a different part of
| the company that's operating the bot) that it's abusive.
___________________________________________________________________
(page generated 2025-10-17 23:01 UTC)