[HN Gopher] Fastmail, Runbox, and Posteo under DDoS extortion at...
___________________________________________________________________
Fastmail, Runbox, and Posteo under DDoS extortion attack
Author : conjuredbytes
Score : 256 points
Date : 2021-10-23 13:10 UTC (9 hours ago)
(HTM) web link (therecord.media)
(TXT) w3m dump (therecord.media)
| Wronnay wrote:
| I don't wonder that Runbox and Posteo are vulnerable, but I would
| have thought that Fastmail is too big for such an attack...
| meribold wrote:
| Fastmail is working just fine for me. At least the web
| interface was briefly offline yesterday, though.
| eloeffler wrote:
| The corresponding posteo blog post is archived here:
| https://archive.md/ZmKAA
| [deleted]
| dubcanada wrote:
| On another note, the amount they are asking for seems really
| reasonable, like how are they making money? A DDoS attack must
| cost more then like 0.06BTC (like $3500 USD) to run all weekend?
| rgrmrts wrote:
| I'd assume they're not actually paying for these attacks and
| have access to a large botnet, which would imply no cost
| (outside of hours spent constructing the attack) to the
| adversary, right?
| jacobobryant wrote:
| They might be renting the botnet.
| stilley2 wrote:
| If they're using a botnet of compromised servers then I suspect
| the marginal cost is negligible.
| bombcar wrote:
| A DDoS can be rented for less than $100/mo depending on how
| much bandwidth you want to flood. Remember, they're compromised
| machines; you're not paying for egress bandwidth.
| jorgesborges wrote:
| Are most machines involved in attacks like this compromised?
| Are we talking generic consumer desktop machines, like my
| grandma's old XP desktop running in her basement?
| IceWreck wrote:
| > Are most machines involved in attacks like this
| compromised?
|
| Yes
|
| > Are we talking generic consumer desktop machines, like my
| grandma's old XP desktop running in her basement?
|
| Those combined with hundreds of thousands of compromised
| VPSes, etc.
| bombcar wrote:
| And don't forget the millions of IoT devices.
| harikb wrote:
| It could also be seed investment to see who is likely to pay
| up. ROI comes from second attack.
| rgrmrts wrote:
| Is the attack on its webmail/website or on their smtp servers?
| I've been wondering about this, but how does one protect smtp
| servers from distributed attacks? Let's assume smaller attackers,
| do you just need good firewalls in front of your servers to
| prevent congestion to the smtp servers? Are there off the shelf
| tools that can be configured to help here (pf maybe)? Do tools
| like fail2ban help?
|
| For context, I've picked up self-hosting again after many years
| and though I don't anticipate being a target for large attacks I
| have been curious what tools individuals have at their disposal
| or if it's a fools errand to even try.
| noaheverett wrote:
| If they're hitting the outbound SMTP servers, there's no way
| (at least that I know of) to protect the IP/servers via
| obfuscation with a service like Cloudflare etc. Email
| deliverability relies heavily on the source IP of the sending
| SMTP for reputation and is going to be viewable in the headers
| of an email. Also changing IPs isn't a small task unless you're
| sitting on a load of good reputation IPs that are pre-warmed
| up. I think a scrubbing service would be the only way to help
| or of course, having enough resources to deal with it directly
| (bandwidth, cpu, etc).
|
| Disclaimer: I run Pingly [1] an email hosting service, but
| ironically our signups are turned off at the moment due to a
| botnet that hits us with fake accounts to send spam that I'm
| working to mitigate completely.
|
| [1] https://pingly.com
| bob1029 wrote:
| Cloudflare has a product where you don't actually need any
| public IPs to host your apps. You install a daemon on each
| server and firewall off the box. It makes it virtually
| impossible for someone to get around the DDOS protection.
|
| https://developers.cloudflare.com/cloudflare-
| one/connections...
| lxchase wrote:
| I've been a server operator and was subject to DDoS attacks.
| I've also tried self hosting since I am familiar with the
| challenges. Short answer is 80% of the time you will be able to
| block a <10gbps attack (assuming that's your uplink speed) at
| the node level. Most boosters all use similar attack methods,
| and it just requires you to drop packets as upstream as
| possible (preferably on the NIC as a hardware upload). In most
| cases you can block common SOURCE ports of packets. Note, there
| may be legitimate uses for some protocols, but I doubt most
| customers use them. If the attack is >10gbps you must likely
| need a BGP scrubber, that can divert your traffic and "tank"
| any bandwidth before it comes to your line lease. If your
| uplink is saturated it doesn't matter how much your hardware
| can filter, good traffic won't be able to get through.
|
| There are theoretical cheap ways where you can assist your
| customer subject to attacks to setup a GRE tunnel to act as
| that scrubbing however it's a bit annoying. Get a VM on a host
| that has that capability already, and route your packets
| through them first. Con is you have one more failure mode and
| increased latency.
| lxchase wrote:
| To add to what I said, the general options and availability
| on the stack is something like:
|
| 1. Network level: - BGP Filtering (meaning in order to reach
| your IP range, packets must first route through another
| company that has sufficient bandwidth to receive AND filter
| out bad packets), then clean traffic will go through to your
| data center, and to your server. - GRE Tunnels, Similar to
| above, but it will not be transparent in that you will likely
| use their IPs.
|
| 2. Datacenter level: - Colocate or find a dedicated server
| that sits behind a dedicated appliance that solely exists to
| act as a filter. You will also need to ask what their
| upstream link speeds are (i.e. 40gbps to that appliance). You
| still might encounter leaks, and rely on the fact that they
| have configured correctly or are willing to apply custom
| filtering if you have an advanced attack.
|
| 3. Node/Server level: - By now, you can only filter whatever
| your line rate is (i.e. 1gbps or 10gbps most likely). There
| are various methods, but all of them require to create custom
| filters and be active in patching up leaks. You'll want to do
| it as far up in the server stack as possible. Best option is
| SmartNICs or NICs that support hardware offloads. Second best
| option is in front of IPtables. Most tutorials online talk
| about IPtables. That assumes you have Linux first, and second
| not the optimal way. Use tc (traffic control) instead, it's
| further up on the network stack.
| LinuxBender wrote:
| _> Is the attack on its webmail /website or on their smtp
| servers?_
|
| It is an attack on the httpd and smtpd daemons. imap has been
| unaffected as far as I can tell.
|
| _> how does one protect smtp servers from distributed
| attacks?_
|
| By design, MX can be as distributed and in large number as you
| can afford or as willing to spend. This can be a combination of
| load balancer virtual IP's distributing load to many MX servers
| behind it and many MX DNS records with the same or different
| priorities. This of course won't help much if the people
| attacking are paying ddos-as-a-service farms to bring on
| massive volume and packet rates that overload all your servers.
| There are DDoS scrubbing services you can pay for that will
| advertise your AS number or use GRE tunnels or VPN's to clean
| the attack data for you. These scrubbing solutions are no
| guarantee of mitigation.
|
| _> Do tools like fail2ban help?_
|
| No. That would be pointless whack-a-mole. If an individual
| person is mad at you and launching a tool from their own PC or
| a handful of VM's, then yes fail2ban will help. Blocking
| individual IP's on your MX servers under a real distributed
| DDoS attack would be futile. Scrubbing centers are about the
| only solution once the attack is big enough. Or if you had
| unlimited funds you could deploy many datacenters or point-of-
| pressence destinations and build your own scrubbing networks
| but that is very expensive.
| rgrmrts wrote:
| Thanks for your response! So using specific examples here for
| smtp, I get a 1gbps guaranteed network from Hetzner so in
| theory I'd need to distribute over 50 servers to withstand
| this attack?
|
| It's not clear to me why fail2ban wouldn't at least help, if
| the botnet is a thousand machines wouldn't I (eventually)
| have them all blocked? And therefore reduce the overall
| duration of the attack? Or is the problem that it's hard to
| differentiate between good clients and bad clients because no
| single client is sending enough traffic to be suspicious?
|
| Also, do you have any specific examples of ddos scrubbing
| services? Would like to take a look specifically at
| affordability for individuals.
| deadbunny wrote:
| You're thinking too small. DDoSaaS use hundreds of
| thousands to millions of compromised machines* to attack.
| You might only see a handful of connections from each IP.
| Whch makes it fruitless to use something like f2b.
|
| * Machines here could be anything from a lightbulb to a
| server.
| mirashii wrote:
| The problem is that fail2ban or any sort of dropping of
| packets at the host is too late, the volume has overloaded
| the link.
| LinuxBender wrote:
| In regards to fail2ban, assuming the attack was purely SMTP
| specific _which it won 't be_ even then blocking IP's would
| be futile. The DDoS-as-a-service farms have hundreds of
| thousands to millions of IP's under their control. Block
| one IP and ten more show up. One IP is not one attacker.
| Those farms have probes that can tell how effective their
| attack is. Some of them even have "proven work" that is
| reported back to the buyer to validate the effectiveness of
| their attack. You may want to research this one as it is a
| very big topic.
|
| In reality however, a large portion of the attack won't
| even show up in logs. The attack will also contain
| _depending on how much the attacker is willing to spend_
| tens of millions or hundreds of millions of packets per
| second of TCP, UDP packets on random ports, no ports,
| random protocols, random sizes, random TTL 's, random
| headers. That is a volumetric attack. Fail2ban and most
| network IDS/IDP's would not even see this attack. It would
| saturate the uplinks to the ISP before you even see
| anything. Your ISP will most likely null route you and
| encourage you to stop advertising your AS number.
|
| In reference to scrubbing centers for individuals, that is
| not a thing unless you have unlimited funds. There are some
| VPS providers that can scrub tiny attacks. Linode, Vultr,
| OVH to name a few but they can only deal with tiny attacks.
| If you want to research this for your business, my
| suggestion would be to get on the NANOG mailing lists and
| discuss it with all the network engineers to find out which
| scrubbing services are currently most effective. This is a
| moving target and a very big investment. I am not a fan of
| any of the companies that provide these services, but that
| is only based on my limited experiences with them.
| rgrmrts wrote:
| Word, thanks for all the details! Yeah, I don't have any
| actual business use-cases outside of a personal interest
| in making my dedicated server reasonably protected.
| LinuxBender wrote:
| You're welcome! For your personal MX servers, the most
| cost effective solution I know of would be to have
| multiple domains and each domain have their own MX
| records and corresponding VM/server on its own unique
| provider to isolate them. Then ensure that
| people/businesses that are important to you know to
| contact you at 2+ email addresses/domains. If someone
| attacks one of your domains or MX servers, you can safely
| ignore the attack. These separate MX servers should be on
| different server/VPS providers in the event that the
| attack causes one of them to suspend your account. Your
| imap client can poll each of the servers/domains so that
| you get your emails.
| rawoke083600 wrote:
| Would a decenterlized service architecture work better
| than an server-client arch ?
|
| Im thinking like a lambda/faas platform where the
| computing endpoint is massively distributed ?
|
| I guess there are (currently) no 'one final solution' but
| like traditional security a good solution consists of
| many many layers/rings of defences ? Can one(in theory)
| decenterlized all 7 of the osi model ?
|
| Trying hard not to use the word (crypto/blockchain) here
| .
| LinuxBender wrote:
| Email is already designed to be decentralized. You can
| have a very large number of mail servers in any number of
| clouds behind multiple MX records, Anycast IP's, Load
| balancers, etc... The scale of distribution required to
| defend against these attacks is possible, but
| prohibitively expensive. Some VPS providers do support
| Anycast so in theory you could dynamically spin up tens
| of thousands of inbound MX nodes that would at least
| spool the emails. Anycast will allow you to have any
| number of MX servers appear as one IP address. Those
| nodes in turn then need to relay that to your centralized
| servers. This brings up more scalability issues as you
| could essentially DDoS yourself in this pattern. There
| may not be enough clouds to support this idea and the
| cost would give just about any CTO sticker shock or
| medical issues.
|
| If you mean distributing the destination servers that
| people access that is basically the same pattern. One
| could in theory spin up cloud instances of the email
| httpd/smtpd servers but there are still diminishing
| returns. This also requires the clients to know how to
| route to the right clusters of servers. Not easy, but
| doable. Ultimately this is the same problem people run
| into with web servers hosted on AWS. You can spin up more
| instances but there are still network link bottlenecks
| going into each region. AWS and the like have
| partnerships with scrubbing centers. This model breaks
| down if the attacker is willing to spend more than $500.
| Most massive scale attacks cost less than $200 on the
| dark web. If the extortionists believe they can get more
| than say $1k, then they might be willing to spend enough
| to even take down some scrubbing centers. It can cost
| hundreds of millions to defend against a $200 attack.
|
| The current email RFC's would not support something like
| crypto/blockchain. At that point you are basically
| inventing a new standard and adoption may take a very
| long time unless there is a compelling business advantage
| to it in my opinion.
| toast0 wrote:
| Function as a service isn't really what I'd call
| decentralized. It's centralized at the service provider.
| But, they mix your load in with everyone elses and they
| serve the load in many places (maybe geographically
| distributed is decentralized?).
|
| But, voip.ms has points of presence all over and they
| were DDoSed effectively recently; distribution can often
| help, but it's not enough. If an attacker sends X
| gigabits of garbage at your San Jose PoP and disables it,
| your other PoPs will likely function, but if they send
| X/N gigabits at each of your N PoPs, that might be enough
| to disable all of them.
|
| Using a large provider can help a lot though. Volumetric
| DDoS is 'solved' by having large pipes and discarding
| lots of traffic. And, where possible, getting upsteams to
| discard lots of traffic before it arrives on your
| network. Large providers have large pipes and good
| relations with their carriers. Smaller providers or DIY
| doesn't.
| tgsovlerkhgsel wrote:
| I agree that you do need some form of first stage that can
| take the traffic volume (usually a third party service, but
| could also be 100+ Gbps of bandwidth you get somewhere and an
| appliance that can do IP filtering), no good way around that.
|
| But once you have that, you should be able to tell them
| "forward me all traffic except <list of IPs>" and then list
| the IPs that are sending you the most (remaining) traffic, or
| even cost (e.g. if the IP is sending little traffic, but
| performing many TLS handshakes). That's where a fail2ban like
| tool would come in, no?
|
| The benefit of this approach is that it works completely
| independently of the protocol you're running. TCP, UDP,
| doesn't matter, as long as the attacker cannot spoof IP
| addresses at scale.
| LinuxBender wrote:
| Agreed adding bandwidth can help against some attacks.
|
| _as long as the attacker cannot spoof IP addresses at
| scale_
|
| That is kindof the rub. Until a majority of tier-1 backbone
| providers implement bcp38 [1] or some derivative of it,
| spoofing from the ddos farms is trivial. There has been
| talk of implementing this for many years but very little
| action. Perhaps when DDoS attacks cost enough tax revenue
| or impact investors, perhaps there may be push for
| legislation in some countries to implement but in an ideal
| world most of the providers would work through this as one
| big team. _I just make some network engineers laugh, or
| smirk, or other_
|
| [1] -
| https://en.wikipedia.org/wiki/Ingress_filtering#Networks
| elaus wrote:
| Curious as you speak of "very little action", yet the
| Wikipedia article you linked says:
|
| > [...] contrary to general opinion about the lack of BCP
| 38 deployment, some 80% of the Internet (by various
| measures) were already applying anti-spoofing packet
| filtering
|
| I know very little about global routing, so I'm wondering
| if it's an ineffective way of filtering against these
| attacks we see today or is 80% not enough to be
| effective?
| LinuxBender wrote:
| It really isn't enough. Everyone needs to do their part
| [1] not just ISP's but also large businesses, cloud
| providers, others. And this has to occur in all regions.
| I admittedly over-simplified the problem statement.
|
| [1] - https://archive.nanog.org/sites/default/files/1_Kae
| o_Ddos_Tr... [PDF]
| tgsovlerkhgsel wrote:
| My understanding is that the attacker can generate a
| relatively small amount of IP-spoofed traffic, which is
| then used to generate a much bigger amount of amplified
| traffic from servers the attacker doesn't control (i.e.
| not IP spoofed).
|
| So you need to be able to handle:
|
| a) the reflected volumetric attack (you can block IPs and
| even protocols/ports here),
|
| b) a much smaller volume of IP-spoofed traffic, which for
| TCP services basically means you have to be handle a SYN
| flood (because if they spoof, they can't get a connection
| established to do the nastier stuff)
|
| c) make sure you don't let your countermeasures against
| a) be triggered by b) in a way that disrupts your
| legitimate customers.
|
| For a TCP based service, I'd expect the following to be
| quite effective:
|
| 1. At the very first stage (has to be able to handle the
| full volume, may be external), discard all UDP traffic,
| any TCP traffic pointing to the wrong port, and any
| traffic from blacklisted IPs.
|
| 2. At the second stage (has to be able to handle the IP-
| spoofable volume), terminate TCP connections using SYN
| cookies, and forward actual connections to the real
| servers.
|
| 3. At the real servers, any remaining incoming connection
| is coming from a non-spoofed IP. Monitor cost (TCP
| handshake, TLS handshake, authentication attempt) and
| successful accounts per IP. Block IPs that cause a high
| cost but no legitimate users, or an extreme cost and few
| legitimate users, using the stage 1 blacklist.
|
| 4. Optionally you can also aggregate information about
| successful logins and block sources that cause a high
| load at stage 2 and have few successful logins using a
| short-lived entry on the stage 1 blacklist.
|
| UDP based services would be much harder since you don't
| have such a trivial stage 2, but for TCP, stages 1+2
| should be something that you can buy.
| cotillion wrote:
| At this scale you pretty much need to apply some sort of DDoS
| scrubbing service. Your ISP might already have one they can
| route traffic through or if you have your own AS you can let a
| DDoS service announce the target prefixes.
|
| A game i run was recently hit with a 102 Ggbps CLDAP reflection
| attack. We were down for a while until our ISPs DDoS protection
| detected it after that we were mostly unaffected. If the attack
| is difficult to separate from legitimate traffic you'll still
| suffer though.
| [deleted]
| trulyrandom wrote:
| Coincidentally, the app password that I've used for Fastmail's
| CalDAV service for years suddenly started causing 403's today. I
| wonder if that's related (but can't think of how it could be)
| muttantt wrote:
| Yet another reason to host your email at your own domain which
| you can move at will.
| kiwijamo wrote:
| Our Bank "A" suffered a major DDoS attack recently. My non-
| techy partner, upset, declared were moving to Bank "B". I
| pointed out that Bank B had also been target of DDoS attacks so
| we would be moving banks only to face the exact same issues.
| Point being you may move to a different service provider
| (email, banking, whatever) to find the new one has the same
| problems anyway.
| qwertox wrote:
| Is it really worth the hassle to move to another provider/self-
| hosted server for such a temporary problem?
|
| Moving to another provider would mean to set up a new account,
| and usually the free tiers won't allow to use your own domain.
| So you would be locked to a new provider where you could face
| the same problems.
|
| Temporarily moving to a self-hosted server would be an option,
| but probably just to receive emails during this time. You'd
| have to set up your certificate, and optionally DKIM, DMARC,
| SPF or whatever is required to ensure that your sent emails
| arrive properly. I can't imagine self-hosting email being
| something which won't give you a hard time every now and then.
|
| Generally yes, I am in favor of owning your email-domain, but
| then using it with a professional provider like mailbox.org
| unless you really know enough about the topic.
|
| But in this case, where the issue is a DDoS attack, I wouldn't
| do anything, since all undelivered email will be re-sent at a
| later time.
| Aachen wrote:
| Or less hassle than completely moving over to a competitor
| (terrorists win in that case, to use counter strike
| terminology): add a backup MX record, perhaps to a small vps
| that just forwards mail to the real server with no retry
| timeout.
| throwaway984393 wrote:
| It sounds like it could work.
|
| Are there any mail admins in the comments who could propose
| an smtp daemon & configs that would store all mail for a
| domain and wait indefinitely to forward it on? I think
| there's a lot of users/businesses who would like to implement
| this.
| TacticalCoder wrote:
| We still hear about DDoS attacks like this once in a while but it
| seems it's not anywhere near as common as it used to be. What
| happened? It looks like the bad guys are really having more and
| more trouble mounting succesful DDoS: how comes? It also looks
| like, in despair, they're targetting smaller fishes. Why? Smaller
| botnets? Cloudflare and OVH and the likes just being too good at
| absorbing everything and anything you can throw at them? Simple
| firewall rules getting rid of 99% of the crap? What's the reason
| it's not as prevalent as it used to be?
| jacques_chester wrote:
| One factor: the main target was internet gambling sites. They
| banded together and collectively agreed not to pay any DDoSers.
|
| Source: _Security Engineering_ by Ross Anderson.
| __s wrote:
| Botnets aren't smaller (IoT has been quite a boon to them)
|
| & according to https://www.comparitech.com/blog/information-
| security/ddos-s... their frequency isn't declining
|
| But yes, the larger sites have gotten their shit together so
| that the cost to DDoS has gone up
|
| Also if you have a botnet you now have to ask: do you want rent
| out DDoS or do you want to mine crypto?
| [deleted]
| TacticalCoder wrote:
| How that's a good point: especially that if you DDoS while
| asking for a ransom, you take the risk that your botnets gets
| taken down. While if you "discretly" mine CPU (and/or GPU?)
| mineable cryptocurrencies, you kinda fly under the radar.
| ryanlol wrote:
| You DDoS from routers and other embedded devices which
| aren't capable of mining anything.
| short12 wrote:
| There have been many attempts to do exactly that with
| varied success.
|
| The best I could find was 250 a day using only 15,000
| hosts. Not bad considering the cost is literally zero for
| the attackers. Scale that up to half a million hosts,
| which is a tiny botnet in reality would make 8000+ a day
| or over 3 million a year based on low hanging fruit
|
| https://www.cnbc.com/2018/03/01/thousands-of-iot-devices-
| can...
| xyzzy123 wrote:
| CloudFlare, AWS, GCP.
| wongarsu wrote:
| Cloudflare I agree, but "give us $10000 or we increase your
| AWS bill by $20000 per day" sounds like a viable extortion
| scheme to me.
| __s wrote:
| https://aws.amazon.com/shield/pricing
|
| Might want to check how much it costs to increase someone's
| billing 20k a day (granted, a botnet makes it cheaper, but
| measure opportunity cost of what else that botnet could be
| doing), see also https://www.reddit.com/r/aws/comments/7z6u
| c3/comment/dutgw6u...
|
| Full disclosure: I work for Azure
| posnet wrote:
| AWS Shield (while expensive $36000 a year) does have `DDoS
| cost protection` as one of it's features. i.e. if you have
| to 10x your server fleet to outscale + outlast the ddos
| attack, then AWS will forgive the additional cost.
| bikingbismuth wrote:
| The general quality of DDoS scrubbing services has dramatically
| improved in the last 10 years. I work for a large tech company
| and Silverline has protected us from 100G+ attacks.
| Kye wrote:
| More protection at the OS and ISP level. ISPs can isolate nodes
| that become part of botnets, and operating systems increasingly
| remove the avenues malicious actors use to cause trouble.
| Microsoft's push for hardware security is justly controversial,
| but the move to TPM by default in Windows 11 is the latest in a
| long line of changes that's made it harder to take over an
| ordinary person's computer. Android has had an equivalent since
| 8, and I'm pretty sure iOS has it.
|
| Put that with the change to more people being mobile-only and
| there's fewer ways to create the botnets behind these.
| birdman914 wrote:
| You are correct on the ISP level. I am a network engineer for
| an ISP, we utilize Corero to monitor and mitigate DDoS
| attacks into our network. Since 99% of the time the DDoS is
| not targeted to us but rather the customer, I also kill the
| active IP addressing to their Modem/ONT, and configure that
| endpoint so it isn't allowed to pull an IP. Once the attack
| stops, re-config the endpoint and have it pull a new address.
| DanAtC wrote:
| That will stop overloading paths inside your network, but
| if your edge can't handle a 100 Gbps DDoS all your other
| customers still suffer.
|
| Better to have the target blackholed upstream. Can usually
| be done with a BGP community of 666 if your peers support
| it.
| kijin wrote:
| Doesn't that simply mean that the customer loses
| connectivity, just as the attacker intended, for the
| duration of the attack?
|
| From the ISP's point of view, you might have prevented an
| overload that could have affected other customers. From the
| customer's point of view, their service was denied all the
| same. Doesn't sound like anything has improved compared to
| 10-20 years ago.
| gsich wrote:
| In both cases the customer has no access. (or a very
| limited one)
| zugi wrote:
| > TPM by default in Windows 11 is the latest in a long line
| of changes that's made it harder to take over an ordinary
| person's computer
|
| Interesting, I think of TPM as being for holding keys for
| bitlocker encryption or personal certificates. Can you
| clarify how TPM makes it harder for to remotely take over a
| computer?
| will4274 wrote:
| It's what you can do with the TPM. With the TPM to hold
| keys, you can require that e.g. bootloader changes be
| signed by the vendor. It's hard for malware to convince an
| ordinary person to go into BIOS and disable vendor locked
| bootloaders. Of course, Microsoft also gets into trouble
| here, because sometimes the vendors (and Microsoft itself)
| don't put the option to disable locking in the BIOS.
| gruez wrote:
| but why does malware need to mess with the bootloader
| when you can launch DDoS attacks from userspace?
| Kye wrote:
| It's a lot easier for something like Windows Defender to
| untangle something confined to user space than something
| that can prevent the OS from protecting its files by
| taking over the boot process.
| Kye wrote:
| Bingo. I've seen multiple instances in the last year or
| so where people were advised to reboot their devices to
| make sure a newly identified and patched out malware was
| removed.
| brownbat wrote:
| Maybe ransomware.
|
| A bit speculative, but my hunch is --
|
| IOT and some other advancements still create opportunities for
| new DDoS attacks, but attackers herd. And the "X as a service"
| support infrastructure is mostly supporting ransomware right
| now, likely because its safer and more lucrative. You can walk
| away from a ransomware target, fire and forget, so you can do
| it at scale. DDoS you have to pick your victims, and monitor
| and maintain the pressure, choose how to allocate your
| resources to targets while they're investigating or waiting you
| out.
|
| Cloudflare might be part of the story, maybe that was enough of
| a headwind to stop the trolls, but for the professional
| criminals, I suspect this is about lucrative alternative
| attacks.
| mobilemidget wrote:
| Yes I have the same suspicion. It is way harder to maintain a
| certain volume of DDOS, and some people use services to keep
| them online during a DDOS. Spending your time on a single
| action, encrypting, and keeping a company hostage without
| further energy I think is more, what's the word, efficient
| for those bad actors.
| 300bps wrote:
| A lot of the old DDoS attacks rely on the ability to spoof your
| IP address. Many networks are now configured to drop packets
| exiting their network that don't have an address from their
| network.
|
| For example, in a Smurf attack the attacker finds broadcast IP
| addresses by sending an ICMP request to an address and counts
| the number of ICMP replies that come back. A broadcast IP
| address is one that sends a packet to every host on a network
| (often with 255 as the last octet like 207.103.0.255 for a
| Class C network of 207.103.0.0/24).
|
| After finding suitably large networks with an open broadcast IP
| address they then send the broadcast IP address packets with a
| spoof IP address of the victim. The attack is then multiplied
| by however many hosts are on the broadcast IP address network.
|
| DNS reflection is another type of DDoS attack that also relies
| on the ability to spoof an IP address of the victim.
| whoopdedo wrote:
| Can't spoof IP on a device behind CGNAT either, can you?
| prdonahue wrote:
| Once you get to a certain scale, you don't really worry about
| those vectors anymore.
|
| The more interesting/difficult to mitigate attacks are those
| that complete handshakes (if TCP) and make fully formed
| requests at L7 that otherwise appear legitimate.
| ryantgtg wrote:
| Wonder if it's the same folks who hit voip.ms last week. That one
| lasted a while.
| 29083011397778 wrote:
| Reading through r/VOIP, I could have sworn Voip.ms was 2 weeks
| ago - then other providers began to be hit with DDoS attacks.
|
| Frankly, it doesn't seem like such a bad strategy. With the
| amount of panic the attack on Voip.ms caused, a number of
| people were discussing switching voip providers. And switching
| targets to other providers just as they get a massive influx of
| new customers seems like a force multiplier IMO.
| Aachen wrote:
| DDoS is said to be very cheap. What if we used that to boot the
| culprits? It's obviously illegal, so just as a thought
| experiment: if those amplifiers (dns resolvers or whatever is
| popular at the moment) started experiencing issues due to their
| servers being a nuisance to others...?
|
| It still seems much easier to just blackhole IPs that are causing
| problems, like collectively (at the edge of your AS) block IPs
| that long-term host a service that is actively involved in
| facilitating DDoS attacks, but for some reason nobody is doing
| that. This could be a more direct way: see where DDoS traffic is
| coming from and... poof
| prdonahue wrote:
| Not really practical or effective to implement. The attacks
| more often than not come from botnets comprised of compromised
| consumer devices. You can't just outright drop traffic from
| residential ISPs.
|
| We didn't disclose it at the time but this 17.2M rps attack
| came from (home) Mikrotik devices that were running proxy
| services: https://blog.cloudflare.com/cloudflare-
| thwarts-17-2m-rps-ddo....
| Aachen wrote:
| Ah right, I was figuring most of those misconfigured udp
| services (dns, ntp, ...) were running on servers rather than
| regular home IPs. That does make it a little different.
|
| Still, if an ISP has had multiple abuse reports for the same
| subscriber and they're not doing anything, after some time it
| starts to become reasonable to block this IP, and in a
| further escalation, this ISP's ranges altogether until they
| clean their act up. I remember getting the Internet
| connection blocked as a teenager on an XS4ALL connection for
| being an ass on the Internet (I tried to DoS a domain
| squatter that tried to sell a domain I wanted for a thousand
| times the price with no added value). The abuse desk which I
| had to contact to unblock it took my promise to not do it
| again seriously (as did I), not sure how other ISPs handle
| this.
| julianlam wrote:
| The sooner these kids realize that DDoS extortion attacks do
| nothing except waste resources (and monetary resource to
| CloudFlare, basically), the sooner they'll give up.
|
| They hit a number of VoIP providers recently, too.
| pan69 wrote:
| Anyone knows what the cost of performing a DDoS is these days?
| As a target, if you can't do anything else, is it best to hang
| in there and wait it out due to mounting costs for the
| attacker?
| muttantt wrote:
| The latest attacks on VoIP providers resulted in a SIP/VoIP
| specific solution from Cloudflare.
|
| I expect we will see Cloudflare mail protection soon enough...
|
| Makes you wonder.
| yarcob wrote:
| I first learned that someone cracked my app from a company
| that offered to help me fight cracked downloads of my app. I
| was immediately suspicious.
| a1369209993 wrote:
| This isn't new; it's called a protection racket and it
| works especially well if you actually do have a mechanism
| to protect against the thing (though of course that's not a
| requirement if you're generating most of the problem
| yourself).
| piperswe wrote:
| Cloudflare Spectrum can already protect arbitrary TCP
| services, such as SMTP or IMAP
| prdonahue wrote:
| Stay tuned ..
|
| Today we can help quite a bit by proxying TCP traffic using
| Cloudflare Spectrum, e.g., spoofed traffic will never reach
| origin as it can't complete a handshake, we can use things
| like SYN cookies to challenge source, etc.
|
| In the future, there's a lot more we plan to do here.
| stilley2 wrote:
| Just to add a data point, I had trouble accessing fastmail early
| this morning (around 7 EDT). I haven't had any issues since then.
| sysadm1n wrote:
| > Since these DDoS attacks started we have worked with our system
| administrators and Internet Service Provider to mitigate the
| attacks
|
| How do you mitigate such an attack though? I know Cloudflare can
| stop this, but how do you create your own bespoke 'DDOS
| mitigation' tool, and what does that look like?
| 0des wrote:
| You simply need bigger pipes to ingest more traffic than the
| attack can provide. It is presumed these days that packet
| analysis in some cases can require too much power costwise
| rather than scaling up the connection to swallow it.
| bombcar wrote:
| And the "further up the chain" you can move the mitigation,
| the easier that is. Mitigating on your box requires a huge
| pipe to your box, but if your provider can mitigate at their
| border router, well those are bigger and already have huge
| traffic to and through them.
|
| And it's in _their_ interest to talk to their providers and
| mitigate even further back if possible. And some mitigations
| are relatively easy (block all DNS traffic to this subnet,
| etc).
| Koenvh wrote:
| This is one such an approach:
| https://www.concordia-h2020.eu/blog-post/setting-up-a-nation...
| capableweb wrote:
| One way is to build your software on top of
| distributed/content-addressed P2P software (not Blockchain, but
| pure P2P). The angle of attack disappears completely then.
|
| Otherwise, with centralized infrastructure, you identify the
| bad traffic and send it elsewhere, an instance that keeps
| connections open but delivers nothing and have a small amount
| of resource usage. Problem is that you still need to be able to
| handle the traffic, but at least you avoid hitting your main
| infrastructure.
| Mister_Snuggles wrote:
| > One way is to build your software on top of
| distributed/content-addressed P2P software (not Blockchain,
| but pure P2P). The angle of attack disappears completely
| then.
|
| I can see how this works for some applications, but how would
| this work for SMTP and IMAP/JMAP?
| capableweb wrote:
| You can't force centralized/federated protocols into
| distributed protocols so you wouldn't.
| a1369209993 wrote:
| > how would this work for SMTP
|
| You can actually get this semi-transparently with a TOR-
| like system. (I'm not sure TOR itself actually implements
| all of this, and also TOR is perennially underprovisioned
| for political reasons, so this mostly won't work in
| practice.)
|
| First, you need a distributed DNS mechanism to publish
| "example.com. 9999 IN MX 1 abcdefghijklmnop.onion". This is
| mostly static, so DDOS doesn't really work.
|
| You then come up with a _sequence_ of rendezvous servers,
| which we 'll number starting from 1. (I think actual TOR
| just assumes a single (or few) rendezvous server is
| sufficient, but I'd need to go dig through the code to be
| sure.)
|
| You then try (in sequence) servers
| 1,2+rand(2),4+rand(4),...,2^k+rand(2^k),... where rand(x)
| picks a random number in [0,x). (0: I'm not sure whether
| distinct rands should share lower bits; see below.) If a
| server is overloaded it just drops traffic on the floor.
|
| The destination server then checks the rendezvous servers
| in sequence until it's gotten enough successes that any
| client would have tried one of the successful rendezvous
| servers in its random sequence ([0] above affects the
| distribution here).
|
| Under heavy load, the destination server also sets a proof
| of work requirement (clientHello statisfies the standard
| hash-has-x-leading-zeros), which allows the rendezvous
| servers to drop most of the incoming traffic. Legitimate
| clients by definition are not spamming connections as fast
| as they can, so they can burn CPU to meet this requirement.
| DDOS clients can _also_ burn CPU on this, but that reduces
| the rate at which they generate traffic.
|
| The end result is volumetric attacks are spread over 2^k
| rendezvous servers, where k is dynamicly chosen such that
| they can handle the load[2], while for faux-legitimate
| attacks[1], DDOS will just push up the computational costs
| for legitimate clients without ever actually shutting down
| the target[3].
|
| This works for anything TCP-like.
|
| 1: You can DDOS anything by just behaving like (absurdly
| many) legitmate clients, eg `while true;do wget
| http\://example.com/;done`.
|
| 2: If the rendezvous and destination servers are all
| similar, then the work per server scales as the square root
| of the attack volume. Or put the other way around, the
| amount of attack traffic this setup can absorb scales as
| the square of the traffic each server can handle.
|
| 3: So for faux-legitimate attacks, the attacker's goal is
| not to overwhelm the _server_ , but to maximize the costs
| to legitimate _clients_ trying to connect; the attacker
| will generate only (roughly) as much traffic as the
| destination can handle, with as large a proof-of-work as
| possible. Assuming the destination server normally runs
| around 50% load, the total work imposed on legitimate
| clients (distributed over all of them) will be about the
| same as the attacker 's available CPU. If the destination
| server normally runs significantly below 50% load, the
| imposed work will be proportionately lower.
| baybal2 wrote:
| P2P networks are on other hand very susceptible to spam
| attack.
|
| You haven't seen Gnutella in its glory days.
| fossuser wrote:
| Urbit solves both of these issues in a clever way with how
| it handles identity on the network.
| theamk wrote:
| Doesn't urbit use the generic "pay to join" identities? I
| don't think it is a scalable solution for general
| applications.
| miked85 wrote:
| Urbit does have the added advantage that very few
| understand what it even is.
| ahmedalsudani wrote:
| Adding capacity, dropping suspect traffic.
| swozey wrote:
| Well that explains it, I literally switched from gmail to
| fastmail 2 days ago and it was going amazingly until last night
| when I couldn't load it. "Oh great, an unreliable service I just
| paid a year for" -- this makes me want to support them even more.
| Their customer support was fantastic. I didn't inquire about the
| outage, instead a dns issue I'd created.
|
| If you're a heavy gmail user you should read this story of my
| last year (out of 10+) with them and IMO migrate off as soon as
| you can. I wish I hadn't used an @gmail as long as I did.
|
| For some unknown reason, they won't tell me, G marked my google
| pay account as "possible fraud, unable to verify identity" at
| some point this year. No idea when, I realized it when CC expired
| and my 3 month old Google Fi account wouldn't pay its bill and it
| wouldn't let me enter a credit card to pay for _any_ G service.
| My G services started to drop like flies once my card expired.
| They locked me out of Google Pay and won 't let me enter a new CC
| to pay for subs/gdrive. I had to clear out the entire 50gb of my
| gdrive so that I could get EMAIL again to my now "free" gmail
| account which is hovering around its 15gb limit, that's how old
| it is. The real kicker; I can't contact support anymore because
| I'm not a "paying customer" who gets their "world class support."
|
| The fun part of this was I was completely without a cell phone
| for a week until I got onto ATT. Support (live chat) was
| completely useless and just sent me to a page to send them a copy
| of my license and a utility bill. I did that about 4 times now
| and I still can't change CCs on my Google account to pay for
| things. And they still won't contact me, won't tell me what the
| deal is or anything. It's been about 6 months.
|
| I was joking to a friend before I switched to Fi "watch this be a
| terrible idea, it breaks and I'm without a phone for a month and
| have nobody to fix it." Welp.
|
| Anyway, I tried out Protonmail months ago and was completely
| unimpressed with its search, interface, etc. I had low
| expectations of fastmail (due to proton exp) but it's great, I'm
| really impressed. It threads conversations like you'd expect and
| I could actually find things I searched for, I sold a house in
| the middle of my Proton use and it was _miserable_ keeping track
| of all of the emails /docs/etc I was getting/sending for some
| reason. I remember searching for important attachments and it
| finding 10s or hundreds of attachments that were my realtors
| signature picture of herself that showed up in every email.
|
| Sorry for the tangent, be very careful of how much you rely on G
| services because this cliche horror story you randomly see on
| twitter/reddit completely happened to me. I'll never be able to
| stop using this @gmail account because I have so, so many things
| tied to it but I'm going to try my best to undo most of that.
|
| My biggest fear now is that my G account gets completely locked
| and all of the things I use G to authenticate to will be lost.
| Undoing all that is a nightmare.
|
| Use a custom domain for your email..
| throwawaysea wrote:
| If you didn't want to depend on a big provider like AWS or
| Cloudflare, what is the approach to fending off a DDoS attack?
| What type of hardware would you need to acquire? What type of
| software? Are there guides on this type of thing?
| neom wrote:
| You could try to do it yourself with firewall rules, reverse
| proxies, things like that, but to fight large scale ddos you
| really need to be moving the traffic around using BGP, and
| you'd want to dump the traffic somewhere, so you'd need
| bandwidth to dump the traffic into, that's why companies like
| cloudflare exist, they're able to work with bgp and they have a
| lot of bandwidth to absorb the traffic on behalf of the
| customer.
| NabiDev wrote:
| Own the whole email provider, use protonmail? Ironic.
| spinax wrote:
| The name on that email is not one of the Runbox folks listed on
| their About page, so one can only guess who that actually is or
| how the email was sent; it could have been a BCC for all we
| know. https://runbox.com/about/runbox-team/
| lrae wrote:
| I don't think they'd publish some random e-mail address of a
| random person.
|
| Are we sure that isn't the address the extortion mail came
| from? Would make more sense?
| jlund-molfese wrote:
| It doesn't look like that email actually belongs to Fastmail
| staff, but if it did, having a backup on another email service
| is a good idea in case there's a DNS issue with Fastmail or
| something.
|
| It's the same idea behind a VPS provider serving their status
| page on another company's infrastructure.
| SavantIdiot wrote:
| I've been with RunBox since 2012 because of Norway's internet
| privacy policies. I support them not giving into to extortion. If
| I don't have email for a few days, or a month, big deal. I
| remember how to use a phone to pay bills.
| LeoPanthera wrote:
| This is a surprisingly insensitive comment. Email is as much a
| a part of modern life as phone calls and postal mail. Asking
| people to "just go without" for a few days may be possible for
| you, but is certainly not possible for a huge fraction of
| people.
| Aachen wrote:
| My employer uses runbox. I'm happy you're so unattached that
| you can go a month without any email (can't sign up anywhere,
| for example) but we'd miss incoming invoices, can't send
| invoices to our customers, can't deliver reports to customers
| (we do security audits so that's kinda important to be able to
| deliver using e.g. pgp), would miss requests for new
| assignments...
|
| I'm fully in favor of not paying into the extortion and not
| going for a global tap system like cloudflare, but we do need
| some solution in at most a few days. In the Netherlands there
| is for example the Nawas (it's also a pun, referring to a
| laundry cleaning thing) that scrubs malicious traffic for ISPs.
| I don't know the details of how they're being attacked, but
| with typical reflector services it's easy to remove that
| traffic based on a source port. Any large Norwegian hosting
| company (they just need an uplink bigger than the DDoS, or work
| with whoever the traffic comes from at their peering points)
| could provide that service as well. Not saying this is trivial
| but there are options other than waiting.
| exegete wrote:
| mailbox.org was also attacked
| https://twitter.com/mailbox_org/status/1451237052855595012?s...
|
| I kept having errors in sending emails and logging into the web
| interface and the mailbox.org status page indicated nothing was
| wrong. Why have the page if that's not where I'm supposed to find
| out something like this?
| omnimus wrote:
| The server was probably running it was just flooded. Probably
| requires different type of checks to catch this.
| psdmm wrote:
| ttHrk jsymt lmd@ lSlb@ Hrk@ htzzy@ bstmrr -
| https://mesael.com/25/%D8%AA%D8%AA%D8%AD%D8%B1%D9%83-%D8%AC%...
| dang wrote:
| Url changed from https://blog.runbox.com/2021/10/runbox-is-under-
| attack-by-ex..., which points to this.
___________________________________________________________________
(page generated 2021-10-23 23:01 UTC)