[HN Gopher] Tell HN: Impassable Cloudflare challenges are ruinin...
___________________________________________________________________
Tell HN: Impassable Cloudflare challenges are ruining my browsing
experience
I travel often. Sometimes I use a VPN, sometimes I don't. I use a
heavily customized Firefox config on Linux. Cloudflare challenges
have made large portions of the web unusable for me. Some recent
examples - The "unsubscribe" button in Indeed's job
notification emails leads me to an impassable Cloudflare challenge.
The "Contact Us" page is also behind an impassable Cloudflare
challenge. - While migrating a non-profit off of A2 Hosting,
their login forces me to re-enter credentials after failing a
challenge, looping endlessly. - On a particularly ironic
note, I tried to complain on the Cloudflare Forums--met with
another impassable challenge. When reachable, customer
support always says "try a mobile data connection", "switch to
Chrome", or some other variant of "too bad, so sad". Is anyone
else dealing with this mess?
Author : blakeashleyjr
Score : 445 points
Date : 2025-01-02 18:19 UTC (1 days ago)
| solardev wrote:
| You're collateral damage in the web's war against bots :(
|
| Unfortunately, I think the Cloudflare challenges are _designed_
| to filter out users similar to your profile... once you stray far
| enough from the norm, it just looks like a bot / suspicious
| traffic to them. Statistically there's not enough users like you
| (privacy-conscious Linux users on nonstandard browsers) for them
| to really care enough to do anything about it. Site owners don't
| care either since you're usually like 1-2% of users at most, and
| typically also the same ones who block ads, etc., so they don't
| mind blocking you... it's sad, but I don't think there is really
| anything you can do about it except conform. It's an ongoing arms
| race and you're caught in the middle.
| blakeashleyjr wrote:
| While you hit the nail on the head, I am still surprised that
| so many tools targeted at people like me (web hosting,
| developer tools, etc.) are protected that way.
| kauegimenes wrote:
| Its not only about protection, most web developers would use
| Cloudflare since its a free CDN and would increase the app
| load time considerably.
| chrisweekly wrote:
| increase -> decrease
| rkagerer wrote:
| Except for those encountering that dreaded captcha.
| solardev wrote:
| You can separately configure (to a large degree) the
| caching vs protection features, though.
| warkdarrior wrote:
| Because if such hosting and developer tools are not protected
| against bots, the tools end up used for phishing, spamming,
| etc.
| rad_gruchalski wrote:
| They are not targeting people like you. Bots are the target.
| If you look like a bot, how are they going to distinguish?
| KronisLV wrote:
| > If you look like a bot, how are they going to
| distinguish?
|
| Some non-existant system of attesting that I'm person X
| (possibly through an e-ID card) who has issued a client
| certificate Y (cert chain, using my e-ID cert to sign) to
| be used with my device Z (presumably with a device
| fingerprint or IP range attached to the cert). Of course,
| this would mean no privacy, but that's not that different
| from being signed in through Google as an identity
| provider, we'd just shift the mechanism to be universal
| (like client certs already are). One of the options that
| would take more coordination than will probably happen
| (though very similar to some e-signature solutions in EU,
| which we already use) but I could see using something like
| that for a variety of professional/service sites, since
| signing in with the e-ID card directly is already a thing
| on some sites here (government sites, banking sites,
| utilities sites).
| rad_gruchalski wrote:
| Okay. Do that globally. And solve the ddos problem as
| you're on it. If you add transparent tls termination,
| edge, caching, dns... maybe I'll have a look!
|
| I had a guy like that working with me. Blocked every
| possible tracker, disabled javascript, used some niche
| browser, proton mail, and then complains that google
| doesn't allow him to sign in. I get it, privacy and what
| not. But the guy was an outlier.
|
| Some random blogs, product pages aren't gov, most likely
| have no way to opt-in for gov eID (maybe they aren't
| based in the EU), and they only care that their service
| is available fast globally and that they get ddos
| protection for free (plus some other convenience
| features).
| KronisLV wrote:
| > Do that globally.
|
| We already do a simpler version of that with TLS and
| HTTPS, there are globally trusted root certs that ship
| with most OSes and browsers. It's just that we haven't
| extended the same approach to client certs and identity
| verification, instead having a bunch of walled gardens
| and governments running legacy methods of figuring out
| who someone is, as opposed to various eID mechanisms.
|
| If I trust news.ycombinator.com because I trust ISRG Root
| X1, I might similarly trust John Doe's iPhone because I
| trust the government of France's CA, as a hypothetical,
| as long as the certification chain is valid there.
|
| It's a problem that's technically solvable (say, in 20-50
| years), but won't get done because good luck getting a
| bunch of governments to collaborate on that across the
| world. It's actually a surprise that we have TLS in the
| first place.
| rad_gruchalski wrote:
| We cannot get them to agree on cookie banners and you're
| talking about something much more complicated.
|
| Hey, by the way, would you trust some Chinese or Russian
| root certificate?
|
| The question is irrelevant, frankly. Consider this:
| you're living in Germany today. You trust the German
| government. They handle all your logins using that eID.
| What if in February AfD comes to power? Do you still
| trust the German government? Governments are formed by
| people. Different people have different interests.
| KronisLV wrote:
| > We cannot get them to agree on cookie banners and
| you're talking about something much more complicated.
|
| Another good example of something that's technically
| feasible and not that complex, but was made infeasible
| due to either ignorance or malice, with all of the dark
| UI patterns and scummy behaviour.
|
| > Hey, by the way, would you trust some Chinese or
| Russian root certificate?
|
| Most people already do: https://chromium.googlesource.com
| /chromium/src/+/main/net/da...
|
| For example: CN=CFCA EV ROOT,O=China
| Financial Certification Authority,C=CN CN=GDCA
| TrustAUTH R5 ROOT,O=GUANG DONG CERTIFICATE AUTHORITY
| CO.,LTD.,C=CN CN=UCA Global G2 Root,O=UniTrust,C=CN
| CN=UCA Extended Validation Root,O=UniTrust,C=CN
| CN=vTrus ECC Root CA,O=iTrusChina Co.,Ltd.,C=CN
| CN=vTrus Root CA,O=iTrusChina Co.,Ltd.,C=CN
|
| If there'd be an issue of not wanting to support a
| certain country, then removing such a group of CAs from a
| store would be trivial for a particular service, same as
| with the above.
|
| Plus, the opposite is also viable, if for example the
| Russian govt. wanted to allow anyone to verify whether
| particular requests come from their citizens, they might
| also run their own CA akin to
| https://www.bleepingcomputer.com/news/security/russia-
| create... except that the attack vector would change from
| MitM to fake identities being issued by them as needed
| (but since the server is the one doing the verification,
| it might as well drop the CA when desired).
|
| > What if in February AfD comes to power?
|
| Revoking the eID and anything dependent on it would be
| akin to your passport being taken away.
|
| Essentially the modern day digital equivalent of getting
| your Google account banned by some bot, if you use that
| account for auth in a bunch of places.
|
| Fundamentally, that's no different from the reality that
| we already face - my regular eID could also be taken away
| if my own government felt like it, same as with my bank
| account and other assets.
|
| Client certs themselves are nothing new, same for PKI.
| It's a cool technology that _could_ but presently cannot
| solve the problem of client identity globally, because we
| just can't have nice things and order.
| rad_gruchalski wrote:
| > Revoking the eID and anything dependent on it would be
| akin to your passport being taken away.
|
| Is it? If my eID is used for logging in to my bank and
| said eID is revoked, I can no longer log in to my bank
| account. That's completely different than a locked up
| passport.
|
| > Essentially the modern day digital equivalent of
| getting your Google account banned by some bot, if you
| use that account for auth in a bunch of places.
|
| Use a custom domain, don't make your kingdom dependent on
| the gmail.com address.
|
| I don't know, for me the perfect amount of government
| oversight is "as little as possible". There's zero need
| for the government to mediate between me and my bank, or
| some random service provider on the internet.
|
| What you're describing sounds like a fun technical
| challenge assuming a perfect world. For example: who
| decides which countries' certificates should be revoked?
| Who decides who is the rogue one? Even that is stretching
| it too far. Can I simply download a browser without some
| selected certificates? If the technology is so great, why
| isn't it widely adopted today
|
| Those are all rhetorical questions. You don't have
| explain PKI to me.
| KronisLV wrote:
| > Is it?
|
| Pretty much the same failure mode, just with different
| immediacy. No more travel, no more ability to start using
| new banking services, no more proving identity for
| becoming employed, pretty much anything that needs you to
| provide valid governmental ID (ID card or passport) and
| doesn't accept alternatives.
|
| On the opposite end of that, both those services might
| accept something like a driver's license and the banking
| service might allow you to log in with their app, or a
| similar identity provider as a backup.
|
| > There's zero need for the government to mediate between
| me and my bank, or some random service provider on the
| internet.
|
| Who else should we depend upon for verifying the identity
| of someone? Because currently it's a hodgepodge,
| especially when some places treat the equivalent of an
| SSN as a secret or have other half baked mechanisms,
| whereas in actuality it's a problem that's been solved
| far better, the same way how e-signatures work here when
| a single competent authority implements them well (certs
| on the e-ID card, you choose what to sign, but there's
| both data integrity and non-repudiation, a service that
| everyone integrates with and it is basically treated as a
| commonplace utility).
|
| > What you're describing sounds like a fun technical
| challenge assuming a perfect world. ...
|
| Yeah, that's about it. Have a good one!
| bigfatkitten wrote:
| > If I trust news.ycombinator.com because I trust ISRG
| Root X1, I might similarly trust John Doe's iPhone
| because I trust the government of France's CA, as a
| hypothetical, as long as the certification chain is valid
| there.
|
| There are a whole ton of privacy problems with this. I am
| happy to demonstrate anonymously that I am not a bot, but
| a random blogger does not need to know that I am John
| Doe, a citizen of France with national ID number
| 12345678.
| gruez wrote:
| Between what you described and having to run a vaguely
| standard browser config, I'll take the latter, thanks.
| peanut-walrus wrote:
| Ok, what does the venn diagram of:
|
| 1) People who anonymize their IP, use Linux, a browser
| with noscript, etc
|
| 2) People who are OK with having a government issued
| digital id and having to use it to access the internet
|
| ...look like, in your opinion?
| Dylan16807 wrote:
| Well, proof of having an ID can be done anonymously.
| Cloudflare even worked on a system for that kind of
| thing.
| rad_gruchalski wrote:
| A non-citizen living in Germany without the German eID
| because they're not a citizen. Their country of origin
| doesn't have any of that. I guess they don't exist in
| that setup? Seems like a steep hill to climb on to solve
| some random login with captcha problem.
|
| Binding login interaction to some government issued
| id...who's entitled here.
|
| Sounds like throwing a baby out with the bathwater.
| peanut-walrus wrote:
| Yeah, this is at least being discussed now for eID.
| Getting it to a point where it is actually usable for
| everyone and trusted by everyone will not be easy though.
| But even in the best case, this would cover maybe 5-10%
| of internet users in 5 years. What do you do with the
| other 90% ?
| Hizonner wrote:
| Their problem. They are not entitled to make it other
| people's problem.
| scarface_74 wrote:
| If I have a process that works for 95% of the people, why
| should I care about outliers who use Linux behind a VPN
| on a heavily customized version of Firefox?
| Hizonner wrote:
| Because they are standards compliant and you aren't, and
| you are legally required to provide an unsubscribe
| service or whatever without undue barriers around it.
| scarface_74 wrote:
| For unsubscribe - yes.
|
| Everything else - no.
|
| But if I am using standards and they have an ad blocker
| that blocks some of the functioning of my site, am I also
| required to test my site against that?
| luckylion wrote:
| > Everything else - no.
|
| I'd include _everything_ important in the "yes" category.
| If I cannot access the customer panel to update settings
| or notify them of a bug that is affecting me because I'm
| using Firefox ("works for 95% of users"), they're just
| not keeping up their end of the contract.
|
| Remember, 95% excludes everything but chromium/webkit-
| engines.
| scarface_74 wrote:
| Every SaaS company I've worked for has had a
| compatibility matrix where we say what we support. If we
| lost customers who were running a highly customized
| Firefox on Linux, so be it.
|
| Every company decides which customers are worth going
| after.
| luckylion wrote:
| Yes, sure, but 5% includes stock firefox, zero
| modifications, zero plugins.
|
| Might still be a business decision, but it's like saying
| "we'll drop any emails that indicate a mail client other
| than apple mail/gmail/outlook".
| scarface_74 wrote:
| While not that strict, see how far you get hosting your
| own email as far as not being rejected or automatically
| classified as spam
| luckylion wrote:
| And I'd include that as well: if your server rejects
| emails because of your spam-decisions, you can't claim
| "we've never received that email". Either you don't use
| email for any legally-binding communication ever, or
| spam-filtering is a you-problem, not an everyone-else-
| problem.
|
| It's not surprising that the strongest protections always
| happen on the unsubscribe links, but not on the
| subscribe-links. That just needs to be fined out of
| existence, just like "you can order with one click, but
| you need 50 clicks and a three-hour-conversation to
| cancel".
| rad_gruchalski wrote:
| If that 5% is 90% of cost to provide the service, forget
| it. Nobody is going to do a Herculean task to support a
| niche user.
| rad_gruchalski wrote:
| > and you are legally required
|
| Where. It's global internet we communicate via.
| shadowgovt wrote:
| It'll be interesting to see what happens if someone takes
| that argument to court.
|
| One side of the argument is that Cloudflare places an
| undue burden. The other side of the argument is that
| without the CF protections, the service provider doesn't
| even have reason to believe the request is coming from a
| human being the law protects.
| olyjohn wrote:
| Maybe you should try to care about something other than
| just your bottom line. I'm sorry if this sounds mean, but
| this attitude just turns the web into a giant monoculture
| because you can't be bothered to care. It actually ends
| up hurting everybody in the long run. Look how long we
| were trapped with IE6. Amazing how people forget history
| so quickly.
| scarface_74 wrote:
| Everyone has limited resources. As a for profit company,
| the focus has to be on your bottom line. How many
| resources should a company use for some obscure corner
| case when the user can make changes?
|
| Of course accessibility is important - ie screen reader
| compatibility.
|
| A typical testing matrix in the US would be
|
| - Safari for iOS
|
| - Chrome for desktop and Android
|
| - maybe Safari for desktop or you just tell Mac users to
| use Chrome
|
| - Firefox if you have the time. But if not, no big deal.
|
| We are definitely not going to test for a highly
| customized Firefox on Linux running over a VPN.
| rad_gruchalski wrote:
| > Maybe you should try to care about something other than
| just your bottom line.
|
| You can do so when your bottom line is healthy. Otherwise
| you go out of business. That's business 101.
| SoftTalker wrote:
| By that logic, why care about accomodating anyone with a
| disability? Your site works for 95% of people, why care
| about those who need to use screen readers?
|
| And before you say "that's their choice," _you 're_ the
| one who is breaking the functionality. Nothing about
| using a VPN or linux or Firefox creates any problem for
| TCP/IP or https.
| scarface_74 wrote:
| One because it's the law and two because the disabled
| can't just make a choice and install Chrome.
|
| However, while the site creator does have to meet the
| disabled halfway, the disabled person is responsible for
| having whatever type of equipment they need to make it
| work - ie screenreaders
| spookie wrote:
| If your website is full of divs generated by JS that are
| full of aria tags that make no sense, those tools don't
| have a chance. Most websites act this way as well. Even
| Facebook used to lock people out of their messages if you
| couldn't use a mouse, at least in the last time I checked
| (infinite feed + no way to skip feed via tab -> can't
| reach right panel).
|
| Just do your job right. Not saying you should test some
| unique Firefox config but at least the default version is
| to be tested.
|
| Hell, I've seen people here indicating that they just
| tell desktop Mac users to "install Chrome". Such
| carelessness is bad for business. Web development sure
| could raise its bar.
| scarface_74 wrote:
| If you're selling a SaaS app, you care more about the
| _customer_ than you do about the _user_. The customer is
| the IT department.
|
| For the longest, Amazon Connect's - AWS hosted call
| center software - call flow builder only worked with
| Chrome.
|
| Even for B2C users, using Chrome is not a deal breaker.
| If they are okay with using shitty Electron apps, they
| will be okay with using Chrome for Mac.
| rad_gruchalski wrote:
| They solved their problem. No matter how upset you are
| about it, the rest of the matter is your problem.
| bluGill wrote:
| Though in this case it is legally their problem as
| unsubscribe links are protected by law in the US.
| luckylion wrote:
| I'm convinced that's mostly incompetence on the side of the
| companies that implement that protection.
|
| "We have a problem with bots" - "Just create a firewall rule,
| whatever"
| rustc wrote:
| What other way would you suggest to protect a free service
| from bots? Cloudflare is often the easiest to implement and
| has a generous limit on their free plan.
| luckylion wrote:
| Oh, they absolutely are, I don't disagree -- I use them
| too.
|
| But the immediate response to bots shouldn't be "make
| everyone go through a captcha". There's lots of nuance
| that you can tune to deal with your particular situation,
| but the first thing I'd do is block known bots or ASNs,
| set up a limit to trigger (bots usually don't make 1
| document request a minute), set up higher limits for
| users who (seem to) have a valid cookie indicating that
| they are logged in, set up different thresholds for
| certain countries that are more risky etc etc.
|
| What you need to protect your service depends on your
| situation, it's not a one-size-fits-all solution. E.g. I
| find that I have no automated contact form spam once I
| add a simple JS to add some data that isn't standard, but
| I'm sure that wouldn't hold up if there was enough
| incentive to try to get past it.
|
| But the OP mentioned not just free services, but e.g.
| webhosting logins. That's just sad, as is Cloudflare's
| community being behind an aggressive captcha. I'm a user,
| I'm logged in, I've posted before, I'm in good standing,
| yet when I go there, I need to solve a captcha. When I
| then go there again an hour later, guess what, another
| captcha.
|
| Either there's another reason I'm not seeing or it's just
| lazyness as in "we need to have a forum but we really
| don't want to spend any resources on it, just put up an
| aggressive captcha that'll filter out most bots and
| everyone but the determined users".
| hombre_fatal wrote:
| Fwiw, Cloudflare does do a multivariate confidence check
| which is why it has multiple tiers: no captcha, a one-
| click captcha, the annoying puzzle captcha once, the
| annoying puzzle captcha six times in a row.
|
| > I'm a user, I'm logged in, I've posted before, I'm in
| good standing, yet when I go there, I need to solve a
| captcha.
|
| Though consider the fact that taking over someone's
| account shouldn't give you (a spammer) unlimited access
| either. The spambots you see on Twitter are mostly cred-
| stuffed accounts. It's a hard problem. Existing accounts
| are more dangerous than fresh accounts.
|
| Imo, "write your own password" should be a thing of the
| past. Services should just auto-gen a password or there
| should be a way to require the OS (like a password
| manager) to generate one to avoid cred-stuffing. We're
| letting down the average person by making them come up
| with unique passwords for every service instead of just
| helping them. Though I'm way off topic.
| luckylion wrote:
| > Though consider the fact that taking over someone's
| account shouldn't give you (a spammer) unlimited access
| either.
|
| But it's not unlimited access -- it's _read_ access at
| that point. This is just when trying to access the forums
| at all, not when trying to post a message. And if they
| were worried about evildoers scraping all the data from
| their forums, they could rate-limit and then require
| captchas (their WAF settings make that trivial). But they
| don't, or the rate limiting is so generous that I've
| never hit it, and their forums are not that active, so I
| don't think that's the reason.
|
| Adding more protection to an endpoint where users send
| posts makes some sense, but for reading? On their
| dashboard you need to solve the captcha on the login-
| form. On the forums, you cannot even get to the login
| (which works via the dashboard, where you'll solve a
| captcha again) until you've solved the captcha.
|
| I use and like CF's products a lot (I'm a paying
| customer, I'm not even looking for free support on the
| forums, but their docs are lacking a lot of information
| that I'm interested in), so I don't believe in "we're
| incompetent", keeping the resource-investment low by
| filtering out bots and a chunk of users makes a lot more
| sense.
| migueldemoura wrote:
| > Fwiw, Cloudflare does do a multivariate confidence
| check which is why it has multiple tiers: no captcha, a
| one-click captcha, the annoying puzzle captcha once, the
| annoying puzzle captcha six times in a row.
|
| That's not correct, Cloudflare challenge pages /
| Turnstile will never show you a puzzle.
| solardev wrote:
| Most developers I've met were actually similarly lazy... we
| just use Chrome on Mac, and don't really want to deal with
| VPNs unless our employers force us to. The last few Firefox
| holdouts also switched after running into various
| WebGL/Canvas/etc issues. The same attitude that leads us to
| focus on "happy path" users and ignore edge cases often also
| causes us to sheeple into that same basic dev group. Long
| gone are the days where most devs custom build Linux boxen
| from scratch and compile custom kernels to our liking...
|
| Anyway, I know the "Cloudflare's monopoly gating is killing
| web openness!" meme is common online, especially on HN, but
| in real life I've never actually heard anyone else complain
| about it (either a fellow dev or a customer or a manager).
| Instead, it's been universal praise for the actual issues
| Cloudflare exists to solve (CDN, bot protection, serverless,
| etc)... they are a godsend for small businesses that
| otherwise get immediately flooded by spam requests,
| especially from China, Russia, and India.
|
| And if you think Cloudflare is bad, it was even worse before
| they became dominant, with terrible services like
| Incapsula/Imperva charging way more but providing both worse
| bot protection AND more false positives, or the really hard
| early reCAPTCHAs (that Cloudflare was largely able to
| replace, for users who DO fit within the "norm"). That, or
| you'd have to fight every random sysadmin with their own lazy
| rules, like firewall rules that blacklisted entire regional
| ISPs and took weeks or months to resolve, if they ever even
| checked their emails.
|
| As inconvenient as Cloudflare is for users who take privacy
| seriously and try to be less trackable, for the other 90% of
| us who don't care as much and easily fit into their "norm"
| model, it's much nicer than what came before. Site downtime
| and slowness are also much less common now, in no small part
| because of their easy CDN and caching.
|
| From the implementation side, I've set up a few Cloudflare
| accounts in my career, but do take the time to try to
| configure it to balance security vs accessibility for any
| given target audience. Sometimes we'd block entire countries,
| other times we'd minimize security to ensure maximum reach,
| but usually we'd customize rulesets in the middle for any
| given company & audience. I never got a complaint about it
| (our emails were still available and not blocked).
|
| This was always a direct response to some business need,
| usually spambots or DDoS attempts that fail2ban etc. couldn't
| catch well enough. For the business, it was usually a "shit,
| our website is down again, what is it this time", and the
| choice between "for free or $20 we can get it back up again
| and not have this issue anymore" or "we can spend thousands
| of dollars and weeks of labor building our own security
| solution" is pretty easy. "What about that one guy who is
| proxied behind TOR and three VPNs with a random user agent
| using a text-only browser he wrote himself?" never really
| factors into that process =/ There's just not enough users
| like that out in the wild vs the very real constant threat of
| bots and malware.
|
| It's a shitty situation that the web is like this today, and
| I wish it weren't the case, but it really is an arms race,
| and these imperfect weapons are just what most of us have
| access to...
| a_gray wrote:
| > spam requests, especially from China, Russia, and India.
|
| On my small website, bot traffic is almost entirely from
| DigitalOcean VPSs.
| udev4096 wrote:
| That is not an excuse to give in to the cloudflare's agenda
| of centralizing everything. Bad things have happened, is
| happening and will continue to happen if one entity has
| this much control over the internet traffic
| anthk wrote:
| >developer chrome, Mac
|
| Maybe in your country, but tons of countries outside of the
| US (first world) avoid Macs like the plague and just use
| Linux/Windows as building machines.
|
| But you are right on Google/Cloudflare, they are the poison
| of the web.
| EGreg wrote:
| I honestly don't see what's so hard about a bot simulating "the
| norm" within the margin of error. This cat-and-mouse game is
| just like a GAN, the end result is indistinguishable even by a
| bot.
| nullc wrote:
| Bot authors are lazy and won't until they have to.. once you
| do, you can then pretend they aren't bots and include them in
| the engagement numbers you feed prospective shareholders.
| tokioyoyo wrote:
| Agreed. From my past experiences though, a very good chunk
| of them will give up once there is a resistance. Basically,
| you want your bot protection to just be a little better
| than your competitor. Then the bot author will target them
| instead, because of the path of least resistance.
| EGreg wrote:
| Outrun the friend not the bear? Hehe
| viraptor wrote:
| It depends on the defences. It starts trivial - just make a
| http request. Then there's http version, user agent header,
| other headers, header ordering, cookies, TLS ciphers, session
| resolution, timing, behaviour for page resources, ... and so
| many other things. It takes time, even if you order headless
| chrome.
| shiomiru wrote:
| The sad part is that it's trivial to get around CF's bot
| protection if you're writing a bot (just use curl-impersonate
| and buy residential IPs), but it's pretty much impossible to
| bypass as a human if their magical black box doesn't like your
| browser and/or IP address.
| solardev wrote:
| How does it get around captchas?
| gjsman-1000 wrote:
| You pay contract workers in a third world country a tiny
| amount of money per day, to spend all day clicking boxes.
| tedivm wrote:
| If they don't think you're suspicious they don't make you
| do the captchas, and as others have mentioned you can
| always outsource it to captcha farms. There are also AI
| models which do a fairly decent amount, and since most
| captchas let you repeat attempts with new patterns you can
| have a pretty high error rate to get past them. Then
| there's the ADA, which requires accessibility- many
| captchas have an audio component as a backup and those are
| easy to interpret by models.
| gruez wrote:
| Cloudflare turnstile isn't even a captcha. The user just
| has to tick a box. Behind the scenes there's a javascript
| challenge to make sure you're vaguely a browser and not
| some script a bazillion requests per minute.
| xdfgh1112 wrote:
| It's also used for proof of work as many scrapers are
| using thousands of IPs but only a few CPUs
| michaelmior wrote:
| curl-impersonate doesn't solve CAPTCHAs, but the goal is to
| look enough like a human that Cloudflare doesn't present a
| CAPTCHA in the first place.
| derefr wrote:
| > it's pretty much impossible to bypass as a human if their
| magical black box doesn't like your browser and/or IP address
|
| There are residential-IP-backed VPN services that you can use
| just like commercial VPN services -- but they're mostly built
| on the backs of botnets, so it's ethically questionable to
| use them.
| devilbunny wrote:
| You could also use Tailscale back to your own IP if the
| goal is not having to trust public WiFi.
| michaelmior wrote:
| FWIW, StarVPN claims to have "ethically sourced" IPs. That
| is, not from botnets. Their pricing is quite a bit higher
| than many (cheapest plan is $20/month), but could be worth
| trying.
|
| https://www.starvpn.com/
| mike_d wrote:
| The "residential VPN" providers setup fake ISPs or buy
| AT&T/Verizon business circuits with large blocks of IPs
| and sell them as residential.
|
| They are easily detected if you are buying IP
| intelligence from one of the higher quality providers:
| https://app.spur.us/context?q=STARVPN_PROXY
| duckmysick wrote:
| The linked page shows a sign-in screen.
| michaelmior wrote:
| Spur access requires a free account.
| michaelmior wrote:
| That's helpful to know. I wasn't aware of this.
| makeitdouble wrote:
| To note, IP is only a part of it, and the full extent of
| what's baked into a CF score will never be explicited (for
| obvious reasons).
|
| CloudFront being way past the simple blocking of IP
| addresses, I wouldn't be surprised if a mismatch between
| your IP block and your language/cookies would be enough to
| lower your score.
| shadowgovt wrote:
| Surprisingly, it still works as intended. Yes, it won't keep
| professionals and dedicated bot-fabricators out, but that's
| like 5% of the botters out there; the rest are the bot
| equivalent of script kiddies who can't be bothered, and it
| filters them great. Meanwhile, the script kiddies have a
| process that still works on non-CF sites, so they don't need
| to improve their process.
| ghxst wrote:
| This is great for bypassing the server side bot detection but
| not the client side one, where it will attempt to verify the
| integrity of your browser environment.
| hedora wrote:
| Well yeah, if you're a legitimate user, CF will block you.
|
| It's only easy to bypass if you're scraping or doing
| nefarious stuff.
| homebrewer wrote:
| It's the same for spam email, yet most spam gets caught in
| spamassassin rules that were written 20 years ago and haven't
| seen much improvement since then. Most bad guys just don't
| bother to do anything above the bare minimum. For example, I
| see lots of email getting caught in a rule that checks for
| incorrectly formatted pseudo-Outlook mailer header, which is
| trivial to circumvent if you pay any attention to it (the
| difference is in excessive whitespace, or a slightly
| incorrect "Outlook" version, or something like that).
| somat wrote:
| see also: The surprising effectiveness of simply asking the
| spam server to try again(sometimes called graylisting). It
| shouldn't work at all, but proves to filter an awful lot of
| the worst mail noise.
|
| http://man.openbsd.org/spamd
| hedora wrote:
| We bypassed it by switching to starlink. Now my IP address is
| a too-big-to-fail CGNAT.
|
| The old IP address was a mom-and-pop CGNAT.
|
| Thanks CF, for protecting us from capitalism, I guess?
| modzu wrote:
| it is discrimination
| winrid wrote:
| I don't buy this because using Chrome is what most bots
| probably do right? Headless chrome is easy.
| kevincox wrote:
| > Site owners don't care either since you're usually like 1-2%
| of users at most, and typically also the same ones who block
| ads, etc., so they don't mind blocking you
|
| I do believe that it is true that many site owners wouldn't
| care. But I suspect that in the vast majority of cases they
| don't actually know. Cloudflare probably shows them a nice
| dashboard about all of these blocked "threats" and they don't
| know better than to question it.
| Zaheer wrote:
| I'd expect this to increase with the proliferation of AI Crawlers
| and scraping becoming easier with AI.
| kauegimenes wrote:
| Can't you have a normal firefox profile for such cases? Do you
| have any javascript filters? I bet the issue must be related to
| configs messing with the JS runtime.
| ghjfrdghibt wrote:
| The issue is scummy companies like cloudflare which are causing
| these issues. If your software is blocking legitimate users
| then your software is shit at its job. It's not the users
| fault.
| gruez wrote:
| >The issue is scummy companies like cloudflare which are
| causing these issues. If your software is blocking legitimate
| users then your software is shit at its job. It's not the
| users fault.
|
| But if you're going out of your way to look suspicious (ie.
| "I use a heavily customized Firefox config on Linux"), surely
| you'd agree at some point it goes from "your software is shit
| at its job" to "it's your fault for looking suspicious"? If
| you walk into bank wearing a balaclava and get stopped by
| security, it's not really "security is shit at its job".
| ghjfrdghibt wrote:
| Yeah we could start blaming victims.
|
| Maybe we should not be allowed to use software we want to
| use. Everyone should only be allowed to use windows and a
| chrome browser variant with no ad blocking. Cloudflare 100%
| should be allowed to arbitrarily block anyone not using
| this set up because they are suspicious.
| gruez wrote:
| >Everyone should only be allowed to use windows and a
| chrome browser variant with no ad blocking. Cloudflare
| 100% should be allowed to arbitrarily block anyone not
| using this set up because they are suspicious.
|
| Seems like a slippery slope argument, but isn't
| reflective of reality. They still allow Tor browser to
| pass, of all things.
| ghjfrdghibt wrote:
| It wasn't meant to be taken seriously, I was using it to
| show the ridiculousness of blaming a user for the
| shortcomings of cloudflare.
|
| But if you like: the arbitrarily blocked user if not at
| fault, cloudflare is at fault.
| gruez wrote:
| >I was using it to show the ridiculousness of blaming a
| user for the shortcomings of cloudflare.
|
| That doesn't advance the conversation, or show that
| cloudflare should be always as fault, as you seem to
| imply. Even if people are pro privacy/freedom, I think
| most wouldn't give the individual (as opposed to the
| security provider) unlimited leeway, as seen in the bank
| example.
| ghjfrdghibt wrote:
| No one said that the user gets unlimited leeway. And
| we're not talking about that. But you can strawman as
| much as you like.
|
| We're talking about browsing the internet; and being
| blocked from doing that because of the incompetence of a
| "security" company.
| gruez wrote:
| >No one said that the user gets unlimited leeway. And
| we're not talking about that. But you can strawman as
| much as you like.
|
| Then why are you refusing to address the fact that the OP
| seemingly went out of his way to seem suspicious, by
| using a "heavily customized Firefox config on Linux"?
| There might be legitimate reasons for doing that, and
| it's not illegal to customize your browser, but those
| factors are true for wearing balaclavas as well. Why
| don't you think cloudflare is justified in refusing
| service to the OP, given that whatever he's doing is
| probably in the top 0.1% of suspiciousness?
| ghjfrdghibt wrote:
| There you go again with your strawman.
|
| You seem to be confusing suspiciousness with using
| software set up the way you want it set up. There is no
| fact about the OP going out their way to be suspicious.
|
| The *fact* you and cloudflare seem to think using
| software set up the way the OP wants is suspicious is the
| actual issue.
|
| No idea why you seem to be obsessed with balaclavas but
| the analogy doesn't work, and is simply there to try and
| make your strawman argument look less like one.
|
| The implicit assumption of you final question is that the
| OP is in some way acting suspiciously, they are not. They
| are using their browser set up how they like, and are
| doing _NOTHING_ wrong. I 'll refer you back to my
| previous point: The *fact* you and cloudflare seem to
| think using software set up the way the OP wants is
| suspicious is the actual issue.
| gruez wrote:
| >There you go again with your strawman.
|
| Please look up what a strawman is. Disagreeing what
| counts as "suspicious" is a disagreement, not a strawman.
|
| >You seem to be confusing suspiciousness with using
| software set up the way you want it set up. There is no
| fact about the OP going out their way to be suspicious.
|
| By that logic would you say that wearing a balaclava in a
| bank isn't suspicious either, because it's "wearing
| clothes you want to wear"?
| ghjfrdghibt wrote:
| > The typical straw man argument creates the illusion of
| having refuted or defeated an opponent's proposition
| through the covert replacement of it with a different
| proposition and the subsequent refutation of that false
| argument, instead of the opponent's proposition
|
| You're not addressing the actual issue but instead
| replacing it with your twisted take, and then refuting
| it. In case you missed it you're attempting to make the
| OPs behaviour out as suspicious. And then having made it
| suspicious stating the cloudflare is well within their
| rights because the OP is acting suspiciously. Strawman.
| And your broken analogy involving a balaclava (now in a
| bank) feeds into this false narrative.
|
| The OPs use of their software set up how they like is not
| equivalent to the OP entering a bank in a balaclava.
|
| Cloudflares behaviour is the equivalent of some sort of
| secret police setting up a roadblock and refusing to let
| you pass unless you are dressed in a certain way, whilst
| also stripping you naked and grabbing all you bio data to
| boot.
| gruez wrote:
| >You're not addressing the actual issue but instead
| replacing it with your twisted take, and then refuting
| it. In case you missed it you're attempting to make the
| OPs behaviour out as suspicious. And then having made it
| suspicious stating the cloudflare is well within their
| rights because the OP is acting suspiciously. Strawman.
| And your broken analogy involving a balaclava (now in a
| bank) feeds into this false narrative.
|
| "not addressing the actual issue but instead replacing it
| with your [...] take" is literally how analogies are
| supposed to work. If you disagree with it, you have to
| point out why it's not applicable. Over the last few
| comments you have failed to do this.
|
| >The OPs use of their software set up how they like is
| not equivalent to the OP entering a bank in a balaclava.
|
| Why?
|
| >Cloudflares behaviour is the equivalent of some sort of
| secret police setting up a roadblock and refusing to let
| you pass unless you are dressed in a certain way, whilst
| also stripping you naked and grabbing all you bio data to
| boot.
|
| I can't tell whether you're not self-aware, or are
| purposely engaging in bad analogies as some sort of way
| to get back at me for my balaclava in bank analogy.
|
| Regardless, comparing browser fingerprints to "stripping
| you naked and grabbing all you bio data to boot" is a
| stretch. On a theoretical level, your device fingerprint
| isn't tied to your body, and can be changed, unlike
| actual fingerprints. You can have multiple devices.
| Devices can be shared. You can change around settings to
| yield different fingerprints. At a practical level, most
| people care far more about being "naked" and "bio data"
| than people knowing that they're using an iPhone 15 Pro
| (or whatever), which realistically is what fingerprinting
| amounts to. Moreover, "setting up a roadblock" is only
| objectionable because it's interfering with your ability
| to travel freely. If I want to go to a concert, and a
| gang sets up a road block on the interstate, my ability
| to go to the concert is being being impeded by a third
| party. However, cloudflare isn't a third party. They're
| contracted as the security vendor by the website you're
| trying to visit. Therefore, it's less like "secret police
| setting up a roadblock" and more close to a bouncer not
| letting you in because he doesn't like the look of you.
| Since the site is basically private property, private
| property owners are free to choose who to admit onto
| their premises, and the site owner has delegated that
| authority to cloudflare, cloudflare is free to reject you
| entry for whatever reason, just like a bouncer is free to
| reject you entry for whatever reason.
| ghjfrdghibt wrote:
| I see no point in engaging with you further. You
| repeatedly attempt to sidetrack the conversation by
| changing the subject and providing various strawman
| arguments and bad faith statements. You have nothing
| valid to add or offer, as proven by you most recent
| replies. You can continue your disingenuous nonsense if
| you like but your points are moot since you are simply
| addressing your own off topic analogies. I'm done feeding
| the troll. Or corporate shill?
| johnklos wrote:
| Does "But if you're going out of your way to look
| suspicious" advance the conversation?
| gruez wrote:
| It advances the conversation because it refutes the
| argument that "It's not the users fault" brought up a few
| comments ago, by using the balaclavas in bank analogy.
| ghjfrdghibt wrote:
| It refutes nothing, it attempts to place the blame for
| cloudflare incompetence at the feet of a user who has
| done nothing wrong.
| gruez wrote:
| Replied in the other comment. In short: if someone wears
| a balaclava to a bank, would you say that person also
| "has done nothing wrong"?
| ghjfrdghibt wrote:
| In short, just answering this question (which as stated
| in the other comment is not the issue), yes that person
| has t done anything wrong. There is no offence for
| wearing hats in banks.
| gruez wrote:
| >There is no offence for wearing hats in banks.
|
| But banks aren't mandated to admit you either. Just
| because it's legal, doesn't mean a private establishment
| has to let you in. When it comes to denying entry, banks
| are relatively tame. Some establishments go beyond that,
| by denying entry unless you wear formal clothing, or
| presenting proof of identity.
| ghjfrdghibt wrote:
| This is now completely off topic, well done.
| Zak wrote:
| Mobile operating systems with remote attestation (that's
| both Android and iOS) aren't far off from that with
| regard to native apps. It doesn't affect the web _yet_ ,
| but Google did propose adding an attestation mechanism to
| Chrome.
| ghjfrdghibt wrote:
| I vaguely remember this from this last year though I
| can't remember all the details. That's a scary slippery
| slope.
|
| Of course it'll be presented as a security feature,
| because users are dumb, whilst also allowing vendors to
| lock you into their ecosystem; similar to how passkeys
| are currently being push by these same companies.
| natch wrote:
| Agreed, but I think the point was that the user has a
| workaround. Use a standard browser for the like five minutes
| it might take to unsubscribe from these mailing lists, a one-
| time operation per business, done.
|
| If on the other hand unsubscribing from mailing lists is not
| the true use case and we are actually being asked to help a
| bot bypass safeguards... then Cloudflare is doing a great job
| here.
| blakeashleyjr wrote:
| What I don't understand is why you have to protect areas that
| require login so harshly?
|
| If I can log in, especially with 2-factor, you can safely assume
| I am not a bot, or you have a larger problem.
|
| If I have entered bad credentials 5+ times, okay, you can start
| backing me off or challenging me.
|
| What am I missing? Fail2ban has been around a long time.
| gjsman-1000 wrote:
| 40% of the internet's traffic now is bots, with about half of
| those being malicious. Fail2ban is decent for a very small
| DDoS, but useless for one with any substance, and also useless
| against bots scraping data or probing for weaknesses.
|
| Also remember, especially on AWS, bandwidth is expensive. A CDN
| cache + blocking bots = big savings.
| noprocrasted wrote:
| Problem is that a significant chunk of the technology industry
| _still_ relies on "engagement" as its business model. The
| objective of slapping an overzealous bot protection system
| isn't to protect high-risk endpoints like logins/etc, it's to
| ensure a _human_ is "engaging" and _human_ time is being
| wasted by making even legitimate automated usage impossible.
|
| From their perspective, the blocking of power users with
| unusual setups is actually a happy coincidence, as those are
| unlikely to "engage" with the product in the desired way (they
| run ad & spyware blockers, don't fall for dark patterns, and
| are more likely to fight back if they get defrauded by the
| corporation).
| duskwuff wrote:
| > What am I missing? Fail2ban has been around a long time.
|
| Modern threat actors can spread requests out over large pools
| of source IPs. Rate limiting login attempts by IP isn't an
| effective means of preventing credential stuffing attacks.
| Terr_ wrote:
| I'm really afraid of what kind of internet we'll have when these
| kinds of un-diagnosable un-appealable false-positives are not
| just transient blips, but become metadata companies use to
| blindly and permanently kill off accounts on other services.
|
| I think it may have been what happened my since-2010 Reddit
| account was mysteriously killed a couple years ago, and literally
| the only cause I can think of is that I might've used the wrong
| public wifi for an evening.
| _xander wrote:
| I'm experiencing the same issue which is definitely exacerbated
| by straying from a 'default' configuration e.g. using a custom
| browser screen reader, browsing from Brazil, using a VPN, using
| Firefox. I think eventually I'll be completely locked out of the
| 'mainstream' web
| Animats wrote:
| > The "unsubscribe" button in Indeed's job notification emails
| leads me to an impassable Cloudflare challenge.
|
| That's a CAN-SPAM act violation.
|
| FTC: _" Tell recipients how to opt out of receiving future
| marketing email from you. Your message must include a clear and
| conspicuous explanation of how the recipient can opt out of
| getting marketing email from you in the future. Craft the notice
| in a way that's easy for an ordinary person to recognize, read,
| and understand. Creative use of type size, color, and location
| can improve clarity. Give a return email address or another easy
| Internet-based way to allow people to communicate their choice to
| you. You may create a menu to allow a recipient to opt out of
| certain types of messages, but you must include the option to
| stop all marketing messages from you. Make sure your spam filter
| doesn't block these opt-out requests."_[1]
|
| Experian was recently fined for making it hard to opt out of
| their marketing emails.
|
| The actual regulation text:
|
| _SS 316.5 Prohibition on charging a fee or imposing other
| requirements on recipients who wish to opt out._
|
| _Neither a sender nor any person acting on behalf of a sender
| may require that any recipient pay any fee, provide any
| information other than the recipient 's electronic mail address
| and opt-out preferences, or take any other steps except sending a
| reply electronic mail message or visiting a single Internet Web
| page, in order to:_
|
| _(a) Use a return electronic mail address or other Internet-
| based mechanism, required by 15 U.S.C. 7704(a)(3), to submit a
| request not to receive future commercial electronic mail messages
| from a sender; or_
|
| _(b) Have such a request honored as required by 15 U.S.C.
| 7704(a)(3)(B) and (a)(4)._
|
| That seems to cover it. File a CAN-SPAM act complaint
| (spam@uce.gov). Send a copy to the legal department of the
| sender.
|
| [1] https://www.ftc.gov/business-guidance/resources/can-spam-
| act...
| LeifCarrotson wrote:
| "Visiting a single Internet Web page" is considerably more
| involved than that. In practice, it means making a request to
| the DNS servers and running Javascript that's injected by the
| CDN/proxy which "verifies" (runs some heuristics) that you're
| allowed to load that page.
|
| It's like a restaurant that complies with a local food access
| requirement to be open at a certain time... but only by having
| a drive-through that requires you to not just be a human being,
| but also to drive a car to get to the restaurant.
| mdaniel wrote:
| I would _suspect_ that OP is choosing the webpage out of
| convenience but that there is a List-Unsubscribe: header hiding
| in the raw version of the email, cheerfully nuking the FTC
| complaint. Now, demonstrating that the List-Unsubscribe
| _worked_ is left as an exercise to the reader, but let 's be
| honest, it's the same with the web page variant with bonus
| points for those pages usually ending it "yeah, we'll get
| around to it is 364 business days" or some shit
| salzig wrote:
| Thanks for that note. I receive ,,spam" by a US based Car
| Rentel/Leasing Company, cause they prevent me from
| unsubscribing because i am in European IP-Range (geo-blocking).
| Especially ,,nice" cause they send me contract specific details
| of one of their customers, who misspelled his email address.
| montecarl wrote:
| I'm in a similar boat. A UK bank thinks I'm one of their
| customers (someone with a similar name). The reply address is
| no-reply@ and I'm not about to call a foreign bank.
| dboreham wrote:
| Quick note that if you use a proper email hosting service,
| or host yourself, you can add a sender block rule to
| eliminate this nuisance.
| causality0 wrote:
| Are there email services which don't allow you to block
| addresses or keywords?
| luckylion wrote:
| I had the same happen with a AU insurance company that also
| made it hard to reach them.
|
| I sent an email to their regulator that this company keeps
| sending me confidential information about one of their
| clients. It took one day until I received an email from the
| company informing me that they've corrected the mistake and
| I shall no longer receive any emails, and it worked, I
| haven't received a single one since.
| happymellon wrote:
| I received spam for quite a while from Robinhood, back when
| they suggested they were going to enter the UK market and to
| sign up for more details.
|
| They didn't but I still recieved spam which I couldn't opt
| out of because they wanted me to log into my account, even
| for support, which obviously didn't exist.
|
| At least back then we had Twitter and messaging them publicly
| got a customer service response.
| johnklos wrote:
| Is this Hertz? Somehow Hertz in Mexico decided to add my
| email address to their mailing list, and I tried to complain
| to every level of Hertz to get them to stop. Their hosters
| didn't care, their upstream didn't care.
|
| I decided to download larger files from their web site a few
| tens of millions of times, which I think cost them a few
| hundred dollars. Unethical? Perhaps, but I'm not the kind of
| person who just accepts that companies are too large to have
| humans that can communicate and that I should just accept
| their harassment.
|
| It worked, though. I finally got a response from Hertz saying
| they were going to "get to the bottom of it", and I finally
| stopped getting their spam.
| driverdan wrote:
| Unfortunately the government seems to have given up on
| enforcing the CAN-SPAM act. If they actually enforced it spam
| companies like Salesforce would face massive fines.
| bluGill wrote:
| You can press charges yourself and get lawyer fees for your
| efforts. Probably not worth it, but you don't need the
| government to do this.
| ycombinatrix wrote:
| Yeah, the same way I don't _need_ to pay taxes...
| antonvs wrote:
| Don't worry, the next administration is likely to eliminate any
| rules like that.
| numpad0 wrote:
| It feels weird that a completely US based Recruit acquisition
| shows such a typical Recruit behavior...
| solatic wrote:
| Makes me wonder. If it's illegal to deploy bot protection on
| unsubscribe links, and there are massive lists of leaked email
| addresses available, surely _someone_ has tried to mass-
| unsubscribe tons of email addresses from tons of mailing lists?
| dolmen wrote:
| More likely: someone tried to automate unsubscribing one
| e-mail address from massive list of mailing lists.
| ycombinatrix wrote:
| brb unsubscribing the world from Indeed
| mg wrote:
| If it is triggered by the customizations you did in Firefox, then
| running a fresh Firefox in a container might help:
| docker run -it --rm -e DISPLAY --net=host -v
| $XAUTHORITY:/root/.Xauthority -v /tmp/.X11-unix:/tmp/.X11-unix
| debian:12-slim
|
| Then inside the container, run: apt update
| apt install firefox-esr firefox
| stonogo wrote:
| what is the advantage here over just running 'firefox
| -ProfileManager' and making a clean profile?
| theamk wrote:
| All host info not accessible via X11 protocol is hidden, for
| example font list, is replaced with generic one.
|
| For even more protection, run VNC server with common
| resolution in the container and connect to it using VNC
| viewer. In this case firefox provides a super generic profile
| (latest debian with mesa GPU), making this browser very hard
| to distinguish from others. This has some downsides however:
| First, you cannot resize window. Second, a lot of actual bots
| use same config, so it might be blocked.
| veeti wrote:
| Isn't it suspicious bot-like behavior to only have the bare
| minimum fonts installed? :-)
| ghxst wrote:
| To be fair, Firefox out of the box prevents against font
| fingerprinting more than Chrome, it's considerably easier
| to get Firefox to run in a docker container and pass all
| the client side challenges than Chrome in my experience,
| you still have a valid point though.
| homebrewer wrote:
| mullvad browser is pretty much this, but without messing
| around with containers. One fingerprint for all users, with
| the same font list, resolution, canvas behavior, etc.
|
| https://mullvad.net/browser
| theamk wrote:
| looking at https://mullvad.net/en/browser/hard-facts ,
| Mullvad browser is much more extreme: many APIs blocked,
| always incognito mode... I would not be surprised if this
| blocks some sites.
|
| the container approach on the other hand is bog-standard
| firefox.
| rmholt wrote:
| OP mentioned that they run a heavily modified browser, I
| think it means compiled with changes - docker means stock
| Firefox
| ghjfrdghibt wrote:
| The suggestion you should have to bend over backwards for
| shitty software like cloudflare is bad enough; but if you were
| going to surely creating a new browser profile is far easily
| than spinning up a debain docker image, updating it and the
| installing Firefox and the running it?
| zufallsheld wrote:
| > - The "unsubscribe" button in Indeed's job notification emails
| leads me to an impassable Cloudflare challenge.
|
| Maybe indeed could be held liable here? From the can spam act (if
| you're from the US):
|
| > You can't charge a fee, require the recipient to give you any
| personally identifying information beyond an email address, or
| make the recipient take any step other than sending a reply email
| or visiting a single page on an Internet website as a condition
| for honoring an opt-out request.
|
| https://www.ftc.gov/business-guidance/resources/can-spam-act...
| _bin_ wrote:
| this nevertheless happens all the time. i have an old linkedin
| account i haven't logged into in years and can't be bothered to
| dig up the credentials so one of my e-mails gets stupid
| "network updates". one must log in to disable these and
| navigate to some obscure settings page in one of the most
| heinously overcrowded UIs on the web.
|
| so i just flagged it all as spam and hoped it hurts their
| deliverability a little.
| ToucanLoucan wrote:
| Honestly I click an unsubscribe link but if it requires me to
| complete a survey or fill out a form, I just nix the tab and
| spam filter the email. I'm nobody's fucking admin assistant
| and my time is valuable: you know my fucking email and could
| easily add it to the think, or at the most, ask me to type it
| into a box if you MUST. Anything more than that, if I have to
| manually opt out of "types" of messages or whatever, nah.
| Fuck you.
|
| I didn't ask for your fucking emails and I sure as shit am
| not going to do the homework you're assigning me to make them
| stop.
| ryandrake wrote:
| Yep, I just spam filter the E-mails now. If that act adds
| 0.0001% to that sender having future E-mail deliverability
| problems, then all the better. If it's commercial or
| political and I didn't explicitly ask for the sender to
| E-mail, then it's spam.
| datadrivenangel wrote:
| It does! Reporting as spam will cause them to have issues
| if enough people do it.
| krior wrote:
| If the survey has text fields and I have enough spite left
| in me I fill them with "[object Object]" in the hopes it
| makes someones day more miserable than mine.
| ToucanLoucan wrote:
| _takes notes_
| kachapopopow wrote:
| I have experience bypassing these.
|
| The primary cause of this is most likely any kind of
| 'optimizations' you have in your browser (or missing
| fingerprints).
|
| If you want to 'bypass' these I recommend removing any use of
| Proxy[1] (via extensions). You should also look into disabling
| any kind of forced backgrounding. Make sure service workers are
| working.
|
| 1: They catch Proxy usage by using exceptions and analyzing the
| stacktrace. I assume you know what a javascript proxy is, but
| incase you don't: It's something that allows you to override any
| kind of object function such as navigator.hardwareConcurrecy.
| wewtyflakes wrote:
| > They catch Proxy usage by using exceptions and analyzing the
| stacktrace
|
| That is really clever, I am guessing this is why various
| browser automation companies are using custom forks of
| Chromium.
| dboreham wrote:
| I ran into this, or something similar recently when our main
| connection went down (solar powered) and we switched to Starlink.
| Due to Starlink NAT issues I had tunneled our traffic to to a box
| colocated in a data center. This broke a number of web sites in
| weird ways. Became so annoying that I ended up bringing up a
| tunnel to our office in town to get back to the regular IP we
| used. Weird problems went away.
| hulitu wrote:
| > Cloudflare challenges have made large portions of the web
| unusable for me.
|
| I guess the best web experience is when one filters Cloudfare,
| Google and Microsoft at the firewall.
| bastard_op wrote:
| I deal with this fairly commonly, presumably because I use linux,
| and we all know only botnets use linux. Occasionally with
| cloudflare I'll just get summary rejection and supposed blocking
| of my IP, but either it's summary rejection or a pass without
| challenge.
|
| Recently I had to deal with this for alibaba just to look at
| something, which I usually just use torbrowser with, and finally
| gave up as I couldn't pass the challenge. I suppose I shouldn't
| be surprised at that though, they trust me as much as I trust
| them.
|
| The worst is usually adobe and cookielaw with all their related
| tracking crap, where I can't even get the captcha to render as
| it's so many layers buried in scripting I can't enable enough
| sites between ublock, noscript, privacy badger, and firefox
| strict modes. I treat adobe like malware, but unfortunately
| things like albertsons.com for groceries and other mega companies
| love to use it, and their sites literally do not work without
| allowing their heavy scripting/tracking.
|
| There are other usually smaller captcha players that I haven't
| been human enough to pass with, I forget the names of the stupid
| to shame, but a few when I see them I recognize to just close the
| window and forget about whatever it was I was looking for there
| (like twitter/x).
|
| Hooray commerce!
| krunck wrote:
| >...when I see them I recognize to just close the window and
| forget about whatever it was I was looking for there
|
| This is the way.
| TiredOfLife wrote:
| My main desktop for the past year has been Steamdeck with
| linux. And don't get any excess Cloudflare challenges.
| choobacker wrote:
| Nice idea! How's that working out for you? Stock OS? Bazzite?
| TiredOfLife wrote:
| Stock. Browser (Chrome/Firefox) doesn't have hw
| acceleration for video decode. But other than that it's
| fine. Fast and silent. VS Code and Jetbrains tools work
| fine.
| ringer wrote:
| I visited albertsons.com out of curiosity, but I was instantly
| banned. Even using an unmodified Chromium browser, I couldn't
| access the site. It's ridiculous what's happening on the
| _internet_ today.
|
| The error: ``` Access denied Error 16 www.albertsons.com
| 2025-01-03 09:30:00 UTC What happened? This request was blocked
| by our security service Your IP: xxx Proxy IP: xxx (ID xxx)
| Incident ID: XXX Powered by Imperva ```
| bastard_op wrote:
| Their Imperva WAF usually challenges me repeatedly during use
| trying to buy groceries from my pc, and most of the time I
| get tired of having to disable every security extension I use
| with Firefox to use Albertsons because of their shitty
| website. Never outright block though.
|
| Might be worth checking some enterprise threat lists for
| whatever IP's your popping up on (ie Imperva and Cloudflare),
| or something uniquely fingerprints you from your browser. I
| use multiple extensions to block whatever they each can, and
| even I'm not treated that badly as you for wherever you are
| coming online from.
|
| Here's Fortinet's you can check your IP against, they all
| tend to roughly use the same lists eventually:
| https://www.fortiguard.com/iprep
| Kwpolska wrote:
| Immediate bans might be related to the country you're in.
| This is a US retailer, and there is zero reason for someone
| outside of the US to visit that site. Blocking foreign
| visitors allows them to ignore GDPR, for example.
| ringer wrote:
| In this case, they may display a message like: 'This page
| is intended for USA visitors only. Our services do not
| operate outside the USA.', but no, they say you are banned
| because just...
| ghjfrdghibt wrote:
| It seems that if you use Firefox with an adblocker then
| cloudflare spam is all you see. Though I have experienced this in
| plain Firefox too.
|
| Cloudflare are a scummy company trying to force you to use one
| browser and view all ads.
| robhlt wrote:
| It can't be just that. I use Firefox on Linux with ublock
| origin, strict tracking protection, and clear cookies on exit,
| and I've never ever seen a cloudflare challenge. Not even on
| sites with that "verifying your browser" page enabled.
| ghjfrdghibt wrote:
| Maybe you're right, I see it all the time. Assume cloudflare
| do other dumb stuff too then like up ranges and just being
| generally crap at their jobs.
| gruez wrote:
| >I use a heavily customized Firefox config on Linux.
|
| This is probably the cause, especially if you're doing stuff like
| spoofing user agent. It's not cloudflare "cracking down on
| privacy" or whatever either. Unmodified tor browser passes
| turnstile challenges just fine.
| jillyboel wrote:
| It's up to users to choose their user agent.
| gruez wrote:
| And it's up to site owners and website security vendors to
| choose which user agents to admit.
| jillyboel wrote:
| No, that's discrimination.
| dylan604 wrote:
| My local TV station's website refuses to allow my to view their
| page and instead presents an a modal that cannot be blocked
| accusing me of using an ad blocker. The funny thing is that only
| happens on a mobile device using the default browser with no
| extensions. When I visit the same site on my laptop with uBO, the
| site is viewable with no blocking modals.
|
| Sometimes you miss what you were aiming for I guess
| ugotjelly wrote:
| What do you mean impassable challenge...? Why isn't it passable?
| Are you a robot?
| gruez wrote:
| The challenge is a small javascript program that checks the
| execution environment is consistent with a real browser. For
| instance, if your user agent says it's chrome, but it's missing
| features that'd normally be supported by chrome, it'll fail
| you. The OP mentioned "heavily customized Firefox config", so
| he might be doing stuff like this that makes his browser look
| suspicious.
| numpad0 wrote:
| From website perspective, yes. GP is likely using extreme ad-
| blocking and/or coming from regions where tons of bots and/or
| unwanted traffic are also from. In those cases, some/many human
| users could be misidentified as bots with little incentives to
| website admins to rectify.
|
| And it's discriminatory, yes.
| viraptor wrote:
| CrimeFlare is not interested in these problems for the users. If
| you have access to the hosting side, you can adjust the bot score
| for specific connections/clients. But consumers don't matter to
| CF so apart from jumping through their hoops, there's nothing
| better you can do.
|
| Unless you accept the racket of course, start paying them and
| proxy your traffic through the CF workers
| https://github.com/pellaeon/cloudflare-worker-proxy and magically
| most barriers will disappear.
| gruez wrote:
| >Unless you accept the racket of course, start paying them and
| proxy your traffic through the CF workers
| https://github.com/pellaeon/cloudflare-worker-proxy and
| magically most barriers will disappear.
|
| Source this actually works? ie. that using cloudflare workers
| allows you to bypass cloudflare protection?
| viraptor wrote:
| https://jychp.medium.com/how-to-bypass-cloudflare-bot-
| protec... and many other posts. Haven't looked into this in a
| while, so can't tell you exactly how effective it is today.
| (Definitely corrects the high bot score of your IP though)
| gruez wrote:
| Sounds like all it does is make your IP reputation slightly
| better than tor, which is a pretty low bar to cross. You'd
| likely get the same effect from using any other VPN
| service, so it's not exactly evidence that cloudflare is
| running a "racket" with its worker product. The linked blog
| post even touts the fact it's free as an advantage. Rackets
| typically aren't free.
| viraptor wrote:
| You also change the headers / TLS signature, because it's
| their worker doing the connection. That covers quite a
| lot already.
|
| The racket is not in the workers themselves, but rather
| cloudflare both protecting from internet abuse and
| protecting sites which sell the abuse services. (For
| example hosting WebStresser) I meant that by giving them
| more traffic and accepting that as a workaround, we'd be
| saying "I'm ok with that".
| gruez wrote:
| >You also change the headers / TLS signature, because
| it's their worker doing the connection.
|
| pip install curl_cffi
|
| Even easier than spending 15 minutes setting up
| cloudflare workers.
|
| >The racket is not in the workers themselves, but rather
| cloudflare both protecting from internet abuse and
| protecting sites which sell the abuse services. (For
| example hosting WebStresser) I meant that by giving them
| more traffic and accepting that as a workaround, we'd be
| saying "I'm ok with that".
|
| Do you think it's a "racket" for gun shops to sell guns
| for home defense, but also to sell guns to criminals?
| slig wrote:
| > If you have access to the hosting side, you can adjust the
| bot score for specific connections/clients
|
| Only for Enterprise customers [1].
|
| [1]: https://developers.cloudflare.com/bots/plans/bm-
| subscription...
| magic_smoke_ee wrote:
| I can't use any of the kerbalspaceprogram.com domains because of
| improper discrimination against IPv6 clients triggered by
| CloudFlare. Error 1015 Ray ID: .... * xxxx-xx-
| xx xx:xx:xx UTC You are being rate limited What
| happened? The owner of this website
| (wiki.kerbalspaceprogram.com) has banned you temporarily from
| accessing this website.
|
| This sort of monoculture creates an Orwellian SPoF.
| TiredOfLife wrote:
| Cloudflare owns kerbalspaceprogram?
| LeifCarrotson wrote:
| No, wiki.kerbalspaceprogram.com is a customer of Cloudflare,
| but the outcome is the same.
| TiredOfLife wrote:
| Then ask them to disable Cloudflare.
| metalliqaz wrote:
| good luck with that
| freitasm wrote:
| I don't think it's an IPv6 problem. IPv6 clients are more
| static than IPv4, which is usually shared amongst many clients
| (at home) or at the network level (CGNAT).
|
| It could be the address is being reused - is it home, cloud or
| corporate? Have you tried different browsers? Incognito mode?
|
| I have an IPv6 block at home and have no problem accessing that
| site.
| duskwuff wrote:
| That isn't "triggered by Cloudflare". The operator of the web
| site has deliberately configured it to block your IP range, and
| Cloudflare is obeying those instructions.
| 015a wrote:
| I've honestly only experienced the opposite; their captcha is
| reasonably easy to bypass, and I've successfully automated access
| to a few sites "protected" by the Cloudflare captcha (behind a
| VPN, no less).
|
| > I use a heavily customized Firefox config on Linux.
|
| If you really care about privacy, you should blend in to look
| like everyone else. Avoiding being tracked raises alarm bells.
| You have to let them track something; but no one ever said it had
| to be you.
| antfie wrote:
| I found a GitHub captcha to be unsolvable. That captcha properly
| stressed me out.
| SoftTalker wrote:
| Yes, I run into it from time to time. I just move on. If someone
| is going to make their website inaccessible to me, I'm not going
| to bend over backwards to try to work around that.
|
| Incidentally, since I configured DNS over HTTPS in Firefox, using
| Cloudflare's DNS, it seems I see this much less often.
| afh1 wrote:
| Same here, but Cloudflare's captchas in particular are actually
| the easiest to pass in my experience. Google's ones are the
| killers. But yeah everything has a captcha if you're using a VPN
| or Firefox.
| UniverseHacker wrote:
| I had similar issues as an (also heavily customized) Firefox
| user, but was able to fix it by installing Cloudflare's Privacy
| Pass browser extension.
|
| It seems ironic that as a human I can't seem to reliably prove I
| am a human with a realistic amount of effort via these systems,
| but having installed a specific automated browser extension does?
|
| I am not a fan of Cloudflare and don't like the idea of running
| their software on my computer, but it seemed like the only
| options to continue using the internet at all.
| tempaccount420 wrote:
| I didn't know that extension existed, so after failing to fix
| it by reinstalling Firefox and removing extensions, I just gave
| up and installed Chrome.
| 93po wrote:
| I wish we could popularize some extension that pays a penny per
| page load or something using some shitcoin both as a means to
| support our favorite sites but also to validate that I'm not a
| bot, or at least if I am, I am willing to spend a lot of money in
| a DDOS that goes directly in your pocket
| shadowgovt wrote:
| Unfortunately, your setup makes you look like a scraper: no
| history for Cloudflare to identify, the sort of browser / OS
| config someone would use to homebrew an automated "I sure am not
| a bot, look at how authentic my user-agent is!" bot, and so on.
| If you also have JavaScript disabled and clear your cookies
| frequently, Cloudflare can't fingerprint your machine to know you
| passed a trust-check in the past.
|
| Maybe keeping a heavily-sandboxed Chrome in a VM for situations
| where Cloudflare is getting in your way might help?
|
| (In the large: this has been an issue a long time coming. Quite a
| bit of cyberpunk predicts the future where the web bifurcates
| into the "regular" web that is sanitized, corporate, controlled,
| and used by most people... And the "everyone else" web that is
| not, with all the pros and cons that entails. The tech has
| evolved to the point that companies that want a service provider
| "keeping the bad guys away" for them can pay to have that done,
| at the cost of false-positives... But at their scale, the false-
| positives may not matter to them).
| doubleorseven wrote:
| I use Whonix quite a lot, Most of the internet is unusable since
| i get into the "check the box" loop.
| mikequinlan wrote:
| If you can't pass the captcha you have to ask yourself, are you
| really a human being or have you just been programmed to believe
| that you are?
| PaulHoule wrote:
| It's ironic but I was having terrible problems accessing
| archive.today when I was using Cloudflare DNS (1.1.1.1) that
| cleared up when I switched to either my ISP's provider or
| Google's 8.8.8.8. I was not the only one
|
| https://news.ycombinator.com/item?id=38063548
|
| What's funny about it is that as a human I get tormented by those
| things all the time but I have been writing bots since 1999 and
| have yet to have had CAPTCHAs affect a webcrawling project in a
| big way: for instance I have a bot that collected 800,000 images
| from 4 web sites since last April, at times I thought they had
| anti-bot countermeasures but I realized that when they were
| having problems it was because the wheels were coming off their
| web site (don't blame me, that is 0.03 requests/second and are
| not parallelized and pipelined like the requests from a web
| browser.) I'm also prototyping one that can look at an article
| like
|
| https://phys.org/news/2025-01-diversifying-dna-origami-gener...
|
| see if there are links to journal articles in there, determine if
| the articles are Open Access and pick out an image for social...
| so far no problems. But if I want to pay my electric bill there's
| a CAPTCHA -- I mean, what kind of bot wants to pay my electric
| bill? (Kinda seems like it is asking for a lawsuit in this day
| and age if it prevents anyone 'differently abled' from accessing
| essential services...)
| duskwuff wrote:
| > I was having terrible problems accessing archive.today when I
| was using Cloudflare DNS (1.1.1.1)
|
| That's because that web site returns bad results to Cloudflare
| DNS, ostensibly because they take issue with the way it handles
| EDNS0. The fact that it fails to work is a deliberate choice by
| the site operator; it isn't Cloudflare's fault.
| johnklos wrote:
| That's oversimplifying a bit and missing some critical
| information.
|
| Cloudflare wants to "protect" people from exposing even their
| general region. This has the side effect of making CDNs that
| aren't Cloudflare work worse. Cloudflare are being dicks
| because they do to others what they wouldn't want to be done
| to themselves, or what they themselves don't do to
| themselves.
|
| It's not even that people are _choosing_ to opt in to
| Cloudflare 's bullshit. If you use Firefox in the US (and
| many other areas, but the US for sure) and you haven't
| manually configured Firefox or set up a canary domain, all
| your DNS lookups are going to Cloudflare, and they're using
| that to make other CDNs work less well. That's definitely
| shady and definitely bad on Cloudflare's end.
|
| I'm glad some people are taking a stand.
| whimsicalism wrote:
| the russian archive site and cloudflare have been having a
| dispute for a while now
| stavros wrote:
| That's not because of spam blocking (directly), it's because of
| a particularity between how the operator of archive.today wants
| to handle DNS and how the Cloudflare resolver handles it.
| webspinner wrote:
| Please do not use that term! I cannot fly! I don't believe in
| that sort of thing, either. I'm libertarian, and would rather
| not sue over much of anything! Especially something that would
| lack standing. Oddly enough, I haven't been interrupted by
| CloudFlare too much. I do use Firefox on Windows, but haven't
| gotten into Linux as of yet. Although it might be fun, I'd
| probably break it too much lol! I do run adblock, mostly for
| accessibility reasons. I don't want ads all over the page, when
| I'm trying to navigate. That makes the web suck a whole lot
| less! I do like RSS, I wish it was still supported in the
| browser, without an extension.
| PaulHoule wrote:
| (1) I was working for a small town web design shop in a town
| dominated by two higher ed institutions circa 2005 where I
| was finishing up a system which handled applications for an
| internship program run by the NSF. (e.g. complex forms,
| scanners to stop people from uploading CVs in Word that are
| full of malware, etc.) I was talking w/ the principal about
| how bad the usability was of applications in higher ed and
| that I was surprised that there hadn't been a lawsuit to the
| effect that "I tried to apply to organization X and couldn't
| because their website was screwed up" and he said that he
| thought they'd probably been settled out of court.
|
| I work at the Uni now and circa 2015 we had a lawsuit against
| us because we made people use terrible quality applications
| that weren't accessible. I'd make the case that that sort of
| organization which has a rigid social hierarchy (e.g. grad
| student, postdoc, assistant professor, associate professor,
| full professor, department head, provost, ...) finds it close
| to impossible to confront quality problems that it finds
| invisible. (e.g. if you submitted a bad paper to a journal or
| had sex with an undergraduate it could understand that but a
| web site could set your computer on fire and they wouldn't
| see a problem with that.)
|
| Since then all higher ed organizations feel a lot of need to
| offer accessible applications. My unit sells a subscription
| service to a data product and in sales talks and other
| conversations with our customers we find accessibility is a
| priority so it is a priority for me as a web dev.
|
| (2) Don't get me started about RSS. I think it is great,
| kinda. Fir $10 a month I can pay Superfeedr to scrape 110
| news sites and send them to my web hook which queues them in
| SQS and lets my RSS reader YOShInOn ingest them at its own
| convenience. I'd like to subscribe to 2000 or so independent
| blogs but don't want to pay a $100+ month scraping bill.
|
| Could I write my own crawler? Sure! But polling is for the
| birds. You really want to get a ping just when the event
| happens (ActivityPub? PubSubHubbub? AT Protocol? XMPP?) but
| instead you have to poll. There are two kinds of polling: (a)
| too fast, (b) too slow. Should I run it at home over my slow
| ADSL connection (is my wife having trouble using the internet
| because my crawler is having a bad day?) or should I run it
| the cloud where trying to save $5 a month on my bill could
| cause EBS volumes to go swap crazy costing me $500 a month?
| It's awful for people who run feeds, see
|
| https://rachelbythebay.com/w/2024/05/27/feed/
|
| although she should (a) just get a CDN and get over it or (b)
| give up on RSS. Sorry, people write stupid stateless crawlers
| with curl and making your crawler stateful enough to respect
| her silly 429 protocol makes RSS no longer a simple protocol.
|
| On top of that people keep failing with the same failing user
| interfaces for RSS readers that have been failing with 1999
| with no insight that "people tried that in 2001 and it
| failed". People like Dave Weiner have no insight why the
| world couldn't care less about RSS because they just won't
| recotnize there are problems.)
|
| (e.g. if you gotta know, YOShInOn works like TikTok... I
| never "mark as read", it doesn't show me little windows that
| show me the top N from 20 different sites, none of that.)
|
| (3) If it's your electric bill it really is an essential
| service that there is no competition for. Frequently markets
| work, but not in that case, even if Enron was able to fool
| some legislators that they would work in that case for a
| while.
| sbierwagen wrote:
| Polling for time sensitive things sucks. But for RSS, why
| not just poll once a day? Even if you're polling once an
| hour, a 304 response is, what, a few hundred bytes?
|
| I just checked my own server logs, and HTTP 200 responses
| were 12.7% of the total requests against my rss.xml. Which
| is suboptimal I guess (I haven't made a single post this
| year) but isn't outrageously terrible.
| webspinner wrote:
| Sure I believe in accessibility, just not lawsuits, yes two
| different things! If I don't like the way a service works,
| I cut ties with that particular service, and don't
| complain. Sometimes I'll contact them first, to advise them
| they have an accessibility issue, but I'm nice about it. I
| did this last year actually, I was testing feeder, I
| contacted them, to let them know I would be testing the
| service. When it really wasn't working for me, I needed
| them to make some tweaks, when the tweaks they made didn't
| work for me, mostly change the tabbed navigation that I
| wasn't using, I left. That being said, I like RSS a lot!
| However RSS readers these days leave a whole lot to be
| desired! I've been looking for one since 2013. I did use
| Miniflux for a while, until the service that was managing
| it for me shut down. I didn't learn terminal when I was
| young lol. I also have chronic migraines, so not sure if I
| could try and fix that now.
| bigfatkitten wrote:
| > I mean, what kind of bot wants to pay my electric bill?
|
| None, but they do want to use your electricity company's credit
| card payment facility to test stolen card numbers.
| oliwarner wrote:
| Cloudflare's --and most similar services'-- stance here comes
| from these VPN funnelling not just people like you, but also
| attackers. It's untrustworthy traffic from their perspective.
|
| Use a VPN but use a normal network. VPN back to your home, your
| office. Your traffic will probably take a throughput and latency
| hit but it looks like real residential traffic, and that's a lot
| less sus.
| Liquix wrote:
| but then all of your traffic comes from a single IP which is
| eventually associated with your identity. this defeats one of
| the core purposes of using a VPN to circumvent surveillance
| capitalism.
| oliwarner wrote:
| I'm not saying you're wrong, but in the context of travel, I
| would suggest most people use the VPN because they don't
| trust the networks they're connecting to, more than wanting
| to avoid surveillance, which would apply without the travel
| component.
|
| I also can't think of one of the popular VPNs that get
| heavily advertised that I'd trust to actually protect my
| privacy.
| idop wrote:
| Yes. I wrote about this on my blog six months ago [1].
|
| CloudFlare has positioned itself as the doorman of the Internet,
| deciding who gets to visit shitty websites written by AIs and who
| doesn't. Every time I try to visit a website and get blocked by
| this company and its unnecessary services, I congratulate myself
| for avoiding yet another terrible website and move on with my
| life.
|
| [1] https://ido50.net/content/what-chafes-my-groin-9.html
| squigz wrote:
| It seems a bit shortsighted to think that CloudFlare only does
| this for 'shitty websites written by AIs'
| idop wrote:
| I thought it was obvious I was being facetious.
| squigz wrote:
| It wasn't to me, apologies.
| gervwyk wrote:
| The doorman for the internet. well said. Someone need to study
| how this is likely the most successful marketing campaign ever
| for a cloud provider.
| theamk wrote:
| I don't think they needed much marketing? A lot of website
| operators want bot/DDoS protection, and cloudfare offers
| service which works (at least for overwhelming majority of
| users), and is absolutely free.
|
| Offering free stuff which works and that many people want is
| how internet companies get big.
| focusedone wrote:
| Exact same situation here. Linux, fairly funky firefox setup,
| eventually couldn't use half of the internet without hitting CF
| prompts, often wasn't able to get around them.
|
| I wound up removing / reinstalling firefox...same exact setup
| otherwise. No more cloudflare (or vastly fewer) prompts. The
| internet is usable again.
|
| Hope that helps.
| tempaccount420 wrote:
| I had to switch to Chrome. Reinstall was not enough for me.
| inetknght wrote:
| > _I use a heavily customized Firefox config on Linux._
|
| I also use a (not-so-heavily) customized Firefox config on Linux.
| I also see repeated abuse of my network activity by Cloudflare.
| sphericalkat wrote:
| I spent a few days agonizing over this same problem, and the
| culprit turned out to by my user-agent modifier extension.
| frereubu wrote:
| People are focusing on your very non-standard setup, but I've
| experienced this - less than you to be sure - on a standard MacOS
| setup with Firefox and only uBlock Origin installed. If I
| switched to Chrome without uBlock Origin it worked. This was on
| the English National Ballet's ticketing website.
| throwaway314155 wrote:
| Same problems here. Mac OS with firefox + ublock origin (and a
| dns based ad blocker) jams me up. Switching to Safari (with dns
| ad blocker still on) makes it work
|
| Has become increasingly more common in the past few months
| across several sites.
| therealmarv wrote:
| I do NOT like it at all but I just want to show a way how it
| works with Cloudflare and to make it painless with them.
| Basically fully assimilating to them because Resistance is Futile
| ;)
|
| 1) Privacy Pass Extension
|
| Install Privacy Pass Client Extension in your browser, here for
| Chrome https://chromewebstore.google.com/detail/silk-privacy-
| pass-c...
|
| 2) Use Cloudflare Warp (which is a VPN by Cloudflare basically,
| it's free):
|
| https://one.one.one.one/
| stebalien wrote:
| The privacy pass extension still requires you to pass a
| cloudlare turnstile which is impossible in some browser
| configurations. E.g., if you disable browser performance-
| debugging/timing features (these used to be a vector for
| Spectre timing attacks).
| casenmgreen wrote:
| Cloudflare works much, much better than Google - Google captchas
| for me, on Tor, are flatly impossible, always. They never let get
| through, no matter whether you get them right or wrong. You
| always get "try again".
|
| The problem I do have with CF is their captchas seem to require
| human interaction on the page, and this makes getting through
| them problematic when you open half a dozen tabs, and each loads
| a CF captcha, and you have to move the mouse around for ten
| seconds just to get the captcha to load, and loading is _not_
| reliable. Often you need to reload the page. It 's this type of
| performance, and poor performance, which is breaking web-pages
| for me.
| jeffbee wrote:
| That sounds like a feature. Tor is for abuse, so you don't want
| Tor people hanging around on your page.
| SahAssar wrote:
| > Tor is for abuse
|
| No. Tor is for anonymization. Some might use that for abuse,
| but that is not it's raison d'etre.
| homebrewer wrote:
| No it isn't. I discuss politically sensitive topics through
| it basically every other day, because by doing it directly in
| my country you will quickly end up in prison. (No, there's no
| _scarecrow of the day_ involved, just discussing things you
| take for granted in your liberal democracy.)
| yuumei wrote:
| But at least with Google captchas you can use AI to solve them.
| I use the buster captcha extension to solve them. It moves the
| mouse around like a human and solves automatically. I pay for
| captcha solvers for hcaptcha which is worse but cloudflare is
| just cancer. It's made the web unusable
| pixelesque wrote:
| Ehhh... maybe...
|
| Last week I had a run of (legacy) Cloudflare captchas on sites
| protected by CF to solve of "select all the boxes with
| motorcycles in", and despite doing it fastidiously and
| correctly (although I never know how to handle the boxes with
| like 3 pixels of object in but are otherwise clear), I had to
| do it like 5 times with different images, until suddenly it was
| happy.
| theamk wrote:
| legacy Cloudflare captchas?
|
| I thought they eliminated them back in 2023? Their
| announcement is pretty clear on them:
|
| "Cloudflare will never issue another visual puzzle to anyone,
| for any reason."
|
| https://blog.cloudflare.com/turnstile-ga/
|
| Are you sure it's not fake? For example archive.is sometimes
| sends me orange-colored CAPTCHAs (with "select all the boxes"
| style) that are never accepted; but if one looks closer at
| them, it actually never says "cloudflare" on them anywhere,
| nor there is a logo (it does this because it has a long-
| standing feud with cloudflare re users' privacy).
| pixelesque wrote:
| I thought they had as well, which was why I was surprised
| to see them.
|
| I can't remember the site I saw them on, so I don't know
| for certain, but the site was definitely protected by
| Cloudflare, and I'm not really sure what you mean by "fake"
| - they were definitely CAPTCHAs with image tiles, but I
| guess I don't know for certain they were coming from
| Cloudflare servers.
| omgin wrote:
| Try creating a cloudflare.com account and stay logged into it.
| I.e. every few days go into the cloudflare dashboard.
|
| Don't know if it will help but they use lots of methods to see if
| you are hostile, and being logged in and authenticated with them
| can't harm
| exabrial wrote:
| Just in time: https://doom-captcha.vercel.app
| idunnoman1222 wrote:
| This is pretty tough on mobile
| mppm wrote:
| Amen. Another fun one is logging into bank and government sites
| while roaming... with sms delivered intermittently and with a 5
| minute delay.
| ravenstine wrote:
| I've had to give up obfuscating my user agent because Cloudflare
| becomes nearly impassable as a result, and Cloudflare seems to
| own most web traffic now.
| tonymet wrote:
| i recommend everyone test the web with TOR to see how dead the
| public internet is. Reddit won't respond. Many sites have a
| 10-minute captcha challenge (e.g. substack).
|
| So many sites have deployed countermeasures like Cloudflare, but
| they aren't actively monitoring the failure mode on those
| countermeasures.
|
| The web is on it's knees and these countermeasures are another
| nail in the coffin if we don't act fast.
| Vampiero wrote:
| Why would anyone care about how a site displays on TOR aside
| from privacy nerds?
|
| The average internet user doesn't even know what TOR is. Though
| they might have heard the words "Dark Web" once or twice.
| tonymet wrote:
| it's a canary
| nicolas_t wrote:
| I absolutely hate cloudflare for the same reason you have.
| Besides traveling and using a VPN, I like in Hong Kong, a country
| that many sites have decided to block completely. It's very
| frustrating that cloudflare easily enables those kind of blanket
| bans for no reasons.
|
| Cloudflare is the enemy of open web.
| 7e wrote:
| You're using a dirty IP and not using Apple Safari, which has
| solved this via Private Access Tokens. Move out of the sticks.
| klntsky wrote:
| From the other perspective, I use Cloudflare for DNS and HTTPS
| certificates. Having an alternative that would cover these two
| use cases without the need for manually running letsencrypt would
| be enough for me to switch.
|
| I don't want to think about HTTPS, my websites are low risk,
| mostly static pages (and there are tens of them).
| superasn wrote:
| I appreciate you bringing up this issue about the Cloudflare
| challenges making it hard to browse. I had a similar experience
| where I couldn't access jsfiddle even without using a VPN. As a
| result, I switched to a different platform for my coding
| experiments.
|
| JsFiddle used to be my favorite for quickly testing out code
| snippets. It's a shame that due to Cloudflare hurdles, I've
| stopped using it and don't plan on going back.
|
| It may not be much but as more websites and businesses lose
| genuine web traffic like this, Cloudflare might eventually listen
| and fix this mess.
| panic wrote:
| One concrete thing we can do is to stop seeing Cloudflare as an
| easy, unproblematic solution. Bring up issues like this when
| people suggest using it.
| peanut-walrus wrote:
| The problem is that any solution so far proposed for this is very
| privacy-unfriendly.
|
| For example, Google proposed https://github.com/explainers-by-
| googlers/Web-Environment-In... and this was shot down by privacy
| advocates (for very good reasons).
|
| So basically the choice for website operators is either to fight
| the bots and accept that their service will be unusable for some
| subset of their users or not fight the bots, which will lead to
| their service becoming unusable for everyone.
|
| More and more, you see services pushing you very hard towards
| using their app and the reason is that with the app, they are
| able to actually verify that you are likely not a bot (or rather,
| in reality, that at least the app is running on an actual
| physical device, mobile phone bot farms are unfortunately also a
| thing).
|
| As for Cloudflare - they offer it as a service, so when the
| website operator has a choice between using them or allocating
| several engineers for bot-fighting, why would they not just go
| with Cloudflare? Doing it yourself can be slightly higher
| fidelity, as you know your customers better, but it is also a lot
| of effort which could be better spent elsewhere.
| soerxpso wrote:
| > either to fight the bots and accept that their service will
| be unusable for some subset of their users or not fight the
| bots, which will lead to their service becoming unusable for
| everyone.
|
| 2/3 of the issues OP listed would not make the service unusable
| for anyone if the botcheck were removed. 1. What would be the
| problem with allowing "bots" to opt out of receiving marketing
| emails? Why do I need to be a human to tell you to stop
| spamming me? Who is running such a bot, for what purpose? 2.
| What would be the problem with allowing a "bot" to log in to an
| already-verified human account a single time?
|
| The only situations where you actually need to confirm that a
| user "looks human" is for repeated connection attempts in quick
| enough succession to matter (DDoS prevention), or when they
| want to do something that someone would actually write a
| nefarious bot to do (mainly just creating posts/messages
| visible to other users).
| Raed667 wrote:
| Just an idea, what if malicious bots started unsubscring
| thousands of email addresses to harm your business.
|
| Even if you send a confirmation email afterwords that's
| potentially millions of emails you are sending because of
| bots.
| ryao wrote:
| The challenges are configurable by cloudflare's customers. The
| challenge can either be from turnstile, which is a captcha
| replacement service that websites use on their own pages, or a
| cloudflare CDN security setting that will block access to pages
| until a challenge is passed. It is not clear which one the
| original poster means.
|
| Cloudflare's customers can largely disable these and rely on
| other means of detecting bots.
|
| In the case of turnstile, it has three modes, two of which are
| entirely automatic and work by interrogating the web browser,
| with the other requiring a client:
|
| https://developers.cloudflare.com/turnstile/concepts/widget/
|
| Cloudflare CDN's security setting on its free tier also has an
| essentially off setting that will basically eliminate challenges
| when browsers accessing pages protected by cloudflare unless
| there are exceptional circumstances. I believe it can be fully
| turned off for the enterprise tier.
|
| Whenever I configure cloudflare for a website, I always turn off
| challenges since they are annoying to users. There is an
| interesting write up about how cloudflare's bot detection works
| here:
|
| https://blog.capmonster.cloud/en/blog/web-scraping1/how-clou...
|
| Note that I have yet to use turnstile, so I am speaking from
| documentation I read rather than from actual experience with it.
| I have used cloudflare's CDN and I am speaking from experience
| with it.
|
| Anyway, the website author is the one that should be blamed here.
| brunojppb wrote:
| My workaround for this as a person who travels a lot was to buy 2
| raspberry Pi's and put them at my family houses in different
| countries and use Tailscale on them as exit nodes, behaving like
| my own VPN. The residencial IP address makes things a lot easier
| when connecting from random places.
| rtrgrd wrote:
| Slightly off topic, but Microsoft ones are even worse - when I
| tried to sign up to OpenAI/get a new Microsoft account, the
| captcha were so difficult that it took me 5 minutes to solve
| (unsuccessfully). As a libre wolf user with very strict settings,
| I think privacy-aware users bear the externalities of this bot vs
| server arms race.
| trhway wrote:
| well, looks like a business opportunity - a service using AI to
| automatically pass the challenges like this so the people like
| the original poster could, for a small service subscription fee,
| use the Internet hassle-free again.
| chrismorgan wrote:
| AWS WAF is even worse. I recently moved from Australia to India,
| and quite a few high-profile websites are now completely
| inaccessible to me because WAF seems to be legitimately broken.
| Two such sites: https://officeworks.com.au/ and
| https://centrecom.com.au/. You successfully complete their
| annoying thingummy, and it redirects you... to the same Human
| Verification CAPTCHA. This has been the case for at least half a
| year, so it's not a recent breakage.
|
| If I tunnel via my VPS which is still in Australia, _then_ I can
| access it.
|
| But complete blocks via Cloudflare have also been a problem: I
| had to do something with VicRoads as part of selling my car, and
| was blocked outright when I got to the actual form page. Had I
| not had my VPS in Australia, I don't know what I would have done.
|
| My IP address is massively shared (CGNAT) with plenty of botnet
| around, so I'm frequently troubled by Cloudflare, but not often
| outright blocked, and if challenged rather than blocked, I've
| never had any problem with it. Linux, Firefox.
| bobnamob wrote:
| As another aussie expat abroad, leaving a box behind at my
| parents place for an Australian residential IP has got to be
| one of the most unexpectedly great things I've done.
|
| Wireguard/Tailscale and my parents having access to cheap
| renewable power are the real enablers ofc.
|
| To anyone moving abroad in the near future - leave a box behind
| with your parents/close friends, it's well worth the trouble if
| they're ok with you occasionally mooching some bandwidth. You
| absolutely won't regret it
| imhoguy wrote:
| And don't be tempted to run any upgrades on it until you come
| back :)
| bobnamob wrote:
| Yeah, I'll admit there's some paranoia about losing access
| after a botched upgrade.
|
| I'm considering investing in a https://tinypilotkvm.com/,
| but that can wait till I've lost ssh at least once. I'm not
| hosting aws on the thing so I can afford to play it fast
| and loose :)
| ycombinatrix wrote:
| imo it's more flexible to just have two boxes, especially
| if they're cheap.
| ycombinatrix wrote:
| Oof. Been there.
| neilv wrote:
| Cloudflare is so embedded into so many important services (like
| some other companies, including Google), that they need to be
| thinking of their role as having some government-like
| responsibilities.
|
| For example, for starters, Cloudflare and Google need to find
| ways so that individual people who're wrongly being locked out of
| services by the company, have some way to get that unlocked. Not
| "sux2bu we dont do support bro".
|
| (Then they can start thinking about the next step, which is due
| process, and what it means to wrongly lock out someone in the
| first place.)
|
| That said, as an immediate pragmatic matter, one debugging tip
| with your Firefox is to go to the `about:profiles` URL, and
| temporarily create a new profile, and without using any Firefox
| sync feature, and see if Cloudflare lets you through, and then
| incrementally add back in your extensions and preference
| customizations, and see if/when CF stops letting you in. (Not
| that it will necessarily identify the sole and exact trigger,
| since they might be using scores of multiple factors, but it will
| be evidence of one thing that pushes it over the edge. And maybe
| get you to a compromise setup that lets you do your work for
| now.) Also helpful is to have alternate browsers installed;
| personally, I keep Chromium installed, as my "violate me every
| possible way, if you'll just let me access this one page/site I
| really need right now".
| 1vuio0pswjnm7 wrote:
| Try some popular user agent strings first before concluding that
| something else like TLS fingerprinting is the problem. Sometimes
| an acceptable UA string is all it takes.
| ForOldHack wrote:
| Mess? I got a 8 try guess Ilol to try in 5 tries... in an
| indistinguishable font. Ooo... Im gonna fail that one...
|
| I am good at this stuff, and "Cloudflare challenges have made
| large portions of the web unusable for me" too.
| mrayycombi wrote:
| Set up your own vpn on AWS ec2. It will bypass the vpn blocks
| they have. Problem solved.
| udev4096 wrote:
| I would recommend using FlareSolverr as a proxy in your browser
| to bypass the clownflare's captcha
| Havoc wrote:
| Vaguely related, youtube is lately doing a lot of unnecessary
| forced logouts & reconfirm password. I'm literally on a static
| IP. On the same computer & browser. With the same cookies. Not
| accessing anything particularly sensitive. There is no way in
| hell they don't know precisely who I am & that its me.
| edmundsauto wrote:
| Is it possible your account is being attacked in the
| background?
| Havoc wrote:
| Possible I guess. It is yubikeyed though so would need to be
| a pretty sophisticated attack
| demaga wrote:
| Now every couple of minutes when scrolling through Reddit, red
| "Network issue" tab appears. Some comments don't load at all,
| some are labeled "deleted" even though they aren't. Refreshing
| the page usually does the trick, but I hate this new experience.
|
| I guess they're just protecting themselves from bots, and I look
| like a bot in their eyes.
| prmoustache wrote:
| I get this all the time with firefox on linux + ublock origin
| extension. Often ending up with that blocked ip page.
|
| I mostly shrug off and just avoid visiting that kind of sites
| again. For an unsubscribe challenge I just copy paste the url and
| visit it using firefox focus on my smartphone on my mobile
| connection.
| kittikitti wrote:
| Only the expensive bots with residential IP's and mechanical
| turks can survive, humans be damned.
| jvaleski2 wrote:
| CF issue, or site programming issue.?
| Spooky23 wrote:
| Stop acting shady.
| johnklos wrote:
| Cloudflare puts challenges on their abuse contact page and rate
| limits it to much slower than human speed. It's also _still_
| broken after years in that you can 't report abusers who register
| domains through Cloudflare and/or host their DNS using
| Cloudflare.
|
| They really don't want feedback from people who don't pay them.
| citrin_ru wrote:
| Just tried to disable User-Agent in Firefox by setting
| general.useragent.override to an empty string and Cloudflare
| captcha become impenetrable. Cloudflare actively blocks attempts
| to improve privacy :(
| bluGill wrote:
| You don't want that to be an empty string - you would be one of
| the few people in the world with that value and thus easy to
| identify. You instead want that string to be exactly the same
| as everyone else in the world (no matter what the real User-
| Agent really is). there should be about 50 different contents
| of the entire header possible for everyone in the world.
___________________________________________________________________
(page generated 2025-01-03 23:02 UTC)