[HN Gopher] Blocked by Cloudflare
___________________________________________________________________
Blocked by Cloudflare
Author : jrhawley
Score : 391 points
Date : 2023-08-08 13:55 UTC (9 hours ago)
(HTM) web link (jrhawley.ca)
(TXT) w3m dump (jrhawley.ca)
| rejectfinite wrote:
| Using something like Edge (for work), Vivaldi or Brave would be
| better than Chrome and probably let you in instead of Chrome.
| AegirLeet wrote:
| I've had the exact same problem for a while. Here are some of the
| sites I've been unable to access (found by searching for "just a
| moment" in my browser history):
|
| - https://gitlab.com/users/sign_in
|
| - https://steamdb.info/login/
|
| - https://www.zabbix.com/forum/
|
| - https://casetext.com/
|
| - https://namemc.com/login
|
| - https://spinroot.com/
|
| - https://camelcamelcamel.com/
|
| It's really annoying and Cloudflare is apparently doing nothing
| to fix it as this has been going on for months if not years. I
| guess Cloudflare just hates the open web and really wants to
| enforce Chrome/Chromium/Blink hegemony.
| adammartinetti wrote:
| Would you be willing to share a rayID you see during one of
| these looping challenges? I'm the PM for Cloudflare's challenge
| platform, and we'd love to look into this. RayIDs contain no
| PII so you can share publicly, or feel free to drop me an email
| at amartinetti at cloudflare.
|
| We'll also release a reporting mechanism soon, so in the future
| you can let us know when you see these issues and we can react
| to them quickly.
| itscrush wrote:
| Here's some loop samples;
|
| - Gitlab; Ray ID: 7f3961b4ec46c443
|
| - Zabbix; Ray ID: 7f39624d982bc32e
|
| - NameMC; Ray ID: 7f3962e68d251871
|
| - Camelcamelcamel; Ray ID: 7f3962eb9cbb421f
|
| Easily can recreate at least the never ending loop by
| flipping on ublock origin's 3rd party scripts and 3rd party
| frame blocking, which matches their recommended medium
| settings.
| adammartinetti wrote:
| Thanks so much to you and everyone else who's supplied
| these. I'm collecting them now, and the team is looking
| into this.
| executesorder66 wrote:
| It would be nice, once the investigation is concluded, if
| you guys posted the findings on the cloudflare blog.
| Otherwise it would just feel like a "your call is very
| important to us, please hold" kind of situation.
| lopkeny12ko wrote:
| Such a classic and incredibly annoying SaaS PM move. Pinky-
| promise that you mean well, pretend to be invested in the
| issue, ask customers to supply evidence and say you'll look
| into it, followed by radio silence and no follow up
| whatsoever.
|
| Incidentally, another Cloudflare PM for Pages asked me to do
| the same thing--I shared my account ID, the request, the
| problem, timestamps, etc...never heard back ever, request
| went straight into the void.
| sockaddr wrote:
| Yup. It's all show.
|
| A service has injected itself between you and your goal,
| it's going to periodically impede you from reaching that
| goal and then lie to you about why, all while making money
| off of the arrangement.
| ta89313 wrote:
| - Gitlab: 7f39707d0fa023af
|
| - Zabbix: 7f3970eabe8ff196
|
| - SteamDB: 7f396f534b0400d2
|
| - Casetext: (works)
|
| - NameMC: 7f3971a01a22d5a8
|
| - Spinroot: (works)
|
| - Camelcamelcamel: (works)
| 2Gkashmiri wrote:
| gitlab 7f39759e1abe1bce
|
| casetext 7f39762f693733e4
|
| steam 7f397694995aa3b7
|
| all over firefox
| AegirLeet wrote:
| Here's a handful:
|
| - 7f395b5ddfe43a54
|
| - 7f395ca09bfa3a54
|
| - 7f395d8afaf73a54
|
| - 7f395f075e33690d
|
| - 7f396102afef35fd
| blocked-by-cf wrote:
| I also cannot access my VPS provider when using firefox.
|
| ray id 7f3a169d4e630306
|
| I previously had the same problem with ungoogled-chromium as
| well (regular chromium worked), but I guess it works now
| after 2-3 loops.
| amadeuspagel wrote:
| If only there was some open standard for browsers to verify
| that a real human is visiting a website, so that website owners
| wouldn't have to rely on bespoke hacks that only work in
| chrome.
| systemvoltage wrote:
| This would be great if it didn't have any downsides. China
| has a system like that: QR code to login everywhere.
| Everything is linked to your phone number which is given
| after taking a picture of you and official ID.
|
| We are gonna have to live in a slightly bot-rich society to
| keep this at bay.
|
| It starts with browser control. And then, ends with needing
| human verification to ssh into a server that you own. Let's
| just build better security.
| shadowgovt wrote:
| They're booing, but you know you're right. ;)
| clsec wrote:
| It happens to me all the time. And it _has_ been going on for
| years, but it 's getting noticeably worse over time. One way or
| another you have to pay to use the web, be it costing you loss
| of access because of your strict privacy settings or paying by
| giving away your privacy. There's no win here..
| stjohnswarts wrote:
| I see the "are you human" on there with a click, but no
| looping, it goes straight to the website
| adrr wrote:
| Users want to chrome hegemony and don't care about the open web
| or Firefox. Its the number #1 browser on desktop even though it
| doesn't come with the OS. Windows comes with edge and macs come
| with safari. Users have to download Chrome.
| j45 wrote:
| All browsers so far have come and gone in popularity. Even
| when it seemed unimaginable.
| Iulioh wrote:
| But today everything but Firefox is Chromium.
|
| That's a little different
| warrenm wrote:
| And 20 years ago everything was IE (at >90% penetration)
| Iulioh wrote:
| The problem is Chromium ,not Chrome
|
| Like you have the illusion of choice, that's what I'm
| talking about and that's different
| [deleted]
| zer8k wrote:
| I haven't had any problems on Waterfox. However, it is absurd
| to me I need javascript to simply visit a website anymore.
| michaelt wrote:
| Sadly, the fact a given site works for you or me is no
| guarantee it works for someone else.
|
| These bot detection systems tend to use all manner of
| imprecise statistical heuristics and weird fingerprinting.
|
| Perhaps AegirLeet has a graphics card that a popular web
| scraper pretends to have. Maybe they're in a suspicious
| timezone. Maybe they've installed a font usually only found
| on a different operating system. Maybe I'm never blocked
| because I have an excellent IP reputation, due to regular
| visits to approved websites.
| flangola7 wrote:
| Fingerprinting is scarily accurate now, strangely.
| https://fingerprint.com/
| marcosdumay wrote:
| Somehow, it doesn't matter. Like the fact that the author
| shared non-repudiable identification information with the
| site also didn't matter and he was classified as a robot
| anyway.
| stjohnswarts wrote:
| yeah, starting that that invisible pixels and cookie
| tracking are way in the past and unless you're using
| something like tor (and not changing the resolution) then
| they know who you are. I mean you can still block ads and
| cookies, but I figure they really do know who you are.
| veave wrote:
| As a data point, enabling privacy.resistFingerprinting on
| Firefox defeats this website.
| kmlx wrote:
| that's because websites have evolved into web apps.
| adrianN wrote:
| Most websites haven't done that
| megous wrote:
| Yeah, gitlab also blocks me from logging in (via its cloudflare
| use). It did so even when we paid for it. We no longer do. (for
| other reasons, but anyway, good riddance)
| unmole wrote:
| I can access them all fine using Firefox on Android.
| krono wrote:
| The amount of times Cloudflare is making me sit through their 15
| to 30 second "checking your connection" page is insane.
|
| For people going through life with ADHD such as myself, the
| impact of all these delays and disruptions throughout the day can
| be severe. Despite being properly medicated this measure is
| absolutely debilitating and makes for a dreadful and very taxing
| online experience.
| jeroenhd wrote:
| Cloudflare's Privacy Pass may help here:
| https://privacypass.github.io/
|
| It should significantly reduce the amount of CAPTCHAs you see
| in a way that's not terrible for privacy.
|
| For Safari, you can enable Private Access Tokens:
| https://blog.cloudflare.com/how-to-enable-private-access-tok...
|
| Both of these mechanisms are similar to Google's web DRM
| proposal in that they rely on external issuers to generate
| tokens, but unlike Google's attempt they don't guarantee that
| ad blockers are disabled on pages that try to use tokens.
| tomxor wrote:
| Wow, that doesn't sound like a terrible idea!
|
| Which is honestly surprising in this area where it feels like
| privacy, anonymity and human verification are incompatible
| with each other.
|
| I am trying to minimise my time wasted by websites, which is
| hard to balance with privacy, one other one is the repetitive
| consent forms (if you don't retain cookies, it's a never
| ending process). I think consent forms and human verification
| are the 2 biggest human time wasters.
| jeroenhd wrote:
| > Which is honestly surprising in this area where it feels
| like privacy, anonymity and human verification are
| incompatible with each other.
|
| The thing is, it still allows for some correlation between
| attestation provider and the websites themselves,
| potentially exposing part of your browsing history to these
| companies based on how many tokens you use and what
| websites consume them.
|
| That doesn't matter much for Cloudflare's implementation
| (now Cloudflare knows when you visit Cloudflare, oh no!)
| but with Apple's attestation provider the risks increase.
| The smaller the attestation provider gets or the fewer
| parties trust that particular attestation provider, the
| higher the risk becomes.
|
| It's better for your privacy than the current norm (de-
| anonimisation through fingerprinting while you fill out a
| CAPTCHA) but it's still not great. It also allows for
| attestation providers (and their algorithms) to arbitrarily
| deny you access to the web if other websites decide to
| start using them.
|
| Privacy in exchange for power, I'm not so sure about that.
| I imagine for someone suffering from ADHD the small risk
| that Cloudflare decides to screw you in particular is worth
| the massive improvement in browsing experience, but
| everyone will have to determine the pros and cons for
| themselves.
| tracker1 wrote:
| For good or bad, this is why I have the Privacy Pass extension
| installed.
| jeroenhd wrote:
| Some comments I have on this post:
|
| > Worse yet, I know that Cloudflare knows I have those
| certificates. Why? Because it asked for them!
|
| Not really. Cloudflare notices your browser has TLS
| authentication available and asks you for it. That's really
| annoying, but part of the protocol spec. Your browser won't send
| this information unless you pick a certificate and hit OK.
|
| Disable your ad blocker and you'll find that many trackers will
| also ask you to identify yourself this way. It's really annoying,
| browsers need to design better UX for this type of
| authentication.
|
| > * MAC address of my machine that I have previously used to
| access this site
|
| How does it gather your MAC address? Did you disable IPv6 Privacy
| Extensions? Unless the website is sitting behind the same switch
| as your computer or you run some kind of native application that
| sends the MAC address, websites can't read the MAC of your
| network interface. Enable the MAC randomisation that's present
| (sometimes even turned on by default!) in every modern OS if you
| consider the local switch or WiFi network to be a privacy risk.
|
| > Will I be able to create and sync these passkeys myself?
|
| Yes, assuming they follow the standard
|
| > Can only certain types of software use passkeys? If so, who
| decides what software meets this standard?
|
| I don't really understand the question. Any software supporting
| passkeys will be able to prompt you for generating or using a
| passkey.
|
| > Will I only be able to generate passkeys on a device with
| specific hardware/software requirements like a TPM, DeviceCheck,
| or Integrity API?
|
| According to the spec, keys can be stored in software no trouble.
| Websites and apps can ask for securely generated keys, but I
| don't think those are all that common. Hardware can also be faked
| relatively easily in most circumstances.
|
| > Can I, at any time, export my passkeys from one service
| provider and switch to another provider?
|
| Ask your service provider for export options. Most likely, you
| can't just dump the keys and import them elsewhere (that would
| defeat the point).
|
| > If a passkey is invovled in a suspicious event, will that
| suspicious mark propogate to any other device that uses that same
| passkey? Do devices that contain suspicious passkeys also get
| marked as suspicious? If so, would that impact the ability of
| that device to access other independent websites?
|
| That depends on the software using the key for authentication.
| Maybe?
| paradox460 wrote:
| I've seen the misconception a lot; people seem to think that
| random websites can grab your MAC like they can get your IP
| address.
| Ekaros wrote:
| MAC is a weird thing, it is important for ethernet to work
| efficiently. But in actuality it only travels to about next
| router from machine...
| jeroenhd wrote:
| With WiFi, MAC addresses become more of an issue (as you're
| often scanning and roaming), but we have randomisation
| algorithms for that.
|
| The same was true for IPv6 for a while, but that too has
| been solved a long time ago.
|
| For a SLAAC client whose privacy extensions have been
| disabled for some reason, it's very much possible to figure
| out your MAC address. This may be a problem on old hardware
| that doesn't receive updates anymore!
| Dwedit wrote:
| The big problem I have with Cloudflare's integrity check is that
| all the spam domains use a fake version which mimics it, and
| tries to trick you into completing a captcha.
| datavirtue wrote:
| Please rename to "Blocked by Firefox"
| j16sdiz wrote:
| > Worse yet, I know that Cloudflare knows I have those
| certificates. Why? Because it asked for them!
|
| It doesn't make sense for Cloudflare to request any client
| certificates.
|
| I think there are real bugs somewhere.
| Operyl wrote:
| It's requesting client certs for their internal intranet stuff
| hosted behind cloudflare.
| ArchOversight wrote:
| Cloudflare allows you to enable mTLS for websites:
|
| https://developers.cloudflare.com/ssl/client-certificates/en...
|
| https://developers.cloudflare.com/cloudflare-one/identity/de...
|
| This would then require Cloudflare to request a client
| certificate. This is great for securing websites using
| corporate identity that is derived from AD certs for example to
| make sure the device being used has a valid cert on it.
|
| Alongside MDM for example forcing the certificate to have a
| short lifespan (my $CORP uses 7 days) you can validate that the
| device has the correct security posture to access the
| resources.
|
| If for example I let my device not update the version of macOS
| often enough my cert expires and I can't access internal
| resources until I update my OS and MDM software checks that and
| provisions me a new device certificate.
| miyuru wrote:
| I cannot access flyertalk.com, which hosts lot of useful airline
| content from any IP from my country. I tried reaching out via
| email as mentioned in the error page and admin does even have a
| valid email posted anywhere.
|
| I know cloudflare is not to blame here, but they provide way easy
| access to blocking to bad admins.
| 1vuio0pswjnm7 wrote:
| No problem for me accessing Gitlab without using a web browser.
| Moreover one can use Internet Archive, Archive.today, Google's
| cache, etc. to avoid SNI.
|
| The author did not specify which project he was trying to access
| so I picked a random one to test from
| /explore/projects/topics/bioinformatics. No problem accessing it
| without a web browser. TLS1.3. No SNI.
|
| https://one-touch-pipeline.gitlab.io/otp/
| thedaly wrote:
| I've been getting stuck in the "browser integrity check" loop a
| lot on firefox lately. Not an issue in chrome, not using a vpn,
| etc. I assume it is some combination of extensions and/or
| settings in firefox.
| IG_Semmelweiss wrote:
| Its happened a few times here too
|
| I also have maxed out anti fingerprinting etc on FF, so it
| comes with the territory. I have to slowly enable JS on some
| sites to see if the loop will break, or i just navigate away.
|
| I use all browsers except chrome, but i only navigate the web
| with FF
| kelnos wrote:
| Prediction: if Google manages to ram Web Environment Integrity
| down our throats, CloudFlare will implement it as a part of these
| checks.
| thedanbob wrote:
| Just yesterday I realized that I couldn't log into Paypal on
| Safari or Firefox, only a Chromium-based browser. We're getting
| deeper all the time into "this site is best viewed in Google
| Chrome".
| jeroenhd wrote:
| I have this on Twitch. I can't log in with Firefox +
| fingerprint resistance. Apparently 2FA isn't strong enough for
| account protection, I have to let Twitch uniquely identify my
| computer or it won't let me log in.
|
| I just stopped watching Twitch streams.
| buro9 wrote:
| I've been experiencing the PayPal one for a while. Firefox on
| Windows, Linux or Android. Thankfully the app is still signed
| in, but I've used credit card for things that I have PayPal
| money just sitting there ready to spend as I can't get into
| PayPal on Firefox.
| Animats wrote:
| Time to switch from PayPal to FedNow. FedNow is run by banks
| and the Fed, which are regulated as to whom they can refuse
| to server.
| jaywalk wrote:
| It's not available for consumers yet, and there are only 40
| or so banks currently on the network.
| buro9 wrote:
| Never seen an online shop accept that, nor do I know if
| it's open to UK citizens. But thanks
| Animats wrote:
| In the UK, you have Single European Payment Area
| payments. (Despite Brexit, the UK chose to stay within
| the SEPA zone.) The US didn't have a bank-level national
| service for consumer to consumer payments until FedNow.
| Just PayPal, Venmo, etc., which are second-tier services,
| which becomes an issue when they break.
| alberth wrote:
| I'm surprised Apple PAT and Google WEI wasn't mentioned in the
| article.
|
| Especially since Apple has partnered with Cloudflare on PAT.
| joshstrange wrote:
| They do mention it (Google's at least) in this section [0]
|
| [0] https://jrhawley.ca/2023/08/07/blocked-by-
| cloudflare#implica...
| CharlesW wrote:
| Also, the scope of PATs is vastly different than WEI. Think
| of PATs as a "probably a human" signal that mostly replaces
| the need for CAPTCHAs.
|
| https://blog.cloudflare.com/how-to-enable-private-access-
| tok...
| jsnell wrote:
| Their scope is the same [0], both in terms of stated intent
| as well as what kind of things they could be used to
| attest. The only significant differences are that one is
| already deployed in prod while one isn't, one got a
| marketing blitz while the other didn't, and that they're
| done by different companies.
|
| There are no vast technical differences, only incredibly
| subtle ones.
|
| [0] https://www.snellman.net/blog/archive/2023-07-25-web-
| integri...
| dcow wrote:
| So many privacy nuts use Chrome and don't realize this:
|
| > What about Google Chrome?
|
| > I tried all of the above in Firefox. So I naturally tried to
| access the same page in Google Chrome to see if I'd still be
| blocked. Thankfully, I wasn't.
|
| > But of course I wasn't because Chrome doesn't have the same
| privacy- and security-enhancing designs that Firefox does. Chrome
| will happily collect as much private information about me and my
| browsing history and share them with select parties, as needed.
| It also doesn't resist fingerprinting or let me modify settings
| to the same degree that Firefox does because Chrome relies on
| those fingerprinting technologies to ensure that I am targeted by
| ads it deems necessary for me to see.
|
| > Being blocked on Firefox and not blocked on Chrome also tells
| me that Cloudflare is blocking me based on the fingerprint (or
| lackthereof) of my browser. Everything about my connection is
| identical between the two requests, aside from the browser being
| used. It's the same security certificates, same corporate VPN,
| same machine, even the same timeframe when I try to access the
| site.
|
| If you care about _anything_ these days, don 't use Chrome.
| dmix wrote:
| > If you care about anything these days, don't use Chrome.
|
| Or Cloudflare.
| warrenm wrote:
| funny enough... I called out Cloudflare for the pariah it is,
| and got downvoted and flagged
| warrenm wrote:
| and someone's come and done it again
| waithuh wrote:
| I seriously never get people that _love_ CF (or any
| company for that matter). Praising 1.1.1.1, giving it
| free advertising. CF is basically handing over your
| website in return for some less work on your part. I get
| the advantages of it (like less engineer credits wasted,
| less server maintaining work and probably cost, faster)
| but actively giving it free PR just doesnt fit right with
| me. Pay your bucks and sit. They are a Big Tech company,
| they dont need your prayers.
| Dalewyn wrote:
| >CF is basically handing over your website in return for
| some less work on your part.
|
| The older you get, the more valuable being able to just
| dump your shit on other people becomes.
| sph wrote:
| I have done the same to the same result. We must be the
| lunatics, as everyone keep defending their decision to put
| everything, even their personal blog, behind a single
| company, because "they might get DDOSed".
|
| The absolute state of software engineers and systems
| administrators in here, man. Talk about overengineering and
| premature optimisation, let alone being totally oblivious
| that their laziness is what creates a monopoly.
| jwatte wrote:
| I don't quite understand the "ads it deems necessary for me to
| see" comment. You will always get ads on sites that serve ads.
| The thing the tracking might do, is change which particular ads
| you get. The right solution to that, is to use an ad blocker,
| and to pay for sites that have an ad-free alternative.
|
| Also, fingerprinting isn't always "bad" -- any business who
| takes credit cards online, wants to try to exclude people who
| will commit fraud (because they might have done it before.)
| Preventing fingerprinting, means you prevent certain anti-
| fraud, which means that you see higher prices and more friction
| doing commerce online, which also affects your experience. The
| connection is just much less direct.
| baq wrote:
| Tracking is establishing your identity. Try using a private
| mode Firefox via a VPN. Half of the web is completely
| unusable. You get put in unsolvable catchpa hell as
| punishment for being anonymous.
| amadeuspagel wrote:
| Try walking into a real place with a mask on and you might
| also get treated less pleasantly.
| ipaddr wrote:
| That's implausible. Using finger printing for fraud detection
| would only catch someone using different cards on the same
| machine. Once a card is deemed stolen it stops working so
| it's unnecessary for that scenario. That doesn't even go into
| fake fingerprinting some browsers/plugins.
|
| The price is the highest the market will pay. Increasing that
| price means few customers lower revenue. Fraud is a cost to
| the business they must pay out of profits because if they
| tried to increase prices demand would drop.
| ficklepickle wrote:
| It can be an aspect of it. For example, if there are
| suddenly many unique fingerprints making purchases from the
| same residential IP, that might look suspicious.
|
| Granted, I'm not aware of a lack of fingerprint being
| penalized. That said, there are products that allow custom
| rules, in which case anything is possible.
|
| I work for a company in this space. Opinions are my own.
| executesorder66 wrote:
| > So many privacy nuts use Chrome
|
| Really? That's news to me.
| dmix wrote:
| Well, Chromium is quite popular with the security conscious
| on Linux. At least it was when I was using ArchLinux, they
| had some good custom build script versions.
| antonvs wrote:
| Security conscious and privacy conscious aren't the same
| thing, although there's overlap. I can be concerned about
| the security of my system without caring about whether I'm
| being targeted for ads.
| bcoates wrote:
| Some particular build of Chromium and Chrome are vastly
| different systems. A lot of this is philosophical; The
| Mozilla way is to support standards and tut-tut at websites
| for doing overtly malicious things like looking at user-
| agent or asking for widevine, the Chromium way is to treat
| the web as a hostile actor and offensively subvert anti-
| user behaviors.
|
| Any modern browser that doesn't actively fingerprint as
| either most-common Chrome on a laptop, most-common Android
| browser, or most-common iPhone is written by such
| hopelessly naive nerds that they shouldn't be trusted with
| user-facing software with real security considerations.
| shadowgovt wrote:
| I care about accessing the sites I use quickly and efficiently,
| with a minimum of auth and compatibility dance.
|
| Since Chrome is so common that it's basically guaranteed to
| have been tested against the site I'm trying to access, I use
| Chrome.
| afavour wrote:
| I'm no Google fanboy but I wasn't satisfied with this:
|
| > Chrome will happily collect as much private information about
| me and my browsing history and share them with select parties,
| as needed
|
| What information does Chrome provide in this scenario that
| Firefox doesn't? It feels like backward logic: it worked in
| Chrome therefore it must be because Chrome gave extra info. In
| reality it could be a whole bunch of things, something as
| mundane as Firefox being a rarer user agent so subject to more
| filtering.
|
| It strikes me that all of this is an inexact science. I've run
| into rate limit messages with sites before now that go away
| when I switch browsers, no matter what the browser is. I assume
| it's because, with the limited information given, the DDOS
| protection software assumes that same IP + different UA =
| different computer.
|
| I have no clue but I wasn't persuaded that this specific
| scenario works with Chrome because it was giving away more
| information. At a bare minimum at least try a third browser!
| brandon wrote:
| I don't mean to support or refuse the author's main points or
| analysis, but you might like to know that the Chrome team is
| currently working towards shipping the Topics API. I have
| strong opinions about it but I will try not to editorialize.
|
| My high-level understanding is that they're going to run an
| ML model over your browsing history (locally on your device)
| to build a list of "topics" that you care about. Sites you
| browse can use the Topics API to pull a set of these
| interests from the browser to show you "relevant" ads.
| Mozilla has taken a negative position against this standard.
|
| https://privacysandbox.com/proposals/topics/
|
| https://github.com/mozilla/standards-positions/issues/622
| afavour wrote:
| How is that relevant to the topic?
| Santosh83 wrote:
| You asked:
|
| >> Chrome will happily collect as much private
| information about me and my browsing history and share
| them with select parties, as needed
|
| > What information does Chrome provide in this scenario
| that Firefox doesn't?
| afavour wrote:
| Key words: "in this scenario"
|
| Is Cloudflare using an as yet unshipped API as part of
| DDOS protection?
| deadbunny wrote:
| You're conflating a downside of using Chrome and the reason
| they think Cloudflare blocked them.
| factormeta wrote:
| seems like the author mentioned that in FireFox disabling
| "privacy.resistFingerprinting" worked. So looks like Chrome
| by default is allowing the server to collect
| Fingerprinting. If cloud flare is using that, then it is a
| big red flag.
| waithuh wrote:
| Of course they are. Thats the whole point of the
| 'Integrity Check'. Besides, almost every website you
| visit collects your fingerprint nowadays.
| tmpX7dMeXU wrote:
| No. And there's still the central issue of the author
| really hand-waving the specifics of their accusations about
| Chrome. It really seems to come down to "Google bad".
|
| To be clear, I don't even use Chrome, in part because
| "Google bad". This just isn't intellectually honest.
| afavour wrote:
| > So I naturally tried to access the same page in Google
| Chrome to see if I'd still be blocked. Thankfully, I
| wasn't.
|
| > But of course I wasn't because Chrome doesn't have the
| same privacy- and security-enhancing designs
|
| Maybe I'm missing something but it seems the conflation was
| by the article author, not me?
| jbdigriz990 wrote:
| When I started having this problem logging into a certain
| credit card co.'s website beginning with about Firefox
| 105.0.2 on Fedora 38, I was told by their apparently
| outsourced customer service that I had to use Chrome, which I
| don't have installed there and couldn't try. Yeah, they
| wanted me to use LogMeIn so they could fix the problem, too.
| Right.
|
| Firefox on Android was still working, though, loathe as I am
| to put passwords of any significance on my phone. Doesn't
| directly address your question, which I'd like to know the
| answer to as well.
| tmpX7dMeXU wrote:
| This screams nerd angst. Completely reasonable and expected
| response from customer support, and nothing to do with the
| fact that they're "outsourced" as you say.
|
| Back in the day, my university would load balance based on
| the browser being used. When results were released every
| semester, all you needed to do to switch to the fast lane
| instead of contending with other students for resources was
| jump on IE (or even spoof your user agent). None of this
| was public knowledge.
|
| I say this just to make the point that people do all sorts
| of weird stuff with web infrastructure. The inferences made
| in this privacy nut's blog post seem a bit off the cuff.
| edmundsauto wrote:
| Why would they load balance based on user agent? I can't
| think of a scenario where that was a reasonable solution.
| waithuh wrote:
| Maybe back when standarts where on shaky ground and
| different versions of the same content was made? I too
| cant see the performance advantage of it. Deprioritizing
| less mainstream browsers to mess with the nerds?
| LoganDark wrote:
| A third browser... like what? Chrome and Firefox are all that
| exist now, unless you have access to a Mac with Safari.
| waithuh wrote:
| its time to pull out lynx again.
| mdwalters wrote:
| My "third" browser is GNOME Web, however, I uninstalled it
| thanks to performance issues. I installed Chrome from
| Flathub, but with limited permissions, which I only use for
| cross-browser testing. My main browser is Firefox.
| ayewo wrote:
| Chrome will indeed divulge more information than other
| browsers but only on the condition that you have opted-in for
| such collection.
|
| "The Chrome User Experience Report (CrUX) provides user
| experience metrics for how real-world Chrome users experience
| popular destinations on the web. _This data is automatically
| collected by Chrome from users who have opted in_ , . . ."
|
| Taken from https://web.dev/crux-and-rum-differences/
| afavour wrote:
| It's not a real time API, though. It's an aggregated
| dataset available via BigQuery. I don't think Cloudflare
| could use it as part of DDOS protection except in very
| vague ways.
| jonatron wrote:
| Does anyone who knows about GDPR know if being blocked by a CDN
| comes under "Automated individual decision-making"?
| buro9 wrote:
| https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-re...
|
| That's interesting.
| CharlesW wrote:
| > _The next day, I tried accessing a web page internal to my
| company... [...] I couldn't get past a security check page
| because of issues in Cloudflare's software. [...] The silliness
| of it all is that I was on my work device the whole time, which
| was behind my workplace VPN._
|
| This seems more like an "IT department gone mad" problem than a
| Cloudflare problem. I'm surprised they'd rather switch to Chrome
| than submit a support ticket.
|
| Having used passkeys for a month+ now via macOS/iOS/1Password
| betas, I don't understand how they're related or the author's
| concerns. Couldn't you just replace "passkey" with "password" in
| all of their questions?
| tadfisher wrote:
| Passkeys have optional attestation payloads, which is basically
| what WEI is doing. Google in particular doesn't recommend
| requiring attestation except in corporate-security scenarios,
| but the fear is that banking and media sites will require
| attestation anyway, which locks users into whatever attestation
| mechanisms supported by the server; so basically Google, Apple
| and Microsoft.
| nullfield wrote:
| You know I knew there was going to be something I really
| didn't like about passkeys, and here it is
| tadfisher wrote:
| Yeah, unfortunately the point of passkeys is to replace
| multi-factor authentication. Usually you have a
| username+password as the primary factor, and a secret
| that's hard to copy and replay as a second factor (TOTP,
| non-resident WebAuthn credential/FIDO, SMS code). Passkeys
| replace the primary factor with a signed challenge, but the
| second factor is up to the authenticator (such as
| biometrics). WebAuthn relying parties verify that the
| authenticator is locking the primary factor behind the
| second factor, and they do that with attestation.
| Animats wrote:
| Suing Cloudflare for interference with contract[1] might be an
| option. Cloudflare is not protected against lawsuits by some
| EULA, because the outside user has no contract with them. They're
| a third party in the middle. Talk to a lawyer.
|
| Most contract law lawsuits are settled out of court. The great
| advantage of suing someone is that you get past the low-level
| customer support people and talk to someone who's authorized to
| settle.
|
| [1] https://www.lodhs.com/blog/interference-with-contractual-
| or-...
| HumanOstrich wrote:
| Can you provide examples of people suing Cloudflare to get
| around being blocked from accessing certain websites?
| skybrian wrote:
| Fingerprinting is probably load-bearing for captchas and other
| anti-fraud stuff that many Internet services and businesses
| depend on:
|
| https://xkcd.com/2347/
|
| It should be replaced with something better. Unfortunately all
| attempts to do something better get attacked by people who don't
| realize that you can't just get rid of it, or important things
| will break.
| superkuh wrote:
| This has been my experience for a handful of years but of course
| it's getting worse. In the past I'd just be getting blocked from
| access commercial websites or applications and things I didn't
| really need. But in the last few years many scientific publishers
| have put all their content behind cloudflare walls. Pretty much
| my only hope of being able to read a paper these days is that it
| came out long enough ago to be on sci-hub _or_ they published the
| pre-print on arxiv /bioarxiv/etc. Once arxiv goes behind a
| cloudflare I don't know what I'll do.
| parasti wrote:
| Insanely enough, Cloudflare sometimes puts these pages in front
| of API endpoints, as if that JSON were for human eyes only.
| ricardo81 wrote:
| Boils down to gatekeepers doesn't it.
|
| Unfortunately there's also bad actors on the web (and the
| definition of bad varies). I understand reasons to try centralise
| the removal of that so called bad, but obviously a central group
| deciding on the 'bad' just isn't democratic.
|
| Ironically when chatgpt mentioned their UA on a web page the
| other day, users were presented with an anti-bot challenge.
| gsich wrote:
| "Checking if the site connection is secure" what a blatant lie.
| If you can read this you already have a TLS connection.
| derefr wrote:
| If you've ever tried to take apart Cloudflare's various session
| cookies, MITMed scripts sent for "high integrity" pages (or when
| in "super bot-fight" mode), etc., you'll have observed that it's
| basically running a web-worker to heuristically do browser-
| integrity checking. That is, Cloudflare is trying to run a series
| of tests that real browsers operated by users pass, but which
| headless browsers operated by bots will fail.
|
| These range from pretty simple things that check that the browser
| is actually a browser rather than a raw HTML parser (e.g. "draw
| an image on a <canvas>, export it to PNG, hash the PNG, compare
| to an expected result"); to things that check for low-effort
| headless-browsing techniques like the one you get by default
| using Puppeteer in a Lambda/Cloud Function (e.g. "do we have the
| weirder fonts you'd expect to exist on a consumer OS, but which
| these default batteries-included container images don't bother to
| bake in"); to things that work really hard to detect the "scent
| of humanity" _through_ the browser (e.g. "before the user
| activated the integrity-check prompt, did we record a sequence of
| 'extraneous' mouse movements and key events that look like a
| human making individualized mistakes on their way to completing
| the form, and _don 't_ look like a recorded capture of such
| similar to other ones we've seen recently.")
|
| If you're getting caught in a verification loop, it's because
| you're using a browser or device or extension that
| obscures/disables enough of these heuristics that Cloudflare
| can't get _proof positive_ that you 're a person rather than a
| bot -- and so, under whatever settings the site-owner has it set
| at, it will just keep trying to get that proof, rather than
| telling you you've failed and been blocked. (Why? Because telling
| a bot they've failed tells them that they should stop trying
| something that's not working and instead -- in the words of Star
| Trek technobabble -- "rotate their shield frequency" before
| trying again.)
| johnklos wrote:
| How does that explain blocks that happen to less common
| browsers and/or less common platforms?
| derefr wrote:
| Heuristics are about optimizing between false positives and
| false negatives.
|
| Many headless-browser stealth techniques involve rotating
| between the signatures and reflected metrics of real -- but
| niche and/or ancient -- User-Agents. (For some reason, the
| developers of these stealth systems think that variety beats
| commonality. Maybe it makes sense if they're specifically
| trying to overcome Apache mod_security's signature-based UA
| blocking or something.)
|
| It turns out that when you actually see one of these UAs in
| your server logs, it's far more (99.99%) likely to be a
| stealthed bot that picked that UA out of a bag, than it is to
| be an actual niche/ancient UA.
|
| In the case of the niche UAs, this is a tragedy of the
| commons.
|
| In the case of the ancient UAs, though, there's no downside
| to blocking them entirely -- because if the traffic is going
| through Cloudflare at all, then you're already requiring of
| the client a minimum version of TLS that the real old UAs
| can't even speak. So the _only_ things actually saying they
| 're that old device -- but managing to get through an HTTP
| request at all -- are stealthed bots.
| runnerup wrote:
| > Many headless-browser stealth techniques involve rotating
| between the signatures and reflected metrics of real -- but
| niche and/or ancient -- User-Agents
|
| I work in this space and we just use fingerprints we
| collect from actual users over the previous month. The
| hardest work is: 1) reverse-engineering the javascript of
| the CAPTCHA/fingerprinting solutions so that we can collect
| and encapsulate the fingerprints correctly in a way that
| looks native to cloudflare/recaptcha/etc, and 2) Training
| AI models to solve the captchas well enough.
|
| Sounds like most of the people you _catch_ are using
| ancient user-agents. But I doubt most of the people you
| _want to catch_ are.
| derefr wrote:
| I mean, when I'm talking about stealth techniques, I'm
| not talking about what I'm seeing in my own server logs,
| but rather about what I find built into various bits of,
| ahem, "anti-detect software for affiliate marketing" tech
| that I dredge up from fraud Telegram groups, carding
| marketplaces, etc. that the attackers I _do_ catch seem
| to frequent. (Gotta stay five steps ahead!)
|
| I suppose there are verticals where the data is _so_
| valuable, and the garden around it _so_ walled-in, that
| you could build a whole IT business with a custom
| scraping stack just around extracting that data to then
| resell it. (I presume that 's the business you're in.)
|
| But for most verticals, the "attackers" you'll see in
| your logs aren't people building a data-broker business,
| and so aren't building their own secret-sauce anonymity
| from scratch; rather, they're end-users who want to do an
| end-run around your rate-limits, commit promotion fraud,
| etc., and so want to buy _anonymity as a product_ ,
| script-kiddie style. And "anonymity as a product", sold
| publicly (rather than through high-value contracts) tends
| to suck. It's script-kiddies buying from script-kiddies,
| with no real engineering in sight.
|
| > I work in this space and we just use fingerprints we
| collect from actual users over the previous month.
|
| Are you sure you're not in a citogenesis cycle? How sure
| are you that some of those "real users" aren't your
| peers' stealthed bots, who in turn picked up those
| fingerprints from unknowningly observing other stealthed
| bots in _their_ logs, who...
| runnerup wrote:
| > Are you sure you're not in a citogenesis cycle? How
| sure are you that some of those "real users" aren't your
| peers' stealthed bots, who in turn picked up those
| fingerprints from unknowningly observing other stealthed
| bots in their logs, who...
|
| Doesn't matter, they work (at least when used in
| combination with high-quality proxy IP's). If they
| stopped working I'd do something else. We only apply hard
| science when absolutely needed, otherwise it's mostly
| wire and duct tape holding things together -- ruthless
| focus on creating business value.
|
| We definitely only sell this via high-value contracts, so
| you're probably mostly correct there. Though puppeteer-
| stealth deserves at least a quiet shout-out for not
| completely sucking.
|
| That said, we do pay attention to a lot of the research
| in the field, even if we only apply the absolute bare
| minimum needed to create business value. Eric Wustrow[0]
| at UC Boulder does really, really good work in an
| adjacent space, and we've found some his papers/software
| to be helpful, as well as those of some of the colleagues
| he works most closely with. I don't think he'd love our
| applications of his research, but our technological needs
| dovetail well with the needs of the anti-censorship
| research that he works on.
|
| If you were interested in the degree of "citogenesis", I
| think that's something that academic researchers like
| Wustrow et. al would be very well-positioned to
| investigate. Highly recommend any of their papers, they
| make front page of HN surprisingly often.
|
| 0: https://ericw.us/trow/
| megous wrote:
| From the PoV of the user it just looks like the website is
| broken. No exception in the console, contant reload looping of
| the same page, etc.
| alex7734 wrote:
| > Why? Because telling a bot they've failed tells them that
| they should stop trying something that's not working
|
| In my humble opinion if your bot is stuck in a CloudFlare loop
| for 10 minutes that's a pretty strong signal that something's
| not working...
| derefr wrote:
| Keep in mind that a bot will be picking some permutation of
| its stock library of UA+metrics info, _and_ generating truly-
| random values for other more continuously-valued parameters
| (e.g. timing between actions), to try to find a combination
| that satisfies a backend integrity-check.
|
| A "try again" just means "you haven't succeeded yet." If
| that's all you get, you're getting zero bits of new
| information -- so you can't _do_ anything other than to
| assume it was your timing that looked weird, and keep trying.
| (And you might be dealing with even more noise, e.g. trying
| to have the bot calibrate itself toward a very low human-
| tuned request rate limit, where above-rate-limit responses
| look no different than integrity-fail "try again"
| responses.)
|
| Suddenly getting a (maybe permanent) hard-fail, meanwhile,
| means that you said something the integrity-checker _really_
| didn 't like.
|
| Presuming you have a lot of IP addresses to send requests
| from, you can then do many experiments to bisect the
| difference between a hard-fail and soft-fail, and use that to
| blacklist values from your UA+metrics library. It's free
| entropy!
| shadowgovt wrote:
| While I'm inclined to agree, it's an old rule-of-thumb to
| give a potential attacker as little information as possible
| so they have to do the legwork to get un-broken.
|
| (I yearn for a world where auth challenge failures give
| proper error messages so I can figure out why my regular,
| human-used authentication channels aren't working).
| w0ts0n wrote:
| Users in Egypt are unable to visit my Fitness website
| https://musclewiki.com
|
| Cloudflare is a huge part of the internet. Often they won't
| respond and it appears that for whatever reason, their IP range
| is blocked in Egypt. We probably get 10 support emails per week.
| I contacted Cloudflare and they simply said there is nothing they
| can do.
| Temporary_31337 wrote:
| If you don't want to stop using CloudFlare or need a temporary
| solution ask your users from Egypt to use VPN- many already do
| as they come across similar problems for other services
| elashri wrote:
| Egypt do block VPNs in much more aggressive way than
| Cloudflare. Also this is my first time to hear that
| cloudflare is blocked in Egypt. People will even complain
| that they cannot connect to their cooperate VPNs.
|
| Blocking cloudflare ip addresses means that half of the
| internet wouldn't be accessible from Egypt. its closw to
| blocking port 443 because some people use DNS over https.
|
| disclaimer: I'm Egyptian living in the US.
| w0ts0n wrote:
| Have someone in Egypt look at our site. Its been blocked
| for about 3 months.
|
| Apparently there is a pool of IP's that Cloudflare use for
| their CDN and some of them are blocked in Egypt. If you are
| unlucky enough to be one of the websites that is using that
| IP, it's blocked. Apparently they rotate them, but I
| haven't seen it yet, so chances are, when they do rotate
| them, more will be blocked.
| warrenm wrote:
| Sounds like you should stop using Cloudflare
| hiatus wrote:
| I'm surprised to hear that. I actually used Cloudflare Tunnel
| to connect to a corporate intranet about 8 months ago while in
| Egypt, not sure if things have changed though.
| delfinom wrote:
| Yea I noticed the same thing for awhile. Cloudflare actively
| blocks non-Chrome browsers.
| tracker1 wrote:
| https://addons.mozilla.org/en-US/firefox/addon/privacy-pass/
| luuurker wrote:
| I use Firefox (stable and dev) and Waterfox, and Cloudflare
| doesn't block me. My settings are close to the defaults though,
| I don't enable things like privacy.resistFingerprinting.
|
| For a while I did notice that when using certain IP blocks,
| they would show me more captchas when using Firefox than when
| using Chrome, but I haven't had that problem in a while.
| rubatuga wrote:
| Actually cloudflare doesn't have access to your MAC address so
| it's a bit more difficult to attest that you are a legitimate
| user.
| adammartinetti wrote:
| Hi there, I'm the PM for Cloudflare's challenge platform. I'd
| love to look into what the cause of the problem is, so you don't
| see these difficulties.
|
| > Cloudflare detected the high frequency of requests and denials
| (but not their faulty loop that caused this pattern of requests,
| of course), and tagged my browser as suspicious.
|
| I can tell you at least that we don't penalize users for this
| looping behavior, so this wouldn't cause us to see your browser
| as suspicious. I hope we can dig into this more and uncover the
| cause of the problem.
|
| Personally, I'm a big Firefox user, and this isn't behavior I
| see. If there were a widespread Firefox wide issue, automated
| alerts would trigger and we'd consider this a critical incident.
|
| You can drop me an email at amartinetti at cloudflare if you're
| interested in troubleshooting.
| waithuh wrote:
| I dont know which type of Firefox you use, but any reasonably
| tuned browser (in the privacy sense) fails your systems. I
| literally didnt have a single instance of passing them without
| handing over a pixel perfect fingerprint.
| adammartinetti wrote:
| Would you be able to send me a rayID of a failed challenge so
| I can take a loop? It sounds like you can use
| https://gitlab.com/users/sign_in to generate one.
|
| You can either reply in the comments with the ID (no PII), or
| email me at amartinetti at cloudflare.com and I'd love to dig
| into it.
|
| We're building Turnstile because we want to make challenges a
| better system than CAPTCHA. It sounds like for you it's
| worse, and we want to fix that.
| stebalien wrote:
| Not OP, but GitLab always cycles for me on LibreWolf, even
| with "enhanced tracking protection" turned off. It's likely
| because I disable WebGL?
|
| 7f3b42d2bee22efb
| baq wrote:
| What are cloudflare's plans regarding browser attestation?
| jacoblambda wrote:
| FWIW I see this with Firefox when I route my traffic through
| ProtonVPN.
|
| It could be caused by someone else's bad behavior on the VPN
| but I'd hazard a guess that it's more than that.
| adammartinetti wrote:
| Do you see the challenge and then you're able to pass? Or
| does the challenge loop forever?
| jacoblambda wrote:
| I see the challenge almost always and most of the time it
| passes after I manually interact with it but I'll get a
| looping challenge every once in a while and it persists
| until I change VPN servers.
|
| I don't think I've seen it for a week or two now but I've
| certainly encountered it in the past for spans where it'd
| occur at a frequency of maybe once every two or three days
| and then go away for a while.
| jrockway wrote:
| I've definitely seen this from time to time. I used to work
| for an ISP and we would occasionally, in the office, get
| "Your system is sending too many automated requests" from
| Google. Usually one of our customers had gone off the rails
| with some sort of amateur scraping, but this was always a
| pain to debug. I think we talked with Google's NOC and just
| had our rate limit increased or something like that.
| stavros wrote:
| Also anecdotally, I use Firefox and I haven't noticed an uptick
| in the amount of CAPTCHAs I need to solve. I don't even see the
| "connection secure" page.
|
| Could it have something to do with that ticket extension I'm
| using (Privacy Pass, looks like it's called)? I don't know if
| it does anything.
| greggyb wrote:
| I see lots of challenges on Firefox, but I attribute this to my
| use of container tabs. Is this a reasonable expectation?
| mikeravkine wrote:
| I have noticed that on StarLink some sites behind CF go into
| "prove you are human" loops that are impassable.
|
| What causes such loops? Just a challenge over and over.
| 0x_rs wrote:
| > Just a challenge over and over.
|
| It must be intentional. Not unlike the endless loop of
| frustratingly slow-fading reCAPTCHA challenges that don't go
| anywhere. The user gives up after some time, but doesn't see
| any explicit error or page blocking their access. I imagine
| it must be quite effective.
| waithuh wrote:
| in my opinion, the admins do not enable the option to ban
| lower trust users (or set the threshold high), so CF tries
| over and over again instead of doing that.
| baq wrote:
| There's a patent for that. The catchpa loop is basically a
| honeypot for bots... and privacy-conscious legitimate folks.
| shiomiru wrote:
| The cause of the problem is that your software is faulty by
| design.
|
| 1. IP addresses are to be used for packet routing. Certainly
| not for assigning "behavior scores" to users in the background.
| IP addresses say nothing about your visitors, my IP address
| could have been a complete stranger's IP address yesterday.
|
| 2. Deciding who can access half the web based on their TLS
| signature achieves nothing in the long run except reinforce
| browser monopolies, and goes completely against the spirit of
| the open web.
|
| I guess now I have to use Chrome for browsing the web from
| home. Yes, I do run a crawler-like bot as a hobby project, I
| got what I was asking for. (Funnily enough, it still works if I
| just emulate Chrome's TLS signature). But I also have friends
| who have done absolutely nothing of sorts (no technical
| skills), and still got caught up in this latest ban wave.
|
| Let's be honest here. Your service has likely caused millions
| of people harm who one day to the other are suddenly blocked
| from half the WWW - not just nerds, who can get around that one
| way or the other, _real users_ who just got unlucky and now are
| potentially blocked from accessing websites required for their
| daily lives (welcome to the 21th century). This is not a one
| time problem, it has been going on for years; this time it just
| came too suddenly for too many people. And this kind of harm is
| a logical conclusion to the heuristics you use for determining
| who can view a website.
|
| Never mind that it's ridiculous how a single company from
| outside my country has the power to decide on whether I can use
| the web or not. That's kind of on website owners
| unconditionally giving this power to CF anyway.
|
| Now, allow me to return to purchasing proxies from shady
| sources for myself, so I can keep using Firefox. Thanks and
| keep up the good work.
| shadowgovt wrote:
| You're going by the specified, designed use cases of those
| technologies.
|
| Every spec is a three-edged sword: the spec, the intent of
| the spec, and the use of the spec in the wild.
|
| In practice, Cloudflare does a pretty good job on far-more-
| than average of gluing together some heuristics in an
| unspec'd way to filter traffic. It sucks because you can't
| plan around it, but that's rather the point because the
| malicious actors are trying to plan around it also.
|
| (ETA: Hacker News rate-limited this post. In theory, I could
| have set up a sock-puppet to try and work around that, but
| then they would catch that too and I'd be out two accounts.
| So I just waited out the limit. Measure and counter-measure.
| ;) ).
| dcow wrote:
| I sympathize with your frustration, but you also have to
| admit that Cloudflare is tasked with an impossible problem:
| from a sea of requests, identify those that are coming from
| robots that are disguised as humans.
|
| So there is no perfect solution. You can't use strong
| identity because a user can share their identity with a
| robot. You have to use a crapy heuristic that only works most
| of the time (or tell site owners it's an application layer
| problem and use this SASS solution to solve the problem).
|
| I mean you admitted that you run a crawler. Cloudflare has
| detected that you run a crawler and has wants you to prove
| that you're human to access sites on their network. It
| actually sounds like their product worked.
|
| In any event, there should probably be better regulation
| around how this blocking is handled so that users aren't
| being unjustly blocked. If you want to run a crawler, how do
| you do it ethically so that you aren't targeted and your
| traffic blocked? If Cloudflare blocks you from accessing one
| site should that block extend across their whole network? How
| long should it last? How do you appeal the block if
| Cloudflare's heuristics falsely block you? If you're in a
| life and death situation and need immediate access to medical
| information and Cloudflare unjustly blocks your access and it
| causes harm, who's at fault? Etc.
| ipaddr wrote:
| Why are humans only allowed and shouldn't we be proactive
| and accept robots as equals now. We have a history of
| prejudice against groups and we seem clueless that we are
| heading their again.
| lwansbrough wrote:
| Have you ever run an open resource with significant
| traffic before? People are absolutely abusive with their
| use of public websites and APIs. "This is why we can't
| have nice things" is as relevant as ever.
|
| Cloudflare provides a vital service that solves a real
| problem that breaks non-pragmatists brains.
| dcow wrote:
| Of course I'd agree that if a robot is following the
| rules and behaving indistinguishably from a human but
| maybe just a little more quickly, then it shouldn't be
| pre-judged (and our detection should accommodate). But
| here we're talking about robots without agency being e.g.
| used in botnets to abuse services, or otherwise not
| following the rules.
| lambda wrote:
| And even when it doesn't block you completely, it delays
| website loading, makes you jump through frustrating captchas,
| etc.
|
| It's probably third in the list of frustrating web behaviors
| in the past couple of years (behind GDPR popups and
| registration/paywalls that seem to have gotten much worse
| recently).
|
| And somehow there are some sites that I get CF delay walls on
| every time I visit.
|
| This feature is utterly broken for a good web experience; it
| pushes users away from sites which use it.
|
| Every time that "checking your browser" page comes up for a
| legitimate user should be considered a failure. Sure, it can
| maybe happen a few times in a thousand, but the feature is
| utterly broken if it comes up every time I visit the same
| site from the same browser not in private mode.
| llm_nerd wrote:
| It's worth noting that the websites that you are visiting
| _chose_ Cloudflare, and have enabled the features that
| irritate you. They have browser integrity enabled, have bot
| protection enabled, maybe turned the security level up
| (gitlab famously is a nuisance because they lean heavily on
| Cloudflare for protection). Sometimes they 've wholly
| barred VPNs or entire geographic areas! And that is
| entirely the decision of website operators, and note that
| they did all of this before Cloudflare came along.
|
| Cloudflare's customers are website operators, not you the
| end user. Those website operators seem pretty pleased with
| the service, so clearly they are doing a good job for the
| people who they are building it for.
| warrenm wrote:
| And every Cloudflare customer is a company I won't do
| business with (unless there is absolutely no way around
| it)
|
| Cloudflare is running the single biggest, most blatant
| man-in-the-middle attack in history, and far too many
| people are happy about it
| sophacles wrote:
| In what way is it an attack? (I know what a mitm is, I'm
| not asking you to explain that - I'm pretty conversant in
| the concept of a proxy, I'm asking you to explain why
| it's an attack specifically)
| warrenm wrote:
| they block and/or slowdown vast swaths of the internet
|
| if that's not an "attack", I don't know what is
| sophacles wrote:
| I don't get it. They offer a service that that people
| choose to sign up for and take active steps to use. I
| don't see how that's an attack. Honestly I'm still trying
| to understand who is being attacked.
|
| Like is it an attack on the site owner - are you saying
| cloudflare is extorting them or something? That seems
| unlikely but I agree that would be a form of attack... it
| also doesn't seem to be what you're saying.
|
| Is it an attack on the user of the website because the
| website owner successfully denies visitors it does not
| want? Does that mean that login credentials are a form of
| attack too? Would an on-prem load balancer or WAF that
| dropped all traffic from a region or matching patterns
| still be an attack?
|
| It just doesn't make sense that it's an attack.
| cutler wrote:
| Agreed. The same goes for the 3rd party "data privacy"
| popups which simply hide a long list of opt-outs several
| layers deep in a Vendors list. I refuse to use such sites
| and I let them know by email.
| cutler wrote:
| Why not take a screenshot of the CF error and send it to
| the website owner? It would freak me out if I thought a
| significant number of my website's users were being blocked
| by CF.
| brigade wrote:
| The work needed to _maybe_ get it past outsourced
| customer support is not at all worth the effort for any
| site I don't actually need to use
| WheatMillington wrote:
| You're being a little dramatic. It's incredibly unlikely that
| millions of innocent users have been blocked, and unless you
| have data to the contrary you shouldn't make such a claim.
|
| You know what else is harmful to the concept of the open
| internet? The enormous malicious botnets and other endemic
| problems that require a solution like CloudFlare.
| waithuh wrote:
| I dont get your second point. Two things can be harmful to
| the open web at once. CloudFlare is definitely not taking
| the right approach at it, which damages the open web
| alongside botnets. Also, botnet owners are for some reason
| extraordinarily nerdy and smart so they probably will find
| a way to fool CF every other month. Its a cat and mouse
| game for them while actively harming everyone else both
| with their botnets and the increased aggressiveness of CF
| caused by their incorrect solution
| ipaddr wrote:
| For every one user that makes their way on here and finds
| and posts here on this thread probably represent 1,000,000
| plus normal users
|
| An open web is open for everyone/thing not just classes of
| beings you select. Bots and users can both be malicious and
| both can be positive.
| tenacious_tuna wrote:
| I agree with the premise that most people don't know how
| to identity or visibly complain about a given technical
| problem, and so an HN thread with N anecdotes about the
| problem likely corresponds to N * F actual amount of
| real-world incidents, for some value of F > 1.... but
| claiming it's a factor of a _million_ without any backing
| evidence is absolutely an overreach.
|
| > An open web is open for everyone/thing not just classes
| of beings you select. Bots and users can both be
| malicious and both can be positive.
|
| This I agree with. I run an archiver ~monthly on a subset
| of my month's browsing history, and I'd hate if that got
| me blacklisted from Cloudflare-backed sites for a benign
| purpose. (See also the idea of remote attestation)
| ricardo81 wrote:
| > It's incredibly unlikely that millions of innocent users
| have been blocked
|
| Is there a 'town square' where we can talk about being
| presented captchas and similar things from 3rd party
| intermediates.
|
| I think it's incredibly likely that millions of hours have
| been wasted on such challenges.
| amatecha wrote:
| On that note...
|
| https://www.folklore.org/StoryView.py?project=Macintosh&s
| tor...
|
| "Well, let's say you can shave 10 seconds off of the boot
| time. Multiply that by five million users and thats 50
| million seconds, every single day. Over a year, that's
| probably dozens of lifetimes. So if you make it boot ten
| seconds faster, you've saved a dozen lives. That's really
| worth it, don't you think?"
|
| Imagine if people still thought like this about computers
| and software.
| ricardo81 wrote:
| Yes. And cookie splash screens! I admire GDPR's intention
| but hasn't it been a massive human time sink.
|
| Not to take away from your point, just that it's all a
| hindrance.
| grishka wrote:
| Those can at least be blocked with ad blockers and/or
| disabling JS.
| rvnx wrote:
| @adammartinetti : maybe you could consider developing a
| new product where you display a GDPR consent banner
| _once_ , and then these settings apply to all Cloudflare-
| proxied websites (by passing this consent information as
| an additional header to the proxied site)
| meltedcapacitor wrote:
| Sounds inferior to the "no cookies no banner" solution.
|
| The GDPR does not mandate gratuitous and pointless
| personalised spying, which is the only case that requires
| consent. Normal operations (say a shop collecting payment
| details and shipping address to fulfil an order) do not
| require a consent banner.
| ulucs wrote:
| That's more on the websites that track your personal data
| for non-essential purposes. No tracking means no banners
| are necessary.
| ricardo81 wrote:
| Finer points, my point is just about people wishing to
| view web pages.
| thaumaturgy wrote:
| Data point of N+1, but I haven't been able to place online
| orders at Petco for about a year now because they use some
| Cloudflare feature that hates my browser + home internet
| connection. Other Cloudflare-proxied sites seem unaffected,
| and I'm not doing any botting/crawling, nor do I have any
| IoT devices on my home network. There's not enough
| information provided to be able to do any substantive
| troubleshooting.
|
| This became irritating enough that it caused two side
| effects: (a) I stopped shopping at Petco, and (b) I moved a
| pile of sites off of Cloudflare and stopped recommending
| them, and now sometimes recommend against them.
|
| Cloudflare is still a good, quick, cheap option for sites
| that receive unusual volumes of malicious traffic, so I'll
| still recommend them as a solution to some problems. But,
| they're not a good default.
| sophacles wrote:
| So you're mad at Cloudflare because Petco enabled a
| feature that blocked you? If Petco had developed
| something in-house that blocked you, would you be mad at
| the compiler?
| thaumaturgy wrote:
| ...no, I've changed my recommendations for Cloudflare
| because it may prevent ordinary users from using a site,
| and insufficient information is provided for
| troubleshooting purposes, and those users are likely not
| going to go to extraordinary lengths to report the
| problem. Even if they do report it, the site won't be
| able to troubleshoot it either. So, if you don't need it,
| you're probably better off without it.
| boneitis wrote:
| Right. It feels silly to point out specific things when just
| about everything about these verification checks deployed by
| every megacorp throws you into a universe of suffering.
|
| If there's a place to start, it would be with eliminating the
| infinite challenge loops. Bad enough that IP blocks get
| outright blocked. Bad enough that I have to decide whether or
| not that blurred sliver of the edge of the wheel+shadow
| constitutes being part of the bicycle. Not to mention the
| humanitarian betrayal of the absolute highest form to farm
| the free human labor to train AI models when they are simply
| trying to browse the $#@%ing internet.
| NicoJuicy wrote:
| This is a very nasty comment. I was wondering if I could find
| some things that could lead to an exception.
|
| But I'm pretty sure that millions of users aren't using stuff
| like w3m pager (
| https://news.ycombinator.com/item?id=34175754 )
|
| We're all technical here, we are the edge cases. We use
| exotic software / combos. Let's not get carried away here
|
| The PM of cloudflare uses Firefox, I sometimes use Firefox
| and I don't notice any difference ( concerning this use-case
| at least).
|
| If you want help, perhaps describe the actual use-case that
| is blocking you to him. He shared his email.
|
| - country
|
| - software ( VPN, ... )
|
| - browser
|
| - OS
|
| - traceid
|
| - ...
|
| Either way, buying shady proxies as you mentioned is already
| a warning flag.
|
| While using Firefox is not :)
| samcat116 wrote:
| > Let's be honest here. Your service has likely caused
| millions of people harm who one day to the other are suddenly
| blocked from half the WWW
|
| If this was true, Cloudflare wouldn't be a good product used
| by a lot of sites.
| bradly wrote:
| Anecdotaly... I use Firefox and have noticed the Cloudflare
| interception pages verifying I'm human appearing more often
| recently. Usually it is all automatic and isn't a big deal, but
| I have noticed a increase in how often I see these the past
| week.
| stjohnswarts wrote:
| I use a VPN 99% of the time and I definitely see more
| cloudflare on my systems (as opposed to when I'm bored and
| surfing in the library) and see the "checking" screen. It
| does seem to be a lot better than last year though when it
| seemed I was getting a captcha every time I turned around
| JakeAl wrote:
| I have the same issue and often just reconnecting using a
| different node/city resolves it Annoying as hell, but
| sometimes the problem is the nodes in the US show as being
| in EU for some reason.
| [deleted]
| lovegrenoble wrote:
| Same for me
| [deleted]
| amatecha wrote:
| On one of my machines I run OpenBSD with Firefox with "Strict"
| privacy settings and "privacy.resistFingerprinting" enabled.
| There are so many websites I can't access, I get a straight-up
| 403 Forbidden page because CloudFlare has decided I am not
| trustworthy. I mean like pretty big companies like DigiKey, Home
| Depot, Canadian Tire, etc. I simply cannot use their websites, or
| I can load the initial page but then the API calls that provide
| the functionality all fail with a 403. DigiKey did something to
| unblock me and I can use their site again, but I know it's just a
| matter of time before it happens again. It's also a frequent
| problem on smaller sites that are simply using CloudFlare , and I
| never know when I'm going to be blocked from a site arbitrarily.
| It's especially egregious when it's a plain old text-based site
| like hamuniverse.com , or a small independent vendor like
| digirig.net ...
|
| This is one of the things that makes it so clear to me that the
| web is diverging into two, one that is the "clean walled-garden
| capitalism web" and the continuation of the original web that was
| open, freely-accessible and built around sharing and knowledge.
| bradley13 wrote:
| Anecdote: For my programming classes, one example I use is a
| simple browser. It doesn't do CSS or Javacript, so display is
| primitive, but it works.
|
| On some sites. Many sites, especially the big ones, see that it's
| an unknown browser, and refuse to send content. Probably they
| think it's a bot. But even if it were, what's wrong with bots, as
| long as they're well-behaved?
|
| What kind of closed web have we let the megacorps build?
| unmole wrote:
| Playing the devil's advocate: Why shouldn't a server get to
| decide which clients it wants to talk to?
| acedTrex wrote:
| I mean, the site gets to decide if it will service a request.
| theres no requirement for a service to respond to everyone.
| tamimio wrote:
| I was going to assume that the corp VPN is the reason as maybe
| someone is abusing that connection for something else and it's
| getting flagged, but the fact that the site worked using chrome
| says otherwise.
|
| Will using chromium for such cases work while having Firefox for
| the rest of sites?
|
| And what's cloudflare alternative that provides similar services
| for free including traffic analysis?
| warrenm wrote:
| [flagged]
| samcat116 wrote:
| The fact that this person thinks Cloudflare has their MAC address
| leads me to believe they shouldn't be speculating on the
| "implications for the web"
| adamgamble wrote:
| One thing that sometimes gets lost is site owners that use
| cloudflare have sort of global options for how paranoid they want
| to be, then they can make specific WAF rules that can be as
| granular and aggressive as they want. So at least in some cases,
| cloudflare gets blamed for website owners setting really
| aggressive rules. The effect on the end user usually looks
| exactly the same.
|
| Case in point, I set a waf rule that blocked all non verified bot
| traffic from several big datacenters (Google cloud, OVH, digital
| ocean, etc). That turned out to be a mistake because a lot of
| corporations were routing their traffic through those ASNs for
| some reason. Now they're blocked. They could have gotten pissed
| out cloudflare, the error page looks the same, but it was really
| misconfiguring it.
| ChrisArchitect wrote:
| Is there something more going on here like you're using some kind
| of blocker and it's stopping the captcha/security 'widget' from
| loading?
| nfriedly wrote:
| > Anyone who uses a de-Googled Android phone has to go to great
| lengths to ensure hardware attestation is working correctly [...]
| or else they can't using banking apps.
|
| I have a relatively Google'd Android running lineageOS. It passes
| SafteyNet on a fresh install, but even that isn't good enough for
| one of my banking apps (or netflix) - they both also perform a
| CTS Profile (Compatibility Test Suite) check and block me from
| using the app if they don't like what they see.
|
| I ultimately had to root the phone to be able to use my bank's
| app. Rooting allowed me to use a fake CTS Profile, and then
| because it was rooted, SafteyNet started failing and I had to
| install a bypass to work around that.
|
| Now everything works great, except OS updates un-root the phone
| and then "secure" apps stop working again.
|
| (Oh, and if you mention that you're rooted, the LineageOS folks
| will refuse to provide any support, even for unrelated issues.
| Making you choose between friendly help and a usable phone is
| probably the only thing I don't like about LineageOS and, to my
| view, the biggest break from it's CyanogenMod roots.)
| kozhevnikov wrote:
| > I temporarily disabled extensions. I opened a private browsing
| window.
|
| FYI When using Chrome, incognito window carries a lot of baggage.
| For issues like this use Guest profile as it doesn't include
| extensions, caches, storage, etc. Optionally do a Google search
| first to seed it with cookies.
| jsnell wrote:
| Doing a Google search will not populate any cookies Cloudflare
| could access.
| kozhevnikov wrote:
| Of course not, but it is less likely to trigger manual
| recaptcha on the site you're trying to visit in Guest mode.
| Operyl wrote:
| Cloudflare does not use Google's reCAPTCHA anymore, and
| hasn't for some time.
| fireflash38 wrote:
| Mind expanding on what you mean by baggage? Or linking to
| something to start research.
| lopkeny12ko wrote:
| Anyone else find it odd that the author's company-internal work
| intranet, which requires a VPN to access, is deployed behind a
| Cloudflare CDN? Why would anyone do this?
| MichaelZuo wrote:
| This is a bit tangential to the author's point but it does seem
| to indicate that IPv6 is mostly pointless for human users for
| exactly this reason.
|
| Since it's so much easier to hide behind a new unique address,
| compared to IPv4, that any service such as Cloudflare would need
| to be extremely aggressive in blocking to meet their internal
| metrics and customer advertised minimum thresholds.
|
| So much so that it actually costs more to use IPv6 then sticking
| with IPv4.
|
| I imagine the scenario described by the author would become more
| and more common as time goes on as more of the world's internet
| users becomes harder to distinguish.
| crote wrote:
| Not really.
|
| IPv6 doesn't allow you to easily get a new _completely random_
| address. You get a subnet allocated by your ISP, and you can
| use any address within that subnet. Rather than blocking a
| single IPv6 address, a service like Cloudflare can just block
| the entire IPv6 subnet prefix and get the same result as
| blocking an IPv4 address.
| MichaelZuo wrote:
| They obviously don't do it this way and implement more
| roundabout ways because it's not as simple or
| straightforward.
| p1mrx wrote:
| > any service such as Cloudflare would need to be extremely
| aggressive in blocking
|
| Cloudflare needs an algorithm to deduce the IPv6 prefix size
| controlled by a given entity, but the details of that algorithm
| are not obvious. You are jumping to the conclusion that they
| must be doing a bad job because the problem is challenging.
|
| IPv4 abuse detection is also challenging because of (e.g.) the
| prevalence of CGNAT with multiple users sharing an IP address.
|
| Which problem is harder? Which solution is better? I don't
| know, without a lot of proprietary data and analysis.
| tomschlick wrote:
| I would imagine they have a list of the default / max DHCPv6
| blocks that ISPs hand out. They are generally public
| knowledge so it would be easy for them to say Comcast hands
| out /56 so when blocking they block at that level for the
| ASN.
| p1mrx wrote:
| It probably makes more sense to derive that information
| from measurements. There are a lot of ISPs in the world,
| and allocation policies can change, even if you manage to
| find a person who knows what they are.
| johnklos wrote:
| You're basically saying this behavior is acceptable and should
| be considered normal and should be expected to become the norm.
|
| If you think IPv6 is mostly pointless, I think you're unaware
| of the fact that a significant majority of phones already use
| IPv6 most of the time they're on cellular.
| MichaelZuo wrote:
| > You're basically saying this behavior is acceptable and
| should be considered normal and should be expected to become
| the norm. If you think IPv6 is mostly pointless, I think
| you're unaware of the fact that a significant majority of
| phones already use IPv6 most of the time they're on cellular.
|
| Can you point to where I suggested that? Or did you misread
| the comment?
| warrenm wrote:
| He quoted you:
|
| >it does seem to indicate that IPv6 is mostly pointless for
| human users for exactly this reason
| MichaelZuo wrote:
| > He quoted you:
|
| > >it does seem to indicate that IPv6 is mostly pointless
| for human users for exactly this reason
|
| Huh? There is no quote in that comment:
| https://news.ycombinator.com/item?id=37051011.
|
| Unless you are referring to a different comment?
| warrenm wrote:
| I'm referring to _your_ comment that he quoted -
| https://news.ycombinator.com/item?id=37050359
|
| Maybe you're misremembering - but you wrote it :)
| MichaelZuo wrote:
| > I'm referring to your comment that he quoted -
| https://news.ycombinator.com/item?id=37050359 Maybe
| you're misremembering - but you wrote it :)
|
| 'johnklos' did not quote a single word from anyone in the
| comment I linked, nor a single word from my original
| comment you linked, let alone an entire phrase, and that
| was the only response he made from what I can see.
|
| Are you confusing him with a different HN user?
|
| You don't have to believe me, it's accessible to every
| passing reader right there in the comment chain...
| flyinghamster wrote:
| Last time I checked when I hotspotted my T-Mobile (US) phone,
| hotspot clients got only an IPv6 address, with 6to4 [edit:
| no, NAT64] translation used to reach IPv4 addresses.
|
| This breaks OpenVPN, which insists on both endpoints being
| one or the other.
| p1mrx wrote:
| T-Mobile uses NAT64. 6to4 is a (mostly deprecated) protocol
| for tunneling IPv6 over IPv4.
| zer8k wrote:
| Any time a large portion of internet traffic is controlled by a
| single source it brings problems like this with it. All
| cloudflare has to do is arbitrarily decide who and who can't use
| the internet and effectively their word becomes law. Like most
| things it starts with an innocent premise (e.g. "an easy way to
| stop bad actors") and ends up extended to any number of arbitrary
| things. Worse, the argument from privacy advocates rings hollow
| because defending privacy means you have to allow Bad People
| (TM). The average drooler using the internet cannot understand
| the nuance. Even in the most innocent of cases, a bad commit
| getting merged, can bring down the internet. It has happened
| before with Cloudflare.
|
| Companies like Cloudflare, Google, Meta, etc are the reason anti-
| trust law exists. Unfortunately, it appears there is no one with
| any power that is willing to use the laws for their purpose. The
| internet in 20 years will be nothing like we've seen before.
| That's not a good thing.
| NicoJuicy wrote:
| Everyone forgets that websites become almost unusable because
| of crawlers and bots.
|
| Website owners specifically choose for cloudflare to protect
| against this, it's not forced upon them by cloudflare.
| everybodyknows wrote:
| Website owner complains elsewhere in this topic of blocked
| users (country-specific):
|
| https://news.ycombinator.com/item?id=37050774
| NicoJuicy wrote:
| That's not a lot of information for such a complex subject.
|
| Eg. If you live in a dictatorship and use a VPN. You're
| traffic is together with a lot of people.
|
| The website owner can disable cloudflare their checks and
| that will leave their site unprotected. The choice of that
| is up to the website owner, no?
| mlyle wrote:
| It seems like a lot of people are pretending that because
| there's a contractual relationship between Cloudflare and
| many websites, that Cloudflare can't cause outsized harms
| to other parties.
|
| It really sucks for a third party to claim you're a bot
| and as a result you lose access to resources. And the
| potential for harm is only increasing.
|
| Worse still, the false positive rate appears unacceptably
| high (look at all the people here with substantial
| issues) and there's no recourse unless you get a highly
| voted thread on hacker news.
| NicoJuicy wrote:
| Tbh. There's a lot of technical folks, worldwide, here.
| That's a lot of edge cases in comparison.
|
| And their technical skills probably skews perception (
| exotic browsers, behavior, VPN, Tor, ... )
|
| As such, can you say, by indication, what percentage of
| web surfers "a lot" is? ( = Including non technical)
|
| I don't disagree that people can have problems. I'm just
| wondering how representative "a lot" in reality is.
| realusername wrote:
| And who defines is an allowed crawler? Is Google the only
| allowed company to crawl the web? Isn't that the definition
| of monopoly?
|
| Could anybody still create a new search engine nowadays?
| NicoJuicy wrote:
| A valid crawler is following robots.txt
| realusername wrote:
| That's not going to be enough to pass Cloudflare.
| NicoJuicy wrote:
| Based on what?
|
| https://developers.cloudflare.com/support/troubleshooting
| /ge...
| buzer wrote:
| https://developers.cloudflare.com/bots/reference/verified
| -bo...
|
| https://radar.cloudflare.com/traffic/verified-bots
|
| https://blog.cloudflare.com/friendly-bots/
|
| > At Cloudflare, we manually "verify" good bots, so they
| don't get blocked.
| gochi wrote:
| >and honor robots.txt.
| NicoJuicy wrote:
| Nice links!
|
| But cloudflare references robots.txt a lot ( which i
| mentioned before => to respect robots.txt)
|
| Additionally, they solve the authentication problem here.
| As a website owner that got bad crawlers ( that copied
| user agents), i just whitelisted Google's IP's and
| blocked all the other crawlers.
|
| It seems that cloudflare is actually fixing this problem
| and making competition for Google possible here.
|
| I recall an article that Bing actually circumvented
| robots.txt a bit, because site owners were only allowing
| Google and blocked all the rest => Bing. Which gave
| Google an unfair advantage ( searched for it, couldn't
| find it)
|
| Similar article to highlight the issue:
| https://www.fastcompany.com/90709672/the-little-known-
| reason...
|
| Our opinions about this seem to differ severely.
| Cloudflare actually enables good bots to start competing
| ( while respecting robots.txt).
| sschueller wrote:
| I don't agree. Maybe don't run WordPress that makes 100+ dB
| queries before first page load on some cheap $2 vps.
|
| There are all kinds of tools that you can easily deal with
| bots and the large DDOS your ISP can handle for you if you
| are willing to pay for it.
| crote wrote:
| > the large DDOS your ISP can handle for you if you are
| willing to pay for it
|
| That's exactly what people are paying Cloudflare for,
| because contrary to your local ISP they are actually
| _competent_ at blocking a DDOS attack.
|
| People use services like Cloudflare exactly because they
| don't want to spend a fortune on complex infrastructure
| just to deal with abuse. Even a mostly-static page running
| on a reasonably-specced server can easily be overwhelmed by
| an attack. Why spend $1000 / month on hardware when you can
| spend $100 / month on Cloudflare's protection?
| thedaly wrote:
| > That's exactly what people are paying Cloudflare for
|
| Cloudflare actually provides this service for free (for
| simple use cases at least).
|
| I don't know how to come down on this issue. On one hand,
| I am against the centralization of cloudflare and the
| risks that come with it.
|
| On the other hand, cloudflare allows almost anyone to set
| up a simple website and serve it to large numbers of
| people with very little resources/cost and advanced
| protection from DDOS attacks.
| tracker1 wrote:
| Similar mindset... also really intrigued with their
| developer tools as well. Workers, pages, D1, KV, etc. I
| was playing with a static site generator that deploys
| directly to a Cloudflare Pages setup, and it's lighning
| fast everywhere.
| NicoJuicy wrote:
| Psst https://ai.cloudflare.com/
|
| Note: light use-cases ( or should I say shared models?).
| Not heavy GPU tasks
| amadeuspagel wrote:
| Well, if some website you like uses cloudflare to block
| bots, maybe you can offer them some help to pay for these
| tools and to set them up?
| delfinom wrote:
| Hah, you know, most crawlers were fine. The only one that
| actively DDOSed websites was fucking Yandex. It doesn't
| respect robots.txt and it will actively fight against any
| rate limits by spawning connections on new IPs the moment one
| is blocked
| NicoJuicy wrote:
| ? I personally experienced many unbehaving crawlers/bots
| not respecting robots.txt and behaving like another user
| agent.
|
| Feel free to prove me wrong and disrupt cloudflare by only
| handling that use-case
| NicoJuicy wrote:
| Found an old reference of mine what I did to block
| crawlers/bots
|
| https://news.ycombinator.com/item?id=34101988
|
| > Had a lot of spammers with Russian language.
| Implemented expanding xml-bombs, Google Captcha, hidden
| input fields and a couple of other things against bots.
| But the block on the russian language was most effective
| ( and since I was dogfooding it, I didn't see the harm at
| the time. But it's out of scope at this very moment,
| yes).
| kayodelycaon wrote:
| Search engine crawlers are a subset of crawlers. Sometimes
| you're dealing with aggressive screen-scraping from
| competitors or various marketing tools.
|
| I've had to deal with these things easily bringing sites
| down.
| tracker1 wrote:
| Same, and when more than half your links are dynamic
| search results, it can pile on and really bring things to
| a crawl. I worked on a fairly popular auto classifieds
| website, and more than 90% of traffic was various
| scrapers, and some were definitely a burden. Worse, is
| that it doesn't show up in analytics as it's not running
| client-js... Ironically equally bad was when bing started
| scraping with JS and it skewed google analytics.
|
| If all we had to deal with were the users, wouldn't need
| nearly the spend on the site. Started manually blocking
| some of the worst offenders.
| shadowgovt wrote:
| The current internet is nothing like the internet of 20 years
| ago.
| robertlagrant wrote:
| > Companies like Cloudflare, Google, Meta, etc are the reason
| anti-trust law exists
|
| Only if Cloudflare stops you moving to a competitor.
| warrenm wrote:
| They [effectively] do prevent you from moving - they're
| guilty of vendor lock-in and running a passive, public man-
| in-the-middle attack platform (how many folks are using
| Cloudflare for DNS - either directly, or because their ISP
| is?)
| cortesoft wrote:
| Cloudflare does not have nearly enough market share to be an
| anti-trust concern.
| zer8k wrote:
| Cloudflare controls about 19% of websites. Of the websites
| that use a CDN, 80% of them use Cloudflare [0] (in a note at
| the bottom of link).
|
| [0] https://community.cloudflare.com/t/statistically-
| speaking-wh...
| antonvs wrote:
| Afaik, neither of those numbers legally constitute a
| monopoly and wouldn't qualify for antitrust action.
| warrenm wrote:
| They _should_
| [deleted]
| adrr wrote:
| You don't pay Cloudflare money nor are you customer. The
| customers are the sites that pay them money to protect their
| infrastructure. The customers have many choices in CDN/WAF
| providers and Cloudflare isn't even largest, Akamai is. Fastest
| growing CDN is cloudfront. There is healthy competition so why
| would anti trust laws apply?
| zer8k wrote:
| Cloudflare controls about 19% of websites. Of the websites
| that use a CDN, 80% of them use Cloudflare [0] (in a note at
| the bottom of link).
|
| [0] https://community.cloudflare.com/t/statistically-
| speaking-wh...
| adrr wrote:
| Traffic by bandwidth/number of users. Akamai powers Apple,
| Adobe, Microsoft, US government, Walmart, bunch of the
| major streamers.
| xmichael909 wrote:
| Cloudflare sucks, plain and simple, no idea why anyone uses it.
| So many better alternatives.
| aeyes wrote:
| For example? Especially when taking price into account.
| buzer wrote:
| It's also annoying how that check page ends up breaking page
| reload in Firefox. When Cloudflare redirects you back to the page
| it will happen via POST. This initial POST gets captured by
| Cloudflare, but if you reload the page that POST will go to page
| itself and there's pretty good chance it doesn't know what to do
| with that and just shows error.
|
| The only fix is to navigate back to page somehow, either by going
| to address bar and pressing enter (to navigate there again
| instead of reloading) or finding some link that points you back
| to the page.
|
| I wouldn't be surprised if those POSTs will end up banning you
| from some website since they "know" you shouldn't POSTing to that
| page so clearly you are evil bot trying to hack them.
| koito17 wrote:
| Increasingly more sites are getting stuck in a Cloudflare
| verification loop on my end. I use Firefox on the beta channel,
| and I do have a few privacy extensions and a heavily modified
| user.js. If you want to give in to the browser fingerprinting, I
| have found that enabling WebGL, enabling performance timing
| (wow), setting network.http.referer.XOriginTrimmingPolicy to 0,
| among other tweaks, helps me break out of the verification loop.
|
| In other words, if Cloudflare can't reliably fingerprint your
| browser, you are treated as a "bot" and denied access to a huge
| chunk of the web. Well, in that case, I would rather be a bot
| than a human. Being a human seems to be increasingly annoying
| nowadays :)
| [deleted]
___________________________________________________________________
(page generated 2023-08-08 23:01 UTC)