[HN Gopher] Ask HN: Has anyone leveraged GDPR to overturn automa...
___________________________________________________________________
Ask HN: Has anyone leveraged GDPR to overturn automated bans?
There are many headlines of people getting their <FAANG/Big
Company> accounts banned and losing access to a lot of important
documents or services. Often, trying to talk to support on any of
these companies is akin to talking to a wall. However, GDPR has a
clause stating that "The data subject shall have the right not to
be subject to a decision based solely on automated processing".
Which would mean that any EU/EEA citizen should have the right to
have the decision reviewed by a human. Has anyone successfully
overturned a banned account using this method?
Author : ndmrs
Score : 128 points
Date : 2022-01-30 16:47 UTC (6 hours ago)
| blibble wrote:
| getting anything sensitive data out of large companies with the
| GDPR seems to be impossible unless you want to resort to lawyers
|
| I was trying to get _my_ matchmaking data out of Activision
| Blizzard and they flat out refused, saying my data was their
| property
|
| their exact response was:
|
| > "the information requested are trade secret and/or intellectual
| property needed to preserve our game integrity"
|
| I complained to the regulator, who agreed with my assessment, but
| to enforce it I'd have to go to court
|
| seems the GDPR is basically useless
| jdavis703 wrote:
| I hear people saying laws that require police reports, police
| enforcement or interactions with the court are useless. For
| people like yourself who feel this way, what alternatives do
| you propose?
| blibble wrote:
| what's the purpose of a regulator if they agree with you and
| can't do anything to enforce the law?
|
| it's privacy theatre, nothing more
| lamontcg wrote:
| We seriously need an Internet Bill of [Personal] Rights and get
| it into law and use it against the FAANGs. Europe at least seems
| to be trying, along with California sometimes.
| chrisin2d wrote:
| Agreed. We're overdue for a Magna Carta for our new era, lest
| we be absolutely ruled by the ever-growing myriad of algorithms
| and models that govern our participation in society and
| economy.
| leobg wrote:
| There's also the right to be forgotten. How can they ban you if
| they have to delete all information that can be used to identify
| you?
|
| Even hashes of your email address or payment data should be
| something you should be able to request they must delete.
| teraflop wrote:
| Even under the GDPR, the right to be forgotten is not absolute
| or unconditional.
|
| If a person revokes consent for their personal data to be used,
| the data must be deleted _if_ "there is no other legal ground
| for the processing". But if a data processor has an overriding
| "legitimate interest" in storing data about you, then they have
| legal grounds to do so without your consent. The details of
| this will vary depending on the situation (and the
| jurisdiction) but, for example, fraud prevention is explicitly
| called out as a legitimate interest.
|
| https://law.stackexchange.com/questions/37882/google-adwords...
| ilamont wrote:
| _The data subject shall have the right not to be subject to a
| decision based solely on automated processing_
|
| Lots of leeway for FAANG/BigCo management to wriggle out of that
| one. "Sure, Jones in Legal gets an email notification every time
| an account is banned and has the option to review it."
|
| I can only imagine the lobbying and "negotiation" that takes
| place to have legislators water down the requirement for real
| human beings to review or respond to such bans.
| sealeck wrote:
| I doubt that would hold legally speaking because that would
| essentially be purely automated data processing.
| throwawaygamma6 wrote:
| Yes. I once got my account permanently locked at a well known
| service provider when I simply tried to make a payment for the
| first time. Support wasn't useful and all they could do was tell
| me that I somehow violated their Terms of Service for committing
| "fraudulent patterns" over and over again.
|
| I could have and maybe should have just let it go, but it really
| got under my skin. I first tried out of band approaches to
| contacting somebody there. I didn't reach anybody, and you
| quickly realize how everybody else on the Internet just assumes
| you must either be lying or not telling the full story. Maybe
| it's just acceptable losses while doing business at scale.
|
| So I finally just emailed them a polite GDPR request containing
| some spiel about Article 15(h), how I have the right to request
| my personal data, and also have the right to correct any
| inaccuracies in it, which must be the case since I committed no
| such fraudulent actions. I also requested a full list of all
| their data subprocessors, which I couldn't actually find listed
| anywhere on their site.
|
| I'm not a lawyer, and I don't know if my request hit all the
| right notes or not. But literally one hour later, I got my
| account unlocked with a personal apology.
|
| For what it's worth I also let them know that I'm not really
| looking to circumvent their systems, and I'm sure they have to
| deal with a lot of bad actors. But there really needs to be a
| better way to reach somebody to fix things when automated systems
| go wrong.
|
| I also have the feeling that this approach would fall on deaf
| ears for big FAANGs, and there really needs to be some high
| profile ruling to put the fear in them.
| foxfluff wrote:
| > I didn't reach anybody, and you quickly realize how everybody
| else on the Internet just assumes you must either be lying or
| not telling the full story.
|
| I have observed the same. When I evaluate service providers,
| I'm curious to know how they handle dispute with customers..
| it's quite depressing to see that on most online forums, it
| usually goes straight into victim blaming. You must have
| violated the TOS, you must be doing something sketchy, you're
| not telling the whole story, you're just holding a grudge so
| get over it, you're just entitled, etcetra. There's very little
| sympathy, and no giving benefit of the doubt.
|
| > But literally one hour later, I got my account unlocked with
| a personal apology.
|
| Congrats! This is a lovely anecdote, thank you so much for
| sharing.
| mst wrote:
| Having done a bunch of moderation work in various open source
| spaces, it's intensely aggravating that while the vast
| majority of complaints are from people who are lying about
| what happened there are also plenty of mistakes so stupid
| that they -sound- unbelievable and yet are in fact true.
|
| And I'd note that I am very very certain that I've made
| mistakes that stupid over the years.
| lambada wrote:
| From memory I believe FAANG etc all _claim_ that appeals you
| lodge are reviewed by a human.
|
| Now if you don't believe them then you'd need to take them to
| court and show why you think that's not the case.
|
| Which I guess means my question is why don't you believe them and
| how likely is it that they are lying when they claim thy appeals
| are reviewed by a human?
| toomuchtodo wrote:
| > Which I guess means my question is why don't you believe them
| and how likely is it that they are lying when they claim thy
| appeals are reviewed by a human?
|
| Why would we believe them? It's Google's responsibility to
| prove their assertion, versus regulators taking them for their
| (not so good) word. The default should be the assumption that
| the corporation is being dishonest.
| ben_w wrote:
| If you're taking them (or anyone else) to court, isn't the
| burden of proof on you?
| toomuchtodo wrote:
| Highly dependent on the law or regulation in question.
| tyingq wrote:
| There was a recent example with Google Drive where it
| explicitly disabled any way to appeal. I was able to reproduce
| the issue where it was flagging files that consisted of a
| single byte, sometimes followed by \r\n or \n.
|
| Here's the HN story:
| https://news.ycombinator.com/item?id=30060405
|
| Screenshots of trying to "appeal" (Request a review) from when
| I recreated the issue show pretty clearly there is no human
| involved: https://imgur.com/a/5YHQtLi
|
| This wasn't an account ban, so I don't know how well it fits
| the GDPR language. Though I'd be surprised if this was somehow
| the only "fully automated account action" FAANG type companies
| are doing.
| jeffbee wrote:
| Are you suggesting that Europe has established a fundamental
| human right to have Google provide free static hosting
| services?
| gpm wrote:
| No, they're suggesting that given that Google has chosen to
| provide free static hosting, Europe has decided they can't
| moderate it with purely automated systems with no appeals
| process.
|
| This is like running a restaurant in the US, and not being
| able to discriminate by race. You're not required to run a
| restaurant, and certainly aren't required to run one that
| gives away free food, but if you are certain obligations
| come attached.
|
| I'd also argue that your use of the phrase "fundamental
| human right" is misleading. Europe can and does require you
| do things for reasons other than respecting fundamental
| human rights. So does pretty much every other law making
| authority.
| Retric wrote:
| Arguably the opposite where the EU may have in effect
| outlawed may free services be requiring human review of
| many activities.
| foxfluff wrote:
| I think reading the part where they say "I don't know how
| well it fits the GDPR language" would answer your question.
| tgsovlerkhgsel wrote:
| The EU has established that "The data subject shall have
| the right not to be subject to a decision based solely on
| automated processing, including profiling, which produces
| legal effects concerning him or her or similarly
| significantly affects him or her." (https://gdpr-
| info.eu/art-22-gdpr/)
| Someone wrote:
| I don't see how you get from Google's statement _"Was taken
| down for legal reasons and cannot be appealed"_ to _"no human
| was involved"_.
| kelnos wrote:
| I think the point is they, based on the file content, no
| human could have been involved in the decision. If there
| was a human involved, the files never would have been
| flagged.
| SAI_Peregrinus wrote:
| You're assuming the human both has agency, and gives a
| damn. It's more likely the human just rubber-stamps all
| bans, to get their KPI of number of appeals processed per
| day up!
| mdoms wrote:
| That's an automated process then.
| foxfluff wrote:
| They are required to have agency and give a damn. Of
| course, it is hard to (dis)prove that they actually do.
| saghm wrote:
| If the human doesn't have agency, then it's not really a
| "human review", is it?
| rat9988 wrote:
| It still is.
| bsedlm wrote:
| it may still be human, but if it's just rubber stamping
| is it really a review?
| philipov wrote:
| And you've just demonstrated why programmatic enforcement
| of contracts doesn't work. Courts are actually able to
| see through that semantic nonsense, because humans are
| able to discern intent.
| saghm wrote:
| I think the chances that a judge would accept that
| argument is potentially a lot lower than you might
| expect.
| throwaway22032 wrote:
| Here's a human review process for you:
|
| 1) Check whether the output of the machine learning model
| outputs Yes or No.
|
| 2) If yes, ban.
|
| 3) If no, no ban.
|
| This is not a human review process. The review process is
| algorithmic, the human is only involved to relay the
| result.
| foxfluff wrote:
| Spirit of the law is a concept I would encourage anyone
| to think about when arguing about these things. I believe
| most people, and courts in particular, would not agree
| that a human rubber-stamping automated decision is in
| line with the spirit of the law. Clinging onto a
| technicality isn't going to go well.
|
| I'd also like to point out that these laws don't just
| come out of nowhere in a vacuum, to be interpreted
| without any further context. In EU we have recitals and
| guidelines to give context and support the interpretation
| of regulations.
|
| If you're interested, do read Guidelines on Automated
| individual decision-making and Profiling for the purposes
| of Regulation 2016/679 (wp251rev.01).
|
| https://ec.europa.eu/newsroom/article29/items/612053/en
|
| Here's what it says about human intervention: "Any review
| must be carried out by someone who has the appropriate
| authority and capability to change the decision. The
| reviewer should undertake a thorough assessment of all
| the relevant data, including any additional information
| provided by the data subject."
| SpicyLemonZest wrote:
| But the entire premise here is a "letter of the law"
| thing. Online account bans are pretty clearly not within
| the spirit of the GDPR restrictions on automated
| decisionmaking; note how the guidelines you linked,
| despite providing quite a bit of detail about different
| kinds of automated decisionmaking and rules around them,
| don't mention account bans at all.
| foxfluff wrote:
| There's only a handful of examples, and to me it is far
| from clear whether account bans would be in scope of the
| law. It's not meant to be an exhaustive list of all the
| things that are covered.
|
| However, I could make the case that losing an account
| which holds years of your private correspondence and is
| your point of contact for private exchange, services you
| rely on (including where bills, account recovery emails,
| policy changes, warnings & alerts, 2fa codes, and other
| very important messages are sent), potential employers or
| clients, and which doubles as a login for other services
| (see openid) and so on, can have a _significant effect_
| on your life and could potentially fall under "decisions
| that deny someone an employment opportunity or put them
| at a serious disadvantage" or (admittedly vague) "lead to
| the exclusion or discrimination of individuals."
|
| Some of the other examples in the guidelines seem mild by
| comparison (e.g. getting a reduced limit on credit card).
|
| My perspective is colored by both having lost access to
| an email account and also being denied a credit card
| application; the former was a much bigger problem.
| Someone wrote:
| The discussion isn't about the initial flagging, it's
| about the review.
|
| That might be as simple as checking for the existence of
| legal documents claiming copyright infringement, or as
| reading a web page stating "we already removed X other
| copies of this file".
|
| Neither is a fail-safe way of doing such a review, but
| doing a thorough review _might_ be expensive even for
| Google. Does anybody know how many such reviews they do
| each day?
|
| It _might_ also be a bug on their tooling to assist human
| reviewers.
| tyingq wrote:
| Because once a human did get involved, via noise from here
| and twitter, Google admitted that there was a problem.
|
| Also, just the absurdity that a human would review a file
| containing only "1" and decide the decision to flag it was
| correct.
|
| https://twitter.com/googledrive/status/1486038872928792576
| godelski wrote:
| What's interesting is there's another user there that
| followed up 2 days after the tweet noting that other
| numbers weren't fixed. But this is ignored.
|
| Problems shouldn't get fixed just because they got enough
| likes and reshares on Twitter.
| [deleted]
| CodesInChaos wrote:
| I think the better question is what a "human review" entails. I
| assume they have some kind of "human review" in there, but no
| meaningful human review.
| hsbauauvhabzb wrote:
| Aside from using imap to backup my mail, what else should I do to
| help mitigate an arbitrary ban? I've had a gmail account for 20
| years since 12 year old me got caught up in invite fomo. I've
| since moved to other providers but still there's a fair amount
| tied into my account currently.
|
| Mostly I'm scared of 'multifactor' where email access is
| considered a form of identity, but I'm not sure what else
| delroth wrote:
| Not speaking for my employer, but the actual quote from GDPR is:
|
| > The data subject shall have the right not to be subject to a
| decision based solely on automated processing, including
| profiling, _which produces legal effects concerning him or her or
| similarly significantly affects him or her_.
|
| Emphasis mine. This would not include the vast majority of
| automated bans. It's more meant as a way to prevent e.g.
| automated police action via algorithmic selection.
| lmkg wrote:
| This is correct. The Article 22 rights are the most narrowly-
| restricted of any data subject rights granted by GDPR. It only
| applies to things that are a Very Big Deal, like prison
| sentences, voting rights, or eligibility for government
| services.
|
| While not tested by the courts, there is a plausible argument
| that "similarly significantly affects him or her" might apply
| to bans that impact your ability to earn a living. So streamers
| getting banned from YouTube, or AdWords bans for businesses
| where that's their main source of revenue. Bans that are lower-
| stakes than that get harder to justify under Article 22.
| foxfluff wrote:
| I agree that it probably does not include the vast majority of
| automated bans.. but I'd prompt anyone interested to read the
| relevant guidelines to understand what might be in scope as far
| as legal effects or "significant effects" are concerned; it
| goes well beyond profiling by authorities, and commercial data
| controllers are far from exempt.
|
| One example of a legal effect is _cancellation of a contract._
| Examples of significant effect include automatic refusal of an
| online credit application, and e-recruiting practices without
| any human intervention.
|
| Advertising is in scope too: "For example, someone known or
| likely to be in financial difficulties who is regularly
| targeted with high interest loans may sign up for these offers
| and potentially incur further debt."
|
| Pricing is in scope too: "Automated decision-making that
| results in differential pricing based on personal data or
| personal characteristics could also have a significant effect
| if, for example, prohibitively high prices effectively bar
| someone from certain goods or services."
|
| Finally, there's an example of profiling reducing a credit card
| limit. "This could mean that someone is deprived of
| opportunities based on the actions of others."
|
| Anecdotally, getting kicked out of my email account has had far
| bigger effects on me than being rejected my credit card
| application.
|
| https://ec.europa.eu/newsroom/article29/items/612053/en
| Nextgrid wrote:
| The problem is that the GDPR is pretty much not enforced. See
| https://ruben.verborgh.org/facebook/ where the author tries to
| get all his data from Facebook - the case hasn't moved since 3
| years now.
|
| The regulators are useless (especially the Irish one which seems
| happy to shield big tech scum from having to comply with the law)
| which confirms my own experience raising complaints with the ICO
| (the UK privacy regulator).
| MaxBarraclough wrote:
| Same goes the for 'cookie law'. A significant fraction of the
| web is in violation. The lack of enforcement sends the message
| that non-compliance is acceptable, so it's become the norm.
| jtbayly wrote:
| What cookie law? The one that states I have to make my
| website worse for everybody to use?
|
| Yeah, I definitely ignore that law, and I wish 100% of
| website owners did. It feels to me like 99% of them follow
| it.
| ben_w wrote:
| There is no law stating you have to make your website
| worse.
|
| Making your website worse is just a what certain analytics
| providers want you to do so you keep paying for their
| services.
|
| https://github.blog/2020-12-17-no-cookie-for-you/
| debesyla wrote:
| I, personally, like it more when I can say "no, don't track
| me".
|
| It's only worse for the user when the cookie notification
| is blocking the content, there is no "no, I don't agree"
| button or clicking it means clicking trough 100 extra
| toggles.
| SpicyLemonZest wrote:
| Regulators have brought quite a few successful GDPR fines. The
| reason that people think the GDPR isn't enforced, in my
| experience, is usually that they've been misled about what it
| does and doesn't require.
|
| For example, the author you linked to is demanding a portable
| copy of all his personal data from all sources, which Facebook
| has no GDPR obligation to give him. He seems to have been
| misled by a form letter he found, which incorrectly conflates
| Article 15 data access (isn't required to be portable) and
| Article 20 data access (isn't required to include data that he
| didn't initially provide).
| Nextgrid wrote:
| I don't disagree that there's been a few successful fines,
| but by that logic I should quit my job tomorrow because I
| happened to get lucky at the casino a couple times.
|
| GDPR enforcement has been extremely lacking as demonstrated
| by the web being littered by non-compliant data processing
| consent forms. A compliant consent form should make the
| "decline" option as prominent as the "accept" one - the vast
| majority of services currently don't comply (including big
| names like Google or Facebook) and entire businesses such as
| TrustArc have been built on providing non-compliant consent
| forms as a service.
|
| For GDPR enforcement to be considered serious, the fines
| amounts should be higher than the profits of companies built
| on abusing user data. If we look at
| https://www.enforcementtracker.com/?insights we can see that
| 1,6 billion euros has been handed out so far over a period of
| 4 years across the entire EU. How much does Google or
| Facebook profit _in a year_?
|
| The entire _experience_ of reporting violations is also a
| major problem and suggests the regulators (at least the UK
| one) aren 't actually interested in enforcing the regulation.
| The process with the ICO requires that you first get in touch
| with the company and try to resolve your concern. This takes
| time & admin work on your behalf and a malicious actor can
| drag out the process for months. But let's assume that after
| you've done that and haven't gotten anywhere, escalating to
| the ICO merely results in them sending a letter. And when the
| company ignores _that_ too, guess what happens? _Another
| letter_ which they will promptly ignore too.
|
| This sets the example that breaching the GDPR _does_ pay,
| because not only reporting a violation requires so much
| commitment that the vast majority of people won 't bother,
| but _even_ once the violation is reported, the response from
| the ICO isn 't actually an effective deterrent either.
| SpicyLemonZest wrote:
| > A compliant consent form should make the "decline" option
| as prominent as the "accept" one - the vast majority of
| services currently don't comply (including big names like
| Google or Facebook) and entire businesses such as TrustArc
| have been built on providing non-compliant consent forms as
| a service.
|
| Nothing in the text of the GDPR, nor of any regulatory
| guidance that I've seen, suggests that the "decline" option
| has any particular UI requirements beyond merely being
| present. Again, while I don't want to claim that this or
| any other regulatory process is perfect, I think the
| primary reason people in privacy circles find it so
| frustrating is that they keep trying to enforce things that
| aren't actually GDPR requirements.
| Nextgrid wrote:
| The following is from the ICO, which I don't think has
| any reason to interpret the guidelines any stronger than
| what they have to, since they aren't willing to enforce
| any of it anyway:
|
| https://ico.org.uk/for-organisations/guide-to-data-
| protectio... :
|
| > Consent means offering individuals real choice and
| control. Genuine consent should put individuals in
| charge, build trust and engagement, and enhance your
| reputation.
|
| > Consent requires a positive opt-in. Don't use pre-
| ticked boxes or any other method of default consent.
|
| > Be specific and 'granular' so that you get separate
| consent for separate things. Vague or blanket consent is
| not enough.
|
| > Make it easy for people to withdraw consent and tell
| them how.
|
| https://ico.org.uk/for-organisations/guide-to-data-
| protectio... :
|
| > What is an unambiguous indication (by statement or
| clear affirmative action)?
|
| > It must be obvious that the individual has consented,
| and what they have consented to. This requires more than
| just a confirmation that they have read terms and
| conditions - there must be a clear signal that they
| agree. _If there is any room for doubt, it is not valid
| consent._ [emphasis mine]
|
| At this point you could already argue that unless the
| decline option is as prominent (if not more) than the
| accept option then the user didn't actually intend to
| consent and just couldn't figure out how to decline.
|
| > Consent should be given by a clear affirmative act
| [...] Silence, pre-ticked boxes or inactivity should not
| therefore constitute consent.
|
| > The key point is that all consent must be opt-in
| consent, ie a positive action or indication - there is no
| such thing as 'opt-out consent'. Failure to opt out is
| not consent as it does not involve a clear affirmative
| act. _You may not rely on silence, inactivity, default
| settings, pre-ticked boxes or your general terms and
| conditions, or seek to take advantage of inertia,
| inattention or default bias in any other way_. All of
| these methods also involve ambiguity - and for consent to
| be valid it must be both unambiguous and affirmative. It
| must be clear that the individual deliberately and
| actively chose to consent. [emphasis mine]
|
| Seems like that's cut and clear.
| SpicyLemonZest wrote:
| What I would call "cut and clear" would be a specific
| description of how prominent the decline button must be.
| "Inertia, inattention, or default bias" is a very
| nonspecific phrase, and even if it does include UI, it's
| not obvious why the implied standard would be "as
| prominent" rather than, say, 50% as prominent.
|
| Don't get me wrong, I'd really like companies to design
| this way on principles of general user friendliness, but
| I don't see much evidence that anyone involved in
| creating the GDPR intended to require it.
| spiffytech wrote:
| > The problem is that the GDPR is pretty much not enforced.
|
| 17 companies were fined for GDPR violations just this month.
| Last year, Amazon was fined EUR746,000,000, Google
| EUR150,000,000, Facebook EUR60,000,000.
|
| https://www.enforcementtracker.com/
| Nextgrid wrote:
| I knew this link was going to come up so I've addressed it
| here: https://news.ycombinator.com/item?id=30141276
|
| The 60M Facebook fine is a welcome development but my point
| still stands - how much did Facebook profit from breaching
| the regulation for the 4 years since it's been in effect?
| That fine should've had a few extra zeros at the end to
| actually serve its role, otherwise it's just a very small
| cost of doing business.
| newbie789 wrote:
| advisedwang wrote:
| GDPR Article 22 (the rule you refer to) also has exceptions:
|
| > Paragraph 1 shall not apply if the decision...is necessary
| for...performance of, a contract between the data subject and a
| data controller
|
| Which I can see applying as they probably have something in the
| ToS to enforce here.
|
| It also allows automated decision making to comply with EU law. I
| don't know EU copyright law well enough, maybe Google has a
| responsibility to take down that data under copyright law and so
| this exception applies too.
| yccs27 wrote:
| The ToS does not constrain the company! The agreement does not
| stipulate that the platform _has_ to enforce ToS, so this is
| not a necessary action to perform the contract.
___________________________________________________________________
(page generated 2022-01-30 23:01 UTC)