[HN Gopher] Debunking the Myth of "Anonymous" Data
___________________________________________________________________
Debunking the Myth of "Anonymous" Data
Author : gslin
Score : 101 points
Date : 2023-11-11 17:32 UTC (5 hours ago)
(HTM) web link (www.eff.org)
(TXT) w3m dump (www.eff.org)
| jruohonen wrote:
| A good popular take, but they, either intentionally or out of
| ignorance, omit newer, proven techniques like differential
| privacy.
| buttercraft wrote:
| Or they don't want you collecting data in the first place, in
| which case the techniques are irrelevant.
| JohnFen wrote:
| Entirely this. That data is so widely collected is the root
| problem.
| pixl97 wrote:
| In particular areas things like differential privacy may work,
| for example medical sets where there are lots of regulations
| and potential fines for the companies involved.
|
| But do you think your average ad tech company gives a fuck?
| They are going to keep the original data because that's where
| the money is. Yea, maybe they'll have privacy datasets they
| sell/release to other groups, but all the real data will remain
| in a database, and with most companies in this industry, be
| given to government agencies on demand.
| willseth wrote:
| It's not because of the money. Most could make money just
| fine with properly anonymized data. They just don't care
| and/or don't want to take the effort or spend money to do it
| because the consequences for leaking private info are so
| minimal.
| Forbo wrote:
| Still sounds potentially problematic. Per wikipedia:
| "Differential privacy provides a quantified measure of privacy
| loss and an upper bound and allows curators to choose the
| explicit trade-off between privacy and accuracy. It is robust
| to still unknown privacy attacks. However, it encourages
| greater data sharing, which if done poorly, increases privacy
| risk. Differential privacy implies that privacy is protected,
| but this depends very much on the privacy loss parameter chosen
| and may instead lead to a false sense of security. Finally,
| though it is robust against unforeseen future privacy attacks,
| a countermeasure may be devised that we cannot predict."
|
| If I am dependent on the curator to determine the level of
| privacy then I've already lost.
| 95014_refugee wrote:
| You are already dependent on a set of curators to not simply
| outright lie and export the data as captured. This take lacks
| subtlety; unless you are going to abandon the set of
| functionality ("where is the best fried chicken near me?")
| that this sort of metadata facilitates, you need to make
| decisions about which curators you trust, and then
| participate in driving them to honesty and accountability.
|
| From my perspective this article leans too heavily into the
| FUD, and really doesn't succeed at keeping the call to action
| ticking over. On a good day, the EFF can be really good.
| Today, not so much so.
| c0pium wrote:
| The problem with the EFF is that it's full of people who
| made up their minds about what's ok and what isn't over a
| decade ago, and they are largely just playing the hits now.
| No new material, and no consideration that things might
| change.
| c0pium wrote:
| > ...it encourages greater data sharing, which if done
| poorly, increases privacy risk.
|
| This is a really useful argument, because it's the equivalent
| of the FDA's "generally believed to be safe". If you look
| into something and this is the risk you find, then it's safe.
| genewitch wrote:
| GRAS. Btw.
| JohnFen wrote:
| I agree. DP is better than no privacy at all, but it
| certainly doesn't make me feel comfortable.
| godelski wrote:
| There's also Zero Knowledge Proofs which instead has a privacy
| guarantee
|
| Edit: Not trying to start a DP vs ZKP (or Homomorphic
| Encryption) flame war. We're all on the same side of privacy,
| right? Different tools for different situations.
| jandrewrogers wrote:
| Techniques like differential privacy do not work for some types
| of data models, including many of the more interesting/risky
| ones. I've never seen a technique that can deliver an
| analytical model at scale that is both analytically effective
| and anonymous while also robust against sophisticated de-
| anonymization attacks. There is no good theoretical foundation
| to suggest that such things are possible.
|
| Most modern techniques for ensuring anonymization make
| assumptions that won't constrain sophisticated blackhats. They
| are good policy in a legal ass-covering sense and increase the
| cost required to de-anonymize but that is about it.
| dfcowell wrote:
| If your scale is large enough and you don't care about
| identifying individuals, synthetic data does this fairly
| effectively.
| jdietrich wrote:
| GDPR specifically mentions pseudonymous data in Recital 26:
|
| _" The principles of data protection should apply to any
| information concerning an identified or identifiable natural
| person. Personal data which have undergone pseudonymisation,
| which could be attributed to a natural person by the use of
| additional information should be considered to be information on
| an identifiable natural person. To determine whether a natural
| person is identifiable, account should be taken of all the means
| reasonably likely to be used, such as singling out, either by the
| controller or by another person to identify the natural person
| directly or indirectly."_
|
| Under EU law, there is no special class of "personally
| identifying information". _Any_ data that relates to a person or
| _could be_ related to a person is protected. It isn 't enough to
| just strip the name and SSN field out of the database and call it
| anonymous, you need to demonstrate that the data couldn't be
| attributed to anyone through any reasonably practical process.
|
| https://gdpr-info.eu/recitals/no-26/
| c0pium wrote:
| That is almost verbatim the definition of PII from Wikipedia
| and from OMB Memorandum M-07-1616 (the US government definition
| of PII).
| oezi wrote:
| The crucial piece is your link is:
|
| > 4To ascertain whether means are reasonably likely to be used
| to identify the natural person, account should be taken of all
| objective factors, such as the costs of and the amount of time
| required for identification, taking into consideration the
| available technology at the time of the processing and
| technological developments.
|
| This has been interpreted that any objectively possible way to
| de-anonymize is reasonable. In particular, considering that all
| the data of the organization doing the anonymization can be
| used to try to de-anonymize.
| bertil wrote:
| That broad definition opens itself to so much... Even raw movie
| rating data was famously partially de-anonymized.
|
| I'm not sure that I could imagine a detailed database with
| personal activity at a reasonable scale that couldn't be de-
| anonymized, at least partly, if one assumes that people use
| related services and that one has access to them via public
| social media, mostly.
|
| - Emails: of course;
|
| - social media: easy;
|
| - search queries: LOL;
|
| - transport: through commute;
|
| - any cultural good: you can connect to contemporary
| commentary;
|
| - fashion shopping: pretty much trivial with OOTD posts;
|
| - medial record: not everyone, but most major diagnostics could
| be matched to social media for many people;
|
| - https blobs via VNP: most people would connect to the same
| four sites again and again, but exact timestamps and public
| statements on social media;
|
| - grocery shopping: harder, but doable as you'd have
| neighborhood from the store address and a lot of surface...
|
| Is the letter of that law against pseudonymous databases?
| genewitch wrote:
| Is it funny that it is so broad? People don't want their
| information leaked! Why is this so hard to understand?
|
| It makes your job harder? Tough shit. It makes law
| enforcement tougher to do? Ohhhh nooo, they might have to
| work for a living instead of pushing buttons.
|
| Stop trying to defend companies that do this in service of
| capital. It is heinous. Some, if not most of us, want to be
| left alone and not have our addresses and medical history and
| YouTube stats available for the whole world.
|
| There is no such thing as an anonymous dataset. It has been
| proven again, and again.
| bertil wrote:
| No need to be snarky or insulting: I was trying to ask for
| a sincere assessment of that law.
|
| > do this in service of capital
|
| Sir, this is ycombinator.com
|
| If the intent is that no database with an individual-level
| breakdown falls under GDPR IID protections, I presume that
| far more processes and declarations would have to be
| applied to circumstances where there never was an intent or
| a credible option to de-anonymize them. This is not how it
| is enforced, understood, or applied today.
|
| The large companies that you criticize so readily would
| have an issue automating the paperwork or building hashing
| solutions that divert the problem, but their privacy-
| respecting competitors would fall under a lot more
| paperwork and legal risk than they could handle.
|
| Thankfully, there are solutions: a lot of people are now
| handling internal data processes with the same open-source
| tool, dbt. That platform could help change standards if
| they knew current patterns do not respect the letter of the
| law. But their lawyers seem to think otherwise.
| euniceee3 wrote:
| Great article putting all the relevant content in one place. Does
| anyone know of any de-anonymization services? The startup I am
| working at is privacy focused and we are looking for a way to
| demonstrate why you need an additional layer to protect and
| compartmentalize.
|
| Short of us buying up data in bulk and then doing the de-
| anonymization in-house I am not seeing an easy way to do this. Or
| even an advertised partner, seems like all the articles are
| really careful to not do free marketing for companies in this
| space.
| willseth wrote:
| There are a small handful of us out there, targeting slightly
| different but overlapping use cases, but this is me
| https://www.privacydynamics.io. Happy to answer any questions
| about it.
| bertil wrote:
| This is the same problem as white-hat security: you need people
| who know an awful lot about how bad guys work but are good
| guys. Security has developed a legitimate market over the
| years, but that took time.
|
| You could try to ask independent consultant(s) who you trust to
| work on that problem and let them grow that practice. The
| problem is that they would need to have access to marketing
| platforms.
| spiffytech wrote:
| Every time I hear "anonymous data", I think of that time AOL
| published anonymized search logs (for academic research). The
| anonymization was negligent, and an NYT reporter de-anonymized
| and tracked down one of the users with the local & personal info
| present in the search queries.
|
| https://en.wikipedia.org/wiki/AOL_search_log_release
|
| https://web.archive.org/web/20130404175032/http://www.nytime...
| willseth wrote:
| Another fun one was the Netflix Prize, where Netflix published
| an anonymized dataset, but some researchers at UT Austin were
| able to de-identify many/most of the users in the dataset by
| linking them to IMDB profiles based on preferences.
| jeffbee wrote:
| There's no way most of the users in the Netflix data had IMDb
| accounts.
| e12e wrote:
| From a skim, it looks like their main point is that with 8
| movie ratings, it's trivial to link two datasets (eg
| anonymized Netflix rating to public IMDb profile).
|
| https://scholar.google.com/citations?view_op=view_citation&
| h...
|
| https://systems.cs.columbia.edu/private-systems-
| class/papers...
| willseth wrote:
| I work for a data privacy startup, and this article unfortunately
| groups all forms of anonymization together. It is specifically
| criticizing forms of anonymization that only treat direct
| identifiers like names, addresses, and phone numbers. That is
| usually referred to as "pseudonymization", and they are correct
| to point out that an only moderately sophisticated attacker can
| still link people in the dataset using combinations of indirect
| identifiers like age, birthday, and zipcode. Pseudonymization is
| a weak form of privacy.
|
| More sophisticated methods of privacy also anonymize indirect
| identifiers, and in some cases personal attributes. They do this
| by adding noise to the data in such a way that the noise has
| relatively* minimal impact on the results of computations made
| over the dataset, but a significant impact on the ability to re-
| identify someone using indirect identifiers or attributes.
|
| *There is always a tradeoff between privacy and utility. The only
| way to achieve 100% private data is 100% noise, but the privacy-
| utility tradeoff curve isn't linear, and you can still achieve
| very good utility and very good privacy in many cases, especially
| with the best tools. Methods are also improving over time,
| reducing the impact of the tradeoff.
| genewitch wrote:
| In the interest of charitable-ness, can you define "noise" and
| explain how rudimentary denoising algorithms that have existed
| since the 1960s can't penetrate it?
|
| Time and time again people think they've anonymized data and
| they're always proven incorrect.
|
| Just come to grips with the fact you're participating in the
| sale of my (or whoever's) private data. Hope you sleep well!
| willseth wrote:
| "Denoising" algorithms can't remove this kind of noise. It
| doesn't work like that. You're simply mistaken.
|
| Generally the state of the art for adding noise to data is
| differential privacy or microaggregation. In the case of
| differential privacy it's typically gaussian or laplacian
| noise, but it is not a trivial application. Noise applied
| through microaggregation is not a mathematical function
| because microaggregation targets k in k-anonymity, and the
| "noise" can only be measured after treatment, through various
| distortion measures.
|
| My company doesn't sell private data or facilitate the sale
| of private data. We improve privacy within organizations to
| reduce the risk of privacy leaks. Data brokers and ad
| targeters would have no interest in our software. But thanks
| for the condescending comment!
| Xelynega wrote:
| Who do you think "data brokers and ad targets" purchase
| data from?
|
| I don't understand how you could help companies protect the
| private data they collect from users and think that you're
| not facilitating the collection and sale of it.
| Dayshine wrote:
| Pseudonymization is creating a new random identifier and
| keeping a linking table from those new identifiers to the
| original identifiers.
|
| Pseudonymized data is for all intents and purposes completely
| anonymous as long as the people you share the data with cannot
| access the linking table.
|
| That constraint can be enforced technically, contractually, or
| legally, depending on how important the breach would be.
| willseth wrote:
| It's pseudonymization regardless of the replacement data
| format. It can be a token, a format-consistent value, e.g.
| "john.smith@example.com", redacted, etc. All of that is
| considered pseudonymous.
|
| Pseudonymized data is not completely anonymous. For example,
| your name, address, date of birth, and other identifiable
| information are likely available in public datasets like
| property tax records or voter registration data. There are
| also mostly-legal datasets available for purchase with more
| complete demographic and personal information. If your
| birthday, gender, and zipcode are present in the
| pseudonymized dataset, I have an ~87% chance succeeding in a
| linkage attack matching you in of those public datasets.
| Dayshine wrote:
| If your birthday, gender, and zip code are present in the
| dataset then it is not Pseudonymized.
|
| By definition in any modern privacy specification, the only
| difference between anonymous and pseudonymous is that you
| can recover the true identity.
|
| I believe a decade plus ago Pseudonymized meant simply
| making something less identifiable, but not any more.
| willseth wrote:
| That's not how the term typically used. Most people will
| assume you have only protected direct identifiers if you
| tell them data has been pseudonymized.
|
| https://csrc.nist.gov/glossary/term/pseudonymization
| Dayshine wrote:
| https://en.m.wikipedia.org/wiki/Pseudonymization
|
| See section about the new definition under GDPR.
|
| Like I said, modern privacy standards. The US is still
| quite far behind: I've had to do HIPAA training and it
| shows.
|
| I'd advise you to set your bar a lot higher than your
| national standards if you ever plan to do international
| products: nobody in the US will complain if you use
| stricter definitions but you'll instantly be rejected in
| Europe if you call that pseudonymization.
| willseth wrote:
| What bar are you talking about? We use the state of the
| art to treat indirect identifiers without
| pseudonymization, but we pseudonymize as needed if that's
| overkill. The bar is set very high for our service.
| Sadly, very few companies are actually interested in
| treating indirect identifiers or consider anything
| besides direct identifiers a problem.
|
| I'm familiar with GDPR, we work with EU companies, and
| all the ones we work with use original definition. NIST
| link aside, these terms are hardly standards anyway. They
| are nearly colloquial vocabulary, which unfortunately in
| this space I expect to remain imprecise and vague. This
| why we generally only use it in marketing and comms,
| while in the actual product we drill down into
| specifications for direct and indirect identifiers,
| distortion, and risk.
| mbork_pl wrote:
| For the curious: more about the 87% figure, with a link to
| the source paper:
| https://www.johndcook.com/blog/2018/12/07/simulating-
| zipcode.... (Not the author, just a fan of John D. Cook's
| blog.)
| jandrewrogers wrote:
| Many red-teaming attacks don't rely on the identifiers at
| all, they are essentially "duck typing" the entities in the
| data model. Same approach is commonly used for dealing with
| data where a single identity may have multiple pseudonymized
| identifiers.
|
| There are a bunch of regulatory box-checking exercises like
| this that don't actually provide anonymization if the
| attacker is sophisticated.
| __MatrixMan__ wrote:
| I'm usually the first to defend the EFF but I agree that
| they've gone a bit far here. The anonymization script which I
| wrote for my company just replaces every string in the
| customer's database with a cryptographic hash--except a list of
| strings like "failed" and "success".
|
| So unless your city is named "success", it's going to be
| missing from the dataset.
|
| It's a bit bewildering to actually run the app in this mode,
| but here's a lot of diagnostically relevant information you can
| get out of a database like that. You can even confirm bug
| fixes. Meanwhile, somebody looking to harm the user would need
| to already know quite a lot about that user before they could
| make any use out of such a thing.
|
| Good faith efforts to protect user privacy exist, is not
| helpful to lump them in with the rest like this:
|
| > Sometimes companies say our personal data is "anonymized,"
| implying a one-way ratchet where it can never be dis-aggregated
| and re-identified. But this is not possible--anonymous data
| rarely stays this way.
|
| Is it not possible, or not common?
|
| People need to be informed about how to apply scrutiny to
| anonymization techniques, not scared into assuming ill intent
| when they see one.
| candiddevmike wrote:
| These scripts always have the best intentions but typically
| end up neglected and suddenly production data is on a dev
| laptop (at least, in my experience, both as an employee and a
| customer).
|
| Make mock data inputs or snapshot data based on internal/QA
| usage, please!
| ralferoo wrote:
| > replaces every string in the customer's database with a
| cryptographic hash > So unless your city is named "success",
| it's going to be missing from the dataset.
|
| You've made the typical mistake of thinking that just because
| you've made it _harder_ , you've also made it anonymous.
|
| For instance, you might have replaced "Chicago" with
| 9cfa1e69f507d007a516eb3e9f5074e2, but if for instance a lot
| of people with that tokenised city name also have
| transactions for a store that only exists in Chicago, you can
| infer with a reasonable degree of accuracy the reverse
| mapping. A couple such data points, and you can be almost
| certain.
|
| If you know when you made a couple of transactions that are
| in the database, and you can find a set of likely options for
| each time, and then see what fields they have in common. Once
| you know your data, you can infer a whole load of the reverse
| mappings for various fields.
|
| If those transactions involve another user, you can start to
| correlate those mappings with data you know about them, and
| start to build up a web of transactions that person had even
| if you don't necessarily know yet who the other users they
| interact with yet.
|
| All this is possible without the use of rainbow tables, but
| chances are your hash function is a standard one, so with a
| single known mapping you can work out which hash function you
| chose, and look for speculative entries in the data. e.g.
| let's just look for MD5("Chicago"), SHA1("Chicago"),
| SHA256("Chicago") and see if there are any matches. If there
| are, we can use that hash function and trivially create a
| rainbow table for every city in the US, first names,
| surnames, etc.
| chaps wrote:
| That is usually referred to as "pseudonymization"
|
| For your world, maybe. For most things with a public face,
| "anonymization" is what's used, not "pseudonymization". So yes,
| this article is referring to its more-common use.
| hasoleju wrote:
| Anonymization of personal data is a very tricky thing. It is so
| tricky that the GDPR even doesn't really mention it. I even don't
| know of any official SOP for doing it.
|
| Of course k-anonymity and differential privacy can help you with
| a closed data set, but once you add records to a dataset over
| time everything breaks down.
|
| I once tried to find a way to anonymize data on different clients
| and then connect the records of the same entity on the server.
| This method had the main problem that one could artificially
| generate a new non-identifying data point for an individual and
| then see in which record it will end up in the central data base.
| yoaviram wrote:
| More about why location information IS personal information:
| https://consciousdigital.org/location-data-is-personally-ide...
___________________________________________________________________
(page generated 2023-11-11 23:00 UTC)