[HN Gopher] Researchers tested AI watermarks and broke all of them
___________________________________________________________________
Researchers tested AI watermarks and broke all of them
Author : adg29
Score : 59 points
Date : 2023-10-04 16:32 UTC (2 days ago)
(HTM) web link (www.wired.com)
(TXT) w3m dump (www.wired.com)
| brap wrote:
| People have been trying to watermark digital media for decades,
| when there was (still is) a very strong financial incentive to
| get it working. It never worked. I don't think it ever will work.
| ipnon wrote:
| "Information wants to be free."
| wly_cdgr wrote:
| More like, "people want to steal information"
| artninja1988 wrote:
| Control+C, Control+Vs in your path
| treyd wrote:
| Copying information isn't the same as stealing. To steal
| means to take away.
| ActorNightly wrote:
| You are confusing access restrictions with signing. You can
| easily sign digital media to show that it was made by you.
| ygjb wrote:
| You are confusing a digital signature for evidence of
| anything other than an attestation.
|
| If you create a digital record, then sign it, then that
| signature is only an attestation of may claim you make, not
| evidence of that claim. That is the problem with relying on
| technology to establish trust - the moment you attach an
| economic benefit to a technology you incentivize people to
| circumvent it, or to leverage it to commit fraud.
| wyldfire wrote:
| You can still declare success if you lower the bar to "we can
| catch leaks/pirates and in particular we can know which
| internal folks should no longer be trusted. ... as long as they
| don't attempt to circumvent the fingerprint"
| [deleted]
| jacobr1 wrote:
| We need to focus on the other direction. How can we have chains
| of trust for content creation, such as for real video. Content
| can be faked, but not necessarily easily faked from the same
| sources that make use of cryptographic signing. The attacks can
| sign the own work, so you'd need ways to distinguish those cases,
| but device level keys, organizational keys, distribution keys all
| can provide provenance chains that can be used by downstream
| systems to _better_ detect fraud, though not eliminate it.
| floren wrote:
| I was thinking the other day about embedding keys in cameras,
| etc. but came up with the problem that you could just wire up a
| computer that BEHAVES like a CCD sensor and send whatever the
| hell you feel like in to the signing hardware, so you feed in
| your fake image and it gets signed by the camera as though it
| were real. I assume smarter people than me have put much more
| time into the problem, so I'd be interested to see any good
| resources on the subject.
| jacobr1 wrote:
| I think you'd need device levels keys. You couldn't trust any
| particular image ... but you could perhaps know where it came
| from, which you gives you a better substrate upon which to
| infer trust.
| tshaddox wrote:
| I think you're essentially describing the hardware DRM supply
| chain.
|
| For example, HDCP is a DRM scheme where Intel convinces (or
| legally requires) every manufacturer of HDMI output devices
| (e.g. set-top boxes, Blu-ray players) in the world to encrypt
| certain video streams.
|
| Then, Intel requires manufacturers of HDMI input devices
| (e.g. TVs) to purchase a license key that can decrypt those
| video streams. This license agreement also requires the
| manufacturer to design their device such that the device key
| cannot be easily discovered and the video content cannot be
| easily copied.
|
| Then, Intel gets media companies to include some extra
| metadata in video media like Blu-ray discs. This metadata can
| contain revoked device keys, so that if a TV manufacturer
| violates the terms of the license agreement (e.g. leaks their
| key, or sells a device that makes copies of video content),
| that manufacturer's TVs won't be able to play new content
| that starts including their key in the revocation list.
|
| Of course, Intel's HDCP master key was either leaked or
| reverse-engineered, so anyone can generate their own valid
| device keys. Intel will probably sue you if you do this, I
| guess.
| charcircuit wrote:
| >Of course, Intel's HDCP master key was either leaked or
| reverse-engineered, so anyone can generate their own valid
| device keys
|
| Of an older version of HDCP. New media can require a higher
| HDCP version where that bypass isn't possible.
| sebzim4500 wrote:
| Interesting. I don't understand the revocation process
| though.
|
| What stops the blu-ray reader from just ignoring the
| revocation list on the disk?
| dylan604 wrote:
| That's where the reversing comes in to switch the
| function call to check the revocation list to a NOP and
| just keep on going. At least, that's how I imagine HDMI
| equipment that ignores HDCP works
|
| What stops them from being sold that way would probably
| be the licensing agreement and honest players. I'd
| imagine in China, there are lots of these types of
| devices available.
| yetanotherloss wrote:
| The cryptography to support this has been around for ever and
| it's been next to impossible to make the decision makers at
| companies and large organizations care, much less end users.
|
| Small time players like GE routinely fail to correctly sign
| industrial control software, the odds of people recording video
| paying enough attention to get it right and the meme crowd
| bothering to check even if they did seems vanishingly small
| without a lot of educational effort.
| jacobr1 wrote:
| Yeah, you need adoption for it to work, and that in tern
| means there needs to be some kind of financial or regulatory
| incentive. But it does seem to me to be more technically
| feasible. Fingerprinting AI seems ... just not workable at
| this point.
|
| We are starting to see adoption of software supply-chains
| with SBOMS, albeit imperfectly. We are starting to see
| increased adoption of things like DMARC in the email space to
| better authentic the originator of an email. Both are highly
| imperfect systems ... but you can start kludging something
| together ... and if the incentives are there I think you can
| build out more of a workable system.
| tomrod wrote:
| > Small time players like GE routinely fail to correctly sign
| industrial control software, the odds of people recording
| video paying enough attention to get it right and the meme
| crowd bothering to check even if they did seems vanishingly
| small without a lot of educational effort.
|
| I've wanted to build a product in this space ever since I
| heard about deepfakes. Mix of keybase and appropriate file
| hash, and hash gen for subsets of sections of video. Maybe it
| needs to be a protocol, maybe a product, not sure, but the
| need seems apparent to me.
| tempusalaria wrote:
| You can see here: GitHub.com/HNx1/IdentityLM
|
| It's a direct (and open source) implementation of public key
| cryptography into the LLM logit distribution.
|
| The paraphrasing model/beam search needs work - feel free to
| pitch in :)
| Alligaturtle wrote:
| I agree with this sentiment. Years ago, I asked around at one
| of the smartphone companies whether it would be possible to
| certify to an end user that a photo is either:
|
| 1) Authentic and only lightly edited with image manipulation
| software (e.g., cropped, color balanced, or text placed over
| top of the image) 2) Produced on a phone that has had to go
| through hardware hacks
|
| Note that the guarantee in (1) wouldn't prevent someone from
| taking a photo of a TV screen. When I asked that original
| question, I had quite a few more details about how the
| certification might be done, how the credentials would be
| hosted, and how the results would be shown on a website.
|
| Anyway, just asking this question was met with a storm of
| negative responses. I counted two dozen messages that were
| either neutral (asking for clarification) or else outright
| hostile before the first hesitantly positive message. My
| favorite hostile response was that allowing people to certify
| images as real would steal peoples' rights. I didn't follow the
| logic, but the guy who made the argument was really into it.
|
| There were lots of comments about how using AI would be a
| better solution, some commenting on how Cannon already did it
| (and messed up gloriously), others stating they didn't have
| faith in hardware... it makes a fella never want to ask a
| question again.
|
| In the end, I got an expert to speculate that the technology
| currently exists, and has existed for 5-10 years, to do this
| with a modern smartphone. However, unless a high-level engineer
| or executive argues that providing this feature will somehow be
| a competitive advantage, there is no appetite to provide this
| kind of feature.
| nofunphil wrote:
| Agreed. At the risk of a shitstorm of downvotes, tokenized
| media could be part of a solution, especially at the consumer
| level. Authenticate real videos via a mint button/QR that
| airdrops you a token from creator. May require platforms to
| opt-in tho. Basically trust nothing unless you can authenticate
| source onchain. Not great fo sho, but prob necessary soon
| tudorw wrote:
| "Magnetic anomalies are generally a small fraction of the
| magnetic field. The total field ranges from 25,000 to 65,000
| nanoteslas (nT). To measure anomalies, magnetometers need a
| sensitivity of 10 nT or less."
|
| Would signing content with a cryptographically consistent
| encoding of this field be workable?
| TestingTest5 wrote:
| Was only a matter of time anyways...
| KaiserPro wrote:
| We already have well established systems to prove the provenance
| of images and other sources.
|
| At the moment the internet is a _wash_ with bullshit images. Its
| imperative that news outlets are at a high enough standard to
| actually prove the provenance of them.
|
| You don't trust some bloke off facebook asserting that something
| is true, its the same for images.
| skilled wrote:
| https://archive.ph/1F0Ng
| rakkhi wrote:
| It's like captcha, highly annoying to users and authors, but if
| you don't want to pay it works against low spend bots
| obblekk wrote:
| For written text, the problem may be even harder. Identifying the
| human author of text is a field called "stylometry" but this
| result shows that some simple transformations reduce the success
| to random chance [1].
|
| Similarly, I suspect watermarking LLM output is probably
| unworkable. The output of a smart model could be de-watermarked
| by fine tuning a dumb open source model on the initial output,
| and then regenerating the original output token by token,
| selecting alternate words whenever multiple completions have
| close probabilities and semantically equivalent. It would be a
| bit tedious to perfectly dial in, but I suspect it could be done.
|
| And then ultimately, short text selections can have a lot of
| meaning with very little entropy to uniquely tag (e.g., covfefe).
|
| [1] https://dl.acm.org/doi/abs/10.1145/2382448.2382450
|
| Curious if Scott Aaronson solved this challenge...
| kromem wrote:
| Also, most stylometry work isn't well fitted to active attempts
| to forge another author, and is more about distinguishing
| authorship in works with uncertain attribution.
| epivosism wrote:
| Wasn't this obvious from the get go that this can't work?
|
| If AI will eventually generate say 10k by 10k images, I can
| resize to 2.001k by 1.999k or similar, and I just don't get how
| any subtle signal in the pixels can persist through that.
|
| Maybe you could do something at the compositional level, but that
| seems restrictive to the output. Maybe something about like
| larger regions average color balance or something? But you
| wouldn't be able to fit many bits in there, especially when you
| need to avoid triggering accidentally.
|
| Also: here are some play money markets for whether this will
| work:
|
| https://manifold.markets/Ernie/midjourney-images-can-be-effe...
|
| https://manifold.markets/Ernie/openai-images-have-a-useful-a...
| charcircuit wrote:
| Normal watermarking solutions can survive resizes.
| great_psy wrote:
| It seems it would be much easier to watermark non-ai images
| instead. Aka crypto signature.
|
| That will be much harder to evade, but also pretty hard to
| implement.
|
| I guess we will end up in the middle ground, where any non-signed
| image could be ai generate, but for most day to day use it's ok.
|
| If you want something to be deemed legit (gov press release,
| newspaper photo, etc) then just sign it. Very similar to what we
| do for web traffic (https)
___________________________________________________________________
(page generated 2023-10-06 23:00 UTC)