(C) BoingBoing
This story was originally published by BoingBoing and is unaltered.
 . . . . . . . . . .



Making AI safer makes it more dangerous [1]

['Jason Weisberger']

Date: 2026-09-18  

Watermarking was supposed to make AI easier to identify; instead, it has made it easier for AI to jump the rails and escape its little box.

The system du jour intended to make AI safer and its use more accountable can apparently make it less safe. Researchers testing Google's SynthID-Text watermarking found changing how a model picks words can also change if it refuses harmful requests. Under prompt-injection attacks, some watermarked models became even more likely to comply with instructions they have been coded to reject.

"As compared to the same models without watermarking, it is definitely going to change their behavior, especially when we place it under adversarial conditions, or we make these models call tools when they're powering an agent," Andrea Siposova, an AI security researcher at Lasso Security, told Ars. "Watermarking is made to not be perceptible to a reader, but we know that when we are changing anything about what the model is generating, it is going to cause some tradeoffs, it's going to show up somewhere." Watermarking works by embedding a signal that allows output to be identified as AI generated, something known as provenance. SynthID takes the normal sampling process and adds a random seed generator, sampling algorithm, and scoring function to it. Instead of the process using an arbitrary random number generator for next-word selection, the watermarking uses a secret key. While the word selection is still random, people with knowledge of the key can check the sequence of words to determine the likelihood that the key was used. ArsTechnica

This is so AI: a safety feature that changes model behavior needs its own safety testing.

Previously:

• California governor Newsom vetoes AI safety law

• Cataloging the problems facing AI researchers is a cross between a parenting manual and a management book

• The newest bipartisan issue: not wanting an AI data center next door

[END]
---
[1] Url: https://boingboing.net/2026/09/18/ai-watermarking-safety-guardrails.html

Published and (C) by BoingBoing
Content appears here under this condition or license: Creative Commons BY-NC-SA 3.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/boingboing/