[HN Gopher] A Chinese official's use of ChatGPT revealed an inti...
___________________________________________________________________
A Chinese official's use of ChatGPT revealed an intimidation
operation
Author : cwwc
Score : 89 points
Date : 2026-02-27 15:52 UTC (7 hours ago)
(HTM) web link (www.cnn.com)
(TXT) w3m dump (www.cnn.com)
| upupupandaway wrote:
| I was in Shanghai recently and while casually testing one of
| their AI chat bots I typed "What do you think of the situation in
| Taiwan?".
|
| It started discussing like a Western bot would - "it's
| complicated, etc. etc." and around 5s it abruptly stopped and
| regurgitated the same line the CCP uses "... it's an unalienable
| part of China etc. etc.".
|
| After printing the line, a popup opened and my camera was
| activated. The app wanted me to submit my information, presumably
| to decide what to do with me next time I enter China.
|
| 1) All the lights and modern buildings cannot hide that China is
| a creepy authoritarian state underneath.
|
| 2) Given the bot started printing the Western consensus first, I
| bet $10 it was trained by distilling ChatGPT or Gemini.
| titaniumtown wrote:
| > After printing the line, a popup opened and my camera was
| activated. The app wanted me to submit my information,
| presumably to decide what to do with me next time I enter
| China.
|
| Was this on your personal device? I'm just wondering how it
| activated your camera. I would love more details!
| 9999px wrote:
| He's lying.
| parl_match wrote:
| An increasing use of AI is to gather user feedback. The
| Chatbot UI detected an error state, and then loaded a
| feedback vendor, who then popped the camera open for their
| interactive feedback session
|
| I've run into this a few times, now.
|
| So what OP is saying is plausible, I just don't appreciate
| their added and probably incorrect conclusion that it's
| because the government of China wants to do something to
| them
| anvuong wrote:
| Yeah that part is either just bullshit or OP gave the bot
| access to his camera previously, which is just dumb.
| 3rodents wrote:
| If this were true, why didn't the chatbot immediately recognize
| that the word "Taiwan" should trigger the response? Detecting
| the word "Taiwan" has been possible since before most of us
| were born.
|
| China has more restrictions on what you can say than the U.S.
| but what you are describing is not reality. Some westerner
| asking Deepseek about Taiwan is completely uninteresting. Just
| as the government do not chase people over VPN usage.
|
| China doesn't try to hide that they are an authoritarian state.
| They don't need to. Most people in China are no less happy with
| their government than westerners are with their governments.
| Governments reflect culture. And as for foreigners, our view of
| China is far worse than it actually is, China doesn't need to
| hide anything, people who visit China will come away with a
| more positive view of the country than those who do not visit.
| CWuestefeld wrote:
| This is manifestly false.
|
| My wife grew up in Shanghai, and you'll have to go quite some
| distance to find someone more critical of the PRC and CCP
| than she is. And it's with good reason.
|
| She grew up during the cultural revolution, and was largely
| raised by her grandmother because literally every other
| person in her extended family was in prison or work camp, not
| because of anything they had actually done wrong, but for
| political reasons because the whole family was blacklisted.
|
| And that's not just the old days. Her father died as a direct
| result of Chinese Covid policy. During the pandemic her
| cousins still in the country would ask her (on Skype) "is X
| true?", and largely their perception of what was going on was
| false. She would exfiltrate encrypted news reports to them -
| until those started getting blocked. Her dad's estate still
| has affairs that need to be resolved, but we've decided not
| to return to China until Xi is gone, as it's just not safe.
| It doesn't get much airplay, but there are currently a couple
| of hundred Americans who are being illegally detained in
| China right now. It's not worth the risk.
|
| My first trip to China was about 30 years ago, shortly after
| we got married. And back then, I would have said that you
| were right. Honestly, it felt like for the average person in
| their day-to-day-lives, the Chinese were less under the
| governmental thumb than we are. People from the countryside
| would bring their produce into the city to sell, or cook
| dumplings and buns to sell on the side of the street - stuff
| that in America we'd have to get permits for. It seemed that
| the oligarchy had an understanding with the people: let us
| control the big picture, and we'll look the other way for the
| little things. But Chinese politics is a pendulum swinging
| very widely. From Tienanmen Square and Tank Man, it had swung
| quite a bit the other way. But today, it's come back
| 180-degrees. Xi is really trying for a Cultural Revolution
| 2.0.
|
| These impressions largely match what I hear from other
| Chinese immigrants - except for Party members, who tend not
| to want to talk about it at all. I'm afraid that you've been
| listening to too much propaganda.
| 3rodents wrote:
| "there are currently a couple of hundred Americans who are
| being illegally detained in China right now"
|
| Compared to the U.S. which currently has no foreign
| nationals detained illegally?
|
| Pick any country and you will find political dissidents.
| The existence of angry emigrants is not evidence that a
| country is worse than we could ever imagine.
| CWuestefeld wrote:
| The fact that the USA and others are also trending
| authoritarian isn't really relevant. The point I was
| trying to make is that people have legit fears of the PRC
| government, enough so that legitimate business like
| settling a deceased parent's affairs isn't sufficient to
| convince people to enter the country.
|
| You haven't addressed at all the parts about blacklisting
| whole families for political reasons, or horrible return-
| to-normal policies for covid-19 three years ago, or the
| general pendulum-swing-back-to-evil trend.
| hungryhobo wrote:
| i don't doubt your experience, but just know it might be
| skewed and not representative of everyone's opinions
|
| the sense i get from my chinese friends are that the CCP is
| an annoying parent but they understand the challenges both
| domestic and international and largely agree with the
| compromises
| sarchertech wrote:
| >And as for foreigners, our view of China is far worse than
| it actually is, China doesn't need to hide anything, people
| who visit China will come away with a more positive view of
| the country than those who do not visit.
|
| To the extent that's true, it's because they won't let you
| see the uyghur reeducation camps.
| hungryhobo wrote:
| it's been debunked a million times, gotta move on
| sarchertech wrote:
| https://www.icij.org/investigations/china-
| cables/beijings-ba...
| MaxPock wrote:
| We can get videos from remote hellholes of Africa like
| Dafur and Mali but apparently,that's too much to ask in
| Xinjiang.We can't even get satellite images to show us
| evidence of this so called wigur genocide
| sarchertech wrote:
| https://www.abc.net.au/news/2018-11-01/satellite-images-
| expo...
|
| https://www.bbc.co.uk/news/extra/85qihtvw6e/the-faces-
| from-c...
| xanthor wrote:
| If you didn't have British Crown state media wrapping a
| narrative around these images you wouldn't think anything
| of them.
| sarchertech wrote:
| Would you take a group of Swiss journalists?
|
| https://gijn.org/stories/interview-uyghur-victims-
| xinjiang-p...
|
| How about the UN Office of the High Commissioner for
| Human Rights?
| Thlom wrote:
| On the other hand you can travel to Xinjiang, visit
| mosques, Uighur museums, experience Uighur culture,
| observe Uighurs just minding their own business in their
| daily life.
| sarchertech wrote:
| "Subjected to arbitrary arrests and forced labor,
| sterilizations to torture, more than one million Uyghurs,
| Kazakhs, Kyrgyz, and other minorities are estimated to
| have been locked up in so-called "re-education" camps and
| prisons in the region over the last decade, according to
| the UN Office of the High Commissioner for Human Rights."
|
| https://gijn.org/stories/interview-uyghur-victims-
| xinjiang-p...
| xanthor wrote:
| UN High Commissioner on Human Rights Michelle Bachelet
| actually visited Xinjiang and made no such assertions.
| Whoever did release the report you're referencing, they
| waited until immediately after her term ended to release
| it (within hours). Pretty conspicuous.
| sarchertech wrote:
| No it was actually released hours before her term ended
| not after. And the reason she held off releasing until
| the last minute is because of pressure from China to
| refrain from releasing it.
| hmmmmmmmmmmmmmm wrote:
| Same thing happened to me, except it was a facetime call from
| Xi.
| chausen wrote:
| Lol
| jajuuka wrote:
| I love Hacker News fiction. Wild stuff. haha
| imwillofficial wrote:
| This risk is far overstated.
|
| I was talking crap about china from the great wall.
| carabiner wrote:
| What compels people to just make shit up like this? What's
| next, you're gonna say everyone was carrying little red books?
| Was there a telescreen in the bathroom?
| mrdependable wrote:
| Wow, our surveillance helped take down their surveillance. Yay, I
| guess?
| AceJohnny2 wrote:
| "Our glorious oversight vs their barbaric surveillance"
|
| (I kid, mostly. While the US certainly isn't pure, its scale of
| surveillance intrusion is light compared to China)
| pcthrowaway wrote:
| > While the US certainly isn't pure, its scale of
| surveillance intrusion is light compared to China
|
| I assume that for someone to believe this, they either have
| to believe the U.S. has poorer surveillance capability than
| China, or, more likely, they consider U.S. surveillance
| unintrusive and Chinese surveillance intrusive.
| dddddaviddddd wrote:
| More interesting than the fact that ChatGPT was used, was seeing
| all the specific examples of the types of work that this
| individual was doing.
| layer8 wrote:
| I wonder what exactly the trigger conditions are that lead to the
| chats of an account being human-reviewed by OpenAI.
| simlevesque wrote:
| I'm pretty sure they can just prompt any convo in the
| background and ask "is this conversation sensitive ?" and the
| model can answer without this being added to the context of the
| convo.
| spwa4 wrote:
| "Is this someone important enough to spy on?"
|
| One hopes the CIA/Secret service would be willing to provide
| the human to do the reviewing but sadly I've worked for
| European telco's and I know better.
| dlev_pika wrote:
| Sounds like Anthropic is fighting this exact battle, and DOD
| is arguing they don't want to do that lol
| rrr_oh_man wrote:
| > sadly I've worked for European telco's and I know better
|
| Can you elaborate?
| LightBug1 wrote:
| This is -the- question.
| coliveira wrote:
| So, it seems they're openly admitting that OpenAI is a
| surveillance mechanism used at the discretion of the US gov.
| morkalork wrote:
| Such a hot topic these days:
| https://www.cbc.ca/radio/frontburner/chatgpt-and-the-tumbler...
| layer8 wrote:
| This is the report on which the CNN article is based (which it
| doesn't link to):
| https://cdn.openai.com/pdf/df438d70-e3fe-4a6c-a403-ff632def8...
| gitpusher wrote:
| > "This is what Chinese modern transnational repression looks
| like," Ben Nimmo, principal investigator at OpenAI, told
| reporters ahead of the report's release. "It's not just digital.
| It's not just about trolling. It's industrialized. [...]
|
| There's something poetic about OpenAI being asked to comment on
| mis-use of their slop generator, and their answer is composed
| entirely of AI slop.
| 2OEH8eoCRo0 wrote:
| > "It's not just digital. It's not just about trolling. It's
| industrialized. It's about trying to hit critics of the CCP
| [Chinese Communist Party] with everything, everywhere, all at
| once."
| CrzyLngPwd wrote:
| Pushing aside the fact that OpenAI is just a tool of the US
| regime.
|
| Will OpenAI release the same for other government officials from
| any other states?
|
| I can't wait to see Starmer's chats with ChatGPT.
|
| Anyway, all of this smells like 1934, "accusing them of what we
| are already doing"
| guelo wrote:
| I'm assuming they would not disclose such campaigns by the US
| government.
|
| I can't imagine the amount of government secrets, trade secrets,
| business plans, personal secrets, etc that people divulge on
| there.
| kykat wrote:
| The amount of information about everything that people are giving
| OpenAI is astronomical, information that was previously kept
| closely guarded is now just freely flowing through foreign
| servers.
|
| Truly a paradise for american intelligence. Would have expected
| that the chinese officials be briefed on not using us tech
| companies, but opsec is hard to teach, and even harder to always
| follow.
| simmerup wrote:
| But the american silicon valley nerds pinky swear not to look!
|
| How can you not trust them.
| bucketdeveloper wrote:
| Did they though?
|
| I never got to the end of the Terms & Conditions myself.
| dlev_pika wrote:
| Crazy to me that Chinese officials use ChatGPT to discuss
| sensitive operations lmao
| tehjoker wrote:
| i kinda get the impression this was from 2023 and also it is not
| clear what this dissident did, hard to evaluate whether i should
| care without knowing that
| romulussilvia wrote:
| I remember a while back when a few cars with CCP decals driving
| around SoCal to intimidate some dissidents!
| lxgr wrote:
| This seems to be the source report:
| https://openai.com/index/disrupting-malicious-ai-uses/ (since it
| would of course kill CNN, like almost all media outlets, to link
| to a non-affiliated primary source...)
|
| Does this level of detail seem strange to anybody else? Shining
| such a strong light on OpenAI's moderation/manual review efforts
| seems like it would draw unwanted attention to the fact that
| ChatGPT conversations are anything but private, and seems
| somewhat at odds with their recent outrage about the subpoena for
| user chats in the NYT case.
|
| Manual reviews of sensitive data are ok as long as their own
| employees are the reviewers, I suppose?
| coliveira wrote:
| Yes, it is either a lie or an admission that OpenAI is a global
| surveillance mechanism.
| andai wrote:
| Alas! My vision of One Fed Per Child hath come to pass!
| jajuuka wrote:
| This feels very planted. Wouldn't be surprised if this some
| attempt to look patriotic with the DoW turning up the heat
| against Anthropic.
| ticulatedspline wrote:
| that creepy feeling of "being watched" has mostly kept me from
| taking advantage of any SOTA models, i only dabble in a few
| local ones.
|
| The level of detail does not seem surprising. they're both
| charged with maintaining a facade of privacy while eliminating
| any and all miss-use. Certainly they heavily analyze basically
| everything given to them.
|
| And generally as a society we've been ok with basically zero
| privacy as long as the data we send stays inside the company we
| sent it too. Google reads all your emails? Sure thing, read
| away, just don't send them to the popo. Apple knows when you're
| ovulating? no problem, just don't tell Amazon. etc
| Palmik wrote:
| From Anthropics recent blog post:
| https://www.anthropic.com/news/detecting-and-preventing-dist...
|
| > By examining request metadata, we were able to trace these
| accounts to specific researchers at the lab.
|
| > The volume, structure, and focus of the prompts were distinct
| from normal usage patterns
|
| Clearly some employees of Anthropic personally looked at
| individual inputs and outputs of their API
| tehjoker wrote:
| Reading the report, they point to some shitty tweets with like
| 12 views and at most a handful of likes. Reminds me of
| "Russiagate"
|
| When governments and their major corporations level accusations
| at countries that are regarded as official enemies, you have to
| ask two questions (1) what is the actual evidence and (2) what
| is the level of impact
|
| Meanwhile when you want to ask about U.S. influence
| operations... like they time we probably killed a lot of people
| by spreading disinformation about the COVID vaccine because we
| didn't want China to look good.
|
| https://www.aljazeera.com/news/2024/6/14/pentagon-ran-secret...
| cc-d wrote:
| There is mass neurocompromise, assigning agency to specific state
| actors does not make much sense.
|
| Big tech is well aware of this, and much of their industry,
| relies on this. Many people reading this will know this to be
| true, and are very saddened by this, but keep their mouths shut,
| because they are 1. comfy 2. smart (we all know it does not
| matter)
|
| This is such common knowledge that I feel kind of cringe even
| posting about this, but I am not being given a choice, nor a
| choice in this edit.
|
| Edit: If anybody wants to chat to somebody who has had his
| organism compromised by what is, very deniablely, an intelligence
| agency (or a system/organization intelligence agency adjacent),
| just reply to this comment. They have not treated me too poorly.
| zoklet-enjoyer wrote:
| Very creepy on the part of Open AI. Glad I don't use chatgpt
___________________________________________________________________
(page generated 2026-02-27 23:00 UTC)