[HN Gopher] Disrupting a covert Iranian influence operation
___________________________________________________________________
Disrupting a covert Iranian influence operation
Author : saikatsg
Score : 80 points
Date : 2024-08-16 18:39 UTC (4 hours ago)
(HTM) web link (openai.com)
(TXT) w3m dump (openai.com)
| ImHereToVote wrote:
| I wonder if it would be possible to get a list of countries that
| can have influence operations using ChatGPT and countries that
| can't.
| janice1999 wrote:
| > Russia, China, Iran, and Israel
|
| https://www.npr.org/2024/05/30/g-s1-1670/openai-influence-op...
| ein0p wrote:
| This doesn't pass the sniff test. Each of these countries has
| domestically developed, strong LLMs. And even if they did
| not, they'd have absolutely no issue running the larger FOSS
| ones. You don't need GPT4 to generate political drivel.
| axus wrote:
| Sounds like OpenAI marketing is stronger than patriotism
| for home-grown LLMs.
| ein0p wrote:
| More likely someone is trying to pass a few basement
| dwellers as major nation state influence operations.
| kgeist wrote:
| Indeed, Russia has YandexGPT and Gigachat which are more
| than enough for generating articles (although they're far
| worse at reasoning).
| dtquad wrote:
| It takes a lot of GPU to run the larger FOSS LLMs. The
| OpenAI API is still the cheapest way to get high-quality
| LLM generations.
| ein0p wrote:
| Tell that to someone else. I run LLaMA70B in my garage.
| Works fine, cost is negligible.
| wslh wrote:
| Those four results only show the metapropaganda limitations.
| There are more countries in the world that are less "fancy"
| and included in the list. Weird that you don't include
| America though. BTW, your comment reminds me this very basic
| espionage issue [1].
|
| [1] https://www.reddit.com/r/todayilearned/comments/bdjh8o/ti
| l_t...
| lancesells wrote:
| I don't think that would matter. I live in the US and I would
| guess there are plenty of people in this country that work for
| countries that aren't allowed to use it. Same goes for other
| countries having people who work for the US.
|
| I really don't think it would take that many people to run
| propoganda campaigns with modern tools.
| hobo_in_library wrote:
| Only countries with dedicated PsyOps divisions are allowed to
| use ChatGPT for influence operations, obviously
|
| https://en.wikipedia.org/wiki/Psychological_operations_(Unit...
| colordrops wrote:
| I hope they put the same restrictions on Israel but I doubt it.
| Multiple core OpenAI team members have expressed pro-Israel
| comments, some very murderous and ugly, so I doubt their
| ability to be unbiased here.
| jjmarr wrote:
| https://www.nbcnews.com/tech/security/meta-openai-say-
| disrup...
|
| https://openai.com/index/disrupting-deceptive-uses-of-AI-
| by-...
|
| OpenAI has also banned Israeli influence operations. Do you
| think the above isn't going far enough?
|
| It's also strange that you call out all pro-Israel comments
| as a cause for concern.
| akira2501 wrote:
| Wow. What a high value target. [0]
|
| I mean. I get that low rent actors will use low rent services to
| try to generate political garbage. Is there any evidence that
| this is actually having a measurable or meaningful impact?
|
| Who exactly is fooled by these sites? And is it the sites that
| are the problem or the relative lack of sophistication in
| American education when it comes to political corruption?
|
| [0]: https://niothinker.com/
| katzinsky wrote:
| Yeah. Knowing my tools are judging my political positions and
| could self destruct if the authors disagree just makes me love
| using my computer.
|
| This is why I only use local models these days.
|
| EDIT: Out of posts for today but I've been pretty happy with
| Gemma2. The context is short but the performance is very good and
| it's easy to disable refusal.
| guerrilla wrote:
| Just curious, which local models?
| tonetegeatinst wrote:
| I know the Facebook model was good. Checkout r/locallamma for
| guidance, or check the ranking website.
| lambdaba wrote:
| The only thing noteworthy about this is how small-scale this is,
| and that the perpetrators don't even bother/have the means to set
| up their own infrastructure.
| chatmasta wrote:
| The noteworthy thing is that OpenAI is reporting it. They're
| signaling that they are proactively monitoring and
| investigating this activity, and that they're willing to work
| with federal agencies while self-policing their negative
| externalities.
|
| This is all part of an ongoing conversation with lobbyists
| about "safe AI," and it's ultimately done to show that OpenAI
| is making an effort to mitigate the risks that regulators claim
| it creates.
|
| But there's also another signal, which is what they're _not_
| broadcasting: "ChatGPT can be used for propaganda, it works in
| Persian too, and we're happy to sell to the DoD."
| lancesells wrote:
| It's a good thing for them to report this, but for a company
| that decided against watermarking their output they are kind
| of complicit.
|
| I know they might lose their teens and college demographic if
| they do it, but if it's truly this world-changing tool that
| they claim than not watermarking is scraping the serial
| number off the guns they're selling.*
|
| * Maybe a shitty analogy
| programmarchy wrote:
| The linked PDF (Storm-2035 [1]) from Microsoft is more detailed
| and interesting than the blog post. However, what's missing from
| the reports is _how_ they detected those operations and how they
| tied them to different groups. There 's a lot of claims being
| made without showing all of the supporting evidence.
|
| To give them the benefit of the doubt, they likely want to keep
| their detection methods secret to make circumvention more
| difficult. And it all sounds totally plausible of course. But at
| the same time, a degree of skepticism is warranted because
| Microsoft has a huge incentive to fearmonger about AI so they can
| lock it down and capture the market. And what better way is there
| than to use the usual bogeymen to do so.
|
| [1] https://cdn-
| dynmedia-1.microsoft.com/is/content/microsoftcor...
| com2kid wrote:
| Having worked at Microsoft for almost a decade, I remember
| chatting with their security people plenty after meetings. One
| interesting thing I learned is that Microsoft (and all the
| other top tech companies presumably) are under constant
| Advanced Persistent Threat from state actors. From literal
| secret agents getting jobs and working undercover for a decade+
| to obtain seniority, to physical penetration attempts (some
| buildings on MS campus used to have armed security, before
| Cloud server farms were a thing!).
|
| Microsoft is one of the few companies that goes toe to toe with
| world governments every day of the year.
|
| And I imagine balancing that next to all the engineers who
| demand admin access everywhere is a royal pain!
|
| Although the best Government VS Business story I heard was
| during intern orientation at Boeing about French agents
| breaking into Boeing employee's hotel rooms during a conference
| in France while the employees were out to dinner, and going
| through laptops. One of the employees returned earlier than
| expected, and the men in suits shut the laptop, turned around
| and walked out of the room w/o saying anything!
| BoingBoomTschak wrote:
| > Microsoft is one of the few companies that goes toe to toe
| with world governments every day of the year.
|
| It's also the company which was the first and longest member
| of PRISM, meaning very deep ties to the less savory parts of
| the US gov and {five,nine,fourteen} eyes. I know it's a
| boring advice, but I'd take this kind of declaration with a
| truckload of salt.
| greatgib wrote:
| What they don't say on their post but that we can guess and is
| from interest is that they probably had to spy their user
| messages to determinate that they used the account for generate
| content for the influence operation.
|
| For sure the purpose is noble, but it is good to remind everyone
| that everything you type, submit or generate there is not private
| but could be randomly snooped by strangers!
| szundi wrote:
| Naah, probably they used OpenAI apis to read, summarize and
| categorize everything
| TOMDM wrote:
| That or they could store hashes of chunks of output to compare
| to propaganda in the wild.
| simonw wrote:
| From a Google search it looks like this is one of the articles in
| question: https://teorator.com/index.php/2024/08/12/x-censors-
| trumps-t...
|
| I requested the Internet Archive grab this copy:
| https://web.archive.org/web/20240816210620/https://teorator....
| throw310822 wrote:
| Looks exceedingly good to be Ai-generated. Also, it appears to
| be slightly pro-Trump, which doesn't sound like in Iran's
| interest.
| stickfigure wrote:
| It is clear to me that Russia, Iran, North Korea are playing
| a zero-sum game. Anything that creates chaos in US politics
| (or European politics) is in their interest. Donald Trump
| represents chaos.
| ignoramous wrote:
| > _Russia, Iran, North Korea_
|
| You're missing China & Israel: https://www.theguardian.com/
| technology/article/2024/may/30/o... /
| https://archive.is/y0V3A
|
| And, according to OpenAI, they're interfering not just in
| the US: https://www.thehindu.com/elections/lok-
| sabha/openai-says-sta... / https://archive.is/RWn8X
| cedws wrote:
| They want to sow discord. Manipulating US politics in any
| meaningful way is difficult to do. It's easier to divide
| and conquer and turn your enemy against itself.
|
| In my opinion the West is already doing a good enough job
| at polarising itself, doesn't need much help from the
| enemy.
| zmgsabst wrote:
| Of course we are:
|
| Domestic enemies came to the same conclusion.
| gwervc wrote:
| > Donald Trump represents chaos.
|
| Very strange perspective when talking about the _only_ US
| president of our times that didn 't start a war, made
| historical diplomatic advances with North Korea, made a
| deal with talibans to end war in Afghanistan. I'm probably
| forgetting a few but his track record is impressive.
| simonw wrote:
| From the article:
|
| > Some of the X accounts posed as progressives, and others as
| conservatives.
|
| My impression is that these influence campaigns know that
| they need to produce a LOT of content that appears to reflect
| different opinions and perspectives if they're going to
| appear to be "real" - that way they can build trust with an
| audience before attempting to influence them in one direction
| or another.
| Bluestein wrote:
| (it's funny ...
|
| ... until then, they might end up being - even if
| inadvertently - more objective than most outlets.-
| lukev wrote:
| Really? Trump has always been much weaker in diplomacy with
| nations that are historically antagonistic to the US. He'll
| have a meeting with their leader, let them flatter him, and
| come away convinced that they're "very nice" and make a lot
| of concessions.
|
| He's basically an admirer of Putin at this point.
| chrisco255 wrote:
| The Abraham accords were and are a good development he
| implemented. The Obama administration in 2015 attempted to
| bribe Iran into not enriching uranium by effectively paying
| them $100 billion.
| blackguardx wrote:
| Can you cite a source on that? That doesn't match my
| understanding of the events.
| dtquad wrote:
| >Iran's interests
|
| Nobody has cared about "Iran's interests" since 1979. The
| rulers of Iran follow their own pan-Shia Islamist ideology
| that puts "Iran's interests" dead last.
|
| They wouldn't mind baiting another warmongering US president
| into another unwinnable 20+ year trillion dollar war.
| littlestymaar wrote:
| Their reaction since Hamas' attempted to drag them into the
| war shows the opposite: they showed they'd rather take
| humiliating blows rather than go to war.
| joe_the_user wrote:
| I don't think people should jump to any "this is the level Iran
| is at?" conclusions.
|
| Many nations employ "patriotic citizens" in an informal and semi-
| formal fashion along with trained propaganda and "infowar"
| experts. I know of China and Israel doing this but I'd assume
| it's everywhere.
| det2x wrote:
| Iran is definately putting out mass propaganda especially on
| social media.
|
| Israel has one of if not the largest propaganda/bot campaigns
| on the entire planet, followed by either Russia or China. It's
| definately working for them so they won't be stopping anytime
| soon.
| djaouen wrote:
| This is as much an indictment of ChatGPT as it is of the
| Iranians. According to OpenAI, their product produces output that
| no one in their right mind would want to read for any purpose.
| echoangle wrote:
| How is that a bad thing for OpenAI though? It depends on the
| prompting, I wouldn't count the ability to generate
| useless/stupid/misleading content when prompted to do this as a
| negative.
| hobo_in_library wrote:
| It's hard to take an article like this at face value when they
| provide zero evidence for any of their claims.
|
| This is coming soon after Trump decided to accuse Iran of being
| behind his assassination attempt (done by a white 20 year old)
| and Israel literally assassinated Hamas's chief negotiator while
| he was visiting Iran.
|
| It seems like the powers that be are desperate for a war with
| Iran and will continue beating the drum to build consent.
|
| Reminds me of the build up to the 2003 Iraq invasion (you know,
| because "they have WMDs")
| usefulcat wrote:
| I'd be more interested in an analysis of the likely intention of
| the campaign. Is it just an attempt to reduce voter turnout? If
| so, that doesn't seem all that useful by itself.
| dfhvneoieno wrote:
| Iran wants Democrats, Russia wants Republicans.
|
| It's easy to see which one is more successful. Every time you
| blame something on Russia, people come out of the woods
| "explaining" why Russia is actually the victim (Russia is
| ALWAYS the victim, is what I've learned from this people).
| epolanski wrote:
| Russia definitely has a non zero amount of people supporting
| or apologizing it's actions all around the world.
|
| It's dangerous to assume all of the content you see is simply
| some shady operation or fake, having controversial opinions
| on any topic is nothing new, millions of Americans don't
| believe in the moon landings.
|
| In general I don't think it's a good idea to discuss politics
| online, and I feel like many talk too much geopolitics but
| can't even tell the policies or programs of their own city
| mayor candidates which are way more important for their own
| lives and directly impacting.
| aguaviva wrote:
| In regard to local issues being way more impactful in one's
| life:
|
| If one is from, or has close contacts in or from the
| regions affected by geopolitics -- then this is
| unfortunately very much not the case.
|
| As disappointed as I am with local politicians -- they
| aren't bombing my friends into the ground, or forcing them
| to move halfway across the planet indefinitely. Or causing
| these to deal with significant emotional anguish, even if
| they aren't forced to move, or directly under threat.
|
| While a solid contingent on HN regularly either apologies
| for / minimizes their actions, or just seems to shoot from
| its hip based on their hunch as to what is happening,
| without any indication of having done much research or
| questioning their sources. Or even just thinking logically
| about the various narratives they're reading. For most of
| them it seems to be more about ideological abstractions
| than anything real and concrete, in any case.
|
| So that, plus the simple fact that this is a global
| community is why geopolitics floats to the top, as it were.
|
| Whether talking about it online is productive or helpful in
| any way is another matter altogether. Turns out it's
| generally not easy to talk about these things in person,
| even with people one knows rather well (as most of the time
| they'd prefer to talk about pretty much anything but "the
| situation").
|
| Being as the "shit" in question is all too real for them.
| PerilousD wrote:
| These jokers seem like the AI version of "script kiddie" hackers,
| and OpenAI may be engaging in a bit of humble bragging. It
| doesn't take considerable investments in time or money to run
| local LLMs, INCLUDING ChatGPT, where your questions, prompts, and
| results are not sent home to the mothership, so it's a BS article
| as to (the real) actors who may or may not be doing this. NOW, if
| OpenAI or Gemini or LLama, etc, showed how they analyzed social
| media posts and flagged the ones that were AI generated and the
| analysis as to WHY the article is flagged, then that would be
| much more useful, actionable by at least some of the readers and
| would put the accounts spreading the content (particularly the
| rebroadcast fluffers) in the spotlight.
| Arainach wrote:
| It wouldn't be useful at all, and would only serve to educate
| malicious actors how to better evade detection.
|
| It's like claiming a search engine open sourcing its ranking
| algorithm would help people be informed instead of making
| spammers able to perfectly hijack all the results.
| commandpaul wrote:
| Given that any self hosted open source model would have worked
| just as well. I can't see this good faith post as anything more
| than forwarding open ais long campaign for regulatory capture.
| dtquad wrote:
| Self-hosting LLMs is expensive at scale. It's cheaper to use VC
| subsidized model inference like the OpenAI APIs.
| CyberDildonics wrote:
| At "scale"? At what scale?
| littlestymaar wrote:
| There are plenty of VC-subsidized inference provider which
| uses open source LLM for much cheaper than OpenAI (which
| isn't really VC-subsidized at this point but Microsoft-
| subsidized).
| refulgentis wrote:
| My anecdata is most teams I've talked to say its below OpenAI
| at scale, and vLLM is a beast. It's interesting to hear the
| opposite, there's lots of cheaper providers, but the "VC
| dollars" argument can go "turtles all the way down", I
| suppose. Still, reality seems to differ.
| ComplexSystems wrote:
| Why wouldn't Iran just use Llama or something?
___________________________________________________________________
(page generated 2024-08-16 23:00 UTC)