[HN Gopher] Large language models in national security applications
___________________________________________________________________
Large language models in national security applications
Author : bindidwodtj
Score : 63 points
Date : 2024-11-12 17:58 UTC (5 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| robertkoss wrote:
| Some related news: https://investors.palantir.com/news-
| details/2024/Anthropic-a...
| Syonyk wrote:
| Abstract:
|
| > _The overwhelming success of GPT-4 in early 2023 highlighted
| the transformative potential of large language models (LLMs)
| across various sectors, including national security. This article
| explores the implications of LLM integration within national
| security contexts, analyzing their potential to revolutionize
| information processing, decision-making, and operational
| efficiency. Whereas LLMs offer substantial benefits, such as
| automating tasks and enhancing data analysis, they also pose
| significant risks, including hallucinations, data privacy
| concerns, and vulnerability to adversarial attacks. Through their
| coupling with decision-theoretic principles and Bayesian
| reasoning, LLMs can significantly improve decision-making
| processes within national security organizations. Namely, LLMs
| can facilitate the transition from data to actionable decisions,
| enabling decision-makers to quickly receive and distill available
| information with less manpower. Current applications within the
| US Department of Defense and beyond are explored, e.g., the USAF
| 's use of LLMs for wargaming and automatic summarization, that
| illustrate their potential to streamline operations and support
| decision-making. However, these applications necessitate rigorous
| safeguards to ensure accuracy and reliability. The broader
| implications of LLM integration extend to strategic planning,
| international relations, and the broader geopolitical landscape,
| with adversarial nations leveraging LLMs for disinformation and
| cyber operations, emphasizing the need for robust
| countermeasures. Despite exhibiting "sparks" of artificial
| general intelligence, LLMs are best suited for supporting roles
| rather than leading strategic decisions. Their use in training
| and wargaming can provide valuable insights and personalized
| learning experiences for military personnel, thereby improving
| operational readiness._
|
| I mean, I'm glad they suggest that LLMs be used in "supporting
| roles rather than leading strategic decisions," but... no? Let's
| please not go down this route for international politics and
| national security. "Twitch Plays CIA" and "Reddit Plays
| International Geopolitical Negotiations" sound like bad movies,
| let's not make them our new reality...
| gmaster1440 wrote:
| The paper argues against using LLMs for military strategy,
| claiming "no textbook contains the right answers" and strategy
| can't be learned from text alone (the "Virtual Clausewitz"
| Problem). But this seems to underestimate LLMs' demonstrated
| ability to reason through novel situations. Rather than just
| pattern-matching historical examples, modern LLMs can synthesize
| insights across domains, identify non-obvious patterns, and
| generate novel strategic approaches. The real question isn't
| whether perfect answers exist in training data, but whether LLMs
| can engage in effective strategic reasoning--which increasingly
| appears to be the case, especially with reasoning models like o1.
| ben_w wrote:
| LLMs can combine cross-domain insights, but the insights they
| have -- that I've seen them have in the models I've used -- are
| around the level of a second year university student.
|
| I would concur with what the abstract says: incredibly valuable
| (IMO the breadth of easily discoverable knowledge is a huge
| plus all by itself), but don't put them in charge.
| gmaster1440 wrote:
| The "second year university student" analogy is interesting,
| but might not fully capture what's unique about LLMs in
| strategic analysis. Unlike students, LLMs can simultaneously
| process and synthesize insights from thousands of historical
| conflicts, military doctrines, and real-time data points
| without human cognitive limitations or biases.
|
| The paper actually makes a stronger case for using LLMs to
| enhance rather than replace human strategists - imagine a
| military commander with instant access to an aide that has
| deeply analyzed every military campaign in history and can
| spot relevant patterns. The question isn't about putting LLMs
| "in charge," but whether we're fully leveraging their unique
| capabilities for strategic innovation while maintaining human
| oversight.
| ben_w wrote:
| > Unlike students, LLMs can simultaneously process and
| synthesize insights from thousands of historical conflicts,
| military doctrines, and real-time data points without human
| cognitive limitations or biases.
|
| Yes, indeed. Unfortunately (/fortunately depending on who
| you ask) despite this the actual quality of the output is
| merely "ok" rather than "fantastic".
|
| If you need an answer immediately on any topic where
| "second year university student" is good enough, these are
| _amazing_ tools. I don 't have that skill level in, say,
| Chinese, where I can't tell Ni Hao (hello) from Ni Hao
| (mud hole/trench)* but ChatGPT can at least manage mediocre
| jokes that Google Translate turns back into English:
|
| Wen : Shi Yao Dong Xi Yue Xi Yue Zang ? Da : Shui !
|
| But! My experience with LLM translation is much the same as
| with LLM code generation or GenAI images: anyone with
| actual skill in whatever field you're asking for support
| with, can easily do better than the AI.
|
| It's a fantastic help when you would otherwise have an
| intern, and that's a lot of things, but it's not the right
| tool for every job.
|
| * I assume this is grammatically gibberish in Chinese, I'm
| relying on Google Translate here:
| https://translate.google.com/?sl=zh-TW&tl=en&text=Ni %20Hao
| %20%2...
| psunavy03 wrote:
| But the aide won't have deeply analyzed every military
| campaign in history; it will only spout off answers from
| books about those campaigns. It will have little to no
| insight on how to apply principles and lessons learned from
| similar campaigns in the current problem. Wars are not won
| by lines on maps. They're not won by cool gear. They're won
| by psychologically beating down the enemy until they're
| ready to surrender or open peace negotiations. Can LLMs get
| in an enemy's head?
| fragmede wrote:
| Only if the enemy has provided a large corpus of writing
| and other data to submit to train the LLM on.
| ben_w wrote:
| > Can LLMs get in an enemy's head?
|
| That may be much easier for an LLM than all the other
| things you listed.
|
| Read their socials, write a script that grabs the voices
| and faces of their loved ones from videos they've shared,
| synthesise a video call... And yes, they can write the
| scripts even if they don't have the power to clone voices
| and faces themselves.
|
| I have no idea what's coming. But this is going to be a
| wild decade even if nothing new gets invented.
| psunavy03 wrote:
| Creating chaos and confusion is great, but it's only part
| of what a military campaign needs. You have to be able to
| use all levers of government power to put the other
| government or the adversary organization in a point where
| they feel compelled to quit or negotiate.
| ben_w wrote:
| Aye.
|
| FWIW, I hope all those other things remain a long way
| off.
|
| Whoever's doing war game planning needs to consider the
| possibility of AI that can do those other things, but I'm
| going to have to just hope.
| JohnMakin wrote:
| The person you are responding to seems to be promoting a
| concept that is frequently spouted here and other places,
| but to me lacking sufficient or any evidence - that AI
| models, particularly LLMs, are both capable of reasoning
| (or what we consider reasoning) around problems and
| generating novel insights that it hasn't been trained on.
| nyrikki wrote:
| You are using a different definition of strategic than the DoD
| uses, what you are describing is closer to tactical decisions.
|
| They are talking about typically Org wide scope, long-term
| direction .
|
| They aren't talking about planning hidden as 'strategic
| planning' in the biz world.
|
| LLMs are powerful, but are by definition past focused, and are
| still in-context learners.
|
| As they covered, hallucinations, adverse actions, unexplainable
| models, etc are problematic.
|
| The "novel strategic approaches" is what in this domain would
| be tactics, not stratagy which is focused on the unknowable or
| unknown knowable.
|
| They are talking about issues way past methods like
| circumscription and the ability to determine if a problem can
| be answered as true or false in a reasonable amount of time.
|
| Here is a recent primer on the complexity of circumscription as
| it is a bit of a obscure concept.
|
| https://www.arxiv.org/abs/2407.20822
|
| Remember, finding an effective choice function is hard no
| matter what your problem domain is for non trivial issues,
| setting a durable shared direction to communicate in the
| presence of the unknowable future that can't be gamed or
| predictable by an advisory is even more so.
|
| Researching what mission command is may help understand the
| nuances that are lost with overloaded terms.
|
| Strategy being distinct from stratagem is also an important
| distinction in this domain.
| paganel wrote:
| > but are by definition past focused,
|
| To add to that, and because the GP had mentioned (a
| "virtual") Clausewitz, "human"/irl strategy itself has in
| many cases been too focused on said past and, because of
| that, has caused defeats for the adopters of those "past-
| focused" strategies. Look at the Clausewitzian concept of
| "decisive victory" which was adopted by German WW1
| strategists who, in so doing, ended up causing defeat for
| their country.
|
| Good strategy is an art, the same as war, no LLM nor any
| other computer code would be ever able to replicate it or
| improve on it.
| beardedwizard wrote:
| A language model isn't a model of strategic conflict or
| reasoning, but may contain text in its training data related to
| these concepts. I'm unclear why (and it seems the paper agrees)
| you would use the llm to reason when there are better models
| for reasoning about the problem domain - and the main value
| from llm is ability to consume unstructured data to populate
| the other models.
| perihelions wrote:
| The most obvious way the US national security industry could use
| LLM's right now is simply to spam foreign adversaries with
| chatbots. That's their greatest strength right now--a use-case
| they have _amply_ proven themselves for.
|
| This paper comes off as eager to avoid this topic: they (briefly)
| talk about _detecting_ foreign LLM spam, which is called
| propaganda, but sidestep the idea of our own side using it. If we
| were considering talking about that, we wouldn 't choose
| negative-sentiment descriptors like (quoting the paper) "nation-
| state sponsored propaganda", or "disinformation campaigns"; we'd
| use our own netural-sentiment jargon, which is "psychological
| operations" ("psyops") [0].
|
| That we're not widely debating this question _right now_
| *probably* means it 's far too late to have a chance of stopping
| it.
|
| edit: Or, to rephrase this as a question: Is it ethical to spam
| another democracy with millions of chatbots pretending to be
| citizens of that country--if the effect is to manipulate those
| citizens to not go to war with our own side, saving our own
| lives? Is that an atrocity or is that legitimate warfare?
|
| [0]
| https://en.wikipedia.org/wiki/Psychological_operations_(Unit...
| joe_the_user wrote:
| Oh, national security professionals aren't going to be talking
| about psyops, offensive applications and etc, because such
| things make a given state look bad - they're an offense against
| democracy and respect for facts and they make the overt media
| of a given nation look bad. But hey, leave it to HN
| commentators to root for taking the gloves off. Not to worry
| post, I'd bet dollars to donuts the actual classified
| discussions of such things aren't worried about such niceties.
| But even more, in those activities of the US and other secret
| states, that have come to light, these state have propagandized
| not only enemy populations but also their own. After, have to
| counter enemies trying to nefariously prevent wars as well.
| bilbo0s wrote:
| _Or, to rephrase this as a question: Is it ethical to spam
| another democracy with millions of chatbots pretending to be
| citizens of that country--if the effect is to manipulate those
| citizens to [take any action advantageous to US national
| interest]...?_
|
| Just, Devil's Advocate, but ethical or not, that's what we
| should be doing and what we _are_ doing. Every nation has its
| sock puppets out there, our job is to stop everyone else ' sock
| puppets, and do everything we can to extend the reach of our
| own sock puppets.
| JoshTriplett wrote:
| That's not inherently true. If there were a way to reliably
| destroy _all_ the sock puppets, we should, and the world
| would be better off. For instance, reliable bot detection, or
| mechanisms by which major social networks could detect and
| prohibit bot-like activity that isn 't labeled as such.
| exe34 wrote:
| charge one penny for every post. most people can afford it.
| bots become less cost effective and you'd be able to trace
| the source of funds.
| recursive wrote:
| Even if they could afford it, they won't. The UX friction
| of charging money would send engagement off a cliff, even
| if the nominal charge was $0.
| exe34 wrote:
| that's why it should be mandated by law for national
| security reasons!
| Onavo wrote:
| The platform you are referring to is called "Reddit", one of
| YC's portfolio companies.
| Jerrrrrrry wrote:
| If the probability beats human error margin in regards to
| collateral damage, then sure.
|
| That was the sentiment in regards to Level 5 automaton driven
| vehicles.
|
| I see no logical difference, only human sentiment ones.
| joe_the_user wrote:
| The problem you have is there's no way to estimate probability
| in situations like warfare or similar chaotic environments.
| Jerrrrrrry wrote:
| Sure you do, it's accumulated heuristics, no different than
| meteorology, or other macro-sims of chaotic systems.
|
| The difference is that human lives are intentioned for
| different fates; so the negative cognitive dissonance is
| going to persist consciously, then sub-consciously.
| joe_the_user wrote:
| _it 's accumulated heuristics, no different than
| meteorology_
|
| Meteorology is based on physics, meteorology doesn't have a
| hostile agent attempt counter prediction attempts,
| meteorology doesn't involve a constantly changing
| technological landscape, meteorology has access to vast
| amounts data whereas data that's key to military decisions
| is generally scarce - you know the phrase "fog of war"?
|
| I mean, LLMs in fact, don't provide probabilities for their
| predictions and indeed the advance of deep learning has
| hinge "just predict, ignore all considerations of 'good
| statistics' (knowing probabilities, estimating bias)".
| htrp wrote:
| This is how you get CIAGPT
| kevmo wrote:
| PRISMGPT
| Jerrrrrrry wrote:
| https://en.wikipedia.org/wiki/Singleton_(global_governance)
| photochemsyn wrote:
| It's rather conspicuous that the most well-known use of AI
| systems in warfare at present, the Lavender / Gospel / Where's
| Daddy systems used by the IDF, don't get any mention. It's true
| that LLMs are not the central component of these systems, which
| have much more in common with Google's targeted ad serving
| algorithms, in the broader category of machine learning, but a
| no-code LLM interface is a likely component.
|
| In defensive red team scenarios, such an LLM system could be used
| in all kinds of nefarious ways, using prompts like "provide a
| list of everyone associated with the US nuclear weapons program,
| including their immediate friend and family circles, and ranking
| them by vulnerability to blackmail based on their personal web
| browsing history" and so on.
| ofslidingfeet wrote:
| Alternative title: "Obviously Irresponsible, Intellectually Lazy
| Things that We Definitely Haven't Been Doing for Fifteen Years."
___________________________________________________________________
(page generated 2024-11-12 23:01 UTC)