[HN Gopher] Most technologies aren't races
___________________________________________________________________
Most technologies aren't races
Author : HALtheWise
Score : 79 points
Date : 2023-04-05 16:52 UTC (6 hours ago)
(HTM) web link (astralcodexten.substack.com)
(TXT) w3m dump (astralcodexten.substack.com)
| jmartrican wrote:
| I disagree whole heartedly. There are winning countries for
| various technologies. Yes there is tech that can be outsourced to
| another country, passing the winning baton to that country.
| Countries can also spend a lot on R&D to reach parity with
| established winners, hence making multiple winners.
|
| Lets use car manufacturing example. Yes North America and Germany
| became early winners, and eventually Japan and Korea have become
| part of the winning club. But many other countries are not
| winning here. There are countries that can only make subpar cars
| for the local market, or make no cars at all and must import them
| all.
|
| So if they are winning, what exactly are these countries winning?
| They are winning money and influence. US and Japan sell more cars
| overseas, that's more money into their country and into the
| government's coffers.
| droopyEyelids wrote:
| Thanks this was my immediate objection, as well.
|
| And who can say that being an early computer & Internet center
| didn't privilege America? The majority of big Internet
| companies are American, and they're now collecting rent across
| the globe!
| Cthulhu_ wrote:
| Apple didn't "win" the smartphone race, but they had a five year
| head start and ended up becoming the most valuable (publicly
| traded) company in the world.
|
| Tesla didn't "win" the EV market (and will likely dwindle), but
| it did become the most valuable car company and made Musk the
| richest person in the world (for a time).
|
| OpenAI likely won't win the AI market, but they're landing multi-
| million deals left right and center. They've got the home
| advantage as well - US and EU companies don't want to deal with
| Chinese hard- or software.
|
| It's not so much about winning, it's about getting a head start.
| thomasahle wrote:
| It's easy to get a "head start" effect if you pick the top
| companies and make a cutoff line just before those companies
| launched their products.
|
| If you include pre-iphone smartphones and pre-tesla EVs, you'll
| get the opposite conclusion: Apple and Tesla demonstrated that
| a head start isn't necessary to surpass competitors and achieve
| market dominance. By making better technology you can leapfrog
| companies (like Nokia) that have been around for decades.
| anon84873628 wrote:
| The point of the article isn't that some companies/countries
| won't have advantages - he specifically points out all the
| examples where they do - but just that it isn't critical to AI
| alignment concerns.
| larkost wrote:
| Apple did not have any head start, when they entered the market
| there were already smartphones on the market (e.g.; BlackBerry
| and Palm Treo). Apple just brought out a significantly better
| product, and it took years for a good response to come out (and
| for the power of Apple's change to be appreciated fully).
|
| So this is evidence that the first move is not always the
| winning stroke.
| paxys wrote:
| Blackberry and Palm Treo and the rest were called
| "smartphones", sure, but let's be real Apple completely
| rebooted the category. Everything that existed before iPhone
| was instantly irrelevant. Its competitors has to start from
| scratch to incorporate large multitouch screens and reimagine
| the basic UI and apps.
| LudwigNagasena wrote:
| The first iPhone could be barely considered "smart"
| comparatively to its competitors. They weren't instantly
| irrelevant.
| paxys wrote:
| Every iPhone released in the last 15 years has been an
| incremental upgrade on the first one. And in turn every
| other smartphone has been a copy of the equivalent
| iPhone. No one is drawing inspiration from the top of the
| line Nokia or Blackberry from 2008. That entire segment
| is dead.
| nemothekid wrote:
| _Instantly irrelevant_ is certainly one way of looking at
| it. You are talking about a platform where developers
| originally had to beg for an appstore. Even Kim Kardashian
| was using a blackberry until 2015.
| 908B64B197 wrote:
| > Even Kim Kardashian was using a blackberry until 2015.
|
| How much did this cost BlackBerry?
| nemothekid wrote:
| I'm willing to bet they paid nothing; only for the reason
| that RIM completely screwed the pooch on mass market
| smartphones and exclusively focusing on enterprise. I
| remember BBM having incredible penetration in college-
| aged women but Blackberry didn't seem interested in
| anything other than enterprise until it was too late.
|
| Edit: I did a quick Google Search and she was buying them
| _on Ebay_. I think it was one of the many misplays by the
| company.
| jandrese wrote:
| Form factor wise there's no argument. The iPhone
| completely changed the design of smartphones. Every pre-
| iPhone smartphone form factor except for the Blackberry
| was dead within 2 years. Even the Blackberry died out
| over time, it was just sustained by the governments and
| businesses with multi-year contracts a bit longer.
| augment001 wrote:
| It's very unrealistic to believe that it was developers
| begging that caused Apple to create the App Store. It's
| possible the begging might have accelerated the process
| because apple wanted to capitalize on the demand, but
| there is no way there wouldn't have been an App Store.
| causi wrote:
| You could even argue the delay is what let Apple win. Apple
| jumped in the moment they could build a UI paradigm around
| fingers and capacitive screens. Its competitors had spent a
| decade building a UI scheme and software ecosystem around
| either trackballs and keyboards or resistive touchscreens and
| styluses, and weren't willing to throw that away when
| multitouch came along.
| 908B64B197 wrote:
| > You could even argue the delay is what let Apple win.
|
| Apple won because of superior design and engineering. It
| became glaringly obvious when BlackBerry released the Storm
| one month or so before or after Apple released the iPhone
| 3G (and the App Store).
|
| I remember thinking it was a prototype when someone showed
| a Storm to me. Scrolling was janky, the whole phone was
| sluggish and slow and the keyboard barely worked.
| Blackberry couldn't even figure out how to get Wifi on the
| thing, so it was stuck using cellular while my iPhone was
| just streaming video. And of course, no apps. It was 2
| years behind the original iPhone and shipped two years
| later.
| ghaff wrote:
| And there were plenty of people in 2007 happy to dunk on
| Apple for not having a physical keyboard and various other
| aspects of the design that didn't represent tradeoffs that
| people were accustomed to but turned out to be appropriate
| (for _most_ customers).
| paulpauper wrote:
| _Apple didn 't "win" the smartphone race, but they had a five
| year head start and ended up becoming the most valuable
| (publicly traded) company in the world._
|
| The iphone was in 2007. The Blackberry preceded it by many
| years. Same for the Palm Pilot.
| signatoremo wrote:
| This photo is a good if incomplete depiction of cellphones
| before and after iPhone [0]
|
| Many traditional phone makers ceased to exist in the iPhone
| era -- Blackberry, Nokia, Palm, Motorola, Ericsson, etc. Many
| new players took their places -- Google, Samsung, Xiaomi,
| HTC. All smartphones are similar today. In that context Apple
| is a pioneer, with a head start. Wasn't for long, mind you.
|
| [0]-https://www.cultofmac.com/wp-
| content/uploads/2012/02/beforea...
| augment001 wrote:
| I had both, and frankly it's like comparing a TRS-80 to a
| Mac.
| frellus wrote:
| Two things can be true at once:
|
| * being first doesn't guarantee success (otherwise we would all
| own Xerox laptops right now)
|
| * hard research and development is always a race in terms of
| getting to a product before running out of money
|
| Basic economic theory shows there is always a scarcity of
| resources - money, time, market, people. The faster you are in
| front of the pack, the more likely you will get the lions share
| of money, the more people buying your product, and the more
| people wanting to join your venture.
|
| The secret is to never fool yourself that you have unlimited time
| or money. That pathology leads companies to burn cash and take
| too much time choosing the "perfect" over the "really good",
| never getting to market. That's where I believe most startups
| failed in the past few years, and we're seeing that now more
| clearly with the desert of VC investment. You can fool yourself,
| but you cannot pretend basic economic forces don't exist.
| fvdessen wrote:
| You can think like that in the USA or China, but in the rest of
| the world its quite obvious we're not winning any races and we
| know the consequences of not winning them
| goatlover wrote:
| Yeah, but the context of the article is whether we should work
| on alignment because of the possible existential threat to
| humanity if a hard takeoff scenario is plausible. If it is, AI-
| alignment becomes paramount for everyone. If not, then winning
| the race matters more, for countries like the US and China.
| jongjong wrote:
| These arguments make sense.
|
| I think there isn't going to be a single dominant LLM. There will
| be multiple competing ones. It's difficult to see how companies
| will achieve lock in. Natural language is the most flexible
| interface. A front end app could easily switch from one LLM to a
| different LLM behind the scenes with very little effort. It's
| basically one API endpoint...
|
| How intelligent can the LLMs get and still be useful to humans?
| There is a point beyond which intelligence won't have any use for
| us. Humans may have already reached this point in some
| industries. Intelligence above a certain threshold doesn't have
| much value.
| skwirl wrote:
| There seems to be some unfortunate social herding around AI
| beliefs in the 'rationalist' community that seems very un-
| rationalism-like to me. Some of these arguments are really poor.
| iEchoic wrote:
| > But suppose AI is "only" a normal transformative technology, no
| more important than electricity, automobiles, or computers.
|
| Most people who think the AI "race" is important _aren 't_
| supposing this. So I don't think you can just "suppose" this away
| in the second sentence.
| somethoughts wrote:
| I'll throw out that two additional motivations that entities have
| to frame things as a race:
|
| - Delay thoughtful regulation of the technology - if you can
| delay it long enough until the entity gets big enough to fund
| lobbying efforts, its much easier to reach steady state/too big
| to fail status
|
| - Gain access to government funding - as a consultant, if there's
| no existential crisis of being forever stuck in second place (no
| one wants to be behind), its much harder to provide thought
| leadership seminars/get fully paid vacations (i.e. speaking
| events).
| oidar wrote:
| Also, if you goal is full employment, think of the GDP
| advantage of a country having access to these technologies.
| It's a huge workforce multiplier.
| vkou wrote:
| High GDP does not in itself result in high employment. It is
| simply a measure of wealth produced in the _measured_
| economy.
|
| If I owned a fully-robotic factory that produced all the
| goods this country consumed, unemployment would be at 100%,
| but GDP would be the same (Because the amount of wealth
| produced would be the same as it is today.)
|
| Capitalists aren't going to share their profit until we take
| it from them, either through taxes[1], or, ah, 'land
| reform'[2].
|
| [1] The carrot.
|
| [2] The stick.
| deltree7 wrote:
| Yes and No.
|
| Output always creates more work, due to second law of
| thermodynamics (entropy). So, the more output you have, the
| more work you need to fight entropy. May be the Robots can
| deal with entropy too, but we aren't there yet.
|
| So, AI will increase need of workforce / labor
| danaris wrote:
| > Output always creates more work, due to second law of
| thermodynamics (entropy).
|
| ...You're gonna have to explain that one, because I don't
| see how those things are connected at all.
| Imnimo wrote:
| >China is currently about 2 years behind the US in AI.
|
| How can this be true if AI is not a race? What does it even mean
| to be behind in a technology that isn't a race?
| vermilingua wrote:
| Traffic isn't a race either, but you can still be behind other
| cars.
| ilc wrote:
| To ignore what happens when if the "other side" wins a race, is
| ignorant.
|
| Imagine the Germans nuking London in 1941. Imagine the chilling
| effect this might have on the US public's willingness to enter
| WWII?
|
| To say something had no impact is very different than it could
| have a MAJOR impact. And nuclear weapons are clearly an example
| of this.
|
| AI is much the same, though I think 70% of the shift occurred
| with unmanned vehicles. The ability to send something with no
| person in it, to fight and kill a person is a HUGE game changer.
| When that 'something' gets better, smaller, and smarter... the
| edge becomes all the more decisive.
|
| Also the ability to take in much more data... Imagine having
| 100,000 sensors you want to understand reasonably well. You may
| not have enough trained humans. But an AI? Maybe... And maybe it
| can correlate the signals etc.
|
| AI is a game changer, and those saying it is as powerful as the
| bomb may not be underselling it. Mainly because of the precision
| and deniability the end results may offer.
| RcouF1uZ4gsC wrote:
| > You may have GPT-3, GPT-4, some future GPT-5,
|
| One thing that is a little different with these AIs is the
| ability to improve programming.
|
| I bet, GPT-4 can make it simpler for humans to build/deploy
| GPT-5. GPT-5, could do even more of the heavy lifting for GPT-6.
| boringuser2 wrote:
| The first country to develop AI will either permanently rule the
| world or be destroyed with the rest of us.
|
| This isn't rocket science.
| cactusplant7374 wrote:
| I think Elon is trying to delay AI because he is so far behind in
| his AI ambitions. Altman is getting so much press and Elon is
| noticeably jealous.
|
| The pause would give him time to spin up a startup and attempt to
| compete.
| signatoremo wrote:
| Not sure if his latest call is to buy more time, but to say
| that it is his only motive is just wrong. Musk has always
| voiced concern about AI, even if he pushes for it at the same
| time. This is from 2017 when OpenAI was still in his plan:
|
| https://www.cnbc.com/2017/09/04/elon-musk-says-global-race-f...
|
| > "Last month he co-signed a letter with other leaders in
| robotics including Google Deep Mind's Mustafa Suleyman, calling
| on the United Nations to ban killer robots"
| endisneigh wrote:
| hasn't he been saying these things even before GPT, though?
| superfrank wrote:
| Elon's been hesitant about AI for a while now, so I don't think
| it's only jealous, but I do agree that I think that's part of
| it.
| ltdanimal wrote:
| Except he isn't hesitant about the AI that is beneficial to
| him and his company. It doesn't make him wrong but it does
| make him a bit hypocritical.
| superfrank wrote:
| Totally agree that he's a hypocrite, but I've always
| assumed that "okay for me, but not for thee" mindset came
| mainly from ego.
|
| "If Tesla builds an AI, I'm smart enough that I can keep it
| in check. If someone else builds an AI their CEO is
| probably not smart enough to do the same thing."
|
| Elon has a long history of thinking he's smarter than
| everyone else and that even people who are experts in their
| field aren't as capable as he is (cave submarine).
| ftxbro wrote:
| "when the US invented nukes, its enemies were stuck with normal
| bombs; there is no slightly-worse-nuke that can only destroy half
| a city."
|
| I'm not an expert on what happened to Dresden but I heard it was
| ... a lot.
| reaperman wrote:
| Not to mention that
|
| > there is no slightly-worse-nuke that can only destroy half a
| city.
|
| Is itself false. The "Little Boy" that dropped on Hiroshima was
| the "slightly-worse-nuke" with a moderate blast damage radius
| (5 psi) of 1.67 km. The Dong Feng 5, China's largest nuclear
| warhead currently deployed, has a moderate blast damage radius
| (5 psi) of 24.5 km.
| danaris wrote:
| But that's still _a nuke_.
|
| You still need to do every bit as much science and
| engineering to make it; the only difference is you need
| slightly less fissile material.
| ianferrel wrote:
| It was possible to destroy a city pre-nukes with a dedicated
| bombing campaign, but it was not possible to destroy a city
| with a single plane/missile/bomb.
|
| Nuclear powers can credibly threaten to not just destroy a
| single city, but to wipe a modern nation off the map. That's
| something you can't do with conventional bombers, even a lot of
| them.
|
| I think his specific claim is not quite right here, but the
| thrust of his argument is. There's a reason that nuclear powers
| dare not enter direct conflict with each other in a way that
| major military powers used to casually do.
| yamtaddle wrote:
| > Nuclear powers can credibly threaten to not just destroy a
| single city, but to wipe a modern nation off the map. That's
| something you can't do with conventional bombers, even a lot
| of them.
|
| I think the key insight here is rate-of-production (measured
| in, say, tons of TNT equivalent, ready for delivery to a
| target), ease of stockpiling huge levels of destructive
| power, and cost of delivering that destructive power. The
| first two are incredibly higher for nukes vs. conventional
| weapons, and the last, far lower.
|
| 500 late-WWII Allied heavy bombers _couldn 't_ wipe out all
| the major cities in, say, the Soviet Union in a couple days.
| Impossible. The logistics simply don't work, there's no
| halfway-plausible scenario in which they could even get close
| to achieving that, there are _tons_ of obstacles standing in
| the way.
|
| .. with a few hundred nukes, though? Now it 's totally
| possible.
| solveit wrote:
| Dresden took literally hundreds of thousands of bombs...
| thebooktocome wrote:
| Astral Codex Ten (or Slate Star Codex, or,...) would be a lot
| more compelling if one didn't always find some obvious
| historical error like this in the typical article. If only he
| could afford a fact-checker.
| sebzim4500 wrote:
| This is not a historical error. It is objectively true that
| there is no conventional bomb which can destroy half a city.
| thebooktocome wrote:
| Insofar as it is objectively true, it is irrelevant to his
| point; insofar as it is relevant to his point, it is
| historically false.
|
| Dresden, Tokyo, Hanoi--how many more counterexamples do you
| need? The means to level a city with conventional bombs
| already existed. The nuclear bomb's innovation was as a
| weapon of intimidation, not raw destructive force.
| yamtaddle wrote:
| > Dresden, Tokyo, Hanoi--how many more counterexamples do
| you need? The means to level a city with conventional
| bombs already existed. The nuclear bomb's innovation was
| as a weapon of intimidation, not raw destructive force.
|
| Conventional strategic bombers and high explosives would
| never have given us MAD. The raw destructive force is
| absolutely relevant, it was (and remains) infeasible to
| do anything like wipe out a whole country in a day or two
| with conventional weapons, that'd take months or years
| and enormous dedication of resources. With nukes? Now
| it's possible.
|
| I'm really not getting the reading of this as "LOL we
| could blow up cities with regular bombs, he's wrong!"
| Like, WTF was all the worry over nukes in the cold war,
| then? They're _extremely obviously_ a major game-changer.
| A functioning nuke program means multiplying the max
| plausible destructive power a country can deploy inside a
| week by a very large number--no later developments in
| conventional weapons have really closed that gap, it 's a
| whole other _category_ of weapon, and you either have it,
| or you don 't.
| thebooktocome wrote:
| Nuclear warheads on ICBMs are wildly different from what
| the Allies had during WWII, and far beyond the scope of
| this debate. They also represent an incremental advance
| in the state of the art, and not the quantum leap Scott
| is failing to invoke here.
| pdonis wrote:
| He shouldn't need to hire a fact checker. He just needs to
| realize that his error rate is too high when he just tosses
| off these historical claims based on his own background
| knowledge, and _stop doing that_. If the claim is really
| central to the argument, then take the time to check it. If
| it 's not that important, then just leave it out.
|
| Unfortunately, public intellectuals in our current society
| don't hold themselves to these kinds of standards (which
| would have been considered obvious for public intellectuals a
| couple of centuries ago), and no one else holds them to it
| either.
| thebooktocome wrote:
| He's not a public intellectual and he doesn't want to be.
| But people treat him like one regardless, because he
| happened to attain a little clout as a "Rationalist".
| pdonis wrote:
| He doesn't get to decide whether he's a public
| intellectual. If you publish your thoughts and ideas
| about issues that are of interest to the public, you're a
| public intellectual, whether you like it or not. Even
| more so if many people read what you publish, discuss it,
| and give it weight.
| solveit wrote:
| > Unfortunately, public intellectuals in our current
| society don't hold themselves to these kinds of standards
| (which would have been considered obvious for public
| intellectuals a couple of centuries ago)
|
| Have you read anything a couple of centuries old? The
| problem was, if anything, much worse!
| pdonis wrote:
| To emphasize the main substantive point, the US having access
| to nuclear weapons while the other side did not was _not_ a
| factor in who won WW II. By the time the US had nukes, Germany
| had already surrendered and Japan was beaten. (And Japan knew
| it; they had been making peace feelers through various avenues
| for months.) WW II was won by conventional weapons.
|
| In other words, the basic historical claim being made here,
| that the US's "brief period of dominance" in nukes won WW II,
| is false. Not only that, it's "schoolboy error" false: the sort
| of mistake nobody who claims any sort of knowledge about the
| subject they're writing about should make.
| anon84873628 wrote:
| He's not saying there weren't other technologies equally
| destructive as nukes at the time. Just that in the world of
| nuclear bombs there is no half version.
|
| Seems like other replies are trying to harp on this to
| undermine the entire article, when they are just deliberately
| misinterpreting the point. Maybe no analogy is perfect?
| yamtaddle wrote:
| The paragraph _is_ a bit muddled. I took it as pointing to
| American dominance at the end of the war, preventing Soviet
| over-reach or immediate direct confrontation with the Allies.
| After all, "Americas enemies" in the sense of Germany,
| Japan, and Italy _never_ got The Bomb--they still don 't have
| it, even if they could all likely get there pretty fast if
| they wanted to, these days--in the sense of building their
| own nuclear weapons, so he can hardly mean them as the
| enemies who got the bomb "soon after"--that's the Soviets,
| our wartime frenemy, and I expect nukes were a significant
| factor in shaping the wrap-up of that war, even if they
| played only a minor role in bringing the actual victory over
| the axis powers.
|
| But I do think the broader point of nukes being a "you have
| it, or you don't and can't even really approach the
| capability of those who do" sort of tech is accurate.
| yamtaddle wrote:
| There is a bit of a difference between needing to commit dozens
| or hundreds of expensive bombers to kill a city, and only
| needing to commit one. You go from being able to knock out a
| city in a day (... and with, probably, _weeks_ worth of
| munitions production) with an air wing, to being able to knock
| out _all_ the cities in a country in a day with an air wing.
| With enough nukes and a modest number of bombers at one 's
| disposal, all but overwhelmingly-strong air defenses become
| incapable of protecting a country from destruction--enough will
| get through to cause catastrophic damage (talking 40s and 50s
| tech here--as air defense tech got better, ballistic and cruise
| missiles overtook bombers in this kind of scenario).
|
| Concentrating the damage like that gives the enemy far less
| time to adapt or rebuild, and less capacity to do so--you can
| out-produce heavy damage that's spread out over a long period
| of time, you can adapt to it, you can shift tactics and
| resources around to reduce the rate of damage in the future,
| but such efforts are hopeless against nukes.
|
| It wasn't especially in-play during WWII since even the US
| couldn't (yet) build the bombs fast enough for them to really
| matter that much versus what could be done with conventional
| weapons, but it did matter soon after as stockpiles began to
| grow and production rate increased. Nukes represented a huge
| multiplier in the amount of destructive power a state could
| realistically stockpile and deploy in short order.
| travisjungroth wrote:
| I was going to quote that exact sentence. A night of bombing
| Tokyo killed approximately as many people as the nuclear bomb
| on Hiroshima. https://en.m.wikipedia.org/wiki/Bombing_of_Tokyo
| ToValueFunfetti wrote:
| I feel like the fact that that was 334 planes dropping 1665
| tons of mostly 500lb bombs over the course of a night is an
| important distinction from a single bomb that does the same
| damage.
| bluGill wrote:
| Not really as even with WWII tech the bombs could target
| specific areas. A nuclear bomb is more destructive for a
| few blocks around the blast, but get a couple miles out in
| the city and the damage is much less than a smaller bomb
| hit.
|
| Modern military doctrine is hitting a whole city is bad.
| You want to hit specific targets and not collateral damage
| to other things. Discredited WWII doctrine was killing
| civilians would make them surrender so hitting the whole
| city was good. Which lens you look at military doctrine
| says which is better.
| travisjungroth wrote:
| Obviously there are huge differences and it ushered in a
| new era of weaponry. What I want to point out is that
| atomic weapons were not an immediate step-change in
| destructive power. They're actually _not_ an exception to
| the greater point the author was making.
|
| The destruction caused by a single military operation was
| in the same league. The logistics were actually harder in
| some ways than traditional bombing. It would be months
| before the US would have more atomic weapons after the
| first two. So the one bomb versus hundreds doesn't make a
| big difference when you have zero.
|
| This would all change dramatically over the coming years.
| But _at the time of its first use_ , the atomic bomb was
| not as much of a leap forward in capability as you'd
| probably think.
| walkhour wrote:
| > If you don't believe in crazy science fiction scenarios like
| these, fine. But then why are you so sure that it's crucial to
| "win" the AI "race"?
|
| The whole point of the article is that either AI is not that big
| of a deal and like many other technologies (e.g. electricity,
| cars, ... ), or it's so revolutionary that's it's so hard to
| align that we're doomed.
|
| Well, can't it be something else? Can't it be something very safe
| which gives a country a 200 year head start?
| anon84873628 wrote:
| Sure. We just don't have examples in history where the head
| start is as big as 200 years.
|
| Well, there is a difference between the "developed" and
| "developing" world, but that doesn't seem to be due to access
| to a single specific technology...
| walkhour wrote:
| We also don't have examples of one of the two contemplated
| options, we haven't seen a technology that has destroyed the
| world either.
| crazygringo wrote:
| > _But suppose AI is "only" a normal transformative technology,
| no more important than electricity, automobiles, or computers._
|
| That's the weak point of this article.
|
| Because the article then goes on to admit:
|
| > _The most consequential "races" have been for specific military
| technologies during wars; most famously, the US won the "race"
| for nuclear weapons. America's enemies got nukes soon afterwards,
| but the brief moment of dominance was enough to win World War
| II._
|
| And basically the fear is that AI is going to be _exactly_ like
| nuclear weapons in terms of delivering an overwhelmingly decisive
| advantage, and that this could _very_ much be concrete reality
| specifically in a war where China invades Taiwan.
|
| In everything from integrating millions of information sources to
| determine best strategy and tactics, to controlling swarms of
| smart drones and bombs and missiles.
|
| If there were ever a moment to compare something to a
| technological race like the nuclear bomb, I'd say AI fits it to a
| tee.
| bryanlarsen wrote:
| The article has both "suppose it is a normal transformative
| technology" and "suppose it's not".
|
| It's a decision tree supporting the argument for a pause on AI
| research.
|
| If it's a normal transformative technology, then a pause won't
| hurt because it's not a race.
|
| If it is not, then if it is non-aligned it has huge potential
| for catastrophe. Therefore a pause is essential to ensure that
| we end up with an aligned AI rather than a non-aligned one.
|
| Choice A: pause won't hurt. Choice B: pause prevents
| destruction of the world.
| crazygringo wrote:
| But it's not just these two options.
|
| I'll take Choice C: alignment is a largely hypothetical
| problem that probably won't be a major issue at all, while an
| authoritarian China gaining an insurmountable military
| advantage would be a here-and-now disaster for those of us
| who value democracy and self-determination globally.
|
| So I'm more worried about a worryingly realistic problem
| (China gaining military dominance over the West due to
| winning an AI race) over a totally hypothetical idea that
| we're going to reach some kind of singularity where non-
| aligned AI destroys humanity.
|
| If we start getting to a point where a non-aligned AI
| singularity seems to be a year or two away (or even a couple
| of months) then yes, shut it all down -- the same way you'd
| shut down a nuke that turned out to be able to obliterate the
| entire planet. But right now the singularity is entirely in
| the realm of science fiction and futurist philosophers.
|
| So right now, yes AI does look like a race, and it's
| important that democracy win that race rather than
| authoritarianism.
| om2 wrote:
| So you think AI will both be overwhelmingly powerful, but
| also very easy to control reliably? It seems the scenario
| where C applies implies both of those things, because China
| only gets an insurmountable advantage if they can build
| powerful AI but also remain in control. Even today's state
| of the art chatbots seem pretty hard to control, and they
| are much less powerful than true AGI so I'm kind of
| skeptical.
| crazygringo wrote:
| Personally, yes that's what I'm expecting.
|
| I don't expect AGI at all. I expect very powerful
| _domain-specific AI_ (e.g. integrating information,
| battlespace simulation, weapons coordination) that is far
| more effective than human coordination and communication
| can be, but I expect it to still just be sticking to
| pattern recognition, simulating scenarios, and picking
| the most likely scenarios based on relatively simple
| reward functions, and with constant human oversight (in
| the same way managers oversee their employees).
|
| Today's chatbots are controlled perfectly in the sense
| that they're performing exactly how they were designed
| to. They're predicting a plausible next word, that's it.
|
| If we start seeing rapidly progressing AGI then it's an
| entirely different conversation. But there's zero
| evidence that's coming even in a couple of decades. It's
| 100% hypothetical right now.
| inglor_cz wrote:
| War in Europe was won before nukes were deployed. War in the
| Pacific would have been won by conventional means too - it
| would have taken longer, with more casualties, but there was no
| chance for Japan to fight the US _and_ the USSR, which declared
| war on Japan in August 1945, to a stalemate.
|
| What would have looked different would be the political
| outcome. If USSR moved its armies to the East to fight Japan,
| well, entire Korea and half of Japan could have become
| Communist. On the other hand, with the Soviet Army engaged in
| the Far East, countries like Poland, Hungary and Czechoslovakia
| could have plausibly wrestled themselves free from the Soviet
| grip.
| majormajor wrote:
| I think something like "what if Germany had gotten nukes
| before the end of the war in Europe, and before the US" is a
| more interesting counterfactual than the US not getting nukes
| before the end of the Pacific war. Or Russia or another
| European country, to a lesser extent.
| Yoric wrote:
| It is true that both Germany and the US considered it a
| race.
|
| However, as it turns out, by the end of the war, Germany
| was actually pretty far from even seeing the finish line.
| For some reason, murdering or driving away many of your
| brightest minds because they happen to come from the wrong
| branch of the religious tree/diverge from the mainstream
| sexual orientation/don't adhere to the political cult in
| power doesn't help come up with breakthroughs in
| fundamental sciences.
| inawarminister wrote:
| Yep, on the other hand, the US rival right now while a
| bit tyrannical, is actually in the similar tier to
| American academia; maybe a few months or even weeks
| behind.
|
| And lots of people in the unaligned world is ready to
| switch from San Fransisco to Beijing if the latter
| provides cheap LLM for commercial purposes.
| jandrese wrote:
| I think the "Japan surrendered because they were afraid of
| Russia" is revisionist history. Japan knew that Russia had no
| effective way to project power onto the Japanese home
| islands, the Russo-Japanese war had proven that and they were
| clearly exhausted from fighting the Nazis.
|
| Russia saw the writing on the wall and wanted to claim a
| piece of the pie before Japan's inevitable surrender. The
| only way it hastened the end of the war was in convincing the
| Japanese leadership to hurry up and sign the papers before
| any other countries declared war and made claims.
| boringuser2 wrote:
| AI is more decisive than nuclear bombs, it's more akin to
| inventing a nuclear bomb for literally everything rather than
| just killing lots of people.
___________________________________________________________________
(page generated 2023-04-05 23:02 UTC)