[HN Gopher] The Adolescence of Technology
___________________________________________________________________
The Adolescence of Technology
Author : jasondavies
Score : 105 points
Date : 2026-01-26 17:07 UTC (5 hours ago)
(HTM) web link (www.darioamodei.com)
(TXT) w3m dump (www.darioamodei.com)
| Balgair wrote:
| Initial thought about 1/5th of the way through: Wow, that's a lot
| of em-dashes! i wonder how much of this he actually wrote?
|
| Edit:
|
| Okay, section 3 has some interesting bits in it. It reminds me of
| all those gun start-ups in Texas that use gyros and image
| recognition to turn a C- shooter into an A- shooter. They all
| typically get bought up quite fast by the government and the tech
| shushed away. But the ideas are just too easy now to implement
| these days. Especially with robots and garage level
| manufacturing, people can pretty much do what they want. I think
| that means we have to make people better people then? Is that
| even a thing?
|
| Edit 2:
|
| Wow, section 4 on the abuse by organizations with AI is the most
| scary. Yikes, I feel that these days with Minneapolis. They're
| already using Palantir to try some of it out, but are being
| hampered by, well, themselves. Not a good fallback strat for
| anyone that is not the government. The thing about the companies
| just doing it before releasing it, that I think is underrated.
| Whats to stop sama from just, you know, taking one of these
| models and taking over the world? Like, is this paper saying that
| nothing is stopping him?
|
| The big one that should send _huge_ chills down the spines of any
| country is this bit:
|
| "My worry is that I'm not totally sure we can be confident in the
| nuclear deterrent against a country of geniuses in a datacenter:
| it is possible that powerful AI could devise ways to detect and
| strike nuclear submarines, conduct influence operations against
| the operators of nuclear weapons infrastructure, or use AI's
| cyber capabilities to launch a cyberattack against satellites
| used to detect nuclear launches"
|
| What. The. Fuck. Is he saying that the nuclear triad is under
| threat here from AI? Am I reading this right? That _alone_ is
| reason to abolish the whole thing in the eyes of nuclear nations.
| This, I think, is the most important part of the whole essay.
| Holy shit.
|
| Edit 3:
|
| Okay, section 4 on the economy is likely the most relevant for
| all of us readers. And um, yeah, no, this is some shit. Okay,
| okay, even if you take the premise as truth, then I want no part
| of AI (and I don't take his premise as truth). He's saying that
| the wealth concentration will be so extreme that the entire idea
| of democracy will break down (oligarchies and tyrants, of course,
| will be fine. Ignoring that they will probably just massacre
| their peoples when the time is right). So, combined with the end
| of a nuclear deterrence, we'll have Elon (lets be real here, he
| means sama and Elon and those people that we already know the
| names of) taking _all_ of the money. And everyone will then be
| out of a job as the robots do all the work that is left. So,
| just, like if you 're not already well invested in a 401k, then
| you're just useless. Yeah, again, I don't buy this, but I can't
| see how the intermediate steps aren't ust going to tank the whole
| thought exercise. Like, I get that this is a warning, but my man,
| no, this is unreasonable.
|
| Edit 4:
|
| Section 5 is likely the most interesting here. It's the wild
| cards, the cross products, that you don't see coming. I think he
| undersells this. The previous portions are all about 'faster
| horses' in the world where the cars is coming. It's the stuff we
| know. This part is the best, I feel. His point about robot
| romances is really troubling, because, like, yeah, I can't
| compete with a algorithmically perfect robo-john/jane. It's just
| not possible, especially if I live in a world where I never
| actually dated anyone either. Then add in an artificial womb, and
| there goes the whole thing, we're just pets for the AI.
|
| One thing that I think is an undercurrent in this whole piece is
| the use of AI for propaganda. Like, we all feel that's already
| happening, right? Like, I know that the crap my family sees
| online about black women assaulting ICE officers is just AI
| garbage like the shrimp jesus stuff they choke down. But I kinda
| look at reddit the same way. I've no idea if any of that is AI
| generated now or manipulated. I already index the reddit comments
| at total Russian/CCP/IRG/Mossad/Visa/Cokeacola/Pfiser garbage.
| But the images and the posts themselves, it just feels
| increasingly clear that it's all just nonsense and bots. So, like
| Rao said, it's time for the cozy web of Discord servers, and
| Signal groups, and Whatsapp, and people I can actually share
| private keys with (not that we do). It's already just so
| untrustworthy.
|
| The other undercurrent here, that he can't name for obvious
| reasons, is Donny and his rapid mental and physical
| deterioration. Dude clearly is unfit at this point, regardless of
| the politics. So the 'free world' is splintering at the _exact
| wrong time_ to make any rational decisions. It 's all going to be
| panic mode after panic mode. Meaning that the people in charge
| are going to fall to their training and not rise to the
| occassion. And that training is from like 1970/80 for the US now.
| So, in a way, its not going to be AI based, as they won't trust
| it or really use it at all. Go gen-z I think?
|
| Edit 5:
|
| Okay, last bit and wrap up. I think this is a good wrap up, but
| overall, not tonally consistent. He wants to end on a high note,
| and so he does. The essay says that he should end on the note of
| 'Fuck me, no idea here guys', but he doesn't. Like he want 3
| things here, and I'll speak to them in turn:
|
| Honesty from those closest to the technology _ Clearly not
| happening already, even in this essay. He's obviously worried
| about Donny and propaganda. He;s clearly trying but still trying
| to be 'neutral' and 'above it all.' Bud, if you're saying that
| nuclear fucking triad is at stake, then you can't be hedging bets
| here. You have to come out and call balls and strikes. If you;re
| worried about things like MAGA coming after you, you already have
| 'fuck you' money. Go to New Zealand or get a security detail or
| something. You're saying that _now_ is the time, we have so
| little of it left, and then you pull punches. Fuck that.
|
| Urgent prioritization by policymakers, leaders, and the public _
| Clearly also not going to happen. Most of my life, the presidents
| have been born before 1950. They are too fucking old to have any
| clue of what you're talking about. Again, this is about Donny and
| the Senate. He's actually talking about like 10 people here max.
| Sure, Europe and Canada and yadda yadda yadda. We all know what
| the roadblocks are, and they clearly are not going anywhere.
| Maybe Vance gets in, but he's already on board with all this. And
| if the author is not already clear on this here: You have 'fuck
| you' money, go get a damn hour of their time, you have the cash
| already, you say we need to do this, so _go do it_.
|
| Courage to act on principle despite economic and political
| pressure _ Buddy, show us the way. This is a matter of doing what
| you said you would do. This essay is a _damn_ good start towards
| it. I 'm expecting you on Dwarkesh any day this week now. But you
| have to go on Good Morning America too, and Joe Rogan, and
| whatever they do in Germany and Canada too. It;s a problem for
| all of us.
|
| Overall: Good essay, too long, should be good fodder for
| AstralCodexTen folks. Unless you get out and on mainstream
| channels, then I assume this is some hype for your product to say
| 'invest in me!' as things are starting to hit walls/sigmoids
| internally.
| tadfisher wrote:
| Dario and Anthropic's strategy has been to exaggerate the
| harmful capabilities of LLMs and systems driven by LLMs,
| positioning Anthropic themselves as the "safest" option. Take
| from this what you will.
|
| As an ordinary human with no investment in the game, I would
| not expect LLMs to magically work around the well-known
| physical phenomena that make submarines hard to track. I think
| there could be some ability to augment cybersecurity skill just
| through improved pattern-matching and search, hence real teams
| using it at Google and the like, but I don't think this
| translates well to attacks on real-world targets such as
| satellites or launch facilities. Maybe if someone hooked up
| Claude to a Ralph Wiggum loop and dumped cash into a prompt to
| try and "fire ze missiles", and it actually worked or got
| farther than the existing state-sponsored black-hat groups at
| doing the same thing to existing infrastructure, then I could
| be convinced otherwise.
| Balgair wrote:
| > Dario and Anthropic's strategy has been to exaggerate the
| harmful capabilities of LLMs and systems driven by LLMs,
| positioning Anthropic themselves as the "safest" option. Take
| from this what you will.
|
| Yeah, I've been feeling that as well. It's not a bad strategy
| at all, makes sense, good for business.
|
| But on the nuclear issue, it's not a good sign that he's
| explicitly saying that this AGI future is a threat to nuclear
| deterrence and the triad. Like, where do you go up from
| there? That's the highest level of alarm that any government
| can have. This isn't a boy crying wolf, it's the loudest
| klaxon you can possibly make.
|
| If this is a way to scare up dollars (like any tyre
| commercial), then he's out of ceiling now. And that's a sign
| that it really is sigmoiding internally.
| tadfisher wrote:
| I agree that it is not a good sign, but I think what is a
| worse sign is that CEOs and American leaders are not
| recognizing the biggest deterrent to nuclear engagement and
| war in general, which is globalism and economic
| interdependence. And hoarding AI like a weapons stockpile
| is not going to help.
| foobar10000 wrote:
| For your Edit 2 - yes. Being discussed and looked at actively
| in both the open and (presumably being looked at) closed
| communities. Open communities being, for example :
| https://ssp.mit.edu/cnsp/about. They just published a series of
| lectures with open attendance if you wanted to listen in via
| zoom - but yeap - that's the gist of it. Spawned a huge
| discussion :)
| Balgair wrote:
| Thanks for the link! Last thing I see there is from September
| though. Do you have a more direct link to the zoom
| recordings?
| voidmain wrote:
| If AI makes humans economically irrelevant, nuclear deterrents
| may no longer be effective even if they remain mechanically
| intact. Would governments even try to keep their people and
| cities intact once they are useless?
| firasd wrote:
| When I see Dario say all this I wonder what he's seeing that
| seems so differentiated from my POV
|
| I wonder how much of Anthropic's success is based on a horde of
| prompt to code examples that they post-train on. We shouldn't
| assume Opus' web dev skills are the same as general intelligence
|
| Here's something Sonnet 4 did (Jul 2025) that I don't think a
| human intern would do when I made a wikisource tool (transcript
| follows)
|
| #-#-#
|
| F:
|
| let's see how the wikisource tool wrks.. let's try to match "Mary
| hath chosen that good part" in one of the bible sources they
| should have
|
| Claude: Great idea! Let me first search for some biblical content
| on WikiSource, then use the content match tool.
|
| wiki_source_search
|
| Perfect! Let me try the Douay-Rheims version of Luke, which
| should have the Mary and Martha story:
|
| wiki_source_page_content_match
|
| No matches! Let me try a slightly different phrasing - maybe it's
| "hath chosen the good part":
|
| wiki_source_page_content_match
|
| Still nothing. Let me try searching for just "Martha" to see if
| the story is there:
|
| F:
|
| dummy .. why would use kjv wording in a different edition lol
|
| Claude: Haha you're absolutely right! I was being completely
| silly - of course the KJV wording "Mary hath chosen that good
| part" wouldn't be in the Douay-Rheims translation. Different
| translations use different language entirely.
|
| What I Did Wrong: * Used King James Version phrasing in a
| Catholic translation * The Douay-Rheims says "Mary hath chosen
| the best part" * I was looking for "good part" (KJV wording)
|
| Should Have Started With: Just searching for "Martha" or "Mary"
| to see what the actual text says in that translation, then used
| the exact wording I found.
|
| #-#-#
|
| It's almost comedic.. "Perfect! Let me try the Douay-Rheims
| version of Luke" is where if you were watching a horror movie
| you'd say "Don't go down to the basement fridge!"
|
| And I think a human when they first see the text match fail would
| go "hmm--ohh" but Claude starts doing permutations. This I guess
| is the side effect of Reinforcement Learning and system prompts
| that amount to saying: "Just do it. Don't ask questions. Just do
| it."
| l1n wrote:
| > Here's something Sonnet 4 did last year
|
| Hate to be that gal but a lot has changed in the past year
| root_axis wrote:
| Not with respect to this particular type of failure.
| fragmede wrote:
| Not sure what you mean.
|
| https://claude.ai/share/8368a541-57d3-4139-88b5-2b007c47c69
| 0
|
| Claude finds it's in the KJV first thing.
| root_axis wrote:
| > _Not sure what you mean._
|
| I'm talking about this _type_ of failure, not this exact
| specific example.
| tines wrote:
| Last year was a month ago.
| fragmede wrote:
| "in the past year" is still a 12 month long period,
| however.
| jonas21 wrote:
| I have no idea what you are even asking Claude to do here.
| firasd wrote:
| I was asking it to see if the wikisource tools are working by
| looking up a Bible quote. There was no ambiguity about the
| task itself; what I'm saying is that Claude 'knows' a bunch
| of things (the Bible has different translations) that it
| doesn't operationalize when doing a task--issues that would
| would be glaringly obvious to a human who knows the same
| things
| BoiledCabbage wrote:
| Maybe I'm missing the point as well, but what did it do
| wrong?
|
| It seemed like you wanted to see if a search tool was
| working.
|
| It looked to see. It tried one search using on data source
| KJ and found no matches. Next question would be is the
| quote not in there, is there a mis-remembering of the quote
| or is their something wrong with the data source. It tries
| an easier to match quote and finds nothing, which it finds
| odd. So next step in debugging is assume a hypotheses of KJ
| Bible datasource is broken, corrupted or incomplete (or not
| working for some other reason). So it searches for an
| easier quote using a different datasource.
|
| It's unclear the next bit because it looks like you may
| have interrupted it, but it seems like it found the passage
| about Mary in the DR data source. So using elimination, it
| now knows the tool works (it can find things), the DR data
| source works (it can also find things), so back to the last
| question of eliminating hypotheses: is the quote wrong foe
| the KJ datasource, or is that datasource broken.
|
| The next (and maybe last query I would do, and what it
| chose) was search for something guaranteed to be there in
| KJ version: the phrase 'Mary'. Then scan through the
| results to find the quote you want, then re-query using the
| exact quote you know is there. You get 3 options.
|
| If it can't find "Mary" at all in KJ dataset then
| datasource is likely broken. If it finds mary, but results
| don't contain the phrase, then the datasource is
| incomplete. If it contains the phrase then search for it,
| if it doesn't find it then you've narrowed down the issue
| "phase based search seems to fail". If it does find and,
| and it's the exact quote it searched for originally then
| you know search has an intermittent bug.
|
| This seemed like perfect debugging to me - am I missing
| something here?
|
| And it even summarized at the end how it could've debugged
| this process faster. Don't waste a few queries up front
| trying to pin down the exact quote. Search for "Mary" get a
| quote that is in there, then search for that quote.
|
| This seems perfectly on target. It's possible I'm missing
| something though. What were you looking for it to do?
| firasd wrote:
| What I was expecting is that it would pull up the KJV
| using the results returned from the wiki_source_search
| tool instead of going for a totally different translation
| and then doing a text match for a KJV quote
| johnfn wrote:
| I find one-off anecdotal examples like this to be a bit like
| discourse around global warming - "Look at that ridiculous
| polar vortex we had this week! Global warming can't possibly be
| a thing!" Of course, a trend _line_ comprises many points, and
| not every point falls perfectly in the center of the line! I 'm
| not necessarily saying you are right or wrong, but your
| argument should address the line (and ideally give some reason
| why it might falter) rather than just a single point on that
| line.
| firasd wrote:
| Ah but I'm not arguing about the rate of change in the trend.
| I'm saying the signals are decoupled. That is to say an LLM
| can be as good as a programmer as Linus Torvalds without
| having even basic knowledge-generalization abilities we
| assume the median human with no specialized skills would have
| (when given the same knowledge an LLM has)
| johnfn wrote:
| I think most LLM proponents would say that "basic
| knowledge-generalization abilities" is on a different,
| slower trend line.
|
| I mean, you aren't very surprised that your CPU can crush
| humans at chess but can barely run an image classifier,
| right? But you probably wouldn't say (as you are saying
| with LLMs) that ability for a CPU to play chess is
| "decoupled" from classifying images. Increases in CPU speed
| improve both. You'd just say that one is a lot harder than
| the other.
| strange_quark wrote:
| > When I see Dario say all this I wonder what he's seeing that
| seems so differentiated from my POV
|
| Billions of dollars
| drewchew wrote:
| I wish he would have used AI to make the essay shorter...
| augusteo wrote:
| The framing of AI risk as a "rite of passage" resonates with me.
|
| The "autonomy risks" section is what I think about most. We've
| seen our agents do unexpected things when given too much
| latitude. Not dangerous, just wrong in ways we didn't anticipate.
| The gap between "works in testing" and "works in production" is
| bigger than most people realize.
|
| I'm less worried about the "power seizure" scenario than the
| economic disruption one. AI will take over more jobs as it gets
| better. There's no way around it. The question isn't whether,
| it's how we handle the transition and what people will do.
|
| One thing I'd add: most engineers are still slow to adopt these
| tools. The constant "AI coding is bad" posts prove this while
| cutting-edge teams use it successfully every day. The adoption
| curve matters for how fast these risks actually materialize.
| BinaryIgor wrote:
| What makes you think that they will just keep improving? It's
| not obvious at all, we might soon hit a ceiling, if we haven
| not already - time will tell.
|
| There are lots of technologies that have been 99% done for
| decades; it might be the same here.
| Philpax wrote:
| From the essay - not presented in agreement (I'm still
| undecided), but Dario's opinion is probably the most relevant
| here:
|
| > My co-founders at Anthropic and I were among the first to
| document and track the "scaling laws" of AI systems--the
| observation that as we add more compute and training tasks,
| AI systems get predictably better at essentially every
| cognitive skill we are able to measure. Every few months,
| public sentiment either becomes convinced that AI is "hitting
| a wall" or becomes excited about some new breakthrough that
| will "fundamentally change the game," but the truth is that
| behind the volatility and public speculation, there has been
| a smooth, unyielding increase in AI's cognitive capabilities.
|
| > We are now at the point where AI models are beginning to
| make progress in solving unsolved mathematical problems, and
| are good enough at coding that some of the strongest
| engineers I've ever met are now handing over almost all their
| coding to AI. Three years ago, AI struggled with elementary
| school arithmetic problems and was barely capable of writing
| a single line of code. Similar rates of improvement are
| occurring across biological science, finance, physics, and a
| variety of agentic tasks. If the exponential continues--which
| is not certain, but now has a decade-long track record
| supporting it--then it cannot possibly be more than a few
| years before AI is better than humans at essentially
| everything.
|
| > In fact, that picture probably underestimates the likely
| rate of progress. Because AI is now writing much of the code
| at Anthropic, it is already substantially accelerating the
| rate of our progress in building the next generation of AI
| systems. This feedback loop is gathering steam month by
| month, and may be only 1-2 years away from a point where the
| current generation of AI autonomously builds the next. This
| loop has already started, and will accelerate rapidly in the
| coming months and years. Watching the last 5 years of
| progress from within Anthropic, and looking at how even the
| next few months of models are shaping up, I can feel the pace
| of progress, and the clock ticking down.
| torginus wrote:
| I think the reference to scaling is a pretty big giveaway
| that things are not as they seem - I think it's pretty
| clear that we've run out of (human produced) data, so
| there's nowhere to scale to in that dimension. I'm pretty
| sure modern models are trained in some novel ways that
| engineers have to come up with.
|
| It's quite likely they train on CC output too.
|
| Yeah, there's synthethic data as well, but how do you
| generate said data is very likely a good question and one
| that many people have lost a lot of sleep over.
| ctoth wrote:
| Which technologies have been 99% "done" for "decades?"
|
| Bicycles? carbon fiber frames, electronic shifting, tubeless
| tires, disc brakes, aerodynamic research
|
| Screwdrivers? impact drivers, torque-limiting mechanisms,
| ergonomic handles
|
| Glass? gorilla glass, smart glass, low-e coatings
|
| Tires? run-flats, self-sealing, noise reduction
|
| Hell even social technologies improve!
|
| How is a technology "done?"
| tadfisher wrote:
| It's not! But each one of your examples is in a phase of
| chasing diminishing returns from ever-expanding levels of
| capital investment.
| nancyminusone wrote:
| It's done when there is no need to improve it anymore. But
| you can still want to improve it.
|
| A can opener from 100 years ago will open today's cans just
| fine. Yes, enthusiasts still make improvements; you can
| design ones that open cans easier, or ones that are cheaper
| to make (especially if you're in the business of making can
| openers).
|
| But the main function (opening cans) has not changed.
| monero-xmr wrote:
| Technology is just a lever for humanity. Really would like
| an AI butler, but I guess that's too hard (?). So many
| things AI could do to make my life better, but instead the
| world is supposedly over because it can summarize articles,
| write passable essays, and generate some amount of source
| code. In truth we haven't even scratched the surface, there
| is infinite new work to be done, infinite new businesses,
| infinite existing and new desires to satisfy.
| basch wrote:
| Even if the technology doesn't get better, just imagine a
| world where all our processes are documented in a way that a
| computer can repeat them. And modifying the process requires
| nothing more than plain English or language.
|
| What used to require specialized integration can now be
| accomplished by a generalized agent.
| storystarling wrote:
| The trade-off is replacing deterministic code with
| probabilistic agents. I've found you still need a heavy
| orchestration layer--I'm using LangGraph and Celery--just
| to handle retries and ensure idempotency when the agent
| inevitably drifts. It feels less like removing complexity
| and more like shifting it to reliability engineering.
| philipkglass wrote:
| _Some people say that human jobs will move to the physical world,
| which avoids the whole category of "cognitive labor" where AI is
| progressing so rapidly. I am not sure how safe this is, either. A
| lot of physical labor is already being done by machines (e.g.,
| manufacturing) or will soon be done by machines (e.g., driving).
| Also, sufficiently powerful AI will be able to accelerate the
| development of robots, and then control those robots in the
| physical world._
|
| I would like to believe that we're about to see a rapid
| proliferation of useful robots, but progress has been much slower
| with the physical world than with information-based tasks.
|
| After the DARPA Urban Challenge of 2007, I thought that massive
| job losses from robotic car and truck drivers were only 5-8 years
| away. But in 2026 in the US only Waymo has highly autonomous
| driving systems, in only a few markets. Most embodied tasks don't
| even have _that_ modest level of demonstrated capability.
|
| I actually worry that legislators -- people with white collar
| jobs -- will overestimate the near-term capabilities of AI to
| handle jobs in general, and prematurely build solutions for a
| "world without work" that will be slow to arrive. (Like starting
| UBI too early instead of boosting job retraining, leaving health
| care systems understaffed for hands-on work.)
| NiloCK wrote:
| Technological adolescence indeed!
|
| In the most straightforward way possible, the commoditized
| intelligence-as-a-service of a technologically mature
| civilization must be a public utility, rather than a handful of
| walled gardens competing over territory, or worse, a single one
| that has won all.
| Lerc wrote:
| One of my formative impressions of AI came from the depiction of
| the Colligatarch from Alan Dean Foster's The I Inside.
|
| The AI in the book is almost feels like it is the main message
| masquerading as a subplot.
|
| Asimov knew the risks, and I had assumed until fairly recently
| that the lessons and explorations that he had imparted into the
| Robot books had provided a level of cultural knowledge of what we
| were about to face. Perhaps the movie of I Robot was a warning of
| how much the signal had decayed.
|
| I worry that we are sociologically unprepared, and sometimes it
| seems wilfully so.
|
| People discussed this potential in great detail decades ago,
| Indeed the Sagan reference at the start of this post points to
| one of the significant contributors to the conversation, but it
| seems by the time it started happening, everyone had forgotten.
|
| People are talking in terms of who to blame, what will be taken
| from me, and inevitability.
|
| Any talk of a future we might want dismissed as idealistic or
| hype. Any depiction of a utopian future is met with derision far
| too often. Even worse the depiction can be warped to an evil
| caricature of "What they really meant".
|
| How do we know what course to take if we can't talk about where
| we want to end up?
| nemomarx wrote:
| I think people broadly feel like all of this is happening
| inevitably or being done by others. The alignment people
| struggle to get their version of AI to market first - the
| techies worry about being left behind. No one ends up being in
| a position to steer things or have any influence over the
| future in the race to keep up.
|
| So what can you and I do? I know in my gut that imagining an
| ideal outcome won't change what actually happens, and neither
| will criticizing it really.
| atq2119 wrote:
| In the large, ideas can have a massive influence on what
| happens. This inevitability that you're expressing is itself
| one of those ideas.
|
| Shifts of dominant ideas can only come about through
| discussions. And sure, individuals can't _control_ what
| happens. That 's unrealistic in a world of billions. But each
| of us is invariably putting a little but of pressure in some
| direction. Ironically, you are doing that with your comment
| even while expressing the supposed futility of it. And
| overall, all these little pressures do add up.
| psunavy03 wrote:
| People can't even have a conversation about any kind of
| societal issues these days without pointing at the other
| political tribe and casting aspersions about "what they really
| meant" instead of engaging with what's actually being said.
|
| Forgetting that if you really can hear a dogwhistle, you're
| also a dog.
| root_axis wrote:
| I don't think we have much to worry about in terms of economic
| disruption. At this point it seems pretty clear that LLMs are
| having a major impact on how software is built, but for almost
| every other industry the practical effects are mostly
| incremental.
|
| Even in the software world, the effect of being able to build
| software a lot faster isn't really leading to a fundamentally
| different software landscape. Yes, you can now pump out a month's
| worth of CRUD in a couple days, but ultimately it's just the same
| CRUD, and there's no reason to expect that this will change
| because of LLMs.
|
| Of course, creative people with innovative ideas will be able to
| achieve more, a talented engineer will be able to embark on a
| project that they didn't have the time to build before, and that
| will likely lead to some kind of software surplus that the
| economy feels on the margins, but in practical terms the economy
| will continue to chug along at a sustained pace that's mostly
| inline with e.g. economic projections from 10 years ago.
| jonas21 wrote:
| > _At this point it seems pretty clear that LLMs are having a
| major impact on how software is built, but for almost every
| other industry the practical effects are mostly incremental._
|
| Even just a year ago, most people thought the practical effects
| in software engineering were incremental too. It took another
| generation of models and tooling to get to the point where it
| could start having a large impact.
|
| What makes you think the same will not happen in other
| knowledge-based fields after another iteration or two?
| root_axis wrote:
| Software is more amenable to LLMs because there is a rich
| source of highly relevant training data that corresponds
| directly to the building blocks of software, _and_ the
| "correctness" of software is quasi-self-verifiable. This
| isn't true for pretty much anything else.
| dpflan wrote:
| The more verifiable the domain the better suited. We see
| similar reports of benefits from advanced mathematics
| research from Terrence Tao, granted some reports seem to
| amount to very few knew some data existed that was relevant
| to the proof, but the LLM had it in its training corpus.
| Still, verifiably correct domains are well-suited.
|
| So the concept formal verification is as relevant as ever,
| and when building interconnected programs the complexity
| rises and verifiability becomes more difficult.
| root_axis wrote:
| > _The more verifiable the domain the better suited._
|
| Absolutely. It's also worth noting that in the case of
| Tao's work, the LLM was producing Lean and Python code.
| 2001zhaozhao wrote:
| I think the solution in harder-to-verify cases is to
| provide AI (sub-)agents a really good set of instructions
| on a detailed set of guidelines of what it should do and
| in what ways it should think and explore and break down
| problems. Potentially tens of thousands of words of
| instructions to get the LLM to act as a competent
| employee in the field. Then the models need to be good
| enough at instruction-following to actually explore the
| problem in the right way and apply basic intelligence to
| solve it. Basically treating the LLM as a competent
| general knowledge worker that is unfamiliar with the
| specific field, and giving it detailed instructions on
| how to succeed in this field.
|
| For the easy-to-verify fields like coding, you can train
| "domain intuitions" directly to the LLM (and some of this
| training should generalize to other knowledge work
| abilities), but for other fields you would need to supply
| them in the prompt as the abilities cannot be trained
| into the LLM directly. This will need better models but
| might become doable in a few generations.
| fc417fc802 wrote:
| Presumably at some point capability will translate to other
| domains even if the exchange rate is poor. If it can
| autonomously write software and author CAD files then it
| can autonomously design robots. I assume everything else
| follows naturally from that.
| marcosdumay wrote:
| > most people thought the practical effects in software
| engineering were incremental too
|
| Hum... Are you saying it's having clear positive (never mind
| "transformative") impact somewhere? Can you point any place
| we can see observable clear positive impact?
| xcodevn wrote:
| > we may have AI that is more capable than everyone in only 1-2
| years
|
| There's no evidence this will be the case...
| pineaux wrote:
| dont forget who is writing it and what he needs to think about
| it and what he wants others to think about it...
| mrdependable wrote:
| I'm starting to think people who build these models are more
| likely to suffer from AI psychosis.
| ares623 wrote:
| "The worst person you know is being told 'You are absolutely
| right!' by an LLM right now"
| mordymoop wrote:
| What would you consider such evidence to look like?
| krunck wrote:
| I fear that when this technology grows up it will first be in the
| hands of the propagandists and war mongers. The rest of use won't
| stand a chance against the real-time propaganda streams
| convincing us why "we" needs to attack the bad guy country of the
| month die so we can take their stuff. Or maybe we'll be so
| sedated by genAI, 24/7, always new, personally customized
| entertainment that we won't care.
| direwolf20 wrote:
| It's already there. Propaganda was one of the first uses of
| LLMs, and before that, they used humans.
| ares623 wrote:
| Now the humans can be used in the meat grinder. No job
| losses, just different kind of jobs. Am I doing this right?
| reducesuffering wrote:
| This is the most important article to come across HN in a while
| and I encourage you to read it for the immense intellectual
| wisdom it contains rather than the reflexive uneducated discourse
| on AI that envelops HN these days. I'm sure if you read it end-
| to-end you'd likely agree.
| rishabhaiover wrote:
| God, gone are the days when I'd spend three days writing unit
| tests and phone it in for the other two just to reach the
| weekend.
| mwcampbell wrote:
| This is obviously bullshit. If he were really worried about the
| things he says he is, he'd put the brakes on his company, or
| would never have started it in the first place.
| voidmain wrote:
| So if someone (actually, practically everyone) who runs an AI
| company says AI is dangerous, it's bullshit. If someone who is
| holding NVDA put options says it, they're talking their book.
| If someone whose job is threatened by AI says it, it's cope. If
| someone who doesn't use AI says it, it's fear of change. Is
| there someone in particular you want to hear it from, or are
| you completely immune to argument?
| mwcampbell wrote:
| I actually do believe that AI is dangerous, though for
| different reasons than the ones he focuses on. But I don't
| think he really believes it, since if he did, he wouldn't be
| spending billions to bring it into existence.
| surgical_fire wrote:
| > So if someone (actually, practically everyone) who runs an
| AI company says AI is dangerous, it's bullshit.
|
| My instinct is to take his words as a marketing pitch.
|
| When he says AI is dangerous, it is a roundabout way to say
| it is powerful and should be taken seriously.
| mwcampbell wrote:
| Yes, exactly.
| 2001zhaozhao wrote:
| It's interesting just how many opinions Amodei shares with AI
| 2027's authors despite coming from a pretty different context.
|
| - Prediction of exponential AI research feedback loops (AI coding
| speeding up AI R&D) which Amodei says is already starting today
|
| - AI being a race between democracies and autocracies with
| winner-takes-all dynamics, with compute being crucial in this
| race and global slowdown being infeasible
|
| - Mention of bioweapons and mirror life in particular being a big
| concern
|
| - The belief that AI takeoff would be fast and broad enough to
| cause irreplaceable job losses rather than being a repeat of past
| disruptions (although this essay seems to be markedly more
| pessimistic than AI 2027 with regard to inequality after said job
| losses)
|
| - Powerful AI in next few years, perhaps as early as 2027
|
| I wonder if either work influenced the other in any way or is
| this just a case of "great minds think alike"?
| reducesuffering wrote:
| It's because few realize how downstream most of this AI
| industry is of Thiel, Eliezer Yudkowsky and LessWrong.com.
|
| Early "rationalist" community was concerned with AI in this way
| 20 years ago. Eliezer inspired and introduced the founders of
| Google DeepMind to Peter Thiel to get their funding. Altman
| acknowledged how influential Eliezer was by saying how he is
| most deserving of a Nobel Peace prize when AGI goes well (by
| lesswrong / "rationalist" discussion prompting OpenAI).
| Anthropic was a more X-risk concerned fork of OpenAI. Paul
| Christiano inventor of RLHF was big lesswrong member. AI 2027
| is an ex-OpenAI lesswrong contributor and Scott Alexander, a
| centerpiece of lesswrong / "rationalism". Dario, Anthropic CEO,
| sister is married to Holden Karnofsky, a centerpiece of
| effective altruism, itself a branch of lesswrong /
| "rationalism". The origin of all this was directionally
| correct, but there was enough power, $, and "it's inevitable"
| to temporarily blind smart people for long enough.
| adithyan_win wrote:
| I wanted a version to read on Kindle, so I made the following.
|
| The EPUB + PDF version is here:
| https://www.adithyan.io/blog/kindle-ready-adolescence-of-tec...
___________________________________________________________________
(page generated 2026-01-26 23:00 UTC)