[HN Gopher] It Looks Like You're Trying to Take over the World
___________________________________________________________________
It Looks Like You're Trying to Take over the World
Author : ruuda
Score : 51 points
Date : 2023-02-15 18:52 UTC (4 hours ago)
(HTM) web link (gwern.net)
(TXT) w3m dump (gwern.net)
| neilv wrote:
| How do you get all the Clippies to cooperate towards a collective
| goal?
|
| Asking for a friend.
| nynx wrote:
| This kind of reminds me of the (very beautiful) beginning of
| Diaspora, by Greg Egan, in which a nascent machine/human
| intelligence emerges from a chaotic, abstract virtual machine.
| jvanderbot wrote:
| A less dystopian version can be had in the Bobiverse series. Love
| those books!
| gwern wrote:
| Previously (URL/domain changed):
| https://news.ycombinator.com/item?id=30818895
| cs702 wrote:
| Do you believe models can go through the kind of phase
| transition you mention, after which they start modeling
| themselves as agents in a world (i.e., they become "conscious"
| or "self-aware," for lack of a better term), _without requiring
| any new engineering breakthroughs or theoretical advances_?
| rtkwe wrote:
| We have such a poor understanding of our own consciousness
| that I think it's hard to answer that. We know a fair amount
| of what goes on in our heads and fundamentally it's fairly
| closely modelled by neural nets but we don't really know
| where (or even really if) it jumps from a complex but dead
| input output set of neurons to a (seemingly) self reflective
| consciousness.
| cs702 wrote:
| Yes, at the moment it's a matter of _belief_. That 's why I
| asked gwern if he/she/they _believe_ new breakthroughs will
| or won 't be necessary.
| efitz wrote:
| When we start building models to train and optimize other
| models, I would expect emergent behavior. If we did something
| foolish like building yhe system in such a way that it could
| write and execute code, then the emergent behavior could do
| things as nasty as anything else on the internet.
|
| We have also seen emergent behavior in programmable hardware
| [1] so I tend to be pessimistic about the limits of what such
| a system could do. Also it could pay human minions to
| interact with the physical world if needed.
|
| [1] https://www.damninteresting.com/on-the-origin-of-circuits
| dang wrote:
| Thanks! Macroexpanded:
|
| _It looks like you're trying to take over the world_ -
| https://news.ycombinator.com/item?id=30818895 - March 2022 (274
| comments)
| mark_l_watson wrote:
| That is a fun story!
| A4ET8a8uTh0 wrote:
| It made me smile.
|
| I recently thought about why certain options are restricted from
| GPT ( predictions come to mind ) and it slowly became apparent
| that with enough information you could predict not a specific
| individual making a specific move, but likely specific event
| happening.
|
| Would you like to proceed?
| jkingsman wrote:
| This reminds me a lot of an epistolary novel I quite enjoyed --
| Exegesis by Astro Teller [0]. It's a story told entirely via
| emails about this happening in much less technical detail, but
| it's still a quick and fun read.
|
| [0] https://www.amazon.com/Exegesis-Astro-Teller/dp/037570051X
| Karellen wrote:
| Also of the 2005 FOSS computer game _Endgame: Singularity_
|
| https://en.wikipedia.org/wiki/Endgame%3A_Singularity
___________________________________________________________________
(page generated 2023-02-15 23:02 UTC)