[HN Gopher] We're one step closer to reading an octopus's mind
___________________________________________________________________
We're one step closer to reading an octopus's mind
Author : Amorymeltzer
Score : 89 points
Date : 2023-04-10 15:29 UTC (1 days ago)
(HTM) web link (arstechnica.com)
(TXT) w3m dump (arstechnica.com)
| tough wrote:
| [flagged]
| tired_star_nrg wrote:
| Every day we get closer to making this onion article a reality:
|
| https://clickhole.com/the-future-is-here-elon-musk-stuck-a-g...
| mysterydip wrote:
| Probably "swim, swim, hungry" http://dopefish.com/fishinfo.html
| 7373737373 wrote:
| I _HIGHLY_ recommend this talk from the HotChips conference
| "HC29-K1: The Direct Human/Machine Interface and Hints of a
| General Artificial Intelligence", specifically the next 4 minutes
| after this timestamp:
|
| https://youtu.be/PVuSHjeh1Os?t=1168
|
| Also this: "Flashes of Insight: Whole-Brain Imaging of Neural
| Activity in the Zebrafish"
|
| https://youtu.be/eKkaYDTOauQ
| asplake wrote:
| Meanwhile from the sci-fi department, got to mention the Children
| of Time series by Adrian Tchaikovsky
| nozzlegear wrote:
| I really enjoyed Children of Time, and I just started the
| latest entry in the series a couple of days ago. Speaking of
| octopuses though, I highly recommend The Mountain in the Sea by
| Ray Nayler. It was a riveting read and dealt just as much with
| artificial intelligence (timely considering all the hubbub
| around LLMs lately) as it did with the rise of consciousness in
| octopuses in the near future and how humanity might react to
| that. I couldn't put it down.
| asplake wrote:
| Purchased the audiobook, thanks!
| ljlolel wrote:
| If Octopuses could evolve human intelligence and society in
| just a few generations, then why wouldn't they have done so
| in the hundreds of millions of years before?
|
| Hint: they must have
| ninjanomnom wrote:
| One of the longest lived octopus species we know of has a
| lifespan of about 5 years, even if they were more
| intelligent than humans they'd have a hard time
| establishing a society only being able to establish 5 years
| worth of knowledge per individual. Let's make longevity
| treatments for octopi and see if they learn enough to
| make/join society.
| perihelions wrote:
| I wonder what would happen if you prompted an LLM with something
| other than human language, like a large upload of patterns from a
| cephalopod brain. Could it translate concepts?
| toss1 wrote:
| It'll be cool when we get there!!
|
| But, considering that the most massive LLMs do not even attempt
| to abstract and manipulate even basic concepts (they just model
| likely sequences of words using insanely large data sets), that
| would be a definitive "No.".
|
| Plus, the fact that we have no idea how to map even human
| neural activity patterns to concepts, also "No.", it'd be even
| harder to figure out the octopus' conceptual map.
| knome wrote:
| >the most massive LLMs do not even attempt to abstract and
| manipulate even basic concepts
|
| then how does this work? H: translate the
| following sentence to instead use olympic curling related
| metaphors ( do not explicitly mention curling, and do not use
| similes in forming the translation ): man:"I've been spinning
| my tires all day without getting anywhere" woman:"running out
| of gas?" man:"nah, I'm gassed up, geared up, oiled and
| revving. I just can't seem to get in the right lane, you
| know?" woman:"anything knocking?" man:"nah, I'm tuned just
| fine. I can't seem to get my GPS to connect, you know?"
| woman:"take a few laps without a destination, see if you can
| figure out the lay of the land" man:"you know, that might
| help. thanks barbara" R: man: "I've been sweeping the
| ice all day without hitting the target." . woman:
| "Losing momentum?" . man: "Nah, I've got the energy,
| the technique, and the focus. I just can't seem to find the
| right path, you know?" . woman: "Anything coming in
| your way?" . man: "Nah, I'm on point with my aim. I
| just can't seem to get my strategy to align, you know?"
| . woman: "Maybe glide a few times without aiming
| specifically, see if you can figure out the ice conditions."
| . man: "You know, that might help. Thanks, Barbara."
|
| I find it unlikely that these larger models do anything less
| than abstract and manipulate concepts.
|
| ( my apologies to anyone that has seen me paste this same
| example more than once, I find it a very good and succinct
| example for the topic )
| zffr wrote:
| Why not? LLM convert words into vectors and then discover
| relationships between them. If it is possible to convert
| brain patterns into vectors, I can see how an LLM could
| detect patterns between those too.
|
| Converting the brain pattern vectors back to words could be
| difficult, but might be possible if each brain pattern vector
| was annotated with a description of the organism's current
| environment and the organism's current actions.
| iamerroragent wrote:
| I'm curious when you say LLM convert words in to vectors,
| is it similar to vectors in physics, with one part being a
| magnitude and the other part being a direction?
| jesse_cureton wrote:
| It's similar - basically here a vector/tensor is an array
| of magnitudes across N dimensions. Whereas in (undergrad-
| level) physics you might have a 3- or 4-D vector for
| spatial dimensions and time, here the LLMs are embedding
| sequences of tokens into N-dimensional space where N is
| much much larger.
|
| There's a pattern called "embedding search" - you
| precalculate a set of embeddings for a corpus of text.
| Then to do a search, you calculate embeddings for your
| search string. Then you can find the closest vector in
| that N-dimensional space, which finds you the
| semantically closest neighbor from the original corpus.
|
| For an embedding search - the OpenAI Embeddings API gives
| you a ~1500 dimension output vector. When a LLM is
| working with input text as a vector, I am not sure what
| the tokenizer is actually feeding into the model.
| Hopefully someone else can chime in!
| iamerroragent wrote:
| This is really elucidating.
|
| Thank you and your time for writing that out.
| TeMPOraL wrote:
| The magic is really in how absurdly high-dimensional
| those vectors are. The dimensionality of the latent space
| in current breed of LLMs is, IIRC, on the order of
| _hundred thousand_ dimensions.
|
| Now, even if all the model does is 1) turn your prompt
| into high-dimensional vectors, 2) run an adjacency search
| to find the vectors in the latent space nearest to your
| vector, and 3) translate those vectors back to tokens,
| it's more than enough for it to work with concepts.
| Again, we're talking 1000 to 100 000 dimensional vectors
| here. Any kind of semantic similarity between words you
| can think of (tree - green - grass, tree - tall -
| skyscraper, tree - data structure, tree - files, etc.)
| can fit in there - the relevant words (tokens) will be
| close together along _some_ dimensions. So, if you pick a
| vector somewhere in the latent space, and look around (in
| hundred thousand dimensions) for its nearest neighbors,
| the group of points you 'd be looking at is, IMHO, a
| _concept_ in its raw form.
| krisoft wrote:
| This is the goal of the Project Ceti, albeit with whale
| vocalisations and not cephalopod brain readings.
|
| https://www.projectceti.org/
| wrycoder wrote:
| Which one?
| [deleted]
___________________________________________________________________
(page generated 2023-04-11 23:01 UTC)