_______ __ _______
| | |.---.-..----.| |--..-----..----. | | |.-----..--.--.--..-----.
| || _ || __|| < | -__|| _| | || -__|| | | ||__ --|
|___|___||___._||____||__|__||_____||__| |__|____||_____||________||_____|
on Gopher (inofficial)
(HTM) Visit Hacker News on the Web
COMMENT PAGE FOR:
(HTM) How the Brain Parses Language
taeric wrote 10 hours 6 min ago:
This feels too reductive to me. In particular, it makes a hard
distinction between the thinking and the language. I fully accept that
they are distinct, but how distinct? It is hard not to think that some
thinking styles influence how something is heard?
Not just in full language, mind, but consider the last time you heard a
song in a major key? Do you even know what that means? Because many
of us do not.
Same goes for listening to people discuss things like sports. I'm
inclined to think many people effectively run a simulation in their
mind of a game as they listen to it broadcast. This almost certainly
isn't inherent to the language, it is part of the learning of it,
though. Think looking over lists of the moves in a chess game. Then
go from that to laying out the pieces as they are after that list. Or
calling what the next move can be.
Can this be a completely separate set of "circuitry" in our brains that
first parses the language and then builds the simulation? I suppose.
Seems more likely there is something that is active between the two
that can effectively get merged in advanced practitioners.
fallingfrog wrote 11 hours 39 min ago:
I've had the experience of having migraines with aphasia- this is
essentially a migraine aura that affects the part of the brain that
processes language. I can confirm that while this was happening, i was
aware of my surroundings and able to have thoughts, but I was unable to
speak and unable to understand spoken or written language. It all just
looked and sounded like gibberish. I thought about whether I should go
to a hospital, what was going on, wondered whether my loved ones were
concerned, and so on, but was unable to communicate any of those
thoughts to other people. It was a bizarre experience.
rdtsc wrote 12 hours 23 min ago:
> But what if our neurobiological reality includes a system that
behaves something like an LLM?
With every technological breakthrough we always posit that the brain
has to work like the newly discovered thing. At various times brains
were hydraulic, mechanical, electrical, like a computer, like a
network. Now, of course, the brain has to be like an LLM.
jimbokun wrote 10 hours 34 min ago:
All of those analogies were useful in some ways, and LLMs are too.
There's also a progression in your sequence. There were rudimentary
mechanical calculating devices, then electrical devices begat
electrical computers, and LLMs are a particular program running on a
computer. So in a way the analogies are becoming more refined as we
develop systems more and more capable of mimicking human
capabilities.
HarHarVeryFunny wrote 11 hours 42 min ago:
Yes, but at least now we're comparing artificial to real neural
networks, so the way it works at least has a chance of being similar.
I do think that a transformer, a somewhat generic
hierarchical/parallel predictive architecture, learning from
prediction failure, has to be at least somewhat similar to how we
learn language, as opposed to a specialized Chompyskan "language
organ".
The main difference is perhaps that the LLM is only predicting based
on the preceding sequence, while our brain is driving language
generation by a combination of sequence prediction and the thoughts
being expressed. You can think of the thoughts being a bias to the
language generation process, a bit like language being a bias to a
diffusion based image generator.
What would be cool would be if we could to some "mechanistic
interpretability" work on the brain's language generation circuits,
and perhaps discover something similar to induction heads.
paddleon wrote 8 hours 0 min ago:
> comparing artificial to real neural networks
I had a sad day in college when I thought I'd build my own ANN
using C++.
First thing I did was create a "Neuron" class, to mimic the idea of
a human neuron.
Second thing I did was realize that ANNs are actually just Weiner
filters with a sigmoid on top. The base unit is not a "neuron".
aeve890 wrote 9 hours 45 min ago:
>Yes, but at least now we're comparing artificial to real neural
networks
Given that the only similarity between the two of is just the
"network" structure I'd say that point is pretty weak. The name
"artificial neural network" it's just an historical artifact and an
abstraction totally disconnected from the real thing.
HarHarVeryFunny wrote 5 hours 28 min ago:
Sure, but ANNs are at least connectionist, learning
connections/strengths and representations, etc - close enough at
that level of abstraction that I think ANNs can suggest how the
brain may be learning certain things.
rdtsc wrote 10 hours 55 min ago:
> Yes, but at least now we're comparing artificial to real neural
networks, so the way it works at least has a chance of being
similar.
Indeed, and I wasn't even saying it's wrong, it may be pretty
close.
> What would be cool would be if we could to some "mechanistic
interpretability" work on the brain's language generation circuits,
and perhaps discover something similar to induction heads.
Yeah, I wouldn't be surprised. And maybe the more we find out about
the brain, it could lead to some new insights about how to improve
AI. So we'd sort of converge from both sides.
griffzhowl wrote 12 hours 31 min ago:
One disanalogy between human language use and LLMs is that language
evolved to fit the human brain, which was already structured by
millions of years of primate social life. This is more or less the
reverse situation to a neural network trained on a large text corpus.
HarHarVeryFunny wrote 11 hours 36 min ago:
Yes, but animal/human brains (cortex) appear to have evolved to be
prediction machines, originally mostly predicting evolving sensory
inputs (how external objects behave), and predicting real-world
responses to the animal's actions.
Language seems to be taking advantage of this pre-existing predictive
architecture, and would have again learnt by predicting sensory
inputs (heard language), which as we have seen is enough to induce
ability to generate it too.
adamzwasserman wrote 13 hours 37 min ago:
There's an interesting falsifiable prediction lurking here. If the
language network is essentially a parser/decoder that exploits
statistical regularities in language structure, then languages with
richer morphological marking (more redundant grammatical signals)
should be "easier" to parse â the structure is more explicitly marked
in the signal itself.
French has obligatory subject-verb agreement, gender marking on
articles/adjectives, and rich verbal morphology. English has largely
shed these. If you trained identical neural networks on French vs
English corpora, holding everything else constant, you might expect
French models to hit certain capability thresholds earlier â not
because of anything about the network, but because the language itself
carries more redundant structural information per token.
This would support Fedorenko's view that the language network is
revealing structure already present in language, rather than
constructing it. The "LLM in your head" isn't doing the thinking â
it's a lookup/decode system optimized for whatever linguistic code you
learned.
(Disclosure: I'm running this exact experiment. Preregistration: [1] )
(HTM) [1]: https://osf.io/sj48b
mcswell wrote 1 hour 4 min ago:
Written French does have all that inflectional morphology you talk
about, but spoken French has much less--a lot of the inflectional
suffixes are just not pronounced on most verbs (with the exception of
a few, like être and aller--but at least 'be' in English is
inflected in ways that other verbs are not). So there's not that
much redundancy.
As for gender marking on adjectives--or nouns--it does almost no
semantic work in French, except where you're talking about
professional titles (doctor, professor...) that can be performed by
men or by women.
If you want a heavily inflected language, you should look at
something like Turkish, Finnish, Swahili, Quechua, Nahuatl, Inuit...
Even Spanish (spoken or written) has more verbal inflection than
spoken French.
patcon wrote 3 hours 37 min ago:
I suspect you're more right than wrong. I'm a strong believer in this
sort of thing -- that humans are best understood as a cyborg of a
biological and semiotic organism, but mostly a "language symbiont
inside a host". We should perhaps understand this as the strange
creature of language jumping between hosts. But I suspect we're
looking at a mule of sorts: it can't reproduce properly. But this
mule could destroy us if we put it to work doing the wrong things,
with too much agency when it doesn't have the features that give us
the right to trust our own agency as evolved creatures.
You might be interested to look into the Leiden Theory of
Language[1][2]. It's been my absolutely favourite fringe theory of
mind since I stumbled across the rough premise in 2018, and went
looking for other angles on it. [1]
[2] > Language is a mutualist symbiont and enters into a mutually
beneficial relationship with its hominid host. Humans propagate
language, whilst language furnishes the conceptual universe that
guides and shapes the thinking of the hominid host. Language enhances
the Darwinian fitness of the human species. Yet individual
grammatical and lexical meanings and configurations of memes mediated
by language may be either beneficial or deleterious to the biological
host.
EDIT: almost forgot the best link!
Language as Organism: A Brief Introduction to the Leiden Theory of
Language Evolution
(HTM) [1]: https://www.kortlandt.nl/publications/art067e.pdf
(HTM) [2]: https://en.wikipedia.org/wiki/Symbiosism
(HTM) [3]: https://www.isw.unibe.ch/e41142/e41180/e523709/e546679/2004f...
adamzwasserman wrote 1 hour 50 min ago:
Thank you for the Leiden references. I hadn't encountered this
framework before. The "language symbiont" framing resonates with
what I've been circling around: a system that operates with its own
logic, sometimes orthogonal to conscious intention.
The mule analogy is going to stick with me. LLMs have inherited the
statistical structure of the symbiont without the host: pattern
without grounding. Whether that makes them useful instruments for
studying the symbiont itself, or just misleading simulacra, is
exactly what I'm trying to work out.
Going to dig into Kortlandt tonight.
retrac wrote 9 hours 32 min ago:
That presumes that languages with little morphology do not have
equivalent structures at work elsewhere doing the same kind of heavy
lifting.
One classic finding in linguistics is that languages with lots of
morphology tend to have freer word order. Latin has lots of
morphology and you can move the verb or subject anywhere in the
sentence and it's still grammatical. In a language like English
syntax and word order and word choice take on the same role as
morphology.
Inflected languages may indeed have more information encoded in each
token. But the relative position of the tokens to each other also
encodes information. And inflected languages appear to do this less.
Languages with richer morphology may also have smaller vocabularies.
To be fair, this is a contested conjecture too. (It depends a lot on
how you define a morpheme.) But the theory is that languages like
Ojibwe or Sansrkit with rich derivational morphologies and
grammatical inflections simply don't need a dozen words for different
types of snow, or to describe thinking. A single morpheme with an
almost infinite number of inflected forms can carry all the shades of
meaning, where different morphemes might be used to make the same
distinctions, in a less inflected language.
adamzwasserman wrote 7 hours 45 min ago:
These are good points that sharpen the hypothesis. The word order
question is interesting â positional encoding vs morphological
encoding might have different computational properties for a
parser.
One difference I'm betting on: morphological agreement is redundant
(same information marked multiple times), while word order encodes
information once. Redundancy aids error correction and may lower
pattern extraction thresholds. But I'm genuinely uncertain whether
that outweighs the structural information carried by strict word
order.
Do you have intuitions on which would be "easier" for a statistical
learner? Or pointers to relevant literature? The vocabulary size /
morpheme count tradeoff is also something I hadn't fully considered
as a confound.
pessimizer wrote 8 hours 53 min ago:
You saved me from posting this. Strict word order makes a lot of
things easier that have to be done through morphology in the vulgar
Latins.
> Languages with richer morphology may also have smaller
vocabularies. To be fair, this is a contested conjecture too.
I agree with the criticism of this to an extent. A lot of has
seemed to me like it relies on thinking of English as a sort of
normal, baseline language when it is actually very odd. It has so
many vowels, and it also isn't open so has all of these little
weird distinguishing consonant clusters at the end of syllables.
And when you compare it to a language conjugated with a bunch of
suffixes, those suffixes gradually both make the words very long,
and add a bunch of sounds that can't be duplicated very often at
the end of roots without causing confusion.
All of that together means that there's a lot more bandwidth for
more words. English, even though it has a lot more words than other
languages, doesn't have more precise words. Most of them are vague
duplications, including duplicating most of Norman French just to
have special, fancy versions of words that already existed. The
strong emphasis on position in the grammar and the vast number of
vowels also allows it to easily borrow words from other languages
without a compelling reason.
I think all of that is enough to explain why English is such an
outlier on vocabulary size, and I think you see similar in other
languages that share a subset of these features.
tgv wrote 9 hours 53 min ago:
There are more differences between English and French than you just
described, and they can affect your measurement. Even the corpora you
use cannot be the same. There isn't "ceteris paribus" (holding
everything else constant). The outcome of the experiment doesn't say
anything about the hypothesis.
You're also going to use an artificial neural network to make claims
about the human brain? That distance is too large to bridge with a
few assumptions.
BTW, nobody believes our language faculties are doing the thinking.
There are however, obviously, connections to thought: not only the
concepts/meaning, but possibly sharing neural structures, such as the
feedback mechanism that allows us to monitor ourselves.
I have a slightly better proposal: if you want to see the effect of
gender, genderize English or neutralize French, and compare both
versions of the same language. Careful with tokenization, though.
adamzwasserman wrote 7 hours 43 min ago:
The confound concern is fair: no cross-linguistic comparison is
perfectly controlled. The bet is that the effect size (if any) will
be large enough to be informative despite the noise. But you're
right that it's not ceteris paribus in a strict sense.
Your proposal is interesting though. Synthetic manipulation of
morphology within a single language. Have you seen this done? The
challenge I'd anticipate is that "genderized English" wouldn't have
natural text to train on, so you'd need to generate it somehow,
which introduces its own artifacts. But comparing French vs
artificially gender-neutralized French might be feasible with
existing parallel corpora. Worth thinking about as a follow-up.
On the neural network â brain distance: agreed it's a leap. The
claim isn't that transformers are brains, but that if both are
extracting structure from language, they might reveal something
about what structure is there to extract. Fedorenko's own
comparison to "early LLMs" suggests she thinks the analogy has some
merit.
Grosvenor wrote 11 hours 32 min ago:
And we have those French/English text corpora in the form of Canadian
law. All laws in Canada at the federal level are written in English
and French.
This was used to build the first modern language translation systems,
testing them going from
English->french->english. And in reverse.
You could do similar here
, understanding that your language is quite stilted legalese.
Edit: there might be other countries with similar rules in place that
you could source test data from as well.
adamzwasserman wrote 10 hours 46 min ago:
Incredibly, I had not thought to use that data set.
Now I will. Thanks.
seszett wrote 8 hours 35 min ago:
Belgian federal law is also written in Dutch, French and German,
by the way.
But no English so you might not be interested.
fellowniusmonk wrote 12 hours 12 min ago:
Dyslexia seems to be more of an issue in English than other languages
right?
But also, maybe the difficulty of parsing recruits other/executive
function and is beneficial in other ways?
The per phoneme density/efficiency of English is supposed to be quite
high as an emergent trade language.
Perhapse speaking a certain language would promote slower more
intentional parsing, humility through syntax uncertainty, maybe not,
all I know is that from a global network resilience perspective it's
good that dumb memes have difficulty propagating across
cultures/languages.
adamzwasserman wrote 12 hours 3 min ago:
The dyslexia point is interesting; yes, English orthography causes
more reading disorders than languages with more regular
spelling-to-sound mappings (Italian, Finnish, etc.). That's
consistent with the parser having to work harder when the signal is
noisier.
Your intuition about "slower more intentional parsing" connects to
something I'm exploring: we may parse language at two levels
simultaneously; a fast, nearly autonomic level (think: how insults
land before you consciously process them) and a slower deliberate
level. Whether those levels interact differently across languages
is an open question.
tgv wrote 9 hours 11 min ago:
First: dyslexia has little to do with parsing, which is generally
understood to relate to structure/relations between words.
Second: multiple levels of language processing have been
identified, although it's not at all clear how well separated
they are. The higher levels (semantics, pragmatics) are by
necessity lagging behind the lower (phonetics, syntax). The
higher levels also seem more "deliberate."
netfortius wrote 14 hours 17 min ago:
Every time I read something like this reminds me of Maturana (of
autopoiesis fame), who was among the first scientists from where I
started gaining an interest in these areas. Relevant to his view, in
the area of language, is the following:
"We human beings are living systems that exist in language. This means
that although we exist as human beings in language and although our
cognitive domains (domains of adequate actions) as such take place in
the domain of languaging, our languaging takes place through our
operation as living systems. Accordingly, in what follows I shall
consider what takes place in language[,] as language arises as a
biological phenomenon from the operation of living systems in recurrent
interactions with conservation of organization and adaptation through
their co-ontogenic structural drift, and thus show language as a
consequence of the same mechanism that explains the phenomena of
cognition:"
qqxufo1 wrote 15 hours 42 min ago:
If the brain's language network is only for "packaging words" and not
for actual logic or reasoning, why does writing or speaking our messy
thoughts out loud suddenly make them feel more logical? Is language
actually helping us think, or is it just a filter that forces our
chaotic ideas into a structure we can finally understand?
mcswell wrote 40 min ago:
That's a really good question. I don't have an answer, or even the
beginning of an answer, but I would hazard a guess that there is a
feedback loop. So listening to yourself talk (or even better,
putting your thoughts down in print) is sort of like listening to
someone else talk, which puts new ideas into your mind, or causes you
to better organize the ones you already have.
Doing mathematical proofs might be an extreme example of that: a
mathematician has (I am told) an intuition--a thought--but has to
work it out rigorously. Once they've done that, the intuition
becomes much clearer. I guess.
lapcat wrote 16 hours 2 min ago:
I wouldn't read too much into the LLM analogy. The interview is
disappointingly short, filled with a bunch of unnecessarily tall
photgraphs, and the interviewer, the one who brought up LLMs and
ChatGPT and has a history of writing AI articles ( [1] ), almost seemed
to have an agenda to contextualize the research in this way. In
general, except in a hostile context such as politics, interviewees
tend to be agreeable and cooperative with interviewers, which means
that interviews can be steered in a predetermined way, probably for
clickbait here.
In any case, there's a key disanalogy:
> Unlike a large language model, the human language network doesnât
string words into plausible-sounding patterns with nobody home;
instead, it acts as a translator between external perceptions (such as
speech, writing and sign language) and representations of meaning
encoded in other parts of the brain (including episodic memory and
social cognition, which LLMs donât possess).
(HTM) [1]: https://www.quantamagazine.org/authors/john-pavlus/
adamzwasserman wrote 13 hours 30 min ago:
The disanalogy you quote might actually be the key insight. What if
language operates at two levels, like Kahneman's System 1/2?
Level 1: Nearly autonomic â pattern-matched language that acts
directly on the nervous system. Evidence: how insults land before you
"process" them, how fluent speakers produce speech faster than
conscious deliberation allows, and the entire body of work on
hypnotic suggestion, which relies on language bypassing conscious
evaluation entirely.
Level 2: The conscious formulation you describe â the translator
between perception and meaning.
LLMs might be decent models of Level 1 but have nothing corresponding
to Level 2. Fedorenko's "glorified parser" could be the Level 1
system.
lapcat wrote 12 hours 32 min ago:
> LLMs might be decent models of Level 1
I don't think so. Fast speakers and hyponotized people are still
clearly conscious and "at home" inside, vastly more "human" than
any LLM. Deliberation and evaluation imply thinking before you
speak but do not imply that you can't otherwise think while you
speak.
liampulles wrote 16 hours 10 min ago:
What I'm curious about is what the language parts of the human brain
look like for babies and toddlers. Humans obviously have a bunch of
languages they can speak, and toddlers pick up the language that their
guardians speak around their home, so there seems to be machinery there
that is for the task of "online" learning.
trebligdivad wrote 9 hours 34 min ago:
I'd like one stage further - what are the genetics of this area? How
does a dedicated brain area like this get encoded - (Hopefully the
Allen Institute might dig on this one?); but if we can find how the
areas are encoded in the DNA we could presumably see how they
evolved, but then perhaps also spot other areas?
lukeinator42 wrote 9 hours 57 min ago:
It's an interesting area of research, there is even some evidence
that language experienced in utero affects speech perception: [1] .
(HTM) [1]: https://doi.org/10.1111/apa.12098
griffzhowl wrote 12 hours 46 min ago:
One part of the story I found fascinating is the overlap in infants'
brains of the areas involved in tool use and hierarchical syntax.
These diverge and specialize in adults. The homologous brain region
in primates is involved in motor planning.
It's an interesting hint at the deeper evolutionary origins of
language in the ability to plan complex actions, providing a neural
basis for the observation that language and action planning have this
common structure of an overall goal that can be decomposed into a
structure of subgoals, which we see formalized in computer programs
too.
This is an older reference (1991) where I first heard about it. there
are more recent studies reinforcing various aspects of it but I
didn't find one that was as comprehensive
(HTM) [1]: https://doi.org/10.1017/S0140525X00071235
mcswell wrote 55 min ago:
"overlap in infants' brains of the areas involved in tool use and
hierarchical syntax"---you didn't see that in the Quanta article,
right? I went back and looked, but can't find it mentioned
anywhere.
Anon84 wrote 15 hours 22 min ago:
Me too! Babies and toddlers brains are like sponges. We started
teaching my baby 3 languages since birth (essentially I always spoken
with her in my native language, my wife in hers and gets English from
living in the US). Sheâs not even 4 yet an fully fluent in all
three and seemlessly jumps back and forth between them. (To my
surprise, she doesnât mix words from the different languages in the
same sentence)
mcswell wrote 59 min ago:
There's a lot more to language learning than being a "sponge".
Virtually all the grammar we learn is productive/ creative--that
is, we apply it to new words, and say things we never heard anyone
say before. And the grammar is implicit in what we hear, so
children need to extract it in a form that can be generalized to
new thoughts and words.
mbg721 wrote 37 min ago:
This is why learning Latin the way I did (very methodically and
technically, with no real speaking/responding) makes you good at
parsing it, but not at speaking it. There are schools today
where it's taught as if it were a spoken language.
fellowniusmonk wrote 11 hours 48 min ago:
If you look at the rate of "new" word use after the first spoken
word its very clear that word acquisition and categorizing occurs
for a long period before that first word is ever spoken.
Speaking to babies is incredibly important for linguistics but
probably for all types of complex brain function, I don't think
there is an upper bound on how many words we should expose children
too.
phkahler wrote 12 hours 41 min ago:
>> To my surprise, she doesnât mix words from the different
languages in the same sentence
I knew two brothers that would mix words from different languages
while speaking to each other because they shared the same set of
languages and presumably used the best words to express their
thoughts.
Your daughter probably knows other people generally speak and
understand one language at a time and just conforms because its
most effective.
I'm not sure if or at what age it might be good to start mixing
languages with others who can.
lapcat wrote 15 hours 54 min ago:
I think this quote may speak to the question:
> The brainâs general object-recognition machinery is at the same
level of abstractness as the language network. Itâs not so
different from some higher-level visual areas such as the
inferotemporal cortex (opens a new tab) storing bits of object
shapes, or the fusiform face area storing a basic face template.
In other words, it sounds like the brain may start with the same
basic methods of pattern matching for many different contexts, but
then different areas of the brain specialize in looking for patterns
in specific contexts such as vision or language.
This seems to align with the research of Jenny Saffran, for example,
who has studied how babies recognize language, arguing that this is
largely statistical pattern matching.
mullsork wrote 15 hours 40 min ago:
In the series Babies by Netflix some of her research on this topic
is covered. Season 1 Episode 4 "First Words."
alfanick wrote 17 hours 32 min ago:
Anecdotal data, based on a sample of 1 (aka me). I'm originally Polish,
but I would say my mother tongue is English. I also learned Latin as a
kid/teen. Then learning any other languages is much easier, I also
learned German and some Swiss German dialects. I can also do Spanish,
Italian, French, Dutch, Czech, some Serbo-Croation. I think being
Polish makes learning languages easy - as we have a lot of creations in
Polish that do not translate easily to other languages. I think in my
case it's the same part of brain that processes both human language and
computer language. My brain can do another fun party trick: I never
learned cyrillic, but I can read it just fine, my brain does like
pattern matching and statistical analysis when reading cyrillic.
I also learned to think in hmm "concepts", and then apply a language of
my choice to express them. It's a fun skill to have :) Obviously works
of Chomsky are great, especially exploring if language evolves mind or
is the other way around, does mind evolve language? [let's skip his
rather controversial political views lately].
mzs wrote 10 hours 2 min ago:
I completely understand! I'm also Polish American. I have to say it
helps when mother's side of family is GdaÅsk+west and father's
Lublin+east. My wife's family is all from Warsaw area and I had to
translate for my father-in-law during a holiday to WÅadysÅawowo-Hel
(probably helps my aunt's father's side is Kashubian too, mmm...
dessert first).
I was blown-away on holiday to Croatia. It was so unexpectedly
relatively easily understandable after Czechia, Austria, and
Slovenia. I was all, "What just happened!? Shouldn't this be
something more like Italian?"
It took only a month for me to be able to communicate in Ukrainian
with my ESL students, you're totally right about Cyrillic. And I too
think in concepts but switch my brain to express them externally via
language, whatever that language may be at the moment. I am terrible
at translating OTOH, so unnatural!
But it has it's limits, I got to a point after German and Norwegian
that I thought I harbored a super-power. Then I went to school in
Hungary ;) I also had an ESL student from Lithuania, yep
incomprehensible.
Tor3 wrote 14 hours 13 min ago:
I speak several languages too, though definitely not as many as you
do. I'm also in the process of learning a completely new one, at an
advanced age relative to when I last learned a new one (I was in my
thirties then).
To me, my brain most definitely doesn't process human language the
way it handles computer language. It's about as different as it can
get. The latter is "learning", the former is "burn patterns into the
brain", and learning a language can take years, at least at this age.
Computer languages? Those can be picked up in as little as a weekend,
and getting proficient isn't a multi-year or decade long process. It
feels totally different for me (I've been learning new computer
languages at the same time as I've been trying to get up to speed
with a new human language).
vkazanov wrote 12 hours 28 min ago:
Computer languages are much simpler than human languages, and they
also operate in similar kind of logical ways. I definitely remember
how hard was to go from pascal to C to Cpp to Python to prolog to
haskell to SQL... until at some point nothing was new.
tcsenpai wrote 18 hours 25 min ago:
> But what if our neurobiological reality includes a system that
behaves something like an LLM?
It almost seems like we got inspiration from our brain to build neural
networks!
seanmcdirmid wrote 14 hours 20 min ago:
It isnât clear though. Neural networks were inspired by the brain,
but transformers? It is totally plausible but do we really think just
in words?
SAI_Peregrinus wrote 8 hours 18 min ago:
> It is totally plausible but do we really think just in words?
I find that proposition totally implausible. Some people certainly
report only thinking in words & having a continuous inner
monologue, but I'm not one of them. I think, then I describe my
thoughts in words if I'm speaking or writing or thinking about
speaking or writing.
dr_dshiv wrote 19 hours 14 min ago:
> It almost sounds like youâre saying thereâs essentially an LLM
inside everyoneâs brain. Is that what youâre saying?
>Pretty much. I think the language network is very similar in many ways
to early LLMs, which learn the regularities of language and how words
relate to each other. Itâs not so hard to imagine, right?
Yet, completely glosses over the role of rhythm in parsing language.
LLMs arenât rhythmic at all, are they? Maybe each token production is
a cycle, though⦠hmmâ¦
GolDDranks wrote 18 hours 35 min ago:
I think it's obvious that she means that it's something _like_ LLMs
in some aspects. You are correct in that rhythm and intonation are
very important in parsing language. (And also an important cue when
learning how to parse language!) It's clear that the human language
network is not like LLM in that sense. However, it _is_ a bit like an
_early_ LLM (remember GPT2?) in the sense that it can produce and
parse language, not that it makes much deeper sense in it.
Terretta wrote 8 hours 15 min ago:
> It's clear that the human language network is not like LLM in
that sense.
Is it though? If rhythm or tone changes meaning, then just add
symbols for rhythm and tone to LLM input and train it. You'll get
not just words out that differ based on those additional symbols
wrapping words, but you'll also get the rhythm and tone symbols in
the output.
tgv wrote 16 hours 11 min ago:
However ... language production and perception are quite separated
in our heads. There's basically no parallel to LLMs. Note that the
article doesn't give any, and is extremely vague about the
biological underpinnings of language.
GolDDranks wrote 11 hours 28 min ago:
> language production and perception are quite separated in our
heads
Do you have any evidence for this?
I am a former linguistics student (got my masters), and, after
years of absenteeism in academia, interested in the current state
of the affairs. So: "quite separated in our heads" Evidence for?
against?
tgv wrote 9 hours 46 min ago:
Afasia, and general measures of "normal" performance.
There are various kinds of afasia, often linked to specific
brain areas (Wernicke's and Broca's are well-known). And M/EEG
and fMRI research suggests similar distinctions. It is
difficult to reconcile with the idea that there is only one
language system.
And you will also have noticed that your skills in perception
and production differ. You can read/listen better than
write/speak. Timing, ambiguity and errors in perception and
production differ.
And more logically: the tasks are very different. In
perception, you have to perceive the structure and meaning from
a highly ambiguous, but ordered input of sound triggering
auditory nerves, while during production, meaning is given (in
non-linear order), and you have to find a way to fit it in a
linear, grammatical order with matching words, which then have
to be translated to muscle movements.
moralIsYouLie wrote 3 days ago:
reads like a collection of HN comments by commenters who like to build
"chapter 1" textbook agents using instant-noodle "training tools". "and
what would be the time complexity?"
I can't do this anymore.
Al-Khwarizmi wrote 19 hours 2 min ago:
Ev Fedorenko is a highly recognized cognitive scientist that has been
studying how humans parse language for years.
Of course this doesn't mean one shouldn't question what she says
(that would be an obvious authority fallacy), but I do think it's
fair to say that if you want to question it, the argument should be
more elaborate that "this sounds like she has no idea of the topic".
Timwi wrote 16 hours 9 min ago:
I'm not the person you responded to, but I found the article
unreadable because it kept going on about Evâs life instead of
her research. I'm sure her research is valuable and insightful, but
with this style of reporting it is both inaccessible to me, and it
gives me the (probably flawed) impression that her research isn't
the part of her life that's supposed to be important or impressive.
mcswell wrote 45 min ago:
FWIW, that's soft of the way a lot of physics books (not
textbooks) approach the subject: Einstein/ Heisenberg/ Bohr/
Pauli/ Feynman/ Oppenheimer was this kind of person, oh, and by
the way he came up with this theory of X. Apparently a lot of
people like that way of presenting science, but it's not for
everyone.
jimbokun wrote 10 hours 30 min ago:
This is meant for a lay audience so you should probably just read
her research papers.
Also:
> it gives me the (probably flawed) impression that her research
isn't the part of her life that's supposed to be important or
impressive.
I don't see this at all in the article. There's just some human
interest content to make her research more approachable.
(DIR) <- back to front page