[HN Gopher] Cognitive Behaviors That Enable Self-Improving Reaso...
___________________________________________________________________
Cognitive Behaviors That Enable Self-Improving Reasoners
Author : delifue
Score : 240 points
Date : 2025-03-06 01:33 UTC (21 hours ago)
(HTM) web link (arxiv.org)
(TXT) w3m dump (arxiv.org)
| robocat wrote:
| How much has our knowledge of AI training techniques helped to
| discover how to train people to think better?
| sanxiyn wrote:
| So far, I don't think we found anything interesting, yet.
| vasco wrote:
| We've had knowledge of how to eat better to not get extreme
| scenarios like obesity and look at the effectiveness of that.
| Until you have a pill that makes you think better only the
| motivated will do it, and in this case the motivated could
| already do it.
| PeterStuer wrote:
| You seem to imply the 'motivated' can not improve.
|
| I'd say the motivated often reap the rewards of innovations
| more so than the average, as they were pushing the boundaries
| in the first place.
|
| Having a dishwasher or a robot vacuum does not make me lazy.
| It allows me to do more productive things.
| vasco wrote:
| I guess I was focusing on the most obvious cases. If a pill
| makes everyone skinny you'll notice much more that there's
| no more huge people than that skinny people's body fat went
| from an average of 13% to 11% or some such. I do agree the
| motivated have more likelihood to improve, but on a
| societal level I tend to think first of raising the bottom
| of the group. I agree with your points.
| atwrk wrote:
| As someone with an educational background I actually often ask
| myself the opposite: Why don't AI techniques almost never seem
| to use the knowledge we have about human learning to train
| better AI?
| HPsquared wrote:
| Maybe an area worth exploring, if you think there's something
| to it!
| owenpalmer wrote:
| > four key cognitive behaviors -- verification, backtracking,
| subgoal setting, and backward chaining -- that both expert human
| problem solvers and successful language models employ.
|
| As we make AI better, perhaps we'll inadvertently find ways to
| make HI (human intelligence) better too.
|
| I had a personal experience with this when I was studying for an
| exam recently. As I read over practice questions, I spoke aloud,
| replicating the reasoning methods/personality of Deepseek R1. By
| spending a lot of time reading long verbose R1 outputs, I've
| essentially fine-tuned my brain for reasoning tasks. I believe
| this method contributed to my excellent score on that exam.
| jdpage wrote:
| This is a well-known approach: verbalizing your thought process
| (either by speaking aloud, or by writing) is something that's
| long established as a good tactic for making sure that you're
| actually thinking through something, rather than glossing over
| it. Ironically, I've seen people bemoaning that use of AI will
| _rob_ people of that.
|
| I agree that there's potential here, though, and do genuinely
| hope that we find ways to make human intelligence better as
| we're going about AI research. Even pessimistically, I think
| we'll at least surface approaches that people use without
| thinking about, which is on its own a good thing, because once
| you know you're doing something, it becomes a lot easier to
| train yourself to do it better.
| ilrwbwrkhv wrote:
| This is one of the secrets of the top British universities.
| They do a lot of debating in small groups. Even their papers
| are read out loud
| crooked-v wrote:
| > Ironically, I've seen people bemoaning that use of AI will
| rob people of that.
|
| There's that quote from Socrates, recorded by Plato:
|
| > For this invention will produce forgetfulness in the minds
| of those who learn to use it, because they will not practice
| their memory. Their trust in writing, produced by external
| characters which are no part of themselves, will discourage
| the use of their own memory within them.
| neom wrote:
| Pretty much every single spiritual philosopher has said
| some version of that (I'm writing a book on this subject
| right now, heh):
|
| The Buddha (from the Pali Canon, Vinaya Pitaka, Cullavagga
| 10:4):
|
| "Writing is like a drug that weakens memory."
|
| and: "Do not go upon what has been acquired by repeated
| hearing; nor upon tradition; nor upon rumor... But when you
| yourselves know: 'These things are good; these things are
| not blameable; undertaken and observed, these things lead
| to benefit and happiness,' enter on and abide in them."
|
| Confucius (Analects 2:15):
|
| "Learning without thought is labor lost; thought without
| learning is perilous."
|
| Lao Tzu (Tao Te Ching, Chapter 48):
|
| "In the pursuit of learning, every day something is
| acquired. In the pursuit of Tao, every day something is
| dropped."
|
| Jesus (Matthew 16:26):
|
| "For what shall it profit a man, if he shall gain the whole
| world, and lose his own soul?"
|
| Muhammad (Hadith, Sahih Muslim):
|
| "The worst vessel to fill is the stomach; sufficient for
| the child of Adam are a few morsels to keep his back
| straight. If he must fill it, then one-third food, one-
| third drink, and one-third air."
|
| (This Hadith symbolically warns against excessive reliance
| on external consumption diminishing spiritual clarity and
| internal balance.)
|
| Rumi (Masnavi):
|
| "These outward forms are but dust and air; Seek the reality
| beyond appearance and form."
|
| Krishna (Bhagavad Gita, 2:42-43):
|
| "Those who are attached to pleasure and power, whose minds
| are drawn away by such things, have no capacity for
| absorption into higher states of awareness."
| HPsquared wrote:
| Sometimes it's good to forget something, commit to
| written record and let it go. People can carry too much
| old stuff around in their heads, and it can become
| burdensome.
|
| Even things like confession, or therapy, leverage this -
| people letting go of bad things that are hanging around
| in their memory.
| neom wrote:
| A lot of traditions would strongly disagree with that,
| myself included. I would imagine you would find a lot of
| Gnostics who disagree, a lot of "witches" who have been
| literally burned over the years, and also the aboriginal
| people of Australia. Well, you wouldn't find so many of
| them, because those lines of thought have literally been
| beat out of our human societies in the name of progress.
| If it's not worthy of being in the oral tradition, it's
| not worthy of the society, it's not worthy thought.
| (unworthy thought does not need to be rigorously engaged
| with.)
|
| Also remember, your conclusion itself is "the devil" -
| the trap of the analytical mind. :) You will likely do
| everything you can do avoid the fact that you may be
| disagreeing directly with the word of the creator as
| given via various prophets, if you go back to the
| sources, the command is quite clear, however humans will
| interpret it: because the command is too simple and
| terrifying to adhere to. It seems impossible to us that
| we should indeed, be doing nothing but living in nature
| in a state of oral tradition and anything outside of that
| is an unintended state, trusting that energy cannot be
| destroyed and we are nothing but energy. I don't
| particularly like it either tbh, hence I'm writing a book
| about it.
| tankenmate wrote:
| And on the flipside, Guang Ji Bu Ru Dan Mo [0], lit. "a
| good memory is not as good as pale ink", which is these
| days more commonly translated as "the faintest ink is more
| powerful than the strongest memory".
|
| [0] "A Record of Learning about Government" [Zheng Xue Lu ]
| Magistrates handbooks, Author Zheng Duan [Zheng Duan ]
| (compilation), Early Qing Dynasty (1644-1796)
| rTX5CMRXIfFG wrote:
| Y'all can throw pithy sayings at each other all you like
| but memory is not the same as understanding, and AI does
| offer plenty of opportunity for humans to cognitively
| disengage. Doesn't necessarily mean most people will, but
| it's very likely that most people will.
| rvense wrote:
| Especially if we're more concrete: If your job is, say,
| in administration and what the machine answers is correct
| enough that in in 8 out of 10 cases you can basically
| copy-paste it, I'd say it's extremely likely that it's
| going to increase the amount of errors made.
| mjan22640 wrote:
| In the setting you describe I think it will _reduce_ the
| errors to 20%
| psychoslave wrote:
| No, since the setting is not specifying the initial rate,
| it might as well increase or staying stable at 20%.
|
| But there are other factors, like, is the amount of
| outcomes done also changing, thus affecting the absolute
| number of errors?
|
| Also, does the side effect of disengage the person in
| most cases means it has side effects like not paying the
| same attention to what would stand out as a big issue
| that needs more attention and consideration than business
| as usual?
|
| And so on
| devmor wrote:
| It's an interesting thing to think about. From the way
| it's _talked about_ , I would predict that AI will enable
| people who are more cognitively inclined to think in more
| complex and refined ways; while other people that over-
| rely on the results would be the ones that decline.
|
| However, research[1] suggests that relying on AI tools
| degrades reasoning and cognitive ability regardless of
| your cognitive ability, and may even cause users to stop
| making their own choices[2].
|
| 1. https://www.404media.co/microsoft-study-finds-ai-
| makes-human...
|
| 2. https://www.nature.com/articles/s41599-023-01787-8
| FuriouslyAdrift wrote:
| That's more a comment on bureaucracy than it is about
| memory
| jbreckmckye wrote:
| Classical era philosophers weren't completely wrong about
| this. They lived in a more oral literary culture where
| performers could recite entire works from memory.
|
| I don't think anyone today could recite _Beowulf_ from
| heart. But 1500 years ago that 's exactly how it was
| enjoyed.
| psychoslave wrote:
| Maybe not Beowulf, though it's not to exclude either, by
| I have no doubt that some people out there can do that
| for some work.
|
| Look at these people that will declaim Pi digits, just
| because.
|
| Also there are difference in education over different
| cultural era, not only through time but also space. I
| heard that India for example value more repetition, where
| western culture is more in love with innovation. These
| are of course nothing like exclusive tendencies.
|
| Now if you look at antic Greece, it's certainly not like
| everyone would be able to restitute Homer word for word.
| Actually it's easy to forget how divided in term of
| linguistic and social classes this societies where, and
| focus solely on the most renowned figures as if they
| where all part of a tight social group full of solidarity
| and genius. Actually even a guy like Hippias of Elis can
| be both depicted by plato as exhibiting all the
| tremendous admirable feats of the time, including
| mastering the art of mnemonics, and yet turned into a
| clueless bragger that isn't even able to recognize that
| he just doesn't know how to define beauty.
| FuriouslyAdrift wrote:
| People memorize the entire Quran as a religeous
| obligation...
|
| https://en.wikipedia.org/wiki/Hafiz_(Quran)
| pessimizer wrote:
| I'm pretty sure that "recit[ing] entire works from
| memory" was an example of an amazing memory feat that was
| far more common in the past, before writing was as
| common. I do not think that it was a recommendation that
| education should consist of memorizing literary works.
| ducttapecrown wrote:
| I wonder if the number of people who could recite works
| from memory has substantially changed: while surely the
| prevalence of such people decreased, world population
| ballooned! Today we have people that can recite many
| digits of pi, for example.
| 5ahsdGAh wrote:
| I wonder what Socrates would say if your thinking process
| depends on GPUs owned by oligarchs and all your attempts at
| solving a problem are tracked.
|
| What would he say if the collective IQ drops by 30 points
| in case of a power outage?
|
| What would he say if people need a subscription in order to
| "think"?
| vunderba wrote:
| Thinking out loud is an age old practice and is the equivalent
| of "rubber ducking" to yourself.
|
| As someone who comes from a long ancestral line of people who
| talk to themselves while reasoning through problems - it would
| occasionally prove to be a minor handicap during proctored
| exams, as internal monologue isn't really the same thing.
| forthac wrote:
| From what I have seen from split-brain experiments, I am of
| the belief that by vocalizing our thoughts, we are more fully
| engaging both hemispheres of our brain through the auditory
| pathway in addition to the Corpus Callosum.
| rzzzt wrote:
| Pictures tell me that the language area is dominant in a
| single hemisphere, mostly the left side, with motoric stuff
| and thinking about words in the front (Broca's area) and
| hearing in the back (Wernicke's area).
|
| So you may have to use the SLI bridge again just to make
| sense of what the other side is hearing.
| rendall wrote:
| Me, working through a problem: "So, that means..."
|
| Girlfriend, coming in from outside: "Who are you talking to?"
|
| Me: "I talk to myself. You know that."
|
| Gf: "Oh right. You also whisper to yourself, which is scary."
|
| Me: "Scary?"
|
| Gf: "It sounds demonic."
|
| Which, to be fair... Evidently, my internal monologuing gets
| quite a bit vocal even with other people around.
| vijucat wrote:
| Maybe demons were those who learned to cogitate better and
| ended up being the Ted Kacynzkis of their ilk :)
| HPsquared wrote:
| It's your reasoning daemon running in the background.
| burningChrome wrote:
| When I was working at a Bike Shop, I was standing on one
| side of the display, talking about what I was going to do
| and how I was going to do it. It was a very in depth
| external monologue (I have both like yourself). As I was
| coming around the other side of the display, the owner
| happened to be standing there and said, "Wow, that was some
| kind of a conversation you just had with yourself. I hoped
| you were able to solve whatever problem you were
| discussing." He had a big grin and we both laughed about
| it.
|
| He told me later he too does a lot of internal monologue
| for stuff as well and was told by some super successful
| businessman that this is a good thing and a hallmark of
| successful people so don't be discouraged by it.
| yapyap wrote:
| ... some of yall
| thesz wrote:
| Exam is interpolation.
|
| Research is extrapolation.
|
| Neural networks are interpolators, they are notoriously bad at
| extrapolation. For simple example, look at pendigits data set
| [1]. The test part of pendigits is taken from different
| "writers" than train part and neural networks aren't that good
| at it.
|
| [1]
| https://archive.ics.uci.edu/dataset/81/pen+based+recognition...
|
| Humans do extrapolation all the time.
| arjonagelhout wrote:
| I use this method too for programming problems I would normally
| procrastinate on and offload to subconscious thinking.
|
| Actually writing out all thinking steps helps with ironing out
| some wrong steps in my reasoning or going in circles due to
| having limited working memory.
|
| I started doing this more rigorously after seeing how reasoning
| based AI does reasoning, because it seemed like a useful
| thinking technique.
|
| These reasoning AI models help me think on a meta level about
| my own thinking and shows me tools I can use to improve it.
|
| Great to see that I'm not alone in this!
| EncomLab wrote:
| Sounds like you rediscovered the long held practice of
| "Rubber Ducking".
| floatrock wrote:
| eh, writing it out is closer to "proto design document" or
| just plain ol "whiteboarding"
| resize2996 wrote:
| the only reason I haven't written a design doc or busted
| out the felt tip for my rubber duck is because it can't
| read.
| ninetyninenine wrote:
| But now the rubber duck can talk back and also on occasion
| hallucinate and lie to you to confirm your delusions.
| arjonagelhout wrote:
| Haha, that's a good point. I think the main change from
| using these (reasoning) models is that I'm more cognizant
| of my thinking process, rather than there being a novel
| technique.
| mattgreenrocks wrote:
| For me, I think this approach works because I can commit the
| current thoughts to some type of external (to my brain)
| storage, freeing up space to think about how to further
| subdivide those tasks.
|
| In general, this is very helpful for when your executive
| function feels taxed, as it has the effect of coaching
| yourself.
| yapyap wrote:
| My god, that's literally just thinking aloud, doing the rubber
| duck thing.
|
| You are not "emulating R1", you are talking to yourself to make
| sure you understand the concept.
|
| Which is fine but don't act like AI is making this part of life
| better in any way with this example. Nonsense
| owenpalmer wrote:
| I think you're missing a key point, which is that by
| extensively reading the R1 outputs, I'm able to observe how
| R1 thinks about things, which I can then replicate.
|
| There are good ways and bad ways to think aloud, R1 just gave
| me a large set of examples of doing it the "good" way.
|
| It's uncommon to read hundreds of paragraphs of a smart
| person's internal reasoning process. Usually we're only able
| to read the final results of their thoughts.
| nico wrote:
| Super interesting
|
| > As I read over practice questions, I spoke aloud
|
| This is also something that's expected of the applicant in
| technical interviews
|
| The interviewers want to hear the applicants thought process
| and how they develop a strategy to solve the problems presented
| to them as they work them out
| Cheer2171 wrote:
| You computer scientists cosplaying as cognitive scientists
| really never took single psychology class, did you?
| astrange wrote:
| I believe the quote here is "Every time I fire a linguist,
| the performance of the system goes up."
| calf wrote:
| I'm slightly terrified I'm mot consciously trying any of
| these metacognitive things to "self improve" myself.
| swyx wrote:
| do you want to try that again but being more constructive?
| calf wrote:
| Those four parts sound like one unified, cognitive algorithm --
| having an ontology of the problem by breaking it into subgoals;
| checking your work properly; thinking backwards to debug a
| mistake and retrying; and thinking ahead and reasoning backward
| from the end result. It's all just one algorithm for solving
| hard problems. A skill that can be practiced, and then it
| builds on itself.
| idiotsecant wrote:
| I sometimes see these reddit threads of people talking about the
| experience of having an internal monologue. I have no such
| monologue, at least not one that is accessible to the part of my
| mind that calls itself 'me', but I have often wondered if that
| monologue is something like a 'chain of thought'. I feel like
| maybe without access to that 'idea feed' maybe my planning and
| executive functioning is less effective than some other people. I
| do find myself quite more effective with those sort of tasks when
| I do a little 'chain of thought' notepad.
|
| I also suspect I spend less time ruminating and second-guessing
| myself and other anxious behaviours that I imagine would come
| with having someone talking in your ear all day, but that's
| probably off topic.
| kla-s wrote:
| Genuine question, how does multi step reasoning work for you
| then? Like eg if you have some math problem that's trivially to
| solve individually but needs multiple steps, lets say 16 * 3 +
| 5? How does 16 * 3 = 48 land in some 'register' of your brain
| (short term memory), so that you can then add 5 to get to 53?
| Maybe 16 * 3 + 5 is to easy for you and you'll just 'see' it
| but the question still stands, just choose a more complex
| problem.
|
| Isn't the same meta process at play when thinking about more
| fuzzy topics?
| cgriswald wrote:
| I believe I have an internal narrator but I'm not certain
| exactly what others mean by that so I don't know for sure.
|
| However the way I think about math is different than the way
| I plan my day or other things. In my case, it is very much
| like I have registers that would hold the result of 16 x 3 in
| it so I can add the 5 to it later. I have a certain number of
| registers and with effort like repeating what Ive already
| solved I could temporarily create more.
|
| It also feels somewhat physical, as if the register is an
| actual box or has a "location" or like I've put the answer
| down on the desk like a part of something I'm building.
| Perhaps not coincidentally I am one of the many people who
| have a "calendar shape" for the months.
| crooked-v wrote:
| Not that poster, but for me it's directly manipulating
| numbers (for example, "16x3 + 5" turns into "10x3 + 6x3 + 5"
| into "30 + 18 + 5" into "30 + 10 + 8 + 5" into "40 + 13" into
| "53"). There's no language involved, though in some cases I
| might use some spatial reasoning by doing something like
| associating given chunks of an equation with different
| fingers.
| leroyrandolph wrote:
| I don't really follow. Say, during the in-between step
| "10x3 + 6x3 + 5", how do you store and cognize the
| individual numeric and operator elements?
|
| Surely, even if the arithmetics can be simplified and
| "lookup-table'd", you are still aware of the numbers in
| Arabic form or whatever equivalent you're using, right? Or
| do you somehow have 53 individual blobs swirling inside
| your consciousness?
| wizzwizz4 wrote:
| Language (human) and language (theoretical computer
| science) are different concepts.
| idiotsecant wrote:
| Not poster you replied to but it sounds like we might
| have a similar internal model.
|
| I store numbers as pictures of numbers, or a geometric
| representation depending on how big or precise the number
| is.
|
| Are you saying when you think of the concept of 'twelve
| plus twelve' you have the equivalent of someone in your
| head saying 'hmm, well twelve is 2 more than ten, so if I
| add up ten and ten and two and two I get twenty four?'
|
| That's wild if so.
|
| For your reference, I would follow the procedure above
| approximately, but visually with numbers that just do the
| thing that feels right. I think under the hood we're
| probably doing the same thing, just with a different
| interface layer
| jacomoRodriguez wrote:
| I can just speak for me, obviously, but yes, that is
| what's happening. But it's not someone, it is more like
| me explaining / telling it to myself. Depending on the
| complexity this can be more or less verbal - the more
| complex, the less verbal I would say.
| t-3 wrote:
| I speak out loud or write on paper, or just do it a tiny bit
| slower and sometimes have to redo steps when I forget a
| result.
| atwrk wrote:
| It's also very probable that the verbalization the majority
| does internally is just that - a verbalization of the actual
| underlying thought process. That is what much of current
| cognitive linguistics points to as far as I have understood.
|
| (Also a reason why I'm very sceptical that the current LLM
| approach will eventually lead to AGI, BTW)
| idiotsecant wrote:
| I think you're probably right that the verbalization is the
| 'interface layer' but why does that mean LLMs can't
| approach AGI? They also only use words as an 'interface'
| layer. Underlying weights are vectors in an abstract space.
| RGamma wrote:
| So you cannot think in language? Sounds kinda scary to be
| honest.
| kennysoona wrote:
| You never form thoughts in your mind in a linguistic way? Can
| you read a sentence and be aware of it as a sentence in your
| mind, or are you unable to do that?
|
| I don't doubt you or anything like that, just very curious. As
| someone with a very strong internal monologue, it's hard for me
| to imagine not having one.
| PeterStuer wrote:
| For me it has linguistics components for sure, but it is many
| in parallel and a lot less 'linear'.
|
| Where inner language most certainly comes into play is in the
| 'output' phase, be it spoken or written, as serialization is
| required there, but to be honest that often feels like a
| projection or even a reconstruction with an inherrent sense
| of loss as the inner is so much richer and nuanced.
|
| That is not to say linearization has no merits. Even if it
| loses so much it forces consistency and rigor in the lower
| dimensional reasoning.
| kennysoona wrote:
| If you were in a debate, and had to think through your
| reasoning and arguments, refining your argument, not the
| words you want to say but the substance of what you will
| say, would there be an inner voice in that situation?
| HPsquared wrote:
| Non-chain of thought models just "come up with" words to
| say. It's probably a bit like that.
| TeMPOraL wrote:
| So do chain-of-thoughts. The "come up with words to say"
| aka. selecting the best-fitting token to continue the
| sequence, is more-less exactly how my inner monologue
| feels. It sits at the boundary of the conscious and the
| unconscious, and from the point of view of my own
| introspection, the words just _appear_ , complete with an
| emotional undertone of whether they "feel right" and how
| much.
| idiotsecant wrote:
| In my case, no. There is no conscious 'refinement' of
| speech options like a Disco Elysium scene or something
| like that. I mostly just say what I think. There _is_
| definitely some impulse control type function to keep me
| from saying, for example, offensive things, but it 's not
| in the form of myself telling myself 'dont say this
| thing'. It's usually in the form of an intrusive memory
| of a facial expression or negative emotion that I
| associate with that branch of conversation, if that makes
| sense? Same end, I think, just without a verbal
| component.
|
| I have moderate inattentive type ADHD that manifests as
| me being hyper focused on specific sometimes minor things
| and failing to effectively plan larger picture things and
| often results in poor executive function outcomes. Maybe
| that's part of it.
| PeterStuer wrote:
| Best I can put it not one but many (not voices but)
| narratives at once, and shared with stuff that is not
| linguistic, but more like analogue waves of concept
| activation.
|
| In a fast debate you have no time for inner single
| dimension explicit reasoning. It flows more directly.
|
| Compare it to running on uneven terrain. Somehow
| decisions are being made where to put the next foot, but
| most of the time not after an introspective deliberation
| (although these would occur sometimes in e.g. very tricky
| rock climbing moves). The steering of where to go occurs
| at a much higher level, and the steps flow from higher
| level coarse grained directions.
|
| Now in hindsight you can go back and analyze a recording
| or memory of the debate and see how it all makes sense
| _as if_ it was rationally reasoned at every step of the
| way, but _during_ the debate it most certainly is not an
| "inner voice", homunculus reasoning before consciously
| uttering the next phrase.
| idiotsecant wrote:
| No. I don't think in 'first person' words at all. I might
| consciously compose a phrase if I'm doing something like
| writing a poem, which is more akin to arranging a puzzle or
| something or I might recall words of a conversation someone
| said to me and i _do_ think of song lyrics if I have a song
| in my head, but there 's no voice in my head and it's
| absolutely baffling to me to imagine otherwise, as I imagine
| it is for other people to imagine my situation.
|
| When I read a sentence in a book I don't hear any kind of
| narration or anything, but I do assemble a 'scene' of images,
| sounds, facial expressions, motions, etc. not like a movie,
| but more like a series of small related ideas if that makes
| sense?
|
| I find that I understand dialogue and characters in books
| _much_ better when I listen to an audiobook than when I read,
| not sure if that 's related or not.
|
| I am a relatively intelligent successful professional, but I
| wonder sometimes if I am missing some processing hardware
| other people have access to.
| kennysoona wrote:
| Not missing hardware, maybe just different. Those of us
| with an internal monologue will still say there is thinking
| that happens before that is voiced, so maybe you're still
| thinking, maybe the same hardware for an internal monologue
| is there, I guess, but it just isn't being voiced maybe? It
| doesn't sound like you are missing anything important,
| given you are still able to reason just fine.
|
| Anyway thank you for answering!
| ninetyninenine wrote:
| I do have an internal monologue. I can also think in pictures
| and I can also think in terms of neither, just pure raw
| thought.
|
| I would say most people are like me. They have 3 modes of
| thinking and they probably have a primary mode which they
| favor. I favor none and go into all 3 depending on whether I'm
| reading, writing or doing something else.
|
| The second bigger group has only one primary mode of thinking.
| The internal monologue. They can only think in terms of an
| inner voice and this inner voice is so powerful I often
| encountered people who think this inner voice is the definition
| of thought. They assumed thinking was COT.
|
| The even rarer versions you get people who assign colors to
| numbers or people who can't even perceive to think in pictures.
| You're the first person I've encountered who can't even have an
| internal monologue.
| idiotsecant wrote:
| I think you'd be surprised. I never knew that internal
| monologues were a thing until there was a HN thread about it
| and a lot of people had them and a lot of people didn't.
|
| I always thought it was something that we did in TV shows or
| books to give you a sense of what a character was feeling, I
| didn't know this was an actual literal experience people had.
|
| I can certainly have an internal monologue, in the way that
| you could put on a puppet show. I can conciously think to
| myself 'self, this is self. Clean your car out' I can form
| the feeling of those words in my head. But there's nobody
| 'saying' them if that makes sense. I'm playing back a design
| of my conscious self.
| nickpsecurity wrote:
| "models primed with incorrect solutions containing proper
| reasoning patterns achieve comparable performance to those
| trained on correct solutions"
|
| One of the parts most worth a replication study.
| meindnoch wrote:
| At this point I can't tell from the title whether it's a self-
| help psychology fad or an LLM paper.
| Etheryte wrote:
| Soon enough, they'll have LLMs that reason purely from the
| first principles of The Subtle Art of Not Giving a Fuck.
| falcor84 wrote:
| That would actually be really useful
| miksik wrote:
| > four key cognitive behaviors -- verification, backtracking,
| subgoal setting, and backward chaining -- that both expert human
| problem solvers and successful language models employ.
|
| Based on what have they claimed that such methods are used by
| expert human problem solvers?
| th0ma5 wrote:
| Once you suspend disbelief of AI all of this pseudoscience
| becomes just as plausible.
| glass_door wrote:
| Does this also mean giving better system prompts that encourage
| this behaviour also substantially help?
| astrange wrote:
| In my experience models aren't very good at following such
| prompts. Smart "non-reasoning" models like Claude 3.5 could,
| but would generate so much text when thinking they ran out of
| context.
| spwa4 wrote:
| True, but a problem is that self-improving AI leads to a somewhat
| troubling mode of thinking. AIs switch to an internal babbling
| type language that makes no sense but clearly still conveys
| meaning to the AIs, then think in that language (if it's a
| language, though not sure what else it could be) and then produce
| correct results.
|
| Worse, when you use multiple agents to get AI LLMs talking to one
| another, all AI agents switch to this internal language and they
| make progress despite no human understanding what hell is
| happening. This seems very bad.
|
| Illustration:
|
| > How many r in strawberry?
|
| I'm asked how many r in strawberry. I can just spell the word and
| a;dklsjaw; a;ewjraqwpeouypaads;lq qepwiouryaqeopw
| qewrpoiuyoiauysdqw145124rfa.nkjlwh ;45a8345a894ya4a
| q4p58q45jaq;lkjas;dlfkja;j
|
| <answer>There are 3 (three) r's in strawberry</answer>
| hnuser123456 wrote:
| They are not going to invent a new language by themselves, they
| by definition can't even "think" in terms of languages they
| haven't seen. It does not occur to them that the language they
| use may be suboptimal. And surely, any better ways of thinking
| can still be described in English. It seems more likely there
| will be a gradual transition from us teaching LLMs how to
| reason, to LLMs being able to actually gobble and process
| enough data to learn more effective ways to reason, which it
| can then "teach" us. But that's just the LLM reflecting the way
| it was trained and aligned.
| theptip wrote:
| I've heard this described as talking in "Neuralese". It seems
| plausible that this will be the most dense language for model-
| internal dialog (or presumably inter-LLM dialog assuming they
| share the same weights).
|
| You will penalize this inasmuch as your alignment strategy
| depends on Deliberative Alignment. But at some point I assume
| that will come with a real capability cost as Neuralese can be
| more conceptually dense.
| kittikitti wrote:
| ``think''
|
| In the abstract they use different characters for double quotes
| here.
___________________________________________________________________
(page generated 2025-03-06 23:01 UTC)