[HN Gopher] Watch R1 "think" with animated chains of thought
___________________________________________________________________
Watch R1 "think" with animated chains of thought
Author : higuidebot
Score : 123 points
Date : 2025-02-17 16:23 UTC (6 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| ipsum2 wrote:
| It seems kinda silly to use a separate service to generate
| embeddings for t-SNE when you have the embeddings in the model
| already.
| higuidebot wrote:
| Is it generating embeddings or just coordinates? What would be
| a better way?
| gavmor wrote:
| What are embeddings if not "just coordinates"?
| higuidebot wrote:
| Well ... we have to reduce them to a 2D plane to visualize
| them ...
| datameta wrote:
| That just makes them higher order coordinates, no?
| gavmor wrote:
| Higher order, yes, but as these coordinates certainly
| contain less information, it's possible they contain only
| noise.
| mikeshi42 wrote:
| Something needs to generate the document embeddings since the
| LLM itself won't
| qoez wrote:
| Fun experiment but in the back of my mind I suspect this is just
| plotting a random walk.
| vekntksijdhric wrote:
| same, it is unclear what you can get out of this work
| higuidebot wrote:
| Random walk is definitely possible. Also possible that we're
| observing some "search" in the embedding space from an
| initial point. It's hard to tell because the chains are often
| similar lengths, so I don't think it really terminates early.
| It might be interesting to find the closest CoT component to
| the final answer and see how step distance inflects at that
| point
| DiscourseFan wrote:
| Perhaps assign each co-ordinate to musical notation and you
| can get some Scheonberg-esque compositions
| gavmor wrote:
| I was, personally, hoping to see a sort of "spiraling down"
| towards the answer or pathfinding like IDA*[0], but I suppose
| what we're looking at isn't too dissimilar from A* or
| Djikstra's if you squint.
|
| I suspect you recognize dimensionality reduction, but to
| reiterate for my own understanding: t-Distributed Stochastic
| Neighbor Embedding (t-SNE) is one method among a few other,
| (more popular?) ones like Principal Component Analysis (PCA)
| and Uniform Manifold Approximation and Projection (UMAP).
|
| Is t-SNE the most appropriate technique for modeling the
| terrain under a multidimensional "walk"? Possibly a linear
| technique (PCA, LDA, SVD?) or PaCMAP[1], which "dynamically
| employs a particular set of mid-near pairs to capture the
| global structure and then improve the local structure."
| (Qattous H, 2023)
|
| 0. https://qiao.github.io/PathFinding.js/visual/
|
| 1. https://pmc.ncbi.nlm.nih.gov/articles/PMC10756978/
|
| Edit: for reference, the tensor projector:
| https://projector.tensorflow.org/
| jejeyyy77 wrote:
| lol isnt this just plotting noise
| higuidebot wrote:
| Whether or not it's "noise" might depend on if you think Chains
| of Thought are causally relevant? I liked these pieces if you
| want to read more about CoT / O1:
|
| O1 Technical Primer:
| https://www.lesswrong.com/posts/byNYzsfFmb2TpYFPW/o1-a-techn...
|
| Using Search Was a Psyop:
| https://www.interconnects.ai/p/openais-o1-using-search-was-a...
|
| Value Attribution:
| https://www.lesswrong.com/posts/FX5JmftqL2j6K8dn4/shapley-va...
| atorodius wrote:
| Is this using t SNE? Or sth else? I have a feeling similarity is
| not well defined in whatever space this is using. t SNE is
| famously unsuited to plot how "close" two points are. it is for
| clustering
| higuidebot wrote:
| 2D plot is tSNE, consecutive distance comparison is cosine sim
| distance normalized across the chain of thought
| vukadinovic wrote:
| Distances in t-SNE/UMAP don't mean anything. They are clustering
| algorithms
| higuidebot wrote:
| 2D plot is tSNE, consecutive distance comparison is cosine sim
| distance normalized across the chain of thought
| ThouYS wrote:
| does this show anything?
| jurgenaut23 wrote:
| As useless as it gets, surprised that it got to the front page.
| luyu_wu wrote:
| It's a cute little project, arguably far more interesting than
| the political flame wars that make front page.
| higuidebot wrote:
| Thank you for your support Mr. Wu
| higuidebot wrote:
| I too am pleasantly surprised
| ganyu wrote:
| Bear in mind that "any two high-dimensional vectors are almost
| always orthogonal".
| frizkie wrote:
| Is this better rephrased as "any two vectors in a high-
| dimensional space are almost always functionally orthogonal"?
|
| I have mostly a laypersons understanding of this idea but I
| would assume that it would be false to say that they are
| typically _entirely_ orthogonal?
| viraptor wrote:
| https://softwaredoug.com/blog/2022/12/26/surpries-at-hi-
| dime... it's both much more likely to be actually orthogonal
| and almost always very close to orthogonal.
| aithrowawaycomm wrote:
| Yes, one more precise way to phrase this is that the expected
| value of the dot product between two random vectors chosen
| from a vector space tends towards 0 as the dimension tends to
| infinity (I think the scaling is 1/sqrt(dimension)). But the
| probability of drawing two truly orthogonal vectors at random
| (over the reals) is zero - the dot product will be very small
| but nonzero.
|
| That said, for _sparse_ high dimensional datasets, which aren
| 't proper vector spaces, the probability of being truly
| orthogonal can be quite high - e.g. if half your vectors have
| totally disjoint support from the other half then the
| probability is at least 50-50.
|
| Note that ML/LLM practioners use "approximate orthogonality"
| anyway.
| esafak wrote:
| The visualization is useless. IF the 2D embeddings were any
| good they might be useful to R1's developers but still not to
| end users. What am I supposed to with it?
| higuidebot wrote:
| No need to do anything in particular! Perhaps interesting to
| observe
| dehrmann wrote:
| Alternatively, in a high-dimension space, everyone sits in
| their own corner.
| levocardia wrote:
| It would be much more interesting to see PCA (or t-SNE or
| whatever) on the internal representation within the model itself.
| As in the activations of a certain number of layers or neurons,
| as they change from token to token.
|
| I don't think the OpenAI embeddings are necessarily an
| appropriate "map" of the model's internal thoughts. I suppose
| that raises another questions: Do LLMs "think" in language? Or do
| they think in a more abstract space, then translate it to
| language later? My money is on the latter.
| higuidebot wrote:
| Text embeddings are underused WRT model understanding IMO.
| "Interpretability" focuses on more complex tools but perhaps
| misses some of the basics - shouldn't we have some sort of
| visual understanding of model thinking?
| eightysixfour wrote:
| > I suppose that raises another questions: Do LLMs "think" in
| language? Or do they think in a more abstract space, then
| translate it to language later? My money is on the latter.
|
| The processing happens in latent space and then is converted to
| tokens/token space. There is research into reasoning models
| which can spend extra compute in latent space instead of in
| token space: https://arxiv.org/abs/2412.06769
| HarHarVeryFunny wrote:
| I'd have to guess that the "transformations" being made to
| the embeddings at each layer are basically/mostly just adding
| (tagging with) incremental levels of additional
| grammatical/semantic information that has been gleaned by the
| hierarchical pattern matching that is taking place.
|
| At the end of the day our own "thinking" has to be a purely
| mechanical process, and one also based around pattern
| recognition and prediction, but "thinking" seems a bit of a
| loaded term to apply to LLMs given the differences in
| "cognitive architecture", and smacks a bit of anthromorphism.
|
| Reasoning (search-like chained predictions) is more of an
| algorithmic process, but it seems that the "reactive" pass-
| thru predictions of the base LLM are more clearly viewed just
| as pattern recognition and extrapolation/prediction.
|
| Prove me wrong!
| eightysixfour wrote:
| > Prove me wrong!
|
| For future reference, it is hard to parse tone over the
| internet but this "command" read pretty poorly to me. I
| would have preferred if you asked a question or something
| else.
|
| However, assuming best intentions...
|
| > I'd have to guess that the "transformations" being made
| to the embeddings at each layer are basically/mostly just
| adding (tagging with) incremental levels of additional
| grammatical/semantic information that has been gleaned by
| the hierarchical pattern matching that is taking place.
|
| > Reasoning (search-like chained predictions) is more of an
| algorithmic process, but it seems that the "reactive" pass-
| thru predictions of the base LLM are more clearly viewed
| just as pattern recognition and extrapolation/prediction.
|
| I'm having trouble following. Are you saying that:
|
| * The "reactive pass-thru predictions" are just pattern
| matched responses from the training text that come from
| "incremental levels of additional semantic information"
|
| * There is some other algorithmic process which results in
| "search-like chained predictions" from the pattern matched
| responses
|
| * These two capabilities, combined in a single "thing," are
| not analogous to thinking
|
| ?
|
| > At the end of the day our own "thinking" has to be a
| purely mechanical process, and one also based around
| pattern recognition and prediction, but "thinking" seems a
| bit of a loaded term to apply to LLMs given the differences
| in "cognitive architecture", and smacks a bit of
| anthromorphism.
|
| You can pick whatever term you like. What we seem to have
| is a system which can, through the embedded patterns of
| language, create a recursive search through a problem space
| and try and solve it by exploring plausible answers. If my
| dog came up with a hypothesis based on patterns it had
| previously observed, considered that hypothesis, discarded
| it, and then came up with a new hypothesis, I'd say it was
| thinking.
|
| There are clear gaps between where we are and human
| capabilities especially as it relates to memory, in-context
| learning, and maintaining coherence over many iterations
| (well, some humans), but (to me) one of two things is
| probably true:
|
| 1. Models are doing something analogous to thinking that we
| don't understand.
|
| 2. Thinking is just a predict-act-evaluate loop with
| pattern matching to generate plausible predictions.
|
| I lean towards the second. That's not to ignore the
| complexity of the human brain, it is just that the core
| process seems quite clear in the abstract to me via both
| observation and introspection. What can "thinking" (as you
| define it) do that is beyond these capabilities?
| antirez wrote:
| The relation among the internal model representations inside its
| latent space and the embedding of the CoT compressed with a text
| embedding model is, more or less, minimal. Then we take this and
| map it to a 2D space, which captures more or less nothing of the
| original dimentionality and meaning. That's basically plotting
| random points.
| higuidebot wrote:
| Potentially it's useful to understand a model "on its own
| terms" via its observable outputs.
|
| >The relation among the internal model representations inside
| its latent space and the embedding of the CoT compressed with a
| text embedding model is, more or less, minimal.
|
| This may or may not be correct but one way to find out is by
| taking a look!
| stared wrote:
| While I like the idea of measuring subsequent steps, this kind of
| approach of using embeddings is the reason why I wrote: "Don't
| use cosine distance carelessly"
| (https://p.migdal.pl/blog/2025/01/dont-use-cosine-similarity).
|
| In this case, cosine distance one would be in a case when it
| repeats word-by-word. It is not even a "similar thought" but some
| sort of LLM's OCD.
|
| For anything else... cosine similarity says little. Sometimes,
| two steps can have opposite conclusions but have very high cosine
| similarity. In another case, it can just expand on the same
| solution but use different vocabulary or look from another angle.
|
| A more robust approach would be to give the whole reasoning to an
| LLM and ask to grade according to a given criterion (e.g. "grade
| insight in each step, from 1 to 5").
| higuidebot wrote:
| Well, you are certainly correct about how cosine sim would
| apply to the text embeddings, but I disagree about how useful
| that application is to our understanding of the model.
|
| > In this case, cosine distance one would be in a case when it
| repeats word-by-word. It is not even a "similar thought" but
| some sort of LLM's OCD.
|
| Observing that would be helpful in our understanding of the
| model!
|
| > For anything else... cosine similarity says little.
| Sometimes, two steps can have opposite consultation, but they
| have very high cosine similarity. In another case, it can just
| expand on the same solution but use different vocabulary or
| look from another angle.
|
| Yes, that would be good to observe also! But here I think you
| undervalue the specificity of the OAI embeddings model, which
| has 3072 dimensions. That's quite a lot of information being
| captured.
|
| > A more robust approach would be to give the whole reasoning
| to an LLM and ask to grade according to a given criterion (e.g.
| "grade insight in each step, from 1 to 5").
|
| Totally disagree here, using embeddings is much more reliable /
| robust, I wouldn't put much stock in LLM output, too much going
| on
| vale95ntino wrote:
| Pretty cool. Funnily enough I made something similar this weekend
| that converts CoT to Graphs/Trees of Thoughts with an LLM. It
| kind of allows to see when the LRM/LLM changes direction or when
| it finds a path that it wants to follow.
|
| Link: https://github.com/vale95ntino/cot2tot
| higuidebot wrote:
| Very cool!
| jumploops wrote:
| I have the same concern as other commenters (using a separate
| embedding model, utility of cosine similarity, etc.)
|
| BUT this could "seed" a really neat loading graphic for reasoning
| models, beyond seeing the thinking steps.
| higuidebot wrote:
| Ha, if steps have consistent distances you could take the
| average distance at step X and generate a step of that length
| in some direction and be ~approximately correct regardless of
| the actual value
| dan_voronov wrote:
| Why not in a three-dimensional space?
| KTibow wrote:
| I wonder if you could turn it into a graph by adding connections
| between the two most similar entries until everything is
| connected.
| KTibow wrote:
| I tried this: https://github.com/KTibow/thoughtgraph
___________________________________________________________________
(page generated 2025-02-17 23:00 UTC)