[HN Gopher] The Fractured Entangled Representation Hypothesis
___________________________________________________________________
The Fractured Entangled Representation Hypothesis
Author : akarshkumar0101
Score : 45 points
Date : 2025-05-20 15:56 UTC (7 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| akarshkumar0101 wrote:
| Tweet: https://x.com/kenneth0stanley/status/1924650124829196370
| Arxiv: https://arxiv.org/abs/2505.11581
| pvg wrote:
| Sounds like you're one of the co-authors? Probably worth
| mentioning if the case so people know they can discuss the work
| with one of the work-doers.
| akarshkumar0101 wrote:
| I mentioned that in the original post, but I don't see that
| text here anymore (thats why I added links via comment)... I
| am new to hackernews
| messe wrote:
| I believe they just mean that you should edit the comment
| where you added the links to mention that you are the
| author, to add that additional context.
| pvg wrote:
| I just meant 'it's good for people to know one of the
| authors is in the thread because it makes for more
| interesting conversation'. Clearly did not figure out how
| to do that without starting a bunch of meta!
| macintux wrote:
| I believe this could (or should) have been a Show HN, which
| would have allowed you to include explanatory text. See the
| top of this page for the rules.
|
| https://news.ycombinator.com/show
|
| Welcome to the site. There are a lot of features which are
| less obvious, which you'll discover over time.
| pvg wrote:
| Reading material usually can't be a Show HN but you can
| just post your work without that and say you're involved.
| macintux wrote:
| The repo includes runnable code.
|
| > Show HN is for something you've made that other people
| can play with... On topic: things people can run on their
| computers or hold in their hands
| pvg wrote:
| A lot of writing includes runnable code and isn't a Show
| HN. It's a comparatively narrow category.
| ipunchghosts wrote:
| I am interested in doing research like this. Is there any way I
| can be a part of it or a similar group? I have been fighting
| for funding from DoD for many years but to no avail so I
| largely have to do this research on my own time or solve my
| current grant's problems so that i can work on this. In my
| mind, this kind of research is the most interesting and
| important right now in the deep learning field. I am a hard
| worker and a high-throughput thinking... how can i get
| connected to otherwise with a similar mindset?
| scarmig wrote:
| Did you investigate other search processes besides SGD? I'm
| thinking of those often termed "biologically plausible" (e.g.
| forward-forward, FA). Are their internal representations closer
| to the fractured or unified representations?
| timewizard wrote:
| > Much of the excitement in modern AI is driven by the
| observation that scaling up existing systems leads to better
| performance.
|
| Scaling up almost always leads to better performance. If you're
| only getting linear gains though then there is absolutely nothing
| to be excited about. You are in a dead end.
| goldemerald wrote:
| This is an interesting line of research but missing a key aspect:
| there's (almost) no references to the linear representation
| hypothesis. Much work on neural network interpretability lately
| has shown individual neurons are polysemantic, and therefore
| practically useless for explainability. My hypothesis is fitting
| linear probes (or a sparse autoencoder) would reveal linearly
| semantic attributes.
|
| It is unfortunate because they briefly mention Neel Nanda's
| Othello experiments, but not the wide array of experiments like
| the NeurIPS Oral "Linear Representation Hypothesis in Language
| Models" or even golden gate Claude.
| ipunchghosts wrote:
| Is what your saying imply that there is a rotation matrix you
| can apply to each activation output to make it less entangled?
| goldemerald wrote:
| Not quite. For an underlying semantic concept (e.g., smiling
| face), you can go from a basis vector [0,1,0,...,0] to the
| original latent space via a single rotation. You could then
| induce said concept by manipulating the original latent point
| by traversing along that linear direction.
| ipunchghosts wrote:
| I think we are saying the same thing. Please correct me
| though where I am wrong. You could look at the maps in some
| way but instead of the basis being one hot dimensions (the
| standard basis), it could be rotated.
| akarshkumar0101 wrote:
| We mention this issue exactly in the fourth paragraph in
| Section 4 and in Appendix F!
| akarshkumar0101 wrote:
| We mention this issue exactly in the fourth paragraph in
| Section 4 and in Appendix F!
| goldemerald wrote:
| That is addressing the incomprehensibility of PCA and
| applying a transformation to the entire latent space. I've
| never found PCA to be meaningful for deep learning. As far as
| I can tell, polysemous issue with neurons cannot be addressed
| with a single linear transformation. There is no sparse
| analysis (via linear probes or SAEs) and hence the
| unaddressed issue.
| ipunchghosts wrote:
| I am glad they evaluated this hypothesis using weight decay which
| is primarily thought of to induce a structured representation. My
| first thought was that the entire paper was useless if they
| didn't do this experiment.
|
| I find it rather interesting that the structured representations
| go from sparse to full to sparse as a function of layer depth. I
| have noticed that applying weight decay penalty as an exponential
| function of layer depth gives improved results over using a
| global weight decay.
| cwmoore wrote:
| Isn't this simply mirroronic gravitation?
___________________________________________________________________
(page generated 2025-05-20 23:01 UTC)