[HN Gopher] EM-LLM: Human-Inspired Episodic Memory for Infinite ...
___________________________________________________________________
EM-LLM: Human-Inspired Episodic Memory for Infinite Context LLMs
Author : jbotz
Score : 104 points
Date : 2025-05-10 07:49 UTC (4 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| MacsHeadroom wrote:
| So, infinite context length by making it compute bound instead of
| memory bound. Curious how much longer this takes to run and when
| it makes sense to use vs RAG.
| mountainriver wrote:
| TTT, cannon layers, and titans seem like a stronger approach IMO.
|
| Information needs to be compressed into latent space or it
| becomes computationally intractable
| searchguy wrote:
| do you have references to
|
| > TTT, cannon layers, and titans
| najarvg wrote:
| This was the nearest reference I could find. Links to an
| unofficial pytorch implementation on Github are also linked
| in the threads somewhere - https://www.reddit.com/r/LocalLLaM
| A/comments/1i0q8nw/titans_...
| vessenes wrote:
| is titans replicated? I feel like lucidrains couldn't
| replicate.
| logicchains wrote:
| I think something like Titans explains Gemini's excellent
| long context performance. That would explain why the Titan
| team hasn't released the training code or hyperpameters used
| even though they said in the paper that they would, and why
| soon after that it came out that DeepMind would be holding
| off publishing new results for 6 months to avoid giving away
| competitive advantages.
| p_v_doom wrote:
| Interesting. Before there even was attention I was thinking that
| the episodic memory model offers something that could be very
| useful for neural nets, so its cool to see people testing that
| killerstorm wrote:
| Note that this works within a single sequence of tokens. It might
| be consistent with "episodic memory" metaphor if we consider a
| particular transformer run as its experience.
|
| But this might be very different from what people expect from
| "memory" - i.e. ability to learn vast amounts of information and
| retrieve it as necessary.
|
| This is more like a refinement of transformer attention: instead
| of running attention over all tokens (which is very expensive as
| it's quadratic), it selects a subset of token spans and runs
| fine-grained attention only on those. So it essentially breaks
| transformer attention into two parts - coarse-grained (k-NN over
| token spans) and fine-grained (normal).
|
| It might be a great thing for long-context situations. But it
| doesn't make sense when you want millions of different facts to
| be considered - making them into long context is rather
| inefficient.
| yorwba wrote:
| It would be inefficient if you had to do it from scratch for
| every query, but if you can do it once as a preprocessing step
| and reuse the prepared context for many queries, it might start
| to become more efficient than a shorter context that includes
| only some documents but has to be reprocessed because it's
| different every time.
| killerstorm wrote:
| Yes, I think it might be a good solution where you have a
| context up to 10M of tokens and you do a lot of requests with
| that context. It might be relevant for agentic stuff which
| tends to produce long chat logs - especially with some
| gadgets on top, e.g. some 'episodes' might be completely
| removed as obsolete.
|
| But I don't think it's a good solution for bigger amounts of
| data - as in that case it's more beneficial if that can be
| formed into independent memories.
___________________________________________________________________
(page generated 2025-05-14 23:02 UTC)