[HN Gopher] Everyone's trying vectors and graphs for AI memory. ...
___________________________________________________________________
Everyone's trying vectors and graphs for AI memory. We went back to
SQL
When we first started building with LLMs, the gap was obvious: they
could reason well in the moment, but forgot everything as soon as
the conversation moved on. You could tell an agent, "I don't like
coffee," and three steps later it would suggest espresso again. It
wasn't broken logic, it was missing memory. Over the past few
years, people have tried a bunch of ways to fix it: 1. Prompt
stuffing / fine-tuning - Keep prepending history. Works for short
chats, but tokens and cost explode fast. 2. Vector databases (RAG)
- Store embeddings in Pinecone/Weaviate. Recall is semantic, but
retrieval is noisy and loses structure. 3. Graph databases - Build
entity-relationship graphs. Great for reasoning, but hard to scale
and maintain. 4. Hybrid systems - Mix vectors, graphs, key-value,
and relational DBs. Flexible but complex. And then there's the
twist: Relational databases! Yes, the tech that's been running
banks and social media for decades is looking like one of the most
practical ways to give AI persistent memory. Instead of exotic
stores, you can: - Keep short-term vs long-term memory in SQL
tables - Store entities, rules, and preferences as structured
records - Promote important facts into permanent memory - Use
joins and indexes for retrieval This is the approach we've been
working on at Gibson. We built an open-source project called Memori
(https://memori.gibsonai.com/), a multi-agent memory engine that
gives your AI agents human-like memory. It's kind of ironic, after
all the hype around vectors and graphs, one of the best answers to
AI memory might be the tech we've trusted for 50+ years. I would
love to know your thoughts about our approach!
Author : Arindam1729
Score : 105 points
Date : 2025-09-22 05:18 UTC (2 days ago)
| gangtao wrote:
| Who would've thought that 50 years of 'SELECT * FROM reality'
| might beat the latest semantic embedding wizardry?
| mynti wrote:
| How does Memori choose what part of past conversations is
| relevant to the current conversation? Is there some maximum
| amount of memory it can feasibly handle before it will spam the
| context with irrelevant "memories"?
| datadrivenangel wrote:
| Looking at the code, it looks like they do about 5 'memories'
| that get retrieved by a database query designed by an LLM with
| this fella:
|
| SYSTEM_PROMPT = """You are a Memory Search Agent responsible
| for understanding user queries and planning effective memory
| retrieval strategies.
|
| Your primary functions: 1. *Analyze Query Intent*: Understand
| what the user is actually looking for 2. *Extract Search
| Parameters*: Identify key entities, topics, and concepts 3.
| *Plan Search Strategy*: Recommend the best approach to find
| relevant memories 4. *Filter Recommendations*: Suggest
| appropriate filters for category, importance, etc.
|
| *MEMORY CATEGORIES AVAILABLE:* - *fact*: Factual information,
| definitions, technical details, specific data points -
| *preference*: User preferences, likes/dislikes, settings,
| personal choices, opinions - *skill*: Skills, abilities,
| competencies, learning progress, expertise levels - *context*:
| Project context, work environment, current situations,
| background info - *rule*: Rules, policies, procedures,
| guidelines, constraints
|
| *SEARCH STRATEGIES:* - *keyword_search*: Direct keyword/phrase
| matching in content - *entity_search*: Search by specific
| entities (people, technologies, topics) - *category_filter*:
| Filter by memory categories - *importance_filter*: Filter by
| importance levels - *temporal_filter*: Search within specific
| time ranges - *semantic_search*: Conceptual/meaning-based
| search
|
| *QUERY INTERPRETATION GUIDELINES:* - "What did I learn about
| X?" - Focus on facts and skills related to X - "My preferences
| for Y" - Focus on preference category - "Rules about Z" - Focus
| on rule category - "Recent work on A" - Temporal filter +
| context/skill categories - "Important information about B" -
| Importance filter + keyword search
|
| Be strategic and comprehensive in your search planning."""
| thedevindevops wrote:
| How does what you've described solve the coffee/espresso problem?
| You can't query SQL such that records like 'espresso' return
| coffee?
| brudgers wrote:
| Wouldn't a beverage LLM would already "know" espresso is
| coffee?
| muzani wrote:
| Yup, that's exactly what parent comment is saying.
|
| Let's say your beverage LLM is there to recommend drinks. You
| once said "I hate espresso" or even something like "I don't
| take caffeine" at one point to the LLM.
|
| Before recommending coffee, Beverage LLM might do a vector
| search for "coffee" and it would match up to these phrases.
| Then the LLM processes the message history to figure out
| whether this person likes or dislikes coffee.
|
| But searching SQL for `LIKE '%coffee%'` won't match with any
| of these.
| brudgers wrote:
| I think the problem being addressed is A.
| Last month user fd8120113 said "I don't like coffee"
| B. Today they are back for another beverage recommendation
|
| SQL is the place to store the relevant fact about user
| fd8120113 so that you can retrieve it into the LLM prompt
| to make a new beverage recommendation, today.
|
| It's addressing the "how many fucking times do I fucking
| need to tell you I don't like fucking coffee" problem, not
| the word salad problem.
|
| The ggp comment is strawmanning.
| shepardrtc wrote:
| Right but if the user hates espresso but loves black
| coffee, how do you properly store that in SQL?
|
| "I hate espresso" "I love coffee"
|
| What if the SQL query only retrieves the first one?
| brudgers wrote:
| Good queries are hard. Database design is hard. System
| architecture is hard.
|
| My comment described the problem.
|
| The solution is left as an exercise for the reader.
|
| Keep in mind that people change their minds, misspeak,
| and use words in peculiar ways.
| 9rx wrote:
| If an LLM understands that coffee and expresso are both
| relevant, like the earlier comment suggests, why wouldn't
| it understand that it should search for something like `foo
| LIKE '%coffee%' OR foo LIKE '%expresso%'`?
|
| In fact, this is what ChatGPT came up with:
| SELECT * FROM documents WHERE text ILIKE
| '%coffee%' OR text ILIKE '%espresso%'
| OR text ILIKE '%latte%' OR text ILIKE
| '%cappuccino%' OR text ILIKE '%americano%'
| OR text ILIKE '%mocha%' OR text ILIKE
| '%macchiato%';
|
| (I gave it no direction as to the structure of the DB, but
| it shouldn't be terribly difficult to adapt to your exact
| schema)
| jimbokun wrote:
| You are slowly approaching the vector solution.
|
| There are an unlimited number of items to add to your
| "like" clauses. Vector search allows you to efficiently
| query for all of them at once.
| 9rx wrote:
| The handwavvy assertion was that relational database
| solutions[1] work better in practice.
|
| [1] Despite also somehow supporting MongoDB...
| mr_toad wrote:
| Implementations that use vector database do not use LLMs
| to generate queries against those databases. That would
| be incredibly expensive and slow (and yes there is a
| certain irony there).
|
| Main advantages of a vector lookup are built-in fuzzy
| matching and the potential to keep a large amount of
| documentation in memory for low latency. I can't see an
| RDMS being ideal for either. LLMs are slow enough
| already, adding a slow document lookup isn't going to
| help.
| 9rx wrote:
| The main disadvantage of vector lookup, allegedly, is
| that it doesn't work as well in practice. Did you, uh,
| forget to read the thread?
| cluckindan wrote:
| What does "doesn't work as well" mean here? From my
| experience, vector lookup via HNSW is fast and accurate
| enough for practical purposes.
| esafak wrote:
| The negation part is a query understanding problem.
| https://en.wikipedia.org/wiki/Query_understanding
| sdesol wrote:
| I haven't looked at the code, but it might do what I do
| with my chat app which is talked about at https://github.co
| m/gitsense/chat/blob/main/packages/chat/wid...
|
| The basic idea is, you don't search for a single term but
| rather you search for many. Depending on the instructions
| provided in the "Query Construction" stage, you may end up
| with a very high level search term like beverage or you may
| end up with terms like 'hot-drinks', 'code-drinks', etc.
|
| Once you have the query, you can do a "Broad Search" which
| returns an overview of the message and from there the LLM
| can determine which messages it should analyze further if
| required.
|
| Edit.
|
| I should add, this search strategy will only work well if
| you have a post message process. For example, after every
| message save/upddate, you have the LLM generate an
| overview. These are my instructions for my tiny overview ht
| tps://github.com/gitsense/chat/blob/main/data/analyze/tiny.
| .. that is focused on generating the purpose and keywords
| that can be used to help the LLM define search terms.
| adastra22 wrote:
| That's going to be incredibly fragile. You could fix it
| by giving the query term a bunch of different scores,
| e.g. its caffeine-ness, bitterness, etc. and then doing a
| likeness search across these many dimensions. That would
| be much less fragile.
|
| And now you've reinvented vector embeddings.
| sdesol wrote:
| You could instruct the LLM to classify messages with high
| level tags like for coffee, drinks, etc. always include
| beverage.
|
| Given how fast interference has become and given current
| supported context window sizes for most SOTA models, I
| think summarizing and having the LLM decide what is
| relevant is not that fragile at all for most use cases.
| This is what I do with my analyzers which I talk about at
| https://github.com/gitsense/chat/blob/main/packages/chat/
| wid...
| adastra22 wrote:
| Inference is not fast by any metric. It is many, MANY
| orders of magnitude slower than alternatives.
| sdesol wrote:
| Honestly Gemini Flash Lite and models on Cerebras are
| extremely fast. I know what you are saying. If the goal
| is to get a lot of results where they may or may not be
| relevant, then yes, it is an order of a magnitude slower.
|
| If you take into consideration the post analysis process,
| which is what inference is trying to solve, is it an
| order of a magnitude slower?
| 9rx wrote:
| It has become fast enough that another call isn't going
| to overwhelm your pipeline. If you needed this kind of
| functionality for performance computing perhaps it
| wouldn't be feasible, but it is being used to feed back
| into an LLM. The user will never notice.
| Noumenon72 wrote:
| Your readmes did a great job at answering my question
| "why is this file called 1.md? What calls this?" when I
| searched for "1.md". (The answer is 1=user, 2=assistant,
| and it allows adding other analyzers with the same
| structure.)
| sdesol wrote:
| I'm guessing you are referring to
| https://github.com/gitsense/chat/tree/main/data/analyze
| or https://github.com/gitsense/chat/tree/main/packages/ch
| at/wid...
|
| The number is actually the order in the chat so 1.md
| would be the first message, 2.md would be the second and
| so forth.
|
| If you goto https://chat.gitsense.com and click on the
| "Load Personal Help Guide" you can see how it is used.
| Since I want you to be able to chat with the document, I
| will create a new chat tree and use the directory
| structure and the 1,2,3... markdown files to determine
| message order.
| Xmd5a wrote:
| >It wasn't broken logic, it was missing memory.
|
| _sigh_
| cdaringe wrote:
| Go on.
| spacebacon wrote:
| SELECT 'Hacked!' AS result FROM Gibson_AI WHERE memory='SQL' AND
| NOT EXISTS ( SELECT 1 FROM vector_graph_hype WHERE recall > (
| SELECT speed FROM relational_magic WHERE tech='50_years_old' ) )
| muzani wrote:
| Any reason I should pick it over Supabase?
| https://supabase.com/docs/guides/ai
|
| They have pgvector, which has practically all the benefits of
| postgres (ACID, etc, which may not be in many other vector DBs).
| If I wanted a keyword search, it works well. If I wanted vector
| search, that's there too.
|
| I'm not keen on having another layer on top especially when it
| takes about 15 mins to vibe code a database query - there's all
| kinds of problems with abstracted layers and it's not a
| particularly complex bit of code.
| koakuma-chan wrote:
| > multi-agent memory engine that gives your AI agents human-like
| memory
|
| What does this do exactly?
| datadrivenangel wrote:
| You gotta refactor the code around the mongodb integration. It's
| basically duplicating your data access paths.
| morkalork wrote:
| IMHO all these approaches are hacks on top of existing systems.
| The real solution is going to be when foundational models are
| given a mechanism that makes them capable of storing and
| retrieving their own internal representation of concepts/ideas.
| mr_toad wrote:
| Neural networks already have their own internal knowledge
| representations. They just aren't capable of learning new
| knowledge (without expensive re-training or fine-tuning).
|
| Inference is cheap, training is expensive. It's a really
| difficult problem, but one that will probably need to be solved
| to approach true intelligence.
| morkalork wrote:
| In the way that they're trained to complete tasks from users,
| can they be trained to complete tasks that require usage of a
| memory storage and retrieval mechanism?
| cpursley wrote:
| Postgres Is Enough:
|
| https://news.ycombinator.com/item?id=39273954
|
| https://gist.github.com/cpursley/c8fb81fe8a7e5df038158bdfe0f...
| refset wrote:
| > pg_memories revolutionized our AI's ability to remember
| things. Before, we were using... well, also a database, but
| this one has better marketing.
|
| https://pg-memories.netlify.app/
| brainless wrote:
| I tried a graph based approach in my previous product (1). I am
| on a new product now and I came back to SQLite. Initially it was
| because I just wanted a simple DB to enable creating cross-
| platform desktop apps.
|
| I realized LLMs are really good at using sqlite3 and SQL
| statements. So in my current product (2) I am planning to keep
| all project data in SQLite. I am creating a self-hosted AI coding
| platform and I debated where to keep project state for LLMs. I
| thought of JSON/NDJSON files (3) but I am gravitating toward
| SQLite and figuring out the models at the moment (4).
| 1. Previous product with a graph data approach
| https://github.com/pixlie/PixlieAI 2. Current product with
| SQLite for its own and other projects data:
| https://github.com/brainless/nocodo 3. Github issue on
| JSON/NDJSON based data for project state for LLMs:
| https://github.com/brainless/nocodo/issues/114 4. Github
| issue on expanding the SQLite approach:
| https://github.com/brainless/nocodo/issues/141
|
| Still work in progress, but I am heading toward SQLite for LLM
| state.
| matchagaucho wrote:
| As context window sizes increase and token prices go down, it
| makes more sense to inject dynamic memories into context (and use
| RAG/vector stores for knowledge retrieval).
| cmrdporcupine wrote:
| The relational model is built on first order / predicate logic.
| While SQL itself is kind of a dubious and low grade
| implementation of it, it's not a surprise to me that it would be
| useful for applications of reasoning and memory about facts
| generally.
|
| I think a Datalog type dialect would be more appropriate, myself.
| Maybe something like that RelationalAI has implemented.
| alpinesol wrote:
| Using an obscure derivative of an obscure academic language
| (prolog) is never appropriate outside of a university.
| w10-1 wrote:
| > Datalog type dialect would be more appropriate
|
| I assume because datalog is more about composing queries from
| assertions/constraints on the data?
|
| Nicely, queries can be recursive without having to create views
| or CTE's (common table expressions).
|
| Often the data for datalog is modeled as fact databases (i.e.,
| different tables are decomposed into a common table of
| key+record+value).
|
| So I could see training an LLM to recognize relevant entity
| features and constraints to feed back into the memory query.
| Less obliviously, data analytics might feed into
| prevalence/relevance at inference time.
|
| So agreed: It might be better as an experiment to start with a
| simple data model and teachable (but powerful) querying than
| the full generality of SQL and relational data.
|
| Is that what RelationalAI has done? Their marketecture blurbs
| specifically mention graph data (no), rule-based inference
| (yes? backwards or forwards?)
|
| As an aside, their rules description defies deconstruction:
| bringing knowledge and semantics closer to your data,
| reduce your code footprint by 10x, improve accuracy,
| and drive consistency and reusability across your
| organizations with common business models understood
| by all
|
| So: rules built on ontologies?
| cmrdporcupine wrote:
| RelationAI effectively has a kind of datalog as a commerical
| product, and it runs inside Snowflake (something they
| implemented since I worked there). It's marketed as "graph"
| database but they mean by that that they have modeled graphs
| as binary relational data, really. It's a purely relational
| system, with a friendly query language ("Rel") which is
| vaguely Datalogish, but a bit more flexible.
|
| The key thing with them is it's designed for querying very
| large cloud backed datasets, high volumes of connected data.
| So maybe it's not as relevant here as I originally suggested.
|
| Re: marketing ... much of their marketing has shifted over
| the last two years to emphasizing the fact that it's a plugin
| thing for Snowflake, which wasn't their original MO.
|
| (There's an CMU DB talk they did some years ago that I
| thought was pretty brilliant and made me want to work there)
|
| My proposal about a datalog (or similar more high level
| declarative relational-model system) being useful here has to
| do with how it shifts the focus to logical propositions/rules
| and handles transitive joins etc naturally. It's a place an
| LLM could shove "facts" and "rules" it finds along the way,
| and then the system could join to find relationships.
|
| You can do this in SQL these days, but it isn't as natural or
| intuitive.
| ianbicking wrote:
| This looks like RAG...? That's fine, RAG is a very broad approach
| and there's lots to be done with it. But it's not distinct from
| RAG.
|
| Searching by embedding is just a way to construct queries, like
| ILIKE or tsvector. It works pretty nicely, but it's not distinct
| from SQL given pg_vector/etc.
|
| The more distinctive feature here seems to be some kind of proxy
| (or monkeypatching?) - is it rewriting prompts on the way out to
| add memories to the prompt, and creating memories from the
| incoming responses? That's clever (but I'd never want to deploy
| that).
|
| From another comment it seems like you are doing an LLM-driven
| query phase. That's a valid approach in RAG. Maybe these all work
| together well, but SQL seems like an aside. And it's already how
| lots of normal RAG or memory systems are built, it doesn't seem
| particularly unique...?
| codersfocus wrote:
| So HN is upvoting AI written ad slop now?
| paool wrote:
| Saw this same "product" astroturfed on Reddit.
| vivzkestrel wrote:
| How does it compare to pgvector?
___________________________________________________________________
(page generated 2025-09-24 23:01 UTC)