[HN Gopher] Using Vectorize to build an unreasonably good search...
___________________________________________________________________
Using Vectorize to build an unreasonably good search engine in 160
lines of code
Author : ColinWright
Score : 123 points
Date : 2025-12-21 16:36 UTC (4 days ago)
(HTM) web link (blog.partykit.io)
(TXT) w3m dump (blog.partykit.io)
| Supermancho wrote:
| Site has a neat feature where you can see the pointers of other
| people, marked by regional? notations, scrolling through the
| content.
| wormpilled wrote:
| It's amazing! Got so distracted, gotta switch to reader mode
| haha. Never seen anything like that.
| fnord77 wrote:
| that got annoying fast
| wqaatwt wrote:
| Seems fantastic for analytics. I wonder how many news sites do
| that
| TheLNL wrote:
| It doesn't look to be live though, I didn't see anyone reacting
| to weird cursor movements I was making
| mips_avatar wrote:
| There's a lot of previously intractable problems that are getting
| solved with these new embeddings models. I've been building a
| geocoder for the past few months and it's been remarkable how
| close to google places I can get with just slightly enriched open
| street maps plus embedding vectors
| occupant wrote:
| That sounds really interesting. If you're open to it, I'd be
| curious what the high-level architecture looks like (what gets
| embedded, how you rank results)?
| isaachh wrote:
| Id love to hear more about this
| robrenaud wrote:
| What are you embedding? Are you doing a geo restricted area
| (small universe?).
| repeekad wrote:
| What about re-ranking? In my limited experience, adding
| fast+cheap re-ranking with something like Cohere to the query
| results took an okay vector based search and made top 1-5 results
| much stronger
| vjerancrnjak wrote:
| Query expansion works better.
| repeekad wrote:
| Query expansion happens before the retrieval query, reranking
| is applied after the ranked results are returned, both are
| important
| sa-code wrote:
| Query expansion and re ranking can and often do coexist
|
| Roughly, first there is the query analysis/manipulation phase
| where you might have NER, spell check, query
| expansion/relaxation etc
|
| Then there is the selection phase, where you retrieve all
| items that are relevant. Sometimes people will bring in
| results from both text and vector based indices. Perhaps and
| additional layer to group results
|
| Then finally you have the reranking layer using a cross
| encoder model which might even have some personalisation in
| the mix
|
| Also, with vector search you might not need query expansion
| necessarily since semantic similarity does loose association.
| But every domain is unique and there's only one way to find
| out
| sgk284 wrote:
| Reranking is definitely the way to go. We personally found
| common reranker models to be a little too opaque (can't explain
| to the user why this result was picked) and not quite steerable
| enough, so we just use another LLM for reranking.
|
| We open-sourced our impl just this week:
| https://github.com/with-logic/intent
|
| We use Groq with gpt-oss-20b, which gives great results and
| only adds ~250ms to the processing pipeline.
|
| If you use mini / flash models from OpenAI / Gemini, expect it
| to be 2.5s-3s of overhead.
| simonw wrote:
| I was super-excited about vector search and embeddings in 2024
| but my enthusiasm has faded somewhat in 2025 for a few reasons:
|
| - LLMs with a grep or full-text search tool turn out to be great
| at fuzzy search already - they throw a bunch of OR conditions
| together and run further searches if they don't find what they
| want
|
| - ChatGPT web search and Claude Code code search are my favorite
| AI-assisted search tools and neither bother with vectors
|
| - Building and maintaining a large vector speech index is a pain.
| The vector are usually pretty big and you need to keep them in
| memory to get truly great performance. FTS and grep are way less
| hassle.
|
| - Vector matches are weird. So you get back the top twenty
| results... those might be super relevant or they might be total
| garbage, it's on you to do a second pass to figure out if they're
| actually useful results or not.
|
| I expected to spend much of 2025 building vector search engines,
| but ended up not finding them as valuable as I had thought.
| markerz wrote:
| The problem with LLMs using full-text-search is they're very
| slow compared to a vector search query. I will admit the
| results are impressive but often it's because I kick off an
| agent query and step away for 5 minutes.
|
| On the other hand, generating and regenerating embeddings for
| all your documents can be time consuming and costly, depending
| on how often you need to reindex
| leobg wrote:
| Not an apples to apples comparison. Vector search is only
| fast after you have built an index. The same is true for full
| text search. That too, will be blazing fast once you have
| built an index (like Google pre-transformer).
| Someone wrote:
| > The vector are usually pretty big and you need to keep them
| in memory to get truly great performance. FTS and grep are way
| less hassle.
|
| If you find disk I/O for grep acceptable, why would it matter
| for vectors? They aren't much bigger, are they?
| marginalia_nu wrote:
| The ultimate bottleneck in any search application is IOPS;
| how much data can you get off disk to compare within a
| tolerable time span.
|
| Embeddings are huge compared to what you need with FTS, which
| generally has good locality, compresses extremely well, and
| permits sub-linear intersection algorithms and other tricks
| to make the most of your IOPS.
|
| Regardless of vector size, you are unlikely to get more than
| one embedding per I/O operation with a vector approach. Even
| if you can fit more vectors into a block, there is no good
| way of arranging them to ensure efficient locality like you
| can with e.g. a postings list.
|
| Thus off a 500K IOPS drive, given a 100ms execution window,
| your theoretical upper bound is 50K embeddings ranked,
| assuming actual ranking takes no time and no other disk
| operations are performed and you have only a single user.
|
| Given you are more than likely comparing multiple embeddings
| per document, this carriage turns to a pumpkin pretty
| rapidly.
| croemer wrote:
| Doesn't ChatGPT web search use a (vector) search engine under
| the hood, e.g. Bing? Do we know how it works exactly?
| simonw wrote:
| I've not heard about Bing using vector search, at least
| outside of their image search feature
| https://arxiv.org/abs/1802.04914
|
| Information about how Bing text search works appears to be
| pretty sparse though.
|
| One of the great mysteries to me right now is how ChatGPT
| search actually works.
|
| It was Bing when they first launched it, but OpenAI have been
| investing a ton into their own search infrastructure since
| then. I can't figure out how much of it is Bing these days vs
| their own home-rolled system.
|
| What's confusing is how secretive OpenAI are about it! I
| would personally value it a whole lot more if I understood
| how it works.
|
| So maybe it's way more vector-based than I believe.
|
| I'd expect any modern search engine to have aspects of
| vectors somewhere - some kind of hybrid BM25 + vectors thing,
| or using vectors for re-ranking after retrieving likely
| matches via FTS. That's different from being pure vectors
| though.
| windexh8er wrote:
| Given that it's not documented also becomes a trust issue.
| OpenAI is clearly headed towards monetizing results and if
| search is biased / injected with unlabeled ads or
| questionable sources they become a new vector for both
| untrustworthy results and potential misdirection or
| misinformation.
| jdthedisciple wrote:
| In my experience vector search (top 50 results) combined with
| reranking (top 5-15 of those 50 results) yields not only great
| results but is even quite performant if done right (which is
| not hard!).
| softwaredoug wrote:
| The main problem isn't embeddings, in my experience, it's that
| "vector search" is the wrong conceptual framework to think
| about the problem
|
| We need to think about query+content understanding before
| deciding a sub problem happens to be helped by embeddings. RAG
| naively looks like a question answering "passage retrieval"
| problem, when in reality it's more structured retrieval than we
| first assume (and LLMs can learn how to use more structured
| approaches to explore data much better now than in 2022)
|
| https://softwaredoug.com/blog/2025/12/09/rag-users-want-affo...
| bonecrusher2102 wrote:
| Love seeing you in these threads! We use "AI Powered Search"
| as a bible on our team. Thanks for all your contributions to
| the community.
| sa-code wrote:
| Models like bge are small and quantized versions will fit in
| browser or on a tiny machine. Not sure why everyone reaches for
| an API as their first choice
| yuzhun wrote:
| While embeddings are generally not required in the context of
| code, I am interested in how they perform in the legal and
| regulatory domain, where documents are substantially longer.
| Specifically, how do embeddings compare with approaches such as
| ripgrep in terms of effectiveness?
| RomanPushkin wrote:
| You might be getting a good _recall_ rate, since vectorize search
| is ANN, but the _precision_ can be low, because reranker piece is
| missing. So I would slightly improve it by adding 10 more lines
| of code and introducing reranker after the search (slightly
| increasing topK). Query expansion in the beginning can be also
| added to improve recall.
| ____tom____ wrote:
| You didn't build a search engine in 160 lines of code. You build
| a client for a search engine in 160 lines of code. The vector
| database is providing the search.
| ivanjermakov wrote:
| Look, I made a thing in two lines of code!
| import thing from everything thing()
| croemer wrote:
| Missing a (2024)
| daquisu wrote:
| Now it is even easier. Cloudflare has a beta product called AI
| Search that implements most of these 160 lines of code
| abhinavb05 wrote:
| At my workplace we are using vector embedding to build
| recommendation system and the results are amazing
| wg0 wrote:
| Could you elaborate on the storage engine and processing
| pipeline if not confidential?
___________________________________________________________________
(page generated 2025-12-25 23:01 UTC)