[HN Gopher] Exact binary vector search for RAG in 100 lines of J...
___________________________________________________________________
Exact binary vector search for RAG in 100 lines of Julia
Author : lunaticd
Score : 55 points
Date : 2024-05-16 15:09 UTC (1 days ago)
(HTM) web link (domluna.com)
(TXT) w3m dump (domluna.com)
| mik1998 wrote:
| for i in 0:7 c += (r >> i) & 1 end
|
| This is just popcnt, surely Julia has a built in for that.
| ummonk wrote:
| From what I've heard it's actually faster to create a 256 byte
| lookup table than to use popcnt.
| mik1998 wrote:
| It used to be pretty bad on old intel processors but nowadays
| it should be faster than an L1 fetch.
| borodi wrote:
| There is, it's called count_ones. Though I wouldn't be
| surprised if LLVM could maybe optimize some of these loops into
| a popcnt, but I'm sure it would be brittle
| floobertoober wrote:
| I'm not sure what is meant by "exact" here - do they describe
| their binarisation process at all? This seems more like an XOR
| benchmark than a rag benchmark, no mention of recall or other
| relevant performance metrics
| jjnoakes wrote:
| Some (not all) of your questions may be answered by the linked
| article near the top of the submitted article, which goes into
| more detail about how much is lost quantizing to 1 bit (and 1
| byte): https://huggingface.co/blog/embedding-quantization
| klowrey wrote:
| Why not use the built in BitVector type that has specialized code
| for things like xor?
|
| https://docs.julialang.org/en/v1/base/arrays/#Base.BitArray
___________________________________________________________________
(page generated 2024-05-17 23:00 UTC)