[HN Gopher] Non-Machine Learning Image Matching with a Vector DB
       ___________________________________________________________________
        
       Non-Machine Learning Image Matching with a Vector DB
        
       Author : hannah-pdx
       Score  : 31 points
       Date   : 2022-08-23 13:35 UTC (9 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | fzliu wrote:
       | Cool to see a vector search application that leverages
       | traditional image features. I'm curious to know which of these
       | methods performed best on cropped images, which remain somewhat
       | of a challenge for traditional classification models (contrastive
       | models trained specifically with image crops tend to work much
       | better).
       | 
       | Also for the vector database itself, have you considered spinning
       | up your own open-source alternative such as Milvus
       | (https://github.com/milvus-io/milvus), or were you only
       | considering managed services?
        
         | hannah-pdx wrote:
         | The full cropped results are in the "Correctness" tab of the
         | Google sheet linked at the bottom, with more details in the
         | "Scoring" tab, but TL;DR the intensity vector worked best, with
         | Goldberg (the one I chose) a pretty close second. Goldberg
         | correctly returned the correct result highest-scored in 79% of
         | cases, with it present in the result in 90%.
         | 
         | I'm primarily interested in managed services. I've been an SRE
         | and I hope to not be in that role again.
        
       | hannah-pdx wrote:
       | As part of the development of a system that requires searching by
       | image, we needed to compute feature vectors for use with the
       | Pinecone vector database. All the research we could find focused
       | on either ML approaches, which were untenable due to hopes to
       | perform vector generation in the browser, or hamming distance
       | vector comparison, which are untenable for large scale search.
       | 
       | The README here contains my research into several algorithms'
       | performance, and the repo contains the code that performed the
       | data gathering.
       | 
       | The site alt-text.org is still alpha quality and under active
       | development, and the library backing it is quite small so most
       | searches will fail, but feel free to play around with it.
       | 
       | Twitter users can help build the library with the link in the
       | upper right corner, though it does not yet work on mobile.
        
         | dontreact wrote:
         | Why is vector generation in the browser untenable with ML
         | approaches? There are tiny models and tensorflow.js
        
           | hannah-pdx wrote:
           | Hmm, the smallest model I see is still 4.3MB, are there
           | smaller?
           | 
           | My quick read of the stats offered says that the smaller
           | model's accuracy suffers considerably. I could definitely see
           | running similar tests on it though!
           | 
           | All that said, the matching tensorflow offers from my
           | understanding is also not exactly what I'm after. I'm
           | primarily concerned with matching identical-to-humans images,
           | possibly with small modifications such as size changes. Think
           | more "are these two images identical" vs "give me pictures of
           | dogs"
        
             | jmalicki wrote:
             | There is the metric learning problem to learn a hash for
             | similarity https://github.com/tensorflow/similarity
             | 
             | That said, I don't see many good models available for
             | download on tfhub or huggingface optimized for it, but you
             | can always programmatically modify your images (if you
             | truly mean identical to humans) - change white balance,
             | crop, rotate, select adjacent frames from videos, etc. and
             | optimize a network that is small enough for you to be
             | satisfied and see if that works, as a possible alternative.
        
       | rhsimplex wrote:
       | I also developed a service based on the Goldberg paper in the mid
       | 2010s. We flattened and discretized the signature to make it
       | searchable which gave us pretty good results:
       | https://github.com/ProvenanceLabs/image-match
       | 
       | Sorry I haven't maintained the project in years so it's unlikely
       | to work out of the box. But who knows, maybe you'll find
       | something useful here for your project!
        
         | hannah-pdx wrote:
         | Ooh, very cool. Were the quoted insertion rates including the
         | vectorization times, and was that using the ES or Mongo
         | backend?
        
           | hannah-pdx wrote:
           | Oh, and did you explore using more lightness buckets than the
           | 5 from the paper?
        
       ___________________________________________________________________
       (page generated 2022-08-23 23:03 UTC)