https://github.com/raphaelsty/neural-cherche Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Resources + Learning Pathways + White papers, Ebooks, Webinars + Customer Stories + Partners * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing Search or jump to... Search code, repositories, users, issues, pull requests... Search [ ] Clear Search syntax tips Provide feedback We read every piece of feedback, and take your input very seriously. [ ] [ ] Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly Name [ ] Query [ ] To see all available qualifiers, see our documentation. Cancel Create saved search Sign in Sign up You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert {{ message }} raphaelsty / neural-cherche Public * Notifications * Fork 2 * Star 100 Neural Search raphaelsty.github.io/neural-cherche/ License MIT license 100 stars 2 forks Activity Star Notifications * Code * Issues 0 * Pull requests 0 * Actions * Projects 0 * Security * Insights Additional navigation options * Code * Issues * Pull requests * Actions * Projects * Security * Insights raphaelsty/neural-cherche This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. main Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 4 branches 12 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/r] Use Git or checkout with SVN using the web URL. [gh repo clone raphae] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @raphaelsty raphaelsty Update README.md ... 0f4fad7 Nov 16, 2023 Update README.md 0f4fad7 Git stats * 51 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time docs update readme November 16, 2023 23:42 neural_cherche update version November 16, 2023 23:45 .gitignore Update retriever with sparse tensors August 6, 2023 22:49 LICENSE Create LICENSE October 20, 2023 16:03 Makefile update readme November 16, 2023 23:42 README.md Update README.md November 17, 2023 00:02 mkdocs.yml update package November 16, 2023 23:23 pytest.ini Init SparsEmbed August 5, 2023 02:04 setup.cfg Init SparsEmbed August 5, 2023 02:04 setup.py update package November 16, 2023 23:23 View code Neural-Cherche Installation Documentation Quick Start References README.md Neural-Cherche Neural Search [logo] documentation license Neural-Cherche is a library designed to fine-tune neural search models such as Splade, ColBERT, and SparseEmbed on a specific dataset. Neural-Cherche also provide classes to run efficient inference on a fine-tuned retriever or ranker. Neural-Cherche aims to offer a straightforward and effective method for fine-tuning and utilizing neural search models in both offline and online settings. It also enables users to save all computed embeddings to prevent redundant computations. Installation We can install neural-cherche using: pip install neural-cherche If we plan to evaluate our model while training install: pip install "neural-cherche[eval]" Documentation The complete documentation is available here. Quick Start Your training dataset must be made out of triples (anchor, positive, negative) where anchor is a query, positive is a document that is directly linked to the anchor and negative is a document that is not relevant for the anchor. X = [ ("anchor 1", "positive 1", "negative 1"), ("anchor 2", "positive 2", "negative 2"), ("anchor 3", "positive 3", "negative 3"), ] And here is how to fine-tune ColBERT from a Sentence Transformer pre-trained checkpoint using neural-cherche: import torch from neural_cherche import models, utils, train model = models.ColBERT( model_name_or_path="sentence-transformers/all-mpnet-base-v2", device="cuda" if torch.cuda.is_available() else "cpu" ) optimizer = torch.optim.AdamW(model.parameters(), lr=3e-5) X = [ ("query", "positive document", "negative document"), ("query", "positive document", "negative document"), ("query", "positive document", "negative document"), ] for anchor, positive, negative in utils.iter( X, epochs=1, batch_size=32, shuffle=True ): loss = train.train_colbert( model=model, optimizer=optimizer, anchor=anchor, positive=positive, negative=negative, ) model.save_pretrained("checkpoint") References * SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking authored by Thibault Formal, Benjamin Piwowarski, Stephane Clinchant, SIGIR 2021. * SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval authored by Thibault Formal, Carlos Lassance, Benjamin Piwowarski, Stephane Clinchant, SIGIR 2022. * SparseEmbed: Learning Sparse Lexical Representations with Contextual Embeddings for Retrieval authored by Weize Kong, Jeffrey M. Dudek, Cheng Li, Mingyang Zhang, and Mike Bendersky, SIGIR 2023. * ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT authored by Omar Khattab, Matei Zaharia, SIGIR 2020. About Neural Search raphaelsty.github.io/neural-cherche/ Topics google transformers stanford language-model semantic-search neural-search colbert splade sparseembed Resources Readme License MIT license Activity Stars 100 stars Watchers 2 watching Forks 2 forks Report repository Releases 12 1.0.0 Latest Nov 16, 2023 + 11 releases Packages 0 No packages published Languages * Python 99.5% * Makefile 0.5% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time.