https://github.com/boyter/cs Skip to content Toggle navigation Sign up * Product + Actions Automate any workflow + Packages Host and manage packages + Security Find and fix vulnerabilities + Codespaces Instant dev environments + Copilot Write better code with AI + Code review Manage code changes + Issues Plan and track work + Discussions Collaborate outside of code Explore + All features + Documentation + GitHub Skills + Blog * Solutions For + Enterprise + Teams + Startups + Education By Solution + CI/CD & Automation + DevOps + DevSecOps Case Studies + Customer Stories + Resources * Open Source + GitHub Sponsors Fund open source developers + The ReadME Project GitHub community articles Repositories + Topics + Trending + Collections * Pricing [ ] * # In this repository All GitHub | Jump to | * No suggested jump to results * # In this repository All GitHub | Jump to | * # In this user All GitHub | Jump to | * # In this repository All GitHub | Jump to | Sign in Sign up {{ message }} boyter / cs Public * Notifications * Fork 6 * Star 135 command line codespelunker or code search License MIT, Unlicense licenses found Licenses found MIT LICENSE Unlicense UNLICENSE 135 stars 6 forks Star Notifications * Code * Issues 1 * Pull requests 0 * Actions * Projects 0 * Security * Insights More * Code * Issues * Pull requests * Actions * Projects * Security * Insights boyter/cs This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository. master Switch branches/tags [ ] Branches Tags Could not load branches Nothing to show {{ refName }} default View all branches Could not load tags Nothing to show {{ refName }} default View all tags Name already in use A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch? Cancel Create 5 branches 2 tags Code * Local * Codespaces * Clone HTTPS GitHub CLI [https://github.com/b] Use Git or checkout with SVN using the web URL. [gh repo clone boyter] Work fast with our official CLI. Learn more about the CLI. * Open with GitHub Desktop * Download ZIP Sign In Required Please sign in to use Codespaces. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching GitHub Desktop If nothing happens, download GitHub Desktop and try again. Launching Xcode If nothing happens, download Xcode and try again. Launching Visual Studio Code Your codespace will open once ready. There was a problem preparing your codespace, please try again. Latest commit @boyter boyter Update README.md ... 7d4f033 Jun 6, 2023 Update README.md 7d4f033 Git stats * 554 commits Files Permalink Failed to load latest commit information. Type Name Latest commit message Commit time .github/workflows Create codeql-analysis.yml July 7, 2020 08:38 asset appease the linting gods June 5, 2023 11:55 vendor allow build for windows June 6, 2023 17:30 .gitignore preperation June 5, 2023 13:27 .goreleaser.yml allow build for windows June 6, 2023 17:30 .ignore Progress January 24, 2020 16:51 CODE_OF_CONDUCT.md First commit with slightly working stuff July 25, 2019 19:26 CONTRIBUTING.md First commit with slightly working stuff July 25, 2019 19:26 LICENSE First commit with slightly working stuff July 25, 2019 19:26 README.md Update README.md June 6, 2023 17:39 UNLICENSE First commit with slightly working stuff July 25, 2019 19:26 check.sh start to prepare for release June 2, 2023 21:09 console.go tweak for performance June 4, 2023 11:04 cs_http.png http image June 5, 2023 13:06 debounce.go start to prepare for release June 2, 2023 21:09 debounce_test.go start to prepare for release June 2, 2023 21:09 file.go appease the linting gods June 5, 2023 11:55 globals.go appease the linting gods June 5, 2023 11:55 go.mod allow build for windows June 6, 2023 17:30 go.sum allow build for windows June 6, 2023 17:30 helpers.go start to prepare for release June 2, 2023 21:09 http.go appease the linting gods June 5, 2023 11:55 http_helpers.go resolve paging ext issue June 5, 2023 11:03 main.go add windows support June 6, 2023 17:35 ranker.go start to prepare for release June 2, 2023 21:09 search.go appease the linting gods June 5, 2023 11:55 searcher.go start to prepare for release June 2, 2023 21:09 snippet.go start to prepare for release June 2, 2023 21:09 structs.go start to prepare for release June 2, 2023 21:09 tui.go resolve regex region display issue June 5, 2023 10:46 View code [ ] codespelunker (cs) Pitch Install Go Get Nixos Manual FAQ Is this as fast as... You didn't let me finish, I was going to ask if it's as fast as... Does it work on normal documents? Where is the index? How does the ranking work then? How do you get the snippets? What does HTTP mode look like? Usage README.md codespelunker (cs) A command line search tool. Allows you to search over code or text files in the current directory either on the console, via a TUI or HTTP server, using some boolean queries or regular expressions. Consider it a similar approach to using ripgrep, silver searcher or grep coupled with fzf but in a single tool. Dual-licensed under MIT or the UNLICENSE. Go Report Card Coverage Status Cs Count Badge asciicast Pitch Why use cs? * Reasonably fast * Rank results on the fly helping you find things * Searches across multiple lines * Has a nice TUI interface. The reason cs exists at all is because I was running into limitations using rg TERM | fzf and decided to solve my own problem. Install If you want to create a package to install things please do. Let me know and ill ensure I add it here. Go Get If you have Go >= 1.20 installed go install github.com/boyter/cs@v1.1.0 Nixos nix-shell -p codespelunker NixOS/nixpkgs#236073 Manual Binaries for Windows, GNU/Linux and macOS are available from the releases page. FAQ Is this as fast as... No. You didn't let me finish, I was going to ask if it's as fast as... The answer is probably no. It's not directly comparable. No other tool I know of works like this outside of full indexing tools such as hound, searchcode, sourcegraph etc... None work on the fly like this does. While cs does have some overlap with tools like ripgrep, grep, ack or the silver searcher the reality is it does not work the same way, so any comparison is pointless. It is slower than most of them, but its also doing something different. You can replicate some of what it does by piping their output into fzf though if you feel like a flawed comparison. On my local machine which at time of writing is a Macbook Air M1 it can search a recent checkout of the linux source code in ~2.5 seconds. While absolute performance is not a design goal, I also don't want this to be a slow tool. As such if any obvious performance gains are on the table I will take them. Does it work on normal documents? So long as they are text. I wrote it to search code, but it works just as well on full text documents. The snippet extraction for example was tested on Pride and Prejudice. If you had a heap of PDF's you could shell script some use of pdftotext and get something searchable. Note it was designed for code and as such has full .ignore and .gitignore support. Where is the index? There is none. Everything is brute force calculated on the fly. For TUI mode there are some shortcuts taken with caching of results to speed things up. How does the ranking work then? Standard BM25 or TF/IDF or the modified TF/IDF in Lucene https:// opensourceconnections.com/blog/2015/10/16/ bm25-the-next-generation-of-lucene-relevation/ which dampens the impact of term frequency. Technically speaking it's not accurate because it calculates the weights based on what it matched on and not everything, but it works well enough in practice and is calculated on the fly. Try it out and report if something is not working as you expect? How do you get the snippets? It's not fun... https://github.com/boyter/cs/blob/master/snippet.go Have a look at the code. It works by passing the document content to extract the snippet from and all the match locations for each term. It then looks through each location for each word, and checks on either side looking for terms close to it. It then ranks on the term frequency for the term we are checking around and rewards rarer terms. It also rewards more matches, closer matches, exact case matches and matches that are whole words. For more info read the "Snippet Extraction AKA I am PHP developer" section of this blog post https://boyter.org/posts/ abusing-aws-to-make-a-search-engine/ What does HTTP mode look like? It's a little brutalist. scc You can change its look and feel using --template-display and --template-search. See https://github.com/boyter/cs/tree/master/asset /templates for example templates you can use to modify things. cs -d --template-display ./asset/templates/display.tmpl --template-search ./asset/templates/search.tmpl Usage Command line usage of cs is designed to be as simple as possible. Full details can be found in cs --help or cs -h. Note that the below reflects the state of master not a release, as such features listed below may be missing from your installation. $ cs -h code spelunker (cs) code search. Version 1.0.0 Ben Boyter cs recursively searches the current directory using some boolean logic optionally combined with regular expressions. Usage: cs [flags] Flags: --address string address and port to listen to in HTTP mode (default ":8080") --binary set to disable binary file detection and search binary files -c, --case-sensitive make the search case sensitive --exclude-dir strings directories to exclude (default [.git,.hg,.svn]) -x, --exclude-pattern strings file and directory locations matching case sensitive patterns will be ignored [comma separated list: e.g. vendor,_test.go] -r, --find-root attempts to find the root of this repository by traversing in reverse looking for .git or .hg -f, --format string set output format [text, json, vimgrep] (default "text") -h, --help help for cs --hidden include hidden files -d, --http-server start http server for search -i, --include-ext strings limit to file extensions (N.B. case sensitive) [comma separated list: e.g. go,java,js,C,cpp] --max-read-size-bytes int number of bytes to read into a file with the remaining content ignored (default 1000000) --min include minified files --min-line-length int number of bytes per average line for file to be considered minified (default 255) --no-gitignore disables .gitignore file logic --no-ignore disables .ignore file logic -o, --output string output filename (default stdout) --ranker string set ranking algorithm [simple, tfidf, tfidf2, bm25] (default "bm25") -s, --snippet-count int number of snippets to display (default 1) -n, --snippet-length int size of the snippet to display (default 300) --template-display string path to display template for custom styling --template-search string path to search template for custom styling -v, --version version for cs Searches work on single or multiple words with a logical AND applied between them. You can negate with NOT before a term. You can do exact match with quotes, and do regular expressions using toothpicks. Example search that uses all current functionality cs t NOT something test~1 "ten thousand a year" "/pr[e-i]de/" You can use it in a similar manner to fzf in TUI mode if you like, since cs will return the matching document path if you hit the enter key. cat `cs` About command line codespelunker or code search Topics search cli code tui command-line-tool Resources Readme License MIT, Unlicense licenses found Licenses found MIT LICENSE Unlicense UNLICENSE Code of conduct Code of conduct Stars 135 stars Watchers 4 watching Forks 6 forks Report repository Releases 2 v1.1.0 Latest Jun 6, 2023 + 1 release Packages 0 No packages published Contributors 3 * @boyter boyter Ben Boyter * @tlelson tlelson Tim Elson * @nicola-zanardi nicola-zanardi Nicola Zanardi Languages * Go 98.5% * Shell 1.5% Footer (c) 2023 GitHub, Inc. Footer navigation * Terms * Privacy * Security * Status * Docs * Contact GitHub * Pricing * API * Training * Blog * About You can't perform that action at this time. You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.