[HN Gopher] Ripgrep 13.0
___________________________________________________________________
Ripgrep 13.0
Author : mfrw
Score : 315 points
Date : 2021-06-13 11:10 UTC (1 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| jayflux wrote:
| Shoutout to Burntsushi
|
| He knows how to keep a good changelog
| https://github.com/BurntSushi/ripgrep/blob/master/CHANGELOG....
| it's good to read through and see the progress.
|
| He also cares about ergonomics as well as performance which I
| like. I find it much more easier to use than existing Grep tools.
|
| Interestingly, I believe Ripgrep started off as a test harness to
| benchmark the Rust Regex crate (he was the author of that). It
| was originally not intended to be a consumer cli. I guess he
| dogfooded it that well it became a successful tool
| burntsushi wrote:
| Thanks. :-)
|
| That's right. IIRC, I wrote it as a simple harness because I
| wanted to check whether someone _could_ feasibly write a grep
| tool using the regex crate and get similar performance. Once I
| got the harness up and running though, I realized it was not
| only competitive with GNU grep in a larger number of cases than
| I thought it would be (after some perf bug fixes, IIRC), but
| was substantially faster than ag. So that 's when I decided to
| polish it up into a tool for others to use.
| e12e wrote:
| Do you use any tooling for the Changelog/issue tagging (PERF,
| BUG) etc?
| burntsushi wrote:
| Nope. I just try to keep it updated as I go. But in
| practice, I usually forget to update it and it just ends up
| becoming part of the release process.[1]
|
| [1] -
| https://github.com/BurntSushi/ripgrep/blob/master/RELEASE-
| CH...
| a-dub wrote:
| lots of other cool rust goodies in his github account as well.
| kisamoto wrote:
| I unconsciously reach for ripgrep on a daily basis. Finding
| references just is incredibly easy without cluttering it with
| stuff listed in .gitignore (I'm looking at you node_modules..)
|
| This has reminded and motivated me to submit a donation to the
| project.
| kisamoto wrote:
| For those also looking I found it in the FAQ[0]:
|
| ------------------------------------------
|
| How can I donate to ripgrep or its maintainers?
|
| As of now, you can't. While I believe the various efforts that
| are being undertaken to help fund FOSS are extremely important,
| they aren't a good fit for me. ripgrep is and I hope will
| remain a project of love that I develop in my free time. As
| such, involving money---even in the form of donations given
| without expectations---would severely change that dynamic for
| me personally.
|
| Instead, I'd recommend donating to something else that is doing
| work that you find meaningful. If you would like suggestions,
| then my favorites are: The Internet Archive
| Rails Girls Wikipedia
|
| * [0]
| https://github.com/BurntSushi/ripgrep/blob/64ac2ebe0f2fe1c89...
| c7DJTLrn wrote:
| ripgrep is one of the most powerful tools in my toolkit. It is
| amazing being able to scan through huge codebases for strings and
| patterns in what feels like an instant. It eliminates the need
| for code intelligence or a full fledged IDE in my experience.
| nubb wrote:
| Just to jump on the burntsushi bandwagon, his nfldb project
| taught me postgres. I'll always be thankful for that.
| mfrw wrote:
| Highlights: A new short flag, -., has been added. It is an alias
| for the --hidden flag, which instructs ripgrep to search hidden
| files and directories.
|
| BREAKING CHANGES: Binary detection output has changed slightly.
| hawski wrote:
| Lack of this alias was one of the things that put me off
| regarding ripgrep. I'm using the silver searcher instead as it
| is plenty fast and it's CLI makes sense for me and doesn't
| change, AFAIR it's even mostly the same as ack, which I used
| before.
| rektide wrote:
| loving '-.'. I use --hidden a lot.
|
| also noteable, an underlying lib got some vectorization speed
| improvements.
| oweiler wrote:
| The best feature about ripgrep (apart from its great
| performance): Sane defaults out of the box!
| jjice wrote:
| My favorite 'new wave Unix' utility by far. So ergonomic, yet an
| easy switch from egrep.
| ibraheemdev wrote:
| Yep, I've also been loving mcfly, bat, and fd. I mentioned this
| further down in the post, but I compiled a list of all these
| modern unix tools a couple months ago:
| https://github.com/ibraheemdev/modern-unix
| Y_Y wrote:
| Just what I've been looking for! Thanks for making this. I'd
| love to see someone build a "new" userland with all the old
| gnu tools replaced where there's a good modern alternative.
| geoelectric wrote:
| This compilation is very useful. I've been needing to review
| my CLI toolkit for a bit, and this looks like it'll make it
| easy for me to try out some alternatives. Thank you for
| putting the time and energy in to create it.
| bewuethr wrote:
| I bumped into three blog posts a while ago that listed
| similar modern alternatives - lots of overlap:
|
| https://darrenburns.net/posts/tools/
|
| https://darrenburns.net/posts/more-tools
|
| https://darrenburns.net/posts/even-more-tools
| RBerenguel wrote:
| bat [0] (a cat replacement) and fzf [1] are the other two I
| would miss dearly, standing above some other "new" tools I use
| very regularly. [0]:
| https://github.com/sharkdp/bat [1]:
| https://github.com/junegunn/fzf
| kasbah wrote:
| Mine are fd [0] to replace `find` and tldr [1] to replace
| `man`.
|
| [0]: https://github.com/sharkdp/fd
|
| [1]: https://tldr.sh/
| papaf wrote:
| I find repgrep defaults sensible when searching for text
| but fd misses files that I expect it to find -- this has
| happened a few times now and I have gone back to find.
|
| For instance: 'fd .apk' to search for android builds.
| jjice wrote:
| fd is fantastic as well, and I can get away with tldr 80%
| of the time before diving into man.
| jjice wrote:
| fzf is another that is just absurdly powerful. fzf comes in
| handy in so many useful little scripts for me.
| thiht wrote:
| What do you use fzf for? I can't figure out a use case for
| it but everyone seems to love it, what am I missing on?
| lelandbatey wrote:
| The main thing I get out of it is super amazing fuzzy-
| search of my terminal history. Normally with bash's built
| in CTRL-R reverse history search, you pretty much have to
| know EXACTLY what you're searching for, and if there are
| a lot of similar commands between the most recently run
| match for a search and the one you're ACTUALLY searching
| for, you may have a really hard time finding it since you
| have to go backwards through the commands one at a time.
| For myself and many people, we were getting really good
| at typing `history | grep '<regex>'`, maybe with more
| pipes to grep for further refinement.
|
| But with fzf[1], that whole workflow of searching through
| your history is probably two orders of magnitude faster.
| Now you hit CTRL-R and you start typing any random part
| of the command you're trying to remember. If there was
| some other part of the command you remember, hit space
| and type that search term after the first search term.
| FZF will then show you the last 10-ish matches for all
| the search params you just typed, AND it will have done
| all this with no UI lag, no hitching, and lightning fast.
|
| I don't know what other people use FZF for, as this is
| the SINGLE feature that's so good I can't live without it
| anymore.
|
| [1] - https://github.com/junegunn/fzf#key-bindings-for-
| command-lin...
| taude wrote:
| Here's an example with rip grep. Say you search for
| something with a lot of results, you can feed the results
| with a lot of answers into it, and then add additional
| filters. Useful for when you're not "exactly" sure what
| your search term needs to be. I use it when ripgrepping a
| lot of code bases all at once....
|
| I also have a folder of a bunch of curl commands that I
| can search and apply fuzzy finding on the results that
| helps me explore to find.
|
| Contrived example, search for "t" in all these files and
| then pipe to fzf: rg t . | fzf
| Zizizizz wrote:
| 1. built in to zsh so I can fuzzy find my command history
| 2. similarly command t let's me do.. for example ```bat
| <ctrl t> (and fuzzy find whatever file I want``` 3. it's
| a vim plugin so it's a replacement the need for nerd
| tree, can be used as a file explorer, buffer finder,
| fuzzy find lines instead of / search, switch branches,
| etc 4. fzf-tab for zsh gives you fuzzy tab completion in
| your terminal too
| coffeeri wrote:
| Also consider installing ripgrep-all [1], an extension making it
| possible to search through PDFs, eBooks, ZIPs and other
| documents. The integration with fzf is also fantastic!
|
| [1] https://github.com/phiresky/ripgrep-all
| O_H_E wrote:
| Aaaaa, yes yes yes. I need that in my life.
| tekronis wrote:
| If you're on macOS, you can also install ripgrep via MacPorts:
| "sudo port install ripgrep"
| ufo wrote:
| One time, I almost accidentally deleted my files because I typoed
| rm instead of rg. Was saved by the -i flag... Since then, I've
| been using "alias gr=rg"
| moonchild wrote:
| alias rm='rm -i'
| toxik wrote:
| Kinda feel like rm should have been a longer command to type,
| like delete or unlink.
| wonks wrote:
| How does it compare to the_silver_searcher?
| __jem wrote:
| similar ergonomics, better performance
| snth wrote:
| You might be interested in this blog post:
| https://blog.burntsushi.net/ripgrep/
| Ashanmaril wrote:
| There's a comparison to similar tools in the readme
|
| https://github.com/BurntSushi/ripgrep#quick-examples-compari...
| HyperCrowd wrote:
| I'd like to know this, too
| simonw wrote:
| ripgrep is absolutely fantastic software. If you're not using it
| yet I guarantee it's worth taking the time to install and learn
| basic usage.
|
| On macOS: "brew install ripgrep" - then use "rg searchterm" to
| search all nested files in the current directory that contain
| searchterm.
|
| You may be using it without realizing already: the code search
| feature in VS Code uses ripgrep under the hood.
| cute_boi wrote:
| actually most of the time you might want to use rg -i
| searchterm . I don't know why its not default? May be they want
| to make it similar to grep but with perf?
| simonw wrote:
| -i is ignore-case, right?
|
| Usually I'm searching code for things like variable names, so
| actually case-sensitive searches are a better default for me.
| raziel2p wrote:
| --smart-case is worth looking into as well. you can also put
| whatever arguments you want in your ripgreprc file to tweak
| the "defaults" to your liking.
| bostonvaulter2 wrote:
| I think that's a bit of a personal choice. I definitely
| prefer the current default of an exact match including case.
| Partially because I'm very familiar with the common `-i` flag
| to ignore case which is used in `grep`, `git grep`, `ack`,
| `ag` and probably others. But I've never used a flag to make
| a search case-sensitive. Plus it's easy enough to alias `rg`
| to `rg -i` if you prefer that as the default.
| twic wrote:
| The other flag to know about is --no-ignore, or alternatively
| -u/--unrestricted.
|
| By default, ripgrep does not search a bunch of files, in
| particular anything gitignored and anything with a leading
| dot (which the docs call "hidden"):
|
| https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#a.
| ..
|
| These flags override that.
| Hamuko wrote:
| I almost never use -i. I may do rg "[Dd]efault" if I want to
| search for "default" or "Default", but not "DEFAULT". Usually
| I just want to search in a very specific case since I'm
| looking for variables.
| burntsushi wrote:
| It's not the default because I personally find it pretty
| unintuitive, and while I don't have any numbers, I think most
| others do too. When I'm comparing ripgrep with other tools
| that enable _smart_ case (like -i, but is disabled if there
| are any literal capital characters in your pattern) by
| default, I _still_ get tripped by it because I forget about
| it.
|
| On Unix, you can just do `alias rg="rg -S"`, and you get
| smart-case by default. :-) Or if you truly always want case
| insensitive by default, then `alias rg="rg -i"`.
|
| "make ripgrep similar to grep" is certainly _one_ goal, but
| there are other goals that compete with that and whether or
| not ripgrep matches grep 's behavior with respect to case
| sensitivity is only one small piece of that.
|
| It was proposed to enable --smart-case by default:
| https://github.com/BurntSushi/ripgrep/issues/178
| e12e wrote:
| Oh, I hope the current default stays!:)
|
| I usually know when I want case insensitive match - and I
| frequently _want_ case-sensitivity with all lowercase
| search terms. Eg: doing a search for methods or variables
| in ruby - where "model" is right, but "Model" is a class
| or module (and vice-versa - but then smart case would
| work).
| burntsushi wrote:
| Don't worry, it will. :-)
|
| While I do allow for breaking changes between major
| versions, I don't really see myself ever doing any kind
| of major breaking change. Especially one that subtly
| changes match semantics in a really major way.
| buerkle wrote:
| ripgrep also has a configuration file for options without
| the need for an alias. https://github.com/BurntSushi/ripgre
| p/blob/master/GUIDE.md#c...
| skrtskrt wrote:
| As mentioned elsewhere, using --hidden which now has a
| shorthand -. is a good default configuration to add via the
| .ripgreprc.
|
| I wouldn't want to miss anything on my home directory or
| repository-specific dotfiles.
| handrous wrote:
| This would be great, except I rarely--bordering on never--
| want to search .git, and when I do that's the _only_ thing I
| want to search.
| fn-mote wrote:
| Yes but I frequently would like to find files in .gitignore
| (e.g., where is this mess inside generated source code).
|
| Edit: oh, this is different from --no-ignore.
| kzrdude wrote:
| I just have a separate alias for rg --no-ignore
| piinbinary wrote:
| I don't see it listed in the notes, but using the --sort-files
| flag no longer seems to have a performance penalty.
| burntsushi wrote:
| It still disables parallelism. No changes there. If you have
| anneasily reproducible benchmark on a public corpus, I could
| try to provide an analysis. (Note: if the corpus is small
| enough, it might not be possible to witness a perf difference.
| Or it may even be possible that using parallelism is slower due
| to the overhead of spinning up threads.)
| e12e wrote:
| Any idea how this works compared to eg GNU sort --parallel?
| (or clever tricks with partial sort and merge)?
|
| I'm guessing rg can be faster in general - because of less
| memory allocation/copying by sorting before outputting?
| burntsushi wrote:
| I'm not familiar with GNU sort's --parallel flag.
|
| That --sort-files disables parallelism is not really a
| theoretical limitation.
|
| See: https://github.com/BurntSushi/ripgrep/issues/152
| piinbinary wrote:
| Interesting!
|
| Perhaps the difference is then explained by my choice of
| search term. The term I tried after upgrading must happen to
| appear early in the sorted corpus.
|
| I just now tried it with a very rare term, and it does indeed
| take longer overall to complete the search.
| boredprograming wrote:
| How does ripgrep compare to Hyperscan? I always wanted to wrap
| that library in a grep utility and never got around to it
| burntsushi wrote:
| ripgrep's internals are generic over the regex engine, so not
| only is it possible to plug Hyperscan into ripgrep, but someone
| has already done it: https://sr.ht/~pierrenn/ripgrep/
|
| (I should really add a link to that in the README.)
| killercup wrote:
| Ripgrep includes one of the most prominent algorithms from
| Hyperscan internally for some expressions.
|
| Longer story: Ripgrep uses Rust's regex library, which uses the
| Aho-Corasick library. That does not just provide the algorithm
| it is named after, but also "packed" ones using SIMD, including
| a Rust rewrite of the [Teddy algorithm][1] from the Hyperscan
| project.
|
| [1]: https://github.com/BurntSushi/aho-
| corasick/tree/4499d7fdb41c...
| kevincox wrote:
| This post by the author is a great introduction to the
| techniques used in ripgrep
| https://blog.burntsushi.net/ripgrep/
| boredprograming wrote:
| That's awesome!
| infocollector wrote:
| I've been using ugrep these days
| (https://github.com/Genivia/ugrep) - and am pretty happy with it.
| nn3 wrote:
| ripgrep always stroke me like the "industrial strength faberge
| egg" of Knuth/McIlroy fame [1]
|
| Yes it's an impressive achievement to make it that fast, but you
| could get the same or better performance by just using an index
| to search. I've been using GNU id-utils [2] for a long time, that
| is using an index, and it gets comparable performance for a
| fraction of the source code and brain power, and likely energy
| use too.
|
| [1] http://blobthescientist.blogspot.com/2017/10/knuth.html
|
| [2] https://www.gnu.org/software/idutils/
| burntsushi wrote:
| The comparison doesn't really make much sense IMO. Searching
| with and without indexing is targeting two very different use
| cases. I wanted to try idutils and see if it suffered from
| pitfalls (like if the indexing scanner didn't grab a token) or
| how it handled regexes. But I couldn't get it compiled using
| these commands[1]. It failed with: gcc
| -DHAVE_CONFIG_H -I. -D_FORTIFY_SOURCE=2 -march=x86-64
| -mtune=generic -O2 -pipe -fno-plt -MT fseterr.o -MD -MP -MF
| .deps/fseterr.Tpo -c -o fseterr.o fseterr.c fseterr.c:
| In function 'fseterr': fseterr.c:74:3: error: #error
| "Please port gnulib fseterr.c to your platform! Look at the
| definitions of ferror and clearerr on your system, then report
| this to bug-gnulib." 74 | #error "Please port
| gnulib fseterr.c to your platform! Look at the definitions of
| ferror and clearerr on your system, then report this to bug-
| gnulib." | ^~~~~ make[3]: ***
| [Makefile:1590: fseterr.o] Error 1
|
| So I can't even easily get idutils to try it, but suspicion is
| that it misses some key use cases. And from what I can tell,
| its index doesn't auto-update, so you now have to use your
| brain power to remember to update the index. (Or figure out how
| to automate it.)
|
| ripgrep is commonly used in source code repos, where you might
| be editing code or checking out different branches all the
| time. An indexing tool in that scenario without excellent
| automatic updates is a non-starter.
|
| [1] -
| https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=iduti...
| synergy20 wrote:
| I use fd and ripgrep 90% of the time before I need return to
| find/grep these days.
| flakiness wrote:
| Haven't tried ripgrep before but finally installed. It is
| noticeably faster than older tools and more importantly the CLI
| ergonomics is much better! Thanks!
| mfrw wrote:
| on a tangential note: there is a bunch of tools re-written in
| rust, a handy non exhaustive list [1].
|
| [1]: https://zaiste.net/posts/shell-commands-rust/
| tspiteri wrote:
| I love Rust, but I love and recommend ripgrep because of its
| usefulness, reliability and robustness, not because of the
| language it's written in.
|
| I haven't looked into all the tools in that list, but I would
| not for example recommend exa over ls as it is simply not
| reliable enough: if a filename ends in a space, you won't see
| that in exa, and that bug has been reported for years. To me
| that is a clear blocker, and if it is still there, I simply
| cannot trust the file listing from exa, no matter how pretty
| it may look.
| ibraheemdev wrote:
| Here's a hopefully more exhaustive list that isn't restricted
| to Rust: https://github.com/ibraheemdev/modern-unix, meaning
| it can include other awesome tools like fzf (written in Go).
| mgradowski wrote:
| If you like fzf, give skim [1] a go. It's basically faster
| (anecdotally) fzf, but can also be used as a Rust library.
| I made a little personal to-do manager using it, together
| with SQLite for persistence. I also improved my fish shell
| ergonomics with [2].
|
| [1] https://github.com/lotabout/skim [2]
| https://github.com/zenofile/fish-skim
| empiricus wrote:
| For SSD, if the read queue depth is big, the performance improves
| significantly compared to sequential reads. Does ripgrep has an
| option to read many files in parallel?
| burntsushi wrote:
| ripgrep uses parallelism by default.
|
| Parallelism applies both to directory traversal, gitignore
| filtering and search. But that's where its granularity ends. It
| does not use parallelism at the level of single-file search.
| nix23 wrote:
| Can someone help me? Why no use "grep -r"? Is it faster or better
| suited for recursive?
| lalaithion wrote:
| https://github.com/BurntSushi/ripgrep#why-should-i-use-ripgr...
| rakenodiax wrote:
| Check out this post for an in-depth analysis:
| https://blog.burntsushi.net/ripgrep/
| nix23 wrote:
| Thanks allot! Made it clear, HN is a big shit since 2 years,
| i might have to say goodby someday, one asks a neutral
| question an gets downvoted...what a fucking wannabe elite
| mindset.
| kzrdude wrote:
| ripgrep is just one in the line of "more ergonomic greps", for
| example ack and ag were popular before ripgrep
| atonse wrote:
| I didn't know this was the same author as xsv, a tool I find
| indispensable now.
|
| Going to install ripgrep ASAP
| InspiredIdiot wrote:
| Yeah. It almost seems unfair but I'll just use both and be glad
| they are available.
| imiric wrote:
| As a ripgrep user, TIL about xsv. Thanks for mentioning it,
| definitely one for the toolbelt.
|
| Can't believe I wasn't following BurntSushi on GitHub, what a
| track record.
| gigatexal wrote:
| I love ripgrep. It's insanely fast. And very useful to my
| workflow.
| sbergot wrote:
| Fun trivia: vscode uses ripgrep for its search feature
___________________________________________________________________
(page generated 2021-06-14 23:00 UTC)