[HN Gopher] Fast regex search: indexing text for agent tools
       ___________________________________________________________________
        
       Fast regex search: indexing text for agent tools
        
       Author : jxmorris12
       Score  : 19 points
       Date   : 2026-03-24 06:31 UTC (2 days ago)
        
 (HTM) web link (cursor.com)
 (TXT) w3m dump (cursor.com)
        
       | mpalmer wrote:
       | > No matter how fast ripgrep can match on the contents of a file,
       | it has one serious limitation: it needs to match on the contents
       | of all files.
       | 
       | The omission of rg's `-g` parameter is unsurprising in one sense,
       | because it would mostly obviate this entire exercise. How often
       | do you need to search what sounds like hundreds of millions of
       | lines of source for a complex pattern, with zero constraints on
       | paths searched?
       | 
       | > We routinely see rg invocations that take more than 15 seconds
       | 
       | I'm trying to understand the monorepo that is so large that
       | ripgrep takes 15 seconds to return results, when it's benchmarked
       | as searching for a literal in a 9.3GB file in 600ms, or 1.08s to
       | search for `.*` in the entire Linux repo.
       | 
       | And again, that's without using `-g`.
        
         | piker wrote:
         | > -g GLOB, --glob=GLOB
         | 
         | > Include or exclude files and directories for searching that
         | match the given glob. This always overrides any other ignore
         | logic. Multiple glob flags may be used. Globbing rules match
         | .gitignore globs. Precede a glob with a ! to exclude it. If
         | multiple globs match a file or directory, the glob given later
         | in the command line takes precedence. As an extension, globs
         | support specifying alternatives: -g 'ab{c,d}*' is equivalent to
         | -g abc -g abd. Empty alternatives like -g 'ab{,c}' are not
         | currently supported. Note that this syntax extension is also
         | currently enabled in gitignore files, even though this syntax
         | isn't supported by git itself. ripgrep may disable this syntax
         | extension in gitignore files, but it will always remain
         | available via the -g/--glob flag.
         | 
         | > When this flag is set, every file and directory is applied to
         | it to test for a match. For example, if you only want to search
         | in a particular directory foo, then -g foo is incorrect because
         | foo/bar does not match the glob foo. Instead, you should use -g
         | 'foo/*'.
         | 
         | https://man.archlinux.org/man/rg.1.en*
         | 
         | (for those who were unfamiliar with the switch like me)
        
         | mpalmer wrote:
         | Has the Cursor team considered (for instance) that on MDM-
         | managed machines, binaries which do pervasive arbitrary FS
         | reads may be monitored, throttled, or otherwise controlled by
         | overseer programs? That kitchen-sink Electron apps like Cursor
         | using those binaries might compound the red-flag signals?
        
       | open-paren wrote:
       | The creator of fff.nvim[0], Dmitriy Kovalenko, had an interesting
       | analysis of this on Xitter[1]. The TL;DR of this is that
       | Anysphere/Cursor is being somewhat disingenuous and does not
       | include the index-creation and recreation time in the comparison
       | nor do they include the CPU or memory overhead, where rg (and his
       | tool, fff.nvim) are indexless.
       | 
       | ---
       | 
       | 0: https://github.com/dmtrKovalenko/fff.nvim
       | 
       | 1: http://x.com/i/article/2036558670528651264
        
       ___________________________________________________________________
       (page generated 2026-03-26 23:00 UTC)