https://zwischenzugs.com/2022/02/02/grep-flags-the-good-stuff/ Skip to content [4f37af910d] zwischenzugs grep Flags - The Good Stuff zwischenzugs Uncategorized February 2, 2022 5 Minutes While writing a post on practical shell patterns I had a couple of patterns that used grep commands. I had to drop those patterns from the post, though, because as soon as I thought about them I got lost in all the grep flags I wanted to talk about. I realised grep deserved its own post. grep is one of the most universal and commonly-used commands on the command line. I count about 50 flags you can use on it in my man page. So which ones are the ones you should know about for everyday use? I tried to find out. I started by asking on Twitter whether the five flags I have 'under my fingers' and use 99% of the time are the ones others also use. What are your top 5 most used grep flags? Mine are -r -i -l (lowercase L) -I (capital i) -v -- Ian Miell (@ianmiell) December 27, 2021 It turns out experience varies widely on what the 5 most-used are. Here are the results, in no particular order, of my researches. I've tried to categorize them to make it easier to digest. The categories are: * ABC - The Context Ones * What To Match? * What To Report? * What To Read? * Honourable Mention If you think any are missing, let me know in the comments below. ABC - The Context Ones These arguments give you more context around your match. grep -A grep -B grep -C I hadn't included these in my top five, but as soon as I was reminded of them, -C got right back under my fingertips. I call them the 'ABC flags' to help me remember them. Each of these gives a specified context around your grep'd line. -A gives you lines after the match, -B gives you lines before the match, and -C (for 'context') gives you both the before and after lines. $ mkdir grepflags && cd grepflags $ cat > afile < afile < do echo "top of: $f" > head $f > done SHOUT shout let it all out which outputs the heads of all files in the local folder except any files with README in their names. grep -w The -w flag only matches 'whole-word' matches, ignoring cases where submitted words are part of longer words. This is a useful flag to narrow down your matches, and also especially useful when searching through prose: $ cat > afile < afile1 << EOF a EOF $ cp afile1 afile2 $ grep a * afile1:a afile2:a $ grep -h a * a a This is particularly useful if you want to process the matching lines without the filename spoiling the input. Compare these to the output without the -h. $ grep -h a * | uniq a $ grep -h a * | uniq -c 2 grep -o This outputs only the text specified by your regular expression. One match is output per line, but multiple matches may be made per line. This can result in more matches than lines, as in the example below, where you look for words that end lines that end in 'ay', and then any words with the letter 'e' in them (but not at the start or the end of the word). $ rm -f afile1 afile2 $ cat > afile << EOF Yesterday All my troubles seemed so far away Now it looks as though they're here to stay Oh I believe In yesterday EOF $ grep -o ' [^ ]*ay$' afile away stay yesterday $ grep -o ' [^ ]*e[^ ]*' afile troubles seemed they're here believe yesterday grep -l If you're fighting through a blizzard of output and want to focus only on which files your matches are in rather than the matches themselves, then using this flag will show you where you might want to look: $ cat > afile << EOF a a EOF $ cp afile afile2 $ cp afile afile3grep -l $ grep a * afile:a afile:a afile2:a afile2:a afile3:a afile3:a $ grep -l a * afile afile2 afile3 What To Read? These flags change which files grep will look at. grep -r A very popular flag, this flag recurses through the filesystem looking for matches. $ grep -r securityagent /etc grep -I This one is my favourite, as it's incredibly useful, and not so well known, despite being widely applicable. If you've ever been desperate to find where a string is referenced in your filesystem (usually as root) and run something like this: $ grep -rnwi specialconfig / then you won't have failed to notice that it can take a good while. This is partly because it's looking at every file from the root, whether it's a binary or not. The -I flag only considers text files. This radically speeds up recursive greps. Here we run the same command twice (to ensure it's not only slow the first time due to OS file cacheing), then run the command with the extra flag, and see a nearly 50% speedup. $ time sudo grep -rnwi specialconfig / 2>/dev/null sudo grep -rnwi specialconfig / 418.01s user 382.19s system 70% cpu 19:03.09 total $ time sudo grep -rnwi specialconfig / sudo grep -rnwi specialconfig / 434.19s user 411.62s system 70% cpu 19:56.25 total $ time sudo grep -rnwiI specialconfig / sudo grep -rnwiI specialconfig / 33.54s user 322.64s system 52% cpu 11:19.03 total Honourable mention There are many other grep flags, but I'll just add one honourable mention at the end here. grep -E I spent an embarrassingly long time trying to get regular expressions with + signs in them to work in grep before I realised that default grep didn't support so-called 'extended' regular expressions. By using -E you can use those regular expressions just as the regexp gods intended. $ cat > afile <<< aaaaaa+bc $ grep -o 'a+b' a # + is treated literally a+b $ grep -o 'aa*b' a # a workaround aaaaaaaaaab $ grep -oE 'a+b' a # extended regexp aaaaaaaaaab --------------------------------------------------------------------- If you like this, you might like one of my books: Learn Bash the Hard Way Learn Git the Hard Way Learn Terraform the Hard Way LearnGitBashandTerraformtheHardWayBuy in a bundle here --------------------------------------------------------------------- If you enjoyed this, then please consider buying me a coffee to encourage me to do more. [bmc-button] Share this: * Email * * * Tweet * Share on Tumblr * Pocket * * Like this: Like Loading... Related [4f37af910d] Published by zwischenzugs View all posts by zwischenzugs Published February 2, 2022 Post navigation Previous Post Why It's Great To Be A Consultant 2 thoughts on "grep Flags - The Good Stuff" 1. Pingback: Grep Flags - The Good Stuff - The web development company Lzo Media - Senior Backend Developer 2. [558ea] Georg says: February 2, 2022 at 7:01 pm Thanks :) Mine are: -e # for OR-ing -- still miffed there's no good order-independent AND equivalent and more and more using awk in pipes for that reason (awk '/a/ && /b/' is just so much easier than -e 'a.*b' -e 'b.*a' the more complex a and b get ; and once you need fields from the grepped lines anyway...) -v # more and more often in combination with many -e -E # mostly when I need something involving '|' but more complex than just using two '-e's -C / -A / -B # -C was a revelation when, after literally years of "-A n -B n", I (re-)read the man page properly :-D -i Cheers, Georg PS: just discovered your Blog; off to read some more :-) Reply Leave a Reply Cancel reply Enter your comment here... [ ] Fill in your details below or click an icon to log in: * * * * Gravatar Email (required) (Address never made public) [ ] Name (required) [ ] Website [ ] WordPress.com Logo You are commenting using your WordPress.com account. ( Log Out / Change ) Google photo You are commenting using your Google account. ( Log Out / Change ) Twitter picture You are commenting using your Twitter account. ( Log Out / Change ) Facebook photo You are commenting using your Facebook account. ( Log Out / Change ) Cancel Connecting to %s [ ] Notify me of new comments via email. [ ] Notify me of new posts via email. [Post Comment] [ ] [ ] [ ] [ ] [ ] [ ] [ ] D[ ] This site uses Akismet to reduce spam. Learn how your comment data is processed. Follow me on Twitter My Tweets Top Posts & Pages * grep Flags - The Good Stuff * Practical Shell Patterns I Actually Use * Five Ansible Techniques I Wish I'd Known Earlier * Ten Things I Wish I'd Known Before Using Jenkins Pipelines * Bash to Python Converter * Why It's Great To Be A Consultant * A 'Hello World' GitOps Example Walkthrough * How (and Why) I Run My Own DNS Servers * Ten Things I Wish I'd Known About bash * Anatomy of a Linux DNS Lookup - Part I Recent Posts * grep Flags - The Good Stuff * Why It's Great To Be A Consultant * Practical Shell Patterns I Actually Use * Why I Keep Coming Back to Cynefin * Is Agility Related to Commitment? - Money Flows Part II * Five Ansible Techniques I Wish I'd Known Earlier * A 'Hello World' GitOps Example Walkthrough * If You Want To Transform IT, Start With Finance * How To Waste Hundreds of Millions on Your IT Transformation * When Should I Interrupt Someone? * An Incompetent Newbie Takes Up 3D Printing * GitOps Decisions * Five Ways to Undo a Commit in Git * The Halving of the Centre: Covid and its Effect on London Property * Why Do We Have Dev Rels Now? * The Runbooks Project * Some Relatively Obscure Bash Tips * Riding the Tiger: Lessons Learned Implementing Istio * The Astonishing Prescience of Nam June Paik * Notes on Books Read in 2019 * The First Non-Bullshit Book About Culture I've Read * Why Everyone Working in DevOps Should Read The Toyota Way * Surgically Busting the Docker Cache * Software Security Field Guide for the Bewildered * The Lazy Person's Guide to the Info Command * A Hot Take on GitHub Actions * Seven God-Like Bash History Shortcuts You Will Actually Use * How Long Will It Take For The Leavers To Leave? * Goodbye Docker: Purging is Such Sweet Sorrow * Seven Surprising Bash Variables * The Missing Readline Primer * Apple's HQ, Ruskin, Gothic Architecture, and Agile * Eight Obscure Bash Options You Might Want to Know About * 'AWS vs K8s' is the new 'Windows vs Linux' * Pranking the Bash Binary * Bash Startup Explained * Git Hooks the Hard Way * Notes on Books Read in 2018 * Six Ways to Level Up Your nmap Game * Five Things I Wish I'd Known About Git * Eleven bash Tips You Might Want to Know * Learn Bash Debugging Techniques the Hard Way * Why Are Enterprises So Slow? * Anatomy of a Linux DNS Lookup - Part V - Two Debug Nightmares * Anatomy of a Linux DNS Lookup - Part IV * Anatomy of a Linux DNS Lookup - Part III * Anatomy of a Linux DNS Lookup - Part II * Anatomy of a Linux DNS Lookup - Part I * A Docker Image in Less Than 1000 Bytes * Autotrace - Debug on Steroids * Beyond 'Punk Rock Git' in Eleven Steps * Sandboxing Docker with Google's gVisor * Unprivileged Docker Builds - A Proof of Concept * Learn Git Rebase Interactively * Terminal Perf Graphs in one Command * git log - the Good Parts * Five Key Git Concepts Explained the Hard Way * Create Your Own Git Diagrams * Five Things I Did to Change a Team's Culture * Centralise Your Bash History * How (and Why) I Run My Own DNS Servers * Ten More Things I Wish I'd Known About bash * Download a Free Sample of Learn Bash the Hard Way * Ten Things I Wish I'd Known About bash * Project Management as Code with Graphviz * How to Manually Clear Locks in Jenkins * How I Manage My Time * Ten Things I Wish I'd Known About Chef * Vagrant and Ohai / Chef IP Address Hack * 'Towards a National Computer Grid' - Electronic Computers, 1965 * A Complete Chef Infrastructure on Your Laptop * Ten Things I Wish I'd Known Before Using Vagrant * A Checklist for Docker in the Enterprise (Updated) * OpenShift 3.6 DNS In Pictures * Puppeteer - Headless Chrome in a Container * My 20-Year Experience of Software Development Methodologies * A Non-Cloud Serverless Application Pattern Using Git and Docker * Run Your Own AWS APIs on OpenShift * Dockerized Headless Chrome Example * Convert a Server to a Docker Container (Update II) * Automating Dockerized Jenkins Upgrades * Ten Things I Wish I'd Known Before Using Jenkins Pipelines * Five Books I Advise Every DevOps Engineer to Read * Things I Learned Managing Site Reliability for Some of the World's Busiest Gambling Sites * Clustered VM Testing How-To * Easy Shell Automation * 1-Minute Multi-Node VM Setup * Migrating an OpenShift etcd Cluster * A Complete OpenShift Cluster on Vagrant, Step by Step * Learn Kubernetes the Hard Way (the Easy and Cheap Way) * Docker in the Enterprise * Terraform and Dynamic Environments * Bash to Python Converter * Hello world Unikernel Walkthrough * A checklist for Docker in the Enterprise * A Quick Tour of Docker 1.12 * Power 'git log' graphing * ssh -R (reverse tunnel) man page hell * Writing a Technical Book * Interactive Git Tutorials - Rebase and Bisect * Hitler Uses Docker, Annotated * Linux Scales * Play With Kubernetes Quickly Using Docker (Updated) * Convert Any Server to a Docker Container (Updated) * CI as Code Part III: Dynamic Jenkins-Swarm Example * Docker 1.10 Highlights - Updated * CI as Code Part II: Stateless Jenkins With Dynamic Docker Slaves * CI as Code Part I: Stateless Jenkins Deployments Using Docker * Docker Ecosystem Rosetta Stones * Understanding Docker - A Tour of Logical Volume Management * Automating Docker Security Validation * The IT Crowd Was Right - What I learned by reading a lot of RFCs * Understanding Docker - Network Namespaces * DockerConEU 2015 Talk - You Know More Than You Think * Docker Migration In-Flight CRIU * A High Availability Phoenix and A/B Deployment Framework using Docker * Quick Intro to Kubernetes * Take OpenShift for a spin in four commands * RedHat's Docker Build Method - S2I * RedHat's Docker Build Method - S2I * Bash Shortcuts Gem * A CoreOS Cluster in Two Minutes With Four Commands * The Most Pointless Docker Command Ever * My Favourite Docker Tip * Convert Any Server to a Docker Container * A Field Guide to Docker Security Measures * Docker SELinux Experimentation with Reduced Pain * Storage Drivers and Docker * Play With Kubernetes Quickly Using Docker * Play with an OpenShift PaaS using Docker * Scale Your Jenkins Compute With Your Dev Team: Use Docker and Jenkins Swarm * Docker in Practice - A Guide for Engineers * Fight Docker Package Drift! * Win at 2048 with Docker and ShutIt (Redux) * Set Up a Deis (Docker-Friendly) Paas on Digital Ocean for $0.18 Per Hour in Six Easy Steps Using ShutIt * Create your own CoreOS cluster in 6 easy steps for $0.03 * Make Your Own Bespoke Docker Image * Taming Slaves with Docker and ShutIt * Docker - One Year On * Using ShutIt to Build Your Own Taiga Server * Using ShutIt and Docker to play with AWS (Part Two) * Talk on Docker and ShutIt * Using ShutIt and Docker to play with AWS (Part One) * Phoenix deployment pain (and win) * Phoenix Deployment with Docker and ShutIt * Docker, ShutIt and the Perfect 2048 Game (Videos) * Docker, ShutIt and the Perfect 2048 Game (4 - Halfway There) * Docker, ShutIt and the Perfect 2048 Game (3 - Brute Force Escapes) * Docker, ShutIt and the Perfect 2048 Game (2) * Docker, ShutIt, and The Perfect 2048 Game Follow zwischenzugs on WordPress.com Website Built with WordPress.com. * Follow Following + [wpcom-] zwischenzugs Join 229 other followers [ ] Sign me up + Already have a WordPress.com account? Log in now. * + [wpcom-] zwischenzugs + Customize + Follow Following + Sign up + Log in + Copy shortlink + Report this content + View post in Reader + Manage subscriptions + Collapse this bar Loading Comments... Write a Comment... [ ] Email (Required) [ ] Name (Required) [ ] Website [ ] [Post Comment] Send to Email Address [ ] Your Name [ ] Your Email Address [ ] [ ] loading [Send Email] Cancel Post was not sent - check your email addresses! Email check failed, please try again Sorry, your blog cannot share posts by email. %d bloggers like this: [b]