https://blog.robertelder.org/intro-to-tail-command/ Robert Elder Software Inc. * Home * Blog * Contact * Home * Blog * Contact * #linux * | * #commandline * | * #compilers * | * #softwareengineering * | * #cprogramming * ... * View All >> Use The 'tail' Command To Monitor Everything 2021-04-08 - By Robert Elder Introduction This article is about how to use the 'tail' command. Here is an overview of how you can use the 'head' and 'tail' commands to output different parts of a file: Head Command Tail Command Use The Following Commands to output the first 3 Use The Following Commands to output the last lines of the file/steam: 3 lines of the file/steam. head -n 3 business-tips.txt # (POSIX Compatible) tail -n 3 business-tips.txt # (POSIX Compatible) # OR # OR head -n +3 business-tips.txt # (Not POSIX Compatible) tail -n -3 business-tips.txt # (POSIX Compatible) business-tips.txt (green lines will be printed as business-tips.txt (green lines will be printed as output) output) 1) Buy low, sell high. 1) Buy low, sell high. 2) Invest with a plan. 2) Invest with a plan. 3) Follow your heart. 3) Follow your heart. 4) Find your passion. 4) Find your passion. 5) Believe in yourself. 5) Believe in yourself. 6) Take risks, be bold. 6) Take risks, be bold. 7) Play it safe, and be considerate. 7) Play it safe, and be considerate. 8) Realize that there are no limits. 8) Realize that there are no limits. 9) Realize that you have limits. 9) Realize that you have limits. 10) You can rest when you're dead. 10) You can rest when you're dead. 11) Take time to care for yourself. 11) Take time to care for yourself. 12) Ignore what negative people say. 12) Ignore what negative people say. 13) Listen to constructive criticism. 13) Listen to constructive criticism. Use The Following Command to output the 'head' or Use The Following Command to output the 'tail' beginning of the file/stream with the last 3 lines end of the file/stream starting with the 3rd line omitted: continuing until the end: head -n -3 business-tips.txt # (Not POSIX Compatible) tail -n +3 business-tips.txt # (POSIX Compatible) business-tips.txt (green lines will be printed as business-tips.txt (green lines will be printed as output) output) 1) Buy low, sell high. 1) Buy low, sell high. 2) Invest with a plan. 2) Invest with a plan. 3) Follow your heart. 4) Find your passion. 3) Follow your heart. 5) Believe in yourself. 4) Find your passion. 6) Take risks, be bold. 5) Believe in yourself. 7) Play it safe, and be considerate. 6) Take risks, be bold. 8) Realize that there are no limits. 7) Play it safe, and be considerate. 9) Realize that you have limits. 8) Realize that there are no limits. 10) You can rest when you're dead. 9) Realize that you have limits. 10) You can rest when you're dead. 11) Take time to care for yourself. 11) Take time to care for yourself. 12) Ignore what negative people say. 12) Ignore what negative people say. 13) Listen to constructive criticism. 13) Listen to constructive criticism. The table above shows the different ways that you can use the 'head' and 'tail' commands to output the beginning and end of a file. You can get different results by passing a positive or negative number to these commands, although not all combinations are POSIX compliant. Therefore, they may not be supported in all implementations of head/tail. In the examples shown above, the green colour text is what would get printed, and the gray text is the part that would be omitted. The Simplest Use Of The 'tail' Command You can use the 'tail' command to print out the 'tail-end' of a file or stream. If we start with the following file 'business-tips.txt': 1) Buy low, sell high. 2) Invest with a plan. 3) Follow your heart. 4) Find your passion. 5) Believe in yourself. 6) Take risks, be bold. 7) Play it safe, and be considerate. 8) Realize that there are no limits. 9) Realize that you have limits. 10) You can rest when you're dead. 11) Take time to care for yourself. 12) Ignore what negative people say. 13) Listen to constructive criticism. and run the tail command like this: tail business-tips.txt We'll get the following output: 4) Find your passion. 5) Believe in yourself. 6) Take risks, be bold. 7) Play it safe, and be considerate. 8) Realize that there are no limits. 9) Realize that you have limits. 10) You can rest when you're dead. 11) Take time to care for yourself. 12) Ignore what negative people say. 13) Listen to constructive criticism. which shows the last 10 lines in the file. The default number of lines to display is 10, but you can use the '-n' flag to specify a different number. For example, this command: tail -n 5 business-tips.txt will output the following: 9) Realize that you have limits. 10) You can rest when you're dead. 11) Take time to care for yourself. 12) Ignore what negative people say. 13) Listen to constructive criticism. which shows the last 5 lines in the file. Tail's 'Follow' Mode One of the most common uses for the tail command is it's "follow" mode that you get when using the '-f' flag. If we run a command like this: tail -f /var/log/kern.log you'll see the last few lines from the file '/var/log/kern.log' printed to the screen, and the cursor will appear to 'hang' as if it was waiting for something: Tail Command -f Flag In this case, the tail command is waiting for more lines to be appended to the end of the file so it can print them to the screen. The tail command will keep watching for new lines until you interrupt 'tail' by pressing and holding the CTRL key followed by the 'c' key. Using the tail command with the '-f' flag is a common way to monitor log files. By default, the 'tail' command will output a few lines that were already at end of the file even when you use the '-f' flag. This can be confusing if you're only interested in monitoring for log statements that happened after you issued the tail command. Therefore, I like to include the '-n' flag with a value of 0 whenever I use '-f': tail -f -n 0 /var/log/apache2/access.log This way, you won't get distracted by outdated information that was already in the file when you issued this tail command. You can also follow multiple files at once with the 'tail' command like this: tail -f -n 0 /var/log/apache2/access.log /var/log/apache2/errors.log After issuing the above command, tail will output any of the lines that are written to any of the files. When multiple files are followed at once, 'tail' will usually print out a 'header' that indicates which file the corresponding lines came from: ==> /var/log/apache2/access.log <== ==> /var/log/apache2/errors.log <== ==> /var/log/apache2/access.log <== Hello from access log! ==> /var/log/apache2/errors.log <== Hello from error log! Most shells also support wildcards that make it even more convenient to specify a large number of files to follow at once with 'tail': tail -f -n 0 /var/log/apache2/* Following Files That Require Elevated Privileges This is a very specific, but quite common scenario that might cause you some confusion with the 'tail' command: What if you want to use a wildcard to follow multiple files that are in a directory that is only readable by another user? You might think 'oh, I'll just use sudo' and do something like this: sudo tail -f -n 0 /var/log/apache2/* But this doesn't work! You'll still get the permission issue since your shell will attempt to expand the wildcard (the '*' part) before rest of the command is even passed to sudo. The exact solution will depend on which shell you use, but for the bash shell the following will work: sudo sh -c 'tail -f -n 0 /var/log/apache2/*' Use 'tail' To Find The 4 Most Recently Copied Files Let's say you're in the process of backing up some files to an external hard drive, and you'd like to see the progress of the last few files that were copied. You could use the following 'ls' command to list all files in the current directory, with the most recently modified file shown last: ls -ltr which outputs something like this: ...many many lines... -rw-r--r-- 1 robert robert 8839640 Feb 27 00:30 my-cool-file-30356.dat -rw-r--r-- 1 robert robert 194326506 Feb 27 00:30 my-cool-file-3093.dat -rw-r--r-- 1 robert robert 665529770 Feb 27 00:30 my-cool-file-3396.dat -rw-r--r-- 1 robert robert 629536792 Feb 27 00:31 my-cool-file-5005.dat -rw-r--r-- 1 robert robert 987631920 Feb 27 00:31 my-cool-file-7551.dat -rw-r--r-- 1 robert robert 654947606 Feb 27 00:31 my-cool-file-7606.dat -rw-r--r-- 1 robert robert 1294157424 Feb 27 00:31 my-cool-file-8633.dat -rw-r--r-- 1 robert robert 29884416 Feb 27 00:31 my-cool-file-8673.dat but this shows too much information. It would look nicer if we could see only 4 of the most recently modified files. We can do that using the tail command: ls -ltr | tail -n 4 and now we get the following output: -rw-r--r-- 1 robert robert 987631920 Feb 27 00:31 my-cool-file-7551.dat -rw-r--r-- 1 robert robert 654947606 Feb 27 00:31 my-cool-file-7606.dat -rw-r--r-- 1 robert robert 1294157424 Feb 27 00:31 my-cool-file-8633.dat -rw-r--r-- 1 robert robert 29884416 Feb 27 00:31 my-cool-file-8673.dat But it's inconvenient to manually run this over and over. We can also leverage the 'watch' command to run this command over and over again so we get to see the file copy progress as it's happening: watch -n 0.1 -d 'ls -latr | tail -n 4' The use of the '-d' flag above will cause 'watch' to highlight the differences, and the '-n' flag will cause the command to run again every 0.1 seconds instead of the default interval of 2 seconds. Counting From Beginning Instead Of End You may encounter situations where you want to extract the 'end' of a file where you don't know in advance how long the 'end' will be. The 'tail' command can still help you in these situations because you can specify an offset from the beginning where you'd like to start the output. For example, let's consider the file 'some_lines.txt' with the following contents: My Favourite Stonks: GME AMC RKT PLTR We can use the 'cat' command with the '-n' flag to add some line numbers for clarity: cat -n some_lines.txt which produces this output: 1 My Favourite Stonks: 2 GME 3 AMC 4 RKT 5 PLTR If we only wanted the last two lines, we would use the tail command like this: cat -n some_lines.txt | tail -n 2 which provides this output: 4 RKT 5 PLTR But, if we add a plus sign in front of the two, we get a different interpretation: cat -n some_lines.txt | tail -n +2 this prints out the 'end' of the file starting at line 2 continuing until the end (regardless of the total number of lines): 2 GME 3 AMC 4 RKT 5 PLTR If we then add some more data to this file: echo "FOO1" >> some_lines.txt echo "FOO2" >> some_lines.txt those lines will now be output too when we issue the same command: cat -n some_lines.txt | tail -n +2 will output: 2 GME 3 AMC 4 RKT 5 PLTR 6 FOO1 7 FOO2 Using 'tail' With Bytes Instead Of Lines You can also use the 'tail' command to extract the last few bytes instead of the last few lines with the '-c' flag. Here is an example of using the tail command to extract the last byte of the file from the previous example: cat -n some_lines.txt | tail -c 1 This doesn't print any visible characters, so let's pipe the result into xxd to see what the byte is: cat -n some_lines.txt | tail -c 1 | xxd and the result is: 00000000: 0a . which shows that the last byte in the file is a newline, just as we'd expect. Let's do this again and extract the last 4 characters: cat -n some_lines.txt | tail -c 4 which provides this output LTR and if we pipe this into xxd: cat -n some_lines.txt | tail -c 4 | xxd we'll see the following: 00000000: 4c54 520a LTR. Finding The 3 Newest Books In the article on the sort command, we reviewed an example of how to find the 3 newest books from the following unsorted list: Tropic of Cancer,Henry Miller,1934 Housekeeping,Marilynne Robinson,1981 Deliverance,James Dickey,1970 The Sun Also Rises,Ernest Hemingway,1926 The Great Gatsby,F. Scott Fitzgerald,1925 The Corrections,Jonathan Franzen,2001 The Berlin Stories,Christopher Isherwood,1946 Call It Sleep,Henry Roth,1935 Slaughterhouse-Five,Kurt Vonnegut,1969 Light in August,William Faulkner,1932 We showed how you can use the sort command to sort each line in the file according according to the three different columns: sort -t ',' -k 3,3n -k 2,2 -k 1,1 -s books.txt which produces this output: The Great Gatsby,F. Scott Fitzgerald,1925 The Sun Also Rises,Ernest Hemingway,1926 Light in August,William Faulkner,1932 Tropic of Cancer,Henry Miller,1934 Call It Sleep,Henry Roth,1935 The Berlin Stories,Christopher Isherwood,1946 Slaughterhouse-Five,Kurt Vonnegut,1969 Deliverance,James Dickey,1970 Housekeeping,Marilynne Robinson,1981 The Corrections,Jonathan Franzen,2001 We can this pipe this directly into the 'tail' command and tell it to only print out the last 3 lines: sort -t ',' -k 3,3n -k 2,2 -k 1,1 -s books.txt | tail -n 3 which produces this output: Deliverance,James Dickey,1970 Housekeeping,Marilynne Robinson,1981 The Corrections,Jonathan Franzen,2001 As you can see above, we've found the 3 newest books from an unsorted list using a combination of the 'sort' and 'tail' commands. Finding The Most Popular Name In the article on the 'uniq' command, we reviewed an example of how to find the most popular name from a list of names: Verity Jayda Verity Jayda Verity Jayda Verity Jayda Justy Kaiden Christopher Rene Christopher Rene Christopher Rene Branden McKenna Branden McKenna the first step was to make sure the list of names was sorted (as a requirement of the 'uniq' command): cat names.txt | sort then, the list of names is piped into the 'uniq' command with the '-c' flag which provides a count of the number of occurrences for each name: cat names.txt | sort | uniq -c and the output of this is: 2 Branden McKenna 3 Christopher Rene 1 Justy Kaiden 4 Verity Jayda Now, we can use the 'sort' command again, but this time use numeric sorting to order the list according to which name has the most occurrences: cat names.txt | sort | uniq -c | sort -n which provides this output: 1 Justy Kaiden 2 Branden McKenna 3 Christopher Rene 4 Verity Jayda Now, we just need to use the 'tail' command to pick out the last line in the output: cat names.txt | sort | uniq -c | sort -n | tail -n 1 and the output of this command is: 4 Verity Jayda And the above name is the most popular name in the list. This process lets go from an unsorted list of many names to a single record that tells what the most popular name is. Potentially Unbounded Memory Use The tail command can potentially experience unbounded memory requirements in some cases. This becomes obvious when you consider the fact that the tail command must be capable of working with streams where the length of the input it not known in advance. If you want to output the last N lines from a stream, you need to be able to store at least N lines in memory so that when you finally detect the end of the stream you'll be able to output what came before it. In the interest of giving some numbers, here's a contrived example that reads a bunch of lines from /dev/urandom. We then use 'head' to grab the first 10,000,000 lines and pipe them into 'tail' to get the last line. This entire pipe is run as a sub-shell through /usr/bin/time so we can get some performance numbers: /usr/bin/time -v sh -c 'xxd /dev/urandom | head -n 10000000 | tail -n -1 | wc -l' which gives some output that includes these lines: Elapsed (wall clock) time (h:mm:ss or m:ss): 0:25.64 Maximum resident set size (kbytes): 2104 If we start increasing the number of lines that tail should output: /usr/bin/time -v sh -c 'xxd /dev/urandom | head -n 10000000 | tail -n -100000 | wc -l' the max memory usage starts to increase: Elapsed (wall clock) time (h:mm:ss or m:ss): 0:25.01 Maximum resident set size (kbytes): 8768 And increase: /usr/bin/time -v sh -c 'xxd /dev/urandom | head -n 10000000 | tail -n -1000000 | wc -l' Elapsed (wall clock) time (h:mm:ss or m:ss): 0:25.06 Maximum resident set size (kbytes): 68656 And increase... /usr/bin/time -v sh -c 'xxd /dev/urandom | head -n 10000000 | tail -n -10000000 | wc -l' Elapsed (wall clock) time (h:mm:ss or m:ss): 0:25.46 Maximum resident set size (kbytes): 668992 From the above output, we can see that a maximum of 668992KiB or 653.312MiB was required by at one time during the running of the above tail command. Obviously, 10,000,000 lines is quite a lot, but you should keep this in mind for cases where the argument to the 'tail' command might be a variable in a script that could be arbitrarily large. A similar problem can also occur with the non-POSIX feature of the head command that lets you specify a negative value to '-n'. A Surprisingly The Most Using A Piece Of An Introduction Common Mistake Confusing Paper As A To Data Science Involving Wildcards Grep Mistakes Display Terminal On The Linux & The Find Command I've Ever - ed Vs. vim Command Line A Surprisingly Made Using A Piece Of An Introduction Common Mistake The Most Paper As A To Data Science Involving Wildcards Confusing Display Terminal On The Linux & The Find Command Grep Mistakes - ed Vs. vim Command Line Published I've Ever Published Published 2020-01-21 Made 2020-10-05 2019-10-16 Published 2020-11-02 A Guide to What is SSH? An Overview of Using SSH to Recording 660FPS Linux How to Do Connect to Your Video On A $6 Commands For Everything with Raspberry Pi Over Raspberry Pi Camera Beginners Raspberry Pi The Internet A Guide to What is SSH? Cameras Using SSH to Recording 660FPS Linux An Overview of Connect to Your Video On A $6 Commands For How to Do Raspberry Pi Over Raspberry Pi Camera Beginners Everything with The Internet Published Published Raspberry Pi Published 2019-08-01 2017-04-30 Cameras 2019-04-22 Published 2019-05-28 Why Bother Subscribing? * Free Software/Engineering Content. I publish all of my educational content publicly for free so everybody can make use of it. Why bother signing up for a paid 'course', when you can just sign up for this email list? Join My Mailing List * Read about cool new products that I'm Email: building. How do I make money? Glad you [ ] asked! You'll get some emails with [Submit] Privacy examples of things that I sell. You might Policy even get some business ideas of your own :) * People actually like this email list. I know that sounds crazy, because who actually subscribes to email lists these days, right? Well, some do, and if you end up not liking it, I give you permission to unsubscribe and mark it as spam. (c) 2021 Robert Elder SocialSocialSocialSocialSocialSocialSocial Software Inc. Privacy Policy Paypal Acceptable Use Terms of Use