https://www.trevorlasn.com/blog/10-essential-terminal-commands-every-developer-should-know/ /whoami/ all articles Aug 21, 2024 * 13 min read * By Trevor Indrek Lasn Principal Software Engineer @ stackredefine.com , carrentalgateway.com --------------------------------------------------------------------- 10 Essential Terminal Commands Every Developer Should Know List of useful Unix terminal commands to boost your productivity. Here are some of my favorites. Sometimes, tasks that might take hours to code can be accomplished in minutes with the terminal. The command line can significantly streamline your workflow by automating tasks that would otherwise require navigating through a graphical interface. For those familiar with Unix-like systems such as Linux or macOS, the terminal is a key tool for efficiency. It acts as a command-line interface, allowing you to control your system, execute programs, and manage files directly. This article assumes you're already comfortable with basic commands like rm, pwd, and cd. 1. grep Need to find where a function or variable is used in your codebase, or sift through logs to locate specific entries? grep can help you with that. The grep command searches for specific patterns in files. It's like having a supercharged search function that digs into file contents. How and when to use grep The basic syntax for the grep command goes as the following; grep "let's find something" file.[txt,json,js,md,etc] Case-insensitive search: Add the -i flag to ignore case differences. grep -i "REact" compiler/apps/playground/app/index.tsx ? '[DEV] React Compiler Playground' : 'React Compiler Playground' Count occurrences: Use the -c flag to count the number of matching lines. grep -c "React" compiler/apps/playground/app/index.tsx 2 Analyzing logs: If you're troubleshooting an issue, you can use grep to find specific error messages in logs. grep -i "Operation not supported on socket" system.log 09/24 08:51:01 INFO :..settcpimage: Get TCP images rc - EDC8112I Operation not supported on socket. Search for Multiple Patterns: You can search for multiple patterns by using the -e flag multiple times. Match either "error" or "404" in system.log. grep -e "error" -e "404" system.log npm error code E404 npm error 404 'trevorlasn.com@*' is not in this registry. npm error A complete log of this run can be found in: /Users/trevorindreklasn/.npm/_logs/2024-08-20T16_41_32_846Z-debug-0.log Recursive Search: To search for a pattern in all files within a directory and its subdirectories, use the -r (or --recursive) flag. grep -o -r "fs" node_modules | wc -l 22491 This will search through all files in the specified directory and its subdirectories. The -o option tells grep to print only the matched parts of the line. The pipe | takes the output from the command on the left (grep) and uses it as input for the command on the right (wc -l). wc -l counts and displays the number of lines in its input. 2. ls The ls command lists all the files and directories in your current directory. It's the go-to command for getting an overview of what's in a directory. trevorlasn.com git:(master) ls README.md package.json tsconfig.json astro.config.mjs pnpm-lock.yaml vercel.json dist public yarn.lock node_modules src package-lock.json tailwind.config.mjs trevorlasn.com git:(master) Detailed view: To get more information, like file sizes and permissions, use the -l option. trevorlasn.com git:(master) ls -l total 1712 -rw-r--r-- 1 trevorindreklasn staff 2468 Jul 29 20:14 README.md -rw-r--r--@ 1 trevorindreklasn staff 589 Aug 15 14:41 astro.config.mjs drwxr-xr-x 18 trevorindreklasn staff 576 Jul 30 11:20 dist drwxr-xr-x@ 609 trevorindreklasn staff 19488 Aug 20 19:41 node_modules -rw-r--r--@ 1 trevorindreklasn staff 386782 Aug 20 19:41 package-lock.json -rw-r--r--@ 1 trevorindreklasn staff 1106 Aug 19 09:41 package.json -rw-r--r-- 1 trevorindreklasn staff 223398 Jul 29 20:14 pnpm-lock.yaml drwxr-xr-x 5 trevorindreklasn staff 160 Aug 17 12:43 public drwxr-xr-x 12 trevorindreklasn staff 384 Aug 7 15:10 src -rw-r--r-- 1 trevorindreklasn staff 503 Aug 7 20:03 tailwind.config.mjs -rw-r--r-- 1 trevorindreklasn staff 165 Jul 29 20:14 tsconfig.json -rw-r--r-- 1 trevorindreklasn staff 614 Aug 17 11:46 vercel.json -rw-r--r--@ 1 trevorindreklasn staff 236600 Aug 20 19:41 yarn.lock trevorlasn.com git:(master) Show hidden files: Sometimes you need to see files that start with a dot (like .gitignore). trevorlasn.com git:(master) ls -la total 1736 drwxr-xr-x 21 trevorindreklasn staff 672 Aug 19 09:42 . drwxr-xr-x 22 trevorindreklasn staff 704 Jul 31 13:19 .. drwxr-xr-x 5 trevorindreklasn staff 160 Aug 11 19:59 .astro -rw-r--r-- 1 trevorindreklasn staff 37 Jul 29 20:14 .eslintignore -rw-r--r--@ 1 trevorindreklasn staff 722 Aug 17 13:36 .eslintrc.cjs drwxr-xr-x 14 trevorindreklasn staff 448 Aug 19 10:35 .git -rw-r--r-- 1 trevorindreklasn staff 230 Jul 29 20:14 .gitignore drwxr-xr-x 4 trevorindreklasn staff 128 Jul 29 20:14 .vscode -rw-r--r-- 1 trevorindreklasn staff 2468 Jul 29 20:14 README.md -rw-r--r--@ 1 trevorindreklasn staff 589 Aug 15 14:41 astro.config.mjs drwxr-xr-x 18 trevorindreklasn staff 576 Jul 30 11:20 dist drwxr-xr-x@ 609 trevorindreklasn staff 19488 Aug 20 19:41 node_modules -rw-r--r--@ 1 trevorindreklasn staff 386782 Aug 20 19:41 package-lock.json -rw-r--r--@ 1 trevorindreklasn staff 1106 Aug 19 09:41 package.json -rw-r--r-- 1 trevorindreklasn staff 223398 Jul 29 20:14 pnpm-lock.yaml drwxr-xr-x 5 trevorindreklasn staff 160 Aug 17 12:43 public drwxr-xr-x 12 trevorindreklasn staff 384 Aug 7 15:10 src -rw-r--r-- 1 trevorindreklasn staff 503 Aug 7 20:03 tailwind.config.mjs -rw-r--r-- 1 trevorindreklasn staff 165 Jul 29 20:14 tsconfig.json -rw-r--r-- 1 trevorindreklasn staff 614 Aug 17 11:46 vercel.json -rw-r--r--@ 1 trevorindreklasn staff 236600 Aug 20 19:41 yarn.lock trevorlasn.com git:(master) 3. cat The cat command is short for "concatenate." It's used to display the contents of a file, combine files, or create new ones. trevorlasn.com git:(master) cat astro.config.mjs import { defineConfig } from "astro/config"; import mdx from "@astrojs/mdx"; import sitemap from "@astrojs/sitemap"; import tailwind from "@astrojs/tailwind"; import vercel from "@astrojs/vercel/static"; import partytown from "@astrojs/partytown"; // https://astro.build/config export default defineConfig({ site: "https://trevorlasn.com", integrations: [ mdx(), sitemap(), tailwind(), partytown({ config: { forward: ["dataLayer.push"] } }), ], output: "static", adapter: vercel(), }); Combining Files: One of the key features of cat is its ability to combine multiple files into one. For instance, if you want to merge file1.txt and file2.txt into file3.txt, you can do this: cat file1.txt file2.txt > file3.txt This command above takes the content of file1.txt and file2.txt and merges them into file3.txt. The > operator is used to direct the combined output into a new file. Creating New Files: You can also use cat to create new files. Type your text, and when you're done, press Ctrl+D to save and exit. cat > newfile.txt hey ls newfile.txt cat newfile.txt hey cat is helpful for viewing smaller files, but for very large files, it can be overwhelming as it dumps everything at once. In such cases, it's better to use commands like less or head to view files in a more controlled way. 4. head You often don't need to see all the content when working with large files. Instead of using cat to display everything, the head command lets you preview just the first few lines of a file. This is especially useful for checking the structure of CSV files, logs, or any other large text files. By default, head shows the first 10 lines of a file: trevorlasn.com git:(master) head package-lock.json { "name": "trevorlasn.com", "version": "1.0.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "trevorlasn.com", "version": "1.0.0", "dependencies": { If you need more or fewer lines, you can specify the exact number using the -n option: trevorlasn.com git:(master) head -n 5 package-lock.json { "name": "trevorlasn.com", "version": "1.0.0", "lockfileVersion": 3, "requires": true, Previewing CSV Headers: For CSV files, head is perfect for quickly checking the header or structure: head -n 1 username-password-recovery-code.csv Username; Identifier;One-time password;Recovery code;First name;Last name;Department;Location 5. awk awk is a powerful tool for pattern scanning and processing. It's particularly useful for manipulating and analyzing text files and data streams. With awk, you can filter, extract, and transform data in a file or from command output. awk efficiently extracts and combines data from various sources using its associative arrays. Suppose you have two CSV files: employees.csv cat employees.csv ID,Name,Department 101,John Doe,Sales 102,Jane Smith,Engineering 103,Jim Brown,Sales salaries.csv cat salaries.csv ID,Salary 101,50000 102,60000 103,55000 Use awk to merge these files and display each employee's name with their salary. awk -F',' ' NR==FNR {salaries[$1]=$2; next} FNR==1 {next} {print $2, salaries[$1]} ' salaries.csv employees.csv John Doe 50000 Jane Smith 60000 Jim Brown 55000 Rundown * NR==FNR {salaries[$1]=$2; next}: While processing the first file (salaries.csv), store salaries in an associative array. The employee ID ($1) is the key, and the salary ($2) is the value. This runs only for the first file. * FNR==1 {next}: Skip the header line of the second file ( employees.csv). * {print $2, salaries[$1]}: For each line in the second file ( employees.csv), print the employee's name ($2) and their salary from the array (salaries[$1]). You can also save the results to a new file. awk -F',' ' NR==FNR {salaries[$1]=$2; next} FNR==1 {next} {print $2, salaries[$1]} ' salaries.csv employees.csv > combined.csv cat combined.csv John Doe 50000 Jane Smith 60000 Jim Brown 55000 6. sed sed, short for Stream Editor, is a powerful tool for text processing in the terminal. It allows you to find, replace, insert, or delete text within files or streams of data. You can use it for quick edits without opening a text editor, making it great for scripting and automation. Replace a word or pattern in a file: Replacing "Trevor" with "John". cat hello.md My name is Trevor sed -i '' 's/Trevor/John/' hello.md cat hello.md My name is John If you want save the changes, use the -i option. Print Specific Lines: Print only specific lines from a file. trevorlasn.com git:(master) sed -n '2,4p' package-lock.json "name": "trevorlasn.com", "version": "1.0.0", "lockfileVersion": 3, This prints lines 2 through 4. Regular Expressions: sed supports regular expressions, allowing for complex search-and-replace operations. For example, replace all digits with "X": cat combined.csv John Doe 50000 Jane Smith 60000 Jim Brown 55000 sed 's/[0-9]/X/g' combined.csv John Doe XXXXX Jane Smith XXXXX Jim Brown XXXXX Renaming Files in Bulk: Let's say you have multiple files with the extension .txt and you want to rename them to .md. ls 1.txt 2.txt 3.txt for file in *.txt; do mv "$file" "$(echo "$file" | sed 's/.txt$/.md/')" done ls 1.md 2.md 3.md sed is highly versatile, and these examples just scratch the surface 7. tail tail is the counterpart to head. It allows you to view the last few lines of a file rather than the first. It's commonly used to monitor log files or check the end of a document. By default, tail shows the last 10 lines of a file. trevorlasn.com git:(master) tail package.json }, "devDependencies": { "@typescript-eslint/eslint-plugin": "^7.3.1", "@typescript-eslint/parser": "^7.3.1", "eslint": "^8.57.0", "eslint-plugin-astro": "^0.32.0", "eslint-plugin-jsx-a11y": "^6.8.0", "typescript": "^5.4.2" } } Viewing More or Fewer Lines: You can adjust the number of lines shown using the -n option. trevorlasn.com git:(master) tail -n 15 package.json "astro": "^4.13.3", "clsx": "^2.1.0", "sharp": "^0.33.3", "tailwind-merge": "^2.2.2", "tailwindcss": "^3.4.1" }, "devDependencies": { "@typescript-eslint/eslint-plugin": "^7.3.1", "@typescript-eslint/parser": "^7.3.1", "eslint": "^8.57.0", "eslint-plugin-astro": "^0.32.0", "eslint-plugin-jsx-a11y": "^6.8.0", "typescript": "^5.4.2" } } Real-Time File Monitoring: One of the most powerful features of tail is the -f option, which allows you to follow a file as it grows. This is especially useful for watching log files in real-time. tail -f 1.md 11 Changing file As new lines are added to 1.md, tail will automatically display them. 8. chmod Each file has three sets of permissions: owner, group, and others. These are typically represented in a format like rwxr-xr-- * r: Read permission * w: Write permission * x: Execute permission ls -l sensitive.md -rw-r--r--@ 1 trevorindreklasn staff 0 Aug 21 15:22 sensitive.md The file permissions -rw-r--r-- indicate that: * Owner (trevorindreklasn): Has read (r) and write (w) permissions. * Group (staff): Has read (r) permissions. * Others: Have read (r) permissions. The @ symbol indicates that the file has extended attributes, which are additional metadata beyond standard file permissions. File permissions control who can read, write, or execute a file, ensuring security and proper access management by preventing unauthorized users from modifying or viewing sensitive data. To restrict access to sensitive.md so that only the root user or superadmins can view and write to it, you can use the chmod command to modify the file's permissions. First, ensure the file is owned by the root user or a superadmin. You might need sudo for changing ownership: ls -l sensitive.md -rw-r--r--@ 1 root staff 0 Aug 21 15:22 sensitive.md sudo chown root:admin sensitive.md ls -l sensitive.md -rw-r-----@ 1 root admin 0 Aug 21 15:22 sensitive.md sudo chmod 600 sensitive.md textfiles ls -l sensitive.md -rw------- 1 root admin 0 Aug 21 15:22 sensitive.md Only the owner (root) has read and write access. While the group and others have no permissions. This restricts access to the file, making it readable and writable only by the owner. Improper file permissions can lead to security issues or system problems * Unauthorized Access: File containing sensitive information, like passwords or financial data, has overly permissive settings (e.g., chmod 777), anyone on the system can read or modify it. This could lead to data breaches or unauthorized access to sensitive information. * Malware Installation: A file or directory with write permissions for all users (e.g., chmod 777) could be exploited by attackers to place malicious scripts or software, potentially compromising the entire system. * Data Corruption: If files that should be read-only (e.g., logs or system configurations) are accidentally given write permissions, users or applications might inadvertently corrupt or erase critical data, leading to system instability or loss of important information. 9. xargs The xargs command builds and runs commands using input from other commands. It's used to pass a list of items as arguments to another command. Suppose you have a list of files that you want to delete. ls 1.txt 2.txt 2.md 3.md sensitive.md # Find all .txt files and delete them find . -name "*.txt" | xargs rm ls 2.md 3.md sensitive.md Instead of deleting them one by one, you can use xargs to pass the list of files to rm. find . -name "*.tmp" | xargs rm Creating Multiple Directories: If you have a list of directory names in a file and want to create all of them, you can use xargs with mkdir. cat dirs.txt src temp utils public ls 2.md 3.md dirs.txt sensitive.md # Creates each directory listed in the file. cat dirs.txt | xargs mkdir ls 2.md dirs.txt sensitive.md temp 3.md public src utils Compressing Files: If you have multiple files that you want to compress using gzip, you can use xargs to pass the filenames to gzip. # Compresses all .log files in the current directory ls *.log | xargs gzip 10. find Search and locate files and directories within your file system based on various criteria. It's highly customizable and can be combined with other commands for complex tasks. find [path] [expression] The find command searches for occurrences of "astro" within the node_modules directory, returning paths to files and directories with that name, including executables and package files. trevorlasn.com git:(master) find node_modules -name "astro" node_modules/.bin/astro node_modules/astro node_modules/astro/dist/runtime/server/render/astro Cleanup old log files: Regularly delete log files older than a month to free up disk space. find /var/log -type f -name "*.log" -mtime +30 -delete Backup important files: Locate and copy all .docx files from home directory to a backup location. find ~/Documents -name "*.docx" -exec cp {} /path/to/backup/ \; The find command is incredibly versatile and can be tailored to suit a wide range of file management tasks. Honorable Mentions * jq: Slice, filter, map and transform structured data with the same ease that sed, awk, grep. * cut: A command used to remove sections from each line of files. * netstat: A network utility to display network connections, routing tables, and interface statistics. * ping: A tool to test the reachability of a host on an IP network and measure round-trip time. * ifconfig: A command to configure network interfaces (deprecated in favor of ip). * telnet: A program to interact with remote hosts over a network using the Telnet protocol. * sftp: An interactive file transfer program that uses the SSH protocol. * wget: Network utility to retrieve files from the World Wide Web using HTTP and FTP. * ps: Displays information about running processes. * top: A task manager to view and manage running processes in real-time. * kill: A command used to send signals to processes, typically to terminate them. * comm: A command that compares two sorted files line by line. --------------------------------------------------------------------- Hi, my name is Trevor. I write about software engineering, startups, security, and more. If you like my work, feel free to visit my other website. This article was published on trevorlasn.com . all articles Back to top (c) 2017 - 2024 | about the author linkedin / github / rss --------------------------------------------------------------------- I level up engineering teams and help companies become more competitive. Please consider whitelisting this website in your adblocker. It helps with web traffic analytics and sponsorships.