SED Ściąga ===================================================================================================== sed -e "s/co_szukamy/na_co_zfamieniamy/g" pliw_wyjściowy (bez "g" zamienia tylko pierwsze wystąpienie wzorca) sed -e "25,100s/co_szukamy/na_co_zamieniamy/g" plik wyjściowy (podajemy zakres działania, od linii do linii) sed -e "/UNIX/ s/co_szukamy/na_co_zamieniamy/" plik_wyjściowy (edycja dotyczy tylko i wyłącznie linii, które zawierają słowo UNIX) sed -n "1,10p" plik_wyjściowy (pierwsze 10 linii z pliku plik_wejściowy zapisz jako plik_wyjściowy, zakres można podać dowolny) sed -e "s/;/:/g" plik_wyjściowy (zamienia w plik_wejściowy wszystkie ";" na ":" i zapisuje w plik_wyjściowy) sed -e "s/\t/ /g" plik_wyjściowy (zamienia wszystkie znaki TAB na SPACJA) sed '/ */ /g' plik_wyjściowy (zastąpi wszystkie wielokrotne spacje na jedną spację) sed '/^$/d' plik_wyjściowy (usuwa wszystkie puste wiersze z pliku) sed '/^BEGIN/,/^END/d' plik_wyjściowy (usuwa wszystkie wiersze pomiędzy wierszami BEGIN i END) sed '/[A-Za-z]/d' plik_wyjściowy (usuwa wszystkie wiersze z pliku zawierające litery) sed 's/BESTIA/bestia/gw bestia.bis' (zamieni wszystkie słowa BESTIA na bestia i zapisze w pliku bestia.bis) sed -e '/szukana_fraza/w jest.txt' -e 'szukana_fraza/!w nie_ma.txt' plik_wyjściowy (jeśli więcej poleceń edycyjnych na plik umieszczamy je w skrypcie sedfile.sh) W pliku sedfile.sh umieszczamy dwie linie: a\ *** sed -f sedfile.sh plik_wyjściowy (po każdym wierszu w plik_wejściowy wstawi *** i zapisze w plik_wyjściowy) ===================================================================================================== USEFUL ONE-LINE SCRIPTS FOR SED (Unix stream editor) Dec. 29, 2005 Compiled by Eric Pement - pemente[at]northpark[dot]edu version 5.5 Latest version of this file (in English) is usually at: http://sed.sourceforge.net/sed1line.txt http://www.pement.org/sed/sed1line.txt FILE SPACING: # double space a file sed G # double space a file which already has blank lines in it. Output file # should contain no more than one blank line between lines of text. sed '/^$/d;G' # triple space a file sed 'G;G' # undo double-spacing (assumes even-numbered lines are always blank) sed 'n;d' # insert a blank line above every line which matches "regex" sed '/regex/{x;p;x;}' # insert a blank line below every line which matches "regex" sed '/regex/G' # insert a blank line above and below every line which matches "regex" sed '/regex/{x;p;x;G;}' NUMBERING: # number each line of a file (simple left alignment). Using a tab (see # note on '\t' at end of file) instead of space will preserve margins. sed = filename | sed 'N;s/\n/\t/' # number each line of a file (number on left, right-aligned) sed = filename | sed 'N; s/^/ /; s/ *\(.\{6,\}\)\n/\1 /' # number each line of file, but only print numbers if line is not blank sed '/./=' filename | sed '/./N; s/\n/ /' # count lines (emulates "wc -l") sed -n '$=' TEXT CONVERSION AND SUBSTITUTION: # IN UNIX ENVIRONMENT: convert DOS newlines (CR/LF) to Unix format. sed 's/.$//' # assumes that all lines end with CR/LF sed 's/^M$//' # in bash/tcsh, press Ctrl-V then Ctrl-M sed 's/\x0D$//' # works on ssed, gsed 3.02.80 or higher # IN UNIX ENVIRONMENT: convert Unix newlines (LF) to DOS format. sed "s/$/`echo -e \\\r`/" # command line under ksh sed 's/$'"/`echo \\\r`/" # command line under bash sed "s/$/`echo \\\r`/" # command line under zsh sed 's/$/\r/' # gsed 3.02.80 or higher # delete leading whitespace (spaces, tabs) from front of each line # aligns all text flush left sed 's/^[ \t]*//' # see note on '\t' at end of file # delete trailing whitespace (spaces, tabs) from end of each line sed 's/[ \t]*$//' # see note on '\t' at end of file # delete BOTH leading and trailing whitespace from each line sed 's/^[ \t]*//;s/[ \t]*$//' # insert 5 blank spaces at beginning of each line (make page offset) sed 's/^/ /' # align all text flush right on a 79-column width sed -e :a -e 's/^.\{1,78\}$/ &/;ta' # set at 78 plus 1 space # center all text in the middle of 79-column width. In method 1, # spaces at the beginning of the line are significant, and trailing # spaces are appended at the end of the line. In method 2, spaces at # the beginning of the line are discarded in centering the line, and # no trailing spaces appear at the end of lines. sed -e :a -e 's/^.\{1,77\}$/ & /;ta' # method 1 sed -e :a -e 's/^.\{1,77\}$/ &/;ta' -e 's/\( *\)\1/\1/' # method 2 # substitute (find and replace) "foo" with "bar" on each line sed 's/foo/bar/' # replaces only 1st instance in a line sed 's/foo/bar/4' # replaces only 4th instance in a line sed 's/foo/bar/g' # replaces ALL instances in a line sed 's/\(.*\)foo\(.*foo\)/\1bar\2/' # replace the next-to-last case sed 's/\(.*\)foo/\1bar/' # replace only the last case # substitute "foo" with "bar" ONLY for lines which contain "baz" sed '/baz/s/foo/bar/g' # substitute "foo" with "bar" EXCEPT for lines which contain "baz" sed '/baz/!s/foo/bar/g' # change "scarlet" or "ruby" or "puce" to "red" sed 's/scarlet/red/g;s/ruby/red/g;s/puce/red/g' # most seds gsed 's/scarlet\|ruby\|puce/red/g' # GNU sed only # reverse order of lines (emulates "tac") # bug/feature in HHsed v1.5 causes blank lines to be deleted sed '1!G;h;$!d' # method 1 sed -n '1!G;h;$p' # method 2 # reverse each character on the line (emulates "rev") sed '/\n/!G;s/\(.\)\(.*\n\)/&\2\1/;//D;s/.//' # join pairs of lines side-by-side (like "paste") sed '$!N;s/\n/ /' # if a line ends with a backslash, append the next line to it sed -e :a -e '/\\$/N; s/\\\n//; ta' # if a line begins with an equal sign, append it to the previous line # and replace the "=" with a single space sed -e :a -e '$!N;s/\n=/ /;ta' -e 'P;D' # add commas to numeric strings, changing "1234567" to "1,234,567" gsed ':a;s/\B[0-9]\{3\}\>/,&/;ta' # GNU sed sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta' # other seds # add commas to numbers with decimal points and minus signs (GNU sed) gsed -r ':a;s/(^|[^0-9.])([0-9]+)([0-9]{3})/\1\2,\3/g;ta' # add a blank line every 5 lines (after lines 5, 10, 15, 20, etc.) gsed '0~5G' # GNU sed only sed 'n;n;n;n;G;' # other seds SELECTIVE PRINTING OF CERTAIN LINES: # print first 10 lines of file (emulates behavior of "head") sed 10q # print first line of file (emulates "head -1") sed q # print the last 10 lines of a file (emulates "tail") sed -e :a -e '$q;N;11,$D;ba' # print the last 2 lines of a file (emulates "tail -2") sed '$!N;$!D' # print the last line of a file (emulates "tail -1") sed '$!d' # method 1 sed -n '$p' # method 2 # print the next-to-the-last line of a file sed -e '$!{h;d;}' -e x # for 1-line files, print blank line sed -e '1{$q;}' -e '$!{h;d;}' -e x # for 1-line files, print the line sed -e '1{$d;}' -e '$!{h;d;}' -e x # for 1-line files, print nothing # print only lines which match regular expression (emulates "grep") sed -n '/regexp/p' # method 1 sed '/regexp/!d' # method 2 # print only lines which do NOT match regexp (emulates "grep -v") sed -n '/regexp/!p' # method 1, corresponds to above sed '/regexp/d' # method 2, simpler syntax # print the line immediately before a regexp, but not the line # containing the regexp sed -n '/regexp/{g;1!p;};h' # print the line immediately after a regexp, but not the line # containing the regexp sed -n '/regexp/{n;p;}' # print 1 line of context before and after regexp, with line number # indicating where the regexp occurred (similar to "grep -A1 -B1") sed -n -e '/regexp/{=;x;1!p;g;$!N;p;D;}' -e h # grep for AAA and BBB and CCC (in any order) sed '/AAA/!d; /BBB/!d; /CCC/!d' # grep for AAA and BBB and CCC (in that order) sed '/AAA.*BBB.*CCC/!d' # grep for AAA or BBB or CCC (emulates "egrep") sed -e '/AAA/b' -e '/BBB/b' -e '/CCC/b' -e d # most seds gsed '/AAA\|BBB\|CCC/!d' # GNU sed only # print paragraph if it contains AAA (blank lines separate paragraphs) # HHsed v1.5 must insert a 'G;' after 'x;' in the next 3 scripts below sed -e '/./{H;$!d;}' -e 'x;/AAA/!d;' # print paragraph if it contains AAA and BBB and CCC (in any order) sed -e '/./{H;$!d;}' -e 'x;/AAA/!d;/BBB/!d;/CCC/!d' # print paragraph if it contains AAA or BBB or CCC sed -e '/./{H;$!d;}' -e 'x;/AAA/b' -e '/BBB/b' -e '/CCC/b' -e d gsed '/./{H;$!d;};x;/AAA\|BBB\|CCC/b;d' # GNU sed only # print only lines of 65 characters or longer sed -n '/^.\{65\}/p' # print only lines of less than 65 characters sed -n '/^.\{65\}/!p' # method 1, corresponds to above sed '/^.\{65\}/d' # method 2, simpler syntax # print section of file from regular expression to end of file sed -n '/regexp/,$p' # print section of file based on line numbers (lines 8-12, inclusive) sed -n '8,12p' # method 1 sed '8,12!d' # method 2 # print line number 52 sed -n '52p' # method 1 sed '52!d' # method 2 sed '52q;d' # method 3, efficient on large files # beginning at line 3, print every 7th line gsed -n '3~7p' # GNU sed only sed -n '3,${p;n;n;n;n;n;n;}' # other seds # print section of file between two regular expressions (inclusive) sed -n '/Iowa/,/Montana/p' # case sensitive SELECTIVE DELETION OF CERTAIN LINES: # print all of file EXCEPT section between 2 regular expressions sed '/Iowa/,/Montana/d' # delete duplicate, consecutive lines from a file (emulates "uniq"). # First line in a set of duplicate lines is kept, rest are deleted. sed '$!N; /^\(.*\)\n\1$/!P; D' # delete duplicate, nonconsecutive lines from a file. Beware not to # overflow the buffer size of the hold space, or else use GNU sed. sed -n 'G; s/\n/&&/; /^\([ -~]*\n\).*\n\1/d; s/\n//; h; P' # delete all lines except duplicate lines (emulates "uniq -d"). sed '$!N; s/^\(.*\)\n\1$/\1/; t; D' # delete the first 10 lines of a file sed '1,10d' # delete the last line of a file sed '$d' # delete the last 2 lines of a file sed 'N;$!P;$!D;$d' # delete the last 10 lines of a file sed -e :a -e '$d;N;2,10ba' -e 'P;D' # method 1 sed -n -e :a -e '1,10!{P;N;D;};N;ba' # method 2 # delete every 8th line gsed '0~8d' # GNU sed only sed 'n;n;n;n;n;n;n;d;' # other seds # delete lines matching pattern sed '/pattern/d' # delete ALL blank lines from a file (same as "grep '.' ") sed '/^$/d' # method 1 sed '/./!d' # method 2 # delete all CONSECUTIVE blank lines from file except the first; also # deletes all blank lines from top and end of file (emulates "cat -s") sed '/./,/^$/!d' # method 1, allows 0 blanks at top, 1 at EOF sed '/^$/N;/\n$/D' # method 2, allows 1 blank at top, 0 at EOF # delete all CONSECUTIVE blank lines from file except the first 2: sed '/^$/N;/\n$/N;//D' # delete all leading blank lines at top of file sed '/./,$!d' # delete all trailing blank lines at end of file sed -e :a -e '/^\n*$/{$d;N;ba' -e '}' # works on all seds sed -e :a -e '/^\n*$/N;/\n$/ba' # ditto, except for gsed 3.02.* # delete the last line of each paragraph sed -n '/^$/{p;h;};/./{x;/./p;}' SPECIAL APPLICATIONS: # remove nroff overstrikes (char, backspace) from man pages. The 'echo' # command may need an -e switch if you use Unix System V or bash shell. sed "s/.`echo \\\b`//g" # double quotes required for Unix environment sed 's/.^H//g' # in bash/tcsh, press Ctrl-V and then Ctrl-H sed 's/.\x08//g' # hex expression for sed 1.5, GNU sed, ssed # get Usenet/e-mail message header sed '/^$/q' # deletes everything after first blank line # get Usenet/e-mail message body sed '1,/^$/d' # deletes everything up to first blank line # get Subject header, but remove initial "Subject: " portion sed '/^Subject: */!d; s///;q' # get return address header sed '/^Reply-To:/q; /^From:/h; /./d;g;q' # parse out the address proper. Pulls out the e-mail address by itself # from the 1-line return address header (see preceding script) sed 's/ *(.*)//; s/>.*//; s/.*[:<] *//' # add a leading angle bracket and space to each line (quote a message) sed 's/^/> /' # delete leading angle bracket & space from each line (unquote a message) sed 's/^> //' # remove most HTML tags (accommodates multiple-line tags) sed -e :a -e 's/<[^>]*>//g;/output_file sed -r -e :a -e 's/\[[^\][0-9]+\]//g;/output_file # extract multi-part uuencoded binaries, removing extraneous header # info, so that only the uuencoded portion remains. Files passed to # sed must be passed in the proper order. Version 1 can be entered # from the command line; version 2 can be made into an executable # Unix shell script. (Modified from a script by Rahul Dhesi.) sed '/^end/,/^begin/d' file1 file2 ... fileX | uudecode # vers. 1 sed '/^end/,/^begin/d' "$@" | uudecode # vers. 2 # sort paragraphs of file alphabetically. Paragraphs are separated by blank # lines. GNU sed uses \v for vertical tab, or any unique char will do. sed '/./{H;d;};x;s/\n/={NL}=/g' file | sort | sed '1s/={NL}=//;s/={NL}=/\n/g' gsed '/./{H;d};x;y/\n/\v/' file | sort | sed '1s/\v//;y/\v/\n/' # zip up each .TXT file individually, deleting the source file and # setting the name of each .ZIP file to the basename of the .TXT file # (under DOS: the "dir /b" switch returns bare filenames in all caps). echo @echo off >zipup.bat dir /b *.txt | sed "s/^\(.*\)\.TXT/pkzip -mo \1 \1.TXT/" >>zipup.bat cat filename | sed '10q' # uses piped input sed '10q' filename # same effect, avoids a useless "cat" sed '10q' filename > newfile # redirects output to disk ===================================================================================================== Z TUTORIALA: ----------------------------------------------------------------------------------------------------- The slash as a delimiter The character after the s is the delimiter. It is conventionally a slash, because this is what ed, more, and vi use. It can be anything you want, however. If you want to change a pathname that contains a slash - say /usr/local/bin to /common/bin - you could use the backslash to quote the slash: sed 's/\/usr\/local\/bin/\/common\/bin/' new Gulp. Some call this a 'Picket Fence' and it's ugly. It is easier to read if you use an underline instead of a slash as a delimiter: sed 's_/usr/local/bin_/common/bin_' new Some people use colons: sed 's:/usr/local/bin:/common/bin:' new Others use the "|" character. sed 's|/usr/local/bin|/common/bin|' new Pick one you like. As long as it's not in the string you are looking for, anything goes. And remember that you need three delimiters. If you get a "Unterminated `s' command" it's because you are missing one of them. Using & as the matched string Sometimes you want to search for a pattern and add some characters, like parenthesis, around or near the pattern you found. It is easy to do this if you are looking for a particular string: sed 's/abc/(abc)/' new This won't work if you don't know exactly what you will find. How can you put the string you found in the replacement string if you don't know what it is? The solution requires the special character "&." It corresponds to the pattern found. sed 's/[a-z]*/(&)/' new You can have any number of "&" in the replacement string. You could also double a pattern, e.g. the first number of a line: % echo "123 abc" | sed 's/[0-9]*/& &/' 123 123 abc Let me slightly amend this example. Sed will match the first string, and make it as greedy as possible. I'll cover that later. If you don't want it to be so greedy (i.e. limit the matching), you need to put restrictions on the match. The first match for '[0-9]*' is the first character on the line, as this matches zero or more numbers. So if the input was "abc 123" the output would be unchanged (well, except for a space before the letters). A better way to duplicate the number is to make sure it matches a number: % echo "123 abc" | sed 's/[0-9][0-9]*/& &/' 123 123 abc The string "abc" is unchanged, because it was not matched by the regular expression. If you wanted to eliminate "abc" from the output, you must expand the regular expression to match the rest of the line and explicitly exclude part of the expression using "(", ")" and "\1", which is the next topic. Extended Regular Expressions Let me add a quick comment here because there is another way to write the above script. "[0-9]*" matches zero or more numbers. "[0-9][0-9]*" matches one or more numbers. Another way to do this is to use the "+" meta-character and use the pattern "[0-9]+" as the "+" is a special character when using "extended regular expressions." Extended regular expressions have more power, but sed scripts that treated "+" as a normal character would break. Therefore you must explicitly enable this extension with a command line option. GNU sed turns this feature on if you use the "-r" command line option. So the above could also be written using % echo "123 abc" | sed -r 's/[0-9]+/& &/' 123 123 abc Mac OS X and FreeBSD uses [153]-E instead of [154]-r. For more information on extended regular expressions, see [155]Regular Expressions and the [156]description of the -r command line argument Using \1 to keep part of the pattern I have already described the use of "(" ")" and "1" in my tutorial on [158]regular expressions. To review, the escaped parentheses (that is, parentheses with backslashes before them) remember a substring of the characters matched by the regular expression. You can use this to exclude part of the characters matched by the regular expression. The "\1" is the first remembered pattern, and the "\2" is the second remembered pattern. Sed has up to nine remembered patterns. If you wanted to keep the first word of a line, and delete the rest of the line, mark the important part with the parenthesis: sed 's/\([a-z]*\).*/\1/' I should elaborate on this. Regular expressions are greedy, and try to match as much as possible. "[a-z]*" matches zero or more lower case letters, and tries to match as many characters as possible. The ".*" matches zero or more characters after the first match. Since the first one grabs all of the contiguous lower case letters, the second matches anything else. Therefore if you type echo abcd123 | sed 's/\([a-z]*\).*/\1/' This will output "abcd" and delete the numbers. If you want to switch two words around, you can remember two patterns and change the order around: sed 's/\([a-z]*\) \([a-z]*\)/\2 \1/' Note the space between the two remembered patterns. This is used to make sure two words are found. However, this will do nothing if a single word is found, or any lines with no letters. You may want to insist that words have at least one letter by using sed 's/\([a-z][a-z]*\) \([a-z][a-z]*\)/\2 \1/' or by using extended regular expressions (note that '(' and ')' no longer need to have a backslash): sed -r 's/([a-z]+) ([a-z]+)/\2 \1/' # Using GNU sed sed -E 's/([a-z]+) ([a-z]+)/\2 \1/' # Using Apple Mac OS X The "\1" doesn't have to be in the replacement string (in the right hand side). It can be in the pattern you are searching for (in the left hand side). If you want to eliminate duplicated words, you can try: sed 's/\([a-z]*\) \1/\1/' If you want to detect duplicated words, you can use sed -n '/\([a-z][a-z]*\) \1/p' or with extended regular expressions sed -rn '/([a-z]+) \1/p' # GNU sed sed -En '/([a-z]+) \1/p' # Mac OS X This, when used as a filter, will print lines with duplicated words. The numeric value can have up to nine values: "\1" thru "\9." If you wanted to reverse the first three characters on a line, you can use sed 's/^\(.\)\(.\)\(.\)/\3\2\1/' /g - Global replacement Most UNIX utilities work on files, reading a line at a time. Sed, by default, is the same way. If you tell it to change a word, it will only change the first occurrence of the word on a line. You may want to make the change on every word on the line instead of the first. For an example, let's place parentheses around words on a line. Instead of using a pattern like "[A-Za-z]*" which won't match words like "won't," we will use a pattern, "[^ ]*," that matches everything except a space. Well, this will also match anything because "*" means zero or more. The current version of Solaris's sed (as I wrote this) can get unhappy with patterns like this, and generate errors like "Output line too long" or even run forever. I consider this a bug, and have reported this to Sun. As a work-around, you must avoid matching the null string when using the "g" flag to sed. A work-around example is: "[^ ][^ ]*." The following will put parenthesis around the first word: sed 's/[^ ]*/(&)/' new If you want it to make changes for every word, add a "g" after the last delimiter and use the work-around: sed 's/[^ ][^ ]*/(&)/g' new /1, /2, etc. Specifying which occurrence With no flags, the first matched substitution is changed. With the "g" option, all matches are changed. If you want to modify a particular pattern that is not the first one on the line, you could use "\(" and "\)" to mark each pattern, and use "\1" to put the first pattern back unchanged. This next example keeps the first word on the line but deletes the second: sed 's/\([a-zA-Z]*\) \([a-zA-Z]*\) /\1 /' new Yuck. There is an easier way to do this. You can add a number after the substitution command to indicate you only want to match that particular pattern. Example: sed 's/[a-zA-Z]* //2' new You can combine a number with the g (global) flag. For instance, if you want to leave the first word alone, but change the second, third, etc. to be DELETED instead, use /2g: sed 's/[a-zA-Z]* /DELETED /2g' new I've heard that combining the number with the g command does not work on The MacOS, and perhaps the FreeSBD version of sed as well. Don't get /2 and \2 confused. The /2 is used at the end. \2 is used in inside the replacement field. Note the space after the "*" character. Without the space, sed will run a long, long time. (Note: this bug is probably fixed by now.) This is because the number flag and the "g" flag have the same bug. You should also be able to use the pattern sed 's/[^ ]*//2' new but this also eats CPU. If this works on your computer, and it does on some UNIX systems, you could remove the encrypted password from the password file: sed 's/[^:]*//2' /etc/password.new But this didn't work for me the time I wrote this. Using "[^:][^:]*" as a work-around doesn't help because it won't match a non-existent password, and instead delete the third field, which is the user ID! Instead you have to use the ugly parenthesis: sed 's/^\([^:]*\):[^:]:/\1::/' /etc/password.new You could also add a character to the first pattern so that it no longer matches the null pattern: sed 's/[^:]*:/:/2' /etc/password.new The number flag is not restricted to a single digit. It can be any number from 1 to 512. If you wanted to add a colon after the 80th character in each line, you could type: sed 's/./&:/80' new You can also do it the hard way by using 80 dots: sed 's/^................................................................................/&:/' new /p - print By default, sed prints every line. If it makes a substitution, the new text is printed instead of the old one. If you use an optional argument to sed, "sed -n," it will not, by default, print any new lines. I'll cover this and other options later. When the "-n" option is used, the "p" flag will cause the modified line to be printed. Here is one way to duplicate the function of grep with sed: sed -n 's/pattern/&/p' new Note that a space after the '/I' and the 'p' (print) command emphasizes that the 'p' is not a modifier of the pattern matching process, , but a command to execute after the pattern matching. Multiple commands with -e command One method of combining multiple commands is to use a -e before each command: sed -e 's/a/A/' -e 's/b/B/' new A "-e" isn't needed in the earlier examples because sed knows that there must always be one command. If you give sed one argument, it must be a command, and sed will edit the data read from standard input. The long argument version is sed --expression='s/a/A/' --expression='s/b/B/' new Filenames on the command line You can specify files on the command line if you wish. If there is more than one argument to sed that does not start with an option, it must be a filename. This next example will count the number of lines in three files that don't begin with a "#:" sed 's/^#.*//' f1 f2 f3 | grep -v '^$' | wc -l Let's break this down into pieces. The sed substitute command changes every line that starts with a "#" into a blank line. Grep was used to filter out (delete) empty lines. Wc counts the number of lines left. Sed has more commands that make grep unnecessary. And grep -c can replace wc -l. I'll discuss how you can duplicate some of grep's functionality later. Of course you could write the last example using the "-e" option: sed -e 's/^#.*//' f1 f2 f3 | grep -v '^$' | wc -l sed -n: no printing The "-n" option will not print anything unless an explicit request to print is found. I mentioned the "/p" flag to the substitute command as one way to turn printing back on. Let me clarify this. The command sed 's/PATTERN/&/p' file acts like the cat program if PATTERN is not in the file: e.g. nothing is changed. If PATTERN is in the file, then each line that has this is printed twice. Add the "-n" option and the example acts like grep: sed -n 's/PATTERN/&/p' file Nothing is printed, except those lines with PATTERN included. The long argument of the -n command is either sed --quiet 's/PATTERN/&/p' file or sed --silent 's/PATTERN/&/p' file sed -f scriptname If you have a large number of sed commands, you can put them into a file and use sed -f sedscript new where sedscript could look like this: # sed comment - This script changes lower case vowels to upper case s/a/A/g s/e/E/g s/i/I/g s/o/O/g s/u/U/g When there are several commands in one file, each command must be on a separate line. The long argument version is sed --file=sedscript new sed in shell scripts If you have many commands and they won't fit neatly on one line, you can break up the line using a backslash: sed -e 's/a/A/g' \ -e 's/e/E/g' \ -e 's/i/I/g' \ -e 's/o/O/g' \ -e 's/u/U/g' new Quoting multiple sed lines in the C shell You can have a large, multi-line sed script in the C shell, but you must tell the C shell that the quote is continued across several lines. This is done by placing a backslash at the end of each line: #!/bin/csh -f sed 's/a/A/g \ s/e/E/g \ s/i/I/g \ s/o/O/g \ s/u/U/g' new Quoting multiple sed lines in the Bourne shell The Bourne shell makes this easier as a quote can cover several lines: #!/bin/sh sed ' s/a/A/g s/e/E/g s/i/I/g s/o/O/g s/u/U/g' new Passing arguments into a sed script Passing a word into a shell script that calls sed is easy if you remembered [187]my tutorial on the UNIX quoting mechanism. To review, you use the single quotes to turn quoting on and off. A simple shell script that uses sed to emulate grep is: #!/bin/sh sed -n 's/'$1'/&/p' However - there is a subtle problem with this script. If you have a space as an argument, the script would cause a syntax error, such as sed: -e expression #1, char 4: unterminated `s' command A better version would protect from this happening: #!/bin/sh sed -n 's/'"$1"'/&/p' Using sed in a shell here-is document You can use sed to prompt the user for some parameters and then create a file with those parameters filled in. You could create a file with dummy values placed inside it, and use sed to change those dummy values. A simpler way is to use the "here is" document, which uses part of the shell script as if it were standard input: #!/bin/sh echo -n 'what is the value? ' read value sed 's/XYZ/'$value'/' <new Patterns Many UNIX utilities like vi and more use a slash to search for a regular expression. Sed uses the same convention, provided you terminate the expression with a slash. To delete the first number on all lines that start with a "#," use: sed '/^#/ s/[0-9][0-9]*//' I placed a space after the "/expression/" so it is easier to read. It isn't necessary, but without it the command is harder to fathom. Sed does provide a few extra options when specifying regular expressions. But I'll discuss those later. If the expression starts with a backslash, the next character is the delimiter. To use a comma instead of a slash, use: sed '\,^#, s/[0-9][0-9]*//' The main advantage of this feature is searching for slashes. Suppose you wanted to search for the string "/usr/local/bin" and you wanted to change it for "/common/all/bin." You could use the backslash to escape the slash: sed '/\/usr\/local\/bin/ s/\/usr\/local/\/common\/all/' It would be easier to follow if you used an underline instead of a slash as a search. This example uses the underline in both the search command and the substitute command: sed '\_/usr/local/bin_ s_/usr/local_/common/all_' This illustrates why sed scripts get the reputation for obscurity. I could be perverse and show you the example that will search for all lines that start with a "g," and change each "g" on that line to an "s:" sed '/^g/s/g/s/g' Adding a space and using an underscore after the substitute command makes this much easier to read: sed '/^g/ s_g_s_g' Ranges by line number You can specify a range on line numbers by inserting a comma between the numbers. To restrict a substitution to the first 100 lines, you can use: sed '1,100 s/A/a/' If you know exactly how many lines are in a file, you can explicitly state that number to perform the substitution on the rest of the file. In this case, assume you used wc to find out there are 532 lines in the file: sed '101,532 s/A/a/' An easier way is to use the special character "$," which means the last line in the file. sed '101,$ s/A/a/' The "$" is one of those conventions that mean "last" in utilities like cat -e, vi, and ed. "cat -e" Line numbers are cumulative if several files are edited. That is, sed '200,300 s/A/a/' f1 f2 f3 >new is the same as cat f1 f2 f3 | sed '200,300 s/A/a/' >new Ranges by patterns You can specify two regular expressions as the range. Assuming a "#" starts a comment, you can search for a keyword, remove all comments until you see the second keyword. In this case the two keywords are "start" and "stop:" sed '/start/,/stop/ s/#.*//' The first pattern turns on a flag that tells sed to perform the substitute command on every line. The second pattern turns off the flag. If the "start" and "stop" pattern occurs twice, the substitution is done both times. If the "stop" pattern is missing, the flag is never turned off, and the substitution will be performed on every line until the end of the file. You should know that if the "start" pattern is found, the substitution occurs on the same line that contains "start." This turns on a switch, which is line oriented. That is, the next line is read and the substitute command is checked. If it contains "stop" the switch is turned off. Switches are line oriented, and not word oriented. You can combine line numbers and regular expressions. This example will remove comments from the beginning of the file until it finds the keyword "start:" sed -e '1,/start/ s/#.*//' This example will remove comments everywhere except the lines between the two keywords: sed -e '1,/start/ s/#.*//' -e '/stop/,$ s/#.*//' Delete with d Using ranges can be confusing, so you should expect to do some experimentation when you are trying out a new script. A useful command deletes every line that matches the restriction: "d." If you want to look at the first 10 lines of a file, you can use: sed '11,$ d' out will append the file "end" at the end of the file (address "$)." The following will insert a file after the line with the word "INCLUDE:" sed '/INCLUDE/ r file' out You can use the curly braces to delete the line having the "INCLUDE" command on it: #!/bin/sh sed '/INCLUDE/ { r file d }' The # Comment Command As we dig deeper into sed, comments will make the commands easier to follow. The older versions of sed only allow one line as a comment, and it must be the first line. SunOS (and GNU's sed) allows more than one comment, and these comments don't have to be first. The last example could be: #!/bin/sh # watch out for a '/' in the parameter # use alternate search delimiter sed -e '\_#INCLUDE <'"$1"'>_{ # read the file r '"$1"' # delete any characters in the pattern space # and read the next line in d }' Append a line with 'a' The "a" command appends a line after the range or pattern. This example will add a line after every line with "WORD:" #!/bin/sh sed ' /WORD/ a\ Add this line after every line with WORD ' You could eliminate two lines in the shell script if you wish: #!/bin/sh sed '/WORD/ a\ Add this line after every line with WORD' Insert a line with 'i' You can insert a new line before the pattern with the "i" command: #!/bin/sh sed ' /WORD/ i\ Add this line before every line with WORD ' Change a line with 'c' You can change the current line with a new line. #!/bin/sh sed ' /WORD/ c\ Replace the current line with the line ' A "d" command followed by a "a" command won't work, as I discussed earlier. The "d" command would terminate the current actions. You can combine all three actions using curly braces: #!/bin/sh sed ' /WORD/ { i\ Add this line before a\ Add this line after c\ Change the line to this one }' Leading tabs and spaces in a sed script Sed ignores leading tabs and spaces in all commands. However these white space characters may or may not be ignored if they start the text following a "a," "c" or "i" command. In SunOS, both "features" are available. The Berkeley (and Linux) style sed is in /usr/bin, and the AT&T version (System V) is in /usr/5bin/. To elaborate, the /usr/bin/sed command retains white space, while the /usr/5bin/sed strips off leading spaces. If you want to keep leading spaces, and not care about which version of sed you are using, put a "\" as the first character of the line: #!/bin/sh sed ' a\ \ This line starts with a tab ' Adding more than one line All three commands will allow you to add more than one line. Just end each line with a "\:" #!/bin/sh sed ' /WORD/ a\ Add this line\ This line\ And this line ' Address ranges and the above commands You may remember that earlier I warned you that some commands can take a range of lines, and others cannot. To be precise, the commands "a," "i," "r," and "q" will not take a range like "1,100" or "/begin/,/end/." The documentation states that the read command can take a range, but I got an error when I tried this. The "c" or change command allows this, and it will let you change several lines into one: #!/bin/sh sed ' /begin/,/end/ c\ ***DELETED*** ' If you need to do this, you can use the curly braces, as that will let you perform the operation on every line: #!/bin/sh # add a blank line after every line sed '1,$ { a\ }' Print line number with = The "=" command prints the current line number to standard output. One way to find out the line numbers that contain a pattern is to use: # add line numbers first, # then use grep, # then just print the number cat -n file | grep 'PATTERN' | awk '{print $1}' The sed solution is: sed -n '/PATTERN/ =' file Earlier I used the following to find the number of lines in a file #!/bin/sh lines=$(wc -l file | awk '{print $1}' ) Using the "=" command can simplify this: #!/bin/sh lines=$(sed -n '$=' file ) The "=" command only accepts one address, so if you want to print the number for a range of lines, you must use the curly braces: #!/bin/sh # Just print the line numbers sed -n '/begin/,/end/ { = d }' file Since the "=" command only prints to standard output, you cannot print the line number on the same line as the pattern. You need to edit multi-line patterns to do this. Transform with y If you wanted to change a word from lower case to upper case, you could write 26 character substitutions, converting "a" to "A," etc. Sed has a command that operates like the tr program. It is called the "y" command. For instance, to change the letters "a" through "f" into their upper case form, use: sed 'y/abcdef/ABCDEF/' file Here's a sed example that converts all uppercase letters to lowercase letters, like the tr command: sed 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/' lowercase If you wanted to convert a line that contained a hexadecimal number (e.g. 0x1aff) to upper case (0x1AFF), you could use: sed '/0x[0-9a-zA-Z]*/ y/abcdef/ABCDEF' file This works fine if there are only numbers in the file. If you wanted to change the second word in a line to upper case, and you are using classic sed, you are out of luck - unless you use multi-line editing. (Hey - I think there is some sort of theme here!) However, GNU sed has a uppercase and lowercase extension. The -s Separate argument Normally, when you specify several files on the command line, sed concatenates the files into one stream, and then operates on that single stream. If you had three files, each with 100 lines, then the command sed -n '1,10 p' file1 file2 file3 would only print the first 10 lines of file file1. The '-s' command tells GNU sed to treat the files are independent files, and to print out the first 10 lines of each file, which is similar to the head command. Here's another example: If you wanted to print the number of lines of each file, you could use 'wc -l' which prints the number of lines, and the filename, for each file, and at the end print the total number of lines. Here is a simple shell script that does something similar, just using sed: #!/bin/sh FILES=$* sed -s -n '$=' $FILES # print the number of lines for each file sed -n '$=' $FILES # print the total number of lines. The 'wc -l' command does print out the filenames, unlike the above script. A better emulation of the 'wc -l' command would execute the command in a loop, and print the filenames. Here is a more advanced script that does this, but it doesn't use the '-s' command: #!/bin/sh for F in "$@" do NL=$(sed -n '$=' < "$F" ) && printf " %d %s\n" $NL "$F" done TOTAL=$(sed -n '$=' "$@") printf " %d total\n" $TOTAL The -i in-place argument I've already described in Editing multiple files the way I like to do this. For those who want a simpler method, GNU Sed allows you to do this with a command line option - "-i". Let's assume that we are going to make the same simple change - adding a tab before each line. This is a way to do this for all files in a directory with the ".txt" extension in the current directory: sed -i 's/^/\t/' *.txt The long argument name version is sed --in-place 's/^/\t/' *.txt This version deletes the original file. If you are as cautious as I am, you may prefer to specify an extension, which is used to keep a copy of the original: sed -i.tmp 's/^/\t/' *.txt And the long argument name version is sed --in-place=.tmp 's/^/\t/' *.txt In the last two versions, the original version of the "a.txt" file would have the name "a.txt.tmp". You can then delete the original files after you make sure all worked as you expected. Please consider the backup option, and heed my warning. You can easily delete the backed-up original file, as long as the extension is unique. The GNU version of sed allows you to use "-i" without an argument. The FreeBSD/Mac OS X does not. You must provide an extension for the FreeBSD/Mac OS X version. If you want to do in-place editing without creating a backup, you can use sed -i '' 's/^/\t/' *.txt The --follow-symlinks argument The in-place editing feature is handy to have. But what happens if the file you are editing is a symbolic link to another file? Let's assume you have a file named "b" in a directory called "tmp", with a symbolic link to this file: $ ls -l b lrwxrwxrwx 1 barnett adm 6 Mar 16 16:03 b.txt -> tmp/b.txt If you executed the above command to do in place editing, there will be a new file called "b.txt" in the current directory, and "tmp/b.txt" will be unchanged. Now you have two versions of the file, one is changed (in the current directory), and one is not (in the "tmp" directory). And where you had a symbolic link, it has been replaced with a modified version of the original file. If you want to edit the real file, and keep the symbolic link in place, use the "--follow-symlinks" command line option: sed -i --follow-symlinks 's/^/\t/' *.txt This follows the symlink to the original location, and modifies the file in the "tmp" directory, If you specify an extension, the original file will be found with that extension in the same directory as the real source. Without the --follow-symlinks command line option, the "backup" file "b.tmp" will be in the same directory that held the symbolic link, and will still be a symbolic link - just renamed to give it a new extension. =====================================================================================================