I. Whitespace Normalization

Replace every whitespace character with a single space character, preserving count (no collapsing). Whitespace here means the full POSIX [[:space:]] class: horizontal tab, vertical tab, form feed, carriage return, and space. Newlines require special handling because ed and sed are line-at-a-time tools; see the tr entry for newline-inclusive mapping.

A. ed — featured one-liner

The ed approach operates in-place with no temporary file. The address , is shorthand for 1,$ (all lines); the substitution replaces each whitespace character 1:1 with a space; w writes back; q quits.

EXAMPLE 1 — ed in-place whitespace-to-space, count preserved

ed file.txt <<'EOF'
,s/[[:space:]]/ /g
w
q
EOF

The heredoc delimiter is single-quoted ('EOF') to suppress shell expansion inside the script body. Double-quoting or leaving it bare would allow the shell to expand $variables and backticks before ed sees them — almost never what is intended.

NOTE. ,s/[[:space:]]/ /g does not touch the newline that terminates each line; that byte is the record separator consumed by ed before the substitution runs. To also flatten newlines, join all lines first: prepend ,j (join) before the substitution, or use tr (Example 3 below).

B. ed — silent mode

By default ed prints the byte count after w and the line count after an error. Pass -s to suppress all diagnostic output, which is useful in scripts:

  ed -s file.txt <<'EOF'
  ,s/[[:space:]]/ /g
  w
  q
  EOF

C. sed — stdout, count preserved

sed reads line-at-a-time and writes to standard output. The same POSIX character class applies. Redirect to capture the result.

  sed 's/[[:space:]]/ /g'  file.txt

For in-place editing, GNU sed accepts -i; BSD/macOS sed requires an explicit (possibly empty) backup suffix:

  sed -i     's/[[:space:]]/ /g' file.txt   # GNU (Linux)
  sed -i ''  's/[[:space:]]/ /g' file.txt   # BSD / macOS

D. tr — stdin→stdout, newlines included

tr maps characters 1:1 at the byte stream level, so newlines are visible to it unlike the line-at-a-time tools. This is the only standard utility that handles all whitespace in a single expression without preprocessing:

  tr '[:space:]' ' ' < file.txt

Note the absence of -s (squeeze); count is preserved. The output is a single unbroken line if the input contained newlines.

E. Comparison

Tool	Handles newlines	True in-place	No temp file	POSIX
ed	No (line-at-a-time)	Yes	Yes	Yes
sed (stdout)	No	No	Yes	Yes
sed -i	No	Emulated	No (temp file)	No
tr	Yes	No (stdout only)	Yes	Yes
awk	No	No	Yes	Yes

F. Collapsing vs. Preserving Count

For reference: to collapse runs of whitespace into a single space (the opposite of the above), use the + quantifier in sed or the -s flag in tr.

  sed 's/[[:space:]]\+/ /g'  file.txt   # GNU sed: collapse runs
  sed 's/[[:space:]][[:space:]]*/  /g'  file.txt   # POSIX sed: same
  tr -s '[:space:]' ' '  < file.txt    # tr: collapse and flatten

II. In-Place File Editing

ed is the only standard utility that edits files in place without a temporary file. It loads the file into a buffer, applies edits, and writes back to the same inode. The file’s hard link count, inode number, and permissions are unchanged.

sed -i creates a new file and renames it over the original. This resets the inode and breaks hard links. It is not POSIX. On macOS, the -i '' form is required.

A. Global Substitution

  ed file.txt <<'EOF'
  ,s/foo/bar/g
  w
  q
  EOF

B. Addressed Range

Edit only lines 5 through 10. Addresses precede the s command exactly as in sed.

  ed file.txt <<'EOF'
  5,10s/WARNING/ERROR/g
  w
  q
  EOF

C. Pattern-Addressed Edit

The g/pattern/cmd form applies cmd to every line matching the pattern. This is the origin of the name grep: g/re/p (global, regular expression, print).

  ed file.txt <<'EOF'
  g/^#/d
  w
  q
  EOF

Deletes every line that begins with # (comment lines in many configuration formats).

D. Append Text After a Matching Line

  ed file.txt <<'EOF'
  /^SECTION/a
  --- inserted line ---
  .
  w
  q
  EOF

The a command appends; the . on its own line ends the appended text. Likewise, i inserts before and c changes (replaces) the addressed line.

E. Delete Blank Lines

  ed -s file.txt <<'EOF'
  g/^$/d
  w
  q
  EOF

The regex ^$ matches a line whose only content is the newline itself — an empty line. Lines containing only spaces or tabs are not matched; use ^[[:space:]]*$ for those.

III. Line Operations

A. Print a Specific Line

  sed -n '5p'           file.txt   # line 5
  awk 'NR==5'           file.txt   # same
  ed -s file.txt <<'EOF'
  5p
  q
  EOF

B. Print a Range of Lines with Line Numbers

The ed n command is the numbered counterpart to p: it prints each addressed line prefixed by its line number and a tab. 5,10n prints lines 5 through 10 with their absolute file line numbers prepended.

  ed -s file.txt <<'EOF'
  5,10n
  q
  EOF

Equivalents outside ed:

  grep -n '' file.txt | sed -n '5,10p'          # grep -n '' injects numbers inline
  awk 'NR>=5 && NR<=10 {print NR"\t"$0}' file.txt
  cat -n file.txt      | sed -n '5,10p'          # number first, then slice

The three equivalent forms differ subtly: grep -n '' uses a colon separator (5:line text); ed n uses a tab; awk and cat -n let you choose the delimiter in the format string. Only the ed form operates in-place and prints selectively without passing the whole file through a second tool.

C. Print a Range of Lines

  sed -n '5,10p'        file.txt
  awk 'NR>=5 && NR<=10' file.txt
  ed -s file.txt <<'EOF'
  5,10p
  q
  EOF

D. Print Last N Lines Without tail

  awk -v n=5 'NR>n { print buf[NR%n] } { buf[NR%n]=$0 }' file.txt

Maintains a circular buffer of n lines. Once the line count exceeds n, printing the oldest slot gives the trailing n lines on termination.

E. Number Lines

  nl                    file.txt   # skips blank lines by default
  cat -n                file.txt   # numbers every line including blank
  awk '{printf "%6d\t%s\n", NR, $0}' file.txt  # custom format

F. Reverse Line Order Without tac

  sed -n '1!G;h;$p'     file.txt
  awk '{a[NR]=$0} END{for(i=NR;i>=1;i--) print a[i]}' file.txt

The sed idiom: G appends the hold space to the pattern space; h copies back; on the last line ($), print. The hold space accumulates lines in reverse.

G. Deduplicate Adjacent Identical Lines

  uniq                  file.txt   # standard utility
  awk '!seen[$0]++'     file.txt   # deduplicate non-adjacent lines too
  sed '$!N;/^\(.*\)\n\1$/!P;D'  file.txt  # sed sliding-window uniq

H. Count Lines, Words, Bytes

  wc -l file.txt    # lines
  wc -w file.txt    # words
  wc -c file.txt    # bytes
  awk 'END{print NR}' file.txt  # line count without wc

IV. Pattern Substitution

A. Case-Insensitive Substitution (GNU sed)

  sed 's/error/ERROR/gI'  file.txt   # GNU: I flag = case-insensitive
  awk '{gsub(/error/,"ERROR"); print}' file.txt  # awk: always case-sensitive
  # POSIX portable: bracket-expand each letter
  sed 's/[eE][rR][rR][oO][rR]/ERROR/g' file.txt

B. Substitute Only on Lines Matching a Pattern

  sed '/^WARN/s/low/HIGH/g'   file.txt
  awk '/^WARN/{gsub(/low/,"HIGH")} 1' file.txt

The /pattern/ prefix is an address; the s command runs only on lines where the address matches. The trailing 1 in the awk form is a true condition that triggers the default print action for every line.

C. Substitute on Lines NOT Matching a Pattern

  sed '/^#/!s/old/new/g'    file.txt   # skip comment lines

The ! negates the address.

D. Multi-Expression sed

  sed -e 's/foo/bar/g' \
      -e 's/baz/qux/g' \
      -e '/^$/d'        file.txt

Each -e expression is applied in sequence on every line. Equivalent in ed with multiple commands in the heredoc.

E. Back-References

  sed 's/\(word\)/[\1]/g'          file.txt   # POSIX BRE
  sed -E 's/(word)/[\1]/g'         file.txt   # ERE with -E (GNU/BSD)
  awk '{gsub(/word/,"[&]")} 1'     file.txt   # awk: & = matched text

In awk, & in the replacement string refers to the entire matched text. In sed BRE, \1 through \9 refer to captured groups.

V. Field Extraction

A. Single Field by Delimiter

  cut -d: -f1           /etc/passwd   # username field
  awk -F: '{print $1}'  /etc/passwd   # same
  cut -d, -f2,4         data.csv      # fields 2 and 4 from CSV

B. Last Field (Variable Column Count)

  awk '{print $NF}'     file.txt   # NF = number of fields
  rev file.txt | cut -d' ' -f1 | rev   # last space-delimited word

C. Character Column Range

  cut -c1-8             file.txt   # characters 1 through 8
  cut -c10-             file.txt   # character 10 to end of line
  awk '{print substr($0,10)}'  file.txt  # same with awk

D. Sum a Numeric Column

  awk '{sum+=$2} END{print sum}'   file.txt   # sum of column 2
  awk -F: '{sum+=$3} END{print sum}' /etc/passwd  # sum UIDs

E. Unique Values in a Field

  awk -F: '{print $7}' /etc/passwd | sort -u   # unique shells
  awk -F: '!seen[$7]++ {print $7}' /etc/passwd  # same, order-preserving

VI. Process Substitution and Redirection

A. Diff Two Command Outputs

  diff <(sort file1.txt) <(sort file2.txt)

<(cmd) is a bash process substitution: it presents the output of cmd as a named pipe (file descriptor), which diff reads as if it were a file. Requires bash or zsh; not POSIX sh.

B. Tee to File and Stdout Simultaneously

  some-command | tee output.log | grep ERROR

C. Heredoc as Input to Any Command

  sort <<'EOF'
  banana
  apple
  cherry
  EOF

Single-quoting the delimiter disables variable expansion and command substitution inside the heredoc body. Leave the delimiter unquoted to allow expansion.

D. Herestring (Single-Value stdin)

  wc -w <<<"count the words in this string"
  grep -o '[0-9]\+' <<<"error at line 42, column 7"

<<< feeds a single string as standard input. Available in bash, zsh, and ksh93.

E. Capture stderr Separately

  cmd 2>errors.log          # stderr to file, stdout to terminal
  cmd >out.log 2>err.log    # both redirected separately
  cmd >combined.log 2>&1   # stderr merged into stdout stream

VII. tr Idioms

tr operates on the raw byte stream, making it the right choice when character-level mapping must cross line boundaries. It does not support regex; it maps character sets 1:1.

A. Case Conversion

  tr '[:lower:]' '[:upper:]' < file.txt   # uppercase
  tr '[:upper:]' '[:lower:]' < file.txt   # lowercase
  awk '{print toupper($0)}' file.txt        # awk equivalent

B. Delete a Character Class

  tr -d '[:digit:]'  < file.txt   # strip all digits
  tr -d '\r'         < file.txt   # strip carriage returns (DOS→Unix)
  tr -dc '[:print:]' < file.txt   # keep only printable chars

-d deletes; -c complements the set (inverts). Combined, -dc set means “delete everything not in set”.

C. Squeeze Repeated Characters

  tr -s ' '          < file.txt   # collapse runs of spaces to one
  tr -s '[:space:]' ' ' < file.txt   # all whitespace to single space

-s squeezes; this is the collapse behavior, in contrast to the count-preserving form without -s from Section I.

D. ROT13

  tr 'A-Za-z' 'N-ZA-Mn-za-m' < file.txt

Notes

POSIX compliance: ed, sed, tr, awk, cut, sort, uniq, wc, and nl are all specified by POSIX.1-2017. Process substitution (<()), sed -i, sed -E, and herestrings (<<<) are extensions not covered by POSIX.
macOS / BSD sed vs. GNU sed: BSD sed (macOS default) requires -i '' for in-place editing with no backup. GNU sed (Linux default, Homebrew gsed on macOS) accepts bare -i. The I flag for case-insensitive substitution is GNU-only. For portable scripts, prefer ed for in-place work.
ed address forms: . = current line • $ = last line • , = 1,$ (all lines) • % = same as , in some implementations • /pat/ = next line matching pattern • ?pat? = previous line matching pattern • n,m = line range • +n / -n = relative offset.
awk field separator: Default field separator is any run of whitespace. Set with -F on the command line or FS= in a BEGIN block. Setting FS=" " (a literal space) restores the default whitespace-splitting behavior; setting FS=" " (space) is not the same as the default.
Character class portability: [[:space:]], [[:digit:]], [[:alpha:]], and the other POSIX named classes are portable across locales and character encodings. Avoid [a-z] range notation in locale-aware contexts; the range includes locale-collation characters on some systems.

WARNING. All destructive one-liners (ed with w, sed -i) modify the file immediately and without confirmation. Test the substitution expression against a copy or use ed’s p command to preview before issuing w. There is no undo.

Name

Synopsis