Slice and summarize text with grep, sort, uniq, cut, awk and sed

Logs, CSV exports and command output are all just lines of text. Instead of opening a spreadsheet, you can answer most quick questions with a short pipeline. The examples use a small web server log called access.log:

203.0.113.5 - - [10/Oct/2026:10:00:01] "GET /index.html HTTP/1.1" 200 512
198.51.100.7 - - [10/Oct/2026:10:00:02] "GET /about HTTP/1.1" 404 128
203.0.113.5 - - [10/Oct/2026:10:00:03] "POST /login HTTP/1.1" 200 64
192.0.2.9 - - [10/Oct/2026:10:00:04] "GET /index.html HTTP/1.1" 200 512
203.0.113.5 - - [10/Oct/2026:10:00:05] "GET /missing HTTP/1.1" 404 128

grep: keep matching lines

grep ' 404 ' access.log        # lines containing " 404 "
grep -c ' 404 ' access.log     # count them
grep -v 'GET' access.log       # lines that do NOT match
grep -i 'post' access.log      # ignore case
grep -n 'POST' access.log      # show line numbers
grep -E '40[0-9]' access.log   # extended regular expressions

cut: pick columns

cut splits each line on one delimiter character and keeps the fields you ask for. It is perfect for simple CSV files without quoted commas.

cut -d' ' -f1 access.log             # first space-separated field: the IP
cut -d, -f2 users.csv | tail -n +2    # second CSV column, skip header

sort and uniq: count things

uniq only removes duplicates that sit next to each other, so you almost always sort first. uniq -c adds a count.

$ cut -d' ' -f1 access.log | sort | uniq -c | sort -rn
   3 203.0.113.5
   1 198.51.100.7
   1 192.0.2.9

Useful sort flags: -n numeric, -r reverse, -u unique, -k2 sort by the second field, -t, use comma as the field separator, -h human sizes like 4K and 2M.

awk: columns with logic

awk splits each line into fields $1, $2, and so on (by whitespace by default). $NF is the last field and NF is the number of fields. A rule looks like condition { action }.

# Print IP and path for every 404
awk '$(NF-1) == 404 { print $1, $6 }' access.log

# Sum the bytes column
awk '{ total += $NF } END { print total }' access.log

# CSV: use -F to set the separator
awk -F, 'NR > 1 { print $1 }' users.csv

NR is the current line number, and the END block runs once after the last line, which makes totals and averages easy.

sed: find and replace

sed edits a stream of text. Its most common job is substitution: s/old/new/ replaces the first match on each line, and s/old/new/g replaces all of them.

sed 's/http:/https:/g' links.txt          # print the changed text
sed -n '10,20p' big.log                    # print only lines 10 to 20
sed '/^#/d' config.ini                     # delete comment lines
sed -E 's/([0-9]+) ms/\1ms/' timings.txt   # capture groups with -E

Editing files in place: macOS vs Linux

sed -i changes the file directly, but the flag behaves differently. GNU sed (Linux) takes an optional backup suffix attached to -i. BSD sed (macOS) requires a suffix argument, and an empty one must be written as a separate ''.

# Linux (GNU sed)
sed -i 's/web/frontend/' users.csv

# macOS (BSD sed)
sed -i '' 's/web/frontend/' users.csv

# Works on both: keep a backup copy named users.csv.bak
sed -i.bak 's/web/frontend/' users.csv

The portable -i.bak form is a good habit anyway: if your expression is wrong, the original is still there. Delete the .bak file once you have checked the result.

Putting it together

# Top 3 paths that returned 404
grep ' 404 ' access.log | awk '{ print $6 }' | sort | uniq -c | sort -rn | head -n 3

When a pipeline grows beyond one line, or the data has quoted fields with commas inside, switch to a real CSV or JSON tool such as jq, or a short Python script.