jq cheat script: pretty-print, filter and flatten JSON from a file or stdin

jq is the standard tool for JSON on the command line, but its filter language is a second language to remember. This script wraps the dozen operations most people need behind plain subcommands, prints the jq filter it used so you learn it as you go, and reads either a file or stdin so it fits at the end of a curl or kubectl pipeline. It only reads.

The script

#!/usr/bin/env bash
set -euo pipefail

usage() {
  cat <<USAGE >&2
Usage: $(basename "$0") COMMAND [ARGS] [FILE]

Reads FILE, or stdin when FILE is missing or "-". Never modifies anything.

  pretty                 indent with 2 spaces
  compact                one line
  sorted                 pretty with keys sorted (good for diffs)
  keys                   top-level keys
  paths                  every path in the document, one per line
  type                   type of the top-level value
  count PATH             number of items in the array at PATH (e.g. .items)
  get PATH               value at PATH (e.g. .metadata.name), raw strings
  pick PATH K1,K2,...    for each item in array PATH keep only these keys
  where PATH KEY VALUE   items in array PATH whose KEY equals VALUE
  grep PATH KEY REGEX    items in array PATH whose KEY matches REGEX
  csv PATH K1,K2,...     array PATH to CSV with these columns (header first)
  tsv PATH K1,K2,...     same as csv but tab separated
  lines PATH             one compact JSON object per line (for xargs or while read)
  flatten                nested object to "a.b.c": value pairs
  raw FILTER             run any jq filter you type

Examples:
  curl -s https://api.github.com/repos/jqlang/jq | $(basename "$0") pick . name,stargazers_count
  kubectl get pods -o json | $(basename "$0") where .items .status.phase Running
USAGE
  exit 2
}

command -v jq >/dev/null 2>&1 || { echo "jq is not installed (apt install jq / brew install jq)" >&2; exit 1; }
[[ $# -ge 1 ]] || usage

cmd=$1; shift
args=()
file="-"

case "$cmd" in
  pretty|compact|sorted|keys|paths|type|flatten) [[ $# -le 1 ]] || usage ;;
  count|get|lines|raw) [[ $# -ge 1 && $# -le 2 ]] || usage ;;
  pick|csv|tsv) [[ $# -ge 2 && $# -le 3 ]] || usage ;;
  where|grep) [[ $# -ge 3 && $# -le 4 ]] || usage ;;
  -h|--help|help) usage ;;
  *) echo "unknown command: $cmd" >&2; usage ;;
esac

case "$cmd" in
  pretty|compact|sorted|keys|paths|type|flatten) [[ $# -eq 1 ]] && file=$1 ;;
  count|get|lines|raw) args=("$1"); [[ $# -eq 2 ]] && file=$2 ;;
  pick|csv|tsv) args=("$1" "$2"); [[ $# -eq 3 ]] && file=$3 ;;
  where|grep) args=("$1" "$2" "$3"); [[ $# -eq 4 ]] && file=$4 ;;
esac

if [[ "$file" != "-" && ! -r "$file" ]]; then
  echo "cannot read $file" >&2
  exit 1
fi

jqopts=()
filter=""

case "$cmd" in
  pretty)  filter='.' ;;
  compact) jqopts=(-c); filter='.' ;;
  sorted)  jqopts=(-S); filter='.' ;;
  keys)    jqopts=(-r); filter='if type == "object" then keys[] else "top level is \(type), not an object" end' ;;
  paths)   jqopts=(-r); filter='[paths(scalars)] | map(map(tostring) | join("."))[]' ;;
  type)    jqopts=(-r); filter='type' ;;
  flatten) filter='[paths(scalars) as $p | {key: ($p | map(tostring) | join(".")), value: getpath($p)}] | from_entries' ;;
  count)   filter="${args[0]} | length" ;;
  get)     jqopts=(-r); filter="${args[0]}" ;;
  lines)   jqopts=(-c); filter="${args[0]}[]" ;;
  raw)     filter="${args[0]}" ;;
  pick)
    jqopts=(--arg keys "${args[1]}")
    src="${args[0]}[]"; [[ "${args[0]}" == "." ]] && src="."
    filter="(\$keys | split(\",\")) as \$k | $src | . as \$o | reduce \$k[] as \$key ({}; . + {(\$key): (\$o | getpath(\$key | split(\".\")))})" ;;
  where)
    jqopts=(--arg k "${args[1]#.}" --arg v "${args[2]}")
    filter="${args[0]}[] | select((getpath(\$k | split(\".\")) | tostring) == \$v)" ;;
  grep)
    jqopts=(--arg k "${args[1]#.}" --arg re "${args[2]}")
    filter="${args[0]}[] | select((getpath(\$k | split(\".\")) | tostring) | test(\$re))" ;;
  csv|tsv)
    fmt='@csv'; [[ "$cmd" == "tsv" ]] && fmt='@tsv'
    jqopts=(-r --arg keys "${args[1]}")
    filter="(\$keys | split(\",\")) as \$k | (\$k | $fmt), (${args[0]}[] | . as \$o | [\$k[] as \$key | \$o | getpath(\$key | split(\".\"))] | $fmt)" ;;
esac

[[ -t 2 ]] && printf 'jq %s%s\n' "${jqopts[*]:+${jqopts[*]} }" "'$filter'" >&2

if [[ "$file" == "-" ]]; then
  jq ${jqopts[@]+"${jqopts[@]}"} "$filter"
else
  jq ${jqopts[@]+"${jqopts[@]}"} "$filter" -- "$file"
fi

How to use it

install -m 755 jqc.sh ~/.local/bin/jqc      # or anywhere on your PATH

# Pretty-print an API response
curl -s https://api.github.com/repos/jqlang/jq | jqc pretty

# Just two fields
curl -s https://api.github.com/repos/jqlang/jq | jqc pick . full_name,stargazers_count

# Work with a file
jqc keys package.json
jqc get .scripts.test package.json
jqc count .dependencies package.json

# Arrays of objects
jqc where .items .status.phase Running pods.json
jqc grep .items .metadata.name '^web-' pods.json
jqc csv .items metadata.name,status.phase pods.json
jqc lines .items pods.json | while read -r item; do echo "$item" | jq -r .metadata.name; done

# Compare two configs regardless of key order
diff <(jqc sorted a.json) <(jqc sorted b.json)

# Anything else
jqc raw '.items | map(.status.phase) | group_by(.) | map({(.[0]): length}) | add' pods.json

Whenever stderr is a terminal the script prints the exact jq command it ran, so after a few uses you can type the filter yourself. Redirecting stderr hides that line in pipelines and cron.

Why it is safe

• jq never modifies its input, and the script passes the file name after -- so a name starting with - cannot become an option.
• Keys and values from the command line go into jq through --arg, never by string interpolation into the filter, so a value such as "); rm is just text. Only the PATH argument and the filter given to raw are inserted into the filter text; type those yourself, not from untrusted input.
• Careful: jqc pretty data.json > data.json destroys the file. The shell truncates data.json before jq opens it. Write to a new name, or use sponge from moreutils.

Official documentation

• jq manual
• jq download and installation