Files and data

Process JSON with jq

Extract and validate structured data with jq instead of matching JSON using grep or regular expressions.

10 minute lesson

~~~

Sooner or later a script needs data from an API response or a JSON config file. Resist the urge to grep it. JSON strings can contain whitespace, escapes, and reordered fields — a regex that works today breaks the day the producer changes formatting. A JSON parser understands the structure. On the command line, that parser is jq.

Extract values

Say services.json describes what should run:

{
  "services": [
    { "name": "web", "active": true },
    { "name": "worker", "active": false },
    { "name": "cron", "active": true }
  ]
}

Read active service names:

jq -r '.services[] | select(.active == true) | .name' services.json
# web
# cron

The filter reads left to right: .services[] streams the array elements, select keeps the matching ones, .name extracts the field. The -r flag prints raw strings instead of quoted JSON, which is what a shell script wants to consume.

Capture the result with command substitution when you need it in a variable:

active=$(jq -r '.services[] | select(.active) | .name' services.json)

Verify against real variation

Test reordered properties, escaped strings, and an empty array. jq handles all three identically because it parses structure, not text: the same data with fields in a different order produces the same output. That is the whole argument against grep, demonstrated in thirty seconds.

Let jq control script success

Add jq -e when the filter result should control script success. With -e, the exit status is non-zero when the result is false, null, or missing:

if jq -e '.services[] | select(.name == "web")' services.json >/dev/null; then
  printf '%s\n' 'web is configured'
else
  printf '%s\n' 'web is missing from services.json' >&2
  exit 1
fi

Without -e, jq exits zero even when the filter finds nothing, and your check silently passes. That’s the realistic mistake: a guard that guards nothing.

One safety rule to close. Treat output as data, not shell code — extracted values go into quoted variables and arguments. Never pass it to eval: a hostile "name": "web; rm -rf ~" must stay an odd-looking string, not become a command.

Lesson completed

Take this course offline

Get every free book and course as PDF and EPUB files.

Get the download library →