JSONL: a practical guide to JSON Lines

By

Learn how JSONL stores one JSON value per line, how to write, stream, and query it, and why I use it as Factory Log's event store.

~~~

JSONL is one of those formats that looks almost too simple.

It is JSON, but every line is a separate JSON value.

I use it when I want to keep adding structured records to a file without loading and rewriting the whole file. Logs, events, exports, and communication between small programs are all good uses for it.

Let’s see how it works, then I’ll show you how I use it in Factory Log.

What JSONL looks like

JSONL means JSON Lines. It is also called newline-delimited JSON or NDJSON.

Here is a small events.jsonl file:

{"timestamp":"2026-08-27T08:00:00Z","kind":"task.started","title":"Add search"}
{"timestamp":"2026-08-27T08:18:00Z","kind":"task.reported","summary":"Added the search form"}
{"timestamp":"2026-08-27T08:31:00Z","kind":"task.archived","summary":"Finished search and verified the build"}

This file contains three JSON objects.

Each object is complete on its own. The newline after it marks the end of the record.

The JSON Lines format has three main rules:

Objects are the most common values, but a line can contain any valid JSON value. It can be an array, string, number, boolean, or null too.

A blank line is not a JSON value, so blank lines are invalid.

Files commonly use the .jsonl extension. You will also see .ndjson, especially in systems that follow the NDJSON specification.

JSONL compared to a JSON array

We could store the same records in a regular JSON array:

[
  {"timestamp":"2026-08-27T08:00:00Z","kind":"task.started","title":"Add search"},
  {"timestamp":"2026-08-27T08:18:00Z","kind":"task.reported","summary":"Added the search form"},
  {"timestamp":"2026-08-27T08:31:00Z","kind":"task.archived","summary":"Finished search and verified the build"}
]

This is one JSON value: an array containing three objects.

Adding another record is inconvenient. We must insert a comma and a new object before the closing bracket. Many programs read the array, change it in memory, then rewrite the file.

With JSONL, we append one line:

{"timestamp":"2026-08-27T09:05:00Z","kind":"task.started","title":"Improve the empty state"}

Existing records do not change.

This also makes JSONL useful for streams. A reader can process the first record before the writer produces the last one.

The tradeoff is that the complete file is not one valid JSON document. You cannot pass the whole file to JSON.parse(). You must parse one line at a time.

If JSON syntax is new to you, start with my introduction to JSON.

Write a JSONL file with Node.js

Let’s append an event using Node.js:

import { appendFile } from 'node:fs/promises'

const event = {
  timestamp: new Date().toISOString(),
  kind: 'task.started',
  title: 'Add dark mode'
}

const line = `${JSON.stringify(event)}\n`

await appendFile('events.jsonl', line, 'utf8')

JSON.stringify() turns the object into compact JSON without line breaks. We then add \n and append the result.

Always add the final newline. It makes the next append unambiguous and lets us concatenate JSONL files safely.

Do not pretty-print each object with JSON.stringify(event, null, 2). Pretty printing spreads one value over several physical lines, which breaks the format.

A string may contain a line break, but JSON encodes it as \n inside the string. The record itself still stays on one physical line.

Read a small JSONL file

For a small file, we can read the text and parse each line:

import { readFile } from 'node:fs/promises'

const text = await readFile('events.jsonl', 'utf8')
const lines = text.split('\n')

if (lines.at(-1) === '') {
  lines.pop()
}

const events = lines.map((line, index) => {
  if (line === '') {
    throw new Error(`Blank line at ${index + 1}`)
  }

  return JSON.parse(line)
})

Removing the last empty string handles the recommended trailing newline. We still reject empty lines in the middle of the file.

JSON.parse() tells us when one record is malformed. In a real importer, I would catch that error and include the line number so the person can repair the file.

Stream a large JSONL file

Reading the complete file is fine when it is small. For a large log or export, process it one line at a time:

import { createReadStream } from 'node:fs'
import { createInterface } from 'node:readline'

const lines = createInterface({
  input: createReadStream('events.jsonl'),
  crlfDelay: Infinity
})

for await (const line of lines) {
  if (line === '') {
    throw new Error('Found a blank line')
  }

  const event = JSON.parse(line)
  console.log(event.kind)
}

This keeps memory usage almost constant. The file can contain thousands or millions of records without becoming one huge JavaScript array.

crlfDelay: Infinity makes the reader handle both Unix \n and Windows \r\n line endings.

Inspect JSONL from the command line

JSONL works well with normal command-line tools.

Count the records:

wc -l events.jsonl

Show the three most recent events:

tail -n 3 events.jsonl

Use jq to select archived tasks:

jq -c 'select(.kind == "task.archived")' events.jsonl

jq reads each JSON value and runs the filter on it. The -c option keeps every result on one line.

Use --slurp when you intentionally want one array containing all records:

jq --slurp 'map(select(.kind == "task.archived"))' events.jsonl

Be careful with --slurp on large files. It loads the complete input into memory, removing one of JSONL’s main advantages.

Appending is not the same as concurrency safety

The simple Node.js example works when one process owns the file.

It is not enough when several processes may write at the same time. JSONL defines the record format, but it does not provide locks, transactions, validation, or crash recovery.

Two writers could both inspect the same state and append conflicting records. A process could also stop after writing only part of a line.

For multiple writers, use one dedicated writer, a cross-process lock, or a storage system that provides the guarantees you need.

This distinction matters in Factory Log.

How I use JSONL in Factory Log

Factory Log keeps a private, local history of work completed by coding agents.

An agent starts a task, reports meaningful milestones, then archives the task when the work is finished:

factorylog start \
  --title "Add account settings" \
  --summary "Started the settings screen" \
  --source codex

The command returns JSON containing a generated taskID. The agent preserves that ID for later updates:

factorylog report \
  --task-id task_123 \
  --summary "Added validation and tests"

When the task is complete:

factorylog archive \
  --task-id task_123 \
  --summary "Finished and verified account settings"

Each command appends one event to this file:

~/Library/Application Support/Factory Log/events.jsonl

A real record has this shape:

{"id":"9d61dbef-f829-4437-aaa1-371015450427","kind":"task.started","project":{"name":"flaviocopes.com","path":"/Users/flavio/Projects/flaviocopes.com"},"schemaVersion":1,"source":{"tool":"codex"},"summary":"Drafting a JSONL tutorial grounded in Factory Log's real event store","taskID":"task_f4380f90ddd0488682823b8299376adc","taskTitle":"Write JSONL tutorial","timestamp":"2026-08-27T10:42:53Z"}

Every event is self-contained. It repeats the project, task title, source, and schema version instead of depending on another file or database row.

Events with the same taskID form one task history:

task.started
task.reported
task.reported
task.archived

The native macOS app watches the JSONL file. It groups events by taskID, derives whether each task is Doing or Done, and turns the reports into a dashboard and daily project chronicle.

Factory Log only stores short reports written by the agent. It does not collect source code, diffs, secrets, or terminal output.

Why I chose JSONL for Factory Log

The format matches the product.

A coding agent produces a sequence of events. JSONL lets the CLI append each event immediately without rewriting earlier history.

The file also remains useful without the app. I can open it in a text editor, inspect it with tail, filter it with jq, copy it, or write a different importer later.

I did not need a database server or a custom binary format. The source of truth is one local UTF-8 file made of ordinary JSON records.

But I did not treat appendFile() as a complete storage design.

Several coding agents may report at the same time, so Factory Log uses a sibling lock file. Validation and append happen while holding the same cross-process lock.

The writer opens the data file in append mode, writes one complete newline-terminated record, then calls fsync() before releasing the lock.

Readers handle failure at the record level. If the last write was interrupted and the file ends with an incomplete line, Factory Log ignores that final fragment. Earlier events remain readable.

If a non-empty complete line contains malformed JSON or an unsupported schema version, the app reports its line number while keeping the other readable records available. It blocks new mutations until the broken history is repaired.

Each event includes schemaVersion: 1 so the format can evolve deliberately instead of guessing what an old record means.

This is the part I like most about JSONL: the basic format stays tiny, while the application adds exactly the guarantees it needs.

When JSONL is a good fit

I reach for JSONL when records are naturally sequential and mostly immutable:

I would not use a plain JSONL file when I need frequent updates to old records, indexes across many fields, relational queries, or transactions involving several records.

At that point, I use SQLite or another database.

JSONL sits in a useful place between unstructured text and a database. It keeps the simplicity of a file while giving every line a structure that programs already know how to parse.

Tagged: CLI · All topics

Want me to talk about your product? You can sponsor this site.

~~~

Related posts about cli: