JSON Lines (NDJSON) Explained: When One Object Per Line Beats an Array

Quick answer: JSON Lines (JSONL, also called NDJSON) is a text format where each line is one complete, compact JSON value, usually an object, separated by newline characters. There is no outer array and no commas between records. It exists so that files can be streamed, appended to, and split without parsing the whole thing, which is why logs, bulk imports, and machine-learning datasets use it. A .jsonl file is not itself valid JSON; each line is.

A service writes one JSON object per request into a log file. After a month the file is 6 GB. If it were a JSON array, nothing could read it without loading all 6 GB, and the process that wrote it would have had to keep seeking back to fix the closing bracket. Because it is JSON Lines, tail -f shows the latest request, grep finds an order ID, and a Python loop processes it in constant memory. That is the entire pitch for the format.

Inspect complete individual records in the JSON Viewer, or use the JSONPath Query Tester to select fields from a parsed record. Neither tool replaces an incremental stream parser.

What JSON Lines Is, Precisely

The specification fits in four rules:

1. The file is UTF-8 encoded text.
2. Each line is a complete, valid JSON value. Objects are overwhelmingly the norm, but any value is allowed.
3. Lines are separated by \n. A \r\n ending is tolerated by most readers because JSON treats the carriage return as whitespace.
4. The last line may or may not end with a newline. Readers should accept both.

Nothing else. No opening [, no trailing commas, no header line. Because a line break ends a record, every value must be written in compact form; you cannot pretty-print a record across several lines. Newlines inside strings are not a problem, since JSON already escapes them as \n.

Here is the same three users as a JSON array and as JSON Lines:

// users.json (one document)
[
  { "id": 1, "name": "Ada", "role": "admin" },
  { "id": 2, "name": "Linus", "role": "dev" },
  { "id": 3, "name": "Grace", "role": "dev" }
]

// users.jsonl (three documents)
{"id":1,"name":"Ada","role":"admin"}
{"id":2,"name":"Linus","role":"dev"}
{"id":3,"name":"Grace","role":"dev"}

The second form is slightly shorter, but size is not the point. The point is that line 2 can be read, skipped, or rewritten without touching lines 1 and 3.

Why One Object Per Line Beats an Array

A JSON array is a single value. A parser must see the closing ] before it can hand you anything, and a writer must emit that bracket last. Those two facts cause every operational pain point with large JSON files, and JSON Lines removes both:

The last two rows are the cost. Because the file is not one document, generic JSON tools reject it, and you cannot format the whole thing as a single indented tree. For a 50-record config file, that trade is a bad one; use an array. For a 50-million-record export, it is the only sane choice. The JSON file size limits guide covers where arrays start to hurt in practice.

A quick rule for choosing: if the data is a bounded document that something reads whole, such as a config file, an API response, or a fixture, use an array. If records arrive over time, are produced by one process and consumed by another, or will not fit in memory, use JSON Lines. When in doubt, ask whether anyone will ever want to tail the file. If yes, it is JSON Lines.

Where You Will Meet JSON Lines

• Application logs. Structured loggers such as pino and Docker's default json-file driver write one JSON object per line. The structured logging guide covers the field conventions.
• LLM fine-tuning and evaluation datasets. Training files are commonly JSONL, one example per line, so a dataset can be shuffled, sampled, and split with plain Unix tools.
• Bulk database loads. BigQuery accepts newline-delimited JSON as a load format. Elasticsearch's _bulk API takes NDJSON where action lines and document lines alternate.
• Data exports and warehouse dumps. Export jobs that stream rows naturally produce one line per row, and downstream tools can process them in parallel by splitting the file.
• Streaming HTTP responses. An API can send results as they are ready, each on its own line, and the client can start processing before the response finishes.

Reading and Writing JSON Lines in Python

No library needed. Iterate the file, skip blank lines, parse each one:

import json

def read_jsonl(path):
    with open(path, encoding="utf-8") as f:
        for line_no, line in enumerate(f, start=1):
            line = line.strip()
            if not line:
                continue
            try:
                yield json.loads(line)
            except json.JSONDecodeError as e:
                raise ValueError(f"line {line_no}: {e.msg}") from e

def write_jsonl(path, records):
    with open(path, "w", encoding="utf-8") as f:
        for record in records:
            f.write(json.dumps(record, ensure_ascii=False) + "\n")

errors = [r for r in read_jsonl("app.jsonl") if r.get("level") == "error"]
write_jsonl("errors.jsonl", errors)

read_jsonl is a generator, so the 6 GB log from the introduction never sits in memory at once. Re-raising with the line number is the detail that saves you an hour when a single bad record is buried in the middle of the file. Do not pass indent to json.dumps here; it would spread one record over several lines and corrupt the file.

Reading and Writing JSON Lines in Node.js

Node's readline module splits a stream on newlines, which is exactly the boundary you need:

import fs from "node:fs";
import readline from "node:readline";

async function* readJsonl(path) {
  const rl = readline.createInterface({
    input: fs.createReadStream(path, "utf8"),
    crlfDelay: Infinity,
  });
  let lineNo = 0;
  for await (const line of rl) {
    lineNo++;
    if (!line.trim()) continue;
    try {
      yield JSON.parse(line);
    } catch (err) {
      throw new Error("line " + lineNo + ": " + err.message);
    }
  }
}

const out = fs.createWriteStream("errors.jsonl");
for await (const record of readJsonl("app.jsonl")) {
  if (record.level === "error") {
    out.write(JSON.stringify(record) + "\n");
  }
}
out.end();

crlfDelay: Infinity makes \r\n count as a single line break, which matters for files that passed through Windows. JSON.stringify with no indent argument produces the compact form the format requires. For more on the parse side, see JSON.parse and JSON.stringify in JavaScript.

JSON Lines With jq

jq reads JSON Lines natively: given a stream of values it applies the filter to each one. The -c flag keeps output compact, which is what makes jq the standard converter in both directions:

-s (slurp) reads every value into one array before running the filter, so it loads the whole file. -n '[inputs]' gives the same result but is the idiom jq documentation recommends for large inputs. The validation row works because jq exits non-zero and prints the line where parsing failed. For the full flag list, keep the jq cheat sheet open.

Converting Without jq

If jq is not installed, two Python one-liners do the job. Array to JSON Lines:

python -c "import json,sys; [print(json.dumps(o)) for o in json.load(sys.stdin)]" < data.json > data.jsonl

And JSON Lines back to an array:

python -c "import json,sys; print(json.dumps([json.loads(l) for l in sys.stdin if l.strip()], indent=2))" < data.jsonl > data.json

Gotcha: A .jsonl File Is Not Valid JSON

Paste a JSON Lines file into any JSON validator and it fails. The parser reads the first object, then finds { where it expects the end of input and reports something like Unexpected token '{' at line 2, column 1. This is correct behaviour; the file was never one document. Three ways to work with it:

1. Validate one record. Copy a single line and paste it into JSON Prettifier. It formats, validates, and highlights that record on its own, which is usually what you want when debugging a specific log entry.
2. Validate the whole file. Run jq -c . file.jsonl > /dev/null, or the Python reader above, which reports the exact failing line.
3. Format the whole file. Convert it to an array with jq -s . and open the result in the formatter. For files larger than a few megabytes, the large JSON validation guide explains what the browser can handle.

Common Mistakes

• Pretty-printing records. Any writer configured with an indent produces multi-line records. The next reader sees { alone on a line and fails. Always serialize compact.
• Adding commas or brackets. Someone "fixes" the file into an array by hand, then the streaming reader breaks on line 1. Pick one format per file and name it accordingly.
• Blank lines. Many readers tolerate them, some do not. Skip them on read and never write them.
• Byte order mark. A BOM at the start of the file makes line 1 fail to parse while every other line succeeds. Strip it, or open with utf-8-sig in Python.
• Mixed shapes. The format allows each line to be a different structure, but your consumers will not thank you. Keep a consistent set of keys, and see arrays vs objects for choosing the record shape.
• Unescaped newlines in strings. If you build lines by string concatenation rather than a serializer, a multi-line message splits one record into two broken ones.

For AI responses that deliver fragments of one document rather than independent lines, see how to parse streaming JSON from AI APIs safely. The completion and validation rules differ from an NDJSON record stream.