JSON Lines Makes Records Easy to Stream. It Does Not Make an Incomplete File Complete.

A file contains ninety-nine valid records and a hundredth line cut off halfway through an object. A line-by-line importer has already published the first ninety-nine. Should it report success, reject the whole delivery or preserve a partial result for review?

JSON Lines solves a representation problem: one JSON value per line can be handled one record at a time. It does not choose the application's completeness or publication policy. That policy needs to be explicit before the importer encounters an interrupted delivery.

The format documentation specifies UTF-8, valid JSON values on each line and newline separation. This guide builds an original bounded Python importer with a stricter application contract: each record must be an object, and duplicate keys and nonstandard numeric constants are rejected.

Separate format rules from application rules

JSON Lines permits any valid JSON value on a line, including null. A blank line is not a JSON value. A final newline is recommended but not required, so absence of that newline alone does not prove truncation.

An application can impose a narrower schema, such as one object per line with a stable record ID. Label that as an application rule rather than claiming the file format requires every record to be an object.

Know the parser's defaults

Python's json documentation explains that its default decoder accepts nonstandard constants such as NaN and permits repeated object names, retaining the last value. Those defaults may not match a strict import contract.

A parser accepting text does not establish that a numerical value is suitable for the application. A JSON number can be syntactically valid while far outside an allowed range. Preserve the distinction between parsing and domain validation.

Size limits matter too. Reading a complete untrusted file into memory before checking its size defeats a bounded-import design. The example below limits line bytes, total bytes and record count while reading.

Stage a small batch before returning it

The following complete example uses a binary file-like object. Line limits include the line terminator. Decimal preserves parsed decimal numbers without converting a large finite JSON number into a floating-point infinity; application range checks are still required.

import io
import json
from decimal import Decimal


def reject_constant(value):
    raise ValueError(f"Nonstandard numeric constant: {value}")


def unique_object(pairs):
    result = {}
    for key, value in pairs:
        if key in result:
            raise ValueError(f"Duplicate key: {key}")
        result[key] = value
    return result


def stage_jsonl(stream, *, max_line=1024, max_total=8192, max_records=10):
    if min(max_line, max_total, max_records) <= 0:
        raise ValueError("Limits must be positive")
    staged, total = [], 0
    while True:
        raw = stream.readline(max_line + 1)
        if not raw:
            return staged
        total += len(raw)
        if len(raw) > max_line or total > max_total:
            raise ValueError("Input exceeds byte limits")
        if len(staged) >= max_records:
            raise ValueError("Too many records")
        if raw.startswith(b"\xef\xbb\xbf"):
            raise ValueError("Byte order mark is not allowed")
        text = raw.decode("utf-8", errors="strict")
        if not text.strip():
            raise ValueError("Blank line is not a record")
        value = json.loads(text, parse_constant=reject_constant,
                           parse_float=Decimal,
                           object_pairs_hook=unique_object)
        if not isinstance(value, dict):
            raise ValueError("This application requires object records")
        staged.append(value)


rows = stage_jsonl(io.BytesIO(b'{"id":"A","amount":1.25}\n'))
assert rows == [{"id": "A", "amount": Decimal("1.25")}]

This function returns the staged list only after reaching the end of the provided stream. If a later line fails, it raises instead of returning an accepted partial batch. A caller that needs partial recovery should design and label that behavior separately.

Understand the limits of the limits

The example is a small-batch teaching implementation. It limits allocation from line reads and the number of retained records, but it is not a complete sandbox for hostile parsing. Deep nesting can still cause decoder failure, and production systems may need process, time and depth controls appropriate to their input.

The limits should be positive integers supplied by trusted configuration. The sample checks positivity, not a complete configuration schema. Do not let an uploaded record choose its own resource limits.

A compressed source adds another boundary. A small compressed file can expand substantially, so apply limits to the decompressed stream as well as the transport or archive. The example does not implement decompression.

Test malformed and ambiguous records

A duplicate key can hide conflicting values. A blank line can reveal an unexpected exporter behavior. A partial final object can indicate an interrupted write. Rejecting these cases with an explicit reason is more informative than silently skipping them.

Keep a source reference and physical line number in a production error record, while avoiding unnecessary exposure of sensitive record contents. The teaching function keeps error handling compact; an operational importer should add appropriate context around the parser.

Do not catch every exception and return an empty list. That would make a failed delivery look like a successful batch containing no records.

Add a delivery manifest

A file ending after ninety-nine complete objects may be syntactically perfect even if the producer intended one hundred. Parsing alone cannot recover that intention. A separate manifest can state expected count, source version and an integrity digest for the delivered artifact.

Verify the manifest through the delivery contract and compare its expectations with the actual batch. A digest establishes correspondence to particular bytes, not that those bytes contain truthful business facts. The producer's completion signal and the consumer's acceptance rule both matter.

For an actively written file, decide whether the consumer waits for a finalized artifact or follows an append protocol with checkpoints. Do not treat a temporary end-of-file observation as a universal completion signal.

Publish with a clear outcome

After parsing, apply record-schema checks, uniqueness and cross-record requirements. Then publish through the destination's supported transaction or versioning mechanism. Returning a Python list does not make a later series of database writes atomic.

As a self-check, a final line has no newline but contains a complete object. Is it invalid JSON Lines? No. Now ask whether the delivery is complete: that is a separate question requiring the agreed evidence.

For another time-and-completeness distinction, read streaming watermarks and revisions. A usable record and a complete reporting period are different claims.

Sources and limits

• JSON Lines format documentation: encoding, valid values and line termination.
• Python json documentation: decoder behavior, hooks and resource considerations.

Checked October 23, 2026. The code is original and tested on bounded fixtures. It does not provide authentication, a business schema, decompression or an external publication transaction.