Duplicate Keys in JSON: What the Spec Says and What Parsers Actually Do

Short answer: Duplicate keys are not forbidden by the JSON specification. RFC 8259 says object names should be unique and that behavior is unpredictable when they are not. In practice almost every mainstream parser (JavaScript, Python, Go, Jackson, jq, PostgreSQL jsonb) silently keeps the last value, while a few (Rust serde into a struct, Jackson in strict mode) raise an error. Never rely on either behavior; treat a duplicate key as invalid input.

A config file gets merged by two people in the same week. Each adds a "timeout" line in a different place. The file is still valid JSON, the validator is happy, the deploy goes out, and one of the two values is simply gone. Nobody finds out until the wrong timeout fires. That is the whole problem with duplicate keys in one paragraph: they are legal, they are silent, and the result depends on which parser happened to read the file.

Browse the parsed structure in the JSON Viewer and compare two sample outputs with JSON Compare. Keep the precision and duplicate-key caveats in this guide in mind when interpreting parsed results.

What RFC 8259 Says About Duplicate Keys

Section 4 of RFC 8259 is short enough to quote in spirit: the names within an object SHOULD be unique. When they are unique, all implementations agree on the result. When they are not, the spec notes that implementations have been observed to report the last pair only, report an error, or report all pairs, and that software consuming such an object has unpredictable behavior.

"SHOULD" is the key word. In RFC language it means there may be valid reasons to ignore the rule, but the implications must be understood. So a document like this is well-formed JSON:

{ "role": "user", "role": "admin" }

It will pass any grammar-level validator. What it means is undefined. ECMA-404, the other JSON standard, is even less committal: it does not require uniqueness and leaves the semantics entirely to the application.

What Each Parser Does With Duplicate Keys

Here is the observed behavior of common parsers. The pattern is overwhelmingly "last wins", because the natural implementation is to insert each pair into a map as it is read.

Seeing it is more convincing than reading it:

// JavaScript
JSON.parse('{"a": 1, "a": 2}');
// { a: 2 }

# Python
>>> import json
>>> json.loads('{"a": 1, "a": 2}')
{'a': 2}

# jq
$ echo '{"a": 1, "a": 2}' | jq -c .
{"a":2}

And the two that push back:

// Rust, deserializing into a struct
#[derive(Deserialize)]
struct Cfg { a: u32 }
serde_json::from_str::<Cfg>(r#"{"a": 1, "a": 2}"#);
// Err: duplicate field `a` at line 1 column 15

// Java Jackson with strict detection
ObjectMapper mapper = new ObjectMapper();
mapper.enable(JsonParser.Feature.STRICT_DUPLICATE_DETECTION);
mapper.readTree("{\"a\": 1, \"a\": 2}");
// JsonParseException: Duplicate field 'a'

The PostgreSQL row deserves a second look because it is the one place where two behaviors live side by side. The json type stores the text exactly as written, duplicates included, and reprocesses it on every query. The jsonb type parses on insert and, as documented, keeps only the last value. The PostgreSQL and MySQL JSON guide covers the other differences between the two types.

Why Duplicate Keys Are a Security Problem

One parser, one answer, no harm done. The danger appears when a request passes through two parsers that disagree. Consider a payment flow:

1. An API gateway written in one language validates the request and checks that amount is within the user's limit.
2. The backend, written in another language, reads the same body and charges the card.

{ "amount": 10, "amount": 10000 }

If the gateway happens to take the first value (some streaming validators and hand-rolled parsers do), it approves 10. If the backend takes the last, it charges 10000. Neither component has a bug on its own; the mismatch is the exploit. Security researchers have documented this class of "JSON interoperability" or parser-differential attack across many parser pairs, including cases where one side treats the key as a duplicate and the other does not because of Unicode normalization or comment handling.

The defensive rules follow directly:

• Reject duplicate keys at the outermost boundary, before any other component sees the body.
• Parse once and pass the resulting object downstream, not the raw text.
• Where a second parse is unavoidable, use the same parser library and settings on both sides.

The rest of the hardening checklist is in JSON security best practices.

How to Detect Duplicate Keys in JSON

The catch with detection is that a normal parser has already collapsed the duplicate by the time you get an object back. You need something that sees the raw pairs.

Python: object_pairs_hook

json.loads accepts a hook that receives the ordered list of pairs for every object before the dict is built. Raise inside it and the whole parse fails with a useful message:

import json

def reject_duplicates(pairs):
    seen = {}
    for key, value in pairs:
        if key in seen:
            raise ValueError(f"duplicate key: {key!r}")
        seen[key] = value
    return seen

json.loads(text, object_pairs_hook=reject_duplicates)
# ValueError: duplicate key: 'role'

This is cheap enough to leave on permanently for config loading and inbound webhooks. Swap the raise for a logging call if you only want visibility.

Command line: jq --stream

In streaming mode jq emits one event per leaf without building the object, so both values of a duplicate key show up. Extract the paths, sort them, and let uniq -d print the repeats:

jq -c --stream 'select(length == 2) | .[0]' config.json | sort | uniq -d
# ["role"]
# ["servers",0,"timeout"]

An empty result means no duplicate leaf paths. Array elements have distinct indexes, so repeated values inside an array will not produce false positives.

Editor and tool checks

VS Code's built-in JSON language service underlines the second occurrence of a key with a "Duplicate object key" warning, which catches most hand-edited mistakes before they are saved. The validating JSON in VS Code post shows where that and related settings live.

A browser formatter cannot flag duplicates directly, but it can expose them. Paste the document into JSON Prettifier, turn on Sort Keys, and format. Then compare the key list against your source: a key that appears twice in the source and once in the sorted output is your duplicate, and the value that survived is the last one. For large files this is faster than it sounds, because sorted keys are easy to scan.

JavaScript: no hook, so validate elsewhere

JSON.parse has a reviver argument, but it runs after each object is built, so it never sees the losing value. If you need the check in Node, either tokenize the text with a streaming parser, run the Python or jq check in CI, or add a lint rule: the JSON plugins for ESLint include a no-duplicate-keys rule for .json files in the repository.

Where Duplicate Keys Come From

You cannot produce a duplicate from JSON.stringify, json.dumps, or any serializer that starts from a map, because the map already de-duplicated. Duplicates come from text manipulation:

• Hand-edited config files, especially after merge conflicts.
• String concatenation or templating that assembles JSON from fragments.
• Scripts that append a key with sed instead of parsing, editing, and re-serializing.
• Case changes: userId and userid are different keys to every parser, but a case-insensitive consumer may treat them as the same field. Consistent naming, as discussed in JSON key naming conventions, removes that ambiguity.

The fix is the same each time: load, modify the object, serialize. The moment you treat JSON as text rather than data, duplicates become possible.

Common Mistakes

• Assuming a validator catches duplicates. Grammar validation checks syntax. Most validators, including browser-based ones, accept duplicate keys without comment because the spec allows them. The errors a validator does catch are listed in common JSON errors.
• Relying on "last wins" as a feature. Some people deliberately put a default first and an override second. It works until the file is read by a tool that keeps the first value, or a strict parser rejects it outright.
• Using duplicate keys for multi-valued fields. "tag": "a", "tag": "b" is not a list. Use an array.
• Forgetting that json and jsonb differ. Migrating a PostgreSQL column from one to the other can change what a query returns if the stored documents contain duplicates.