Robin van der Vleuten

A JSONPath engine that follows the spec and can't run code

I needed to pull a few fields out of some JSON. Nothing exotic, a couple of nested values and one filtered list. So I did the ordinary thing, reached for a JSONPath library, wrote a query, and moved on.

A week later I swapped that library for an unrelated reason, and the same query came back with different results. Not wrong exactly, just different. My $..author matched one extra node, a slice counted from the other end, and a filter that worked before now threw. Welcome to JSONPath, where for the better part of two decades every implementation agreed on the dollar sign and improvised the rest.

The standard nobody had

Stefan Gössner's original write-up from 2007 sketched the idea and left the corners to the reader. Everyone filled them in differently. Does $..book[0] give you the first book in each array or the first book overall? Do you write a filter [?(@.price)] or [[email protected]]? What does a negative index do at the edge of an array? Pick two libraries and you could get two answers to each.

That changed in February 2024, when the IETF published RFC 9535 and wrote the rules down: the selectors, the filter grammar, the meaning of a missing value, with a compliance test suite so you can check yourself against them. JSONPath went from folklore to specification.

So I wanted a library that actually followed it. Reading the spec, I noticed the part that makes JSONPath worth using is also the part that keeps turning up in security advisories.

The filter that runs a shell

Filters are the reason you pick JSONPath over a hand-written loop. $..book[[email protected] < 10] reads like a sentence and does real work: keep the nodes where the comparison holds.

Evaluating that comparison is where libraries cut corners. jsonpath-plus, the most-downloaded option on npm, built its filters on generated code. That works right up until part of the query comes from somewhere you don't control. At that point a filter is no longer a filter, it is a program running with your process's permissions, and that is exactly how it went: CVE-2024-21534 was a remote code execution through a crafted JSONPath, with more to follow when the first patch left a gap.

There is a quieter cost too. Generating code at runtime needs 'unsafe-eval' in your Content Security Policy, and if you have already shipped a strict CSP, granting that back undoes most of the reason you shipped it.

A scout that stays on the path

padvinder is my answer to both. The name is Dutch for "pathfinder", and also the word for a scout, which fit something whose entire job is to go find a node and come back without disturbing anything on the way. It implements RFC 9535 in roughly 2.75KB min+gzip with no dependencies, and it clears all 456 valid-selector cases in the official suite.

import { query, find } from 'padvinder';

const data = { store: { book: [
  { title: 'Sayings of the Century', price: 8.95, category: 'reference' },
  { title: 'Sword of Honour', price: 12.99, category: 'fiction' },
  { title: 'Moby Dick', price: 8.99, category: 'fiction' },
] } };

find('$..book[[email protected] < 10].title', data);
// => ['Sayings of the Century', 'Moby Dick']

find runs a query once. For a path you will reuse, query compiles it and returns a runner you can point at different data:

const fiction = query('$.store.book[[email protected] == "fiction"].title');
fiction(data); // => ['Sword of Honour', 'Moby Dick']

Hand it a broken path and it throws a SyntaxError at compile time, not partway through your data. A path that matches nothing returns [].

The parts of the spec worth carrying around

Selectors are the dependable half. If you have written JSONPath they hold no surprises: $ for the root, .name or ['name'] for a child, [0] and [-1] for indexes, [1:3:2] for a stepped slice, * for every child, .. for a recursive descent, and [0, 2] to union a handful of selectors.

Filters are where RFC 9535 spends its pages, and a few of its rulings are worth keeping in your head:

  • Existence is not truthiness. A bare [[email protected]] keeps every child that has a discount, even one set to null or 0. It asks whether the path exists, not whether the value is truthy.

  • A missing path is "nothing", not an error. In a comparison, [[email protected] == 1] just fails to match when a is absent instead of throwing, so filters stay safe to write against ragged data.

  • == is deep equality. [[email protected] == ["new", "sale"]] compares structure, and none of JavaScript's loose coercion tags along.

  • $ is still the root inside a filter. A child can be measured against something elsewhere in the document: [[email protected] > $.store.bicycle.price].

Five functions ship built in, taken straight from the RFC: length(), count(), value(), and the regex tests match() for a whole string and search() for a substring. For anything else you pass your own and call it with @ as the current node:

find('$..book[?discounted(@)].title', data, { discounted: b => b.price < 9 });

That registry is the single place executable code enters, and it enters from your side, by name. Whoever writes the query chooses which of your functions to call, never what it does.

The part I tried to borrow

padvinder started out leaning on xprsn. A JSONPath filter is an expression over the current node, so my first version handed each [?...] straight to xprsn and let it do the evaluating. The easy queries passed. The spec did not.

RFC 9535 filters only look like ordinary expressions. Their operands are queries as often as values, so @.a.b is a small path that runs and may match nothing at all. A bare @.discount is an existence test, not a truthiness check. Equality is structural. A missing singular query turns into a special "nothing" that compares equal to no value on earth. A general expression evaluator has no reason to think in any of those terms, and bending xprsn to fit would have meant loading it with JSONPath rules it has no business carrying.

So I stopped borrowing. The filter grammar moved into padvinder as its own recursive-descent parser written against the RFC, and the xprsn dependency came out with it. That is the real reason padvinder has zero runtime dependencies: the one piece it could have reused is the one piece the standard needed it to own.

Why it can't be turned into code

The safety is the same compile-to-closures technique I wrote up for xprsn, so here is only its shape. Every selector and every filter becomes a small function that already exists in the shipped source, and a parser wires them together. Query text never gets reassembled into JavaScript, which leaves nothing to inject into and nothing for a CSP to block.

The test suite keeps that honest by running under node --disallow-code-generation-from-strings, the flag that turns every eval-shaped call into a thrown error, exactly as a strict CSP does in the browser.

The prototype escapes are closed as well. __proto__, constructor, and prototype never match, in a path or a filter, so [[email protected]] finds nothing rather than climbing toward Function. Matching reads own properties only, and a query never writes back to the data you gave it. A test snapshots the input and compares it afterward to prove that last one.

Is it the right tool?

If your paths are all written by you, committed to your repo, and never near a strict CSP, you do not need any of this, and a bigger library with a wider feature set is a reasonable pick. padvinder is built for the narrower spot: the query might come from a user or a config row, the page runs under a CSP you will not weaken, and you would rather the results match the standard than one project's reading of it. The 456-case suite is what I lean on there. It is not my definition of correct, it is the RFC's.

Getting started

npm install padvinder

The README has the full selector table, the filter grammar, and the safety notes. Use find when you want an answer now and query when you will ask the same question a lot.

Closing thoughts

The satisfying part of this one was not the size or even the safety. It was having a compliance suite play judge. Most of my small libraries are correct by my own definition, which is a comfortable and slightly circular place to sit. Here, 456 cases I did not write got a vote, and "I think this is right" became "the standard says this is right." That judge is also what ended the borrowing: you do not approximate your way through 456 cases, so the evaluator I had hoped to reuse had to go. I would take the trade on more projects if the specs existed for them.

If you have been running JSONPath over input you did not write yourself, I would like to hear how you drew the line around it.