FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Feature: JSON.FILTER by devBaunz · Pull Request #1465 · RedisJSON/RedisJSON · GitHub

  1. RedisJSON
  2. RedisJSON

Repository navigation

Feature: JSON.FILTER - #1465

Open
devBaunz wants to merge 1 commit into
RedisJSON:masterfrom
devBaunz:master
ConversationCommits (1)ChecksFiles changed
Open

Feature: JSON.FILTER#1465
devBaunz wants to merge 1 commit into
RedisJSON:masterfrom
devBaunz:master

Conversation

devBaunz commented Nov 25, 2025 •
edited
Loading

Copy link
Copy Markdown

Suggestion for an API - JSON.FILTER

Syntax

JSON.FILTER key1 [key2 ...] < path > < filter-expression >

Description

The command evaluates the filter-expression against each key's JSON document. If the filter returns non-empty results, the path is fetched from that document; otherwise, null is returned for that key position.

Complexity:

O(M*N) where M is the number of keys and N is the size of the document. Both filter-expression and path are evaluated against each document, with filter evaluation occurring first. When filter or path evaluate to multiple values: O(N1+N2+...+Nm) where m is the number of keys and Ni is the size of the i-th key

Basically a lightweight way of filtering and accessing the JSON tree at different points while adhering to JSONPath.
This allows user to have more complex operations without suffering a big performance impact, so that we can do things such as:

  1. Filter data at ROOT such as $.[?($.type == "metadata")] - to fetch any element that is a "metadata"
  2. Allow implementing a arr.filter((item) => predicate(item)) without having to first fetch the entire arr
  3. Allow detaching the filtering aspect of JSONPath from the actual data-fetch, so we can check something along the tree but retrieve a different element

Implementation details:

  • Follows JSON.MGET pattern for consistency
  • Validates filter syntax upfront using compile()
  • Uses calc_once() for filter evaluation
  • Supports both legacy (.) and modern ($) path formats
  • Returns array with results or null for non-matching keys

NOTE: This is an initial draft, it lacks tests and a more in-depth SPEC file, but this is mainly to discuss the concept, idea and reasoning of why this should/not be added

CLAassistant commented Nov 25, 2025 •
edited
Loading

Copy link
Copy Markdown


All committers have signed the CLA.

devBaunz mentioned this pull request Nov 25, 2025
Open
devBaunz marked this pull request as ready for review November 27, 2025 14:21

ephraimfeldblum commented Nov 30, 2025

Copy link
Copy Markdown
Contributor

thank you for the contribution.

there's a small typo in https://github.com/RedisJSON/RedisJSON/pull/1465/files#diff-fe8692f5492557c8fe9633a9784c4369d58ae070dd14d9e63696f57256a4efe9R1047 (missing a :).

can you please add tests verifying correctness of both the happy path as well as any error paths. eg something like

def testFilterCommandErrors(env):
    env.expect('JSON.FILTER', '{doc}:1', '$').raiseError()
    env.expect('JSON.FILTER', '{doc}:1', '$', '$[?(@.a>0)]').raiseError()
    env.cmd('JSON.SET', '{doc}:1', '$', '{"a":1}')
    env.expect('JSON.FILTER', '{doc}:1', '$', '$[?(@.a>').raiseError()
    env.expect('JSON.FILTER', '{doc}:1', '$..[', '$[?(@.a>0)]').raiseError()
def testFilterCommandBasic(env):
    env.cmd('JSON.SET', '{doc}:1', '$', '{"name":"Alice","age":30,"city":"NYC","active":true}')
    env.cmd('JSON.SET', '{doc}:2', '$', '{"name":"Bob","age":25,"city":"LA","active":false}')
    env.cmd('JSON.SET', '{doc}:3', '$', '{"name":"Charlie","age":35,"city":"NYC","active":true}')
    env.cmd('JSON.SET', '{doc}:4', '$', '{"name":"Diana","age":28,"city":"SF","active":true}')

    res = env.cmd('JSON.FILTER', '{doc}:1', '{doc}:2', '{doc}:3', '{doc}:4', '$', '$[?(@.active==true)]')
    env.assertEqual(len(res), 4)
    env.assertNotEqual(res[0], None)
    env.assertEqual(res[1], None)
    env.assertNotEqual(res[2], None)
    env.assertNotEqual(res[3], None)

    res = env.cmd('JSON.FILTER', '{doc}:1', '{doc}:2', '{doc}:3', '{doc}:4', '$.name', '$[?(@.age>28)]')
    env.assertEqual(len(res), 4)
    env.assertEqual(json.loads(res[0]), ["Alice"])
    env.assertEqual(res[1], None)
    env.assertEqual(json.loads(res[2]), ["Charlie"])
    env.assertEqual(res[3], None)

    res = env.cmd('JSON.FILTER', '{doc}:1', '{doc}:2', '{doc}:3', '{doc}:4', '$', '$[?(@.city=="NYC")]')
    env.assertEqual(len(res), 4)
    env.assertNotEqual(res[0], None)
    env.assertEqual(res[1], None)
    env.assertNotEqual(res[2], None)
    env.assertEqual(res[3], None)

    res = env.cmd('JSON.FILTER', '{doc}:1', '{doc}:missing', '{doc}:3', '$', '$[?(@.active==true)]')
    env.assertEqual(len(res), 3)
    env.assertNotEqual(res[0], None)
    env.assertEqual(res[1], None)
    env.assertNotEqual(res[2], None)

    env.cmd('SET', '{doc}:wrong_type', 'not a json')
    res = env.cmd('JSON.FILTER', '{doc}:1', '{doc}:wrong_type', '{doc}:3', '$', '$[?(@.active==true)]')
    env.assertEqual(len(res), 3)
    env.assertNotEqual(res[0], None)
    env.assertEqual(res[1], None)
    env.assertNotEqual(res[2], None)

    env.cmd('JSON.SET', '{doc}:nested1', '$', '{"user":{"name":"Eve","score":85},"status":"active"}')
    env.cmd('JSON.SET', '{doc}:nested2', '$', '{"user":{"name":"Frank","score":92},"status":"inactive"}')
    env.cmd('JSON.SET', '{doc}:nested3', '$', '{"user":{"name":"Grace","score":78},"status":"active"}')

    res = env.cmd('JSON.FILTER', '{doc}:nested1', '{doc}:nested2', '{doc}:nested3', '$.user.name', '$[?(@.user.score>80)]')
    env.assertEqual(len(res), 3)
    env.assertEqual(json.loads(res[0]), ["Eve"])
    env.assertEqual(json.loads(res[1]), ["Frank"])
    env.assertEqual(res[2], None)

    env.cmd('JSON.SET', '{doc}:arr1', '$', '{"items":[1,2,3],"count":3}')
    env.cmd('JSON.SET', '{doc}:arr2', '$', '{"items":[4,5],"count":2}')
    env.cmd('JSON.SET', '{doc}:arr3', '$', '{"items":[6,7,8,9],"count":4}')

    res = env.cmd('JSON.FILTER', '{doc}:arr1', '{doc}:arr2', '{doc}:arr3', '$.count', '$[?(@.count>2)]')
    env.assertEqual(len(res), 3)
    env.assertEqual(json.loads(res[0]), [3])
    env.assertEqual(res[1], None)
    env.assertEqual(json.loads(res[2]), [4])

    res = env.cmd('JSON.FILTER', '{doc}:1', '{doc}:2', '{doc}:3', '.name', '.[?(@.age>28)]')
    env.assertEqual(len(res), 3)
    env.assertEqual(json.loads(res[0]), "Alice")
    env.assertEqual(res[1], None)
    env.assertEqual(json.loads(res[2]), "Charlie")

    res = env.cmd('JSON.FILTER', '{doc}:1', '{doc}:2', '{doc}:3', '$', '$[?(@.age>100)]')
    env.assertEqual(len(res), 3)
    env.assertEqual(res[0], None)
    env.assertEqual(res[1], None)
    env.assertEqual(res[2], None)

    env.cmd('JSON.SET', '{doc}:multi1', '$', '{"a":1,"nested":{"a":2,"b":3}}')
    env.cmd('JSON.SET', '{doc}:multi2', '$', '{"a":4,"nested":{"a":5,"b":6}}')

    res = env.cmd('JSON.FILTER', '{doc}:multi1', '{doc}:multi2', '$..a', '$[?(@.a>0)]')
    env.assertEqual(len(res), 2)
    env.assertEqual(json.loads(res[0]), [1, 2])
    env.assertEqual(json.loads(res[1]), [4, 5])
def testFilterCommandComplex(env):
    env.cmd('JSON.SET', '{doc}:store1', '$', '{"store":{"book":[{"category":"reference","price":8.95},{"category":"fiction","price":12.99}]}}')
    env.cmd('JSON.SET', '{doc}:store2', '$', '{"store":{"book":[{"category":"fiction","price":8.99},{"category":"fiction","price":22.99}]}}')
    env.cmd('JSON.SET', '{doc}:store3', '$', '{"store":{"book":[{"category":"reference","price":15.00}]}}')

    res = env.cmd('JSON.FILTER', '{doc}:store1', '{doc}:store2', '{doc}:store3', '$.store.book[*].price', '$.store.book[?(@.category=="fiction")]')
    env.assertEqual(len(res), 3)
    env.assertNotEqual(res[0], None)
    env.assertNotEqual(res[1], None)
    env.assertEqual(res[2], None)

    env.cmd('JSON.SET', '{doc}:deep1', '$', '{"level1":{"level2":{"value":10}}}')
    env.cmd('JSON.SET', '{doc}:deep2', '$', '{"level1":{"level2":{"value":5}}}')

    res = env.cmd('JSON.FILTER', '{doc}:deep1', '{doc}:deep2', '$..value', '$..level2[?(@.value>7)]')
    env.assertEqual(len(res), 2)
    env.assertEqual(json.loads(res[0]), [10])
    env.assertEqual(res[1], None)

devBaunz commented Nov 30, 2025

Copy link
Copy Markdown
Author

@ephraimfeldblum Sure thing, sorry about it - was actually copy-pasting from my local fork (which pre-dates the macro usage) so didnt check it in-depth.

I'll get to it today - will also differentiate JSON.FILTER (new impl) and JSON.MFILTER (current impl)
as per discussion at #1466

This branch has not been deployed

No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

ephraimfeldblum Awaiting requested review from ephraimfeldblum

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

Back | FazBrowse Home | New Git URL