Skip to content

Add ALL and EXACTLY_ONE array filter operators for MongoDB and Postgres - #326

Merged
lrathod merged 14 commits into
mainfrom
ASP-3008/array-match-all-one-operators
Sep 7, 2026
Merged

Add ALL and EXACTLY_ONE array filter operators for MongoDB and Postgres#326
lrathod merged 14 commits into
mainfrom
ASP-3008/array-match-all-one-operators

Conversation

@lrathod

@lrathod lrathod commented Sep 7, 2026

Copy link
Copy Markdown
Collaborator

Note: This PR continues from #323 (same change, now from an upstream branch so CI runs normally). All review discussion and resolutions are in #323.

Summary

Adds two new ArrayOperator values for filtering on array-valued attributes:

  • ALL — array attribute must contain every value specified in the filter. Set-containment semantics: order and duplicates are irrelevant on both sides ([red, red] ALL [red] is true).
  • EXACTLY_ONE — array attribute must contain exactly one element, and that element must be one of the specified values. The cardinality check counts raw elements, not distinct values ([red, red] EXACTLY_ONE [red] is false).

Both work on top-level and nested array fields (e.g. props.colors, scope.environmentScope.environmentIds), on MongoDB and Postgres.

MongoDB

  • ALL{"$expr": {"$setIsSubset": [<values>, <guarded array>]}}
  • EXACTLY_ONE$and of $size == 1 and $in on $arrayElemAt [path, 0]
  • Non-array guard: <guarded array> is {"$cond": [{"$isArray": "$path"}, "$path", []]} — documents holding a missing/null/non-array scalar value simply do not match instead of erroring ($setIsSubset/$size reject non-array operands), matching the Postgres behaviour.

Postgres

  • Native array columns (flat collections): ALLcol @> ?; EXACTLY_ONEarray_length(col, 1) = 1 AND col && ?. No COALESCE — NULL arrays are excluded by WHERE semantics anyway, and the unwrapped column reference keeps the filter GIN-indexable (SARGable).
  • JSONB array paths (nested documents): ALL(CASE WHEN jsonb_typeof(path) = 'array' THEN path ELSE '[]'::jsonb END) @> ?::jsonb; EXACTLY_ONEjsonb_array_length(<guarded>) = 1 AND <guarded> <@ ?::jsonb (single bound param — with exactly one element, membership ≡ containment). The runtime jsonb_typeof guard is retained only for schemaless JSONB paths.
  • Compile-time element types: the array element type is resolved from the field expression's DataType (ArrayIdentifierExpression#getElementDataType()), falling back to inference from the filter values only when the field carries no type info.

Design note

Both parsers require the inner RelationalExpression to carry a constant value list; a non-constant RHS throws UnsupportedOperationException. This is intentional — ALL/EXACTLY_ONE are set-level operators, unlike ANY which supports arbitrary per-element sub-filters. Empty value lists are rejected at construction by ConstantExpression.

Future consideration (noted in the enum javadoc): an EXACTLY operator for set equality — array contains exactly the filter values, no more and no less.

Test coverage / EXACTLY_ONE semantics

MATCH_EXACTLY_ONE describes the stored array cardinality, not the length of the RHS:

  • Stored array must have exactly 1 element
  • That element must be in the RHS list (RHS can be 1 or many candidates)

Example with column tags:

  • Entity A: ["A", "B"]
  • Entity B: ["A"]
  • Entity C: ["B"]
  • Entity D: ["A", "B", "C"]
Query Result
MATCH_ALL ["A", "B"] A and D (D matches because extras are allowed; ALL is subset / containment)
MATCH_EXACTLY_ONE ["A"] B only
MATCH_EXACTLY_ONE ["A", "B"] B and C (not A, not D — size ≠ 1)

Same translation on both stores: Mongo $size=1 + $in; Postgres native array_length=1 AND && / JSONB length + <@.

Test plan

  • MongoArrayFilterParserTest — operator structure, $isArray guards, nested paths, non-constant RHS rejection, no double $expr wrapping
  • PostgresQueryParserTest — ALL/EXACTLY_ONE × JSONB/native array, nested JSONB paths, compile-time type precedence over value inference, UNSPECIFIED fallback, non-constant RHS rejection
  • DocStoreQueryV1Test (nested ArrayMatchAllOneOperatorTest) — integration tests on both datastores: nested JSONB array fields, native array columns (typed + untyped via PostgresArrayTypeProvider), JSONB array column on flat collections, 3-level nested paths with missing intermediate objects, non-array scalar values, duplicates ([red, red]), and order-independence — run in CI
  • :document-store:build (compile + unit tests + spotless) passes locally

Made with Cursor

Nested arrays

  • Nested arrays: operators apply to the outermost array only — an element that is itself an array is opaque and never matches a scalar RHS value; RHS lists are scalar-only. Pinned by IT testAllAndOneTreatNestedArrayElementsAsOpaque.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown

Test Results

  127 files  + 2    127 suites  +2   39s ⏱️ -4s
  889 tests +34    888 ✅ +34  1 💤 ±0  0 ❌ ±0 
1 259 runs  +67  1 258 ✅ +67  1 💤 ±0  0 ❌ ±0 

Results for commit cc785ec. ± Comparison against base commit 64e8bc1.

♻️ This comment has been updated with latest results.

lrathod and others added 13 commits September 7, 2026 14:20
Add two new ArrayOperator values for filtering on array-valued attributes:
- ALL: array attribute must contain every value specified in the filter
- ONE: array attribute must contain exactly one element, and that element
  must be one of the specified values

MongoDB: ALL uses $setIsSubset with an $ifNull guard; ONE combines
$size == 1 with $in on the first element via $arrayElemAt.

Postgres: native array columns use @> (ALL) and array_length + && (ONE);
JSONB array paths use jsonb_typeof-guarded @> containment and
jsonb_array_length respectively.

Both parsers require the inner filter to carry a constant value list;
non-constant RHS expressions throw UnsupportedOperationException since
these are set-level operators, not per-element predicates like ANY.

Co-authored-by: Cursor <cursoragent@cursor.com>
…mantics

- Resolve native array element type from the compile-time type info on the
  field expression (ArrayIdentifierExpression/IdentifierExpression DataType)
  instead of inferring from filter values; value inference is now only a
  fallback when no type info is present. The runtime jsonb_typeof guard is
  retained only for schemaless JSONB/nested array paths.
- Add unit + integration tests covering ALL/ONE on nested array fields
  (e.g. props.colors, scope.environmentScope.environmentIds) for both
  MongoDB and Postgres.
- Add integration test documenting that ALL is set-containment: duplicates
  in the document array ([red, red] ALL [red]) still match in both backends.

Co-authored-by: Cursor <cursoragent@cursor.com>
Consolidate the ALL/ONE integration tests into DocStoreQueryV1Test as a
nested ArrayMatchAllOneOperatorTest class, per review feedback:
- Nested JSONB array path (props.colors) covered on both MongoDB and
  Postgres via the shared document collection
- Native array columns (tags TEXT[], flags BOOLEAN[]) covered on the flat
  collection with both typed (compile-time DataType) and untyped
  (value-inference fallback) ArrayIdentifierExpression variants
- JSONB array column (props.colors) covered on the flat collection
- Duplicate-containing arrays ([red, red] ALL [red] -> true) covered via a
  dedicated collection, documenting set-containment semantics on real DBs

Co-authored-by: Cursor <cursoragent@cursor.com>
Negative coverage:
- ALL/ONE reject a non-constant RHS with UnsupportedOperationException
  in both Mongo and Postgres parsers
- Empty value lists are rejected at construction by ConstantExpression
- Integration: non-array JSONB values do not match and do not error on
  Postgres, exercising the jsonb_typeof guard

Semantics documentation via integration tests on both datastores:
- ALL is order-independent: [red, blue] ALL [blue, red] matches
- ALL/ONE on a three-level nested array field (props.metadata.colors),
  including docs with missing intermediate objects

Co-authored-by: Cursor <cursoragent@cursor.com>
…ONE semantics

- Native Postgres arrays: drop COALESCE - NULL arrays are excluded by WHERE
  semantics anyway, and the unwrapped column reference keeps the filter
  GIN-indexable (SARGable)
- JSONB ONE: replace the per-value OR chain with a single <@ containment
  against the full filter list (with exactly one element, membership and
  containment are equivalent) - one bound param instead of N
- Mongo: guard ALL/ONE with $cond/$isArray so documents holding a non-array
  scalar no longer error out ($setIsSubset/$size reject non-array operands),
  matching the Postgres jsonb_typeof behavior; subsumes $ifNull
- Document that ONE counts raw elements, not distinct values
  ([red, red] ONE [red] is false), with an integration test on both stores;
  non-array scalar test now runs on Mongo too

Co-authored-by: Cursor <cursoragent@cursor.com>
Aligns with the service-level MATCH_EXACTLY_ONE name and reads
unambiguously ("exactly one element, in the given set"). Also notes a
future EXACTLY (set-equality) operator in the enum javadoc.

Co-authored-by: Cursor <cursoragent@cursor.com>
[true,false] matches ids 5 and 8 (each a one-element flags array), not a parser bug; add [true]→1 to pin the singleton-true case.

Co-authored-by: Cursor <cursoragent@cursor.com>
…fy IN semantics

- Flat JSONB scalar no-match test (jsonb_typeof guard on the flat path)
- Flat native TEXT[] variants of order-independence and duplicate-element
  tests (WITH_TYPE/WITHOUT_TYPE), mirroring array_match_test.json
- Javadoc: inner IN is element membership; ALL = subset, EXACTLY_ONE =
  singleton whose element is in the RHS set
- Flat-collection postgres parser unit tests

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Document (mongo + postgres) and flat native-array variants asserting that
rows with missing/NULL or empty array fields never match, with positive
controls so the tests are not vacuous.

Co-authored-by: Cursor <cursoragent@cursor.com>
…_ONE length check

jsonb @>/<@ are total on non-array scalars (false/NULL, never error), so the
CASE guard was unnecessary for ALL and for the EXACTLY_ONE containment
conjunct. jsonb_array_length raises on non-arrays and Postgres does not
guarantee WHERE conjunct evaluation order, so the length check keeps its
CASE guard.

Co-authored-by: Cursor <cursoragent@cursor.com>
…top-level semantics

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@lrathod
lrathod force-pushed the ASP-3008/array-match-all-one-operators branch from c884914 to 16ec81e Compare September 7, 2026 08:51
@codecov

codecov Bot commented Sep 7, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.32110% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.16%. Comparing base (64e8bc1) to head (cc785ec).

Files with missing lines Patch % Lines
...1/vistors/PostgresFilterTypeExpressionVisitor.java 82.66% 7 Missing and 6 partials ⚠️
...ore/mongo/query/parser/MongoArrayFilterParser.java 90.62% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main     #326      +/-   ##
============================================
+ Coverage     81.06%   81.16%   +0.10%     
- Complexity     1617     1679      +62     
============================================
  Files           243      243              
  Lines          7656     7762     +106     
  Branches        755      769      +14     
============================================
+ Hits           6206     6300      +94     
- Misses          960      967       +7     
- Partials        490      495       +5     
Flag Coverage Δ
integration 81.16% <85.32%> (+0.10%) ⬆️
unit 58.50% <85.32%> (+1.34%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

… SHA)

Co-authored-by: Cursor <cursoragent@cursor.com>

@suddendust suddendust left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Review here: #323

@lrathod
lrathod merged commit db61d59 into main Sep 7, 2026
7 checks passed
@lrathod
lrathod deleted the ASP-3008/array-match-all-one-operators branch September 7, 2026 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants