feat(spec)!: $empty joins FILTER_OPERATORS, and is_empty / is_not_empty lower to it (#20446)

.changeset/20446-empty-joins-filter-operators.md

@@ -0,0 +1,25 @@
+---
+'@objectstack/spec': minor
+'@objectstack/driver-memory': minor
+'@objectstack/service-analytics': patch
+---
+
+feat(spec)!: `$empty` joins `FILTER_OPERATORS`, and the view operators `is_empty` / `is_not_empty` lower to it (#20446)
+
+A stored 「is empty」 / 「is not empty」 — `['field', 'is_empty', …]`, `isempty`, `is_not_empty`, `isnotempty`, in a view rule, a sharing rule or any filter array — now lowers to `{ field: { $empty: true | false } }` instead of `$null`. `$empty` is answered by the field's DECLARED type: a text-like field is empty when it is null or `''`, a multi-value field (multiselect, checkboxes, tags, or a select / radio / lookup / user / file / image with `multiple: true`) when it is null or `[]`, and every other type only when it is null. So an 「is empty」 rule on a text field now also finds `''`, and on a multi-value field also finds `[]`, which the `$null` lowering missed. `is_not_empty` is its exact complement. `$empty` is in `FILTER_OPERATORS` (and `ALL_OPERATORS`) now, and `canonicalAstOperator` folds the empty pair onto `is_empty` / `is_not_empty` rather than onto `is_null` / `is_not_null`. On `@objectstack/driver-memory`, a QueryAST comparison node (`{ type: 'comparison', operator: 'is_empty' }`) is answered by the same declared-type arm.
+
+**BREAKING**: two things accepted before are refused now, each loudly and with its fix.
+
+- **A `{ $empty: … }` object written as a field value** (a `where` pasted into an insert or update payload) is refused with `VALIDATION_FAILED` (`invalid_type`, "$empty is a filter operator, not a value"). Before, a text-like field stored it as data.
+  FROM `update('task', { title: { $empty: true } })` → TO write the value itself (`{ title: '' }`, `{ title: null }`); a filter belongs in `where`.
+- **`is_empty` / `is_not_empty` where no face holds the column's declared type** is refused with `INVALID_FILTER` / 400 (`READ_SCOPE_COMPILE_FAILED` / 500 on an analytics read scope). The `$null` lowering answered these. The compositions:
+  - the built-in `id`, which no object declares. FROM `['id', 'is_empty', true]` → TO `['id', 'is_null', true]` / `is_not_null`;
+  - a federated (external) object on a driver that does not implement `registerExternalObject` (driver-memory, driver-mongodb). The boot already reports such an object as NOT bound to its remote table, naming it, and its reads answered from a table named after the object. FROM `is_empty` on such an object → TO bind it on a driver that implements federation (driver-sql and its heirs, driver-turso);
+  - an `AnalyticsService` constructed without `sourceFieldMeta`. FROM such a host → TO pass `sourceFieldMeta` (the package README shows it), or filter with `is_null` / `is_not_null`;
+  - a multi-value column on a SQL dialect `driver-sql` does not model (a knex client other than SQLite, PostgreSQL or MySQL). FROM `['tags', 'is_empty', true]` there → TO `['tags', 'is_null', true]` / `is_not_null`.
+
+Stored sharing rules and views that use 「is empty」 are not rewritten; they are re-read under the new meaning. Production rules that use 「is empty」 on a text or multi-value field were not measured; each finds more rows (the `''` / `[]` ones) from this release.
+
+Clause-②: yes (narrowing)
+
+<!-- adr-0087: registered filter-is-empty-lowers-to-empty-operator -->

content/docs/references/data/filter.mdx

@@ -120,7 +120,7 @@ const result = ComparisonOperatorSchema.parse(data);
 | **$ilike** | `string` | optional | Whole-string pattern match like $like — "%" / "_" wildcards bound by the caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) and ONLY ASCII case: "café" does NOT match "CAFÉ", the same Q1 = A boundary $icontains declares, because SQLite's fold is ASCII-only and three of the five backends are SQLite underneath. Staged with $like — see FILTER_OPERATORS. |
 | **$null** | `boolean` | optional | Is-null check. `true` matches rows where the field is null, `false` matches rows where it is not null. Lowered to `IS NULL` (true) / `IS NOT NULL` (false) on the SQL family and to `{ field: null }` (true) / `{ $ne: null }` (false) on MongoDB. |
 | **$exists** | `boolean` | optional | Has-a-value check — the exact inverse of `$null`. `true` matches rows where the field holds a value (`!= null`), `false` matches rows where it holds none. Portable across every backend the platform ships: lowered to `IS NOT NULL` (true) / `IS NULL` (false) on the SQL family and to `{ $ne: null }` (true) / `{ $eq: null }` (false) on MongoDB. |
-| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. STAGED: declared ahead of its backends and absent from FILTER_OPERATORS. Until each face has its arm, the query executors refuse it and the write-side check matcher matches no record; the view operators is_empty / is_not_empty still lower to $null. |
+| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. A face that answers by the declared type refuses the operator on a column whose declaration it does not hold (the built-in id, for one) rather than guess a row; use $null there for "has no value". The view operators is_empty / is_not_empty lower to this operator. |
 
 ### Nested Shape: `FieldOperators.$gt`
 
@@ -247,7 +247,7 @@ Type: `[FilterArray](#filterarray)[]`
 | **$ilike** | `string` | optional | Whole-string pattern match like $like — "%" / "_" wildcards bound by the caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) and ONLY ASCII case: "café" does NOT match "CAFÉ", the same Q1 = A boundary $icontains declares, because SQLite's fold is ASCII-only and three of the five backends are SQLite underneath. Staged with $like — see FILTER_OPERATORS. |
 | **$null** | `boolean` | optional | Is-null check. `true` matches rows where the field is null, `false` matches rows where it is not null. Lowered to `IS NULL` (true) / `IS NOT NULL` (false) on the SQL family and to `{ field: null }` (true) / `{ $ne: null }` (false) on MongoDB. |
 | **$exists** | `boolean` | optional | Has-a-value check — the exact inverse of `$null`. `true` matches rows where the field holds a value (`!= null`), `false` matches rows where it holds none. Portable across every backend the platform ships: lowered to `IS NOT NULL` (true) / `IS NULL` (false) on the SQL family and to `{ $ne: null }` (true) / `{ $eq: null }` (false) on MongoDB. |
-| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. STAGED: declared ahead of its backends and absent from FILTER_OPERATORS. Until each face has its arm, the query executors refuse it and the write-side check matcher matches no record; the view operators is_empty / is_not_empty still lower to $null. |
+| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. A face that answers by the declared type refuses the operator on a column whose declaration it does not hold (the built-in id, for one) rather than guess a row; use $null there for "has no value". The view operators is_empty / is_not_empty lower to this operator. |
 
 ### Nested Shape: `NormalizedFilter.$or[number][string]`
 
@@ -271,7 +271,7 @@ Type: `[FilterArray](#filterarray)[]`
 | **$ilike** | `string` | optional | Whole-string pattern match like $like — "%" / "_" wildcards bound by the caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) and ONLY ASCII case: "café" does NOT match "CAFÉ", the same Q1 = A boundary $icontains declares, because SQLite's fold is ASCII-only and three of the five backends are SQLite underneath. Staged with $like — see FILTER_OPERATORS. |
 | **$null** | `boolean` | optional | Is-null check. `true` matches rows where the field is null, `false` matches rows where it is not null. Lowered to `IS NULL` (true) / `IS NOT NULL` (false) on the SQL family and to `{ field: null }` (true) / `{ $ne: null }` (false) on MongoDB. |
 | **$exists** | `boolean` | optional | Has-a-value check — the exact inverse of `$null`. `true` matches rows where the field holds a value (`!= null`), `false` matches rows where it holds none. Portable across every backend the platform ships: lowered to `IS NOT NULL` (true) / `IS NULL` (false) on the SQL family and to `{ $ne: null }` (true) / `{ $eq: null }` (false) on MongoDB. |
-| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. STAGED: declared ahead of its backends and absent from FILTER_OPERATORS. Until each face has its arm, the query executors refuse it and the write-side check matcher matches no record; the view operators is_empty / is_not_empty still lower to $null. |
+| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. A face that answers by the declared type refuses the operator on a column whose declaration it does not hold (the built-in id, for one) rather than guess a row; use $null there for "has no value". The view operators is_empty / is_not_empty lower to this operator. |
 
 ### Nested Shape: `NormalizedFilter.$not[string]`
 
@@ -295,7 +295,7 @@ Type: `[FilterArray](#filterarray)[]`
 | **$ilike** | `string` | optional | Whole-string pattern match like $like — "%" / "_" wildcards bound by the caller, backslash escapes — but ignoring ASCII case (A-Z against a-z) and ONLY ASCII case: "café" does NOT match "CAFÉ", the same Q1 = A boundary $icontains declares, because SQLite's fold is ASCII-only and three of the five backends are SQLite underneath. Staged with $like — see FILTER_OPERATORS. |
 | **$null** | `boolean` | optional | Is-null check. `true` matches rows where the field is null, `false` matches rows where it is not null. Lowered to `IS NULL` (true) / `IS NOT NULL` (false) on the SQL family and to `{ field: null }` (true) / `{ $ne: null }` (false) on MongoDB. |
 | **$exists** | `boolean` | optional | Has-a-value check — the exact inverse of `$null`. `true` matches rows where the field holds a value (`!= null`), `false` matches rows where it holds none. Portable across every backend the platform ships: lowered to `IS NOT NULL` (true) / `IS NULL` (false) on the SQL family and to `{ $ne: null }` (true) / `{ $eq: null }` (false) on MongoDB. |
-| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. STAGED: declared ahead of its backends and absent from FILTER_OPERATORS. Until each face has its arm, the query executors refuse it and the write-side check matcher matches no record; the view operators is_empty / is_not_empty still lower to $null. |
+| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. A face that answers by the declared type refuses the operator on a column whose declaration it does not hold (the built-in id, for one) rather than guess a row; use $null there for "has no value". The view operators is_empty / is_not_empty lower to this operator. |
 
 
 ---
@@ -342,7 +342,7 @@ Type: `[FilterArray](#filterarray)[]`
 | :--- | :--- | :--- | :--- |
 | **$null** | `boolean` | optional | Is-null check. `true` matches rows where the field is null, `false` matches rows where it is not null. Lowered to `IS NULL` (true) / `IS NOT NULL` (false) on the SQL family and to `{ field: null }` (true) / `{ $ne: null }` (false) on MongoDB. |
 | **$exists** | `boolean` | optional | Has-a-value check — the exact inverse of `$null`. `true` matches rows where the field holds a value (`!= null`), `false` matches rows where it holds none. Portable across every backend the platform ships: lowered to `IS NOT NULL` (true) / `IS NULL` (false) on the SQL family and to `{ $ne: null }` (true) / `{ $eq: null }` (false) on MongoDB. |
-| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. STAGED: declared ahead of its backends and absent from FILTER_OPERATORS. Until each face has its arm, the query executors refuse it and the write-side check matcher matches no record; the view operators is_empty / is_not_empty still lower to $null. |
+| **$empty** | `boolean` | optional | Is-empty check by the field's DECLARED type. `true` matches rows whose field is empty, `false` is its exact complement. What counts as empty: text-like types (text, textarea, email, url, phone, password, secret, markdown, html, richtext, code, color, signature, qrcode) = null or '' (the empty string); multi-value types (multiselect, checkboxes, tags, and select, radio, lookup, user, file or image with multiple: true) = null or [] (the empty list); every other type = null only. A face that holds no field declaration judges by the value: null, '' and [] are empty. A face that answers by the declared type refuses the operator on a column whose declaration it does not hold (the built-in id, for one) rather than guess a row; use $null there for "has no value". The view operators is_empty / is_not_empty lower to this operator. |
 
 
 ---

packages/drivers/driver-memory/src/filter-refusal.ts

@@ -364,13 +364,12 @@ export const SUPPORTED_FIELD_OPERATORS: ReadonlySet<string> = new Set<string>([
   ...FILTER_OPERATORS,
   '$like',
   '$ilike',
-  // [#20444] The staged emptiness flag, admitted BY HAND for the reason the
-  // `$like` paragraph above gives, and under its ordering rule: both arms land
-  // with this entry — the reference matcher judges the stored value
-  // (`isEmptyFilterValue`, the spec's reading for a face holding no field
-  // declaration) and the live query path the field's DECLARED row
-  // (`expandEmptyOperator`, from the declaration `syncSchema` recorded).
-  '$empty',
+  // [#20444] `$empty` was admitted here BY HAND, with both its arms (the
+  // reference matcher by value through `isEmptyFilterValue`, the live query
+  // path by the field's DECLARED row through `expandEmptyOperator`), while it
+  // was staged out of `FILTER_OPERATORS`. [#20446] It arrives by DERIVATION
+  // now, after `$exists` in the spec's order, so the hand entry is gone — the
+  // `$icontains` direction above, with the arms already in place.
 ]);
 
 /** The vocabulary as it appears in a refusal message, in declaration order. */

packages/drivers/driver-memory/src/memory-20444-empty-operator.test.ts

@@ -1,7 +1,7 @@
 // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
 
 /**
- * [#20444] The staged `$empty` operator on this package's three filter faces.
+ * [#20444] The `$empty` operator on this package's three filter faces.
  *
  * - **The live query path** (`InMemoryDriver.find` → mingo) holds the field
  *   declarations `syncSchema` recorded, so it answers by the field's DECLARED