Skip to content

Scalar expressions: ROUND(x, 1) returns an integer, function results fail to compile under an operator or in GREATEST, a simple CASE matches NULL or fails over an expression, and ORDER BY GREATEST(…) fails #416

Open

Description

@fupelaqu

These defects are in how a scalar expression is rendered to Painless. They sit in the same neighbourhood as #380, which #412 fixes: a value, a type or a null guard ends up wrong relative to the operator or clause that consumes it. Items 1 and 4 return HTTP 200 with wrong values; the others fail loudly or are refused.

Measured on main (f63222d) and on #412's head (0687815), executed on Elasticsearch 8.18.3 through the gateway. PostgreSQL 16.14 ran on the same rows. Every item behaves the same on both builds unless stated.

Fixture

CREATE TABLE t (id KEYWORD, g KEYWORD, s KEYWORD, s2 KEYWORD, n INT, x DOUBLE, y DOUBLE, d DATE, d2 DATE, ts TIMESTAMP, b BOOLEAN, PRIMARY KEY (id)). "—" means the field is absent from the document. In PostgreSQL an absent field is NULL, and ROUND(x, 1) is written ROUND(x::numeric, 1) because PostgreSQL has no round(double precision, integer).

id g s n x y d d2 ts
r1 g1 1 3 0.25 1.55 2024-01-31 2024-02-01 2024-01-31T10:30:00Z
r2 g1 5 5 1.5 1.45 2024-03-01 2024-02-28 2024-03-01T06:00:00Z
r3 g2 12 -2 — — 2023-12-25 — 2023-12-25T23:59:59Z
r4 g3 — — 2.0 2.25 — 2024-01-31 —
r5 g3 3 10 -1.25 -1.55 2024-02-29 2024-02-29 2024-02-29T00:00:00Z

Values are listed in id order r1 … r5.

1. ROUND with a scale returns an integer, even in a plain SELECT

statement main = #412 PostgreSQL 16
SELECT id, ROUND(x, 1) AS v FROM t 0, 2, NULL, 2, -1 0.3, 1.5, NULL, 2.0, -1.3
SELECT id, ROUND(x, 2) AS v FROM t 0, 2, NULL, 2, -1 0.25, 1.50, NULL, 2.00, -1.25
SELECT id, ROUND(y, 1) AS v FROM t 2, 1, NULL, 2, -2 1.6, 1.5, NULL, 2.3, -1.6
SELECT id FROM t WHERE ROUND(x, 1) = 1.5 no row r2
SELECT id FROM t WHERE ROUND(y, 1) = 1.6 no row r1
SELECT id FROM t WHERE ROUND(x, 1) > 0.25 r2, r4 r1, r2, r4
SELECT id, ROUND(x, 1) + n AS v FROM t 3, 7, NULL, NULL, 9 3.3, 6.5, NULL, NULL, 8.7
SELECT id, NULLIF(ROUND(x, 1), 1.5) AS v FROM t 0, 2, NULL, 2, -1 0.3, NULL, NULL, 2.0, -1.3

emits

def param2 = Math.pow(10, 1); … Long.valueOf(Math.round((param1 * param2) / param2))

The division by 10^scale happens inside Math.round, so the value is rounded to an integer. Measured in SELECT, WHERE, arithmetic and NULLIF; computed columns and per-group calculations were not measured.

  • Expected: PostgreSQL's values.
  • Fix direction: Math.round(x * 10^s) / 10^s as a double. Choose the tie rule for negative halves too: PostgreSQL's numeric gives -1.25 → -1.3, while Java's Math.round would give -1.2.

2. A function result fails to compile under an operator or as a GREATEST / LEAST argument, on every document

statement main = #412 PostgreSQL 16
SELECT id, ROUND(x, 1) - 1 AS v FROM t (also + 1, * 2) 400, Cannot cast from [java.lang.Long] to [int] -0.7, 0.5, NULL, 1.0, -2.3
SELECT id, ABS(x) + 1 AS v FROM t (x is DOUBLE) 400, all shards failed 1.25, 2.5, NULL, 3, 2.25
SELECT id, GREATEST(ABS(n), 1) AS v FROM t 400, compile error Cannot cast null to a primitive type [double] 3, 5, 2, 1, 10
SELECT id, GREATEST(CASE WHEN n > 0 THEN n ELSE 0 END, x) AS v FROM t 400, compile error 3, 5, 0, 2, 10
SELECT id, GREATEST(n, NULL) AS v FROM t 400, compile error 3, 5, -2, NULL, 10

GREATEST(SQRT(x), 1) fails with the same compile error. Adding WHERE <column> IS NOT NULL changes nothing, so these are not a missing-value problem. Since #412, ABS(n) + 1 over an INT runs (4, 6, 3, NULL, 11).

Two mechanisms: the function's boxed result is combined with an int literal (Cannot cast from [java.lang.Long] to [int]), and the function argument of GREATEST / LEAST reports nullable = false, so the reducer applies Math.max to a value that can be null.

  • Expected: PostgreSQL's values.
  • Fix direction: guard the NULL around the whole arithmetic, give the function's result the primitive (or def) type the operator needs, and let a function argument of GREATEST / LEAST report that it can be NULL. A bare NULL argument is skipped, as any NULL argument is.

3. A simple CASE over an expression fails on every document

statement main = #412 PostgreSQL 16
SELECT id, CASE YEAR(d) WHEN 2024 THEN 1 END AS v FROM t 400, runtime dynamic method [java.lang.Integer, toInstant/0] not found 1, 1, NULL, NULL, 1
SELECT id, CASE CAST(n AS VARCHAR) WHEN '3' THEN 1 ELSE 0 END AS v FROM t 400, runtime cannot explicitly cast def [java.lang.String] to long 1, 0, 0, 0, 0
SELECT id, CASE WHEN YEAR(d) = 2024 THEN 1 END AS v FROM t (searched form) 1, 1, NULL, NULL, 1 same

The operand is handled in its column's type rather than the expression's: YEAR(d) is an integer, but param1.toInstant() is called on it as on the date d.

  • Fix direction: take the operand's type from the expression, not from its column.

4. A simple CASE matches a NULL operand

statement main = #412 PostgreSQL 16
SELECT id, CASE n WHEN n THEN 1 ELSE 0 END AS v FROM t 1, 1, 1, 1, 1 1, 1, 1, 0, 1
SELECT id, CASE x WHEN x THEN 1 ELSE 0 END AS v FROM t 1, 1, 1, 1, 1 1, 1, 0, 1, 1
SELECT id, CASE WHEN n = n THEN 1 ELSE 0 END AS v FROM t (searched form) 1, 1, 1, 0, 1 same

The script emits param2 == param2 ? 1 : 0, and in Painless null == null is true. In SQL NULL = NULL is unknown, so the ELSE branch applies. (Since #412 the values are the integers 1 / 0; main answered 1.0 over a DOUBLE operand.)

  • Fix direction: guard the operand and each WHEN value against NULL, as the searched form does.

5. A CASE or a function over an aggregate is refused per group

statement main = #412 PostgreSQL 16
SELECT g, CASE WHEN MAX(n) > 1 THEN 1 ELSE 0 END AS v FROM t GROUP BY g refused at parse: Non-aggregated fields CASE WHEN MAX(n) > 1 THEN 1 ELSE 0 END AS v cannot be selected when GROUP BY is present g1 1, g2 0, g3 1
SELECT g, ROUND(AVG(x), 1) AS v FROM t GROUP BY g the same refusal g1 0.9, g2 NULL, g3 0.4
SELECT g FROM t GROUP BY g HAVING CASE WHEN MAX(n) > 1 THEN 1 ELSE 0 END = 1 refused: HAVING cannot be applied to CASE WHEN … it cannot be rendered without a document … g1, g3

The refusal is loud. A workaround runs: MAX(CASE WHEN n > 1 THEN 1 ELSE 0 END) → 1.0, 0.0, 1.0. HAVING ABS(COUNT(*)) > 1 and the rest of the numeric function family are refused on purpose (dql_statements.md: the boxing conversion compiles in some positions of a group-filter script and not others). An integral rendering of ABS in the bucket_selector compiled and kept the right groups on Elasticsearch 8.18.3, measured on a variant of #412 that was not kept.

  • Expected: computed per group, as arithmetic over aggregates already is.
  • Fix direction: render a CASE or scalar function whose operands are aggregates as a per-group calculation (bucket_script / bucket_selector), with a rendering that compiles in every position.

6. DATE_TRUNC to a sub-day unit fails when compared with a DATE

statement main = #412 PostgreSQL 16
SELECT id FROM t WHERE DATE_TRUNC(d, HOUR) > CAST('2024-01-31' AS DATE) 400, runtime dynamic method [java.time.LocalDate, truncatedTo/1] not found r2, r5
SELECT id FROM t WHERE DATE_TRUNC(d, HOUR) = CAST('2024-01-31' AS DATE) same error r1
SELECT id FROM t WHERE DATE_TRUNC(d, MINUTE) < CAST('2024-02-01' AS DATE) same error r1, r3
SELECT id FROM t WHERE DATE_TRUNC(ts, HOUR) > CAST('2024-01-31' AS DATE) same error r1, r2, r5
SELECT id, DATE_TRUNC(d, HOUR) AS v FROM t midnights, as PostgreSQL timestamps at midnight
SELECT id FROM t WHERE DATE_TRUNC(d, HOUR) = d r1, r2, r3, r5 r1, r2, r3, r5

In the comparison, the operand is narrowed to a LocalDate before it is truncated: ….toLocalDate().truncatedTo(ChronoUnit.HOURS).

  • Expected: compare as timestamps. PostgreSQL promotes the DATE.
  • Fix direction: truncate first, or promote the DATE operand.

7. A BIGINT literal past the INT range fails to compile, except in GREATEST / LEAST

statement #412 PostgreSQL 16
SELECT id, n + 3000000000 AS v FROM t 400, compile error Invalid int constant [3000000000]. If you want a long constant then change it to [3000000000L]. 3000000003, 3000000005, 2999999998, NULL, 3000000010
SELECT id, COALESCE(n, 3000000000) AS v FROM t same error 3, 5, -2, 3000000000, 10
SELECT id, CASE WHEN n > 0 THEN 3000000000 ELSE 0 END AS v FROM t same error 3000000000, 3000000000, 0, 0, 3000000000
SELECT id FROM t WHERE n + 0 > 3000000000 400, compile error no row

The literal is rendered without the L suffix, and Painless reads 3000000000 as an int constant. main renders the same scripts (read from the request). #412 adds the suffix inside GREATEST / LEAST only: GREATEST(n, 3000000000) answers 3000000000 (a Long) on every row there.

  • Expected: PostgreSQL's values.
  • Fix direction: render a whole-number literal past the INT range with the L suffix wherever a script is emitted.

8. ORDER BY GREATEST(…) sorts by the alias as a field and fails

statement main = #412 PostgreSQL 16
SELECT id, GREATEST(n, x) AS v FROM t ORDER BY GREATEST(n, x) DESC 400, No mapping found for [v] in order to sort on r5, r2, r1, r4, r3 (10, 5, 3, 2, -2)
SELECT id, GREATEST(n, 2) AS v FROM t ORDER BY GREATEST(n, 2) DESC same error r5, r2, r1, r3, r4 (10, 5, 3, 2, 2)

The ORDER BY expression is matched to the SELECT item, then sorted by its alias as if v were a field of the index.

  • Expected: sorted by the computed value.
  • Fix direction: sort by a script over the expression, or refuse by name if a script sort cannot express it (see the ORDER BY <arithmetic> limitation in operators.md).

Evidence level

RUN on Elasticsearch 8.18.3, both builds, every row. PostgreSQL 16.14 answers measured. Scripts and types were read from the rendered request on each build. Item 5's integral ABS variant was measured on a development build, not on either commit.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions