Skip to content

feat(pg-functions): add justify_hours, justify_days and justify_interval - #435

Merged
sunng87 merged 1 commit into
datafusion-contrib:masterfrom
Ishaan400:pg-functions-justify
Oct 14, 2026
Merged

sunng87 merged 1 commit into
datafusion-contrib:masterfrom
Ishaan400:pg-functions-justify

Conversation

@Ishaan400

Copy link
Copy Markdown
Contributor

Closes #434

Adds the PostgreSQL interval justification functions to the datetime category of datafusion-pg-functions:

Function Behavior
justify_hours(interval) moves whole 24-hour periods from the time field into days
justify_days(interval) moves whole 30-day periods from days into months
justify_interval(interval) justifies both, with PostgreSQL's sign adjustments

Implementation

  • Each function is a direct port of interval_justify_{hours,days,interval} from PostgreSQL's src/backend/utils/adt/timestamp.c:

    • 30-day months and 24-hour days;
    • truncating division;
    • the same overflow checks;
    • the same sign borrows.

    Behavior for finite values is the same in PostgreSQL 16 through 18.

  • justify_interval follows PostgreSQL's own steps rather than calling justify_days(justify_hours(x)). PostgreSQL moves days into months early when days and time share a sign. It also borrows a month when days are zero and time has the opposite sign, so 1 mon -00:00:00.000001 becomes 29 days 23:59:59.999999.

  • Code layout:

    • one UDF per file under src/datetime/;
    • a small private src/datetime/interval.rs holding the shared signature, Arrow array iteration and constants;
    • registration in datetime::register behind the datetime feature.
  • functions.md marks the three functions 🔧 and updates the Date/Time and TOTAL counts.

Design choices (agreed in #434)

  • Input types: Interval(MonthDayNano), Interval(DayTime) and Interval(YearMonth) are accepted and converted losslessly. The result is always Interval(MonthDayNano), matching PostgreSQL's single interval type.
  • Overflow: month or day overflow raises interval out of range, as PostgreSQL does.
  • Tests: datetime is added to the sqllogictest target's required-features, along with a new datetime.slt. The PostgreSQL 16.15 result is noted above each case.

Sub-microsecond precision

  • PostgreSQL stores intervals in microseconds and rounds finer input when parsing. Arrow MonthDayNano keeps nanoseconds, and DataFusion can produce sub-µs values (e.g. INTERVAL '1 day -1 nanosecond').
  • These functions keep the nanoseconds and justify them exactly, so the result always equals the input.
  • The difference is visible only for sub-µs input. justify_hours(INTERVAL '1 day -1 nanosecond') returns 23:59:59.999999999 (clients see 23:59:59.999999 because arrow-pg truncates to µs), where PostgreSQL returns 1 day.
  • Reasons for this choice:
    • PostgreSQL has no result for sub-µs input to match.
    • DataFusion's own interval arithmetic keeps nanoseconds.
    • Rounding here would set a precision rule for the whole datetime category.
  • The behavior is documented in the rustdoc and pinned by two datetime.slt cases.

Tests

Run locally on this branch (based on master @ 4776015, DataFusion 55.2.0):

Command Result
cargo fmt -- --check pass
cargo clippy --features datafusion-pg-functions/full -- -D warnings pass
cargo clippy --all-features -- -D warnings pass
cargo clippy -p datafusion-pg-functions --features full --all-targets -- -D warnings pass
cargo test pass
cargo test --features datafusion-pg-functions/full pass (pg-functions: 71 unit tests, sqllogictest including datetime.slt)
cargo test --all-features pass

I also compared months, days and microseconds field by field against a PostgreSQL 16.15 server. Results were identical for:

  • 50 chosen inputs per function, including i32 month/day limits and the justify_days / justify_interval overflow errors;
  • 2,000 random mixed-sign inputs per function.

The integration suites that need nix and podman were not run locally.

Known limitations

  • Sub-microsecond time is kept rather than rounded (see above).
  • Interval(DayTime) and Interval(YearMonth) can't be produced from SQL literals in DataFusion. Those inputs are covered by unit tests.
  • PostgreSQL 17+ returns ±infinity intervals unchanged. Arrow has no infinite interval, so there is nothing equivalent to support.

Out of scope

  • Enabling datetime in datafusion-postgres's default features. The server currently enables only math, so these functions aren't yet reachable through it.
  • Existing server behavior unrelated to these functions:
    • NULL interval values are encoded as 00:00:00 by arrow-pg.
    • Execution errors are reported with SQLSTATE XX000.

Implement the PostgreSQL interval justification functions in the
datetime category, ported from interval_justify_{hours,days,interval} in
PostgreSQL's timestamp.c.

- Accept Interval(MonthDayNano), Interval(DayTime) and Interval(YearMonth)
  and always return Interval(MonthDayNano).
- Raise "interval out of range" on month/day overflow, as PostgreSQL does.
- Keep Arrow's nanosecond precision and justify it exactly.
- Register the UDFs under the `datetime` feature, mark them implemented in
  functions.md, and add datetime.slt (`datetime` is now a required feature
  of the sqllogictest target).

Refs datafusion-contrib#434
@sunng87
sunng87 merged commit 05f5458 into datafusion-contrib:master Oct 14, 2026
14 checks passed
@sunng87

sunng87 commented Oct 14, 2026

Copy link
Copy Markdown
Member

Thank you @Ishaan400

@Ishaan400

Copy link
Copy Markdown
Contributor Author

Thanks for merging!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pg-functions: implement justify_days / justify_hours / justify_interval?

2 participants