Skip to content

feat(pg-functions): add age(timestamp, timestamp) and age(timestamp) - #437

Merged
sunng87 merged 3 commits into
datafusion-contrib:masterfrom
Ishaan400:pg-functions-age
Oct 16, 2026
Merged

sunng87 merged 3 commits into
datafusion-contrib:masterfrom
Ishaan400:pg-functions-age

Conversation

@Ishaan400

Copy link
Copy Markdown
Contributor

Closes #436

Adds PostgreSQL's age() to the datetime category, ported from timestamp_age / timestamptz_age in PostgreSQL's timestamp.c. Today age(...) fails with Invalid function 'age'.

  • The timestamps are subtracted field by field, and negative fields borrow from the next larger one. A day borrows the length of the earlier timestamp's month, so age('2024-03-01', '2024-01-31') is 1 mon 1 day and the reverse is -1 mons -1 days.
  • Accepts timestamps of any unit, with or without a time zone, dates and string literals (age('2024-03-01', '2024-01-31') works, as in PostgreSQL). Always returns Interval(MonthDayNano); other argument types are a planning error.
  • age(x) is rewritten to age(current_date, x) during planning, which is how PostgreSQL defines it.
  • Time zones: PostgreSQL takes both arguments' fields in the session TimeZone. Here, when an argument carries an Arrow time zone, both arguments are cast to the first such zone. A timestamp in another zone keeps its instant; a plain timestamp, date or literal is read as local time in that zone. Which argument is earlier is decided by the instants, as PostgreSQL does. This matches PostgreSQL when the session is in that zone: for example 1 day across a DST change in America/New_York, where UTC gives 23:00:00.
  • Nanoseconds are kept, as in the justify_* functions.
  • chrono (already a workspace dependency) becomes an optional dependency enabled by the datetime feature. functions.md marks age implemented under Date/Time; the xid form under System Information is unchanged.

Tests

  • 13 unit tests in datetime/age.rs, with expected values recorded from PostgreSQL 16.15. They cover month-end and leap-day borrows, negative results, sub-second borrows, DST, mixed time zones, dates and mixed units, NULL, empty input, out-of-range timestamps, and the SQL forms (two arguments, string literals, one argument, rejected types).
  • New age section in datetime.slt, with each PostgreSQL result noted.
  • cargo test and cargo test --features datafusion-pg-functions/full: pass. cargo clippy --features datafusion-pg-functions/full -- -D warnings: clean. cargo fmt -- --check: clean.
  • Differential check against PostgreSQL 16.15, running datafusion-postgres-cli and PostgreSQL side by side and sending the same queries through psql:
    • 27 of 28 hand-picked cases identical. The other differs only in how intervals with fractional seconds are displayed (.250000 vs PostgreSQL's .25), which justify_hours and plain interval literals also show.
    • 21,000 random timestamp pairs (1700–2260, biased toward month ends, leap days and microsecond borrows) identical.
    • 11 time-zone cases identical, using zone-tagged timestamps against PostgreSQL with the session in that zone: both DST changes in New York, month borrows in Asia/Kolkata, two different zones, zoned vs plain timestamps, dates near midnight and string literals.

Limitations

  • In a session whose TimeZone is not UTC, timestamptz values reach DataFusion as UTC timestamps, so age() computes in UTC, as extract, date_trunc and ::date do today. For example, in an America/New_York session extract(day from timestamptz '2024-03-01 02:00:00+00') gives 1 here and 29 in PostgreSQL.
  • age(xid) is not provided. Arrow timestamps have no infinity, so PostgreSQL's infinite cases do not arise.

Implement PostgreSQL's age() in the datetime category, ported from
timestamp_age/timestamptz_age in PostgreSQL's timestamp.c: the two
timestamps are subtracted field by field, negative fields borrow from the
next larger one, and a day borrows the length of the earlier timestamp's
month.

- Accept timestamps of any unit, with or without a time zone, dates and
  string literals; always return Interval(MonthDayNano).
- Rewrite age(x) to age(current_date, x) during planning, as PostgreSQL
  defines it.
- Take the calendar fields of a zoned timestamp in its own zone, and
  decide which argument is earlier by the instants, as PostgreSQL does.
- Keep Arrow's nanosecond precision.
- Register the UDF under the `datetime` feature, which now enables the
  workspace's chrono dependency, mark it implemented in functions.md, and
  add age cases to datetime.slt.
PostgreSQL takes the calendar fields of both age() arguments in the
session time zone. age() took each argument in its own zone, so a
timestamp in New York against one in Kolkata, or a zoned timestamp
against a plain one, produced mixed-sign results such as
"-1 mons 30 days 14:00:00" where PostgreSQL returns "00:30:00".

When an argument carries a time zone, coerce every argument to the
first such zone: a timestamp in another zone keeps its instant, and a
plain timestamp, date or string literal is read as local time there, as
PostgreSQL reads them in the session zone.
The time-zone fix takes both arguments in the first argument's zone; the
functions.md note still said each timestamp uses its own zone.
@sunng87

sunng87 commented Oct 16, 2026

Copy link
Copy Markdown
Member

Awesome, thank you @Ishaan400 !

@sunng87
sunng87 merged commit 94f7f4c into datafusion-contrib:master Oct 16, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support age(timestamp, timestamp) and age(timestamp)

2 participants