Repository navigation
in duckdb==1.5.3, DuckDBPyRelation.to_arrow_table() raises InternalException for GEOMETRY('EPSG:xxx') columns #475
Copy link
Copy link
Closed
Labels
Description
What happens?
I was trying to get ibis working with duckdb 1.5.3 and came across a crash in duckdb that is triggered by some of the ibis geospatial unit tests
I've had copilot put together a minimal reproducer - see below
To Reproduce
"""
Minimal reproducer for DuckDB 1.5.3 regression:
DuckDBPyRelation.to_arrow_table() raises InternalException when a query
result contains a GEOMETRY column with an associated EPSG CRS
(i.e. type reported as GEOMETRY('EPSG:xxxx')).
Root cause (from stack trace):
CoordinateReferenceSystem::TryConvert() calls
Transaction::Get(ClientContext, AttachedDatabase) while building the Arrow
schema, but there is no active transaction at that point.
ArrowGeometry::WriteCRS -> CoordinateReferenceSystem::TryConvert ->
Transaction::Get -> "TransactionContext::ActiveTransaction called without
active transaction" -> INTERNAL assertion.
The workaround (rel.to_arrow_reader().read_all()) succeeds because it goes
through the streaming Arrow IPC path, which serialises the CRS differently and
does not trigger the catalog lookup.
Tested on: duckdb==1.5.3, Python 3.13.13
"""
import json
import os
import tempfile
import duckdb
print(f"duckdb version: {duckdb.__version__}")
con = duckdb.connect()
con.load_extension("spatial")
# Minimal GeoJSON with an explicit EPSG:2263 CRS
# (any EPSG code reproduces the issue; 2263 = NY State Plane Long Island)
geojson = {
"type": "FeatureCollection",
"crs": {"type": "name", "properties": {"name": "urn:ogc:def:crs:EPSG::2263"}},
"features": [
{
"type": "Feature",
"properties": {},
"geometry": {"type": "Point", "coordinates": [935996.0, 191376.0]},
},
{
"type": "Feature",
"properties": {},
"geometry": {"type": "Point", "coordinates": [935000.0, 190000.0]},
},
],
}
with tempfile.TemporaryDirectory() as tmp:
path = os.path.join(tmp, "test_epsg.geojson")
with open(path, "w") as f:
json.dump(geojson, f)
rel = con.sql(f"SELECT * FROM ST_READ('{path}')")
# DuckDB correctly reports the GEOMETRY type with the embedded CRS:
print(f"Column types: {rel.dtypes}")
# Expected: [BIGINT, GEOMETRY('EPSG:2263')]
# Workaround: to_arrow_reader() works fine
print("\n--- to_arrow_reader().read_all() (workaround) ---")
tbl = rel.to_arrow_reader().read_all()
print(f"OK — schema:\n{tbl.schema}\n")
# Bug: to_arrow_table() raises InternalException
# Stack:
# DuckDBPyResult::FetchArrowTable
# ArrowConverter::ToArrowSchema
# SetArrowFormat -> SetArrowExtension -> ArrowGeometry::PopulateSchema
# ArrowGeometry::WriteCRS
# CoordinateReferenceSystem::TryConvert <-- needs active transaction
# Transaction::Get(ClientContext, AttachedDatabase)
# "TransactionContext::ActiveTransaction called without active transaction"
print("--- rel.to_arrow_table() (should raise InternalException) ---")
try:
tbl2 = rel.to_arrow_table()
print(f"Unexpectedly OK: {tbl2.schema}")
except duckdb.InternalException as e:
print(f"REGRESSION CONFIRMED — duckdb.InternalException: {e}")
OS:
Linux (Ubuntu 24.04 x86_64)
DuckDB Package Version:
1.5.3
Python Version:
3.13.13
Full Name:
Jonathan Clarke
Affiliation:
FiveSigma Finance
What is the latest build you tested with? If possible, we recommend testing with the latest nightly build.
I have tested with a stable release
Did you include all relevant data sets for reproducing the issue?
Yes
Did you include all code required to reproduce the issue?
- Yes, I have
Did you include all relevant configuration to reproduce the issue?
- Yes, I have
Reactions are currently unavailable
Activity
JonAnCla commentedon May 29, 2026
It could be that this ought to go in the "spatial" repo, LMK if so. Thanks!
evertlammerts commentedon Jun 16, 2026
Fixed in #495 , will be part of v1.5.4 (this week's release)