Skip to content

rust(bug): fix python bindings segfault - #804

Merged
solidiquis merged 4 commits into
mainfrom
python/rust-bindings-segfault
Sep 29, 2026
Merged

solidiquis merged 4 commits into
mainfrom
python/rust-bindings-segfault

Conversation

@solidiquis

@solidiquis solidiquis commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

rust(fix): stop the sift-stream-bindings tokio runtime before python exit

Relevant issues:

Problem

User reported segfault. Happens consistently in the tokio-rt-worker thread. Happens when futures finish
on the tokio side but Python interpreter already shut down.

Fix

The bindings now own their tokio runtime (src/runtime.rs) and plug it into pyo3_async_runtimes::generic, so task locals, cancellation, and future resolution are unchanged.
All sixteen future_into_py call sites route through it.

  • New shutdown(timeout=5.0) stops the runtime and joins its threads with the GIL released. The module registers it with atexit at import, so a normal interpreter exit always runs it while the interpreter is still intact. It can also be called directly, for example before os._exit().
  • New is_shut_down() reports whether it has run. Calls that need the runtime afterwards raise RuntimeError instead of crashing.
  • Runtime threads are named sift-stream-worker so future crash logs identify them.

Verification

User provided a repro script which was used for verification.

Linux container (Debian 13, aarch64, CPython 3.11.16, glibc 2.41), running the user's repro script unchanged:

Build User repro, 150 runs Deterministic repro, 20 runs
main (0.5.1) 134 clean, 10 SIGABRT, 6 SIGSEGV 20 clean, pyo3 panics on every worker
this branch 150 clean 20 clean, no stderr
  • Bindings test suite: 63 passed on Linux 3.11, macOS 3.11, and macOS 3.13.
  • cargo clippy --all-targets -D warnings clean. Generated .pyi stub unchanged.

Releases

  • sift-stream-bindings 0.5.1 to 0.5.2, changelog entry under Unreleased in rust/CHANGELOG.md.
  • Once this is merged we will release a new version of the bindings then follow-up with another PR bumping the Python version and release that.

@solidiquis
solidiquis requested a review from tsift September 29, 2026 19:33
tsift
tsift previously approved these changes Sep 29, 2026
Comment thread rust/crates/sift_stream_bindings/src/lib.rs Outdated
@solidiquis
solidiquis merged commit c8606e7 into main Sep 29, 2026
24 checks passed
@solidiquis
solidiquis deleted the python/rust-bindings-segfault branch September 29, 2026 22:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants