fix: restore signal handlers on lock release - #1329
davidberenstein1957 wants to merge 3 commits into
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1329 +/- ##
==========================================
+ Coverage 91.70% 91.73% +0.03%
==========================================
Files 49 49
Lines 5157 5178 +21
==========================================
+ Hits 4729 4750 +21
Misses 428 428 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
94541ee to
984f4b7
Compare
`Lock` installed SIGINT/SIGTERM handlers and threw away the previous ones, so the host application's handlers were destroyed and Ctrl-C stopped raising KeyboardInterrupt. Save the previous handlers, chain to them from `_handle_exit`, and restore them in `release()`, unregistering the atexit hook so a released lock is not pinned. Handlers are installed in `acquire()` after `open(LOCKFILE, "x")` succeeds, not in `__init__`. On the "another instance is already running" path `acquire()` raises, the tracker sets `_another_instance_already_running`, and `stop()` returns at its early guard without ever reaching `release()` -- so handlers installed in the constructor stayed hijacked for the life of the process. The thread lock is reentrant: `_handle_exit` calls `release()`, which takes `_thread_lock`, so a signal delivered while the same thread was inside `acquire()`/`release()` deadlocked on a plain `Lock`. `release()` is also idempotent now (moved here from #1336, since it edits the same few lines this branch already rewrites), and the `_atexit_hook` indirection is dropped: `atexit.unregister()` compares with `==`, not identity, so a bound method unregisters fine. Tests cover the default and ignored signal dispositions, and the deadlock test unregisters its atexit hook so a reverted lock.py fails the suite instead of wedging the interpreter at exit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
984f4b7 to
004a7e3
Compare
Verdict: 🔧 Request changesThe bug is real.
Must fix:
Nits:
|
- release() removes the lock file first and only restores signal handlers on the main thread, so stop() from a worker thread still writes its CSV. - With a default previous handler, raise SystemExit(128 + signum) (or KeyboardInterrupt for SIGINT) instead of re-sending the signal, so finally / __exit__ still write the final emissions. - Treat a None previous handler (installed from C) as SIG_DFL. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Made the changes in f132178: worker-thread release and SIGTERM cleanup fixed.
|
Description
codecarbon/lock.pynow saves the signal handlers replaced bysignal.signal()inself._previous_handlers(main thread only, once the lock file is acquired) instead of discarding them._handle_exitreleases the lock and then delegates to the handler it replaced: it calls a previous Python handler, does nothing forSIG_IGN, and for the default disposition raisesSystemExit(128 + signum)orKeyboardInterruptfor SIGINT, sofinally/__exit__/@track_emissionsstill write the final emissions.release()removes the lock file first, then restores the saved handlers, but only when the currently installed handler is still the one it set, so an application that registered its own handler afterward is not clobbered; restoring is skipped off the main thread, wheresignal.signal()raises, sostop()from a worker thread still writes its CSV and removes the lock file. The internal thread lock is now anRLock, since_handle_exitcan callrelease()on a thread already holding it.Related Issue
Fixes #1310
Motivation and Context
Lockis constructed wheneverallow_multiple_runs=False. Because it overwrote the process signal disposition permanently, restoring the default SIGINT handler never happened, soKeyboardInterruptstopped being raised at all — everyexcept KeyboardInterrupt:in an embedding application became dead code, including CodeCarbon's own incodecarbon/cli/monitor.py. Applications that had registered a graceful-shutdown SIGTERM handler also lost it silently.Behavior change: previously any SIGINT/SIGTERM ended in
SystemExit(1). Now the process does whatever the application asked for; with no application handler, SIGTERM exits with the conventional code 143 (after cleanup) and SIGINT raisesKeyboardInterrupt.How Has This Been Tested?
tests/test_lock.py(TestLockSignalHandlers) covers: handlers restored on release, forwarding to a previous Python handler,SystemExit(143)on SIGTERM with the default disposition,SIG_IGNkept, release from a worker thread, aNoneprevious handler, no deadlock on re-entrant release, and a failedacquire()leaving handlers alone.Screenshots (if appropriate):
N/A
Types of changes
AI Usage Disclosure
Checklist: