You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I was running some experiments in which the evaluator file did not directly implement a hard timeout for a given program, thus a program could run for an arbitrary amount of time. The code in process_parallel.py (L532-778) first finds a given future that is done, and then proceeds to a try-except block to recover the result. We only enforce the timeout with a future.result(timeout=timeout) call, but this is only done to futures that were completed, i.e., the ones with future.done() == True. I think this only handles futures that are done but are stuck in returning their results, but not futures that haven't been completed and are stuck or running for more than the timeout budget. In my experiments, the timeout was 60s, but some futures took over 1500s to complete, which leads to all sorts of weird situations. I know this could be solved by implementing a hard timeout in the evaluator file, but I think it would be nice if this was also enforced by openevolve itself. If this makes sense, I can make a PR later.
Fixed on main by #485. OpenEvolve already applies evaluator.timeout to every evaluation stage, but when a stage timed out, the worker still waited for the orphaned thread at shutdown. That is why futures could run far past the timeout. Evaluations now use a dedicated executor, so a timed-out evaluation returns promptly and the worker moves on. One limitation remains: Python can't kill a running thread, so a runaway program keeps using CPU in the background until it finishes. For hard kills, run the candidate in a subprocess from your evaluator. Closing, but please reopen if you still see stuck futures.
Activity
codelion commentedon Feb 12, 2026
Yes this would be good to handle in OpenEvolve.
codelion commentedon Sep 28, 2026
Fixed on main by #485. OpenEvolve already applies
evaluator.timeoutto every evaluation stage, but when a stage timed out, the worker still waited for the orphaned thread at shutdown. That is why futures could run far past the timeout. Evaluations now use a dedicated executor, so a timed-out evaluation returns promptly and the worker moves on. One limitation remains: Python can't kill a running thread, so a runaway program keeps using CPU in the background until it finishes. For hard kills, run the candidate in a subprocess from your evaluator. Closing, but please reopen if you still see stuck futures.