Repository navigation
Skip inaccessible processes in health check script detection - #5543
Conversation
scripts_are_running() called process.cmdline() on every process without handling psutil errors. On Windows Server 2025, protected processes raise psutil.AccessDenied, which propagated out of the health check RequestHandler and made the server close the connection without sending a response. As a result, the MIG health check never succeeded and the instances were recreated by autohealing. Skip processes whose command line cannot be read, matching the handling already done in terminate_processes_matching_cmd_line().
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request improves the process handler's robustness by catching exceptions (such as AccessDenied, NoSuchProcess, and OSError) when retrieving a process's command line, ensuring that protected or inaccessible system processes are skipped rather than causing the health check to fail. A corresponding unit test was added to verify this behavior. Feedback on the changes points out a potential issue where a list is modified during iteration, which can lead to skipped elements, and suggests iterating over a copy of the list instead.
g-ortuno
added
the
ready for GOSST review
| try: | ||
| process_cmd_line = process.cmdline() | ||
| except (psutil.AccessDenied, psutil.NoSuchProcess, OSError): | ||
| # Some processes (e.g. protected system processes on Windows Server |
There was a problem hiding this comment.
NIT:
Can you add a debug level log so that we can later verify which processes were skipped?
There was a problem hiding this comment.
Note: We can now use DEBUG level logs, but for them to be visible one would have to start a bot with the env var: LOG_LEVEL = DEBUG so no worries here
There was a problem hiding this comment.
I think we wouldn't have quite useful information in that log, because the exception occurs when trying to get the command line info. So we could see that something was skipped but not really know if that was relevant
4 participants
scripts_are_running()calledprocess.cmdline()on every process without handling psutil errors. On Windows Server 2025, protected processes raise psutil.AccessDenied, which propagated out of the health check RequestHandler and made the server close the connection without sending a response. As a result, the MIG health check never succeeded and the instances were recreated by autohealing.Skip processes whose command line cannot be read, matching the handling already done in
terminate_processes_matching_cmd_line()insrc/clusterfuzz/_internal/system/process_handler.py