fix: detect free-threaded Python from venv and uv before setting PYTHONMALLOC by not-matthias · Pull Request #539 · CodSpeedHQ/codspeed

not-matthias · 2026-09-16T13:53:51Z

Problem

Free-threaded CPython (3.13t+) refuses to start with PYTHONMALLOC=malloc. The valgrind executor already skips that override when it detects a free-threaded interpreter, but the detection only probed python on the PATH. It missed the interpreter the benchmark actually runs under when that interpreter lives in a virtual environment or is selected by uv, so runs using astral-sh/setup-uv with python-version: 3.13t still failed.

Change

Probe every interpreter the benchmark command could resolve to, and skip the override if any of them is free-threaded:

python / python3 on the PATH
$VIRTUAL_ENV/bin/python
.venv/bin/python in the benchmark working directory
the interpreter uv python find $UV_PYTHON resolves to (setup-uv exports UV_PYTHON from its python-version input). uv downloads interpreters lazily, so when the request is not installed yet the request string itself (3.13t, +freethreaded) decides.

The probe now reads sys.abiflags instead of sysconfig.get_config_var('Py_GIL_DISABLED'). import sysconfig fails when _PYTHON_SYSCONFIGDATA_NAME is set for a different interpreter (common in Nix shells), which turned the old check into a silent false negative for exactly the venvs it needs to detect.

Verification

Release-mode smoke runs against real interpreters created with uv venv:

scenario	free-threaded detected
no venv, PATH python 3.12	no
plain 3.12 `.venv` in cwd	no
3.14t `.venv` in cwd	yes
`UV_PYTHON=3.14t` (installed), no venv	yes
`UV_PYTHON=3.13t` (not installed), no venv	yes (request fallback)
`UV_PYTHON=3.12`, no venv	no
`VIRTUAL_ENV=<3.14t venv>`, cwd has no venv	yes

Plus rstest cases for the UV_PYTHON request-string classifier.

…ONMALLOC Free-threaded CPython (3.13t+) refuses to start with PYTHONMALLOC=malloc, so the valgrind executor skips that override when the interpreter is free-threaded. The detection only probed `python` on the PATH, which misses the interpreter the benchmark actually runs under when it lives in a virtual environment or is selected by uv. Probe every interpreter the command could resolve to and skip the override if any of them is free-threaded: - `python` / `python3` on the PATH - `$VIRTUAL_ENV/bin/python` - `.venv/bin/python` in the benchmark working directory - the interpreter `uv python find $UV_PYTHON` resolves to; when uv has not downloaded it yet, classify the request string (`3.13t`, `+freethreaded`) directly The probe reads `sys.abiflags` instead of `sysconfig`, since `sysconfig` fails to import when `_PYTHON_SYSCONFIGDATA_NAME` is set for a different interpreter, which turned the old check into a false negative for exactly the free-threaded venvs it needs to detect.

codspeed · 2026-09-16T14:00:40Z

Merging this PR will not alter performance

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 17 untouched benchmarks

_{Comparing cod-296-fails-with-python313-freethreaded (51f8918) with main (f198706)}

not-matthias · 2026-09-16T14:13:20Z

NOTE: This PR tries to fix the known cases when a free-threaded python can be used and try to not use the environment variable. However, there are many more cases that can bypass our checks (e.g. using custom compiled python).

In the future, we can explore removing PYTHONMALLOC, but this would be a breaking change so it should only be included in the next runner release (as to why this partial solution exists)

greptile-apps · 2026-09-16T14:16:13Z

Confidence Score: 4/5

The PR is not safe to merge until free-threaded interpreters invoked through supported explicit names, paths, or console-script entrypoints are detected.

Findings

**Interpreter Candidates Remain Incomplete** ▶

Fix with agent prompt

### Issue 1
src/executor/valgrind/helpers/python.rs:20-25
Benchmark commands can invoke a free-threaded interpreter through a name or path outside this fixed candidate list, such as `python3.13t script.py` or a `pytest` console script from a custom-named virtual environment. Those commands are accepted as arbitrary shell commands, but detection checks only `python`, `python3`, two conventional virtual-environment paths, and `UV_PYTHON`. The detector therefore returns false, `measure` sets `PYTHONMALLOC=malloc`, and the free-threaded interpreter refuses to start. Resolve or probe the interpreter used by the configured command rather than assuming this list covers every possible interpreter.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR broadens free-threaded Python detection before configuring the Valgrind benchmark environment.

Probes PATH, active and project virtual environments, and UV_PYTHON-selected interpreters.
Replaces the sysconfig-based probe with a sys.abiflags check.
Passes the benchmark working directory into detection.
The fixed candidate list still does not cover arbitrary interpreter paths and entrypoints accepted by the benchmark command interface.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  C[Configured shell command] --> A[Actual interpreter resolved by shell or shebang]
  D[Free-threaded detector] --> P[PATH python and python3]
  D --> V[VIRTUAL_ENV and project .venv]
  D --> U[UV_PYTHON resolution]
  P --> F{Any candidate free-threaded?}
  V --> F
  U --> F
  F -- No --> M[Set PYTHONMALLOC=malloc]
  F -- Yes --> S[Leave allocator unset]
  A -. Versioned name, explicit path, or console-script shebang may bypass candidates .-> M

_{Reviews (1) · Last reviewed commit: "fix: detect free-threaded Python from ve..."}

greptile-apps · 2026-09-16T14:16:16Z

+
+    let mut candidates: Vec<PathBuf> = vec![PathBuf::from("python"), PathBuf::from("python3")];
+    if let Some(venv) = std::env::var_os("VIRTUAL_ENV") {
+        candidates.push(Path::new(&venv).join("bin/python"));
+    }
+    candidates.push(cwd.join(".venv/bin/python"));


Benchmark commands can invoke a free-threaded interpreter through a name or path outside this fixed candidate list, such as python3.13t script.py or a pytest console script from a custom-named virtual environment. Those commands are accepted as arbitrary shell commands, but detection checks only python, python3, two conventional virtual-environment paths, and UV_PYTHON. The detector therefore returns false, measure sets PYTHONMALLOC=malloc, and the free-threaded interpreter refuses to start. Resolve or probe the interpreter used by the configured command rather than assuming this list covers every possible interpreter.

Knowledge Base Used: Valgrind measurement

Prompt To Fix With AI

This is a comment left during a code review. Path: src/executor/valgrind/helpers/python.rs Line: 20-25 Comment: Benchmark commands can invoke a free-threaded interpreter through a name or path outside this fixed candidate list, such as `python3.13t script.py` or a `pytest` console script from a custom-named virtual environment. Those commands are accepted as arbitrary shell commands, but detection checks only `python`, `python3`, two conventional virtual-environment paths, and `UV_PYTHON`. The detector therefore returns false, `measure` sets `PYTHONMALLOC=malloc`, and the free-threaded interpreter refuses to start. Resolve or probe the interpreter used by the configured command rather than assuming this list covers every possible interpreter. **Knowledge Base Used:** [Valgrind measurement](https://app.greptile.com/codspeed/-/custom-context/knowledge-base/codspeedhq/codspeed/-/docs/valgrind-measurement.md) --- For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Add an experimental flag that unsets PYTHONMALLOC for simulation runs, allowing integrations to validate workloads without the forced malloc allocator before it becomes the default behavior.

not-matthias marked this pull request as ready for review September 16, 2026 14:11

not-matthias requested a review from GuillaumeLagrange September 16, 2026 14:11

greptile-apps Bot reviewed Sep 16, 2026

View reviewed changes

feat(simulation): add Python allocator opt-out

51f8918

Add an experimental flag that unsets PYTHONMALLOC for simulation runs, allowing integrations to validate workloads without the forced malloc allocator before it becomes the default behavior.

Navigation Menu

Uh oh!

fix: detect free-threaded Python from venv and uv before setting PYTHONMALLOC - #539

fix: detect free-threaded Python from venv and uv before setting PYTHONMALLOC#539
not-matthias wants to merge 2 commits into
mainfrom
cod-296-fails-with-python313-freethreaded

not-matthias commented Sep 16, 2026

Uh oh!

codspeed Bot commented Sep 16, 2026 •

edited

Loading

Uh oh!

not-matthias commented Sep 16, 2026

Uh oh!

greptile-apps Bot commented Sep 16, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot Sep 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

not-matthias commented Sep 16, 2026

Problem

Change

Verification

Uh oh!

codspeed Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

not-matthias commented Sep 16, 2026

Uh oh!

greptile-apps Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 4/5

Findings

Summary

Uh oh!

greptile-apps Bot Sep 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

codspeed Bot commented Sep 16, 2026 •

edited

Loading

greptile-apps Bot commented Sep 16, 2026 •

edited

Loading