Respect no_proxy environment variable - #2171
Conversation
An explicitly configured proxy was used for every request, even when the target host matched the no_proxy / NO_PROXY environment variable. Add a _proxies_for_url helper that returns an empty proxy mapping for bypassed hosts so libcloud behaves consistently with other HTTP clients.
There was a problem hiding this comment.
🟡 Changes recommended
The new regression test’s environment cleanup can leak no_proxy state (and doesn’t isolate NO_PROXY), potentially causing flaky tests or side effects across the test suite.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates Libcloud’s Requests integration so that explicitly configured proxies (via set_http_proxy() / proxy_url) are bypassed for targets matching no_proxy / NO_PROXY, aligning behavior with Requests/curl/urllib semantics.
Changes:
- Add
LibcloudBaseConnection._proxies_for_url(url)to selectively disable proxies whenshould_bypass_proxies()indicatesno_proxyshould apply. - Pass
proxies=per-request inLibcloudConnection.request()to allow bypass behavior for matching hosts. - Add a regression test and a CHANGES entry documenting the behavior change.
File summaries
| File | Description |
|---|---|
| libcloud/http.py | Adds per-URL proxy selection and wires it into requests made by LibcloudConnection. |
| libcloud/test/test_connection.py | Adds a regression test validating proxy bypass behavior for no_proxy hosts. |
| CHANGES.rst | Documents the new no_proxy/NO_PROXY proxy-bypass behavior. |
Review details
- Files reviewed: 3/3 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| os.environ["no_proxy"] = "internal.example.com" | ||
| self.addCleanup(os.environ.pop, "no_proxy", None) |
…ars in test cleanup and pin NO_PROXY
|
Thanks @micafer — good catch. The test now snapshots both no_proxy and NO_PROXY up front, pins NO_PROXY to unset for the test's duration, and the addCleanup restores the original values instead of unconditionally popping no_proxy after tearDown. The externally-set NO_PROXY case you (and Copilot) flagged can't leak in or out anymore. |
micafer
left a comment
There was a problem hiding this comment.
I checked and it seems that returning an empty proxy mapping is not sufficient to bypass a proxy configured on Session.proxies.
Requests merges the per-request proxies argument with the session proxy configuration. When {} is passed, the session-level http / https proxies can therefore be merged back in, so the request may still use the explicitly configured proxy even though _proxies_for_url() returned {}.
The current regression test only asserts the return value of _proxies_for_url(), so it doesn't exercise this merge behavior.
Could you add an end-to-end-ish test around LibcloudConnection.request() which verifies the effective proxy configuration after Requests processing? You may need to explicitly override the configured schemes with None (e.g. {"http": None, "https": None}) when should_bypass_proxies() returns true, rather than returning {}.
Respect no_proxy environment variable
Description
Fixes #2077.
When a proxy is configured explicitly (via
set_http_proxy()or thehttp_proxy/https_proxyenvironment variables), libcloud used it for every request, includinghosts listed in
no_proxy/NO_PROXY. Other HTTP clients (curl, requests via itsown env handling, urllib) all honour
no_proxy, so this was surprising and made itimpossible to talk to an internal endpoint directly while a proxy was configured.
This adds a
_proxies_for_url(url)helper onLibcloudBaseConnectionthat returnsan empty proxy mapping (
{}) when the target host matchesno_proxy, andNoneotherwise so the session default applies. It is passed as
proxies=inLibcloudConnection.request(). Matching is delegated torequests.utils.should_bypass_proxies,so libcloud inherits the exact same
no_proxysemantics as requests rather thanreimplementing them.
Status
done, ready for review
Checklist (tick everything that applies)