feat: cache and back off Electricity Maps carbon intensity - #1358
davidberenstein1957 wants to merge 2 commits into
Conversation
|
CI fix pushed. The failing test ( What changed is the test only: it now patches The caching behaviour itself remains covered by Verified locally: |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1358 +/- ##
==========================================
+ Coverage 91.43% 91.84% +0.41%
==========================================
Files 49 49
Lines 5057 5201 +144
==========================================
+ Hits 4624 4777 +153
+ Misses 433 424 -9 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
`find_green_window` fetched the forecast and then asked /latest for the current intensity, a second HTTP call whose value only fed a "saves ~X%" line and the --threshold short-circuit. The forecast's first point is that same period, so use it and drop the call, the fallback and the try/except with it. Add --finish-by as the complement to --deadline: --deadline bounds the start, --finish-by bounds the end and is what most people mean. It is a subtraction, not a second search path. The Electricity Maps request extraction this branch used to carry now lives in its base branch (#1358) where it belongs, so `clear_cooldown` is gone: request() clears its own location's cooldown on a usable response. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`find_green_window` fetched the forecast and then asked /latest for the current intensity, a second HTTP call whose value only fed a "saves ~X%" line and the --threshold short-circuit. The forecast's first point is that same period, so use it and drop the call, the fallback and the try/except with it. Add --finish-by as the complement to --deadline: --deadline bounds the start, --finish-by bounds the end and is what most people mean. It is a subtraction, not a second search path. The Electricity Maps request extraction this branch used to carry now lives in its base branch (#1358) where it belongs, so `clear_cooldown` is gone: request() clears its own location's cooldown on a usable response. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
8d9862e to
73a4188
Compare
`find_green_window` fetched the forecast and then asked /latest for the current intensity, a second HTTP call whose value only fed a "saves ~X%" line and the --threshold short-circuit. The forecast's first point is that same period, so use it and drop the call, the fallback and the try/except with it. Add --finish-by as the complement to --deadline: --deadline bounds the start, --finish-by bounds the end and is what most people mean. It is a subtraction, not a second search path. The Electricity Maps request extraction this branch used to carry now lives in its base branch (#1358) where it belongs, so `clear_cooldown` is gone: request() clears its own location's cooldown on a usable response. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dec7cad to
0d96e25
Compare
Carbon intensity was fetched from the Electricity Maps API on every emissions computation, so a long run with a short `measure_power_secs` issued thousands of requests for a value the grid publishes hourly. A failing token produced one doomed request per measurement tick for the whole run. `get_carbon_intensity()` is extracted from `get_emissions()`, its result is cached for 60 s per location and token, and the API goes into a flat 60 s cooldown after a failure. `get_emissions()` is unchanged for callers. A 60 s TTL is deliberate: 5 minutes silently overrode the ~2 minute `api_call_interval` cadence and halved the intensity resolution. Both the cache and the cooldown are keyed by location and token, so trackers with different tokens do not share a value and one tracker's bad token or unreachable network cannot block another tracker's good one. The token is used directly as part of the in-process dict key and is never rendered into a log line. Cooldown raises a dedicated error logged at debug, so a bad token no longer produces one error line per tick. Cache and cooldown state are read-modify-written from the background measurement thread, so they are guarded by one module-level lock, never held across the HTTP request. Behaviour change worth calling out: a non-200 whose body is not the expected JSON error object now surfaces `resp.text` instead of raising a `JSONDecodeError` (or `ElectricityMapsAPIError(None)` when the body is JSON without `error`/`message`). Covered by a test on a 502 HTML body. Refs #1354 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
6699eab to
de92ca8
Compare
Verdict: 🔧 Request changes (small)Extracting Must fix:
Low: Nit:
|
… cache docs Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Made the changes in d9b956f: cooldown now 30 s doubling to 1 h (reset on success, tested), docs no longer claim 3-in-4 savings, intensity timeout cut to 10 s, and |
Description
Slice 1 of 5 of the pluggable carbon-intensity provider work described in #1354. It fixes two live defects in the current Electricity Maps path without introducing any new abstraction, config key, or output field.
get_carbon_intensity(geo, token) -> floatis extracted out ofget_emissions(), the intensity is cached for 60 seconds keyed by location (lat/lonorcountryCode) and token, and after a failure the location+token goes into an exponential cooldown doubling from 30s to a 1-hour ceiling during which no request is issued. Skipped requests raiseElectricityMapsAPICooldownError, logged at debug level, so a bad token produces one error line instead of one per API call.get_emissions()keeps its exact signature and return value.Related Issue
Refs #1354 (no issue closed by this PR directly)
Motivation and Context
codecarbon/core/electricitymaps_api.pyfetched grid carbon intensity on every emissions computation. On a long run that is a lot of HTTP requests for a value the grid publishes hourly at most, and when the token is wrong or the network is down, every one of them is doomed and logs an error — one per measurement tick for the whole run.How Has This Been Tested?
tests/test_electricitymaps_cache.py, all network mocked withresponses: cache hit within TTL, a long run bounded to one request, refetch after expiry, per-location keying, no request while in cooldown, cooldown doubling to the ceiling, cooldown reset after success, and cooldown isolation between tokens.Screenshots (if appropriate):
N/A
Types of changes
AI Usage Disclosure
Checklist:
Behaviour change, not a pure optimisation
The 60s TTL means measurements inside that window convert energy with the same intensity value rather than a freshly fetched one. The TTL is deliberately shorter than the default
api_call_interval × measure_power_secs(~2 minutes), so the cache only collapses the extra calls that tasks andstop()add on top. Documented indocs/how-to/configuration.md.What is deferred (future slices of #1354)
Slice 2 lifts today's bundled-data branches into
StaticProviderunchanged. Slice 3 puts Electricity Maps behind the protocol. Slice 4 addsresolve_intensity()with a fallback chain and thecarbon_intensity_providersconfig key. Slice 5 exposescarbon_intensity_g_co2e_kwhandcarbon_intensity_sourceonEmissionsData. Further out: ENTSO-E and WattTime providers, time-weighted intensity. Also deliberately not in this slice: stale-serve (returning an expired cached value on API error), which should land with the slice-5is_livereporting.This PR was opened as a draft pending review of the overall direction in #1354.