feat(api): reuse connections, add timeouts, and stamp emissions with measurement time

[davidberenstein1957]

Description

ApiClient._request called bare requests.get/post/patch with a hardcoded timeout=2, no Session and no retry policy, and http.py:_emit logged failures away. This adds a Session with a jittered Retry adapter, configurable timeouts (api_timeout, api_retries), and a backoff guard around run creation. It also stamps emissions with their measurement time. POST requests use a deliberately narrower retry policy than GET/PATCH, since carbonserver has no idempotency key on POST /emissions and the dashboard sums emission rows, so retrying a POST whose response was lost after the insert committed would silently inflate a user's reported emissions.

Related Issue

Part of #1338

Motivation and Context

Without connection reuse or retries, every emission POST opened a new TCP connection, and a single transient failure (503, connection error) permanently dropped a measurement row with no visibility that it happened. This is stacked on #1341 (scaling/01-emission-timestamps), so its diff is limited to its own two commits; retarget to master once #1341 lands.

How Has This Been Tested?

Measured against a loopback stub server (no real network): 50 sequential emission POSTs went from 50 TCP connections / 0.085s to 1 connection / 0.023s, and a row hitting two transient 503s went from 0/1 rows delivered to 1/1 delivered. New test file tests/test_api_client_retry.py, 14 tests covering retry-then-succeed on 503, give-up after N, no-retry on 4xx, retry on connection error, connection reuse, timeout pass-through, both run-creation backoff directions, and pinning of the POST-specific policy (read timeout on POST not retried, 503/504 on POST handled per the split policy, read timeout on GET still retried). Plus test_api_timeout_and_retries_reach_the_api_client in tests/test_emissions_tracker.py, asserting the two config knobs survive the constructor path. Tests use a stdlib loopback server rather than requests_mock, since requests_mock bypasses the HTTPAdapter/Retry layer under test. Known flake, verified pre-existing on master: tests/test_emissions_tracker_flush.py and tests/test_logging_output.py flake on the test machine independent of this change.

Screenshots (if appropriate):

N/A

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

AI Usage Disclosure

• 🟥 AI-vibecoded
• 🟠 AI-generated
• ⭐ AI-assisted
• ♻️ No AI used

Checklist:

• My code follows the code style of this project.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have read the docs/how-to/contributing.md document.
• I have added tests to cover my changes.
• All new and existing tests passed.

api_timeout (default 5s) and api_retries (default 2) are documented in docs/how-to/configuration.md, including which failures are retried on POST and why. Not included in this PR: aligning the direct requests.get calls in electricitymaps_api/geography.py with the same policy, and a delivery-failure counter in the final log line.

[codecov]

Codecov Report

❌ Patch coverage is 96.77419% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 91.71%. Comparing base (3ec31a0) to head (72c53b7).
⚠️ Report is 41 commits behind head on master.

Files with missing lines	Patch %	Lines
codecarbon/output_methods/http.py	50.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1340      +/-   ##
==========================================
+ Coverage   91.43%   91.71%   +0.27%     
==========================================
  Files          49       49              
  Lines        5057     5176     +119     
==========================================
+ Hits         4624     4747     +123     
+ Misses        433      429       -4     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[benoit-cty]

🤖 This review comment was written and posted by Claude Opus 5.5 (AI assistant), at the request of @benoit-cty. Findings were checked by reading the code and running tests locally (merged with current master where relevant), but please double-check before acting on them.

Verdict: ✅ Approve with nits (please merge #1339 first)

Session reuse (with close() wired into CodeCarbonAPIOutput.exit()), the (3.05, 10) connect/read timeouts, and stamping emissions with EmissionsData.timestamp all look good. 80 tests pass, and the loopback test that counts connections is nice. There are no retries at all, so there is no risk of a replayed POST creating duplicate emissions.

Issues:

1. The description doesn't match the code. It describes:

   • a Retry adapter with a split POST vs GET/PATCH policy
   • api_timeout / api_retries config, and docs for them
   • a run-creation backoff guard
   • tests/test_api_client_retry.py
   • "stacked on fix(api): stamp emissions with measurement time, not send time #1341" (now closed)

   None of these are in the diff. Please update the description, or add the missing parts.

2. A hung API now blocks the measurement thread much longer (api_client.py:37, output_methods/http.py:62-68).

   • When run_id is None, _emit calls _ensure_api_run() and then add_emission(), and each can call _create_run.
   • Against a hung API, one live_out can block the scheduler thread for about 2 × 13 s ≈ 26 s, versus about 4 s on master. That is longer than the default 15 s tick.
   • Suggested fix: at most one _create_run attempt per _emit, plus a cooldown after a failed run creation (the "backoff guard" from the description).
   • Together with refactor(scheduler): run ticks on one daemon thread instead of chained Timers #1339 (5 s join cap in stop()), this also makes it more likely that the final flush overlaps an in-flight tick.

Nit:

• _measurement_timestamp (api_client.py:40) takes a naive, second-precision local time and calls .astimezone() on it. In the repeated hour at the DST fall-back, the offset can be wrong by an hour. Creating a timezone-aware timestamp at the source (emissions_tracker.py:1076, e.g. datetime.now().astimezone()) avoids that.

[davidberenstein1957]

Made the changes in 72c53b7: 60 s cooldown after a failed run creation.

On the 2 x 13 s concern: _create_run re-raises every failure, so _emit stops after the first attempt; with the cooldown a down API now costs one ~13 s call per minute, not per tick. Stays compatible with #1339 (no shared files).

Not done: rewriting the description (my edit was blocked; the retry/config/backoff sections are stale, the diff is Session + timeouts + measurement timestamp + cooldown). Skipped the DST nit: making the tracker timestamp offset-aware changes the CSV timestamp format, which needs a maintainer call.

[davidberenstein1957]

Deferring the DST timestamp nit: making the timestamp timezone-aware at the source changes the CSV timestamp format, so it will ship in the next minor release together with #1334 rather than in this PR.