perf(deps): drop pandas from the default install

[davidberenstein1957]

Part of #1338. Split out of #1343.

Stacked: on scaling/03-dependency-extras (#1343, numpy removal), which is itself on fix/powermetrics-nan-totals (#1345). Review #1343 first; this PR's diff is only the pandas half. Retarget once the stack lands.

pandas was used shallowly — reading three bundled reference CSVs, a handful of row filters, and the emissions.csv writer — and pulled numpy in behind it. It is now an extra, replaced by the stdlib csv module. Nothing a user writes changes.

This was split off #1343 deliberately: it reimplements read_csv/dropna semantics by hand across 14 files, which is the entire regression risk of the dependency work, and it should be reviewed on its own rather than bundled with a three-line numpy swap.

Measured on a clean 3.12 venv (uv venv && uv pip install .), whole stack

	master	stack
installed size	130 MB	68 MB
packages	39	37
cold import codecarbon	78.4 ms	37.8 ms

Import time is the median of 7 python -X importtime runs.

A default CSV tracker on master pulled pandas, numpy, prometheus_client and pycountry. prometheus_client was imported before the if OutputMethod.PROMETHEUS check; that import is now inside the branch.

The real risk surface

The original proposal's inventory listed six shallow call sites and "no vectorised arithmetic", and missed output_methods/file.py entirely — the writer for the user's emissions.csv, which did dtype-preserving in-place row updates by run_id and all-NA column dropping. That, not the reference-data parsing, is what to review. It is now on csv.DictWriter.

⚠️ Public API break — needs a release note

DataSource.get_cloud_emissions_data() and DataSource.get_cpu_power_data() now always return list[dict] and emit a DeprecationWarning. The supported names are get_cloud_emissions_rows() and get_cpu_power_rows().

An earlier revision returned a DataFrame when pandas happened to be importable and rows otherwise. That makes the return type of a public method depend on the environment, which is worse than a clean break: downstream code would work on the author's machine and fail on a bare install, at the .iloc/.query() call rather than at the import. It is a straight break now, warned and named.

Nothing in-tree breaks (three test modules and codecarbon/viz/ updated — viz builds its own frame), but these are importable, undecorated, documented-by-name methods. This repo has no CHANGELOG — release notes come from .github/release-drafter.yml — so please label this PR breaking.

Behaviour changes, both fixes

1. Emissions.get_cloud_geo_region returned pandas' NaN for 30 of 40 cloud regions that have a city but no state, because if state is not None is true for NaN. It now returns the city, as the code plainly intended.
2. Float means/sums are exactly rounded (fmean/fsum) instead of matching numpy's pairwise order — differences at 1e-16. Two tests were hardcoding pandas'/numpy's summation order in the last ulp and are now tolerance-based. A side-by-side run of the old and new paths over mock_intel_power_gadget_data.csv agrees on all 22 columns to within 4.4e-16.

Risks

CSV float formatting now comes from str(float) rather than pandas' to_csv. These agree on repr-shortest for normal values but not on NaN: str(float("nan")) is the literal "nan" where to_csv wrote an empty cell. _as_csv_row renders any non-finite float as "", covered by tests/output_methods/test_file.py::test_non_finite_values_are_written_as_empty_cells (verified to fail when the guard is removed).

Blanking infinities is a deliberate divergence from pandas: to_csv wrote the literal inf, this writes an empty cell. An infinite energy figure is not a measurement, and writing inf propagates a garbage number into everything that reads the file, where an empty cell reads back as missing.

append mode previously called dropna(axis=1, how="all") on the single new row, writing a short, column-misaligned row whenever a field was None; this writes an empty cell instead — correct, but not byte-identical to that old broken output. Three malformed TDP values in cpu_power.csv (27.29.5, 33.34.8, 29.32.9) still raise on float() exactly as before; left alone, with a comment on why the column stays text.

extrasaction="ignore" on the DictWriter is defensive only and carries a comment saying so.

Compatibility

pip install codecarbon installs less and behaves the same; codecarbon[carbonboard] and [viz-legacy] still get pandas and numpy; a new all extra pulls everything; pandas moved to the dev group, since ten test modules read output CSVs with it.

The CLI/auth extras split from the original proposal is not here. It is worth 23 MB and 8 packages with zero runtime benefit — those imports are already off the hot path — and it breaks pip install codecarbon && codecarbon monitor, a documented headline feature. pycountry (21 MB) also stays: it sits on the live geolocation path, and a lazy import with an empty fallback would be a silent country_iso_code/country_name regression.

Tests

uv run pytest tests/ -q --ignore=tests/test_viz_data.py → 633 passed, 21 skipped. test_viz_data.py fails to collect on master identically — dash not installed.

New guard in tests/test_package_integrity.py: runs a full offline tracker in a subprocess and asserts pandas, numpy and prometheus_client are absent from sys.modules. End-to-end in a bare venv, two update-mode runs produce the same 38-column, 2-row emissions.csv as master.

Not included: the CI bare-install job (that is #1342) and the docs/README install-instruction sweep.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 94.56522% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.44%. Comparing base (eb2cfef) to head (47abab9).
⚠️ Report is 3 commits behind head on scaling/03-dependency-extras.

Files with missing lines	Patch %	Lines
codecarbon/core/cpu.py	86.36%	3 Missing ⚠️
codecarbon/emissions_tracker.py	0.00%	1 Missing ⚠️
codecarbon/output_methods/file.py	96.55%	1 Missing ⚠️

Additional details and impacted files

@@                      Coverage Diff                      @@
##           scaling/03-dependency-extras    #1391   +/-   ##
=============================================================
  Coverage                         91.43%   91.44%           
=============================================================
  Files                                49       49           
  Lines                              5057     5097   +40     
=============================================================
+ Hits                               4624     4661   +37     
- Misses                              433      436    +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[benoit-cty]

🤖 This review comment was written and posted by Claude Opus 5.5 (AI assistant), at the request of @benoit-cty. Findings were checked by reading the code and running tests locally (merged with current master where relevant), but please double-check before acting on them.

Verdict: 🔧 Request changes (blocker + public API)

Replacing pandas with the stdlib csv module for the reference data and the emissions.csv writer is a good direction. The parts I checked look right:

• The dashboard builds its own DataFrame in viz/data.py behind the extra.
• The Prometheus import is now lazy.
• The CSV writer respects the header order already on disk.
• NaN and inf are written as empty cells.
• The get_cloud_geo_region NaN fix is correct.

Blocker:

1. import codecarbon fails on a default install.
   • codecarbon/core/powermetrics.py:10 still does import numpy as np. It's imported at module load through external/hardware.py:16 and core/resource_tracker.py:5.
   • numpy only ever came in through pandas. In a clean Python 3.12 venv, from codecarbon import EmissionsTracker raises ModuleNotFoundError: No module named 'numpy'.
   • Your own guard, tests/test_package_integrity.py::test_core_does_not_import_pandas_or_numpy, fails on the PR head (core tracker path imported ['numpy']).
   • Fix: bring in perf(powermetrics): drop numpy for statistics.fmean/math.fsum #1343's numpy removal from powermetrics (statistics.fmean / math.fsum), or keep numpy as a core dependency.

Public API, per maintainer direction (@benoit-cty): breaking the public API is not acceptable. It needs a new name (or a new version of the endpoint).
2. DataSource.get_cloud_emissions_data() and get_cpu_power_data() (codecarbon/input.py:191, :266) now return list[dict] instead of a DataFrame. Downstream .loc / .query calls would break at runtime. Please instead:

• Keep the existing names returning a DataFrame. Import pandas lazily inside them. If pandas is missing, raise an ImportError with a clear hint (pip install codecarbon[carbonboard] / [all]).
• Add new names that return list[dict] (e.g. get_cloud_emissions_records() / get_cpu_power_records()), and use them internally everywhere.
• Optionally, deprecate the DataFrame versions with a warning that points to the new names, and remove them in a future major version.
• Add a release note for users who relied on pandas coming in transitively.

Housekeeping:
3. The CI results are stale. They ran when the base was scaling/03-dependency-extras (#1343, now closed without merging), and the base is now master. Please push again to re-run CI, and rewrite the "stacked on #1343 → #1345" part of the description.

Nit:

• task_out indexes rows[0], which would crash on an empty list. Current callers never pass one, but a guard is cheap.