Repository navigation
Proposal: an optional evidence check on pull requests (demo on 5 merged PRs) #452
Description
Hi @thodson-usgs, thanks again for the review and merge on #440.
I've been building proofcheck (https://github.com/bhaskargurram-ai/proofcheck). It's a GitHub check that tests a PR by running it. It runs the changed tests on the base and head commits, then reverts each changed hunk to see which ones the passing tests depend on.
To see whether it's useful here, I replayed five recently merged dataretrieval fixes in a fork. All five come back ✅:
| Upstream PR | Fail before the fix, pass after | Code hunks the tests depend on |
|---|---|---|
| #427 qualifier filter as JSON array | 3 tests | 4 of 4 |
| #429 numeric columns stay float64 | 7 tests | 4 of 4 |
| #431 reject unusable get_nearest_continuous inputs | 25 tests | 3 of 4 |
| #395 keep censored peaks | 9 tests | 1 of 1 |
| #432 Virgin Islands naming | 2 tests | 3 of 3 |
Cards: https://github.com/bhaskargurram-ai/dataretrieval-python/pulls
Docstring-only edits are marked separately and not counted. One honest caveat: my first run of #395 used the squash commit message as the description, and a per-commit note in it ("no behavior change") was misread as a claim about the whole PR. The real description says the opposite, so I re-ran it without that text.
If useful, I can open a PR adding it as a comment-only workflow that never fails CI. If not, feel free to close this.
Activity