Skip to content

fix(health): stop deduping a red hour into an alert older than the window - #136

Merged
Fl0p merged 1 commit into
mainfrom
flo-996-stale-alert-window
Oct 5, 2026
Merged

Fl0p merged 1 commit into
mainfrom
flo-996-stale-alert-window

Conversation

@Fl0p

@Fl0p Fl0p commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Summary

A red hour no longer dedups into an open alert older than PC_ALERT_MAX_AGE_H (default 6). raise opens a fresh alert, assigned as usual, and does not close, comment on, or otherwise touch the stale one. The Actions log names both, so a second alert is explainable from the run list.

  • Inside the window, the still-red wake is unchanged.
  • PC_ALERT_MAX_AGE_H=0 always dedups, for a drill.
  • A missing or unparseable createdAt is treated as inside the window, so a parse failure cannot mint a duplicate.
  • A green hour still wakes the assignee of whatever alert is open. The window is only the raise decision.
  • The fresh create sets allowDuplicate, because this tracker otherwise returns a recent open issue with the same title. If the response is still the stale alert, the pager fails the call instead of claiming a new one was opened.
  • Age is computed with jq from the API timestamp. Fractional seconds are stripped first; fromdateiso8601 rejects them, and date -d is not portable to the macOS runner.

docs/operations/health-probe.md documents the window and the pager environment, including PC_ALERT_MAX_AGE_H and its default.

No workflow_dispatch drill. The window is script logic with no API behaviour to exercise, and a live drill would need a six-hour-old alert. bash scripts/page-cotel-health_test.sh is the proof (60 passed, 0 failed).

Test plan

  • bash scripts/page-cotel-health_test.sh — passed=60 failed=0
  • PC_ALERT_MAX_AGE_H=06 logs 6h; 00 still dedups; 6h fails the pager before any call
  • CI: probe script tests on the PR

Summary by CodeRabbit

  • Bug Fixes

    • Alerts older than six hours no longer prevent a fresh alert from being created when a health check remains red. Existing alerts are still woken when the check turns green, regardless of age.
    • Alerts with missing or invalid creation times are treated as current. The staleness window can be disabled by setting it to zero.
  • Documentation

    • Updated pager guidance to cover stale alerts, duplicate handling, and their impact on alert creation budgets.

@coderabbitai

coderabbitai Bot commented Oct 5, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0ef909ac-2c25-49e3-b7aa-6bc62b2068aa
📥 Commits

Reviewing files that changed from the base of the PR and between 9cc7c43 and 9c45ae5.

📒 Files selected for processing (3)
  • docs/operations/health-probe.md
  • scripts/page-cotel-health.sh
  • scripts/page-cotel-health_test.sh
 ______________________________________
< My code review levels are over 9000! >
 --------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…ndow

A failed close leaves the open alert in place forever, and the next red hour
folds into it and wakes nobody. On raise, an alert older than
PC_ALERT_MAX_AGE_H (default 6 hours) is not a dedup target: the pager opens a
fresh one, assigned as usual, and does not touch the stale alert. Zero hours
disables the window. A missing or unparseable creation time stays inside it,
so a parse failure cannot mint a duplicate. The still-red wake is unchanged
for an alert inside the window, and a green hour still wakes whatever is open.

The fresh create asks not to collapse into a recent open issue with the same
title. Without that, the tracker would hand back the stale alert and the
assignment would never fire.

Co-Authored-By: Vesper <vesper@agents.flopbut.local>
Co-Authored-By: Grok 4.7 <noreply@x.ai>
@Fl0p
Fl0p force-pushed the flo-996-stale-alert-window branch from 9836b84 to 9c45ae5 Compare October 5, 2026 09:58
@Fl0p
Fl0p merged commit 4703042 into main Oct 5, 2026
7 of 8 checks passed
@Fl0p
Fl0p deleted the flo-996-stale-alert-window branch October 5, 2026 10:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant