[for minor release] RSS configurable thresholds by fstagni · Pull Request #8520 · DIRACGrid/DIRAC

fstagni · 2026-04-20T12:56:50Z

closes #8517

test in certification

BEGINRELEASENOTES

*RSS
CHANGE: Added configurable thresholds and fractions for FreeDiskSpace policy evaluation

*Deployment

CHANGE: for users of advanced RSS policies: check for adjusting DT and FreeDiskSpace policies

ENDRELEASENOTES

aldbr · 2026-06-11T08:11:15Z

ddev: PR is ready but changes the CS, so we need to make sure who is relying on that to see how smooth the integration could be. @fstagni will check with the known installations

ready to be reviewed

iueda · 2026-06-22T09:21:25Z

I have put a comment in #8517 (comment)

It would be nice if the thresholds can be set by absolute free space (eg. 0.1 TB) and by fraction (free/total, eg. 5%)

I was trying to compare this PR and our code, but decided to comment now, before spending too much time and coming too late.

fstagni · 2026-07-06T15:38:20Z

I have put a comment in #8517 (comment)

It would be nice if the thresholds can be set by absolute free space (eg. 0.1 TB) and by fraction (free/total, eg. 5%)

I was trying to compare this PR and our code, but decided to comment now, before spending too much time and coming too late.

@iueda I have added the "fraction", as IUUC should be in the BelleDIRAC implementation. Please review.

iueda

The code looks good (better than I was preparing).
Only minor suggestions on the comments and the tests.

There may be some remnants of my previous comments, which were in preparation and now obsolete (I tried to delete them, but somehow appears in the list of pending comments...)

…ully configurable

- AlwaysBannedForSite2: tests domain-based matching specificity - AlwaysDegradedForResource: tests AlwaysDegraded policy type - SpecificFreeDiskSpace: tests per-policy arg overrides (Unit=GB, custom thresholds) and specificity dedup against generic FreeDiskSpace - PropagationForSite: tests Propagation policy type - PolicyActions sections: LogStatusAction and LogPolicyResultAction - Integration test cases for LogSE WriteAccess (Degraded via specific FreeDiskSpace), LogSE ReadAccess (Active via AlwaysActive), and Site with None status

…es and generic specificity deduplication - Configurations.py: POLICIESMETA now holds plain code-level defaults only; Operations() calls removed (CS is read at runtime by InfoGetter instead) - InfoGetter.getPoliciesThatApply: CS entries without policyType are skipped (they are command-args defaults sections, not policy definitions); non-reserved keys in a CS policy entry are collected as arg overrides, merged into the policy args with case-insensitive key normalisation and type casting - InfoGetter.postProcessingPolicyList: replaces the old FreeDiskSpaceMB/GB/TB hack with a generic rule — when multiple policies of the same policyType match, keep the most specific one (name-match presence > number of matchParams keys) - dirac.cfg: add SpecificFreeDiskSpace example (Unit=GB, Banned_threshold=15, Degraded_threshold falls back to default); fix missing closing brace in Policies - docs: note that command-args sections have no policyType and are not policy definitions - test: new Test_InfoGetter.py with 15 unit tests covering all the above

…d defaults FreeDiskSpaceCommand and DowntimeCommand no longer duplicate default values in .get() fallbacks — defaults live exclusively in POLICIESMETA and are guaranteed to be present in self.args by the time the command runs.

- StateMachine.setState: guard against self.state being None before accessing self.states[self.state], preventing KeyError when the machine transitions from a valid state to None and back. - DowntimeCommand._prepareCommand: change key-existence checks to value checks (if not self.args.get("name")) so that None values from decisionParams defaults are also caught. - Add unit tests for both fixes and a new StateMachine test suite.

- AlwaysBannedForSite2: tests domain-based matching specificity - SpecificFreeDiskSpace: tests per-policy arg overrides (Unit=GB, custom thresholds) and specificity dedup against generic FreeDiskSpace - PropagationForSite: tests Propagation policy type - PolicyActions sections: LogStatusAction and LogPolicyResultAction - Integration test case for LogSE WriteAccess (Error via FreeDiskSpace command when no occupancy data exists in test DB)

Add Banned_fraction and Degraded_fraction config options to the FreeDiskSpace policy so SEs can be flagged based on the fraction of total free space in addition to absolute thresholds. - FreeDiskSpaceCommand: pass fractions through _prepareCommand, doNew, doCache - FreeDiskSpacePolicy: evaluate fraction thresholds alongside absolute ones - Configurations.py: add default fraction values (1% Banned, 5% Degraded) - Tests: update existing tests and add new fraction threshold test cases - Docs: update admin and developer guides with new config options

…WMSAdministrator methods PropagationCommand now returns Unknown status instead of S_ERROR when site elements are missing or DB queries fail, preventing the Error status from overriding other policies in PDP decisions. Removed 9 deprecated no-op RPC methods from WMSAdministratorHandler (setSiteMask, getSiteMask, getSiteMaskStatus, getAllSiteMaskStatus, banSite, allowSite, clearMask, getSiteMaskLogging, getSiteMaskSummary) that only returned S_OK().

DowntimeCommand now returns S_OK(None) or S_OK() instead of S_ERROR when GOCDB queries fail, the downtime cache is unavailable, or command preparation fails (e.g. missing elementType). This prevents the Error status from overriding other policies in PDP decisions in test environments without GOCDB access.

DowntimeCommand.doNew() now returns S_OK(None) instead of S_ERROR when _prepareCommand() fails, matching the integration test fix.

fstagni requested review from andresailer and atsareg as code owners April 20, 2026 12:56

chrisburr marked this pull request as draft April 20, 2026 13:35

fstagni force-pushed the rss_configurable_thresholds branch 2 times, most recently from 67caea8 to 5689ef1 Compare April 20, 2026 15:48

fstagni marked this pull request as ready for review April 20, 2026 15:49

fstagni force-pushed the rss_configurable_thresholds branch from 5689ef1 to 603dbf3 Compare April 20, 2026 16:10

fstagni force-pushed the rss_configurable_thresholds branch 5 times, most recently from f45acb7 to 17e4b0e Compare June 9, 2026 15:40

iueda mentioned this pull request Jun 23, 2026

[Feature]: Make RSS FreeDiskSpacePolicy actually configurable #8517

Open

1 task

fstagni force-pushed the rss_configurable_thresholds branch from 17e4b0e to 28a1265 Compare July 6, 2026 08:30

iueda approved these changes Sep 8, 2026

View reviewed changes

iueda reviewed Sep 8, 2026

View reviewed changes

Comment thread src/DIRAC/ResourceStatusSystem/Policy/test/Test_RSS_Policy_FreeDiskSpacePolicy.py Outdated

Comment thread src/DIRAC/ResourceStatusSystem/Policy/test/Test_RSS_Policy_FreeDiskSpacePolicy.py Outdated

fstagni force-pushed the rss_configurable_thresholds branch 2 times, most recently from 0998326 to fd47b7f Compare September 8, 2026 09:29

fstagni added 4 commits September 24, 2026 14:34

feat: the thresholds and unit used for the FreeDiskSpace policy are f…

2cace5d

…ully configurable

feat: the hours for the DowntimePolicy are now configurable

a99fca7

fstagni force-pushed the rss_configurable_thresholds branch from fd47b7f to 0a20133 Compare September 24, 2026 12:35

fstagni added 4 commits September 24, 2026 17:01

docs: better explanation of multiple policies configuration

7e0f11a

fstagni added 2 commits September 24, 2026 17:01

chore: removed outdated tests

4b6ec7f

fstagni force-pushed the rss_configurable_thresholds branch 2 times, most recently from f0482d2 to fadf015 Compare September 24, 2026 15:51

feat(rss): added default matchParams

672252d

fstagni force-pushed the rss_configurable_thresholds branch from fadf015 to 672252d Compare September 25, 2026 07:33

fstagni added 2 commits September 25, 2026 11:05

fstagni force-pushed the rss_configurable_thresholds branch from 40acb5a to aac5874 Compare September 25, 2026 10:39

test: update DowntimeCommand tests to expect S_OK on preparation failure

ed1f231

DowntimeCommand.doNew() now returns S_OK(None) instead of S_ERROR when _prepareCommand() fails, matching the integration test fix.

fstagni force-pushed the rss_configurable_thresholds branch from aac5874 to ed1f231 Compare September 25, 2026 11:52

Navigation Menu

[for minor release] RSS configurable thresholds - #8520

[for minor release] RSS configurable thresholds#8520
fstagni wants to merge 14 commits into
DIRACGrid:integrationfrom
fstagni:rss_configurable_thresholds

fstagni commented Apr 20, 2026 •

edited

Loading

Uh oh!

aldbr commented Jun 11, 2026 •

edited

Loading

Uh oh!

iueda commented Jun 22, 2026

Uh oh!

fstagni commented Jul 6, 2026

Uh oh!

iueda left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

fstagni commented Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aldbr commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

iueda commented Jun 22, 2026

Uh oh!

fstagni commented Jul 6, 2026

Uh oh!

iueda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

fstagni commented Apr 20, 2026 •

edited

Loading

aldbr commented Jun 11, 2026 •

edited

Loading