Repository navigation
HDDS-16965. Initialize disk balancer state from configured values when diskBalancer.info is absent - #11503
HDDS-16965. Initialize disk balancer state from configured values when diskBalancer.info is absent#11503shuan1026 wants to merge 3 commits into
Conversation
Gargi-jais11
left a comment
There was a problem hiding this comment.
Thanks @shuan1026 for finding out this issue.
Please find the inlined comment.
slfan1989
left a comment
There was a problem hiding this comment.
Thanks for inviting me to review this PR.
I left one inline comment about an edge case in threshold validation.
|
|
||
| @PostConstruct | ||
| public void validate() { | ||
| if (threshold <= 0d || threshold >= 100d) { |
There was a problem hiding this comment.
NaN bypasses this validation because both comparisons evaluate to false.
Since the configuration parser uses Double.parseDouble(), an XML value of NaN reaches this method and is subsequently persisted to diskBalancer.info.
Once balancing runs, comparisons against the resulting NaN threshold are always false. For example, newUtilization >= upperThreshold cannot reject a move, which may cause containers to be selected without a valid utilization limit.
Could we reject non-finite values here and in setThreshold(), for example:
if (!Double.isFinite(threshold) || threshold <= 0d || threshold >= 100d)
Please also add a test using a raw NaN configuration value.
There was a problem hiding this comment.
Thanks for the review @slfan1989!
Done, NaN is now rejected in both validate() and setThreshold(), with tests added.
slfan1989
left a comment
There was a problem hiding this comment.
Thanks for addressing the feedback.
The non-finite threshold validation is now shared by configuration initialization and the setter, and the tests cover NaN and infinities.
LGTM, +1!
What changes were proposed in this pull request?
Bug: On first DiskBalancer init (
diskBalancer.infoabsent), onlyshould.run.defaultwas taken from injected config; threshold, bandwidth, parallel thread, stop-after-disk-even, and container states were seeded vianew DiskBalancerConfiguration()and persisted as Java defaults. Later restarts keep the wrong file; XML changes alone do not fix it.Impact: Operator
ozone-site.xmltuning is ignored on new datanodes. Balancing may use wrong I/O limits / stop condition / movable states;diskbalancer statusreports the persisted defaults with no warning.Fix: Seed initial
DiskBalancerInfofromconf.getObject(DiskBalancerConfiguration.class). Existing-file load path unchanged.What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16965
How was this patch tested?
TestDiskBalancerService#testInitialDiskBalancerInfoUsesConfiguredValues: empty info dir + non-default config viasetFromObject; asserts in-memory info and written YAML match configured values.