Skip to content

HDDS-16965. Initialize disk balancer state from configured values when diskBalancer.info is absent - #11503

Open
shuan1026 wants to merge 3 commits into
apache:masterfrom
shuan1026:HDDS-16965
Open

shuan1026 wants to merge 3 commits into
apache:masterfrom
shuan1026:HDDS-16965

Conversation

@shuan1026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Bug: On first DiskBalancer init (diskBalancer.info absent), only should.run.default was taken from injected config; threshold, bandwidth, parallel thread, stop-after-disk-even, and container states were seeded via new DiskBalancerConfiguration() and persisted as Java defaults. Later restarts keep the wrong file; XML changes alone do not fix it.

Impact: Operator ozone-site.xml tuning is ignored on new datanodes. Balancing may use wrong I/O limits / stop condition / movable states; diskbalancer status reports the persisted defaults with no warning.

Fix: Seed initial DiskBalancerInfo from conf.getObject(DiskBalancerConfiguration.class). Existing-file load path unchanged.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16965

How was this patch tested?

@Gargi-jais11 Gargi-jais11 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @shuan1026 for finding out this issue.
Please find the inlined comment.

@Gargi-jais11 Gargi-jais11 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, +1!

@slfan1989 slfan1989 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for inviting me to review this PR.

I left one inline comment about an edge case in threshold validation.


@PostConstruct
public void validate() {
if (threshold <= 0d || threshold >= 100d) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NaN bypasses this validation because both comparisons evaluate to false.

Since the configuration parser uses Double.parseDouble(), an XML value of NaN reaches this method and is subsequently persisted to diskBalancer.info.

Once balancing runs, comparisons against the resulting NaN threshold are always false. For example, newUtilization >= upperThreshold cannot reject a move, which may cause containers to be selected without a valid utilization limit.

Could we reject non-finite values here and in setThreshold(), for example:

if (!Double.isFinite(threshold) || threshold <= 0d || threshold >= 100d)

Please also add a test using a raw NaN configuration value.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @slfan1989!
Done, NaN is now rejected in both validate() and setThreshold(), with tests added.

@slfan1989 slfan1989 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the feedback.

The non-finite threshold validation is now shared by configuration initialization and the setter, and the tests cover NaN and infinities.

LGTM, +1!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants