Skip to content

utils: do not require an IPv4 default route to find the default NIC - #14060

Open
wido wants to merge 1 commit into
apache:mainfrom
wido:netutils-no-ipv4-default-route
Open

utils: do not require an IPv4 default route to find the default NIC#14060
wido wants to merge 1 commit into
apache:mainfrom
wido:netutils-no-ipv4-default-route

Conversation

@wido

@wido wido commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Description

A CloudStack management server does not need an IPv4 default route. It can be IPv6-first, or — as in the setup that triggered this — have IPv4 connectivity to the POD network and the KVM agents without any IPv4 default gateway. Today such a management server does not boot:

ERROR [o.a.c.s.l.CloudStackExtendedLifeCycle] (main:[]) (logid:) Error on configuring bean RootCAProvider -
Cannot invoke "java.net.NetworkInterface.getInterfaceAddresses()" because "nic" is null
java.lang.NullPointerException: Cannot invoke "java.net.NetworkInterface.getInterfaceAddresses()" because "nic" is null
        at com.cloud.utils.net.NetUtils.getAllDefaultNicIps(NetUtils.java:298)
        at org.apache.cloudstack.ca.provider.RootCAProvider.loadManagementKeyStore(RootCAProvider.java:409)
        at org.apache.cloudstack.ca.provider.RootCAProvider.setupCA(RootCAProvider.java:521)
        at org.apache.cloudstack.ca.provider.RootCAProvider.configure(RootCAProvider.java:548)
        ...
ERROR [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:[]) (logid:) Failed to load module [root-ca]

RootCAProvider calls NetUtils.getAllDefaultNicIps() to collect the IPs that go into the management server certificate's SANs. That in turn calls NetUtils.getDefaultEthDevice(), which ran:

ip route show default 0.0.0.0/0 | head -1 | awk '{print $5}'

Two problems with that:

  1. It only looks at IPv4, and it does not always return a device name. Script merges stderr into stdout (ProcessBuilder.redirectErrorStream(true)), so anything the shell or ip writes on stderr becomes the "device name". That non-null, non-device string is handed to NetworkInterface.getByName(), which returns null, and the unchecked dereference on the next line throws the NPE above — which aborts the whole root-ca module and therefore the management server boot.

  2. Column 5 is not the device. $5 only happens to be the device for routes shaped like default via <gw> dev <name> .... For an on-link default route it is wrong:

    route old $5 new
    default via 10.0.0.1 dev eth0 proto static metric 100 eth0 eth0
    default dev eth0 scope link link eth0
    default via fe80::1 dev eno1 proto ra metric 1024 expires 1798sec pref medium eno1 eno1
    default dev eno1 proto kernel metric 256 pref medium kernel eno1
    multipath (default proto static + nexthop ... dev eth0 ...) (empty) eth0

Fix

Small change, all in NetUtils:

  • The device name is taken from the token that follows dev, rather than from a fixed column. Correct for every route layout above, including multipath.
  • ip -4 route show default is queried first; if there is no IPv4 default route, ip -6 route show default is used. An IPv6-first management server now resolves its default NIC and gets its addresses (both families) into the management server certificate, instead of getting nothing.
  • The results of NetworkInterface.getByName() are null-checked in both getAllDefaultNicIps() and getDefaultHostIp(), so an unresolvable device name degrades to "no default NIC found" with a warning in the log instead of killing the boot.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Four unit tests added to NetUtilsTest, covering the IPv4→IPv6 fallback, the IPv4-preferred path, an unresolvable device name, and no default route at all.

The awk expression was checked against the route outputs in the table above.

Note

NetUtilsTest#testAllIpsOfDefaultNic fails on JDK 26 with UnsupportedOperationException from Collections.reverse() in getNetworkParams(), because NetworkInterface.getInterfaceAddresses() now returns an immutable list. That is pre-existing on main and unrelated to this PR.

A management server does not need an IPv4 default route. It can be
IPv6-first, or reach its POD network and KVM agents over IPv4 without
having a default gateway on that family at all.

getDefaultEthDevice() only looked at the IPv4 routing table and picked
column 5 of the route as the device name:

  ip route show default 0.0.0.0/0 | head -1 | awk '{print $5}'

That has two problems:

* Anything the shell or 'ip' writes on stderr ends up in the captured
  output (Script merges stderr into stdout), so the method can return a
  string that is not a device name at all. getAllDefaultNicIps() then
  passes it to NetworkInterface.getByName(), which returns null, and the
  unchecked dereference throws a NullPointerException. On a host without
  an IPv4 default route this aborts the boot of the management server:

    ERROR [o.a.c.s.l.CloudStackExtendedLifeCycle] Error on configuring
    bean RootCAProvider - Cannot invoke
    "java.net.NetworkInterface.getInterfaceAddresses()" because "nic" is
    null
      at com.cloud.utils.net.NetUtils.getAllDefaultNicIps(NetUtils.java:298)
      at o.a.c.ca.provider.RootCAProvider.loadManagementKeyStore(RootCAProvider.java:409)
    ERROR [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] Failed to load
    module [root-ca]

* Column 5 is only the device for routes of the form
  "default via <gw> dev <name> ...". For an on-link default route such as
  "default dev eth0 scope link" or "default dev eno1 proto kernel metric
  256" it returns "link" or "kernel".

The device name is now taken from the token following "dev", which is
correct for every route layout including multipath routes, and the IPv6
routing table is consulted when there is no IPv4 default route. The
results of NetworkInterface.getByName() are null checked in both
getDefaultHostIp() and getAllDefaultNicIps() so an unresolvable device
name degrades to "no default NIC" instead of an exception.
@wido
wido requested a review from weizhouapache September 4, 2026 20:42
@wido wido added this to the 24.0 milestone Sep 4, 2026
@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 81.25000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 19.78%. Comparing base (44ab7dd) to head (766dc99).

Files with missing lines Patch % Lines
...ls/src/main/java/com/cloud/utils/net/NetUtils.java 81.25% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##               main   #14060   +/-   ##
=========================================
  Coverage     19.77%   19.78%           
- Complexity    19989    19997    +8     
=========================================
  Files          6371     6371           
  Lines        575899   575914   +15     
  Branches      70495    70499    +4     
=========================================
+ Hits         113912   113953   +41     
+ Misses       449563   449533   -30     
- Partials      12424    12428    +4     
Flag Coverage Δ
uitests 3.53% <ø> (ø)
unittests 21.06% <81.25%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant