Repository navigation
(17e) dataplane: advertise the gateway with LLDP on every port - #1923
daniel-noland wants to merge 1 commit into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
f0fe306 to
8fdf188
Compare
e4343df to
ac920e5
Compare
8fdf188 to
f46855f
Compare
b9a3a44 to
b31d0c0
Compare
f46855f to
07ff3b5
Compare
b31d0c0 to
fec09dd
Compare
07ff3b5 to
14cf0c0
Compare
fec09dd to
f12f51e
Compare
14cf0c0 to
90235a7
Compare
Run the LLDP transmitter on every port the control-plane bridge holds. What each port advertises: - Chassis ID: locally assigned, the gateway's name from its k8s config. Nothing is advertised until a config has named the gateway: a chassis ID must not change under a listening neighbour, and the k8s name is the one a person recognises. - Port ID: the interface name the port was configured with. - Port Description: the device behind the port, bus address and driver (DPDK device name and driver; sysfs for the kernel driver). - System Name, System Description (dataplane and version), System Capabilities (router, enabled), 802.3 Maximum Frame Size (MTU plus Ethernet header and FCS). - Management Address: the host's default-route source address, looked up on the host runtime, since only the host namespace sees it. Plumbing: - The bridge keeps a per-port handle for other control-plane services: a clone of the tap's injection sender, the port's link state, and the identity the datapath reported (which `PortIdentity` now extends with the hardware description). - Both drivers publish link state from their supervisors' one-second tick: DPDK from `rte_eth_link_get_nowait`, the kernel driver from sysfs carrier. DPDK previously read the link once, at bring-up. - mgmt publishes the gateway's name after each successful apply. Without the bridge (the datapath sharing the control namespace) there is no way to send on the ports, so LLDP is off and says so. The shutdown frame on process stop is best-effort: workers see cancellation at the same moment and may stop draining before it goes out. An n-vm test drives the bridge, agent and transmitter against real taps: a port reported up under a named gateway produces LLDP frames carrying both, and stopping produces a shutdown frame. The bridge holds each port's identity until shutdown, not until the datapath drops its end: the kernel driver reports each port once and drops its end as soon as it has started, and LLDP stops advertising on a port whose identity closes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Daniel Noland <daniel@githedgehog.com>
90235a7 to
a0b50d9
Compare
mvachhar
left a comment
There was a problem hiding this comment.
Note
AI-assisted review: This review was drafted with Claude and reviewed by @mvachhar.
The wiring looks right: the management address is looked up from the host namespace, and nothing new runs on the packet workers beyond one injected frame per port per interval. Requesting changes for one thing: LLDP should start as soon as the dataplane does, rather than waiting for a config, so it's there to help debug a gateway whose config never arrives (inline).
| management: Option<IpAddr>, | ||
| ) -> Option<Advertisement> { | ||
| let identity = identity?; | ||
| let gateway_name = gateway_name?; |
There was a problem hiding this comment.
Note
AI-assisted review: This comment was drafted with Claude and reviewed by @mvachhar.
No LLDP until a config arrives means no LLDP exactly when we'd want it most. If the config can't be fetched (k8s unreachable, wrong name, config error), the gateway never advertises, and nothing on the switch side shows that a gateway is even cabled up. LLDP should start as early as possible so it can help with debugging.
The name doesn't actually have to wait for the config: init_name() in runtime.rs already settles it at startup (--name, else the hostname), and it's the name the dataplane uses to fetch its own GatewayAgent config (MgmtParams.hostname). In the fabric the pod runs with hostNetwork and is pinned with kubernetes.io/hostname = gw.Name, so that hostname is the gateway name. Could the agent:
- advertise from startup with the chassis ID taken from
init_name(), which is stable across restarts; - use the config's name for System Name once it arrives (System Name can change freely);
- say
(awaiting configuration)in System Description until a config is applied, soshow lldp neighbors detailon the switch tells you the gateway is up but unconfigured?
If the config's name ever differs from the startup name, #1922's transmitter already handles that cleanly (shutdown frame for the old chassis ID, reinitDelay, then the new one), so there's no risk of a stale neighbour.
Native LLDP for the gateway. The dataplane owns its NICs, so nothing else can speak LLDP for them, and
lldpd/lldpadare hostile to automation. LLDP itself is small, so the dataplane transmits and receives it.This used to be one PR. It is now a stack of seven, each building on the one before:
net: TLV framing and the mandatory TLVsnet: the optional TLVsnet: the LLDPDU andHeaders::parselldp: the transmitter and the neighbour tabledataplane: advertise on every port ← this PRdataplane: hear neighbours,show lldp neighbors, all-multicastStacked on (16) #1905, capture. (18) #1907, rate limiting, and (19) #1919, traffic protection, build on top.
This PR: advertise the gateway on every port
What a port advertises
0000:03:00.0 (mlx5_pci)Nothing is advertised until a config has named the gateway. A chassis ID must not change while neighbours are listening, and the k8s name is the one a person recognises.
Wiring
rte_eth_link_get_nowaitfor DPDK and sysfs carrier for the kernel driver. DPDK used to read link state only once, at bring-up.Testing
show lldp neighborsshows each gateway with the TLVs above.🤖 Generated with Claude Code