Skip to content

(17e) dataplane: advertise the gateway with LLDP on every port - #1923

Open
daniel-noland wants to merge 1 commit into
pr/daniel-noland/lldp-cratefrom
pr/daniel-noland/lldp-advertise
Open

daniel-noland wants to merge 1 commit into
pr/daniel-noland/lldp-cratefrom
pr/daniel-noland/lldp-advertise

Conversation

@daniel-noland

@daniel-noland daniel-noland commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Native LLDP for the gateway. The dataplane owns its NICs, so nothing else can speak LLDP for them, and lldpd/lldpad are hostile to automation. LLDP itself is small, so the dataplane transmits and receives it.

This used to be one PR. It is now a stack of seven, each building on the one before:

Stacked on (16) #1905, capture. (18) #1907, rate limiting, and (19) #1919, traffic protection, build on top.

This PR: advertise the gateway on every port

What a port advertises

TLV Value
Chassis ID locally assigned: the gateway's name from its k8s config
Port ID the interface name from the command line
Port Description the device: bus address and driver, e.g. 0000:03:00.0 (mlx5_pci)
System Name the gateway's name
System Description dataplane and version
System Capabilities router, enabled
Management Address the host's default-route source address (host namespace)
802.3 Max Frame Size MTU plus Ethernet header and FCS

Nothing is advertised until a config has named the gateway. A chassis ID must not change while neighbours are listening, and the k8s name is the one a person recognises.

Wiring

  • Per-port handles: the control-plane bridge gives each port an injection handle, a link-state watch and an identity watch.
  • Identities are held until shutdown, not just until the datapath drops its end. The kernel driver reports each port once and then drops its end, and LLDP stops advertising on a port whose identity closes. vlab found this one.
  • Link state is polled once a second from the driver supervisors: rte_eth_link_get_nowait for DPDK and sysfs carrier for the kernel driver. DPDK used to read link state only once, at bring-up.
  • Inputs from elsewhere: mgmt publishes the gateway's name, and the host runtime finds the management address.

Testing

  • n-vm, against real taps: the bridge, agent and transmitter together produce LLDP frames naming the gateway and the port. A reported identity outlives the datapath's end.
  • vlab (kernel driver, two gateways): the spines' show lldp neighbors shows each gateway with the TLVs above.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: f47bbdd4-b9ed-4d63-a8e4-a3c1974db28f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/lldp-advertise branch from f0fe306 to 8fdf188 Compare October 16, 2026 05:47
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/lldp-crate branch from e4343df to ac920e5 Compare October 16, 2026 05:47
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/lldp-advertise branch from 8fdf188 to f46855f Compare October 16, 2026 08:43
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/lldp-crate branch 2 times, most recently from b9a3a44 to b31d0c0 Compare October 17, 2026 03:26
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/lldp-advertise branch from f46855f to 07ff3b5 Compare October 17, 2026 03:26
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/lldp-crate branch from b31d0c0 to fec09dd Compare October 17, 2026 05:59
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/lldp-advertise branch from 07ff3b5 to 14cf0c0 Compare October 17, 2026 05:59
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/lldp-crate branch from fec09dd to f12f51e Compare October 17, 2026 06:17
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/lldp-advertise branch from 14cf0c0 to 90235a7 Compare October 17, 2026 06:17
Run the LLDP transmitter on every port the control-plane bridge holds.

What each port advertises:

- Chassis ID: locally assigned, the gateway's name from its k8s config.
  Nothing is advertised until a config has named the gateway: a chassis
  ID must not change under a listening neighbour, and the k8s name is
  the one a person recognises.
- Port ID: the interface name the port was configured with.
- Port Description: the device behind the port, bus address and driver
  (DPDK device name and driver; sysfs for the kernel driver).
- System Name, System Description (dataplane and version), System
  Capabilities (router, enabled), 802.3 Maximum Frame Size (MTU plus
  Ethernet header and FCS).
- Management Address: the host's default-route source address, looked
  up on the host runtime, since only the host namespace sees it.

Plumbing:

- The bridge keeps a per-port handle for other control-plane services:
  a clone of the tap's injection sender, the port's link state, and the
  identity the datapath reported (which `PortIdentity` now extends with
  the hardware description).
- Both drivers publish link state from their supervisors' one-second
  tick: DPDK from `rte_eth_link_get_nowait`, the kernel driver from
  sysfs carrier.  DPDK previously read the link once, at bring-up.
- mgmt publishes the gateway's name after each successful apply.

Without the bridge (the datapath sharing the control namespace) there is
no way to send on the ports, so LLDP is off and says so.  The shutdown
frame on process stop is best-effort: workers see cancellation at the
same moment and may stop draining before it goes out.

An n-vm test drives the bridge, agent and transmitter against real taps:
a port reported up under a named gateway produces LLDP frames carrying
both, and stopping produces a shutdown frame.

The bridge holds each port's identity until shutdown, not until the
datapath drops its end: the kernel driver reports each port once and
drops its end as soon as it has started, and LLDP stops advertising on a
port whose identity closes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Daniel Noland <daniel@githedgehog.com>
@daniel-noland
daniel-noland force-pushed the pr/daniel-noland/lldp-advertise branch from 90235a7 to a0b50d9 Compare October 17, 2026 22:06
@daniel-noland
daniel-noland marked this pull request as ready for review October 18, 2026 18:03
@daniel-noland
daniel-noland requested a review from a team as a code owner October 18, 2026 18:03
@daniel-noland
daniel-noland requested review from sergeymatov and removed request for a team October 18, 2026 18:03

@mvachhar mvachhar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

AI-assisted review: This review was drafted with Claude and reviewed by @mvachhar.

The wiring looks right: the management address is looked up from the host namespace, and nothing new runs on the packet workers beyond one injected frame per port per interval. Requesting changes for one thing: LLDP should start as soon as the dataplane does, rather than waiting for a config, so it's there to help debug a gateway whose config never arrives (inline).

management: Option<IpAddr>,
) -> Option<Advertisement> {
let identity = identity?;
let gateway_name = gateway_name?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

AI-assisted review: This comment was drafted with Claude and reviewed by @mvachhar.

No LLDP until a config arrives means no LLDP exactly when we'd want it most. If the config can't be fetched (k8s unreachable, wrong name, config error), the gateway never advertises, and nothing on the switch side shows that a gateway is even cabled up. LLDP should start as early as possible so it can help with debugging.

The name doesn't actually have to wait for the config: init_name() in runtime.rs already settles it at startup (--name, else the hostname), and it's the name the dataplane uses to fetch its own GatewayAgent config (MgmtParams.hostname). In the fabric the pod runs with hostNetwork and is pinned with kubernetes.io/hostname = gw.Name, so that hostname is the gateway name. Could the agent:

  • advertise from startup with the chassis ID taken from init_name(), which is stable across restarts;
  • use the config's name for System Name once it arrives (System Name can change freely);
  • say (awaiting configuration) in System Description until a config is applied, so show lldp neighbors detail on the switch tells you the gateway is up but unconfigured?

If the config's name ever differs from the startup name, #1922's transmitter already handles that cleanly (shutdown frame for the old chassis ID, reinitDelay, then the new one), so there's no risk of a stale neighbour.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants