(19) dataplane: a dedicated control queue and worker for BGP, BFD, ARP, ND and LLDP by daniel-noland · Pull Request #1919 · githedgehog/dataplane

daniel-noland · 2026-10-16T02:52:55Z

Stacked on (18) #1907.

BGP, BFD, ARP, IPv6 neighbour discovery and LLDP now keep working while the data plane is saturated. The NIC steers them to a receive queue of their own, which a dedicated control worker polls without the pipeline. On env-5, BGP flapped under load. On a BlueField-3, a 39 Mpps flood made the current dataplane lose about 73% of BGP and BFD probes; with this branch it loses none.

Why

Today every control frame is ordinary traffic until the end of the pipeline:

RSS spreads it over the data queues, where a flood crowds it out (imissed).
A saturated data worker gets to it late, or never.
The whole VPC pipeline runs before the punt decision.
It shares one bounded punt channel with everything else that is punted.
Keepalives leave from worker 0, allocated from the receive pool.

Design

Classes:
- BGP (TCP 179) and BFD (UDP 3784/4784), from configured neighbours to the gateway's addresses
- ICMPv6 to the gateway, link-local or multicast addresses (neighbour discovery included)
- ARP
- LLDP to the nearest-bridge group address
Everything else, including ping and traceroute to the gateway, stays on the data path and is punted as before. The gateway's addresses are underlay addresses, not reachable from VPCs, and keeping the class set narrow means nothing but these protocols can fill the control queue.
A control queue. Each port gets one RX/TX queue pair beyond the data workers'. The RSS redirection table covers only the data queues. Ingress rte_flow rules with a queue action steer the classes to the control queue.
A control worker busy-polls every port's control queue on its own CPU (the last allowed one; data workers get the rest). It hands frames addressed to the port straight to the tap. It also takes over injection, from its own TX queue and mempool, so keepalives no longer depend on worker 0 or the receive pool.
A priority punt channel. The tap's pump drains, in order: the kernel's outgoing frames, then the control queue's punts, then the data workers' punts.
Software fallback. On a port without a control queue (RSS without a redirection table), or for a class whose rule the NIC refuses, data workers classify each frame right after parsing and punt matches on the priority channel, skipping the pipeline.
One class table, two forms. traffic-control builds both from the config's local addresses and neighbours:
- a software classifier;
- the rules for the NIC.
A property test checks they agree, and fails on deliberately broken rules.

Commits

dpdk: EthMatch (destination/source MAC, ethertype) for match_eth.
dpdk: RSS redirection table control: spread_rss, rss_reta.
traffic-control: the class table (ControlPlane → ClassTable), with classify(Key) and rules(), and the differential test.
mgmt: the addresses and peers the classes need, derived from each applied config and published on a watch channel. Addresses come from underlay interfaces, BGP update sources, BFD sources and the VTEP. Peers are BGP neighbours and BFD peers.
dpdk: RETA follow-up. Tables smaller than 64 entries are handled: mlx5 reports W+1 entries when that is a power of two. When the reported size would split unevenly, a 512-entry table is tried first, so one worker doesn't get double load.
traffic-control: no "data" fate, which would have needed an RSS action repeating the port's hash types. Superseded by 12.
dataplane: ports get W+1 queues and a control pool, and RSS is restricted to the data queues. The control worker polls the control queues and owns injection.
dataplane: the control worker installs the class rules and follows config changes, keeping unchanged rules, so there is no window during a change.
dataplane: the priority punt channel.
dataplane: CPU placement: the control worker on its own CPU.
dataplane: the software early-punt fallback.
traffic-control: narrows the classes to the control protocols. The "other local" catch-all is dropped, and with it the priority tiers and the VTEP carve-out.

Tests

Unit and property tests pass for dpdk, traffic-control, mgmt (23) and dataplane (90, in n-vm). Workspace clippy is clean.

BlueField-3:

Setup: port 0 ran the gateway image, which includes FRR (dataplane-init --supervise-frr, 2 data workers). On port 1, the kernel netdev probed every 10 ms with TCP 179 from the configured neighbour, UDP 3784 and ping. testpmd, in flow-isolate mode on the same port, sent the floods.
Steering: mlx5 accepted every class rule. At idle the control worker received 617 of 623 frames and the data workers 6.

Scenario	Base (#1907)	This branch
Idle	0% loss	0% loss
39 Mpps 64 B flood to an unrouted address (both workers saturated)	BGP 73%, BFD 71%, ping 75% loss	BGP/BFD 0% (p50 0.5 ms); ping 75%, as base
UDP flood to the gateway's own address	~99–100% loss	BGP/BFD 0% (p50 0.5 ms); ping 100%

Two notes on the BFD numbers:

The probe counts the gateway kernel's ICMP port unreachable as the answer. Under the own-address flood those replies drown among the flood's own unreachables, so that row was measured with a UDP echo on 3784 standing in for bfdd. Real BFD has bfdd listening.
The base column for the own-address flood was measured before narrowing the classes (12). At that point the branch also lost about 98% there.

Not yet done

Per-class policing and counters, and show control-plane. They matter only if a neighbour floods the control classes themselves.
Hardware policing (METER_MARK) and flow isolation.
The kernel driver: ethtool ntuple steering, IRQ affinity, a control AF_PACKET socket.
Topology-aware placement of the data workers.

🤖 Generated with Claude Code

coderabbitai · 2026-10-16T02:53:05Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI
Review profile: CHILL
Plan: Essentials
Run ID: 559273d1-cfd9-4a9d-869b-23122e2718e8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

codecov · 2026-10-16T03:10:17Z

Codecov Report

❌ Patch coverage is 51.91105% with 692 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
dataplane/src/drivers/dpdk/control.rs	4.73%	161 Missing ⚠️
dataplane/src/drivers/dpdk/port.rs	0.00%	146 Missing ⚠️
dataplane/src/drivers/dpdk/classes.rs	12.29%	107 Missing ⚠️
dataplane/src/drivers/dpdk/mod.rs	0.00%	56 Missing ⚠️
dataplane/src/drivers/cpbridge.rs	0.00%	52 Missing ⚠️
dataplane/src/drivers/dpdk/worker.rs	26.78%	41 Missing ⚠️
dpdk/src/dev.rs	0.00%	37 Missing ⚠️
dataplane/src/drivers/dpdk/placement.rs	54.23%	27 Missing ⚠️
mgmt/src/processor/proc.rs	0.00%	15 Missing ⚠️
traffic-control/src/table.rs	76.27%	14 Missing ⚠️
... and 7 more

📢 Thoughts on this report? Let us know!

match_eth only matched the presence of an Ethernet header. It now takes an EthMatch: destination MAC, source MAC and ethertype, each optional, each lowered to the spec with a full mask when set and left out of the mask when not. Steering ARP (ethertype 0x0806) and LLDP (01:80:c2:00:00:0e, 0x88cc) to a control queue needs both. A property test checks that the lowered spec and mask match exactly the frames the criteria describe. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

DevInfo::reta_size reports the table's size; Dev::spread_rss lays it out round-robin over a given set of receive queues and Dev::rss_reta reads it back. Spreading RSS over the data queues only leaves a queue that receives nothing but what a flow rule steers to it, e.g. control-plane traffic a data flood cannot crowd out. The layout is a pure function: a property test checks that every entry names a given queue, every given queue is named, and shares differ by at most one entry per listing. Tables of no or ragged size and empty queue lists are refused before the driver sees them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Control traffic must keep flowing while the data plane is overloaded, which means taking it off the data path as early as possible: onto a receive queue of its own where the NIC can steer it, first thing after receive where it cannot. This crate decides what control traffic is. A ClassTable is built from the gateway's local addresses, its VTEP and its BGP/BFD neighbours (from configuration, not the FIB, so VXLAN to the VTEP is not mistaken for control traffic). Its classes: - BFD: UDP 3784 and 4784 from a neighbour to a local address - BGP: TCP port 179 from a neighbour to a local address - ICMPv6 to a local, link-local or multicast address (neighbour discovery among it) - ARP, by ethertype - LLDP to the nearest bridge group address - other local: anything else to a local address, VXLAN to the VTEP excepted. BGP and BFD from strangers and BFD echo land here. The table answers in two forms: classify() sorts a frame's Key into a class, and rules() is the same decision as two tiers of prioritized match rules, the shape NIC flow rules take, for the driver to lower. The two are written independently; a property test checks that the rules steer exactly what the classifier classifies, and that rules of one tier never disagree about a frame. Removing the VXLAN exception or changing a port in the rules makes it fail. A Key holds only the outermost headers, as a NIC sees them: a tagged frame is its tag's ethertype, and ports behind IPv6 extension headers or in non-first fragments are unknown. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The control-plane traffic classes are built from the gateway's local addresses, its VTEP and its BGP and BFD neighbours. The config processor now derives them from each config it applies (or rolls back to) and publishes them on a watch channel when they change, as it does the gateway's name. Local addresses are the underlay interfaces' addresses, the BGP update sources and BFD sources given as addresses, and the VTEP's. Neighbours are the BGP neighbours given by address and the BFD peers. Neighbours a listen range admits are not known in advance; their sessions count as other local traffic. The runtime holds the receiving end until the driver installs the classes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mlx5 reports a redirection table as small as the number of receive queues when that is a power of two: 2, 4, 8 entries. The layout refused any size that was not whole 64-entry groups, so spreading RSS failed on exactly the NIC it is for. Tables of any size are now laid out, the last group partial and masked. A table that small also splits unevenly: 7 data queues out of 8 share 8 entries and one queue gets twice the traffic. When the reported size would split unevenly, spread_rss first asks for the largest size DPDK defines (512), which mlx5 resizes to and other drivers refuse, and falls back to the reported size. It returns the size it used, which rss_reta now takes, since the device info cached at open no longer says. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…trol VXLAN to the VTEP was carved out of the catch-all for local traffic by a higher tier rule whose fate was "data". On a NIC that is an RSS action, which would have to repeat the port's hash types exactly, or VXLAN from one peer VTEP (whose entropy is in the outer UDP source port) could land on one worker. Instead the catch-all for the VTEP's address only takes ICMP and TCP; UDP to it (VXLAN, or nothing the gateway expects) matches no rule and keeps the port's own RSS. Every rule now steers to the control queue, so a rule carries a class rather than a fate. BFD and BGP to the VTEP's address are matched by their own rules as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Every DPDK port gets one more queue pair than there are data workers, for a control worker, and a small pool of its own (2047 mbufs, 512 descriptors each way). RSS is spread over the data queues only, so the control receive queue gets nothing but what flow rules steer to it and a data flood cannot fill it. A port that does RSS without a redirection table gets no control receive queue (and warns); a port whose table cannot be restricted is refused, since the control worker has no pipeline to forward data with. The control worker polls every port's control queue and hands what it receives, if addressed to the port, straight to the port's tap, without the pipeline. It also takes over injection from data worker 0: frames from the kernel (BGP keepalives, BFD, ARP and ND replies, LLDP) leave on the control transmit queue, from the control pool, whatever the data workers are doing and whether or not the receive pool is exhausted. It is supervised like the data workers, numbered after them. Nothing steers traffic to the control queue yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The control worker installs the class table's rules on every port with a control receive queue: ingress rte_flow rules matching each class, with a queue action to the control queue, at priority 0 for specific protocols and 1 for the catch-all to local addresses. IP rules name the IP protocol explicitly, so the NIC, like the software classifier, reads the IP header's own protocol field. It follows the control plane mgmt publishes: when the addresses or peers change, rules the new table keeps stay, new ones are created and dropped ones destroyed, so there is no window in which control traffic takes the data path. A rule the NIC refuses is logged and tried again on the next change; its traffic stays on the data path, punted by the pipeline as before. Rules are created and destroyed on the control worker's thread and gone before its queues are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Everything punted to a port's tap shared one bounded channel, so the pings, ICMP errors and other local traffic the data workers punt could fill it ahead of BFD. Each port's bridge now has a second punt channel, which the control worker uses for what its control queue receives. The tap's pump takes, in order: the kernel's own frames for the wire (keepalives and replies, sent at the rate the control plane needs and no faster), then the control channel, then the data workers' punts. It stops once both punt channels have closed. LLDP is still handed to the LLDP agent whichever channel it came on. The ARP round trip test now punts on the control channel; the LLDP test still punts as a data worker does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The control worker is pinned to the last CPU the process may use, and the data workers to the others, so a saturated data path cannot take the control worker's CPU. With a single CPU the data workers are left where they are and share it. If the process's CPUs cannot be read or a thread cannot be pinned, it is logged and the thread floats as before. Placement is a pure function of the allowed CPUs, with a property test. Topology-aware placement of the data workers (hwlocality) is still to come; tokio's threads still float over every CPU. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…software Where the NIC does not steer a class of control traffic to the control queue -- a port without a control receive queue, or a class whose rule the NIC refused -- the data workers now classify each parsed frame against the class table before the pipeline, and punt matches addressed to the port on the control worker's priority channel. That spares the pipeline's cost and verdicts (a rate limit or ACL can no longer drop BFD) and the shared punt channel, though not the shared receive queue. The control worker publishes, per port, which classes are left to software: all of them without a control queue, the refused ones otherwise, nothing when the NIC steers them all. Data workers read it once per burst and skip the check entirely when there is nothing to take. The per-poll buffers a data worker reuses are now one struct. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Drop the "other local" class. It took anything addressed to the gateway onto the control queue, which made the queue fillable by any traffic to the gateway's addresses: on the BF-3, a flood to the gateway's own address took BGP and BFD down with it. The gateway hosts no services and its addresses live in the underlay, so ping and traceroute to it are not worth that; they stay on the data path and are punted from there, as before. What remains are protocols the NIC can recognise exactly: BGP and BFD from configured neighbours to local addresses, ARP, ICMPv6 to us, and LLDP. No two rules then match a frame with different classes, so the priority tiers, the VTEP carve-out (and with it the VTEP in ControlPlane, whose address is simply local), and the catch-all L4Pattern::Any go too. The differential test now asserts that rules never disagree at all. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The data workers hold an armed exit guard, so a panic or unexpected return before shutdown is reported as fatal. The control worker had none. Losing it leaves forwarding running while BGP and BFD go silent, and the supervisor logs the end without failing the process. Give it the same guard. Signed-off-by: Daniel Noland <daniel@githedgehog.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mvachhar

Note

AI-assisted review: This review was drafted with Claude and reviewed by @mvachhar.

Approving. Steering the control protocols to their own queue, worker, pool and transmit queue is the right design, the class rules are correct (both BGP directions, both BFD ports, neighbour-restricted), and the BF-3 results show BGP/BFD surviving a data flood. One remaining gap, inline: a flood of the open control classes (ARP, ICMPv6, LLDP) can still drop BFD on receive and transmit. Please address it here if there's time, otherwise in an issue for the next release.

mvachhar · 2026-10-18T19:51:53Z

+        };
+        // Copied, as every punt is: the far end is a task on the management runtime, which an
+        // mbuf cannot reach. The mbuf goes back to the control pool when this iteration drops it.
+        if punt.try_send(frame.to_vec()).is_err() {


Note
AI-assisted review: This comment was drafted with Claude and reviewed by @mvachhar.

BGP and BFD can still be dropped when the control classes themselves are flooded, on receive and on transmit. This PR fixes the data-flood case properly, and the BF-3 numbers show it. But BGP/BFD share one FIFO path with ARP, ICMPv6 and LLDP at every stage, and those three aren't neighbour-restricted: anything on the link can send them. The description says per-class policing matters "only if a neighbour floods the control classes themselves", but an open-class flood needs no misbehaving neighbour. For example, scanning the underlay link's IPv6 subnet from anywhere makes the leaf send a neighbour solicitation to our link for every scanned address, and those land in the ICMPv6 multicast class. A broadcast storm or ARP flood does the same through the ARP class.

Where BFD gets dropped under such a flood:

Stage Shared resource Drop

Receive: NIC one control queue per port (512 descriptors) an open-class flood faster than the control worker drains overflows the ring

Receive: control worker → pump (here) one control channel per port (1024), FIFO, try_send the pump writes one frame per syscall into the tap, so an open-class flood above that rate fills the channel and BFD is dropped with it

Receive: tap → kernel the kernel's backlog for the tap (netdev_max_backlog) flood frames that reach the tap compete with BFD before bfdd sees it

Transmit: kernel → pump → inject one inject channel (1024), FIFO, try_send a flood of ARP requests or neighbour solicitations to the gateway makes the kernel answer each one, and those replies fill the inject channel ahead of bfdd's packets

Suggestions, roughly in order of value:

Two priorities instead of one: sessions (BGP, BFD: neighbour-restricted) and open (ARP, ICMPv6, LLDP), kept apart at every stage:

NIC: a second control receive queue for sessions, or a hardware meter (METER_MARK) on the open-class rules;

worker → pump: separate channels, sessions drained first;

transmit: classify what the kernel sends with the same table, so BGP/BFD are injected ahead of ARP/NA replies.

Police the open classes with (18) policer: rate limiting with RFC 2697/2698 three color markers #1907's policer at rates that comfortably cover real ARP/ND/LLDP (a few thousand per second per port), so a flood is dropped at the control worker before it reaches the pump, the tap or the kernel.

BF-3 tests alongside the existing table, with BFD probes running in both directions under a line-rate broadcast ARP flood, a neighbour solicitation flood to solicited-node multicast, and ARP-request and NS floods to the gateway's address (the transmit case). Pass criterion: 0% BFD loss, receive and transmit.

Minor, in the same vein: the control worker goes on the last allowed CPU. If that's the hyperthread sibling of a busy data worker, it shares a core under load, so topology-aware placement matters for the control worker in particular.

I'm approving as is, since this is already a large improvement and these changes aren't small. Please make them here if there's time. If they don't land in this PR, please file an issue for them and schedule it for the next release.

daniel-noland force-pushed the pr/daniel-noland/rate-limit branch from 7576e60 to fb942cd Compare October 16, 2026 04:23

daniel-noland force-pushed the pr/daniel-noland/traffic-protection branch from c6c8d54 to e9dc1a9 Compare October 16, 2026 04:23

daniel-noland changed the title ~~dataplane: a dedicated control queue and worker for BGP, BFD, ARP, ND and LLDP~~ (19) dataplane: a dedicated control queue and worker for BGP, BFD, ARP, ND and LLDP Oct 16, 2026

daniel-noland force-pushed the pr/daniel-noland/rate-limit branch from fb942cd to b3bba1a Compare October 16, 2026 05:47

daniel-noland force-pushed the pr/daniel-noland/traffic-protection branch from e9dc1a9 to 5f5af61 Compare October 16, 2026 05:47

daniel-noland force-pushed the pr/daniel-noland/rate-limit branch from b3bba1a to e8733e0 Compare October 16, 2026 08:43

daniel-noland force-pushed the pr/daniel-noland/traffic-protection branch from 5f5af61 to 75ccd06 Compare October 16, 2026 08:43

daniel-noland force-pushed the pr/daniel-noland/rate-limit branch from e8733e0 to 9851286 Compare October 17, 2026 03:26

daniel-noland force-pushed the pr/daniel-noland/traffic-protection branch from 75ccd06 to 9308300 Compare October 17, 2026 03:26

daniel-noland force-pushed the pr/daniel-noland/rate-limit branch from 9851286 to e4af002 Compare October 17, 2026 05:59

daniel-noland force-pushed the pr/daniel-noland/traffic-protection branch 2 times, most recently from 32835d4 to c28ea76 Compare October 17, 2026 06:17

daniel-noland force-pushed the pr/daniel-noland/rate-limit branch from e4af002 to b74607f Compare October 17, 2026 06:17

daniel-noland and others added 8 commits October 17, 2026 16:01

daniel-noland and others added 5 commits October 17, 2026 16:02

daniel-noland force-pushed the pr/daniel-noland/rate-limit branch from b74607f to a855ba1 Compare October 17, 2026 22:06

daniel-noland force-pushed the pr/daniel-noland/traffic-protection branch from c28ea76 to 84f3f45 Compare October 17, 2026 22:06

daniel-noland marked this pull request as ready for review October 18, 2026 18:03

daniel-noland requested a review from a team as a code owner October 18, 2026 18:03

daniel-noland requested review from sergeymatov and removed request for a team October 18, 2026 18:03

mvachhar approved these changes Oct 18, 2026

View reviewed changes

Navigation Menu

Repository navigation

(19) dataplane: a dedicated control queue and worker for BGP, BFD, ARP, ND and LLDP - #1919

(19) dataplane: a dedicated control queue and worker for BGP, BFD, ARP, ND and LLDP#1919
daniel-noland wants to merge 13 commits into
pr/daniel-noland/rate-limitfrom
pr/daniel-noland/traffic-protection

daniel-noland commented Oct 16, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 16, 2026 •

edited

Loading

Review skipped

Uh oh!

codecov Bot commented Oct 16, 2026 •

edited

Loading

Uh oh!

mvachhar left a comment

Uh oh!

mvachhar Oct 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Stage	Shared resource	Drop
Receive: NIC	one control queue per port (512 descriptors)	an open-class flood faster than the control worker drains overflows the ring
Receive: control worker → pump (here)	one `control` channel per port (1024), FIFO, `try_send`	the pump writes one frame per syscall into the tap, so an open-class flood above that rate fills the channel and BFD is dropped with it
Receive: tap → kernel	the kernel's backlog for the tap (`netdev_max_backlog`)	flood frames that reach the tap compete with BFD before bfdd sees it
Transmit: kernel → pump → inject	one inject channel (1024), FIFO, `try_send`	a flood of ARP requests or neighbour solicitations to the gateway makes the kernel answer each one, and those replies fill the inject channel ahead of bfdd's packets

Conversation

daniel-noland commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

Design

Commits

Tests

Not yet done

Uh oh!

coderabbitai Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

codecov Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mvachhar left a comment

Choose a reason for hiding this comment

Uh oh!

mvachhar Oct 18, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

daniel-noland commented Oct 16, 2026 •

edited

Loading

coderabbitai Bot commented Oct 16, 2026 •

edited

Loading

codecov Bot commented Oct 16, 2026 •

edited

Loading