(18) policer: rate limiting with RFC 2697/2698 three color markers

[daniel-noland]

Stacked on (17g) #1925, the top of the LLDP stack ((17a) #1906 – (17g) #1925).

A rate limiting network function. Each rule matches traffic, to start with by source VNI and inner destination prefix, and limits packets per second and/or bytes per second. Every packet is colored by an RFC 2697 (srTCM) or RFC 2698 (trTCM) meter, and each color has an action: pass, remark DSCP, or drop.

Why policing, not shaping

At 100 Gb/s anything that queues is a liability. The three color markers are meters: each packet is colored when it arrives and never held, for a few comparisons a packet. What is hard at line rate is the workers, not the algorithm. RSS spreads one rule's traffic over every worker, and a bucket updated per packet would bounce one cache line between all of them.

Commits

1. policer
   • Profiles for both markers.
   • Limits on bytes, packets, or both.
   • An exact single-owner Meter.
   • SharedMeter/LocalMeter: each worker leases tokens in batches, so the shared lock is taken roughly once per lease, not once per packet. A refused worker stays off the shared state until the bucket could have refilled.
   • The departure from the RFC model is bounded: a burst can stretch by one lease per worker, and the long-term rate is unchanged.
   • Tested against a literal tick-by-tick transcription of both RFCs, with a property test over worker interleavings and a shuttle/loom model. RFC 2697/2698 are vendored and cited with duvet.
2. refactor(acl): the first-match classifier table (rte_acl or reference) moves out of acl-filter into acl::table, so any MatchKey can use it.
3. config: rate limits in the overlay model.
   • Each rule has a name, a VPC, a destination prefix as the VPC addresses it (before NAT), limits, and an action per color.
   • Validation refuses an unknown VPC, a repeated name, and two rules on the same VPC and destination. Nested destinations are allowed; the longest prefix wins.
4. net: DoneReason::RateLimited, and Ipv6::payload_length.
5. policer: Display for limits.
6. rate-limit: the RateLimiter<C: Classify> stage.
   • It is generic over the classifier, so it can police on any key at any point in the pipeline. VpcDestination is the first classifier.
   • A rule that survives a config change with the same name and limits keeps its meter and its counts.
   • Counts are kept per worker, so counting never contends.
7. dataplane:
   • the stage sits after the ACLs (denied packets are not charged) and before NAT;
   • mgmt applies the rules;
   • read-only show rate-limits displays them;
   • rate-limited drops are never punted.

Tests

Passing: policer, rate-limit, config, net, just test dataplane (87) and just test mgmt (22). The mgmt sample-config test now applies a rate limit through rte_acl and checks that it reaches the workers. rte_acl is also checked against the reference backend.

Not yet done

• The k8s/gateway API conversion. Rate limits exist in the internal model only.
• Real-hardware throughput measurement.
• Caching the rule on the flow, to skip classifying packets of known flows.
• Overlay-algebra generation of rate limits.

🤖 Generated with Claude Code

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 94.55865% with 90 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
mgmt/src/tests/mgmt.rs	63.33%	11 Missing ⚠️
acl/src/table.rs	88.63%	10 Missing ⚠️
mgmt/src/processor/proc.rs	0.00%	10 Missing ⚠️
rate-limit/src/stage.rs	85.50%	7 Missing and 3 partials ⚠️
dataplane/src/runtime.rs	0.00%	9 Missing ⚠️
rate-limit/src/rules.rs	92.10%	9 Missing ⚠️
dataplane/src/packet_processor/mod.rs	0.00%	7 Missing ⚠️
cli/bin/cmdtree_dp.rs	0.00%	6 Missing ⚠️
rate-limit/src/display.rs	85.71%	4 Missing ⚠️
config/src/external/overlay/ratelimit.rs	97.63%	3 Missing ⚠️
... and 6 more

📢 Thoughts on this report? Let us know!