From 51e86e761ad506dc6ce73a7e2fa4714c3e823248 Mon Sep 17 00:00:00 2001 From: Pau Capdevila Date: Wed, 7 Oct 2026 17:06:43 +0200 Subject: [PATCH] doc: describe what RoCE mode configures and its limits Co-Authored-By: Claude Sonnet 5.5 Signed-off-by: Pau Capdevila --- docs/user-guide/devices.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/docs/user-guide/devices.md b/docs/user-guide/devices.md index 823fa1d4..0decd1d8 100644 --- a/docs/user-guide/devices.md +++ b/docs/user-guide/devices.md @@ -115,6 +115,30 @@ to the outer packet at the time of encapsulation. Likewise the outer DSCP information is copied to the inner packet when the packet is deencapsulated. This process preserves the traffic classification even through a VXLAN tunnel. +#### What RoCE mode configures + +`roce: true` is a single switch-level setting. It applies a fixed set of +lossless Ethernet configuration to the switch as one unit: + +- Buffer sizing for the lossless queues +- Priority Flow Control (PFC) on the RDMA traffic class +- Explicit Congestion Notification (ECN) marking +- Queue scheduling between the traffic classes in the table above + +Enabling `roce` makes the Fabric agent invoke the switch operating system's +built-in RoCE configuration, which applies these settings. The Fabric API has +no fields for adjusting individual PFC, ECN, buffer or scheduler parameters, so +they can't be changed per fabric or per switch. + +!!! warning + Do not change these settings through the switch CLI. The Fabric agent + does not track individual parameter values, so manual changes are + unsupported and are lost when RoCE mode is disabled and enabled again. + +RoCE mode is enabled per switch. PFC and ECN only work end to end when every +switch on the traffic path uses the same configuration, so enable `roce` on all +switches that carry RoCE traffic, and configure the end hosts to match the DSCP +values in the table above. #### RoCE QPN Hashing Mode