SRv6 Operations C. Filsfils Internet-Draft P. Camarillo, Ed. Intended status: Informational Cisco Systems Expires: 1 May 2027 L. Yuan G. Lu OpenAI J. Brar D. Becker A. Jouhari Oracle K. Pillai IBM A. Abdelsalam Cisco Systems J. Tantsura NVIDIA K. Patel Arrcus, Inc. 28 October 2026 SRv6 for Deterministic Path Placement in AI Backends draft-filsfils-srv6ops-srv6-ai-backend-06 Abstract This document describes how SRv6 NEXT-CSID enables deterministic path placement in AI backend fabrics through L3-L4 integration on end hosts: the transport stack on the NIC encodes each path as an ordered list of segments (a NEXT-CSID network program) in the packet header, while the fabric forwards statelessly. It explains operational benefits including deterministic probing and alignment with hyperscale production deployments. Status of This Memo This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet- Drafts is at https://datatracker.ietf.org/drafts/current/. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." Filsfils, et al. Expires 1 May 2027 [Page 1] Internet-Draft SRv6 for AI Backends October 2026 This Internet-Draft will expire on 1 May 2027. Copyright Notice Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/ license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1. Requirements Language . . . . . . . . . . . . . . . . . . 3 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. AI Traffic Characteristics and Challenges . . . . . . . . . . 5 4. SRv6 for Deterministic Path Placement . . . . . . . . . . . . 6 5. Statically Provisioned Fabric . . . . . . . . . . . . . . . . 7 6. Deterministic Probing . . . . . . . . . . . . . . . . . . . . 7 7. Illustration . . . . . . . . . . . . . . . . . . . . . . . . 8 7.1. SRv6 Fabric Provisioning . . . . . . . . . . . . . . . . 9 7.2. SRv6-Based Deterministic Path Selection . . . . . . . . . 10 7.3. Transport-Controlled Path Selection with Congestion Feedback . . . . . . . . . . . . . . . . . . . . . . . . 12 8. Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . 12 9. Massive Scale . . . . . . . . . . . . . . . . . . . . . . . . 13 10. Deployment . . . . . . . . . . . . . . . . . . . . . . . . . 14 11. Cluster Operation and Maintenance . . . . . . . . . . . . . . 15 11.1. Scale-Out, Maintenance, and Topology Change . . . . . . 15 11.2. Monitoring and Fault Localization . . . . . . . . . . . 15 12. Security Considerations . . . . . . . . . . . . . . . . . . . 16 13. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 16 14. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 16 15. Normative References . . . . . . . . . . . . . . . . . . . . 17 16. Informative References . . . . . . . . . . . . . . . . . . . 17 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 19 Filsfils, et al. Expires 1 May 2027 [Page 2] Internet-Draft SRv6 for AI Backends October 2026 1. Introduction Hyperscale AI training clusters rely on massive GPU-to-GPU data exchanges, where training-step synchronization delays from congestion and packet loss directly degrade training performance and operational cost. These workloads generate *large, predictable flows* that require ultra-low latency, high bandwidth, and precise congestion control to maintain efficiency. Traditional networking approaches, such as ECMP-based per-flow load balancing, suffer from poor entropy due to the limited number of RoCEv2 flows, leading to fabric hotspots, congestion, and slow reconvergence after failures. SRv6 NEXT-CSID enables *L3-L4 integration* on end hosts in AI backend fabrics: the transport stack on the NIC (i.e., SmartNIC, DPU) controls which path each packet follows by encoding an ordered list of segments in the outer IPv6 header, while switches perform simple, static forwarding without per-flow state. This ensures predictable performance, fine-grained traffic control, and rapid reaction to congestion without fabric reconvergence. This model is deployed at hyperscale. OpenAI, Microsoft, and Oracle Cloud Infrastructure operate training clusters using Multipath Reliable Connection (MRC) over static SRv6 source routing. MRC extends RoCEv2 with multipath packet spraying and transport-layer path selection; SRv6 provides a deterministic mapping from each path identifier to a unique physical path through the fabric. Section 10 summarizes these deployments and provides references. [SRv6-E2E-Frontend-WAN] explains how SRv6 NEXT-CSID is applied to an end-to-end DC Frontend and WAN fabric. 1.1. Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 2. Terminology SRv6 Segment Routing over IPv6 [RFC8986]. NEXT-CSID NEXT Compressed SID as defined in [RFC9800]. Filsfils, et al. Expires 1 May 2027 [Page 3] Internet-Draft SRv6 for AI Backends October 2026 The industry term _uSID (micro SID)_ originated in vendor-specific publications and individual Internet-Drafts and predates the formal name in [RFC9800]. It has been widely used across operators, vendors, open-source implementations, and multi-vendor interoperability reports. This document uses the formal name NEXT-CSID. The terms uSID and NEXT-CSID refer to the same encoding. uSID Micro-segment. Equivalent to NEXT-CSID. ECMP Equal-Cost Multi-Path uN The uN is a short notation for the End behavior with NEXT-CSID, PSP, and USD flavors as defined in [RFC9800]. uA The uA local behavior is a short notation for the End.X behavior with NEXT-CSID, PSP, and USD flavors [RFC9800]. ROCEv2 RDMA over Converged Ethernet version 2 [IBTA-ROCEv2]. NIC Network Interface Card, a hardware component that connects a computer to a network. SmartNIC A Network Interface Card with embedded processing capabilities, designed to offload network and storage tasks from the host CPU. DPU Data Processing Unit, a specialized processor designed to offload and accelerate data-centric tasks, often used in network and storage functions. GPU Graphics Processing Unit, a processor designed for rendering graphics and performing parallel computation tasks, commonly used for AI and machine learning workloads. L3-L4 integration Coordination between the network layer (static SRv6 forwarding in the fabric) and the transport layer (NIC- controlled path selection, congestion response, and probing), without switch-based dynamic routing or per-flow network state. Deterministic path placement Encoding by the source transport of the Filsfils, et al. Expires 1 May 2027 [Page 4] Internet-Draft SRv6 for AI Backends October 2026 path as an SRv6 NEXT-CSID network program (an ordered list of segments in the packet) so each packet or spray round follows a fixed physical path through the fabric. Distinct path identifiers map to disjoint segment programs over the multi-plane topology. Paths are not assigned by a centralized flow scheduler or traffic- engineering controller; the network holds no per-flow state and does not pre-install paths. 3. AI Traffic Characteristics and Challenges AI workloads exhibit highly structured traffic patterns: * *Predictable Elephant Flows*: Collectives' communications require multiple GPUs to exchange data in a structured manner that is known in advance. Flows between GPUs are large, long-lived, high- throughput and predictable. * *Synchronized Bursts*: Model synchronization causes periodic, coordinated traffic spikes. * *Low ECMP Entropy*: Data exchange between GPUs relies on a small number of flows (ROCEv2 Queue Pairs), leading to poor performance of traditional load-balancing solutions. A 5-tuple based ECMP load-balancing results in non-homogenous utilization across the fabric, leading to congestion. * *Resilience*: The fabric must minimize avoidable disruption and support fast, predictable recovery. Even brief hotspots or reconvergence delays can amplify tail latency across a synchronized job. Designs that provide multipath spraying over disjoint encoded segment programs, transport-controlled rerouting, and accurate probing reduce the risk that the network becomes the limiting factor during long training runs. At hyperscale, faults are routine rather than exceptional. In a 54-day Llama 3 405B pre-training run on 16,384 GPUs, Meta reported 419 unexpected interruptions (about 78% hardware-related; 8.4% network switches and cables) [Llama3-Herd]. Synchronized training makes fabric congestion, loss, or degraded paths costly for jobs that run for weeks. Meta's large-scale RoCE backend design is described in [Meta-RoCE-SIGCOMM24]. Filsfils, et al. Expires 1 May 2027 [Page 5] Internet-Draft SRv6 for AI Backends October 2026 4. SRv6 for Deterministic Path Placement The source encodes each path as a NEXT-CSID network program in the packet header; transports spray packets across disjoint network paths, and choose a different path upon congestion. SRv6 enables L3-L4 integration: the transport stack on the NIC controls the AI workload traffic journey through the fabric by encoding an ordered list of segments in the packet header, while the network layer provides stateless static forwarding. * *Control plane and orchestration*: At bring-up, the orchestrator discovers topology and the SRv6 NEXT-CSIDs configured on each link. These NEXT-CSID instructions are statically configured on the routers and are independent of any dynamic routing protocol state. Devices are programmed through the same automation used for other Day-0 fabric configuration (NETCONF/YANG, gNMI, or equivalent). Details are in Section 7.1. - The orchestrator provides to the NICs with topology information, including the NEXT-CSIDs available on each link in the fabric. - Based on that information, the NIC transport composes a path as a sequence of NEXT-CSIDs and encodes the resulting network program in the outer IPv6 Destination Address of each packet. Encoding a path in this way *does not require any per-path communication between the orchestrator and the fabric*. * *Transport stack on the NIC*: Before sending RoCEv2 traffic, the transport stack encapsulates the packet with an outer IPv6 header that carries the NEXT-CSID program selected on the NIC for that packet or spray round. - An outer IPv6 header allows encoding 6 NEXT-CSIDs in the Destination Address. This implies that even with a super-spine in a 3-tier Clos fabric, the entire path can be encoded without an additional Segment Routing Header (SRH). * *Highly Scalable Stateless Fabric*: Routers enforce the path by following SRv6 instructions in the packet header. There is *no per-flow state in the network* (unlike MPLS RSVP-TE, which would require per-path state for each GPU-to-GPU deterministic path). Filsfils, et al. Expires 1 May 2027 [Page 6] Internet-Draft SRv6 for AI Backends October 2026 * *Congestion feedback loop*: The transport stack reacts in real time to congestion notifications (ECN, in-band latency measurement, Packet Trimming, in-band packet loss). At any time, without fabric-wide signaling, the source NIC can change the path by updating the outer IPv6 Destination Address. Only the source changes; intermediate devices remain unchanged. 5. Statically Provisioned Fabric In a dynamically routed fabric, protocols such as BGP maintain reachability across the Clos fabric. When a link fails or the topology changes, switches must reconverge: prefixes are withdrawn or re-advertised, each device updates its RIB, and forwarding entries are reprogrammed. A point-to-point link-down may be detected within milliseconds, but completing BGP reconvergence in a large datacenter fabric typically takes on the order of tens of milliseconds to a sub- second interval in optimized designs, and can be substantially longer when many paths or prefixes are affected [RFC7938]. At extreme scale, convergence may require many round-trip times across the fabric before forwarding is stable [MRC-SRv6-Paper]. In the statically provisioned SRv6 model in this document, GPU traffic paths do not depend on that reconvergence cycle. NEXT-CSID instructions are configured at fabric bring-up and remain in place on the routers. When a link degrades or fails, the fabric does not wait for BGP or other dynamic routing to install a new path. The source NIC or transport stack detects the problem through loss, ECN, latency, or probing, selects another NEXT-CSID network program from the topology information it already holds, and encodes it in the outer IPv6 Destination Address of subsequent packets. That change is local to the sender, takes effect within microseconds, and does not require signaling to intermediate switches or coordination with routing protocol timers [Microsoft-Fairwater]. 6. Deterministic Probing Accurate visibility into fabric health is essential for AI backend operations: scheduling repairs, tuning performance, and correlating transport behavior with physical paths. Traditional approaches face limitations at scale. With ECMP-based forwarding, a probe follows the same path as a data packet only if both share the hash entropy used by the fabric (typically the 5-tuple). In practice they often do not: ICMP or IP- in-IP probes differ from RoCEv2 service headers, so hashing - which is sensitive to those header fields - places them on different ECMP members. After a failure the ECMP set itself changes, so even an identical 5-tuple may take a different path than earlier data Filsfils, et al. Expires 1 May 2027 [Page 7] Internet-Draft SRv6 for AI Backends October 2026 packets. Mechanisms that send probes to remote nodes depend on remote availability and do not always localize faults precisely. ICMP probes to switches are often handled by the control plane, limiting probe frequency. Dynamic routing can change forwarding while probes are in flight, reducing the ground truth of measurements. SRv6 NEXT-CSID source routing enables *deterministic probe pinning*: a probe encoded with the same segment program as data traffic follows the identical physical path through the fabric. There is no ECMP ambiguity, no dependence on switch control-plane ICMP handling for path validation, and no interaction with dynamic routing reconvergence during measurement. * *Path pinning*: Each probe is assigned a specific SRv6 network program, so operators and transport stacks know exactly which links and switches a measurement traverses. * *Dataplane fidelity*: Probes are identical SRv6 packets to the service traffic they validate: same encapsulation, same NEXT-CSID program, and same dataplane forwarding. They therefore observe the impairments that affect GPU traffic (congestion, loss, blackholes, MTU, ACL). Traditional ICMP or IP-in-IP probes (for example Microsoft Netbouncer) use a different packet format from service traffic and can miss failures that are specific to the workload encapsulation. SRv6 probes also enable high-frequency monitoring suitable for large clusters, without relying on switch control-plane ICMP handling. * *Self-probes and localization*: Agents on cluster nodes can source-route probes to a top-of-rack switch and back, or to aggregation switches and back, localizing NIC-to-fabric or fabric- internal faults without requiring a remote peer to be up. * *Transport alignment*: When the transport stack selects paths using SRv6 programs, health probes and background path validation use the same encoding, so measurements reflect the paths data actually uses. Deterministic probing simplifies denylist and spray/path-selection policies on the NIC, supports resurrection of paths after transient failures, and provides operations teams ground-truth telemetry independent of switch control-plane health. 7. Illustration The following figure depicts a typical 2-tier Clos topology. Filsfils, et al. Expires 1 May 2027 [Page 8] Internet-Draft SRv6 for AI Backends October 2026 Spine5 Spine6 | | +--------+----+--------------+-------|-----+ | | | | | | +---------|---+----------|---+---+-----|----+ | | | | | | | | +--+------+ +--+------+ +--+------+ +--+------+ | Leaf1 | | Leaf2 | | Leaf3 | | Leaf4 | +----+----+\ /+----+----+ +----+----+\ /+----+----+ | X | | X | | / \ | | / \ | | / \ | | / \ | +----+----+ +----+----+ +----+----+ +----+----+ | DPU1 | | DPU2 | | DPU3 | | DPU4 | | | | | | | | | | | | | | GPU1 | | GPU2 | | GPU3 | | GPU4 | +---------+ +---------+ +---------+ +---------+ Figure 1: Reference Topology The topology consists of two Spine devices. Each of the Spines is connected to four Leaf devices. There are 4 NICs, which are connected through the host interface (e.g., PCIe) to a GPU. In this example each NIC is dual-homed to two Leaf devices. 7.1. SRv6 Fabric Provisioning At a day0 cluster build-up (fabric bring-up), the topology is provisioned with SRv6 SIDs on the Spine and Leafs devices. These SIDs are statically configured and thus independent of any routing protocol dynamic state. The SID inventory is per device, not per GPU. Each Leaf or Spine instantiates a node SID (uN) and, when hop-by-hop link pinning is used, a small number of adjacency SIDs (uA) - one per fabric interface. A hyperscale cluster with 100,000 GPUs therefore does not require 100,000 SID entries on each switch. The configuration size on a given device is comparable to other Day-0 static items (loopbacks, interface addresses) and is independent of the number of GPUs attached to the fabric. Provisioning uses the same automation already used to bring up the cluster: an orchestration engine that programs devices through NETCONF/YANG, gNMI, or equivalent. CLI templates may be used for smaller fabrics or for bootstrap. The orchestrator is the source of truth for topology and SID assignment; it also distributes the Filsfils, et al. Expires 1 May 2027 [Page 9] Internet-Draft SRv6 for AI Backends October 2026 resulting topology and SID map to NICs (Section 4). Incremental node or link addition (Day 1.5) programs only the new devices and updates the orchestrator's topology database; existing SID entries on other switches remain unchanged. Operational handling of scale-out, repair, and topology change is described in Section 11. The following is provisioned: * SRv6 SID Space in the fabric 5f00:0::/32 * Leaf*1* instantiates the SID 5f00:0:0*1*00::/48 associated with the uN instruction (End with NEXT-CSID, PSP & USD) * Leaf*2* instantiates the SID 5f00:0:0*2*00::/48 associated with the uN instruction (End with NEXT-CSID, PSP & USD) * Leaf*3* instantiates the SID 5f00:0:0*3*00::/48 associated with the uN instruction (End with NEXT-CSID, PSP & USD) * Leaf*4* instantiates the SID 5f00:0:0*4*00::/48 associated with the uN instruction (End with NEXT-CSID, PSP & USD) * Spine*5* instantiates the SID 5f00:0:0*5*00::/48 associated with the uN instruction (End with NEXT-CSID, PSP & USD) * Spine*6* instantiates the SID 5f00:0:0*6*00::/48 associated with the uN instruction (End with NEXT-CSID, PSP & USD) 7.2. SRv6-Based Deterministic Path Selection During a collective, GPU1 and GPU2 send traffic to GPU3. The transport on each NIC sprays packets across disjoint NEXT-CSID network programs, each encoded as an ordered segment list in the outer IPv6 header. The orchestrator supplies topology and NEXT-CSID information at bring-up; path choice and spraying are performed by the NIC transport (e.g., MRC) at send time. Example programs: * GPU1->GPU3: via Leaf1, Spine5, Leaf3 (NEXT-CSID program 5f00:0:0100:0500:0300::) * GPU2->GPU3: via Leaf2, Spine6, Leaf4 (NEXT-CSID program 5f00:0:0200:0600:0400::) When sending RoCEv2 traffic from GPU1 to GPU3: * NIC1: creates a ROCEv2 packet that must be sent to NIC3. NIC1 encapsulates the ROCEv2 packet with an outer IPv6 Header (H.Encaps.Red behavior). Filsfils, et al. Expires 1 May 2027 [Page 10] Internet-Draft SRv6 for AI Backends October 2026 - IPv6 DA: 5f00:0:0100:0500:0300:: - The packet has no SRH. * Leaf1: - Packet in: (IPv6. DA=5f00:0:0100:0500:0300::)(ROCEv2) - Leaf1 has the SID 5f00:0:0100::/48 instantiated with the End with NEXT-CSID, PSP & USD behavior. As a result, it shifts, looks up, and forwards the packet. - Packet out: (IPv6. DA=5f00:0:0500:0300::)(ROCEv2) * Spine5: - Packet in: (IPv6. DA=5f00:0:0500:0300::)(ROCEv2) - Spine5 has the SID 5f00:0:0500::/48 instantiated with the End with NEXT-CSID, PSP & USD behavior. As a result, it shifts, looks up, and forwards the packet. - Packet out: (IPv6. DA=5f00:0:0300::)(ROCEv2) * Leaf3: - Packet in: (IPv6. DA=5f00:0:0300::)(ROCEv2) - Leaf3 has the SID 5f00:0:0300::/48 instantiated with the End with NEXT-CSID, PSP & USD behavior. As a result, it removes the outer IPv6 header and forwards the inner packet. - Packet out: (ROCEv2) * NIC3: receives the ROCEv2 packet, processes it, and passes data to the GPU3. *Note that Leaf1, Spine5, and Leaf3 do not hold any state for this specific flow*. It is a single NEXT-CSID instruction per node instantiated upon cluster build-up and reused by all traffic using that program. GPU2->GPU3 uses the second example program; forwarding is the same stateless model on each hop. While in this example we have used the uN instruction, it can also be encoded using uA instructions specifying the sequence of interfaces. Filsfils, et al. Expires 1 May 2027 [Page 11] Internet-Draft SRv6 for AI Backends October 2026 7.3. Transport-Controlled Path Selection with Congestion Feedback At any time during the execution of the AI job, Spine5 may experience congestion. The transport stack on NIC1 detects this via ECN, in- band latency, packet trimming, or loss feedback. Within microseconds, without fabric signaling or new state at intermediate devices, the transport stack steers traffic to a different path. NIC1 switches the path from to by encapsulating new traffic for GPU1->GPU3 with IPv6 DA 5f00:0:0100:0600:0300::. *This is not switch adaptive routing* (e.g., dynamic ECMP or BGP reconvergence). Path changes are made only at the source NIC; switches continue static SRv6 forwarding. The fabric is entirely stateless, and the packet path is encoded in the IPv6 header built at the source. Separating transport-controlled path selection from switch-based adaptive routing avoids unpredictable interactions at scale and is essential because AI workloads cannot tolerate slow reconvergence [Microsoft-Fairwater]. 8. Benefits * *Deterministic Path Placement*: SRv6 allows the NIC to encode, per packet or spray round, distinct NEXT-CSID network programs that pin traffic to disjoint paths through the fabric. * *Minimum-MTU*: A plain outer IPv6 encapsulation allows to encode 6 NEXT-CSIDs in the outer DA. This implies that without the need of additional extension headers, only with 40 Bytes of IPv6 encapsulation, we can encode up to 6 intermediate waypoints allowing to enforce a path in a 3-tier Clos network. This is sufficient to control a path hop-by-hop (link by link) through a leaf, spine, super-spine, spine, leaf. * *Congestion Feedback Loop*: Instant rerouting at the source based on ECN, in-band measured One-Way and Two-Way latency, Packet Trimming feedback and in-band packet loss, without any dependency of routing protocols. There is neither any control-plane signaling involved between the GPU and the fabric, nor between the AI orchestrator and the fabric devices. * *Standardization*: Open, vendor-agnostic implementation Filsfils, et al. Expires 1 May 2027 [Page 12] Internet-Draft SRv6 for AI Backends October 2026 * *Ease of operation*: As opposed to black-box proprietary solutions that pack opaque layer-2 optimizations, the SRv6 solution is minimalistic, IP-based, fully standardized, and supported by a rich ecosystem (vendor, merchant silicon, and open source). The deterministic and open nature of the solution simplifies troubleshooting. * *Production validation*: Hyperscale AI training clusters operate static SRv6 source routing at scale; see Section 10. 9. Massive Scale AI workloads are deployed across thousands of GPUs in multi-tier Clos networks, requiring a networking architecture that scales efficiently. SRv6 NEXT-CSID ensures deterministic path placement while maintaining scalability through the following mechanisms: * *Stateless Fabric*: Unlike RSVP-TE or MPLS-TE, which require per- flow state on network devices, SRv6 enforces paths by including all instructions in the packet header. This eliminates state explosion as the number of GPUs increases. * *NEXT-CSID Encapsulation*: The SRv6 NEXT-CSID encoding allows paths to be efficiently encoded even in multi-tier topologies, reducing encapsulation overhead while supporting large deployments. If more than 6 instructions are required, a simple IPv6 Segment Routing Extension Header can encode additional instructions. * *Multi-plane topologies*: High-radix, multi-plane Clos designs spread NIC capacity across parallel network planes, improving physical redundancy and enabling clusters well beyond 100K GPUs in two-tier fabrics while keeping latency low. * *Cross-Datacenter Extension*: The same SRv6-based mechanism can extend beyond a single cluster to multi-datacenter AI fabrics (inter-DC AI training), where deterministic path placement ensures efficient inter-cluster data transfers. SRv6 network programs can be extended to forward between clusters using the same path- encoding model. * *Overlay Tenant Separation*: SRv6 can provide per-tenant network segmentation, ensuring AI workloads from different tenants or jobs are isolated while sharing the same physical infrastructure. Dedicated network resources can be assigned on a per-tenant basis in the fabric, providing resource isolation so that bandwidth, paths, and forwarding capacity for one tenant are not conflated with another. By adding VPN Service SIDs into the encoded network Filsfils, et al. Expires 1 May 2027 [Page 13] Internet-Draft SRv6 for AI Backends October 2026 program, distinct path identifiers and network planes per tenant can be enforced at the network level without additional overlay encapsulations. 10. Deployment Static SRv6 NEXT-CSID source routing is deployed at hyperscale in production AI training clusters together with Multipath Reliable Connection (MRC), an RDMA transport specified by the Open Compute Project and developed collaboratively by OpenAI, Microsoft, NVIDIA, AMD, Intel, and Broadcom [OCP-MRC] [MRC-SRv6-Paper] [OpenAI-MRC]. MRC extends RoCEv2 Reliable Connection with multipath packet spraying, selective retransmission, packet trimming for incast, and transport-layer path health management; it runs Ethernet in best- effort mode and relies on fast recovery at the transport layer rather than Priority Flow Control. In these fabrics, dynamic routing in switches is disabled: each packet's path is encoded in the outer IPv6 destination using NEXT-CSID, path identifiers map algorithmically to SRv6 network programs, and the transport stack gains deterministic control while routers forward statelessly. This combination has been used to train frontier large language models on clusters exceeding 100,000 GPUs, with implementations on 400 and 800 Gb/s RDMA NICs and SRv6 forwarding across multiple switch platforms and NOS distributions [MRC-SRv6-Paper]. The same architecture spans OpenAI, Microsoft, and Oracle Cloud Infrastructure (OCI) training sites that operate as one coherent design rather than isolated experiments. OpenAI runs MRC over static SRv6 on its largest NVIDIA GB200 supercomputers [OpenAI-MRC] [MRC-SRv6-Paper]; Microsoft's Fairwater supercomputer applies the same model on a two-tier, multi-plane fabric, removing BGP and other dynamic routing from the scale-out network in favor of compact NEXT- CSID source routing, with probe traffic following the same paths as data for ground-truth visibility [Microsoft-Fairwater]; and OCI's Oracle Acceleron multiplanar networking deployed MRC and SRv6 source- based routing at scale, including the Stargate datacenter in Abilene, Texas, with path intelligence at the NIC and simple static forwarding in the network [Oracle-Acceleron-MRC] [Oracle-Acceleron-Arch]. Across these environments, multipath spraying and SRv6 path pinning reduce flow collisions, improve utilization across network planes, and allow large synchronous training jobs to continue through link flaps and partial failures that previously caused restarts [OpenAI-MRC] [Microsoft-Fairwater]. Microsoft has additionally open- sourced MRC software interfaces and SONiC SRv6 enhancements for AI backend networks [Microsoft-Fairwater]. Filsfils, et al. Expires 1 May 2027 [Page 14] Internet-Draft SRv6 for AI Backends October 2026 11. Cluster Operation and Maintenance This section describes typical operational scenarios in a statically provisioned SRv6 AI backend: scale-out, equipment maintenance, topology change, and monitoring. The fabric holds no per-flow state, so these events do not trigger routing reconvergence for GPU traffic. Host path selection uses the topology and NEXT-CSID map supplied by the orchestrator, together with live dataplane feedback. 11.1. Scale-Out, Maintenance, and Topology Change Adding switches or links (cluster scale-out, Day 1.5) programs NEXT- CSIDs only on the new devices, using the same automation as Day 0. Existing SID entries on other Leaf and Spine devices are left unchanged. The orchestrator updates its topology database and distributes the delta to NICs so that transport stacks can compose additional NEXT-CSID network programs that use the new capacity. Traffic already in flight continues on previously encoded paths; no fabric signaling is required. When a switch is taken offline for repair, or a link fails, the fabric does not withdraw prefixes or recompute ECMP. Source NICs detect the impairment through loss, ECN, in-band latency, packet trimming, or deterministic probes, and stop using the affected NEXT- CSID programs (denylist). That reaction is local to the host and takes effect within microseconds. Hosts do not need a topology push to stop using a failed path: they already hold the program inventory and the dataplane signals which programs are unhealthy. When the device or link returns, the same probes can resurrect the path without re-provisioning SIDs. Topology and SID information on the end host therefore needs an update only when the available path set changes in a way the host cannot discover from dataplane feedback alone - principally when new devices or links are added and should be used, or when SIDs are reassigned. Failed or drained equipment does not require a host topology rewrite for correctness; denylist and probing are sufficient until the orchestrator optionally removes the stale entries. 11.2. Monitoring and Fault Localization Cluster monitoring combines deterministic SRv6 probing (Section 6) with in-band signals already consumed by the transport stack: ECN, one-way and two-way latency, packet trimming, and loss. Because probes and service packets are identical SRv6 packets that follow the same NEXT-CSID program, measurements apply to the paths that carry GPU traffic. Filsfils, et al. Expires 1 May 2027 [Page 15] Internet-Draft SRv6 for AI Backends October 2026 Operators can pin a probe to a specific Leaf-Spine-Leaf sequence, or bounce a self-probe at a ToR or aggregation switch, and thereby distinguish NIC-to-fabric faults from fabric-internal faults without depending on a remote peer or on switch control-plane ICMP. High- frequency probing, together with per-path denylist and resurrection on the NIC, yields efficient detection and localization at hyperscale without a fabric-wide control-plane event. 12. Security Considerations This document is informational and does not define a new protocol. Security for the SRv6 data plane and segment programming is covered in [RFC8986]. The deployment model assumes a dedicated AI backend fabric in a single administrative domain: an orchestrator statically provisions NEXT-CSIDs on routers and supplies topology to NICs, and transport stacks on GPU hosts encode segment programs in each packet while the fabric forwards without per-flow state. Security therefore depends on integrity of provisioning, access control over router configuration, and path-selection logic on each host. Because the source encodes the full segment list, a compromised or misconfigured host could steer traffic along unintended paths. Operators SHOULD limit which workloads may send SRv6-encapsulated traffic and constrain hosts to programs authorized by the orchestrator. Backend fabrics are typically isolated from untrusted networks, and operators SHOULD maintain that separation. Security of RoCEv2 and Multipath Reliable Connection (MRC) transports is outside the scope of this document. 13. Contributors The following people contributed significantly to this document: Chris Martin (Cisco Systems), Weiqiang Cheng (China Mobile), 14. Acknowledgements The authors thank the teams behind the MRC and SRv6 production deployments described in Section 10, including contributors at OpenAI, Microsoft, Oracle Cloud Infrastructure, NVIDIA, AMD, Intel, and Broadcom. The authors would like to recognize the work of Lihua Yuan, Guohan Lu, Rita Hui, and Riff Jiang at Microsoft. Filsfils, et al. Expires 1 May 2027 [Page 16] Internet-Draft SRv6 for AI Backends October 2026 Pablo Camarillo and Rita Hui presented this use case at NANOG 96; a recording is available at https://www.segment-routing.net/ conferences/2026-02-02-NANOG96-SRv6-AI-Backend-Microsoft. Clarence Filsfils and Guohan Lu presented related material at OCP EMEA 2026; a recording is available at https://www.segment-routing.net/ conferences/2026-OCP-EMEA-summit-scalable-ai-protocol-stack. The authors would like to acknowledge the work of the developers who have enabled this use-case in the open-source [SONiC] implementation. In particular: Carmine Scarpitta, Abhishek Dosi, Changrong Wu, Kumaresh Perumal, Eddie Ruan, Yuqing Zhao, Rajasekar Raja, and Vivek Venkatraman. 15. Normative References [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, . [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, . [RFC8986] Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer, D., Matsushima, S., and Z. Li, "Segment Routing over IPv6 (SRv6) Network Programming", RFC 8986, DOI 10.17487/RFC8986, February 2021, . [RFC9800] Cheng, W., Ed., Filsfils, C., Li, Z., Decraene, B., and F. Clad, Ed., "Compressed SRv6 Segment List Encoding", RFC 9800, DOI 10.17487/RFC9800, June 2025, . 16. Informative References [SRv6-E2E-Frontend-WAN] Filsfils, C., Camarillo, P., Michielsen, K., and A. Gorovoy, "SRv6 End-to-End DC Frontend and WAN", Work in Progress, Internet-Draft, draft-filsfils-srv6ops-srv6-e2e- dc-frontend-wan-01, 2026, . [IBTA-ROCEv2] InfiniBand Trade Association, "InfiniBand Architecture Specification Volume 1, Release 1.2.1, Annex A17: ROCEv2", Filsfils, et al. Expires 1 May 2027 [Page 17] Internet-Draft SRv6 for AI Backends October 2026 2 September 2014, . [SONiC] Linux Foundation, "SONiC", . [OCP-MRC] Open Compute Project, "Multipath Reliable Connection (MRC) Specification, Revision 1.0", March 2026, . [MRC-SRv6-Paper] Araujo, J., Chow, A., Handley, M., Lu, G., and L. Yuan, "Resilient AI Supercomputer Networking using MRC and SRv6", May 2026, . [OpenAI-MRC] OpenAI, "Supercomputer networking to accelerate large scale AI training", May 2026, . [Microsoft-Fairwater] Cutts, V. and J. Jose, "Building resilient networks for AI supercomputers", May 2026, . [Oracle-Acceleron-MRC] Vincent, P., "First Principles: Unlocking Oracle Acceleron Multiplanar Fabric with Multipath Reliable Connection", May 2026, . [Oracle-Acceleron-Arch] Vincent, P., "First Principles: Oracle Acceleron Multiplanar Networking Architecture", May 2026, . [RFC7938] Lapukhov, P., Premji, A., and J. Mitchell, Ed., "Use of BGP for Routing in Large-Scale Data Centers", RFC 7938, DOI 10.17487/RFC7938, August 2016, . [Llama3-Herd] Dubey, A., Jauhri, A., and A. Pandey, "The Llama 3 Herd of Models", July 2024, . Filsfils, et al. Expires 1 May 2027 [Page 18] Internet-Draft SRv6 for AI Backends October 2026 [Meta-RoCE-SIGCOMM24] Gangidi, A., Miao, R., and S. Zheng, "RDMA over Ethernet for Distributed AI Training at Meta Scale", August 2024, . Authors' Addresses Clarence Filsfils Cisco Systems Belgium Email: cf@cisco.com Pablo Camarillo Garvia (editor) Cisco Systems Spain Email: pcamaril@cisco.com Lihua Yuan OpenAI United States of America Email: lihua@openai.com Guohan Lu OpenAI United States of America Email: lgh@openai.com Jag Brar Oracle United States of America Email: jag.brar@oracle.com David Becker Oracle United States of America Email: david.b.becker@oracle.com Abderrahman Jouhari Oracle United States of America Email: abderrahman.jouhari@oracle.com Filsfils, et al. Expires 1 May 2027 [Page 19] Internet-Draft SRv6 for AI Backends October 2026 Kiran Pillai IBM United States of America Email: Kiran.Pillai@ibm.com Ahmed Abdelsalam Cisco Systems Italy Email: ahabdels@cisco.com Jeff Tantsura NVIDIA United States of America Email: jefftant.ietf@gmail.com Keyur Patel Arrcus, Inc. United States of America Email: keyur@arrcus.com Filsfils, et al. Expires 1 May 2027 [Page 20] .