InfiniBand vs Ethernet for AI Clusters: The Fabric War Inside the AI Factory

Inside every AI factory, the quiet argument is not only about which GPU or custom accelerator wins. It is about the GPU fabric that stitches thousands of accelerators into one training job—and the different fabric that serves inference at the edge of the same campus. That contest is commonly framed as InfiniBand vs Ethernet AI, but the reality in 2025–2026 is more nuanced: both camps now ship 400G and 800G RDMA-capable scale-out networks, both chase lossless behavior and low tail latency, and both are racing toward co-packaged optics and multi-site “scale-across” designs.

This article maps the fabric war for practitioners who must choose—or deliberately mix—AI cluster networking technologies. It covers training versus inference traffic shapes, latency and RDMA (including RoCE), congestion control, NVIDIA Quantum InfiniBand and Spectrum-X, Broadcom Tomahawk and Jericho3-AI, the Ultra Ethernet Consortium, cost and operations, and the near-term outlook. Every numerical claim below is tied to a source listed at the end.

Scale-up, scale-out, and why the fabric matters

A modern GPU cluster is not one network. Dell’s AI fabric framing, which matches how hyperscalers talk about the problem, separates at least four planes: a scale-up domain (NVLink, Infinity Fabric / xGMI, and similar in-node or in-pod links), a scale-out or back-end east–west fabric between nodes, a front-end fabric for storage and applications, and a management network. The InfiniBand-versus-Ethernet debate is almost entirely about the scale-out GPU fabric—the path that carries collective communications (AllReduce, AllGather, ReduceScatter, all-to-all) while a model trains or while a large inference graph is sharded.

NVIDIA’s public networking narrative goes further: NVLink for scale-up, Quantum InfiniBand or Spectrum-X Ethernet for scale-out, and Spectrum-XGS for multi-data-center “scale-across.” The product story matters because it shows that even a single vendor now treats Ethernet and InfiniBand as parallel answers to the same AI-factory problem, not as mutually exclusive religions.

Training fabrics versus inference fabrics

Training and inference punish the network differently.

Distributed training is synchronous, bursty, and collective-heavy. Meta’s SIGCOMM-facing write-up on RoCE for AI training describes low flow entropy, millisecond-scale on/off bursts, and elephant flows that can hit NIC line rate. When thousands of GPUs finish a compute step together, they flood the fabric with large, repetitive messages. Tail latency and micro-congestion directly stretch job completion time. That is why AI training back-ends insist on RDMA, careful topology (often rail-optimized Clos), and either native InfiniBand credit-based flow control or a carefully engineered RoCE stack with PFC, deep buffers, adaptive routing, or receiver-driven admission.

Inference is more heterogeneous. Single-node or small multi-GPU serving may barely touch the scale-out fabric. Large-model serving with tensor or pipeline parallelism, disaggregated prefill/decode, or KV-cache movement across nodes still needs high bandwidth and predictable latency—but traffic is often less synchronized than a full-cluster AllReduce, and operators care more about tail latency for time-to-first-token and tokens-per-second under multi-tenant load. NVIDIA explicitly positions Spectrum-X for multi-tenant generative AI clouds, with performance isolation as a first-class requirement—an inference- and cloud-shaped concern more than a classic single-tenant HPC training concern.

In practice, many operators still standardize one back-end technology for both, then tune QoS, tenancy, and oversubscription. The important design point is not “InfiniBand for training, Ethernet for inference” as a slogan. It is: match lossless behavior and effective bandwidth to the collective pattern you actually run, and do not assume a front-end Ethernet leaf-spine will behave like a back-end GPU fabric without RDMA, congestion policy, and topology-aware scheduling.

Latency, RDMA, and the RoCE bridge

Both InfiniBand and modern AI Ethernet win by keeping the CPU out of the data path. RDMA lets a remote NIC write into registered memory (and, with GPUDirect RDMA, often directly between GPU memories). InfiniBand was designed around that model. Ethernet caught up through RoCE (RDMA over Converged Ethernet): RoCE v1 was essentially InfiniBand over an Ethernet L2 header; RoCE v2, standardized by the InfiniBand Trade Association in 2014, puts RDMA in UDP/IP so traffic can be routed across L3 fabrics—the form hyperscalers actually deploy.

TrendForce’s October 2025 comparison, summarizing the scale-out market, places native InfiniBand RDMA latency under about 2 µs and RoCE v2 under about 5 µs at mainstream 800 Gbps ports, with InfiniBand relying on credit-based flow control and RoCE on Ethernet flow-control tools (including PFC and ECN). Those figures are industry-analyst characterizations, not a universal lab result, but they capture the qualitative gap operators still discuss: InfiniBand remains slightly more “native,” while RoCE is “good enough” when the rest of the stack is co-designed.

Meta’s public evidence is stronger than any micro-benchmark slide. In March 2024 the company described two 24,576-GPU H100 clusters: one on Arista 7800 plus OCP Wedge400/Minipack2 RoCE, one on NVIDIA Quantum-2 InfiniBand, both at 400 Gbps endpoints. Through co-design of network, software, and model architecture—including topology-aware scheduling and NCCL-aware routing—Meta reported successful large GenAI training on both, including Llama 3 training on the RoCE cluster, without network bottlenecks. That single deployment permanently weakened the claim that only InfiniBand can train frontier models at scale.

Congestion control: credits, PFC, Spectrum-X, and Ultra Ethernet

InfiniBand’s credit-based world

InfiniBand’s link-layer credit-based flow control only sends when the receiver has buffer credits. Combined with a subnet manager, adaptive routing, and (on NVIDIA Quantum platforms) SHARP in-network collectives and telemetry-based congestion control, the fabric aims for lossless, ordered delivery with deterministic bandwidth. NVIDIA’s Quantum-X800 platform documentation describes end-to-end 800 Gb/s InfiniBand, 144-port switches, SHARP v4, adaptive routing, and telemetry-based congestion control aimed at trillion-parameter-scale training.

RoCE’s operational reality

RoCE v2 on Ethernet typically depends on Priority Flow Control (PFC) to pause loss-sensitive classes, ECN marks for congestion notification, and historically DCQCN-style rate control on NICs. Meta’s August 2024 engineering post is unusually candid: after trouble tuning DCQCN at 400G, Meta ran production AI training for an extended period with PFC and without DCQCN, relying on deep-buffer spines, receiver-driven clear-to-send admission in the collective path, and high-priority treatment of CTS packets. The lesson for builders is not “turn off congestion control,” but that AI collectives need workload-aware controls; storage-era DCQCN defaults do not automatically transfer to GPU training.

NVIDIA Spectrum-X

Spectrum-X is NVIDIA’s answer when the customer wants Ethernet operations with InfiniBand-like AI behavior. NVIDIA describes it as delivering about 1.6× the network performance of off-the-shelf Ethernet for AI, with accelerated RoCE, adaptive routing (packet spray coordinated with SuperNIC reordering / direct data placement), congestion control, and performance isolation for multi-tenant clouds. Spectrum-X pairs Spectrum Ethernet switches with ConnectX SuperNICs and BlueField DPUs; Spectrum-XGS extends the story across sites. In NVIDIA’s framing, Quantum InfiniBand remains the low-latency AI-factory and HPC choice with in-network computing, while Spectrum-X targets Ethernet AI clouds.

Broadcom Tomahawk and Jericho3-AI

Broadcom attacks the same problem from merchant silicon. Tomahawk 5 (51.2 Tbps class, up to 64×800GbE configurations in Broadcom’s AI networking materials) anchors endpoint-scheduled fabrics: NICs and endpoints schedule; high-radix Tomahawk switches forward with dynamic/adaptive load balancing. Jericho3-AI (BCM88890) anchors switch-scheduled fabrics with Ramon spines: Broadcom documents up to 18×800GbE (or 36×400GbE) network-facing ports per device, RoCEv2 support, perfect cell-based load balancing, end-to-end scheduling to limit head-of-line blocking, and scaling claims on the order of tens of thousands of 800G accelerators in a scheduled fabric (Broadcom cites 32,000 × 800G in its AI/ML white paper and product materials). Meta’s OCP 2024 networking post shows both families in hyperscale designs: Jericho3-AI-based distributed switch fabric leaves and Tomahawk5-based Minipack3 51.2T switches.

TrendForce’s 2025 silicon table also places Broadcom Tomahawk 6 at 102.4 Tbps (1.6 Tbps/port × 64) with Ultra Ethernet–oriented features such as multipath packet spraying, link-layer retry, and credit-based flow control options—evidence that Ethernet silicon is absorbing mechanisms long associated with InfiniBand.

Ultra Ethernet Consortium

The Ultra Ethernet Consortium (UEC), launched in 2023 with founding members including AMD, Arista, Broadcom, Cisco, Eviden, HPE, Intel, Meta, and Microsoft, released Specification 1.0 on 11 June 2025: a full-stack Ethernet approach for AI and HPC with modern RDMA-style transport, multi-vendor interoperability, and stated scalability toward millions of endpoints. Specification 1.1 (12 October 2026) adds scale-up optimizations (including a Unified Forwarding Header that UEC says saves 20 bytes versus a traditional IP packet), UET Verbs mapping for application compatibility, optional in-network collectives, Congestion Signaling (CSIG) in-band telemetry tags, and Programmable Congestion Management APIs. UEC is not “RoCE with a new logo”; it is an attempt to standardize the next Ethernet transport for AI so operators are not locked into a single vendor’s proprietary RoCE extensions.

Cost, operations, and the vendor landscape

Cost comparisons are where vendor white papers diverge most sharply—and where buyers should demand apples-to-apples bills of materials. TrendForce’s 2025 table characterizes InfiniBand hardware cost as roughly 3× Ethernet at comparable mainstream 800G ports (analyst estimate, not a public price list). Dell’s balanced fabric guide emphasizes softer costs: Ethernet staff familiarity, multi-vendor competition, and the ability to reuse aging Ethernet switches on front-end, storage, or management fabrics; InfiniBand often wins on faster out-of-box back-end bring-up (subnet manager versus extensive RoCE/PFC/ECN tuning) and Unified Fabric Manager operational tooling. Broadcom argues that higher Ethernet radix and bandwidth reduce tiers, optics, and power for large clusters; NVIDIA argues that Quantum’s in-network computing and Spectrum-X’s end-to-end co-design raise effective bandwidth and shorten job time enough to repay premium silicon.

The vendor map in late 2026 roughly looks like this:

Cisco, Marvell, and others publish AI-fabric benchmarks against InfiniBand and Spectrum-X; treat those as vendor-sponsored until reproduced in your topology. The durable fact is competition: AI Ethernet is no longer a science project, and InfiniBand is no longer the only serious GPU fabric.

What the lab and production data actually show

Vendor micro-benchmarks can make InfiniBand and Ethernet look interchangeable on a two-node latency graph. Production tells a noisier story. Meta’s public AllGather tuning narrative for its 24k-GPU clusters is instructive: out-of-the-box large-cluster collective bandwidth utilization was initially inconsistent—sometimes far below the 90%+ range seen on small optimized clusters—until topology-aware scheduling and NCCL-aware routing restored large-cluster performance to the small-cluster regime. That is an operations finding as much as a silicon finding: the GPU fabric only delivers after the scheduler, routing entropy, and collective library agree.

An SC ’25 workshop study comparing RDMA over Converged Ethernet with InfiniBand for LLM training similarly stresses configuration. The authors report that RoCE can achieve near-linear scaling comparable to InfiniBand when properly configured, and that RDMA—on Ethernet or InfiniBand—matters far more than TCP/IP for large-model scaling behavior. Cisco’s Silicon One AI/ML fabric posts likewise frame Ethernet as competitive when dynamic load balancing and packet spray are measured against Spectrum-X and InfiniBand baselines on dense H100 pods. None of these results license a blank check for commodity leaf-spine Ethernet without RoCE engineering; they do license an evidence-based RFP that scores effective bandwidth under collective load, not brand allegiance.

For inference-heavy AI clouds, add noisy-neighbor tests. NVIDIA’s Spectrum-X positioning and Dell’s multi-tenant Ethernet arguments both imply that performance isolation under concurrent jobs is a first-order requirement. A fabric that wins a dedicated training soak test can still lose when tenancy, storage bursts, and control-plane chatter share buffers. Budget time for multi-job soak tests before you cut a purchase order.

Outlook: coexistence, CPO, and scale-across

The near-term equilibrium is coexistence, not a knockout. InfiniBand retains a strong hold wherever single-tenant training clusters prioritize SHARP collectives, plug-and-play lossless behavior, and NVIDIA’s full stack. Ethernet—RoCE today, Ultra Ethernet tomorrow—continues to take share wherever operators want open bidding, shared networking talent, multi-tenancy, and reuse across front-end and back-end roles. Meta already proved RoCE can train Llama-class models; UEC 1.1 is trying to make that class of performance a multi-vendor standard rather than a heroic co-design exercise.

Two physical-layer trends will reshape both camps. First, co-packaged optics (CPO) and silicon photonics move optics onto the switch package to cut power and reach as port speeds climb toward 1.6T; TrendForce and NVIDIA both describe CPO roadmaps on Quantum and Spectrum platforms, while Broadcom has shipped successive Tomahawk CPO generations. Second, scale-across—linking AI factories between buildings or campuses with optical long-haul and specialized congestion/NCCL behavior—extends the fabric war beyond a single hall. NVIDIA’s Spectrum-XGS claims and the industry’s August 2025 “scale-across” messaging (as summarized by TrendForce) show that the next bottleneck after the leaf-spine may be the metro fiber between halls.

For builders writing RFPs today, a pragmatic checklist beats tribal loyalty:

1. Profile the workload. Synchronous LLM training, mixture-of-experts all-to-all, and multi-tenant inference isolation are different fabrics even if they share a switch ASIC.
2. Require RDMA end-to-end on the back-end (InfiniBand or RoCEv2/UE), plus GPUDirect-class paths where applicable.
3. Test collectives, not only ping. NCCL/RCCL job completion time under failure and under multi-job noise matters more than unloaded latency.
4. Price the whole system: switches, optics, NICs/SuperNICs, spares, power, and the people who can debug PFC storms or subnet manager events at 3 a.m.
5. Ask about the standards path. RoCE extensions, Spectrum-X features, Jericho scheduling, and UEC 1.1 CSIG/INC will not all interoperate on day one—plan for islands or insist on written multi-vendor commitments.

Conclusion

InfiniBand vs Ethernet for AI is no longer a simple performance hierarchy. InfiniBand still offers a tightly integrated, credit-based, SHARP-accelerated path that many AI factories prefer for dedicated training. Ethernet—with RoCE in production at Meta scale, Spectrum-X for NVIDIA Ethernet AI clouds, Broadcom Tomahawk/Jericho scheduled fabrics, and UEC specifications 1.0/1.1—has closed enough of the gap that cost, operations, multi-tenancy, and vendor strategy often decide the RFP. The real “fabric war inside the AI factory” is a systems war: topology, congestion control, collective libraries, optics, and scheduling must be designed together. Choose the GPU fabric that your workload, staff, and roadmap can actually operate at full effective bandwidth—not the one that wins a slide-deck micro-benchmark.

Related reading

• How Distributed AI Superfactories Stretch the Network Between Campuses
• Multi-Trillion-Parameter Training: What the Next Silicon Generation Must Deliver
• Microsoft Fairwater and the Rise of the Distributed AI Superfactory

Images: original illustrations generated for AndBeyond.Tech; no logos or real people depicted.