DriveNets, best known for bringing cloud-native, software-centric networking to service providers, recently released a series of Ethernet packages to meet the unique needs of AI data centers.
While the technology mania for AI initially centered on silicon, IT leaders are starting to understand that the network plays a critical role in the success of AI. The role of the network is why NVIDIA spent nearly $7 billion to acquire Mellanox in 2019. Since then, the GPU leader’s CEO, Jensen Huang, has continually reiterated that the network is a differentiator.
Traditional Connectivity and AI
The average network, however, doesn’t have the necessary performance to support AI. One option is InfiniBand, which offers great performance for AI but has several negatives. First, InfiniBand is only supported by one vendor, making it a closed technology that creates vendor lock-in. This might be fine for some companies, but most organizations want to have a more open technology that enables long-term choices and a broad ecosystem. Also, while InfiniBand has been around a long time, a limited number of engineers have worked with it, as the technology has historically been used only in niche situations.
In a recent ZK Research study, I asked the question, “Which networking technology do you prefer to support AI workloads?” and 59% of respondents stated Ethernet. In a follow-up response as to why, they referenced the current ubiquity of Ethernet, their familiarity with it and concerns over lock-in.
That said, current Ethernet options are not ideally suited for the rigors of AI. Despite Ethernet’s versatility, it doesn’t guarantee that every packet will reach its destination, and it has too much latency and bandwidth limitations to support AI. AI training and inferencing demand lossless connectivity, extremely low latency and high bandwidth for fast data transfer between compute nodes. This is why current, enhanced Ethernet options require DPUs to be deployed in the servers, to offload networking functions and to spray packets in a manner that bypasses network bottlenecks.
DriveNets’ Approach
DriveNets has a different approach with its Fabric Scheduled Ethernet, an architecture that uses standard Ethernet connections toward the client side but implements a hardware-based, cell-based scheduled fabric system to ensure predictable, lossless performance. This enables it to provide high throughput and low latency, making it ideal for AI clusters.
The technology enables network engineers to connect a series of switches over a lossless fabric, like what FibreChannel did for storage. Historically, data centers were built on chassis-based switches. DriveNets disaggregated the chassis into top-of-rack and fabric switches, with a cell-based protocol from Broadcom connecting them. This allows the fabric to scale out horizontally, enabling companies to start small and grow to a massive network when required.
To ensure traffic is distributed across the fabric evenly, DriveNets uses a technique called “cell spraying” to load balance traffic across the different switches. It also uses virtual output queuing, which is a buffering technique where each input port maintains separate queues for each output port, preventing head-of-line blocking. This isolation of traffic destined for different outputs enables multiple tenants to share the same physical network infrastructure, without their traffic interfering with each other. Congestion on one output queue doesn’t affect traffic destined for other outputs.
A Look at the Benefits
Multi-tenant AI networks have many benefits, such as the following:
-
Improved resource management.
-
Data sharing and collaboration between companies and departments.
-
Managed service providers can offer network services in an “as a service” or subscription model.
DriveNets’ fabric approach has several benefits. The first, and perhaps most important for AI networks, is guaranteed performance. This approach brings the performance benefits of InfiniBand and combines them with the ease of deployment and management of Ethernet. This is also done independent of GPU, NIC or DPU, giving customers the freedom to pick and choose technologies up the stack. In addition to Ethernet’s ease of deployment, the fabric-based scheduling approach eases the fine-tuning process and substantially accelerates the AI cluster setup time, driving huge savings in time and money.
The deployment isn’t quite plug and play, but it’s close. Network engineers can connect DriveNets switches, which operate on white boxes, and the system automatically configures itself to form an AI cluster. Teams can scale the network out by adding switches to the spine.
Final Thoughts
I don’t expect InfiniBand to go away any time soon, but the growth in AI networking will come from Ethernet. In fact, the transition is already underway. Early adopters can withstand the complexity of running InfiniBand. But for AI to scale, the network needs to shift to Ethernet, as it’s much simpler to work with, and the skills to run it are nearly ubiquitous. Not all Ethernet is created equal, however, and customers should do their due diligence to understand all the options.
Speak Your Mind