InfiniBand (IB) is an advanced computer network communication standard developed by the InfiniBand Trade Association (IBTA). InfiniBand technology enjoys a high reputation in HPC connections for supercomputers, storage, and even LAN networks. InfiniBand has numerous advantages, including simplified management, high bandwidth, complete CPU offloading, ultra-low latency, cluster scalability and flexibility, quality of service (QoS), and SHARP support, among others.
InfiniBand is a critical communication technology for data transmission, suitable for various applications. It has evolved to dominate network speeds of 100G EDR or 200G HDR and is progressing towards even faster speeds like 400G NDR and 800G XDR. InfiniBand adheres to strict latency requirements, approaching zero latency. It excels in applications requiring rapid and precise data processing, commonly used for tasks such as extensive data analysis, machine learning, deep learning training, inference, prediction, and forecasting in supercomputing.
InfiniBand HDR Product Solutions in Supercomputing
NVIDIA GPUs and networking products, particularly Mellanox HDR Quantum QM87xx switches and BlueField DPUs, have established a dominant position in the interconnect of over two-thirds of supercomputers.
InfiniBand HDR Switch
NVIDIA offers two types of InfiniBand HDR switches, namely the HDR CS8500 modular chassis switch and the QM87xx series fixed switches. The 200G HDR QM87xx switches come in two models: MQM8700-HS2F and MQM8790-HS2F. CQM8700 and QM8790 switches typically serve two connection applications. One is to connect with a 200G HDR network interface card (NIC) using 200G-to-200G AOC/DAC cables for direct connection.
Another common application is to connect with a 100G HDR NIC, requiring the use of a 200G-to-2X100G cable to split one physical 200G (4X50G) QSFP56 port of the switch into two virtual 100G (2X50G) ports. After splitting, the port symbols change from x/y to x/Y/z, where “x/Y” represents the original symbol of the port before splitting, and “z” represents the port number (1,2) of the single-channel port, with each sub-physical port considered as a separate port.
InfiniBand HDR Network Interface Cards (NICs)
Compared to HDR switches, HDR Network Interface Cards (NICs) come in various types. In terms of speed, there are two options: HDR100 and HDR. In addition to the data rates for each interface, NICs of each speed can also be selected based on business requirements for single-port, dual-port, and PCIe types.
The HDR InfiniBand network architecture is straightforward yet offers a range of hardware options. For 100Gb/s speed, there are solutions like 100G EDR and 100G HDR100. At 200Gb/s speed, options include HDR and 200G NDR200. There are significant differences in the switches, NICs, and accessories used in various applications.
conclusion
InfiniBand high-performance HDR and EDR switches, Smart NIC cards, as well as solutions combining NADDOD/Mellanox/Cisco/HPE AOC & DAC & optical module products, provide more advantageous and valuable optical network products and comprehensive solutions for applications such as data centers, high-performance computing, and edge computing. This significantly enhances customers’ business acceleration capabilities while offering low cost and excellent performance.
Click to read more related content: Exploring InfiniBand Network, HDR and Significance of IB Applications in Supercomputing | FS Communi
Advantages and Applications of 2x200G HDR Splitter | FS Community
