High-performance computing (HPC) solutions require high bandwidth, low latency components with CPU offloads to get the highest server efficiency and application productivity. The Mellanox ConnectX-2 VPI Single-port and Dual-port QDR InfiniBand host channel adapters (HCAs) deliver the I/O performance that meets these requirements. Data centers and cloud computing also require I/O services such as bandwidth, consolidation and unification, and flexibility, and the Mellanox HCAs support the necessary LAN and SAN traffic consolidation.
Note: The dual-port adapter, 81Y1535, has now been withdrawn from marketing and is no longer available for ordering from IBM.
High-performance computing (HPC) solutions require high bandwidth, low latency components with CPU offloads to get the highest server efficiency and application productivity. The Mellanox ConnectX-2 VPI Single-port and Dual-port Quad Data Rate (QDR) InfiniBand host channel adapters (HCAs) deliver the I/O performance that meets these requirements. Data centers and cloud computing also require I/O services such as bandwidth, consolidation and unification, and flexibility, and the Mellanox HCAs support the necessary LAN and SAN traffic consolidation.
Figure 1 shows the Mellanox ConnectX-2 VPI Dual-port QDR InfiniBand host channel adapter.
Figure 1. Mellanox ConnectX-2 VPI Dual-port QDR InfiniBand host channel adapter
Did You Know?
Mellanox ConnectX-2 VPI Single-port and Dual-port QDR InfiniBand host channel adapters make it possible for any standard networking, clustering, storage, and management protocol to seamlessly operate over any converged network by using a consolidated software stack. With auto-sense capability, each ConnectX-2 port can identify and operate on InfiniBand, Ethernet, or Data Center Bridging (DCB) fabrics. ConnectX-2 with Virtual Protocol Interconnect (VPI) simplifies I/O system design and makes it easier for IT managers to deploy an infrastructure that meets the challenges of a dynamic data center.
Part number information
Table 1 shows the part numbers and feature codes for the Mellanox ConnectX-2 VPI QDR InfiniBand HCAs.
Table 1. Ordering part numbers and feature codes
|Description||Mellanox part number||IBM part number||IBM Feature Code|
|Mellanox ConnectX-2 VPI Single-port QSFP QDR IB/10GbE PCI-E 2.0 HCA||MHQH19B-XTR||81Y1531||5446|
|Mellanox ConnectX-2 VPI Dual-port QSFP QDR IB/10GbE PCI-E 2.0 HCA||MHQH29B-XTR||81Y1535||5447|
The adapters support the transceivers and direct-attach copper (DAC) twin-ax cables listed in Table 2.
Table 2. Supported transceivers and DAC cables
|Description||IBM Feature Code|
|3 m QLogic Optical QDR InfiniBand QSFP Cable||3731|
|10 m QLogic Optical QDR InfiniBand QSFP Cable||3732|
|30 m QLogic Optical QDR InfiniBand QSFP Cable||3733|
|0.5 m QLogic Copper QDR InfiniBand QSFP 30AWG Cable||3725|
|1 m QLogic Copper QDR InfiniBand QSFP 30AWG Cable||3726|
|3 m QLogic Copper QDR InfiniBand QSFP 28AWG Cable||3727|
|3 m IBM Optical QDR InfiniBand QSFP Cable||5989|
|10 m IBM Optical QDR InfiniBand QSFP Cable||5990|
|30 m IBM Optical QDR InfiniBand QSFP Cable||5991|
The Mellanox ConnectX-2 VPI QDR InfiniBand host channel adapters are part of the IBM Intelligent Cluster solution (formerly IBM Systems Cluster 1350). IBM Intelligent Cluster is your key to a fully integrated HPC solution. IBM clustering solutions include servers, storage, and industry-leading OEM interconnects that are factory-integrated, fully tested, and delivered to your door, ready to plug into your data center, all with a single point of contact for support.
Figure 2 shows the Mellanox ConnectX-2 VPI Single-port QDR InfiniBand host channel adapter.
Figure 2. Mellanox ConnectX-2 VPI Single-port QDR InfiniBand host channel adapter
Features and benefits
The Mellanox ConnectX-2 VPI Single-port and Dual-port QDR InfiniBand host channel adapters have the following features:
ConnectX-2 delivers low latency, high bandwidth, and computing efficiency for performance-driven server and storage clustering applications. Efficient computing is achieved by offloading from the CPU routine activities, which makes more processor power available for the application. Network protocol processing and data movement impacts, such as InfiniBand RDMA and Send/Receive semantics, are completed in the adapter without CPU intervention. Graphical processing unit (GPU) communication acceleration provides additional efficiencies by eliminating unnecessary internal data copies, which significantly reduces application runtime. The ConnectX-2 advanced acceleration technology enables higher cluster efficiency and scalability of up to tens of thousands of nodes.
RDMA over converged Ethernet
ConnectX-2 utilizes the InfiniBand Trade Association's RDMA over Converged Ethernet (RoCE) technology to deliver similar low-latency and high-performance over Ethernet networks. Leveraging Data Center Bridging capabilities, RoCE provides efficient low latency RDMA services over Layer 2 Ethernet. The RoCE software stack maintains existing and future compatibility with bandwidth and latency sensitive applications. With link-level interoperability in the existing Ethernet infrastructure, network administrators can use existing data center fabric management solutions.
Applications utilizing TCP/UDP/IP transport can achieve industry leading throughput over InfiniBand or 10 GbE adapters. The hardware-based stateless offload engines in ConnectX-2 reduce the CPU impact of IP packet transport, allowing more processor cycles to work on the application.
ConnectX-2 with Virtual Intelligent Queuing (Virtual-IQ) technology provides dedicated adapter resources and guaranteed isolation and protection for virtual machines within the server. I/O virtualization with ConnectX-2 gives data center managers better server utilization and LAN and SAN unification while reducing cost, power, and cable complexity.
A consolidated compute and storage network achieves significant cost-performance advantages over multi-fabric networks. Standard block and file access protocols can use InfiniBand RDMA for high-performance storage access. T11 compliant encapsulation (FCoIB or FCoE) with full hardware offload simplifies the storage network while keeping existing Fibre Channel targets.
All Mellanox adapter cards are supported by a full suite of drivers for Microsoft Windows, Linux distributions, VMware, and Citrix XENServer. ConnectX-2 VPI adapters support OpenFabrics-based RDMA protocols and software. Stateless offload is fully interoperable with standard TCP/ UDP/IP stacks. ConnectX-2 VPI adapters are compatible with configuration and management tools from OEMs and operating system vendors.
The adapters have the following specifications:
- Low-profile adapter form factor
- Ports: One or two 40 Gbps InfiniBand interfaces (40/20/10 Gbps auto-negotiation) with QSFP connectors
- ASIC: Mellanox ConnectX-2
- Host interface: PCI Express 2.0 x8 (5.0 GTps)
- Interoperable with InfiniBand or 10G Ethernet switches
- IBTA Specification 1.2.1 compliant
- RDMA, Send/Receive semantics
- Hardware-based congestion control
- 16 million I/O channels
- 256 to 4 KB MTU, 1 GB messages
- Nine virtual lanes: Eight data and one management
Enhanced InfiniBand specifications:
- Hardware-based reliable transport
- Hardware-based reliable multicast
- Extended Reliable Connected transport
- Enhanced Atomic operations
- Fine grained end-to-end quality of server (QoS)
- IEEE 802.3ae 10Gb Ethernet
- IEEE 802.3ad Link Aggregation and Failover
- IEEE 802.1Q, 1p VLAN tags and priority
- IEEE P802.1au D2.0 Congestion Notification
- IEEE P802.1az D0.2 ETS
- IEEE P802.1bb D1.0 Priority-based Flow Control
- Jumbo frame support (10 KB)
- 128 MAC/VLAN addresses per port
Hardware-based I/O virtualization:
- Address translation and protection
- Multiple queues per virtual machine
- VMware NetQueue support
Additional CPU offloads:
- TCP/UDP/IP stateless offload
- Intelligent interrupt coalescence
- Compliant with Microsoft RSS and NetDMA
- Fibre Channel over InfiniBand ready
- Fibre Channel over Ethernet ready
Management and tools:
- Interoperable with third-party subnet managers
- Firmware and debug tools (MFT and IBDIAG)
- MIB, MIB-II, MIB-II Extensions, RMON, and RMON 2
- Configuration and diagnostic tools
- Open MPI, OSU MVAPICH, Intel MPI, MS MPI, and Platform MPI
- TCP/UDP, EoIB, IPoIB, SDP, and RDS
- SRP, iSER, NFS RDMA, FCoIB, and FCoE
The adapters have the following physical specifications (without the bracket):
- Single port: 2.1 in x 5.6 in (54 mm x 142 mm)
- Dual port: 2.7 in. x 6.6 in. (69 mm x 168 mm)
The adapters are supported in the following environment:
Operating temperature: 0 to 55° C
Air flow: 200 LFM at 55° C
Power consumption (typical):
- Single-port adapter: 7.0 W typical
- Dual-port adapter: 8.8 W typical (both ports active)
Power consumption (maximum):
- Single-port adapter: 7.7W maximum for passive cables only; 9.7W maximum for active optic modules
- Dual-port adapter: 9.4W maximum for passive cables only, 13.4W maximum for active optic modules
One year limited warranty. When installed in an IBM System x server, these cards assume your system’s base warranty and any IBM ServicePac® upgrades.
The adapters are supported in the System x servers listed in Table 3.
Table 3. Supported servers in the IBM Intelligent Cluster portfolio
Mellanox ConnectX-2 VPI Single-port QSFP QDR IB/10GbE PCI-E 2.0 HCA
Mellanox ConnectX-2 VPI Dual-port QSFP QDR IB/10GbE PCI-E 2.0 HCA
Supported operating systems
The adapters support the following operating systems:
- SUSE Linux Enterprise Server (SLES) 10 and 11
- Red Hat Enterprise Linux (RHEL) 4, 5.3, 5.4
- Microsoft Windows Server 2003
- Microsoft Compute Cluster Server 2003
- Microsoft Windows Server 2008
- Microsoft Windows HPC Server 2008
- OpenFabrics Enterprise Distribution (OFED)
- OpenFabrics Windows Distribution (WinOF)
- VMware ESX Server 3.5/vSphere 4.0
For more information, refer to these documents:
- Publications for the IBM Intelligent Cluster portfolio:
- IBM Intelligent Cluster home page:
- Mellanox ConnectX-2 VPI Single-port and Dual-port QDR InfiniBand HCA product page:
(Select Products > InfiniBand cards > ConnectX-2.)
This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment. publib-b.boulder.ibm.com