More options
Abstract
Over the past 10 years, InfiniBand networks have become the preferred means for interconnecting high performance computing (HPC) resources. The QLogic 12200 and 12300 are a 36-port Quad Data Rate (QDR, 40 Gbps) InfiniBand switches designed to cost-effectively link workgroup resources into a cluster or provide an edge switch option for a larger fabric. The 12200 is fixed configuration, externally managed InfiniBand switch, and 12300 is modular, internally or externally managed InfiniBand switch.
Contents
Over the past 10 years, InfiniBand networks have become the preferred means for interconnecting high performance computing (HPC) resources. The QLogic 12200 and 12300 are 36-port Quad Data Rate (QDR, 40 Gbps) InfiniBand switches designed to cost-effectively link workgroup resources into a cluster or provide an edge switch option for a larger fabric. The 12200 is a fixed configuration, externally managed InfiniBand switch; the 12300 is a modular, internally or externally managed InfiniBand switch.
Both switches are part of the 12000 Series of products that delivers an unmatched set of high speed networking features and functions.
Figure 1 shows the rear of the 12200 switch. The 12300 is very similar in port layout.
Figure 1. QLogic 12200 QDR InfiniBand switch
Did you know?
QLogic 12200 and 12300 InfiniBand switch series are part of IBM Systems Cluster 1350 solution. The Cluster 1350 is your key to a fully integrated HPC solution. IBM clustering solutions include servers, storage, and industry-leading OEM interconnects that are factory-integrated, fully tested, and delivered to your door, ready to plug into your data center, all with a single point of contact for support.
Part number information
Table 1 shows the part numbers to order these switches.
Table 1. IBM part numbers for ordering
| Description | IBM part number |
| QLogic 12200 36-port QDR InfiniBand Switch Bundle | 0449-019 |
| QLogic 12200 (iDPx) 36-port QDR InfiniBand Switch Bundle | 0449-020 |
| QLogic 12300 36-port QDR InfiniBand Switch Bundle | 0449-021 |
The QLogic 12200 and 12300 QDR InfiniBand edge switches are based on QLogic TrueScale ASIC technology and deliver the next evolution in switching for high performance computing (HPC). The available models are tailored for the requirements of the solution:
- The QLogic 12200 Fixed Configuration switch for the Cluster 1350 (0449-019) provides 36 QDR ports in a compact 1U 14.5-inch deep fixed form factor, making it the ideal edge switch for large HPC fabric deployments.
- The QLogic 12200 Fixed Configuration switch for iDataPlex (0449-020) provides 36 QDR ports in a compact 1U 14.5-inch deep fixed form factor. It includes AC power entry, air flow, and mounting hardware tailored for IBM iDataPlex, making it the ideal embedded or edge switch for iDataPlex solutions deploying InfiniBand.
- The QLogic 12300 Configurable switch for Cluster 1350 (0449-021) is an ideal edge switch for enterprise clustering. It provides 36 QDR ports in a standard 1U 26-inch deep form factor and includes a management processor supporting chassis management and an embedded InfiniBand Subnet Manager. The highly available 12300 design is built around state-of-the-art fault detection and recovery capabilities. It ships with redundant hot-swap power and cooling modules.
Benefits
The QLogic 12200 and 12300 InfiniBand switches offer the following benefits:
- Low latency: The QLogic 12200 and 12300 provide scalable, predictable low latency, even at 90% traffic use. Predictable latency means HPC applications can be scaled easily without worrying about diminished cluster performance or costly system tuning efforts.
- Flexible partitioning: The QLogic 12200 and 12300 advanced design is based on an architecture that provides comprehensive virtual fabric partitioning capabilities that enable the InfiniBand fabric to support the evolving requirements of an organization. The TrueScale architecture, together with IFS, allows the fabric to be shared by mission-critical applications while delivering maximum bandwidth utilization.
- Investment protection: The 12000 series of switch products adhere to the IBTA version 1.2 specification, ensuring the ability to interoperate with all other IBTA compliant devices.
- Highly reliable: The highly available 12200 and 12300 design is built around state of the art fault detection and recovery capabilities. In addition, 12300 ships with hot swappable, redundant power and cooling modules.
- Easy to manage: The 12200 takes advantage of QLogic’s advanced InfinBand Fabric Suite Software to facilitate quicker installation and configuration. IFS has advanced tools to verify fabric configuration, topology and performance. Faults are automatically isolated to the component level and reported. In addition, an optional embedded capability of 12300 can be used for switch management.
- Power optimized: Maximum performance is delivered with minimal power and cooling requirements as part of QLogic’s Star Power commitment to developing green solutions for the data center.
Features and specifications
The QLogic 12200 and 12300 InfiniBand switches include the following features and functions:
- 36 ports of InfiniBand QDR (40Gbps) performance with support for DDR and SDR
- 40/20/10-Gbps auto-negotiation links
- Supports Quad Small Form Factor Pluggable (QSFP) optical and copper cable specifications
- TrueScale architecture, with scalable, predictable low latency
- 2.88-Tbps aggregate bandwidth
- Switching latency: < 140 ns
- Multiple Virtual Lanes (VLs) per physical port
- Virtual lanes: 8 plus 1 management
- Maximum MTU size: 4096 bytes
- Maximum multicast table size: 1024 entries
- Supports virtual fabric partitioning
- External chassis management via optional InfiniBand Fabric Suite (IFS) management solution, which provides an expanded set of fabric views and fabric tools.
- Complies with InfiniBand Trade Association (IBTA) version 1.2 standard
The QLogic 12200 InfiniBand switch supports the following management methods:
- Optional external server-based InfiniBand compliant subnet manager
- IBTA compliant SMA and PMA
The QLogic 12300 InfiniBand switch supports the following management methods:
- Command line interface
- Optional external server-based InfiniBand compliant subnet manager
- Optional embedded fabric management
- IBTA compliant SMA,PMA, and BMA
- Chassis management GUI
- SNMP support
- Access methods:
- 10/100 Ethernet Base-T
- Serial port (RS-232 with DB9)
The following LEDs are located on QLogic 12200 InfiniBand switches:
- One per InfiniBand port
- Two for InfiniBand switch status
The following LEDs are located on QLogic 12300 InfiniBand switches:
- One per InfiniBand port
- One for 10/100 Ethernet interface
- Two for InfiniBand switch status
Supported System x and BladeCenter servers, adapters, and cables
The QLogic 12200 and 12300 InfiniBand switches support IBM System Cluster 1350 nodes with InfiniBand adapters installed in the System x or BladeCenter servers. The tables below show core solution components from the IBM System Cluster 1350 Service Component Guide Release 9A Version 1.0.
QDR InfiniBand solution with System x servers and QLogic 12200 or 12300 series switches
Table 2 shows core components to create a full-speed QDR (40 Gbps) InfiniBand solution using QLogic 12200 or 12300 InfiniBand switches and System x servers.
Table 2. QDR InfiniBand solution core components for System x
| Description | IBM part number or machine type |
| InfiniBand Host Channel Adapters | |
| Mellanox ConnectX Single-Port 4X QDR InfiniBand x8 PCI-E 2.0 HCA | 46M2203 |
| Mellanox ConnectX Dual-Port 4X QDR InfiniBand x8 PCI-E 2.0 HCA | 46M2199 |
| Compute Nodes | |
| IBM System x3450 | 7948 |
| IBM System x3550 M2 | 7946 |
| IBM System x3650 M2 | 7947 |
| IBM System x3655 | 7943 |
| IBM System x3455 | 7940, 7941 |
| IBM System x3755 | 7163 |
| IBM System x iDataPlex dx360 M2 | 7321, 7323 |
| Cables | |
| 3m QLogic Optical QDR InfiniBand QSFP Cable | 59Y1920 |
| 10m QLogic Optical QDR InfiniBand QSFP Cable | 59Y1924 |
| 30m QLogic Optical QDR InfiniBand QSFP Cable | 59Y1928 |
DDR InfiniBand (20 Gbps) solution with System x servers and QLogic 12200 or 12300 switches
Table 3 shows core components to create a full-speed DDR (20 Gbps) InfiniBand solution using QLogic 12200 or 12300 InfiniBand switches and System x servers.
Table 3. DDR InfiniBand solution core components for System x
| Description | IBM part number or machine type |
| InfiniBand Host Channel Adapters | |
| Mellanox Single-Port 4X DDR InfiniBand x8PCI-E x8 HCA | 44R9976 |
| Mellanox ConnectX Dual-Port 4X DDR InfiniBand x8 PCI-E 2.0 HCA | 44R8728 |
| Compute Nodes | |
| IBM System x3450 | 7948 |
| IBM System x3550 M2 | 7946 |
| IBM System x3650 M2 | 7947 |
| IBM System x3655 | 7943 |
| IBM System x3455 | 7940, 7941 |
| IBM System x3755 | 7163 |
| IBM System x iDataPlex dx360 M2 | 7321, 7323 |
| Cables | |
| 3m QLogic Optical DDR InfiniBand QSFP-to-CX4 Cable | 59Y1908 |
| 10m QLogic Optical DDR InfiniBand QSFP-to-CX4 Cable | 59Y1912 |
| 30m QLogic Optical DDR InfiniBand QSFP-to-CX4 Cable | 59Y1916 |
DDR InfiniBand (20 Gbps) solution with BladeCenter servers and QLogic 12200 or 12300 switches
Table 4 shows core components to create a full-speed DDR (20 Gbps) InfiniBand solution using QLogic 12200 or 12300 InfiniBand switches and BladeCenter servers.
Table 4. DDR InfiniBand solution core components for BladeCenter
| Description | IBM part number or machine type |
| InfiniBand Host Channel Adapters | |
| 4X DDR InfiniBand CFFh Expansion Card for BladeCenter | 43W4423 |
| Chaasis and Compute Nodes | |
| IBM BladeCenter H chassis | 8852 |
| IBM BladeCenter HS21 server | 8853 |
| IBM BladeCenter HS21 XM server | 7995 |
| IBM BladeCenter HS22 server | 7870 |
| IBM BladeCenter LS22 server | 7901 |
| IBM BladeCenter LS42 server | 7902 |
| IBM BladeCenter JS22 server | 7998 |
| IBM BladeCenter QS22 server | 0793 |
| Pass-thru Modules and Cables | |
| 4X InfiniBand Pass-thru HSSM for BladeCenter | 43W4419 |
| 3m QLogic Optical DDR InfiniBand QSFP-to-CX4 Cable | 59Y1908 |
| 10m QLogic Optical DDR InfiniBand QSFP-to-CX4 Cable | 59Y1912 |
| 30m QLogic Optical DDR InfiniBand QSFP-to-CX4 Cable | 59Y1916 |
Related publications
For more information, see the following documents:
- IBM US Announcement Letter
http://ibm.com/common/ssi/cgi-bin/ssialias?infotype=dd&subtype=ca&&htmlfid=897/ENUS109-284 - Publications for the Cluster 1350
http://www.ibm.com/support/docview.wss?uid=psg1MIGR-5080504 - QLogic 12000 InfiniBand switch publications
http://www.qlogic.com - QLogic 12200 Series home page
http://www.qlogic.com/Products/HPC_products_infibandswitches12200.aspx - QLogic 12300 Series home page
http://www.qlogic.com/Products/HPC_products_infibandswitches12300.aspx
Special Notices
This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.