IBM Virtual Fabric 10Gb Switch Module for IBM BladeCenter

IBM Redbooks Product Guide

Abstract

In today’s environment as data center space is constrained and IT budgets get smaller, clients are looking for ways to increase utilization of their hardware. Virtualization offers the perfect platform by allowing clients to implement multiple virtual machines per server. To meet client requirements, BladeCenter offers the right balance of compute and I/O by offering high-performance servers, such as HS23 and next-generation 10Gb networking from IBM System Networking.

Changes in the April 8 update:
* Added 0.5 m and 7 m IBM Passive DAC cables
* Added UFP support
* Added QLogic VFA support
* Added 10Gb Interposer Card support
* Updated documentation links
* Updated QLogic GbE and FC card part number
* Updated QLogic CNA part number

Contents


The IBM Virtual Fabric 10Gb Switch Module for IBM BladeCenter offers the most bandwidth of any blade switch and represents the perfect migration platform for clients who are still at 1 Gb outside the chassis by seamlessly integrating in the existing 1 Gb infrastructure. This is the first 10 Gb switch for IBM BladeCenter that is convergence ready (that is, able to transmit Converged Enhanced Ethernet (CEE) to a Fibre Channel over Ethernet (FCoE) capable top-of-rack switch). This new feature is available with firmware release 6.1 and newer.

In addition, this switch is a key part of the IBM Virtual Fabric offering, which allows clients to form up to eight virtual network interface controllers (vNICs) from one physical adapter (14 vNICs with an adapter plus the onboard 10 GbE controller with the BladeCenter HS23) and to manage them in virtual groups. This switch can be managed via a command-line interface (CLI) or a graphical interface of the switch, providing all the benefits of I/O Virtualization at 10 Gb speeds. Figure 1 shows the switch module.


Figure 1. IBM Virtual Fabric 10Gb Switch Module

Did you know

As virtualization has become a prevalent standard in the data center, more people have been looking for ways to virtualize I/O to reduce cost and complexity while also maximizing I/O resources. Virtual Fabric for IBM BladeCenter provides fast, flexible, easy, and reliable I/O using the IBM Virtual Fabric 10Gb Switch Module and the Emulex Virtual Fabric Adapter, which provides the abilities to carve up an adapter into eight virtual NICs, dynamically allocate bandwidth, manage virtual groups, and so on. With the BladeCenter HS23, a total of 14 vNICs can be configured using the Emulex VFA II for HS23 plus the onboard Emulex controller.

The switch is designed to support a number of different types of configurations from the blade: 1 Gb, 10 Gb, virtual NIC, Converged Enhanced Ethernet (CEE/FCoE), and iSCSI. If you have a chassis with multiple servers, some operating at 1 Gb, some at 10 Gb, and some transmitting converged packets, this single switch can handle all these workloads and can connect to a 1 Gb infrastructure or a 10 Gb infrastructure, or both.


Part number information

Table 1 shows the part numbers for ordering these modules and additional options for them.

Table 1. IBM part numbers and feature codes for ordering

DescriptionIBM part numberIBM feature code
Switch module
IBM Virtual Fabric 10Gb Switch Module for IBM BladeCenter46C71911639
SFP+ transceivers - 10 GbE
IBM 10GBase-SR 10GbE 850 nm Fiber SFP+ Transceiver44W44084942
IBM SFP+ SR Transceiver46C34475053
IBM SFP+ LR Transceiver90Y9412A1PM
IBM SFP+ ER Transceiver90Y9415A1PP
SFP transceivers - 1 GbE
IBM SFP RJ45 Transceiver81Y16183268
IBM SFP SX Transceiver81Y16223269
IBM SFP LX Transceiver90Y9424A1PN
SFP+ direct-attach cables - 10 GbE
0.5m IBM Passive DAC SFP+ Cable00D6288A3RG
1m IBM Passive DAC SFP+ Cable 90Y9427A1PH
3m IBM Passive DAC SFP+ Cable90Y9430A1PJ
5m IBM Passive DAC SFP+ Cable90Y9433A1PK
7m IBM Passive DAC SFP+ Cable00D6151A3RH

The module part number, 46C7191, include the following items:
  • One IBM Virtual Fabric 10Gb Switch Module
  • 3-meter mini-USB-to-DB9 serial console cable
  • One filler module
  • IBM Virtual Fabric 10Gb Switch Module Installation Guide
  • IBM BladeCenter user license agreement
  • Important Notices document
  • Documentation CD-ROM

Note: Transceivers and cables are not included with the switch module. (See Table 1 for options.)

With the flexibility of the switch, clients can take advantage of the technologies they require for multiple environments. For 1 Gb uplinks, they can take advantage of SFP transceivers. For 10 GbE uplinks, they have a choice of either SFP+ transceivers for longer distances or more cost effective and lower power consuming options such as SFP+ direct-attached copper (DAC) cables, which can be between 0.5 and 7 meters in length and are ideal for connecting chassis together, connecting to a top-of-rack switch, or even connecting to an adjacent rack.


Benefits

The IBM Virtual Fabric 10Gb Switch Module offers the following benefits:
  • Integration and consolidation: This switch offers integration within the BladeCenter chassis, consolidating full Layer 2-3 LAN switching and routing capabilities into a single chassis. This helps flatten the data center infrastructure and reduces the number of discrete devices, management consoles, and equipment that administrators must deal with, helping lower costs and simplifying deployment.
  • Cost: The switch is priced extremely competitively compared with external switches, especially when you factor in that no cables are required between the blade and the switch. In addition, the ability to use direct-attach cables with the switch can help clients save even more compared with the more expensive CX4, XFP, or X2 transceivers.
  • Performance: With support for ten 10 Gb uplinks, clients can exploit not only up to 200 Gbps of bi-directional uplink bandwidth, but also an extremely low oversubscription (14 to 10), which can support even the most performance-intensive environments (up to 7.2 Gbps per blade server port). Clients wanting extreme performance can use up to four switches and the quad port 10 Gb adapter and get up to 1.92 Tbps of data per BladeCenter H chassis.
  • Lower power consumption: Using only 75 W per switch delivers extreme performance per watt, which is second to none with its support for up to 6.4 Gbps per watt of power.
  • Layer 3 functionality: The switch module includes Layer 3 functionality, which provides security and performance benefits as inter-VLAN traffic stays within the chassis. This switch also provides the full range of Layer 3 protocols from static routes for technologies such as Open Shortest Path First (OSPF) and Border Gateway Protocol (BGP) for enterprise customers.
  • Interoperability: IBM switches interoperate seamlessly with other vendors' upstream switches.
  • Management: IBM provides a choice of interfaces that IT staff can use for management which includes IBM System Networking graphical user interface or an industry-based CLI (Cisco-like) for those familiar with IOS, and a full-function Web-based GUI for the latest in simplicity.
  • Fault tolerance: These switches learn alternate routes automatically and perform faster convergence in the unlikely case of a link, switch, or power failure. The switch uses proven technologies like L2 trunk failover, advanced VLAN-based failover, VRRP, IGMP V3 snooping, and OSPF.
  • Converged fabric: The switch is designed to support CEE and connectivity to FCoE modules, making the IBM solution fully FCoE capable. CEE will help enable clients to combine storage, messaging traffic, VoIP, video, and other data on a common data center Ethernet infrastructure. FCoE will help enable highly efficient block storage over Ethernet for consolidating server network connectivity. As a result, clients can deploy a single server interface for multiple data types, which can simplify both deployment and management of server network connectivity, while maintaining the high availability and robustness required for storage transactions.

    With the combination of the switch and the QLogic Virtual Fabric Extension Module for IBM BladeCenter, IBM is able to deliver a full integrated FCoE blade solution that appeals to those clients who require separate LAN and SAN traffic outside the chassis due to internal politics or for those wanting to implement FCoE but are extremely cost-sensitive. For information about the QLogic Virtual Fabric Extension Module, see:

    http://www.redbooks.ibm.com/abstracts/tips0717.html

    In addition IBM will also support CEE/DCB which allows clients to forward the FCoE traffic upstream to a Top-of-Rack Gateway device that will split out the LAN traffic (Ethernet) and the SAN (Fiber Channel) traffic. For the latest official FCoE gateway devices that are supported visit:

    http://www.ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5073016.


The IBM switch comes with a one year warranty or picks up the warranty of the chassis whichever is longer. The software license will match the warranty and allows for the download of new features, such as:
  • Virtual Fabric for IBM BladeCenter: This feature enables clients to implement and manage the Virtual Fabric solution via a graphical interface or CLI. It is through this that administrators can perform tasks such as activating a Virtual Fabric, selecting the number of virtual NICs (vNICs), dynamically allocating bandwidth, creating virtual groups to simplify management, and setting up the failover and isolation capabilities to the vNICs for better availability and security. This switch also offers the benefit of IBM’s next-generation vNIC - Unified Fabric Port (UFP). UFP is an advanced, cost-effective solution that provides a flexible way for clients to allocate, reallocate, and adjust bandwidth to meet their ever-changing data center requirements.
  • VMready: VMready is a unique solution that enables the network to be virtual machine-aware. The network can be configured and managed for virtual ports (v-ports) rather than just for physical ports. With VMready, as VMs migrate across physical hosts, so do their network attributes. Virtual machines can be added, moved, and removed while retaining the same ACLs, QoS, and VLAN attributes. VMready allows for a define-once-use-many configuration that evolves as the server and network topologies evolve. VMready works with all virtualization products, including VMware, Hyper-V, Xen, and KVM, without modification of virtual machine hypervisors or guest operating systems. Available as part of the V6.1 or later software code.

UFP support: UFP is supported using Emulex 1212 code on VFA II with HS22 blade servers. UFP support for HS23 blade servers is planned for 2Q 2014.

Features and specifications

The IBM Virtual Fabric 10Gb Switch Module includes the following features and functions:
  • Form-factor
    • Single-wide high-speed switch module
  • Internal ports
    • 14 internal auto-negotiating ports: 1 Gb or 10 Gb to the server blades
    • Two internal full-duplex 100 Mbps ports connected to the management module
  • External ports
    • Up to ten 10 Gb SFP+ ports (also designed to support 1 Gb SFP if required, flexibility of mixing 1 Gb/10 Gb)
    • One 10/100/1000 Mb copper RJ-45 used for management or data
    • An RS-232 mini-USB connector for serial port that provides an additional means to install software and configure the switch module
  • Scalability and performance
    • Stacking support up to 8 switches
    • Autosensing 1 Gb/10 Gb internal and external Ethernet ports for bandwidth optimization
    • Non-blocking architecture with wire-speed forwarding of traffic and full line rate performance of 480 Gbps full duplex
    • Media access control (MAC) address learning: automatic update, supports up to 32 Kb MAC addresses
    • Up to 128 IP interfaces per switch
    • Static, EtherChannel, and LACP (IEEE 802.3ad) link aggregation, up to 100 Gb of total bandwidth per switch, up to 18 trunk groups, and up to eight ports per group
    • Support for jumbo frames (up to 12288 bytes)
    • Broadcast/multicast storm control
    • IGMP snooping for limit flooding of IP multicast traffic (IGMP V1, V2, and V3)
    • Configurable traffic distribution schemes over trunk links based on source/destination IP addresses, MAC addresses, or both
    • Fast port forwarding and fast uplink convergence for rapid STP convergence
  • Availability and redundancy
    • VRRP for Layer 3 router redundancy
    • IEEE 802.1D STP for providing Layer 2 redundancy with PVRST+
    • IEEE 802.1s Multiple STP (MSTP) for topology optimization, up to 128 STP instances supported by single switch
    • IEEE 802.1w Rapid STP (RSTP), providing rapid STP convergence for critical delay-sensitive, traffic-like voice or video
    • Layer 2 Trunk Failover to support active/standby configurations of network adapter teaming on blades
    • Interchassis redundancy (Layer 2 and Layer 3)
  • Converged Enhanced Ethernet
    • CEE support and connectivity to FCoE
    • Connects to a top-of-rack switch for connectivity to a Fibre Channel SAN
    • Supports the QLogic Virtual Fabric Extension Module for IBM BladeCenter, which provides FCoE gateway functionality inside the BladeCenter Chassis.
  • VLAN support
    • Up to 1024 VLANs supported per switch; VLAN numbers ranging from 1 to 4095 (4095 is used for the management module’s connection only.)
    • 802.1Q VLAN tagging support on all ports
    • Protocol-based VLANs
  • Security
    • VLAN-based, MAC-based, and IP-based access control lists (ACLs)
    • 802.1X port-based authentication
    • Multiple user IDs and passwords
    • User access control
    • Radius, TACACS+, LDAP
  • Quality of service (QoS)
    • Up to eight queues per port
    • Support for IEEE 802.1p, IP ToS/DSCP, and ACL-based (MAC/IP source and destination addresses, VLANs) traffic classification and processing
    • Traffic shaping and re-marking based on defined policies
    • Eight weighted round robin (WRR) priority queues per port for processing qualified traffic
  • Layer 3 functions
    • IP forwarding
    • IP filtering with ACLs (up to 4096 ACLs supported)
    • VRRP for router redundancy
    • Support for up to 128 static routes
    • Routing protocol support (Router Information Protocol (RIP) V1; RIP V2; OSPF V1, V2, and V3; BGP-4), up to 1024 entries in routing table
    • Support for DHCP Relay
    • IPv6 management
  • Virtualization
    • Virtual NIC (vNIC) support
      • vNIC Virtual Fabric mode
      • vNIC Switch Independent mode
    • Unified fabric port (UFP) support
    • VMready
  • Manageability
    • Simple Network Management Protocol (SNMP, V1, V2, and V3)
    • HTTP/HTTPS browser GUI
    • Industry-standard CLI or menu-driven CLI
    • Telnet interface for CLI
    • SSH V1/V2
    • Serial interface for CLI
    • Scriptable CLI
    • Firmware image update (TFTP and FTP)
    • Network Time Protocol (NTP) for switch clock synchronization
    • IBM System Networking Switch Center (SNSC) support
  • Monitoring
    • Switch LEDs for external port status and switch module status indication
    • Port mirroring for analyzing network traffic passing through switch
    • Change tracking and remote logging with syslog feature
    • POST diagnostics
  • Special functions
    • Serial over LAN (SOL)

The switch module supports the following IEEE standards:
  • IEEE 802.1D STP with PVRST+
  • IEEE 802.1s MSTP
  • IEEE 802.1w RSTP
  • IEEE 802.1p Tagged Packets
  • IEEE 802.1Q Tagged VLAN (frame tagging on all ports when VLANs are enabled)
  • IEEE 802.1x port-based authentication
  • IEEE 802.2 Logical Link Control
  • IEEE 802.3ad Link Aggregation Control Protocol
  • IEEE 802.3x Full-duplex Flow Control
  • IEEE 802.3ab 1000BASE-T Gigabit Ethernet
  • IEEE 802.3ae 10GBASE-SR 10Gb Ethernet fiber optics short range
  • IEEE 802.3ae 10GBASE-LR 10Gb Ethernet fiber optics long range
  • IEEE 802.3z 1000BASE-SX Gigabit Ethernet


Supported BladeCenter chassis and expansion cards

The IBM Virtual Fabric 10Gb Switch Module is supported in the IBM BladeCenter H and HT chassis listed in the following table.

Table 2. IBM BladeCenter chassis that supports the IBM Virtual Fabric 10Gb Switch Module

I/O module

Part number

BladeCenter S

BladeCenter E

BladeCenter H

BladeCenter T

BladeCenter HT

MSIM

MSIM-HT

IBM Virtual Fabric 10Gb Switch Module

46C7191

N

N

Y

N

Y

N

N

The following table lists the compatibility information for the IBM Virtual Fabric 10Gb Switch Module and the expansion cards.

Table 3. Expansion card compatibility
DescriptionPart numberFeature
code
Supported by the
Virtual Fabric
10Gb Switch Module
Gigabit Ethernet
Gigabit Ethernet integrated on the server system boardNoneNoneNo
Ethernet Expansion Card (CFFv)39Y93102969No
Ethernet Expansion Card (CIOv)44W44751039No
2/4 Port Ethernet Expansion Card (CFFh)44W44795476Supported
QLogic Ethernet and 8 Gb Fibre Channel Expansion Card00Y3270*A3JCNo
10 Gigabit Ethernet
10Gb Interposer Card for HS23 (uses HS23 Integrated Virtual Fabric LOM)94Y8550A244Supported
Broadcom 10 Gb Gen 2 4-port Ethernet Expansion Card46M61640098Supported
Broadcom 10 Gb Gen 2 2-port Ethernet Expansion Card46M61680099Supported
Broadcom 2-port 10Gb Virtual Fabric Adapter81Y3133A1QRSupported
Brocade 2-port 10GbE Converged Network Adapter81Y16505437Supported
Emulex 10GbE Virtual Fabric Adapter49Y42355755Supported
Emulex 10GbE Virtual Fabric Adapter Advanced49Y42752435Supported
Emulex Virtual Fabric Adapter II (CFFh)90Y3550A1XGSupported
Emulex Virtual Fabric Adapter Advanced II (CFFh)90Y3566AIXHSupported
Emulex 10GbE VFA II for IBM BladeCenter HS2381Y3120A287Supported
Emulex 10GbE VFA Advanced II for IBM BladeCenter HS2390Y9332A2ZNSupported
Intel 2-port 10 Gb Ethernet Expansion Card (CFFh)42C18103593No
Mellanox 10 Gb Ethernet Expansion Card (CFFh)90Y3570A1NWSupported
QLogic 2-port 10Gb Converged Network Adapter (CFFh)00Y3280**A3JBSupported
QLogic 10Gb Virtual Fabric Adapter00Y3332A4ACSupported
QLogic 10Gb Virtual Fabric CNA00Y5618A4ADSupported
* Replaces 44X1940.
** Replaces 42C1830.

The five BladeCenter chassis have the following bays:
  • BladeCenter S, E, and T have four standard I/O bays (1, 2, 3, and 4)
  • BladeCenter H has six standard I/O bays (1, 2, 3, 4), two bridge bays (5 and 6) and four high-speed bays (7, 8, 9, and 10)
  • BladeCenter HT has four standard I/O bays (1, 2, 3, 4) and four high-speed bays (7, 8, 9, and 10).

The IBM Virtual Fabric 10Gb Switch Module fits in one of the high-speed I/O bay (bays 7-10). For each supported adapter, the following table lists which specific chassis I/O bays are supported.

Table 4. IBM Virtual Fabric 10Gb Switch Module and BladeCenter chassis I/O bays support
DescriptionPart number
Bay 1 (Standard)
Bay 2 (Standard)
Bay 3 (Standard)
Bay 4 (Standard)
Bay 5 (Bridge)
Bay 6 (Bridge)
Bay 7 (High-speed)
Bay 8 (High-speed)
Bay 9 (High-speed)
Bay 10 (High-speed)
10Gb Interposer Card for HS2394Y8550
N
N
N
N
N
N
Y
N
Y
N
2/4 Port Ethernet Expansion Card (CFFh)44W4479
N
N
N
N
N
N
Y
Y
Y
Y
Broadcom 10 Gb Gen 2 2-port Ethernet Expansion Card46M6168
N
N
N
N
N
N
Y
N
Y
N
Broadcom 10 Gb Gen 2 4-port Ethernet Expansion Card46M6164
N
N
N
N
N
N
Y
Y
Y
Y
Broadcom 2-port 10Gb Virtual Fabric Adapter81Y3133
N
N
N
N
N
N
Y
N
Y
N
Brocade 2-port 10GbE Converged Network Adapter81Y1650
N
N
N
N
N
N
Y
N
Y
N
Emulex 10GbE Virtual Fabric Adapter49Y4235
N
N
N
N
N
N
Y
N
Y
N
Emulex 10GbE Virtual Fabric Adapter Advanced49Y4275
N
N
N
N
N
N
Y
N
Y
N
Emulex 10GbE Virtual Fabric Adapter II90Y3550
N
N
N
N
N
N
Y
N
Y
N
Emulex 10GbE Virtual Fabric Adapter II Advanced90Y3566
N
N
N
N
N
N
Y
N
Y
N
Emulex 10GbE VFA II for IBM BladeCenter HS2381Y3120
N
N
N
N
N
N
Y
Y
Y
Y
Emulex 10GbE VFA Advanced II for IBM BladeCenter HS2390Y9332
N
N
N
N
N
N
Y
Y
Y
Y
Mellanox 2-port 10Gb Ethernet Expansion Card90Y3570
N
N
N
N
N
N
Y
N
Y
N
QLogic 2-port 10Gb Converged Network Adapter42C1830
N
N
N
N
N
N
Y
N
Y
N
QLogic 10Gb Virtual Fabric Adapter00Y3332
N
N
N
N
N
N
Y
N
Y
N
QLogic 10Gb Virtual Fabric CNA00Y5618
N
N
N
N
N
N
Y
N
Y
N
* These ports are available by routing the onboard Emulex BE3 controller of the HS23 blade server through the VFA card.


Connectors and LEDs

Figure 2 shows the front panel of the IBM Virtual Fabric 10Gb Switch Module.

Front panel of the BNT 10-port 10Gb Ethernet Switch Module
Figure 2. Front panel of the IBM Virtual Fabric 10Gb Switch Module

The front panel contains the following components:
  • LEDs display the status of the switch module and the network:
    • OK: indicates that the switch module has passed the power-on self-test (POST) with no critical faults and is operational
    • Switch-module error: indicates that the switch module has failed the POST or detected an operational fault
  • One USB RS-232 console port provides an additional means to install software and configure the switch module. This USB-style connector enables the connection of a special serial cable that is supplied with the switch module.
  • Ten SFP+ port connectors to attach SFP+ modules.
  • One RJ-45 Ethernet port connector.
  • Each external SFP+ port on the switch module contains a link/activity LED, and an RJ-45 port contains Ethernet link OK and Ethernet Tx/Rx activity LEDs.


Popular configurations

The IBM Virtual Fabric 10Gb Switch Module can be used in various configurations.

Basic two-port configuration

Figure 3 shows the use of IBM Virtual Fabric 10Gb Switch Modules to route two Ethernet ports from Emulex 10GbE Virtual Fabric Adapter II cards installed on each server. Two IBM Virtual Fabric 10Gb Switch Modules are installed in bay 7 and bay 9 of the BladeCenter H chassis. All connections between the controller, card, and switch modules are internal to the chassis. No cabling is needed.

A 20Gb solution using two BNT 10-port 10 Gb Ethernet Switch Modules
Figure 3. A dual 10Gb solution using two IBM Virtual Fabric 10Gb Switch Modules

The following table lists the components used in this configuration.

Table 5. Components used when connecting Emulex 10GbE Virtual Fabric Adapter II cards to two IBM Virtual Fabric 10Gb Switch Modules
Diagram referencePart number/
machine type
DescriptionQuantity
1VariesIBM BladeCenter HS22 or other supported server1 to 14
290Y3550Emulex 10GbE Virtual Fabric Adapter II 1 per server
38852 or 8740/8750BladeCenter H or BladeCenter HT1
446C7191IBM Virtual Fabric 10Gb Switch Module2
590Y9430IBM Passive DAC SFP+ cable, 3MUp to 20*
*The IBM Virtual Fabric 10Gb Switch Module has 10 external 10 Gb ports. To communicate outside of the chassis, you must have either one SFP or SFP+ transceiver or SFP+ direct-attached copper (DAC) cable connected. You have the flexibility to expand your bandwidth as you see fit using from one to 10 connections per switch.

4-port configuration

Figure 4 shows the use of IBM Virtual Fabric 10Gb Switch Modules to route four Ethernet ports from a Broadcom 4-port 10Gb Ethernet Expansion Card (CFFh) installed on each server. Four IBM Virtual Fabric 10Gb Switch Modules are installed in bay 7, bay 8, bay 9, and bay 10 of the BladeCenter H chassis. All connections between the controller, card, and switch modules are internal to the chassis. No cabling is needed.

A 40Gb solution using four BNT 10-port 10Gb Ethernet Switch Modules
Figure 4. A four 10Gb port solution using four IBM Virtual Fabric 10Gb Switch Modules

The following table lists the components used in this configuration.

Table 6. Components used when connecting Broadcom 4-port 10Gb Ethernet Expansion Card (CFFh) to four IBM Virtual Fabric 10Gb Switch Modules
Diagram referencePart number/
machine type
DescriptionQuantity
1VariesIBM BladeCenter HS22 or other supported server1 to 14
244W4465Broadcom 4-port 10Gb Ethernet Expansion Card (CFFh)1 per server
38852 or 8740/8750BladeCenter H or BladeCenter HT1
446C7191IBM Virtual Fabric 10Gb Switch Module4
590Y9430IBM Passive DAC SFP+ cable, 3MUp to 40*
*The IBM Virtual Fabric 10Gb Switch Module has 10 external 10 Gb ports. In order to communicate outside of the chassis you must have either one SFP or SFP+ transceiver or SFP+ DAC cable connected. You have the flexibility to expand your bandwidth as you see fit using from one to 10 connections per switch.

4-port configuration on a BladeCenter HS23

This configuration uses a combination of the onboard Emulex-based 10Gb Ethernet controller of the BladeCenter HS23 plus an Emulex CFFh adapter. Two 10 GbE ports are from the onboard Emulex BE3 controller and two more from the Emulex 10GbE Virtual Fabric Adapter II for HS23, 81Y3120. These four physical ports can be configured as up to 14 vNICs. A further two Gigibit Ethernet ports are also available.

Figure 5 shows the use of four IBM Virtual Fabric 10Gb Switch Modules to route the four 10 GbE ports on the HS23:
  • Two ports from the onboard Emulex BE3 controller HS23 are routed to bays 7 and 9
  • Two ports from the Emulex BE3 controller on the Emulex 10GbE Virtual Fabric Adapter II for HS23 are routed to bays 8 and 10

All connections between the LOM, card and switch modules are internal to the chassis. No cabling is needed.


Figure 5. A four 10Gb port solution with the BladeCenter HS23

The following table lists the components used in this configuration.

Table 7. Components used when enabling 10 GbE ports on an HS23 to four IBM Virtual Fabric 10Gb Switch Modules
Diagram referencePart number/
machine type
DescriptionQuantity
17875IBM BladeCenter HS23, either configure-to-order model (CTO) or a model without the 10 Gb LOM Interposer card1 to 14
281Y3120Emulex 10GbE Virtual Fabric Adapter II for HS231 per server
38852 or 8740/8750BladeCenter H or BladeCenter HT1
446C7191IBM Virtual Fabric 10Gb Switch Module4
590Y9430IBM Passive DAC SFP+ cable, 3MUp to 40*


Network cabling requirements

The network cables that can be used with the switch are shown in the following table.

Table 8. IBM Virtual Fabric 10 GbE switch network cabling requirements
TransceiverStandardCableConnector
10 Gb Ethernet
IBM SFP+ SR Transceiver (46C3447)10GBASE-SR850 nm multimode fiber cable (50 µ or 62.5 µ) up to 300 mLC
IBM SFP+ LR Transceiver (90Y9412)10GBASE-LR1310 nm single-mode fiber cable up to 10 kmLC
IBM SFP+ ER Transceiver (90Y9415)10GBASE-ER1310 nm single-mode fiber cable up to 40 kmLC
10GBase-SR SFP+ (MMFiber) transceiver (44W4408)10GBASE-SR850 nm multimode fiber cable (50 µ or 62.5 µ) up to 300 mLC
Direct attach cable10GSFP+CuSFP+ DAC cables up to 7 m (see Table 1)SFP+
1 Gb Ethernet
IBM SFP RJ-45 Transceiver (81Y1618)1000BASE-TUTP Category 5, 5E, and 6 up to 100 metersRJ-45
IBM SFP SX Transceiver (81Y1622)1000BASE-SX850 nm multimode fiber cable (50 µ or 62.5 µ) up to 550 mLC
IBM SFP LX Transceiver (90Y9424)1000BASE-LX1310 nm single-mode fiber cable up to 10 kmLC
Fixed RJ-45 1 GbE port1000BASE-TUTP Category 5, 5E, and 6 up to 100 metersRJ-45


Related publications

For more information see the following IBM Virtual Fabric 10Gb Switch Module product publications, available from:
http://ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5091835
  • Installation Guide
  • Application Guide
  • Command Reference
  • isCLI Reference
  • BBI (Browser-based Interface) Quick Guide
  • Release Notes
  • End User License Agreement


Other documents:

Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.

Profile

Publish Date
26 May 2009

Last Update
08 April 2014


Rating:
(based on 1 review)


Author(s)

IBM Form Number
TIPS0708