Emulex 10GbE Virtual Fabric Adapter and Virtual Fabric Adapter Advanced for IBM BladeCenter (Withdrawn)

IBM Redbooks Product Guide

Lenovo is open and ready for business with System x
IBM x86 products are now products of Lenovo in the U.S. and other countries. IBM will continue to host x86-related content on ibm.com until migrated to Lenovo.
Learn more about the acquisition

Abstract

The Emulex 10GbE Virtual Fabric Adapter for IBM BladeCenter is a dual-port 10 Gb Ethernet card that supports 1 Gbps or 10 Gbps traffic, or up to eight virtual NIC devices. The virtual NICs are configured to meet the client's mix of network connectivity and throughput demands for today's complex server application environments.

Note: These products have now been withdrawn from marketing and are no longer available for ordering from IBM.

Contents


The Emulex 10GbE Virtual Fabric Adapter for IBM BladeCenter is a dual-port 10 Gb Ethernet card that supports 1 Gbps or 10 Gbps traffic, or up to eight virtual NIC devices. The virtual NICs are configured to meet the client's mix of network connectivity and throughput demands for today's complex server application environments. Each physical 10 Gbps port can be divided into four virtual ports with bandwidth allocation in 100 Mbps increments to the maximum 10 Gbps per physical port. The Emulex 10GbE Virtual Fabric Adapter Advanced adds iSCSI hardware initiator functionality.

Emulex Virtual Fabric Adapter (CFFh) for IBM BladeCenter
Figure 1. Emulex 10GbE Virtual Fabric Adapter for IBM BladeCenter

Did you know

This adapter is based on industry-standard PCIe architecture and offers the flexibility to operate as a Virtual NIC Fabric Adapter or as a dual-port 10 Gb or 1 Gb Ethernet device. Because this adapter supports up to eight virtual NICs on a single dual-port Ethernet adapter, you see benefits in cost, power/cooling, and data center footprint by deploying less hardware. The adapter enables you to configure up to 12 NIC devices on a single blade server--eight virtual NICs plus the two onboard NICs plus two from a CIOv expansion card.

The Advanced Adapter is enhanced with the addition of iSCSI protocol support. FCoE support is also planned, delivering a Universal Converged Network Adapter with a future-proof connectivity solution for IBM BladeCenter.


Part number information

Table 1. Ordering part number and feature code
DescriptionPart numberFeature code
Emulex 10GbE Virtual Fabric Adapter for IBM BladeCenter49Y42355755
Emulex 10GbE Virtual Fabric Adapter Advanced for IBM BladeCenter49Y42752435
Emulex 10GbE Virtual Fabric Advanced Upgrade for IBM BladeCenter49Y42652436

The adapter part number includes the following items:
  • One Emulex 10GbE Virtual Fabric Adapter or Emulex 10GbE Virtual Fabric Adapter Advanced
  • Documentation CD
  • Important Notices flyer

The standard Emulex 10GbE Virtual Fabric Adapter can be upgraded to the same features as the Emulex 10GbE Virtual Fabric Adapter Advanced with the addition of the Emulex 10GbE Virtual Fabric Advanced Upgrade option, part number 49Y4265.


Features

Data center managers today are looking to increase hardware utilization to meet the growing needs of their organization and virtualization offers a very attractive value proposition. The next-generation IBM BladeCenter HS22 servers offer the platform with balanced processor and memory to enable them to create more VMs per server, but I/O is already becoming a bottleneck.

The IBM BladeCenter virtual fabric offering addresses these issues and extends these value propositions to non-virtualized environments. This solution enables clients to get up to eight virtual ports that can be tailored to meet the bandwidth requirements of the application. The key components for this offering include the 2-port 10 Gb Virtual Fabric Adapter by Emulex and the Virtual Fabric 10Gb Switch Module from BNT.

This solution makes use of Converged Enhanced Ethernet (CEE) and can significantly help reduce complexity and cost in virtualized environments by reducing switch costs up to 75%, reducing cable and adapter costs up to 50%, and achieving better energy efficiency. At the same time, the virtual fabric solution gives clients the opportunity to allocate and adjust bandwidth based on actual requirements and have investment protection for next-generation technology like I/O convergence.

The Emulex 10GbE Virtual Fabric Adapter has the following summary of features:
  • Connection to either 1 Gb or 10 Gb data center infrastructure (1 Gb and 10 Gb auto-negotiation)
  • PCI Express 2.0 x8 host interface
  • IBM BladeCenter CFFh form factor; can be combined with a CIOv expansion card in the same server
  • Operates either as an 8-port Virtual Fabric Adapter or as a 2-port 1/10 Gb Ethernet adapter
    • Virtual port bandwidth allocation in 100 Mbps increments
    • Supports up to eight virtual ports
  • Wake On LAN support
  • Full-duplex (FDX) capability
  • Bus-mastering support
  • Direct memory access (DMA) support
  • Preboot Execution Environment (PXE) support
  • IPv4/IPv6 TCP, UDP checksum offload
    • Large send offload (LSO)
    • Large receive offload
    • Receive side scaling (RSS)
    • IPV4 TCP Chimney Offload
  • VLAN insertion and extraction
  • Jumbo frames up to 9000 bytes
  • Load balancing and failover support including adapter fault tolerance (AFT), switch fault tolerance (SFT), adaptive load balancing (ALB), teaming support, and IEEE 802.3ad
  • Enhanced Ethernet (draft)
    • Enhanced Transmission Selection (ETS) (P802.1Qaz)
    • Priority-based Flow Control (PFC) (P802.1Qbb)
    • Data Center Bridging Capabilities eXchange Protocol, CIN-DCBX and CEE-DCBX (P802.1Qaz)
  • iSCSI hardware initiator support with Advanced card or Advanced Upgrade
  • FCoE support planned with Advanced card or Advanced Upgrade
  • Configurable from the BNT Virtual Fabric 10Gb Switch Module


Modes of operation

The expansion card has two modes of operation: standard physical port mode (pNIC) and virtual NIC (vNIC) mode.
  • In vNIC mode, each physical port appears to the blade server as four virtual NICs. The default bandwidth for each vNIC is 2.5 Gbps. Bandwidth for each vNIC can be configured from 100 Mbps to 10 Gbps, up to a maximum of 10 Gb per virtual port. vNICs can also be configured to have 0 bandwidth if you must allocate the available bandwidth to fewer than eight vNICs. To support vNIC mode operation, the expansion card must communicate with the BNT Virtual Fabric 10Gb Switch Module (part number 46C7191), which provides independent control for each vNIC.

    Two of the eight vNIC ports can be iSCSI on the Emulex 10GbE Virtual Fabric Adapter Advanced or on the standard Emulex 10GbE Virtual Fabric Adapter with the Emulex 10GbE Virtual Fabric Advanced Upgrade.

  • In pNIC mode, the expansion card can operate as a standard 10 Gbps or 1 Gbps 2-port Ethernet expansion card. When in pNIC mode, the expansion card functions with any available 10 Gb switch or 10 Gb pass-thru module installed in I/O module bays 7 and 9.

    In pNIC mode, the Emulex 10GbE Virtual Fabric Adapter Advanced or the standard Emulex 10GbE Virtual Fabric Adapter with the Emulex 10GbE Virtual Fabric Advanced Upgrade can operate as two physical iSCSI ports.



Supported servers

The Emulex 10GbE Virtual Fabric Adapter and Emulex 10GbE Virtual Fabric Adapter Advanced are supported in the IBM BladeCenter servers listed in Table 2.

Table 2. Supported servers


Expansion card

Part number

HS12

HS21

HS21 XM

HS22

HS22V

HX5

LS21

LS22

LS41

LS42

JS12

JS21

JS22

JS23/JS43

PS700/PS701/PS702

Emulex 10GbE Virtual Fabric Adapter

49Y4235

N

N

N

Y

Y

Y

N

N

N

N

N

N

N

N

N

Emulex 10GbE Virtual Fabric Adapter Advanced

49Y4275

N

N

N

Y

Y

Y

N

N

N

N

N

N

N

N

N

Figure 2 shows where the CFFh card is installed in a BladeCenter server.

Location on the BladeCenter server planar where the CFFh card is installed
Figure 2. Location on the BladeCenter server planar where the CFFh card is installed


Supported BladeCenter chassis and I/O modules

IBM BladeCenter chassis support is based on the blade server type in which the expansion card is installed. Consult the ServerProven Web site to see which chassis each blade server type is supported in: http://ibm.com/servers/eserver/serverproven/compat/us/.

Table 3 lists the supported chassis and I/O module combinations that the Emulex 10GbE Virtual Fabric Adapter supports, in either standard physical port mode (pNIC) or virtual NIC (vNIC) mode.

Table 3. I/O modules supported with the Emulex 10GbE Virtual Fabric Adapter

Description

Part number

BladeCenter S

BladeCenter E

BladeCenter H

BladeCenter T

BladeCenter HT

MSIM

MSIM-HT

vNIC mode









BNT Virtual Fabric 10Gb Switch Module

46C7191

N

N

Y

N

Y

N

N

BNT 6-port 10 Gb High Speed Switch Module

39Y9267

N

N

N

N

N

N

N

10Gb Ethernet Pass-Thru Module

46M6181

N

N

N

N

N

N

N

Cisco Nexus 4001I Switch Module

46M6071

N

N

N

N

N

N

N

Brocade Converged 10GbE Switch Module

69Y1909

N

N

N

N

N

N

N

pNIC mode









BNT Virtual Fabric 10Gb Switch Module

46C7191

N

N

Y

N

Y

N

N

BNT 6-port 10 Gb High Speed Switch Module

39Y9267

N

N

Y

N

Y

N

N

10Gb Ethernet Pass-Thru Module

46M6181

N

N

Y

N

Y

N

N

Cisco Nexus 4001I Switch Module

46M6071

N

N

Y

N

Y

N

N

Brocade Converged 10GbE Switch Module

69Y1909

N

N

N

N

N

N

N

In BladeCenter H, the ports of CFFh cards are routed through the midplane to I/O bays 7, 8, 9, and 10, as shown in Figure 3. The BladeCenter HT is similar in that the CFFh cards are also routed through the midplane to I/O bays 7, 8, 9, and 10.

IBM BladeCenter H I/O topology showing the I/O paths from CFFh expansion cards
Figure 3. IBM BladeCenter H I/O topology showing the I/O paths from CFFh expansion cards

The Emulex 10GbE Virtual Fabric Adapter and Emulex 10GbE Virtual Fabric Adapter Advanced require that two I/O modules be installed in bays 7 and 9 of the BladeCenter H and HT chassis (Table 4).

Table 4. Locations of I/O modules required to connect to the expansion card
Expansion cardI/O bay 7I/O bay 8I/O bay 9I/O bay 10
Emulex 10GbE Virtual Fabric Adapter Supported I/O moduleNot usedSupported I/O moduleNot used
Emulex 10GbE Virtual Fabric Adapter Advanced Supported I/O moduleNot usedSupported I/O moduleNot used

MAC addresses are assigned to the expansion card. In Virtual Fabric Adapter mode (vNIC mode) each of the eight virtual ports is assigned a MAC address. In 2-port 10 Gb Ethernet mode (pNIC mode), each of the two physical ports is assigned a MAC addresses.


Popular configurations

This section shows a configuration that uses the Emulex 10GbE Virtual Fabric Adapter.

Figure 4 shows the use of the Emulex 10GbE Virtual Fabric Adapter to route either two physical ports or eight virtual ports to the BNT Virtual Fabric 10Gb Switch Modules installed in the BladeCenter chassis. Two BNT Virtual Fabric 10Gb Switch Modules are installed in bay 7 and bay 9 of the BladeCenter H chassis. All connections between the controller, card, and the switch modules are internal to the chassis. No cabling is needed.

A 20 Gb solution using two BNT 10-port 10 Gb Ethernet Switch Modules
Figure 4. Configuration using the Emulex 10GbE Virtual Fabric Adapter

Depending on how you configure the switch and expansion card, you can get either:
  • Eight virtual Ethernet ports (Virtual Fabric Adapter or vNIC mode). Each virtual port can be configured to operate at 100 Mbps to 10 Gbps, up to a total maximum of 10 Gb per physical port. The default bandwidth is 2.5 Gb per virtual port.
  • Two 1 Gb or 10 Gb Ethernet ports (physical port or pNIC mode).

The components used in this configuration are listed in Table 5.

Table 5. Components used when connecting Emulex 10GbE Virtual Fabric Adapter to two BNT Virtual Fabric 10Gb Switch Modules
Diagram referencePart number/ machine type DescriptionQuantity
17870IBM BladeCenter HS221 to 14
249Y4235Emulex 10GbE Virtual Fabric Adapter 1 per server
38852 or 8740/8750BladeCenter H or BladeCenter HT1
446C7191BNT Virtual Fabric 10Gb Switch Module2
544W4408IBM 10GBase-SR SFP+ TransceiverUp to 20*
* To communicate outside of the chassis, you must have either one SFP+ transceiver or SFP+ direct-attached cable (DAC) for each of the external 10 Gb ports on the BNT 10Gb High Speed Switch Module that you wish to enable.


Operating environment

This adapter complies with temperature rating requirements for BladeCenter H and BladeCenter HT chassis and is supported in the following environment:
  • Temperature - Operating:
    • 10 to 52° C (50 to 125° F) at an altitude of 0 to 914 m (0 to 3,000 ft)
    • 10 to 49° C (50 to 120° F) at an altitude of 0 to 3,000 m (0 to 10,000 ft)
  • Temperature - Non-operating:
    • 40 to 65°C (-40 to 149° F) at an altitude of 0 to 12,000 m (0 to 39,370 ft)
  • Humidity
    • Operating: 8 to 80%, noncondensing
    • Non-operating: 5 to 80%, noncondensing


Operating system support

The Emulex 10GbE Virtual Fabric Adapter supports the following operating systems:
  • Microsoft Windows Server 2003/2003 R2, Datacenter Edition
  • Microsoft Windows Server 2003/2003 R2, Datacenter x64 Edition
  • Microsoft Windows Server 2003/2003 R2, Enterprise Edition
  • Microsoft Windows Server 2003/2003 R2, Enterprise x64 Edition
  • Microsoft Windows Server 2003/2003 R2, Standard Edition
  • Microsoft Windows Server 2003/2003 R2, Standard x64 Edition
  • Microsoft Windows Server 2008, Datacenter x86 Edition
  • Microsoft Windows Server 2008 R2
  • Microsoft Windows Server 2008, Datacenter x64 Edition
  • Microsoft Windows Server 2008, Enterprise x86 Edition
  • Microsoft Windows Server 2008, Enterprise x64 Edition
  • Microsoft Windows Server 2008, Standard x86 Edition
  • Microsoft Windows Server 2008, Standard x64 Edition
  • SUSE LINUX Enterprise Server 10 for x86
  • SUSE LINUX Enterprise Server 10 for AMD64/EM64T
  • SUSE LINUX Enterprise Server 11 for x86
  • SUSE LINUX Enterprise Server 11 for AMD64/EM64T
  • Red Hat Enterprise Linux 5 Server Edition
  • Red Hat Enterprise Linux 5 Server x64 Edition
  • VMware ESX 3.5

Support for operating systems is based on the combination of the expansion card and the blade server on which it is installed. See the IBM ServerProven Web site for the latest information about the specific versions and service packs supported: http://ibm.com/servers/eserver/serverproven/compat/us/. Select the blade server, and then select the expansion card to see the supported operating systems.

Related publications

For more information refer to the following resources:

Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment. publib-b.boulder.ibm.com

Profile

Publish Date
19 October 2009

Last Update
26 October 2010


Rating:
(based on 2 reviews)


Author(s)

IBM Form Number
TIPS0748