Emulex 10GbE Virtual Fabric Adapter II and III family for IBM System x

IBM Redbooks Product Guide


Abstract

The Emulex 10GbE Virtual Fabric Adapters II and III for IBM® System x® are members of IBM's comprehensive networking portfolio of 10 Gb Ethernet (10GbE) converged network adapters (CNAs). They deliver industry-leading performance and scalability per watt, reducing requirements for power and cooling. Storage protocol offloads enable the efficient use of computing resources and the support of more virtual machines per CPU, and reduce the number of servers required to support data center demands. The adapters support fiber-optic or twin-ax copper cabling to maximize flexibility.

Changes in this update:
* IBM System x3100 M4 and x3250 M4 do not support these adapters

Contents

Table of contents


Introduction

As server virtualization technology becomes more prevalent within data centers, more dynamic performance is needed for network bandwidth to satisfy these demands. The Emulex 10GbE Virtual Fabric Adapters II and III for IBM® System x® can help you break the I/O bottleneck by allowing you to allocate bandwidth where it’s needed, delivering maximum application agility. Offering a full range of virtualization and convergence capabilities, the same network hardware offers Ethernet, iSCSI, or Fibre Channel over Ethernet with bandwidth that is allocated in increments from 100 Mb to 10 Gb. The next-generation Virtual Fabric Adapter for IBM System x is a fast, flexible, easy, and reliable solution for I/O virtualization.

The Emulex 10GbE Virtual Fabric Adapter II and Emulex Dual Port 10GbE SFP+ VFA III are shown in Figure 1 (these adapters look the same).

Emulex 10GbE Virtual Fabric Adapter II (with optional SFP+ transceivers installed)
Figure 1. Emulex 10GbE Virtual Fabric Adapter II and Emulex Dual Port 10GbE SFP+ VFA III (with optional SFP+ transceivers installed)

Did you know

With IBM Virtual Fabric, up to eight virtual network ports (vNICs) can be created with a single two-port 10 GbE network adapter. Converged protocols such as iSCSI and FCoE are also supported on selected configurations. By using a common infrastructure for Ethernet and SAN, and by virtualizing your network adapter, you can reduce your infrastructure capital expense.

Part number information

All adapters in the Emulex 10GbE Virtual Fabric Adapter family listed in Table 1 offer the same features and functions. There is a difference in the manner that they are field-upgradeable from an Ethernet-only-mode-of-operation to an Ethernet, FCoE and iSCSI mode-of-operation:

  • VFA II adapters are designed to use a paper-key license where IBM provides a license security to be used at the Emulex web site once registered. Emulex provides the user with a different license key that is subsequently administered to the server. VFA II adapters are supported on the M2, M3 and X5 generation of IBM System x servers.
  • VFA III adapters provide a much easier and efficient software license enablement process. IBM’s Feature on Demand process reduces the complexity of activating the license to just a few clicks of the mouse. VFA III is supported on the M4 generation of IBM system x Servers.

Customers and system designers need to note that VFA II and VFA III adapters, as well as the subset of VFA III PCIe, Integrated, and Mezzanine form-factors have different license key part numbers.

Table 1. Ordering part number and feature code
DescriptionPart numberFeature code
VFA III adapters for System x M4 servers
Emulex Dual Port 10GbE SFP+ VFA III for IBM System x95Y3762A2U1
Emulex Dual Port 10GbE SFP+ Integrated VFA III for IBM System x*SBB 95Y3768*A2UN
Emulex VFA III FCoE/iSCSI License for IBM System x (for 95Y3762 and SBB 95Y3768)95Y3760A2U2
Embedded VFA III adapters for System x M4 servers (do not consume a regular PCIe slot)
Emulex Dual Port 10GbE SFP+ Embedded Adapter for IBM System x90Y6456A22J
Emulex Embedded VFA III FCoE/iSCSI License for IBM System x (for 90Y6456)90Y5178A2TE
VFA II adapters for System x M3 and X5 servers
Emulex 10GbE Virtual Fabric Adapter II for IBM System x49Y7950A18Z
Emulex 10GbE Integrated Virtual Fabric Adapter II for IBM System xSBB 49Y7940#A148
Emulex VFA II FCoE/iSCSI License for IBM System x (for 49Y7950 and SBB 49Y7940)49Y42745715
Emulex Dual-port VFA II Adapter and FCoE/iSCSI License for IBM System x95Y3751A348
* This adapter can only be ordered via the configure-to-order (CTO) processor or via special bid.
# This adapter comes standard with the most models of the IBM System x3850 X5, x3950 X5 and x3690 X5. It can also can be ordered via CTO or via special bid for those systems only.

The adapters, when shipped as stand-alone options, include the following items:
  • One Emulex 10 GbE Virtual Fabric Adapter
  • 3U bracket attached with 2U bracket included in the box (95Y3762 and 49Y7950 only)
  • Quick Install Guide
  • Warranty information and Important Notices flyer
  • Documentation CD


The Emulex Dual Port 10GbE SFP+ Embedded Adapter is designed to fit into a special slot in selected servers (see Table 4), allowing you to use the PCIe slots for other technologies. The adapter is shown in Figure 2.

Emulex Dual Port 10GbE SFP+ Embedded Adapter
Figure 2. Emulex Dual Port 10GbE SFP+ Embedded Adapter

The Emulex 10GbE Integrated Virtual Fabric Adapter II (Figure 3) has the same functional characteristics as the Emulex 10GbE Virtual Fabric Adapter II (Figure 1). This adapter must be purchased at the same time as the new IBM eX5 server (x3850 X5 and x3690 X5). It is factory installed into a standard PCIe slot, and is offered at a price savings compared to the Emulex 10GbE Virtual Fabric Adapter II . For this special price, there is a limit of one Emulex 10GbE Integrated Virtual Fabric Adapter II per system. The Emulex 10GbE Integrated Virtual Fabric Adapter II is shown in Figure 3.

Emulex 10GbE Integrated Virtual Fabric Adapter II (without optional SFP+ transceivers)
Figure 3. Emulex 10GbE Integrated Virtual Fabric Adapter II (without optional SFP+ transceivers)

Similarly, the Emulex Dual Port 10GbE SFP+ Integrated VFA III has the same functional characteristics as the Emulex Dual Port 10GbE SFP+ VFA III. The integrated adapter must be purchased at the same time as the new supported server (see Table 4). The integrated adapter is factory installed into a standard PCIe slot, and is offered at a price savings compared to the Emulex Dual Port 10GbE SFP+ VFA III. For this special price, there is a limit of one Emulex Dual Port 10GbE SFP+ Integrated VFA III per system.

Supported transceivers and direct-attach cables

The Emulex 10 GbE Virtual Fabric Adapters have two empty SFP+ cages that support SFP+ SR transceivers and twin-ax direct-attached copper cables as listed in Table 2 and Table 3 respectively.

Table 2. Supported transceivers

DescriptionPart numberFeature Code
Brocade 10Gb SFP+ SR Optical Transceiver49Y42160069
QLogic 10Gb SFP+ SR Optical Transceiver49Y42180064
IBM 10Gb SFP+ SR Optical Transceiver46C34475053
IBM SFP RJ45 Transceiver81Y16183268

Table 3. Supported direct-attach cables
DescriptionPart numberFeature Code
Supported switches
Brocade VDX 6730IBM G8124E, IBM G8264Juniper EX4500
Passive direct-attach cables
0.5m IBM Passive DAC SFP+ Cable00D6288A3RGNoYesNo
1 m IBM Passive DAC-SFP+ Cable90Y9427 A1PHNoYesNo
3 m IBM Passive DAC-SFP+ Cable90Y9430 A1PJNoYesNo
5 m IBM Passive DAC-SFP+ Cable90Y9433 A1PKNoYesNo
1m Juniper DAC SFP+ Cable68Y69275986NoNoYes
3m Juniper DAC SFP+ Cable68Y69475987NoNoYes
Active direct-attach cables
1m IBM Active DAC SFP+ Cable95Y0323A25ANoYesNo
3m IBM Active DAC SFP+ Cable95Y0326A25BNoYesNo
5m IBM Active DAC SFP+ Cable95Y0329A25CNoYesNo
1m 10GE Twinax Act Copper SFP+81Y8295A18MYesNoYes
3m 10GE Twinax Act Copper SFP+81Y8296A18NYesNoYes
5m 10GE Twinax Act Copper SFP+81Y8297A18PYesNoYes

For a complete list of supported SFP+ transceivers and DAC cables refer to the IBM RETAIN tip H203108, Support list matrix for SFP and Twinax DAC cables - Converged Network Adapters, available from:
http://ibm.com/support/entry/portal/docdisplay?lndocid=MIGR-5087884

Features

The members of the Emulex 10 GbE Virtual Fabric Adapter family have the following features and benefits:

  • Dual-channel, 10 Gbps Ethernet controller
  • Line-rate 10 GbE performance
  • 2 SFP+ empty cages to support either SFP+ SR or twin-ax copper connections
    • SFP+ SR link is with SFP+ SR optical module with LC connectors
    • SFP+ twin-ax copper link is with SFP+ direct attached copper module/cable
  • TCIP/IP stateless offloads
  • TCP chimney offload
  • Hardware parity, CRC, ECC, and other advanced error checking
  • PCI Express 2.0 x8 host interface
  • Low-profile form-factor or slot-less mezzanine card form-factor design
  • IPv4/IPv6 TCP, UDP checksum offload
  • VLAN insertion and extraction
  • Support for jumbo frames up to 9000 bytes
  • Preboot eXecution Environment (PXE) 2.0 network boot support
  • Interrupt coalescing
  • Load balancing and failover support
  • Based on Emulex OneConnect technology and including FCoE and iSCSI support as a feature entitlement upgrade
  • Simplifies I/O hardware choices for IT managers
  • Maximizes I/O consolidation with high-performance 10GbE ports
  • One network infrastructure reduces CapEx
  • One management console reduces OpEx
  • Leverages existing IT investments
  • Deploy and manage this and other Emulex OneConnect-based adapters with OneCommand Manager

The Emulex VFA II and III adapters can be upgraded in the field to enable FCoE and iSCSI hardware initiator.

Supported servers

The Emulex 10GbE Virtual Fabric Adapter family are supported in the IBM System x and iDataPlex® servers listed in the following table.

Table 4. Supported System x and iDataPlex servers (Part 1)

Part
number
Product description
x3200 M3 (7327, 7328)
x3250 M3 (4251, 4252)
x3400 M3 (7378, 7379)
x3500 M3 (7380)
x3550 M3 (7944)
x3620 M3 (7376)
x3630 M3 (7377)
x3650 M3 (7945)
x3755 M3 (7164)
95Y3762Emulex Dual Port 10GbE SFP+ VFA III
N
N
N
N
N
N
N
N
N
N
SBB 95Y3768Emulex Dual Port 10GbE SFP+ Integrated VFA III
N
N
N
N
N
N
N
N
N
N
95Y3760Emulex VFA III FCoE/iSCSI License
N
N
N
N
N
N
N
N
N
N
90Y6456Emulex Dual Port 10GbE SFP+ Embedded Adapter
N
N
N
N
N
N
N
N
N
N
90Y5178Emulex Embedded VFA III FCoE/iSCSI License
N
N
N
N
N
N
N
N
N
N
49Y7950Emulex 10GbE Virtual Fabric Adapter II
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
SBB 49Y7940Emulex 10GbE Integrated Virtual Fabric Adapter II
N
N
N
N
N
N
N
N
N
N
49Y4274Emulex VFA II FCoE License for IBM System x
Y
Y
Y
Y
Y
Y
Y
Y
Y
Y
95Y3751Emulex Dual-port VFA II Adapter and FCoE/iSCSI License
Y
N
Y
Y
Y
Y
Y
Y
N
Y

Table 4. Supported System x and iDataPlex servers (Part 2)
Part
number
Product description
x3100 M4 (2582)
x3250 M4 (2583)
x3300 M4 (7382)
x3500 M4 (7383)
x3530 M4 (7160)
x3550 M4 (7914)
x3630 M4 (7158)
x3650 M4 (7915)
x3690 X5 (7147)
x3750 M4 (8722)
x3850 X5 (7143)
95Y3762Emulex Dual Port 10GbE SFP+ VFA III
N
N
Y
Y
Y
Y
Y
Y
N
Y
N
N
SBB 95Y3768Emulex Dual Port 10GbE SFP+ Integrated VFA III
N
N
Y
Y
Y
N
Y
N
N
N
N
N
95Y3760Emulex VFA III FCoE/iSCSI License
N
N
Y
Y
N
Y
N
Y
N
Y
N
N
90Y6456Emulex Dual Port 10GbE SFP+ Embedded Adapter
N
N
N
N
N
Y
N
Y
N
N
N
Y
90Y5178Emulex Embedded VFA III FCoE/iSCSI License
N
N
N
N
N
N
N
N
N
N
N
N
49Y7950Emulex 10GbE Virtual Fabric Adapter II
N
N
N
N
N
N
N
N
Y
N
Y
Y
SBB 49Y7940Emulex 10GbE Integrated Virtual Fabric Adapter II
N
N
N
N
N
N
N
N
Y
N
Y
N
49Y4274Emulex VFA II FCoE License
N
N
N
N
N
N
N
N
Y
N
Y
N
95Y3751Emulex Dual-port VFA II Adapter and FCoE/iSCSI License
N
N
N
N
N
N
N
N
Y
N
Y
N

See IBM ServerProven® for the latest information about the System x servers that support this adapter including support for older servers: http://ibm.com/servers/eserver/serverproven/compat/us/

Modes of operation

The Emulex 10GbE Virtual Fabric Adapter II and III support two types of virtual NIC (vNIC) operating modes, and a physical NIC (pNIC) operating mode:

  • IBM Virtual Fabric Mode (also known as vNIC1 mode) will only work with an IBM RackSwitch G8124E and G8264. In this mode, the Emulex adapter communicates with the IBM switch to obtain vNIC parameters (using DCBX). A special tag is added within each data packet and is later removed by the NIC and/or switch for each vNIC group to maintain separation of the virtual data paths.

    In vNIC mode, each physical port is divided into four virtual ports for a maximum of eight (8) virtual NICs per adapter. The default bandwidth for each vNIC is 2.5 Gbps. Bandwidth for each vNIC can be configured via the IBM switch from 100 Mbps to 10 Gbps, up to a total of 10 Gb per physical port. The vNICs can also be configured to have 0 bandwidth if you must allocate the available bandwidth to fewer than four vNICs per physical port. In IBM Virtual Fabric Mode, you can change the bandwidth allocations through the IBM switch user interfaces without requiring a reboot of the server.

    vNIC bandwidth allocation and metering is performed by both the switch and the VFA. In such a case, a bidirectional virtual channel of an assigned bandwidth is established between them for every defined vNIC.

    In vNIC1 mode, storage protocols (FCoE and iSCSI) on vNICs are not supported.

  • In Switch Independent Mode (also known as vNIC2 mode) the adapter works with any 10 Gb Ethernet switch. Switch Independent Mode offers the same capabilities as IBM Virtual Fabric Mode in terms of the number of vNICs and the bandwidth each can be configured to have. Switch Independent Mode extends the existing customer VLANs to the virtual NIC interfaces. The IEEE 802.1Q VLAN tag is essential to the separation of the vNIC groups by the NIC adapter or driver and the switch. The VLAN tags are added to the packet by the applications or drivers at each end station rather than by the switch.

    vNIC bandwidth allocation and metering is only performed by VFA itself. In such a case, a unidirectional virtual channel is established where the bandwidth management is only performed for the outgoing traffic on a VFA side (server-to-switch). The incoming traffic (switch-to-server) uses the all available physical port bandwidth, as there is no metering performed on either the VFA or a switch side.

    In vNIC2 mode, when storage protocols are enabled on the Emulex 10GbE Virtual Fabric Adapters, six vNICs (three per physical port) are Ethernet, and two vNICs (one per physical port) are either iSCSI or FCoE.

  • In pNIC mode the adapter operates as a standard dual-port 10 Gbps Ethernet adapter, and it functions with any 10 GbE switch. In pNIC mode, with the Emulex FCoE/iSCSI License, the card operates in a traditional Converged Network Adapter (CNA) mode with two Ethernet ports and two storage ports (iSCSI or FCoE) available to the operating system.


Although all adapters in the Emulex 10GbE Virtual Fabric Adapter family support pNIC, vNIC1 and vNIC2 modes, availability of each operating mode is also dependent on the top-of-rack switch and the network or storage protocol used (Ethernet only - no storage protocols, iSCSI or FCoE), as shown in the following table.

Table 5. Available IBM VFA operating modes for Ethernet, iSCSI and FCoE configurations
ProtocolSwitchvNIC1vNIC2pNIC
Ethernet onlyIBM G8124EYesYesYes
IBM G8264YesYesYes
Other compatible switchesNoYesYes
iSCSIIBM G8124ENoPlannedPlanned
IBM G8264NoYesYes
Juniper EX4500NoPlannedPlanned
FCoEIBM G8124E (transit switch)NoNoPlanned
Brocade VDX 6730NoYesYes
Cisco Nexus 5010/5020NoYesYes

Supported FCoE and iSCSI configurations are shown in Table 6 and Table 7 respectively.

Table 6. Supported FCoE configurations
Virtual Fabric adapterFCoE upgradeVFA modeTransit switchFCF switchSAN fabricStorage targets
  • VFA III, 95Y3762
  • Integrated VFA III, SBB 95Y3768
  • 95Y3760
Planned
  • Embedded Adapter, 90Y6456
  • 90Y5178
Planned
  • VFA II, 49Y7950
  • VFA II and FCoE/iSCSI License, 95Y3751
  • 49Y4274
    (not needed
    for 95Y3751)
  • vNIC2
  • pNIC
  • None
  • Brocade VDX 6730
  • IBM B-type
  • DS8000
  • IBM SVC
  • V7000
  • IBM XIV
  • VFA II, 49Y7950
  • Integrated VFA II, SBB 49Y7940
  • VFA II and FCoE/iSCSI License, 95Y3751
  • 49Y4274
    (not needed
    for 95Y3751)
  • vNIC2
  • pNIC
  • None
  • Cisco Nexus 5010
  • Cisco Nexus 5020
  • Cisco MDS
  • DS8000
  • IBM SVC
  • V7000

Table 7. Supported iSCSI configurations
Virtual Fabric adapteriSCSI upgradeVFA modeTOR switchStorage targets
  • VFA III, 95Y3762
  • Integrated VFA III, SBB 95Y3768
  • 95Y3760
  • vNIC2
  • pNIC
  • IBM G8264
  • IBM SVC
  • IBM XIV
  • Embedded Adapter, 90Y6456
  • 90Y5178
Planned
  • VFA II, 49Y7950
  • 49Y4274
  • vNIC2
  • pNIC
  • IBM G8264
  • IBM SVC
  • IBM XIV
  • Integrated VFA II, SBB 49Y7940
  • VFA II and FCoE/iSCSI License, 95Y3751
  • 49Y4274
    (not needed
    for 95Y3751)
Planned

Important: Use these tables only as a starting point. Configuration support must be verified through the IBM System Storage® Interoperation Center (SSIC) found at the following website:
http://www.ibm.com/systems/support/storage/ssic

Standards supported

The following IEEE standards are supported:

  • IEEE 802.3ae (10 Gbps Ethernet XAUI)
  • IEEE 802.1q (VLAN)
  • IEEE 802.1Qbb (Priority flow control)
  • IEEE 802.1Qaz (ETS and Congestion Management)
  • IEEE 802.1p (QoS/CoS)
  • IEEE 802.3ad (Link Aggregation)
  • IEEE 802.3x (Flow Control)
  • ANSI INCITS T11 FC-BB-5 2.0, FC-PI-2, FC-GS-4, FC-TAPE and FCP-3
  • PCI Express base spec 2.0, PCI Bus Power Management Interface, rev. 1.2
  • Advanced Error Reporting (AER)

Physical specifications

The standard form-factor adapter has the following physical specifications:

  • Height: 167 mm (6.6 in)
  • Width: 69 mm (2.7 in)
  • Depth: 17 mm (0.7 in)

The mezzanine adapter has the following physical specifications:
  • Height: 60 mm (2.4 in)
  • Width: 160 mm (6.3 in)
  • Depth: 17 mm (0.7 in)

Operating environment

This adapter is supported in the following environment:

  • Temperature:
    • Operating: 0° to 55° C (32° to 131° F)
    • Non-operating: -40° to 70° C (-40° to 158° F)
  • Humidity: 5 to 95%, non-condensing

Warranty

One-year limited warranty. When installed in a System x server, these cards assume your system’s base warranty and any IBM ServicePac® upgrade.

Popular configurations

The following figure shows Emulex 10GbE Virtual Fabric Adapter II installed in a supported rack server. The servers are connected to a 10 Gb Ethernet network using a pair of 10 Gb Ethernet switches.


Figure 4. Popular configuration

The parts used are listed in the following table.

Table 8. Components used with the Emulex 10Gb Virtual Fabric Adapter (Figure 4)

Diagram referencePart number/machine typeDescriptionQuantity
1VariesSupported server (See Table 4)2
2VariesEmulex 10GbE Virtual Fabric Adapter (See Table 1)2
2VariesSupported 10Gb SFP+ Transceivers or direct-attach cables (See Tables 2 and 3)4
Varies10Gb Ethernet Switch (IBM RackSwitch G8124-E is shown in Figure 4)2

Note: If SFP+ transceivers are used, this configuration also requires fiber optic cabling between the servers and the switches.

When Emulex Virtual Fabric adapters are used with FCoE/iSCSI feature enabled, refer to IBM System Storage Interoperation Center (SSIC) for a list of supported FCoCEE configurations: http://www.ibm.com/systems/support/storage/ssic

Supported operating systems

The Emulex 10GbE Virtual Fabric Adapter II and III family support the following operating systems:

  • Microsoft Windows Server 2012
  • Microsoft Windows Server 2008 R2
  • Microsoft Windows Server 2008, Datacenter x64 Edition
  • Microsoft Windows Server 2008, Datacenter x86 Edition
  • Microsoft Windows Server 2008, Enterprise x64 Edition
  • Microsoft Windows Server 2008, Enterprise x86 Edition
  • Microsoft Windows Server 2008, Standard x64 Edition
  • Microsoft Windows Server 2008, Standard x86 Edition
  • Microsoft Windows Server 2008, Web x64 Edition
  • Microsoft Windows Server 2008, Web x86 Edition
  • Microsoft Windows Server 2003/2003 R2, Enterprise Edition
  • Microsoft Windows Server 2003/2003 R2, Enterprise x64 Edition
  • Microsoft Windows Server 2003/2003 R2, Standard Edition
  • Microsoft Windows Server 2003/2003 R2, Standard x64 Edition
  • Microsoft Windows Server 2003, Web Edition
  • Red Hat Enterprise Linux 6 Server Edition
  • Red Hat Enterprise Linux 6 Server x64 Edition
  • Red Hat Enterprise Linux 5 Server Edition
  • Red Hat Enterprise Linux 5 Server x64 Edition
  • Red Hat Enterprise Linux 5 Server Edition with Xen
  • Red Hat Enterprise Linux 5 Server with Xen x64 Edition
  • SUSE LINUX Enterprise Server 11 for AMD64/EM64T
  • SUSE LINUX Enterprise Server 11 for x86
  • SUSE LINUX Enterprise Server 11 with Xen for AMD64/EM64T
  • SUSE LINUX Enterprise Server 10 for AMD64/EM64T
  • SUSE LINUX Enterprise Server 10 for x86
  • SUSE LINUX Enterprise Server 10 with Xen for AMD64/EM64T
  • VMware ESX 4.1
  • VMware ESXi 4.1
  • VMware ESX 4.0
  • VMware ESXi 4.0

See IBM ServerProven at http://ibm.com/servers/eserver/serverproven/compat/us/ for the latest information about the specific versions and service packs supported. Not all servers support all operating systems and versions.

Related publications

For more information refer to these documents:


Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.

Profile

Publish Date
18 November 2011

Last Update
20 May 2013


Rating: Not yet rated


Author(s)

IBM Form Number
TIPS0844