Emulex VFA5 ML2 Dual Port 10GbE SFP+ Adapter for IBM System x

IBM Redbooks Product Guide


Abstract

The Emulex Virtual Fabric Adapter 5 (VFA5) for IBM® System x® builds on the foundation of previous generations of Emulex Virtual Fabric Adapters by delivering performance enhancements and new features that maximize the return on server and storage investments. The Emulex VFA5 delivers a new set of powerful features and capabilities that are designed for the virtualized enterprise environment, multi-tenant and single-tenant cloud environments, I/O intensive environments, and converged infrastructure environments.

Contents


Table of contents


Introduction

The Emulex Virtual Fabric Adapter 5 (VFA5) for IBM® System x® builds on the foundation of previous generations of Emulex Virtual Fabric Adapters by delivering performance enhancements and new features that maximize the return on server and storage investments. The Emulex VFA5 delivers a new set of powerful features and capabilities that are designed for the virtualized enterprise environment, multi-tenant and single-tenant cloud environments, I/O intensive environments, and converged infrastructure environments. The adapter is shown in the following figure.

Emulex VFA5 ML2 Dual Port 10GbE SFP+ Adapter
Figure 1. Emulex VFA5 ML2 Dual Port 10GbE SFP+ Adapter

Did you know?

The Emulex VFA5 adapter supports two methods to virtualize I/O: IBM Virtual Fabric mode and Switch Independent mode. With virtual fabric, up to eight virtual network ports (vNICs) can be created with a single two-port 10 GbE network adapter. Converged protocols iSCSI and FCoE are also supported by the IBM Features on Demand upgrade. By using a common infrastructure for Ethernet and SAN, and by virtualizing your network adapter, you can reduce your infrastructure capital expense.

Part number information

The part numbers to order the adapter are listed in the following table. The FCoE/iSCSI upgrade license is an IBM Features on Demand field upgrade that enables the converged networking capabilities of the adapter.

Table 1. Ordering part number and feature code

Part numberFeature codeDescription
00D1996A40QEmulex VFA5 ML2 Dual Port 10GbE SFP+ Adapter for IBM System x
00D8544A4NZEmulex VFA5 ML2 FCoE/iSCSI License for IBM System x (FoD)

The adapter, when shipped as a stand-alone option, includes the following items:
  • One Emulex adapter
  • 3U bracket that is attached with a 2U bracket that is included in the box
  • Quick Install Guide
  • Warranty information and Important Notices flyer
  • Documentation CD

The FCoE/iSCSI license is delivered in the form of an authorization code that you enter on the IBM Features on Demand website to generate an activation key.

Supported transceivers and direct-attach cables

The Emulex VFA5 ML2 Dual Port 10GbE SFP+ Adapter has two empty SFP+ cages that support SFP+ SR transceivers and twin-ax direct-attached copper cables, as listed in Table 2 and Table 3, respectively.

Table 2. Supported transceivers

DescriptionPart numberFeature Code
Brocade 10Gb SFP+ SR Optical Transceiver49Y42160069
QLogic 10Gb SFP+ SR Optical Transceiver49Y42180064
IBM 10Gb SFP+ SR Optical Transceiver46C34475053
IBM SFP RJ45 Transceiver81Y16183268

Table 3. Supported direct-attach cables
DescriptionPart numberFeature Code
Supported switches
Brocade VDX 6730IBM G8124E, IBM G8264Juniper EX4500
Passive direct-attach cables
0.5m IBM Passive DAC SFP+ Cable00D6288A3RGNoYesNo
1 m IBM Passive DAC-SFP+ Cable90Y9427 A1PHNoYesNo
3 m IBM Passive DAC-SFP+ Cable90Y9430 A1PJNoYesNo
5 m IBM Passive DAC-SFP+ Cable90Y9433 A1PKNoYesNo
1m Juniper DAC SFP+ Cable68Y69275986NoNoYes
3m Juniper DAC SFP+ Cable68Y69475987NoNoYes
Active direct-attach cables
1m IBM Active DAC SFP+ Cable95Y0323A25ANoYesNo
3m IBM Active DAC SFP+ Cable95Y0326A25BNoYesNo
5m IBM Active DAC SFP+ Cable95Y0329A25CNoYesNo
1m 10GE Twinax Act Copper SFP+81Y8295A18MYesNoYes
3m 10GE Twinax Act Copper SFP+81Y8296A18NYesNoYes
5m 10GE Twinax Act Copper SFP+81Y8297A18PYesNoYes

Features

The VFA5 reduces cost by enabling a converged infrastructure and improves performance with powerful offload engines. The adapter has the following features and benefits:

  • Multi-protocol support for 10 GbE

    The Emulex VFA5 supports 2x10 GbE and is cost- and performance-optimized for integrated and converged infrastructures. The VFA5 offers a “triple play” of converged data, storage, and low latency RDMA (RDMA support is planned for 1H/2014) networking on a common Ethernet fabric. The Emulex VFA provides customers with a flexible storage protocol option for running heterogeneous workloads on their increasingly converged infrastructures.

  • Full hardware storage offloads

    The Emulex VFA5 supports an optional hardware offload engine that accelerates storage protocol processing, delivering up to 1.5 million I/O operations per second (IOPS), which enables the server’s processing resources to focus on applications, improving the server’s performance.

  • IBM Features on Demand (FoD)

    The VFA5 uses IBM Features on Demand (FoD) software activation technology. FoD enables the adapter to be initially deployed as a low-cost Ethernet NIC, and then later upgraded in the field to support FCoE or iSCSI hardware offload.

  • Emulex Virtual Network Acceleration (VNeX) technology support

    Emulex VNeX supports Microsoft supported network virtualization by using generic routing encapsulation (NVGRE) and VMware-supported virtual extensible LAN (VXLAN). These technologies create more logical LANs for traffic isolation that are needed by cloud architectures. Because these protocols increase the processing burden on the server’s processor, the VFA5 has an offload engine that is designed specifically for processing these tags. The resulting benefit is that cloud providers can leverage the benefits of VxLAN/NVGRE while not being penalized with a reduction in the server’s performance.

  • Advanced RDMA capabilities (planned for 1H/2014)

    RDMA over Converged Ethernet (RoCE) delivers application and storage acceleration through faster I/O operations with support for Windows Server SMB Direct and Linux NFS protocols.

  • You can deploy faster and manage less when you combine Virtual Fabric adapters (VFAs) and Host Bus Adapters (HBAs) that are developed by Emulex.

    IBM VFAs and HBAs that are developed by Emulex use the same installation and configuration process, streamlining the effort to get your server running, and saving you valuable time. They also use the same Fibre Channel drivers, reducing time to qualify and manage storage connectivity. With Emulex's OneCommand Manager, you can manage IBM VFAs and HBAs that are developed by Emulex through the data center from a single console.


Specifications

Adapter specifications:

  • Dual-channel 10 Gbps Ethernet controller
  • Based on the Emulex XE102 ASIC
  • PCI Express 3.0 x8 host interface
  • Line-rate 10 GbE performance
  • Upgradeable to support converged networking (FCoE and iSCSI) through IBM Features on Demand
  • 2 SFP+ empty cages to support either SFP+ SR or twin-ax copper connections
    • SFP+ SR link is with the SFP+ SR optical module with LC connectors.
    • SFP+ twin-ax copper link is with the SFP+ direct attached copper module/cable.
  • Power dissipation: 14.4 W typical

Virtualization features
  • Four NIC partitions per adapter port
  • Complies with PCI-SIG specification for SR-IOV
  • VxLAN/NVGRE encapsulation and offload
  • PCI-SIG Address Translation Service (ATS) v1.0
  • Virtual Switch Port Mirroring for Diagnostic purposes
  • Virtual Ethernet Bridge (VEB)
  • Sixty-two Virtual functions (VF)
  • QoS for controlling and monitoring bandwidth that is assigned to and used by virtual entities
  • Traffic shaping and QoS across each virtual function (VF) and physical function (PF)
  • Message Signal Interrupts (MSI-X) support
  • VMware NetQue / VMQ support
  • Microsoft VMQ & Dynamic VMQ support in Hyper-V

Ethernet and NIC features
  • NDIS 5.2, 6.0, and 6.2 compliant Ethernet functionality
  • IPv4/IPv6 TCP, UDP, and Checksum Offload
  • IPv4/IPv6 TCP, UDP Receive Side Scaling (RSS)
  • IPv4/IPv6 Large Send Offload (LSO)
  • Programmable MAC addresses
  • 128 MAC/vLAN addresses per port
  • Support for HASH-based broadcast frame filters per port
  • vLAN insertion and extraction
  • 9216 byte Jumbo frame support
  • Receive Side Scaling (RSS)
  • Filters: MAC and vLAN

Remote Direct Memory Access (RDMA) (planned for 1H/2014):
  • Direct data placement in application buffers without processor intervention
  • Supports RDMA over converged Ethernet (RoCE) specifications
  • Linux Opens Fabrics Enterprise Distribution (OFED) Support
  • Low latency queues for small packet sends and receives
  • Local interprocess communication option by internal VEB switch
  • TCP/IP Stack By-Pass

Data Center Bridging / Converged Enhanced Ethernet (DCB/CEE) (with Features on Demand upgrade):
  • Hardware Offloads of Ethernet TCP/IP
  • 802.1Qbb Priority Flow Control (PFC)
  • 802.1 Qaz Enhanced Transmission Selection (ETS) (planned for 1H/2014)
  • 802.1 Qaz Data Center Bridging Exchange (DCBX) (planned for 1H/2014)

Fibre Channel over Ethernet (FCoE) offload (with Features on Demand upgrade):
  • Hardware offloads of Ethernet TCP/IP
  • ANSI T11 FC-BB-5 Support
  • Programmable Worldwide Name (WWN)
  • Support for FIP and FCoE Ether Types
  • Supports up to 255 NPIV Interfaces per port
  • FCoE Initiator
  • Common driver for Emulex Universal CNA and Fibre Channel HBAs
  • 255 N_Port ID Virtualization (NPIV) interfaces per port
  • Fabric Provided MAC Addressing (FPMA) support
  • Up to 4096 concurrent port logins (RPIs) per port
  • Up to 2048 active exchanges (XRIs) per port

iSCSI offload (with Features on Demand upgrade):
  • Full iSCSI Protocol Offload
  • Header, Data Digest (CRC), and PDU
  • Direct data placement of SCSI data
  • 2048 Offloaded iSCSI connections
  • iSCSI initiator and concurrent initiator /target modes
  • Multipath I/O
  • OS-neutral INT13 based iSCSIboot and iSCSI crash memory dump support
  • RFC 4171 Internet Storage Name Service (iSNS)

Standards supported

The following IEEE standards are supported:

  • 802.3-2008 10Gbase Ethernet port
  • 802.1Q vLAN
  • 802.3x Flow Control with pause Frames
  • 802.1 Qbg Edge Virtual Bridging
  • 802.1Qaz Enhanced transmission Selection (ETS) Data Center Bridging Capability (DCBX)
  • 802.1Qbb Priority Flow Control
  • 802.3ad link Aggregation/LACP
  • 802.1AB Link Layer Discovery Protocol
  • 802.3ae (SR Optics)
  • 802.3p (Priority of Service)
  • IPV4 (RFC 791)
  • IPV6 (RFC 2460)
  • 1588v2 Time Sync (planned for 1H/2014)

Supported servers

The Emulex VFA5 ML2 Dual Port 10GbE SFP+ Adapter is supported in the IBM System x and iDataPlex® servers that are listed in the following table.

Table 4. Server compatibility

Part
number
Description
x3100 M4 (2582)
x3250 M4 (2583)
x3300 M4 (7382)
x3500 M4 (7383, E5-2600)
x3530 M4 (7160)
x3550 M4 (7914, E5-2600)
x3630 M4 (7158)
x3650 M4 (7915, E5-2600)
x3690 X5 (7147)
x3750 M4 (8722)
x3850 X5 (7143)
dx360 M4 (7912, E5-2600)
x3250 M5 (5458)
x3500 M4 (7383, E5-2600 v2)
x3550 M4 (7914, E5-2600 v2)
x3650 M4 (7915, E5-2600 v2)
x3650 M4 HD (5460)
x3850 X6/x3950 X6 (3837)
dx360 M4 (7912, E5-2600 v2)
nx360 M4 (5455)
00D1996Emulex VFA5 ML2 Dual Port 10GbE SFP+ Adapter
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
N
Y
N
N

For the latest information about the System x servers that support this adapter, including support for older servers, see IBM ServerProven® at the following website: http://ibm.com/servers/eserver/serverproven/compat/us/

Modes of operation

The Emulex VFA5 ML2 Dual Port 10GbE SFP+ Adapter supports two types of virtual NIC (vNIC) operating modes, and a physical NIC (pNIC) operating mode:

  • IBM Virtual Fabric Mode (also known as vNIC1 mode) works only with an IBM RackSwitch™ G8124E and G8264. In this mode, the Emulex adapter communicates with the IBM switch to obtain vNIC parameters (using DCBX). A special tag is added within each data packet and is later removed by the NIC or switch for each vNIC group to maintain separation of the virtual data paths.

    In vNIC mode, each physical port is divided into four virtual ports for a maximum of eight virtual NICs per adapter. The default bandwidth for each vNIC is 2.5 Gbps. Bandwidth for each vNIC can be configured through the IBM RackSwitch from 100 Mbps to 10 Gbps, up to a total of 10 Gb per physical port. The vNICs can also be configured to have 0 bandwidth if you must allocate the available bandwidth to fewer than four vNICs per physical port. In IBM Virtual Fabric Mode, you can change the bandwidth allocations through the IBM switch user interfaces without requiring a reboot of the server.

    vNIC bandwidth allocation and metering are performed by both the switch and the VFA. In such a case, a bidirectional virtual channel of an assigned bandwidth is established between them for every defined vNIC.

  • In Switch Independent Mode (also known as vNIC2 mode), the adapter works with any 10 Gb Ethernet switch. Switch Independent Mode offers the same capabilities as IBM Virtual Fabric Mode in terms of the number of vNICs and the bandwidth each can be configured to have. Switch Independent Mode extends the existing customer VLANs to the virtual NIC interfaces. The IEEE 802.1Q VLAN tag is essential to the separation of the vNIC groups by the NIC adapter or driver and the switch. The VLAN tags are added to the packet by the applications or drivers at each endpoint rather than by the switch.

    vNIC bandwidth allocation and metering are performed only by VFA itself. In such a case, a unidirectional virtual channel is established where the bandwidth management is performed only for the outgoing traffic on a VFA side (server-to-switch). The incoming traffic (switch-to-server) uses the all available physical port bandwidth, as there is no metering that is performed on either the VFA or a switch side.

    In vNIC2 mode, when storage protocols are enabled on the Emulex 10GbE Virtual Fabric Adapters, six vNICs (three per physical port) are Ethernet, and two vNICs (one per physical port) are either iSCSI or FCoE.

  • In pNIC mode, the adapter operates as a standard dual-port 10 Gbps Ethernet adapter, and it functions with any 10 GbE switch. In pNIC mode, with the Emulex FCoE/iSCSI License, the card operates in a traditional Converged Network Adapter (CNA) mode with two Ethernet ports and two storage ports (iSCSI or FCoE) available to the operating system.

The following table compares the two virtual fabric modes.

Table 5. Comparison of virtual fabric modes
FunctionIBM Virtual Fabric ModeSwitch Independent Mode
DescriptionIntelligence in the IBM Networking OS working with select Emulex adapters. VLAN based.Intelligence in the adapter, independent of the upstream networking device.
Supported switchesG8124, G8264, G8264T, G8264CSAll 10 GbE switches
Number of vNICs per physical 10 Gb port44
Minimum vNIC bandwidth100 MB100 MB
Server to switch bandwidth limit per vNICYesNo
Switch to server bandwidth limit per vNICYesNo
IEEE 802.aq VLAN tagging OptionalRequired
Isolated NIC teaming failover per vNICYesNo
Switch modesTunnel modeAccess, Trunk, or Tunnel Mode
Uplink connectivityDedicatedShare
iSCSI/FCoE support YesYes

The switches that support each of these modes are listed in the following table.

Table 6. Supported modes for Ethernet, iSCSI, and FCoE configurations
SwitchvNIC1vNIC2pNIC
Ethernet only
IBM G8124EYesYesYes
IBM G8264YesYesYes
Other compatible switchesNoYesYes
iSCSI
IBM G8124ENoYesYes
IBM G8264NoYesYes
Juniper EX4500NoYesYes
FCoE
IBM G8124E (FCoE transit switch)NoNoYes
Brocade VDX 6730NoYesYes
Cisco Nexus 5010/5020NoYesYes

Physical specifications

ML2 adapter dimensions (approximate):

  • Height: 69 mm (2.7 in.)
  • Length: 168 mm (6.6 in.)
  • Width: 17 mm (0.7 in.)

Shipping dimensions and weight (approximate):
  • Height: 189 mm (7.51 in.)
  • Width: 90 mm (3.54 in.)
  • Depth: 38 mm (1.50 in.)
  • Weight: 450 g (0.99 lb)

Operating environment

This adapter is supported in the following environment:

  • Temperature:
    • Operating: 0 °C to 55 °C (32 °F to 131 °F)
    • Non-operating: -40 °C to 70 °C (-40 °F to 158 °F)
  • Humidity: 5 - 95%, non-condensing

Warranty

One-year limited warranty. When installed in a System x server, these cards assume your system’s base warranty and any IBM ServicePac® upgrade.

Supported operating systems

The adapter supports the following operating systems:

  • Microsoft Windows Server 2008 R2 SP2
  • Microsoft Windows Server 2008, Datacenter x64 Edition
  • Microsoft Windows Server 2008, Datacenter x86 Edition
  • Microsoft Windows Server 2008, Enterprise x64 Edition
  • Microsoft Windows Server 2008, Enterprise x86 Edition
  • Microsoft Windows Server 2008, Standard x64 Edition
  • Microsoft Windows Server 2008, Standard x86 Edition
  • Microsoft Windows Server 2008, Web x64 Edition
  • Microsoft Windows Server 2008, Web x86 Edition
  • Microsoft Windows Server 2012
  • Microsoft Windows Server 2012 R2
  • Red Hat Enterprise Linux 5 Server Edition U9
  • Red Hat Enterprise Linux 5 Server with Xen x64 Edition U9
  • Red Hat Enterprise Linux 5 Server x64 Edition U9
  • Red Hat Enterprise Linux 6 Server Edition U4
  • Red Hat Enterprise Linux 6 Server x64 Edition U4
  • SUSE Linux Enterprise Server 11 for AMD64/EM64T SP3
  • SUSE Linux Enterprise Server 11 for x86 SP3
  • SUSE Linux Enterprise Server 11 with Xen for AMD64/EM64T SP3
  • VMware vSphere 5.0 (ESXi) U2
  • VMware vSphere 5.1 (ESXi) U1
  • VMware vSphere 5.5 (ESXi)

For the latest information about the specific versions and service packs that are supported, see IBM ServerProven at http://ibm.com/servers/eserver/serverproven/compat/us/. Not all servers support all operating systems and versions.

Related publications

For more information, see these resources:


Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment. publib-b.boulder.ibm.com

Profile

Publish Date
18 February 2014


Rating: Not yet rated


Author(s)

IBM Form Number
TIPS1142