Chelsio T4-based 10GbE Unified Wire Adapters for IBM Intelligent Cluster and iDataPlex

IBM Redbooks Product Guide


Lenovo is open and ready for business with System x
IBM x86 products are now products of Lenovo in the U.S. and other countries. IBM will continue to host x86-related content on ibm.com until migrated to Lenovo.
Learn more about the acquisition

Abstract

The Chelsio dual-port and quad-port Unified Wire adapters for IBM® Intelligent Cluster and IBM iDataPlex™ are 10 Gigabit Ethernet adapter with PCI Express host bus interface optimized for cloud computing, HPC, virtualization, storage, and other data center applications. These adapters are based on fourth-generation (T4) technology from Chelsio, provide the highest 10GbE performance available, and dramatically lower host-system CPU communications overhead with on-board hardware that offloads TCP/IP, iSCSI, FCoE, and iWARP RDMA processing from its host system. As a result, the server can achieve increased bandwidth, lower latency, and lower power consumption.

Contents


The Chelsio dual-port and quad-port Unified Wire adapters for IBM® Intelligent Cluster and IBM iDataPlex™ are 10 Gigabit Ethernet adapter with PCI Express host bus interface optimized for cloud computing, HPC, virtualization, storage, and other data center applications. These adapters are based on fourth-generation (T4) technology from Chelsio, provide the highest 10GbE performance available, and dramatically lower host-system CPU communications overhead with on-board hardware that offloads TCP/IP, iSCSI, FCoE, and iWARP RDMA processing from its host system. As a result, the server can achieve increased bandwidth, lower latency, and lower power consumption.

Figure 1 shows the Chelsio T440-CR Quad-Port (SFP+) 10GbE PCI-E 2.0 Adapter.
Chelsio T440-CR Quad-Port (SFP+) 10GbE PCI-E 2.0 Adapter
Figure 1. Chelsio T440-CR Quad-Port (SFP+) 10GbE PCI-E 2.0 Adapter


Did you know

Chelsio implements the concept of the unified wire, enabling the convergence of server networking, storage networking, and cluster computing interconnects onto a single fabric that enables TCP, UDP, iSCSI, iWARP, and FCoE offload with SR-IOV, EVB/VN Tag, and data center bridging (DCB).

These Chelsio adapters are part of the IBM Intelligent Cluster solution. IBM Intelligent Cluster is your key to a fully integrated HPC solution. IBM clustering solutions include servers, storage, and industry-leading interconnects that are factory-integrated, fully tested, and delivered to your door, ready to plug into your data center, all with a single point of contact for support.


Part number information

Table 1 lists part numbers and feature codes.

Table 1. Ordering part number and feature code
DescriptionIBM part numberFeature codeChelsio part number
Chelsio T420-CR Dual-port (SFP+) 10GbE PCI-E 2.0 Adapter46M22373025T420-CR
Chelsio T420-SO-CR Dual-port (SFP+) 10GbE PCI-E 2.0 Adapter81Y8021A0R2T420-SO-CR
Chelsio T440-CR Quad-Port (SFP+) 10GbE PCI-E 2.0 Adapter46M22413026T440-CR
Chelsio T420-BCH Dual-port 10GbE CFFh Expansion Card46M22453027T420-BCH

Figure 2 shows the Chelsio T420-BCH Dual-port 10GbE CFFh Expansion Card for IBM BladeCenter®.

Chelsio T420-BCH Dual-port 10GbE CFFh Expansion Card
Figure 2. Chelsio T420-BCH Dual-port 10GbE CFFh Expansion Card


Features

Chelsio is enabling a unified wire for LAN, SAN, and cluster traffic. This unified wire was made possible by the high bandwidth and low latency of 10GbE combined with storage and cluster protocols operating over TCP/IP (iSCSI, FCoE, and iWARP, respectively). In parallel, operating systems and hypervisors have incorporated native support for iSCSI, and database applications now support file-based storage protocols such as NFS as an alternative to SANs. Expanding its unified wire approach, Chelsio has added Fibre Channel over Ethernet (FCoE) hardware support to the new T4-based adapters. FCoE provides a transition path from legacy SANs to converged networks.

Chelsio Unified Wire 10GbE adapters for IBM Intelligent Cluster and iDataPlex have the following features:
  • PCI Express Gen2 x8
  • Low latency
  • Supports up to 1M connections
  • Full TCP and UDP offload
  • TCP chimney
  • Full iSCSI, FCoE offload
  • Full iWARP RDMA offload
  • EVB, VEPA
  • PCI-SIG SR-IOV
  • Integrated media streaming offload
  • Hardware-based firewall in the cloud
  • Traffic filtering and management


Applications

The Chelsio Unified Wire 10GbE adapters are ideally suited for use in the solutions listed below.

Data center networking:
  • Scale up servers and NAS systems.
  • Link servers in multiple facilities to synchronize data centers.
  • Consolidate LAN, SAN, and cluster networks.

Cloud computing:
  • Virtualization features to maximize cloud scaling and utilization
  • Runs InfiniBand, Fibre Channel apps unmodified on Ethernet
  • Cloud ready functional and management features
  • QoS and traffic management

Networked storage:
  • Enable high-performance NAS systems and Ethernet-based IP SANs.
  • Develop shared-storage systems providing both file-level and block-level services.

High performance computing:
  • Very low latency Ethernet.
  • Increase cluster fabric bandwidth.
  • Deploy Ethernet-only networking for cluster fabric, LAN, and SAN.


Comparison of the adapter cards

Table 2 compares the Chelsio Unified Wire 10GbE adapters.

Table 2. Ordering part numbers and feature codes
AdapterT420-CRT440-CRT420-BCHT420-SO-CR
IBM part number46M223746M224146M224581Y8021
Number of ports2x 10 G4x 10 G2x 10 G2x 10G
Connector typeSFP+ cageSFP+ cageBladeCenter CFFhSFP+ cage
Host interfacePCIe x8 Gen 2PCIe x8 Gen 2PCIe x8 Gen 2PCIe x8 Gen 2
Form factorLow ProfileHalf sizeBladeCenter CFFh Low profile
Connection capacity32 K32 K32 K128
Full TCP offloadYesYesYesYes
Full UDP/multicast offloadYesYesYesNo
Full iWARP RDMA offloadYesYesYesNo
Full iSCSI offloadYesYesYesYes
Full FCoE offloadYesYesYesYes
SR-IOV, VEB/VEPAYesYesYesYes
Op-iSCSI YesYesYesYes
Op-FCoEYesYesYesYes
Power11.94 W12.65 W12.60 W8.86 W
Recommended forUnified Wire: RDMA, TOE,
NIC, iSCSI, FCoE virtualization
Unified wire: RDMA, TOE,
NIC, iSCSI, FCoE virtualization
Unified Wire: RDMA, TOE, NIC,
iSCSI, FCoE virtualization
Unified Wire: NIC, TOE,
iSCSI, FCoE virtualization


Specifications

The Chelsio Unified Wire 10GbE adapters have the specifications listed below.

Host interface:
  • PCI Express Gen2 x8
  • MSI-X, MSI, and support for legacy pin interrupts

Network interfaces:
  • 10GBASE-SR short-reach optics (850 nm)
  • 10GBASE-LR long-reach optics (1310 nm)

High-performance RDMA:
  • Low-latency and line-rate bandwidth
  • Enhanced RDMA primitives
  • iWARP support on Linux OFED
  • Microsoft Network Direct support

UDP and multicast offload:
  • UDP Sockets API
  • Low user-to-user latency
  • Multicast replication on ingress or egress

Virtualization:
  • PCI-SIG SR-IOV
  • 128 virtual functions (VF), 8 PF
  • 144 port virtual switch
  • EVB, VEPA, Flex10, VNTag
  • 336 virtual MAC addresses

FCoE:
  • Full HBA FCoE (initiator or target)
  • Open-FCoE Offload (initiator)
  • CRC32 offload generation and verification
  • Ingress and egress access control list (ACL)

iSCSI acceleration:
  • Full iSCSI initiator and target mode stack
  • CRC32 offload generation and verification
  • iSCSI proxy switching based on SCSI CDB
  • Full HBA offload

TCP/IP full offload:
  • Full TCP implementation including IPv4 and IPv6
  • Microsoft Chimney support
  • Extensive RFC compliance, fully featured stack
  • Full TCP proxy between any set of connections
  • VLAN support of up to 4096 different VLAN IDs
  • Load balancing and failover capabilities

Stateless offloads:
  • TCP/UDP checksum offload for IPv4 and IPv6
  • TSO, LSO, and GSO for IPv4 and IPv6
  • VLAN filtering, insertion, and extraction
  • Line-rate packet filtering and attack protection
  • Fine granularity time stamping (down to 4 ns)
  • Ethernet routing (packet header rewrite)
  • Packet tracing and packet sniffing

Ethernet standards support:
  • IEEE 802.3ae (10 GbE)
  • IEEE 802.3z (1 GbE) compliant
  • IEEE 802.IP priority
  • IEEE 802.IQ VLAN tagging
  • IEEE 802.1Qbg EVB/VEPA
  • IEEE 802.1Qbh VNTag
  • IEEE 802.3x flow control
  • IEEE 802.3ad load-balancing and failover
  • Ether II and 802.3 encapsulated frames
  • Multiple MAC addresses per interface
  • Jumbo frames up to 9.6 Kbytes


Figure 3 shows the Chelsio T420-CR Dual-port (SFP+) 10GbE PCI-E 2.0 Adapter.

Chelsio T420-CR Dual-port (SFP+) 10GbE PCI-E 2.0 Adapter
Figure 3. Chelsio T420-CR Dual-port (SFP+) 10GbE PCI-E 2.0 Adapter


Physical and environmental specifications

The specifications are:
  • Dimensions without bracket:
    • T420-BCH - 4.92" x 6.26" or 12.5 cm x 15.9 cm
    • T420-CR - 6.6 in. x 2.71 in. or 16.8 cm x 6.9 cm
    • T420-SO-CR - 4.0 in. x 2.7 in. or 10.2 cm x 6.9 cm
    • T440-CR - 6.6 in. x 3.95 in. or 16.8 cm x 10.0 cm
  • Operating temp: 0 to 55 °C or 32 to 131 °F
  • Operating humidity: 5 to 95%

Figure 4 shows the Chelsio T420-SO-CR Dual-port (SFP+) 10GbE PCI-E 2.0 Adapter.

Chelsio T420-SO-CR Dual-port (SFP+) 10GbE PCI-E 2.0 Adapter
Figure 4. Chelsio T420-SO-CR Dual-port (SFP+) 10GbE PCI-E 2.0 Adapter


Supported servers

The Chelsio Unified Wire 10GbE adapters are supported in the IBM Intelligent Cluster program,, as listed in Table 3.

Table 3. Supported servers in the IBM Intelligent Cluster program


Part number

x3550 M3

x3650 M3

x3690 X5

x3755 M3


HS22

HS22V

LS22

LS42

dx360 M2

dx360 M3

Chelsio T420-CR Dual-port (SFP+) 10GbE PCI-E 2.0 Adapter

46M2237

Y

Y

Y

Y

Y

N

N

N

N

Y

Y

Chelsio T420-SO-CR Dual-port (SFP+) 10GbE PCI-E 2.0 Adapter

81Y8021

Y

Y

Y

Y

Y

N

N

N

N

Y

Y

Chelsio T440-CR Quad-Port (SFP+) 10GbE PCI-E 2.0 Adapter

46M2241

Y

Y

Y

Y

Y

N

N

N

N

Y

Y

Chelsio T420-BCH Dual-port 10GbE CFFh Expansion Card

46M2245

N

N

N

N

N

Y

Y

Y

Y

N

N


Supported operating systems

The Chelsio adapters support the following operating systems. Support is for both TOE and NIC mode, unless otherwise stated, along with OFED stack, SR-IOV (vNIC), and other functionalities.
  • Red Hat Enterprise Linux 5 update 4 kernel (RHEL5.4), 2.6.18-164.el5 (32-bit and 64-bit)
  • Red Hat Enterprise Linux 5 update 5 kernel (RHEL5.5), 2.6.18-194.el5 (32-bit and 64-bit)
  • Red Hat Enterprise Linux 6 base kernel (RHEL6), 2.6.32-71.el6 (32-bit and 64-bit)
  • SUSE Linux Enterprise Server 11 SP1 kernel (SLES11SP1), 2.6.32.12-0.7 (32-bit and 64-bit)
  • Windows 7 (32-bit and 64-bit)
  • Windows Server 2008 R2 (64-bit)

See the Chelsio driver readme file (bundled with driver and available on the driver download website) for the latest information about the specific versions and service packs that are supported.

Go to http://www.chelsio.com/support for all Chelsio driver downloads.


Related publications

For more information, see the following resources:

Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment. publib-b.boulder.ibm.com

Profile

Publish Date
25 April 2011


Rating: Not yet rated


Author(s)

IBM Form Number
TIPS0825