Chelsio 10 GbE Adapters for IBM Power Systems

IBM Redbooks Product Guide

Abstract

Chelsio 10 GbE adapters for IBM Power Systems are high performance, 10 Gb Ethernet adapters with PCI Express host bus interfaces optimized for virtualization, high performance computing, and storage applications. The third generation technology from Chelsio used in these adapters provides the highest 10 GbE performance available and dramatically lowers the host system's CPU communications' impact.

Contents


Chelsio 10 GbE adapters for IBM Power Systems are high performance, 10 Gb Ethernet adapters with PCI Express host bus interfaces optimized for virtualization, high performance computing, and storage applications. The third generation technology from Chelsio used in these adapters provides the highest 10 GbE performance available and dramatically lowers host system's CPU communications' impact.

With on-board hardware that off-loads TCP/IP, iSCSI, and iWARP RDMA processing from its host system, single port adapters frees up host CPU cycles for useful applications, which results in increased bandwidth and lower latency. This combination makes it practical to converge other networks that traditionally used niche technologies onto 10 GbE Ethernet adapters. High band-width and extremely low latency make 10 GbE Ethernet adapters with protocol offload the best technology for high-performance cluster computing (HPCC) fabrics.

Figure 1 shows the Chelsio single port 10 GbE adapters for IBM Power Systems.

10 Gigabit Ethernet-SR PCI Express Adapte 10 Gigabit Ethernet-CX4 PCI Express Adapter
Figure 1. The 10 Gigabit Ethernet-SR PCI Express Adapter (left) and 10 Gigabit Ethernet-CX4 PCI Express Adapter (right)



Did you know

Chelsio delivers on the promise of the unified wire, enabling the convergence of server networking, storage networking, and cluster computing interconnects onto a single platform and a single fabric.


Part number information

Table 1. Ordering part number and feature code
DescriptionIBM part numberIBM feature codeChelsio part number
10 Gigabit Ethernet-SR PCI Express Adapter46K78975769S310E-SR+
10 Gigabit Ethernet-CX4 PCI Express Adapter46K78995732S310E-CXA


Features

The 10 Gigabit Ethernet-SR and CX4 PCI Express Adapters are low-profile, high performance adapters. The SR adapter uses a LC Duplex type connector that is capable of transferring data a distance of 300 m over MMF-850 nm fiber cable and conforms to the 10GBASE-SR specification. The CX4 adapter uses a CX4 type connector that is capable of transferring data a distance of 15 m over copper cabling and conforms to the 10GBASE-CX4 specification.

The adapters have the following features:
  • Third generation technology (T3) from Chelsio
  • The Unified Wire interconnect solution for server networking, storage networking, and clustering on a single platform
  • Very low latency Ethernet
  • Reduced host CPU utilization by up to 90% compared to NICs without full offload capabilities
  • PCI Express 1.1 x8 host bus interface
  • Line-rate 10 Gbps full-duplex performance
  • Integrated traffic manager, QoS, and virtualization capabilities
  • TCP sockets, SCSI, RNIC-PI, kDAPL, and OpenFabrics 1.3 software interfaces
  • Ideal fit for servers and NAS systems
  • Consolidated LAN, SAN, and cluster networks with TOE, iSCSI, and iWARP
  • Enabled high performance NAS systems and Ethernet-based IP SANs
  • Deployed Ethernet-only IP SANs throughout the enterprise beyond the data center
  • Developed shared-storage systems providing both file- and block-level services
  • Deployed Ethernet-only networking for cluster fabric, LAN, and SAN
  • Full TCP offload, iSCSI, and iWARP RDMA plus direct data placement (DDP)
  • Increased cluster fabric bandwidth and deployed Ethernet-only networking for cluster fabric, LAN, and SAN

Table 2 compares the adapters.

Table 2. Comparison of the adapter cards
Feature10 Gigabit Ethernet-SR PCI Express Adapter10 Gigabit Ethernet-CX4 PCI Express Adapter
Feature code57695732
Form factorLow profile PCIe cardLow profile PCIe card
Ports11
ConnectorSFP+ transceiverCX4
Power consumption17 W16 W


Specifications

The Chelsio 10 GbE adapters for IBM Power Systems have the following specifications:
  • Network interfaces
    • 10 Gigabit Ethernet-SR PCI Express Adapter: 10GBASE-SR short-reach optics (850 nm)
    • 10 Gigabit Ethernet-CX4 PCI Express Adapter: 10GBASE-CX short-reach copper

The 10 Gigabit Ethernet-SR PCI Express Adapter also supports 10GBASE-LR long-reach optics (1310 nm) with the use of a LR transceiver (optional, available directly from Chelsio using part number SM10G-LR)
  • Ethernet
    • IEEE 802.3ae (10 GbE)
    • IEEE 802.1p Priority and 802.1Q VLAN tagging
    • IEEE 802.3x flow control
    • IEEE 802.3ad link aggregation
    • Ether II and 802.3 encapsulated frames
    • Multiple MAC addresses per interface
    • Jumbo Frames up to 9.6 KB
  • Stateless off-loads
    • TCP checksum offload for IPv4 and IPv6
    • TCP Segmentation Offload (TSO) for IPv4 and IPv6
    • Large Receive Offload (LRO) for IPv4 and IPv6
    • Large Send Offload (LSO) for IPv4 and IPv6
    • UDP checksum offload for IPv4 and IPv6
    • Receive Side Scaling and packet steering
    • Line rate packet filtering and attack protection
  • TCP/IP full offload
    • Full TCP implementation including exceptions
    • High performance even during packet loss
    • Extensive RFC compliance, fully featured stack
    • Zero copy on Tx and Rx
    • Direct Data Placement (DDP)
    • Up to 64 K simultaneous connections capacity
  • Integrated traffic manager
    • Multiple Tx and Rx queues with QoS
    • Simultaneous low latency and high bandwidth
    • Per-connection and per-class rate control
    • Packet loss avoidance
  • Virtualization and firewall
    • Rule-based packet steering and filtering capability
  • iSCSI acceleration
    • Full iSCSI initiator and target mode stack
    • Header and Data Digest (CRC) generation and checking
    • PDU recovery
    • Direct Data Placement (DDP)
  • High performance RDMA
    • Ultra-low latency and line rate bandwidth
    • IETF RDDP and RDMAC iWARP compliance
    • APIs: RNIC-PI, kDAPL, and OpenFabrics 1.3

RDMA features are currently only supported by Linux.


Physical specifications

The adapters have the following physical specifications:
  • Width: 168 mm (6.60 in.)
  • Height: 64 mm (2.52 in.)
  • Depth: 18 mm (0.69 in.)


Operating environment

The adapters are supported in the following environment:

  • Operating temperature: 0 to 55° C (32° to 131° F)
  • Operating humidity: 5 to 95%
  • Typical power consumption:
    • 10 Gigabit Ethernet-SR PCI Express Adapter: 17 W
    • 10 Gigabit Ethernet-CX4 PCI Express Adapter: 16 W
  • Minimum required airflow: 200 lf/m


Supported servers

The Chelsio 10 GbE adapters for IBM Power Systems are supported in the IBM servers that are listed in Table 3.

Table 3. Supported servers

Adapter

Feature

IBM Power 520 Express

IBM Power 560 Express

IBM Power 570

IBM Power 575

IBM Power 595

IBM Power 750

IBM Power 755

IBM Power 770

IBM Power 780

IBM Smart Cube

10 Gigabit Ethernet-SR PCI Express Adapter

5769

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

10 Gigabit Ethernet-CX4 PCI Express Adapter

5732

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y


Supported operating systems

The Chelsio adapters support the following operating systems (including support for both TOE and NIC mode, unless otherwise stated):
  • Red Hat Enterprise Linux 3 base kernel (NIC only; no TOE support)
  • Red Hat Enterprise Linux 4 U3, U4, U5, U6, U7, and U8
  • Red Hat Enterprise Linux 5 base, U1, U2, U3, and U4
  • SUSE Linux Enterprise Server 10 base, SP1, and SP2
  • SUSE Linux Enterprise Server 9 SP3 (NIC only, no TOE support)
  • SUSE Linux Enterprise Server 11 base kernel
  • Kernel.org linux-2.6.9, 2.6.12, 2.6.16, 2.6.20
  • AIX 5.3 (NIC only; no TOE support)
  • AIX 6.1 (NIC only; no TOE support)

Note: The supported servers will have other operating system support requirements that must also be met. Check the Software Requirements section of the relevant announcement letter of the server for the specific operating system levels supported.

See the Chelsio driver README file for the latest information about the specific versions and service packs that are supported. This README file is available at the following address:

http://www.chelsio.com/support


Related publications

For more information, see these resources:

Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment. publib-b.boulder.ibm.com

Profile

Publish Date
14 April 2010


Rating: Not yet rated


Author(s)

IBM Form Number
TIPS0774