Brocade Converged 10GbE Switch Module for IBM BladeCenter

IBM Redbooks Product Guide

Abstract

The Brocade Converged 10GbE Switch Module and Brocade 2-Port 10Gb Converged Network Adapter are part of a leading Converged Ethernet solution for IBM BladeCenter that offers Fibre Channel investment protection, maximum bandwidth and performance, and simplicity in a converged environment. The Brocade Converged 10GbE Switch Module also features Dynamic Ports on Demand capability through the Port Upgrade Key. This capability allows you to enable any combination of Fibre Channel and Ethernet ports based on your infrastructure requirements, and to experience Converged Ethernet benefits without significant investment.

Contents


The Brocade Converged 10GbE Switch Module and Brocade 2-Port 10Gb Converged Network Adapter are part of a leading Converged Ethernet solution for IBM BladeCenter that offers Fibre Channel investment protection, maximum bandwidth and performance, and simplicity in a converged environment. The Brocade Converged 10GbE Switch Module also features Dynamic Ports on Demand capability through the Port Upgrade Key. This capability allows you to enable any combination of Fibre Channel and Ethernet ports based on your infrastructure requirements, and to experience Converged Ethernet benefits without significant investment.

With the base model Converged 10GbE Switch Module, you can enable 16 of the 30 ports on the switch (eight 10Gb CEE external ports, eight 8Gb FC external ports, and 14x 10Gb CEE internal ports). If you purchase the Port Upgrade Key, you can enable all 30 ports on the Switch Module for a fully realized Converged Ethernet solution.

Figure 1 shows the switch module.

Brocade Converged 10GbE Switch Module for IBM BladeCenter
Figure 1. Brocade Converged 10GbE Switch Module for IBM BladeCenter


Did you know?

The Brocade CNAs and Brocade Converged 10GbE Switch Modules for IBM BladeCenter are the industry's first truly converged FCoE BladeCenter solution with native Fibre Channel and Ethernet ports built into a single switch module. The converged switch module provides up-front reduction in SAN/LAN cost and complexity. It requires fewer adapters, switch modules and cabling, which means an up-front reduction in your network investment.


Part number information

Table 1 lists the part numbers to order these modules and additional options for them.


Table 1. IBM part numbers and feature codes for ordering

DescriptionIBM part numberIBM feature code
Brocade Converged 10GbE Switch Module for IBM BladeCenter69Y19097656
Brocade 10Gb SFP+ SR Optical Transceiver49Y42160069
Brocade 8Gb SFP+ SW Optical Transceiver44X19625084
Brocade 14-port Converged 10GbE Switch Port Upgrade69Y19172425
Brocade ISL Trunking software upgrade 26K56074247
Brocade Fabric Watch software upgrade 45W05044398
Brocade Performance Monitor upgrade 45W05084248

Part number 69Y1909 include the following items:
  • One Brocade Converged 10GbE Switch Module for IBM BladeCenter (includes two 10GbE SFP+ modules standard)
  • IBM Important Notices document
  • IBM Warranty and Support Guide
  • One mini-USB console cable with serial connectors
  • BC-HT interposer gasket kit (used if the switch module is inserted in the BladeCenter HT chassis)
  • DB9 to RJ45 adapter
  • The IBM Documentation CD which contains the following documents:
    • Brocade Converged 10GbE Switch Module for IBM BladeCenter Installation and Users Guide
    • IBM Safety Information document (multilingual)
    • IBM Environmental Notices and Users Guide
    • Brocade EULA
    • Brocade CEE Admin guide
    • Brocade CEE CLI guide
    • Brocade MIBS manual

The base model of the Brocade Converged 10GbE Switch Module, 69Y1909, ships with 16 licensed ports. This module supports the Dynamic Ports on Demand (DPOD) feature, where ports are licensed as they come online independent of the port type. In the base model, two external CEE ports (ports 15 and 16) are pre-licensed and are pre-installed with two 10GbE optical transceivers. The remaining 14 licenses are available to be assigned to other internal or external ports. The 16 licensed ports may be any combination of external CEE ports, external FC ports, or internal CEE ports. After all the licenses have been assigned, you can manually move licenses from one port to another.

Additional ports can be activated by purchasing the Brocade Converged 10GbE Switch Port Upgrade, part number 69Y1917. This upgrade option activates the license for the remaining 14 ports. All internal and external ports will then be licensed and can be used.

The switch ships standard with two small form-factor pluggable plus (SFP+) modules. Additional SFP+ transceivers can be ordered separately if required. Table 1 lists the transceivers that are supported.


Benefits and features

The Brocade Converged 10GbE Switch Module with a supported converged network adapter (CNA) provides these benefits:
  • Highly integrated BladeCenter switch module
    The Brocade Converged 10GbE Switch Module offers one of the industry's best integrated I/O solutions. The compact design incorporates Ethernet and Fibre Channel switching and provides a total of 30 ports: eight external 10Gb Ethernet CEE ports for LAN connectivity and eight external 8Gb Fibre Channel ports for storage and SAN connectivity. With the high integration of this module, you can achieve all of your networking and storage I/O needs with a single module.
  • Unique flexible ports and server deployment
    The Dynamic Port on Demand (DPoD) feature of the Brocade Converged 10GbE Switch Module lets you activate any combination of internal or external ports. This means you can to balance server and I/O port assignments to accommodate the workloads of applications and satisfy business needs. DPoD gives you the ability to enable only 10GbE ports for networking needs, only Fibre Channel ports for storage support, or any combination of Ethernet and Fibre Channel ports
  • Support for Data Center Fabric Manager

    IBM System Storage Data Center Fabric Manager (DCFM) is available for use with the switch. DCFM is fully integrated with IBM Systems Director for end-to-end data center management. A 75-day trial version and can be downloaded from:

    http://www.ibm.com/systems/storage/san/b-type/dcfm/

  • Highest bandwidth and low latency
    The total available bandwidth of 144 Gbps (80 Gbps Ethernet and 65 Gbps Fibre Channel) accessible through external ports means the switch module can support high demanding applications and storage solutions. In addition to the high bandwidth available on external ports, the switch module supports an additional 140 Gbps on 14 internal 10GbE ports. The switch module uses cut-through and non-blocking to deliver high performance and low latency for demanding virtual applications and high speed environments.
  • Low total cost of ownership
    The highly integrated design delivers the lowest total cost of ownership (TCO). The Brocade Converged 10GbE Switch Module is two switches in one, Ethernet and Fibre Channel, which reduces the cost of acquisition. The switch module is built around a pay-as-you-go model that gives you a lower price entry point and lets you add ports as your business needs dictate. The switch module ships with two 10GbE SFP+ transceivers to further lower initial investments. Because it is designed for seamless integration into existing networking and storage environments, this module truly protects existing investments. Overall, with this integrated switch module, your server infrastructure will use fewer hardware components with lower costs and higher reliability, which further drives a lower TCO.

The converged switch modules have the following features:
  • Eight external 10 Gb Converged Enhanced Ethernet ports
  • Eight external auto-negotiated Fibre Channel ports (2 Gbps, 4 Gbps, or 8 Gbps)
  • 14 internal auto-negotiated 10 GbE ports (1 Gbps or 10 Gbps)
  • Two internal full-duplex 100 Mbps Ethernet interfaces for management purposes
  • One external RS232 console port with a mini-USB interface for serial console management
  • One external 10/100/1000 Mb RJ45 Ethernet copper port for management
  • CEE features
    • Priority-based Flow Control (PFC): IEEE 802.1Qbb
    • Enhanced Transmission Selection (ETS): IEEE 802.1Qaz
    • Data Center Bridging eXchange (DCBX)
  • Layer 2 features
    • Layer 2 Virtual Local Area Networks (VLANs): 4096
    • VLAN Encapsulation 802.1Q
    • Rapid Spanning Tree Protocol (RSTP)
    • Multiple Spanning Tree MSTP (802.1s): 16 instances
    • Link Aggregation Control Protocol (LACP) IEEE- 802.3ad
    • Brocade enhanced frame-based trunking
    • Advanced PortChannel hashing based on Layer 2, 3, and 4 information
    • Pause Frames (802.3x)
    • Storm Control (unicast, multicast, and broadcast)
    • Address Resolution Protocol (ARP) RFC 826
  • Layer 2 security
    • Ingress Access Control Lists (ACLs)
    • Standard and extended Layer 2 ACLs
    • VLAN-based ACLs (VACLs)
    • Port-based ACLs (PACLs)
    • ACL statistics
    • Port-based Network Access Control: IEEE 802.1X
  • Layer 2 Quality of Service (QoS)
    • Eight priority levels for QoS
    • IEEE 802.1p Class of Service (CoS)
    • Eight hardware queues per port
    • Per-port QoS configuration
    • CoS trust: IEEE 802.1p
    • Per-port Virtual Output Queuing
    • CoS-based egress queuing
    • Egress strict priority queuing
    • Egress port-based scheduling: Weighted Round-Robin (WRR)
  • Layer 3 features
    • Static IP routes
  • External Fibre Channel ports that can operate as F_ports (fabric ports), FL_ports (fabric loop ports), or E_ports (expansion ports)
  • FC fabric services
    • Simple Name Server (SNS)
    • Registered State Change Notification (RSCN)
    • Dynamic Path Selection (DPS)
    • Enhanced Group Management (EGM)
    • ISL Trunking (optional)
    • Fabric Watch (optional)
    • Advanced Performance Monitor (optional)
  • Power-on self-test diagnostics and status reporting


The following software features come with the switch modules:
  • Brocade Web Tools
  • 16-port licensing


The switch supports the following fabric management (all management connections go through the management module except direct serial connection, which goes through the mini-USB port):
  • Web interface through Web Tools
  • Command-line interface (CLI) through the Telnet program
  • A terminal emulation program connection to the mini-USB port interface
  • IBM System Storage Data Center Fabric Manager (DCFM) application
  • Switch's SNMP agent



Supported BladeCenter chassis and expansion cards

The Brocade Converged 10GbE Switch Module is supported in the IBM BladeCenter H and HT chassis as listed in Table 2.

Table 2. IBM BladeCenter chassis that support the Brocade Converged 10GbE Switch Module

I/O module

Part number

BladeCenter S

BladeCenter E

BladeCenter H

BladeCenter T

BladeCenter HT

MSIM

MSIM-HT

Brocade Converged 10GbE Switch Module

69Y1909

N

N

Y

N

Y

N

N

The Brocade Converged 10GbE Switch Module supports the expansion cards listed in Table 3. This table also lists the chassis bays in which the switch module must be installed when used with each expansion card.

Table 3. Brocade Converged 10GbE Switch Module and BladeCenter chassis I/O bays support

Description

Part number

Bay 1 (Standard)

Bay 2 (Standard)

Bay 3 (Standard)

Bay 4 (Standard)

Bay 5 (Bridge)

Bay 6 (Bridge)

Bay 7 (High-speed)

Bay 8 (High-speed)

Bay 9 (High-speed)

Bay 10 (High-speed)

Gigabit Ethernet integrated in the server

None

N

N

N

N

N

N

N

N

N

N

Ethernet Expansion Card (CFFv)

39Y9310

N

N

N

N

N

N

N

N

N

N

Ethernet Expansion Card (CIOv)

44W4475

N

N

N

N

N

N

N

N

N

N

QLogic Ethernet and 4Gb FC Card (CFFh)

39Y9306

N

N

N

N

N

N

N

N

N

N

2/4 Port Ethernet Expansion Card (CFFh)

44W4479

N

N

N

N

N

N

N

N

N

N

QLogic Ethernet and 8Gb FC Card (CFFh)

44X1940

N

N

N

N

N

N

N

N

N

N

NetXen 10Gb Ethernet Expansion Card (CFFh)

39Y9271

N

N

N

N

N

N

N

N

N

N

Broadcom 2-port 10Gb Ethernet Expansion Card (CFFh)

44W4466

N

N

N

N

N

N

N

N

N

N

Broadcom 4-port 10Gb Ethernet Expansion Card (CFFh)

44W4465

N

N

N

N

N

N

N

N

N

N

QLogic 2-port 10Gb Converged Network Adapter (CFFh)

42C1830

N

N

N

N

N

N

Y

N*

Y

N*

Emulex Virtual Fabric Adapter (CFFh)

49Y4245

N

N

N

N

N

N

N

N

N

N

Intel 2-port 10Gb Ethernet Expansion Card (CFFh)

42C1810

N

N

N

N

N

N

N

N

N

N

Brocade 2-port 10GbE Converged Network Adapter

81Y1650

N

N

N

N

N

N

Y

N*

Y

N*
* One Brocade Converged 10GbE Switch Module occupies two adjacent high-speed bays, and the ports on supported cards are physically routed to the bays 7 and 9.

The BladeCenter chassis have the following bays:
  • BladeCenter S, E, and T have four standard I/O bays (1, 2, 3, and 4)
  • BladeCenter H has four standard I/O bays (1, 2, 3, and 4), two bridge bays (5 and 6), and four high-speed bays (7, 8, 9, and 10)
  • BladeCenter HT has four standard I/O bays (1, 2, 3, and 4) and four high-speed bays (7, 8, 9, and 10)

The Brocade Converged 10GbE Switch Module fits in one of the high-speed I/O bay (bays 7-10).


Popular configurations

Figure 2 shows the use of Brocade Converged 10GbE Switch Modules to route two 10 Gb Converged Enhanced Ethernet ports from a Brocade 2-port 10GbE Converged Network Adapter installed into each server. Two Brocade Converged 10GbE Switch Modules are installed in bays 7/8 and bays 9/10 of the BladeCenter H chassis. All connections between the controller, card, and the switch modules are internal to the chassis. No internal cabling is needed.

A converged 20 Gb solution using two Brocade Converged 10GbE Switch Modules
Figure 2. A converged 20 Gb solution using redundant Brocade Converged 10GbE Switch Modules

The components used in this configuration are listed in Table 4.

Table 4. Components used when connecting Brocade 2-port 10GbE Converged Network Adapter to two Brocade Converged 10GbE Switch Modules
Diagram referencePart number/machine typeDescriptionQuantity
1VariesIBM BladeCenter HS22 or other supported server1 to 14 or 1 to 12
281Y1650Brocade 2-port 10GbE Converged Network Adapter1 per server
38852 or 8740/8750BladeCenter H or BladeCenter HT1
469Y1909Brocade Converged 10GbE Switch Module1 to 2
549Y4216Brocade 10Gb SFP+ SR Optical TransceiverUp to 6*
644X1962Brocade 8Gb SFP+ SW Optical TransceiverUp to 6*
*The Brocade Converged 10GbE Switch Module has 16 external ports - eight 10 Gb CEE ports and eight 8 Gb Fibre Channel ports. You must have one transceiver for each port in an I/O module (every switch module comes standard with two 10Gb Ethernet SFP+ transceivers).

Refer to the IBM System Storage Interoperability Center web link for information about the compatibility of Brocade Converged 10GbE Switch Modules with storage systems and fabrics:

http://www.ibm.com/systems/support/storage/config/ssic

Connectors and LEDs

Figure 3 shows the front panel of the Brocade Converged 10GbE Switch Module.

Front panel of the Brocade Converged 10GbE Switch Module
Figure 3. Front panel of the Brocade Converged 10GbE Switch Module

Table 5 explains the components shown in Figure 3.

Table 5. Key to components in Figure 3
Diagram referenceDescription
1Status/Fault LED (amber)
2Power LED (green)
3Ethernet port (RJ45)
4Console port (mini-USB)
5Fibre Channel ports (23-30 - right to left)
6CEE ports (15-22 - right to left)
7CEE port fault status LED (amber)
8CEE port link status LED (green)
9FC port fault status LED (amber)
10FC port Tx/Rx link status LED (green)
11Ethernet port speed LED (green)
12Ethernet port Link LED (green)
13Release levers (2)

The front panel contains the following components:
  • LEDs that display the following information:
    • The status of the switch module and its network connection
    • The status of the external connections to the switch module
  • Eight FC SFP+ port connectors to attach SFP+ modules. They are numbered 23-30 right to left.
  • Eight FCoE/CEE SFP+ port connectors to attach SFP+ modules. They are numbered 15-22 right to left.
  • One RJ-45 Ethernet port connector for debug and field support.
  • One mini-USB console port connector used for switch management. This port requires a mini-USB plug on the serial cable.



Network cabling requirements

The following network cables are supported for the Brocade Converged 10GbE Switch Module:
  • 10GBASE-SR for 10Gb CEE ports
    • 850 Nm wavelength, multimode fiber, 50 µ or 62.5 µ (300 meters maximum), with LC duplex connector
  • 8 Gb FC ports when operating at 8 Gbps speed
    • 850 Nm wavelength, multimode fiber, 50 µ (150 meters maximum) or 62.5 µ (21 meters maximum), with LC duplex connector
  • 1000BASE-T for RJ-45 port
    • UTP Category 6 (100 meters maximum)
    • UTP Category 5e (100 meters maximum)
    • UTP Category 5 (100 meters maximum)
    • EIA/TIA-568B 100-ohm STP (100 meters maximum)


Operating environment

The switch module has the following physical specifications:
  • Dimensions:
    • Approximate height: 4.10 cm (1.61 inches)
    • Approximate width: 29.38 cm (11.57 inches)
    • Approximate depth: 25.70 cm (10.12 inches)
    • Approximate weight: 2.13 kg (4.7 lb)
  • Operating:
    • 0° to 40° C (32° to 104° F) at 0 to 3,048 m (0 to 10,000 ft) operating
  • Non-operating:
    • -20° to 70° C (-4°F to 158° F) at 0 to 10,688 km (0 to 35,000 ft) non-operating
  • Humidity:
    • 10% to 90% (non-condensing) at 29° C operating
    • 5% to 95% (non-condensing) at 38° C non-operating


Related publications
For more information refer to these documents:

Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.

Profile

Publish Date
06 October 2010

Last Update
07 August 2011


Rating:
(based on 1 review)


Author(s)

IBM Form Number
TIPS0789