IBM Flex System FC5022 16Gb SAN Scalable Switches

The IBM Flex System™ FC5022 16Gb SAN Scalable Switch is a high-density, 48-port, 16 Gbps Fibre Channel switch that is used in the IBM Flex System chassis. The switch provides 28 internal ports to compute nodes by way of the midplane, and 20 external SFP+ ports. These SAN switch modules deliver an embedded option for IBM Flex System users deploying storage area networks in their enterprise. They offer end-to-end 16 Gb and 8 Gb connectivity. The N-Port Virtualization mode streamlines the infrastructure by reducing the number of domains to manage while enabling the ability to add or move servers without impact to the SAN. Monitoring is simplified by an integrated management appliance, or clients using end-to-end IBM B-type SAN can exploit the IBM management tools.

Figure 1 shows the switch module.


Figure 1. IBM Flex System FC5022 16Gb SAN Scalable Switch


Did you know?

Diagnostic ports (D_ports) are a new port type supported by the FC5022 16 Gb switches that enables administrators to quickly identify and isolate 16 Gbps optics, port, and cable problems, reducing fabric deployment and diagnostic times. If the optical media is found to be the source of the problem, it can be transparently replaced because 16 Gbps optics are hot-pluggable.

IBM Flex System, a new category of computing and the next generation of Smarter Computing, offers intelligent workload deployment and management for maximum business agility. This chassis delivers high-speed performance complete with integrated servers, storage, and networking for multi-chassis management in datacenter compute environments. Furthermore, its flexible design can meet the needs of varying workloads with independently scalable IT resource pools for higher utilization and lower cost per workload. While increased security and resiliency protect vital information and promote maximum uptime, the integrated, easy-to-use management system reduces setup time and complexity, providing a quicker path to ROI.


Part number information

The part numbers to order the switch and optional upgrades are shown in the following table.

Table 1. Part numbers and feature codes for ordering

Description	Part number	Feature code (x-config / e-config)
Switch modules
IBM Flex System FC5022 16Gb SAN Scalable Switch	88Y6374	A1EH / 3770
IBM Flex System FC5022 24-port 16Gb SAN Scalable Switch	00Y3324	A3DP / ESW5
IBM Flex System FC5022 24-port 16Gb ESB SAN Scalable Switch	90Y9356	A2RQ / 3771
Features on Demand upgrades
IBM Flex System FC5022 16Gb SAN Scalable Switch (Upgrade 1)	88Y6382	A1EP / 3772
IBM Flex System FC5022 16Gb SAN Scalable Switch (Upgrade 2)	88Y6386	A1EQ / 3773
IBM Flex System FC5022 16Gb Fabric Watch Upgrade	00Y3320	A3HN / ESW3
IBM Flex System FC5022 16Gb ISL/Trunking Upgrade	00Y3322	A3HP / ESW4

The part numbers for the switches include the following items:

• One IBM Flex System FC5022 16Gb SAN Scalable Switch (12-port or 24-port or 24-port with ESB)
• Two 16 Gb SFP+ optical transceivers (included with 24-port 16Gb SAN Scalable Switch only)
• Important Notices Flyer
• Warranty Flyer
• Documentation CD-ROM

The part numbers for the Features on Demand upgrades include the following items:

• Features on Demand Activation Flyer
• Upgrade activation key

The switch does not include a serial management cable; however, IBM Flex System Management Serial Access Cable, 90Y9338, is supported and contains two cables, a mini-USB-to-RJ45 serial cable and a mini-USB-to-DB9 serial cable, either of which can be used to connect to the switch locally for configuration tasks and firmware updates.

The FC5022 12-port and 24-port ESB SAN switches come without SFP+, which must be ordered separately to provide outside connectivity. The FC5022 24-port SAN switch comes standard with two Brocade 16 Gb SFP+ transceivers, additional SFP+ can be ordered if required. The following table lists supported SFP+ options.

Table 2. Supported SFP+ transceivers

Description	Part number	Feature code (x-config / e-config)
Brocade 8Gb SFP+ SW Optical Transceiver	88Y6416	5084 / 5370
Brocade 16Gb SFP+ Optical Transceiver	88Y6393	A22R / 5371

The following table provides a feature comparison by model for IBM Flex System FC5022 switches.

Table 3. Feature comparison by model

Feature	24-port 16 Gb ESB SAN switch	24-port 16 Gb SAN switch	16 Gb SAN switch
Number of licensed ports	24	24	12
Number of SFP+ included	0	2x 16 Gb SFP+	0
Full fabric	Included	Included	Included
Access Gateway	Included	Included	Included
Advanced zoning	Included	Included	Included
Enhanced Group Management	Included	Included	Included
ISL Trunking	Included	Optional	Optional
Adaptive Networking	Included	Not available	Not available
Advanced Performance Monitoring	Included	Not available	Not available
Fabric Watch	Included	Optional	Optional
Extended Fabrics	Included	Not available	Not available
Server Application Optimization	Included	Not available	Not available

The following table provides compatibility information for the switch modules and Features on Demand upgrades for them.

Table 4. Features on Demand upgrade compatibility

Features on Demand upgrade	Part number	24-port 16 Gb ESB SAN switch	24-port 16 Gb SAN switch	16 Gb SAN switch
IBM Flex System FC5022 16Gb SAN Scalable Switch (Upgrade 1)	88Y6382	No	No	Yes
IBM Flex System FC5022 16Gb SAN Scalable Switch (Upgrade 2)	88Y6386	Yes	Yes	Yes
IBM Flex System FC5022 16Gb Fabric Watch Upgrade	00Y3320	No	Yes	Yes
IBM Flex System FC5022 16Gb ISL/Trunking Upgrade	00Y3322	No	Yes	Yes

With Dynamic Ports on Demand (DPOD), ports are licensed as they come online. With the 16 Gb SAN switch, the first 12 ports reporting (on a first-come, first-served basis) on boot-up are assigned licenses. These 12 ports can be any combination of external or internal Fibre Channel (FC) ports. After all licenses have been assigned, you can manually move those licenses from one port to another. Because this is dynamic, no defined ports are reserved except ports 0 and 29. The 24-port 16 Gb SAN switches have the same behavior, the only difference is the number of ports.

The following table shows the total number of port licenses on the switch after applying compatible port upgrades.

Table 5. Ports on Demand upgrades

Ports on Demand upgrade	Total number of port licenses
	24-port 16 Gb ESB SAN switch	24-port 16 Gb SAN switch	16 Gb SAN switch
Included with base	24	24	12
Upgrade 1, 88Y6382 (adds 12 port licenses)	Not supported	Not supported	24
Upgrade 2, 88Y6386 (adds 24 port licenses)	48*	48*	48*

* With the dual-port FC adapters installed in compute nodes, up to 34 ports (14 internal, 20 external) can be utilized. The remaining 14 internal ports are reserved for future use.


Benefits

The switches offer the following key benefits:

• Exceptional price/performance for growing SAN workloads

The FC5022 16Gb Switch delivers exceptional price/performance for growing SAN workloads through a combination of market-leading 1600 MB/sec throughput per port and an affordable high-density form factor. The 48 FC ports produce an aggregate 768 Gbps full-duplex throughput, plus any external eight ports can be trunked for 128 Gbps Inter-Switch Links (ISLs). Because 16 Gbps port technology dramatically reduces the number of ports and associated optics/cabling required through 8/4 Gbps consolidation, the cost savings as well as simplification benefits are substantial.

• Accelerating fabric deployment and serviceability with diagnostic ports

Diagnostic Ports (D_Ports) are a new port type supported by the FC5022 16Gb Switch that enables administrators to quickly identify and isolate 16 Gbps optics, port, and cable problems, reducing fabric deployment and diagnostic times. If the optical media is found to be the source of the problem, it can be transparently replaced because 16 Gbps optics are hot-pluggable.

• A building block for virtualized, private cloud storage

The FC5022 16Gb Switch supports multi-tenancy in cloud environments through VM-aware end-to-end visibility and monitoring, Quality of Service (QoS), and fabric-based advanced zoning features. The switch enables secure distance extension to virtual private or hybrid clouds with dark fiber support, as well as in-flight encryption and data compression. Internal fault-tolerant and enterprise-class RAS features help minimize downtime to support mission-critical environments.

• Simplified and optimized interconnect with Access Gateway operational mode

The FC5022 16Gb Switch can be deployed as a full-fabric switch or as an Access Gateway, which simplifies fabric topologies and heterogeneous fabric connectivity. Access Gateway mode utilizes N_Port ID Virtualization (NPIV) switch standards to present physical and virtual servers directly to the core of SAN fabrics. This makes it transparent to the SAN fabric, greatly reducing management of the network edge. The IBM SAN Scalable Switch in Access Gateway mode can connect servers to NPIV-enabled IBM, Brocade B-Series, or other vendor SAN fabrics.

• Maximizing investments

To help optimize technology investments, IBM offers a single point of serviceability backed by industry-renowned education, support, and training. In addition, the IBM 16/8 Gbps SAN Scalable Switch is in the ServerProven® program, ensuring compatibility among a variety of IBM and partner products. IBM recognizes that customers deserve the most innovative, expert integrated systems solutions.

Features and specifications

The FC5022 16Gb Switches have the following features and specifications:

• Internal ports
  • 28 internal full-duplex 16 Gb FC ports (up to 14 internal ports can be activated with Ports on Demand feature, remaining ports are reserved for future use)
  • Internal ports operate as F_ports (fabric ports) in native mode or in access gateway mode
  • Two internal full-duplex 1 GbE ports connected to the chassis management module
• External ports
  • 20 external ports for 16 Gb SFP+ or 8 Gb SFP+ transceivers supporting 4 Gb, 8 Gb, and 16 Gb port speeds (supported SFP+ modules are listed in Table 2). Ports are activated with Ports on Demand feature.
  • External ports can operate as F_ports (fabric ports) or E_ports (expansion ports) in native mode or as N_ports (node ports) in access gateway mode
  • One external 1 GbE port (1000BASE-T) with RJ-45 connector for switch configuration and management
  • One RS-232 serial port (mini-USB connector) that provides an additional means to configure the switch module
• Access gateway mode (N_Port ID Virtualization - NPIV) support
• Power-on self-test diagnostics and status reporting
• Inter-Switch Link (ISL) Trunking (licensable), which allows up to eight ports (at 16, 8, or 4 Gbps speeds) to combine to form a single, logical ISL with a speed of up to 128 Gbps (256 Gbps full duplex) for optimal bandwidth utilization, automatic path failover, and load balancing
• Fabric OS (FOS) version 7.0.x or higher, which delivers distributed intelligence throughout the network and enables a wide range of value-added applications, such as Advanced Web Tools and Advanced Fabric Services (on certain models)
• Supports up to 768 Gbps I/O bandwidth
• 420 million frames switch per second, 0.7 microseconds latency
• 8192 buffers for up to 3750 km extended distance at 4 Gbps FC (Extended Fabrics license required)
• In-flight 64 Gbps Fibre Channel compression and decompression support on up to two external ports (no license required)
• In-flight 32 Gbps encryption and decryption on up to two external ports (no license required)
• 48 Virtual Channels (VCs) per port
• Port mirroring to monitor ingress or egress traffic from any port within the switch
• Two I2C connections able to interface to redundant management modules
• Hot pluggable — up to 4 hot pluggable switches per chassis
• Single fuse circuit
• Four temperature sensors
• One internal real-time clock
• Managed with Web Tools
• Supports a minimum of 128 domains in Native mode and Interoperability mode
• Full fabric switch change to Access Gateway mode
• Nondisruptive code load in Native mode and Access Gateway mode
• 255 N_port logins per physical port
• D_port support on external ports
• Class 2 and Class 3 frames
• SNMP v1 and v3 support
• SSH v2 support
• SSL support
• NTP client support (NTP V3)
• FTP support for firmware upgrades
• SNMP/MIB monitoring functionality contained within the Ethernet Control MIB-II (RFC1213-MIB)
• UUID put into MIB structure
• End-to-end optics and link validation
• Ability to send switch events and syslogs to the IBM Chassis Management Module (CMM)
• Traps identify cold start, warm start, link up/link down, and authentication failure events
• Support for IPv4 and IPv6 on the management ports

IBM Flex System FC5022 switches come standard with the following software features:

• Full Fabric mode: Enables high performance 16 Gb or 8 Gb fabric switching
• Access Gateway mode: Leverages NPIV to connect to any fabric without adding switch domains to reduce management complexity
• Dynamic Path Selection: Enables exchange-based load balancing across multiple Inter-Switch Links for superior performance
• Advanced Zoning: Segments a SAN into virtual private SANs to increase security and availability
• Enhanced Group Management: Enables centralized and simplified management of SAN fabrics through IBM Network Advisor

The 24-port ESB model (90Y9356) is a fully enabled switch with a complete set of licensed features that maximizes performance, ensures availability, and simplifies management for the most demanding applications and expanding virtualization environments. The factory-installed licenses include:

• ISL Trunking: Consolidates ISLs into fault-tolerant and load-balanced trunks with bandwidth of up to 128 Gbps for greater performance and simplified management
• Adaptive Networking: Ensures high-priority connections that obtain the bandwidth necessary for optimum performance
• Advanced Performance Monitoring: Provides end-to-end visibility into fabric performance to maximize performance tuning and the utilization of fabric resources
• Fabric Watch: Monitors and creates alerts based on the health of switch and fabric elements
• Extended Fabrics: Extends Fibre Channel SANs beyond traditional distance limitations for replication and backup at full bandwidth
• Server Application Optimization: Allows virtual machine quality of service, authentication, and monitoring from SAN through adapter to Virtual Machine and associated virtualized application

Supported Fibre Channel standards

The switch supports the following Fibre Channel standards:

• FC-AL-2 INCITS 332: 1999
• FC-GS-5 ANSI INCITS 427 (includes the following):
  • FC-GS-4 ANSI INCITS 387: 2004
• FC-IFR INCITS 1745-D, revision 1.03 (under development)
• FC-SW-4 INCITS 418:2006 (includes the following):
  • FC-SW-3 INCITS 384: 2004
• FC-VI INCITS 357: 2002
• FC-TAPE INCITS TR-24: 1999
• FC-DA INCITS TR-36: 2004 (includes the following):
  • FC-FLA INCITS TR-20: 1998
  • FC-PLDA INCIT S TR-19: 1998
• FC-MI-2 ANSI/INCITS TR-39-2005
• FC-PI INCITS 352: 2002
• FC-PI-2 INCITS 404: 2005
• FC-PI-4 INCITS 1647-D, revision 7.1 (under development)
• FC-PI-5 INCITS 479: 2011
• FC-FS-2 ANSI/INCITS 424:2006 (includes the following):
  • FC-FS INCITS 373: 2003
• FC-LS INCITS 433: 2007
• FC-BB-3 INCITS 414: 2006 (includes the following):
• FC-BB-2 INCITS 372: 2003
• FC-SB-3 INCITS 374: 2003 (replaces FC-SB ANSI X3.271: 1996; FC-SB-2 INCITS 374: 2001)
• RFC 2625 IP and ARP Over FC
• RFC 2837 Fabric Element MIB
• MIB-FA INCITS TR-32: 2003
• FCP-2 INCITS 350: 2003 (replaces FCP ANSI X3.269: 1996)
• SNIA Storage Management Initiative Specification (SMI-S) Version 1.2 (includes the following):
  • SNIA Storage Management Initiative Specification (SMI-S) Version 1.03 ISO standard IS24775-2006. Replaces (ANSI INCITS 388: 2004)
  • SNIA Storage Management Initiative Specification (SMI-S) Version 1.1.0
  • SNIA Storage Management Initiative Specification (SMI-S) Version 1.2.0

Supported chassis and adapter cards

The switches are installed in switch bays in the rear of the IBM Flex System Enterprise Chassis as shown in the following figure. Switches are normally installed in pairs since I/O adapter cards installed in the compute nodes route to two switch bays for redundancy and performance.


Figure 2. Location of the switch bays in the IBM Flex System Enterprise Chassis

The connections between the adapters installed in the compute nodes to the switch bays in the chassis are shown diagrammatically in the following figure. The figure shows both half-wide servers, such as the x240 with two adapters, and full-wide servers, such as the p460 with four adapters.


Figure 3. Logical layout of the interconnects between I/O adapters and I/O modules

The IBM Flex System FC5022 16Gb SAN Scalable Switches can be installed in bays 1, 2, 3, and 4 of the Enterprise chassis. A supported adapter card must be installed in a corresponding slot of the compute node (slot A1 when switches are installed in bays 1 and 2 or slot A2 when switches are in bays 3 and 4).

With compute nodes that have an integrated dual-port 10 GbE network interface controller (NIC) these switches can only be installed in bays 3 and 4 because integrated NICs ports are routed to bays 1 and 2 with a specialized periscope connector and the FC adapter card cannot be installed in slot A1. However, when needed, the periscope connector can be replaced with an FC adapter card, in which case the integrated NIC will be disabled and FC switches can be used in bays 1 and 2.

The following table shows the connections between adapters installed in the compute nodes to the switch bays in the chassis with IBM Flex System FC5022 16Gb SAN Scalable Switches and supported dual-port FC adapters.

Table 6. Adapter to I/O bay correspondence

I/O adapter slot in the server	Port on the adapter	Corresponding I/O module bay in the chassis
Slot 1	Port 1	Module bay 1
	Port 2	Module bay 2
Slot 2	Port 1	Module bay 3
	Port 2	Module bay 4
Slot 3 (full-wide compute nodes only)	Port 1	Module bay 1
	Port 2	Module bay 2
Slot 4 (p460 only) (full-wide compute nodes only)	Port 1	Module bay 3
	Port 2	Module bay 4

The following table lists the storage I/O adapters supported by the FC5022 16Gb SAN Scalable Switches. With 8 Gb FC adapters, internal ports on a switch will operate at 8 Gb speeds.

Table 7. Storage adapters

Description	Part number	Feature code (x-config / e-config)	Support for FC5022 switch
16 Gb Fibre Channel
IBM Flex System FC5022 2-port 16Gb FC Adapter	88Y6370	A1BP / None	Yes
8 Gb Fibre Channel
IBM Flex System FC3172 2-port 8Gb FC Adapter	69Y1938	A1BM / 1764	Yes
IBM Flex System FC3052 2-port 8Gb FC Adapter	95Y2375	A2N5 / None	Yes

The adapters are installed in slots in each compute node. Figure 4 shows the locations of the slots in the x240 Compute Node. The positions of the adapters in the other supported servers are similar.


Figure 4. Location of the I/O adapter slots in the IBM Flex System x240 Compute Node


Connectors and LEDs

Figure 5 shows the front panel of the IBM Flex System FC5022 16Gb SAN Scalable Switches.


Figure 5. Front panel of the IBM Flex System FC5022 16Gb SAN Scalable Switches

The front panel contains the following components:

• LEDs that display the status of the switch module and the fabric:
  • OK LED indicates that the switch module has passed the power-on self-test (POST) with no critical faults and is operational.
  • Identify: This Blue LED can be used to identify the switch physically, by illuminating via the management software.
  • Error LED (switch module error) indicates that the switch module has failed the POST or detected an operational fault.
• One mini-USB RS-232 console port that provides an additional means to configure the switch module. This mini-USB-style connector enables connection of a special serial cable (cable is optional and it is not included with the switch; see Part number information section for details).
• One 1 Gb Ethernet RJ-45 port for switch configuration and management.
• 20 external SFP+ port connectors to attach SFP+ modules for 16 Gb, 8 Gb or 4 Gb connections to external Fibre Channel devices.
• An FC link error LED and an FC Tx/Rx LED for each external port on the switch.

Cabling requirements

The cables required for the switch are as follows:

• 16 Gb FC SW SFP+ transceivers (see Table 2)
  • 850 nm communication using multimode fiber cable (50 µ or 62.5 µ) up to 100 m at 16 Gbps speed, LC duplex connector
• 8 Gb FC SW SFP+ transceivers (see Table 2)
  • 850 nm communication using multimode fiber cable (50 µ or 62.5 µ) up to 150 m at 8 Gbps speed, LC duplex connector
• 1000BASE-T (external 1 GbE management port)
  • UTP Category 6
  • UTP Category 5e (100 meters maximum)
  • UTP Category 5 (100 meters maximum)
  • EIA/TIA-568B 100-ohm STP (100 meters maximum)
• RS-232 serial cable: Console cable DB-9-to-mini-USB or RJ-45-to-mini-USB (nonstandard use of USB connector) that come with optional IBM Flex System Management Serial Access Cable, 90Y9338

Warranty

There is a 1-year, customer-replaceable unit (CRU) limited warranty. When installed in a chassis, these switches assume your system’s base warranty and any IBM ServicePac® upgrade.


Physical specifications

The approximate dimensions and weight of the switch are as follows:

• Height: 30 mm (1.2 in.)
• Width: 401 mm (15.8 in.)
• Depth: 317 mm (12.5 in.)
• Weight: 3.2 kg (7.1 lb)

Shipping dimensions and weight (approximate):

• Height: 114 mm (4.5 in.)
• Width: 508 mm (20.0 in.)
• Depth: 432 mm (17.0 in.)
• Weight: 3.6 kg (8.1 lb)

Operating environment

• Temperature

• Operating: 0°C to 35°C (32°F to 95°F)
• Non-operating: 5°C to 45°C (41°F to 113°F)

• Altitude

• Up to 3,000 m (10,000 ft) above sea level

• Humidity

• Operating: 20% to 80%, noncondensing at 29°C
• Non-operating: 8% to 80%, noncondensing at 38°C

Regulatory compliance

The switch conforms to the following standards:

• US: UL, CB, TUV reports and certificates per EN60950-1 2001 / IEC60950-1 2001
• Canada: CSA C22.2 No. 950 or 60950
• Germany: TUV/VDE IEC 950/EN 60950 (TUV component report & IEC60950 CB Report and Certificate)
• FCC - Title 47 CFR Part 15, Class A
• Canada - ICES-003
• Australia/New Zealand - C-Tick - AS/NZS CISPR 22:2006
• CE Mark - EN55022:2006+A1:2007 and EN55024:1998+A1:2001+A2:2003
• VCCI Class A - Japan VCCI 2009
• Korea’s EMC with KCC marking (previously MIC marking)
• MIC Notice No. 1996-78 (Korea)
• CISPR 22 Class A
• Taiwan BSMI – CNS 13438
• Turkey Communiqué No. 2004/9 and No. 2004/22
• Saudi Arabia – EMC.CVG, 28 October 2002
• China – GB 9524:1998

Popular configurations

The following figure shows use of the IBM Flex System FC5022 24-port 16Gb ESB SAN Scalable Switch to route two ports on the adapter card installed in slot 2 of the compute node. Each compute node has two 16 Gb FC ports, and two FC5022 ESB SAN switches are installed in bays 3 and 4 of the Enterprise Chassis. The connections between the adapter card and the switches are internal to the chassis. No cabling is needed.


Figure 5. Using FC5022 24-port 16Gb ESB SAN Scalable Switch with FC5022 16 Gb FC adapters installed in compute nodes

Table 8. Components used when connecting the 16 Gb FC adapters to the 16 Gb FC switches

Part number/machine type	Description	Quantity
8737-x2x	IBM Flex System x240 Compute Node with Embedded 10Gb Virtual Fabric Adapter	1 to 14
88Y6370	IBM Flex System FC5022 2-port 16Gb FC Adapter	1 per server
8721-A1x	IBM Flex System Enterprise Chassis	1
49Y4294	IBM Flex System Fabric EN4093 10Gb Scalable Switch	2
90Y9356	IBM Flex System FC5022 24-port 16Gb ESB SAN Scalable Switch	2

Refer to the IBM System Storage® Interoperation Center (SSIC) for a list of supported Fibre Channel configurations:
http://ibm.com/systems/support/storage/ssic


Related publications

For more information see the IBM Flex System FC5022 16Gb SAN Switches product publications, available from the IBM Flex System Information Center:
http://publib.boulder.ibm.com/infocenter/flexsys/information/index.jsp

• User's Guide

The following Fabric Operating System publications are available from:
http://ibm.brocadeassist.com/public/FabricOSv7xRelease

• Fabric OS Administrator’s Guide
• Network Advisor SAN User Manual
• Access Gateway Administrator’s Guide
• Fabric OS Command Reference Manual
• Fabric OS MIB Reference Manual
• Fabric OS System Error Message Reference Manual
• Web Tools Administrator’s Guide
• Secure Fabric OS Administrator’s Guide
• Fabric Watch Administrator’s Guide

Other documents:

• IBM US Announcement Letter for the 16Gb SAN and 24-port ESB SAN switches
  http://ibm.com/common/ssi/cgi-bin/ssialias?infotype=dd&subtype=ca&&htmlfid=897/ENUS112-053
• IBM US Announcement Letter for the 16Gb 24-port SAN switch
  http://ibm.com/common/ssi/cgi-bin/ssialias?infotype=dd&subtype=ca&&htmlfid=897/ENUS112-139
• IBM US Announcement Letter for the Features on Demand upgrades
  http://ibm.com/common/ssi/cgi-bin/ssialias?infotype=dd&subtype=ca&&htmlfid=897/ENUS112-185
• IBM Flex System Enterprise Chassis Product Guide
  http://www.redbooks.ibm.com/abstracts/tips0865.html
• IBM Flex System FC5022 2-port 16Gb FC Adapter
  http://www.redbooks.ibm.com/abstracts/tips0891.html
• IBM Flex System Products and Technology, SG24-7984
  http://www.redbooks.ibm.com/abstracts/sg247523.html
• IBM Flex System Interoperability Guide
  http://www.redbooks.ibm.com/abstracts/redpfsig.html?Open
• IBM Redbooks® Product Guides for IBM Flex System servers and options
  http://www.redbooks.ibm.com/portals/puresystems
• Configuration and Option Guide
  http://www.ibm.com/systems/xbc/cog/