IBM Flex System EN4023 10Gb Scalable Switch

Product Guide


Lenovo is open and ready for business with System x
IBM x86 products are now products of Lenovo in the U.S. and other countries. IBM will continue to host x86-related content on ibm.com until migrated to Lenovo.
Learn more about the acquisition

Abstract

The IBM® Flex System™ EN4023 10Gb Scalable Switch is a high-performance 10 Gigabit Ethernet (GbE) embedded switch that supports the most demanding business applications. It offers advanced storage support with multiple storage connectivity options, including FCoE, Fibre Channel, iSCSI, and NAS storage. Clients deploying the new Brocade Virtual Cluster Switching (VCS) Fabric can now extend the features to the Flex System chassis using the EN4023 switch module. It is specifically designed to improve network utilization, maximize application availability, increase scalability, and dramatically simplify network architecture in virtualized data centers, as described in this IBM Redbooks® Product Guide.

Changes in the December 1 update:
* IBM Flex System CN4052 2-port 10Gb Virtual Fabric Adapter now supported
* IBM Flex System CN4058S 8-port 10Gb Virtual Fabric Adapter now supported

Contents


Table of contents


Introduction

The IBM® Flex System EN4023 10Gb Scalable Switch is a high-performance 10 Gigabit Ethernet (GbE) embedded switch that supports the most demanding business applications. It offers advanced storage support with multiple storage connectivity options, including FCoE, Fibre Channel, iSCSI, and NAS storage. Clients deploying the new Brocade Virtual Cluster Switching (VCS) Fabric can now extend the features to the Flex System chassis using the EN4023 switch module. It is specifically designed to improve network utilization, maximize application availability, increase scalability, and dramatically simplify network architecture in virtualized data centers, as described in this IBM Redbooks® Product Guide.

The following figure shows the EN4023 10Gb Scalable Switch.

IBM Flex System EN4023 10Gb Scalable Switch

Figure 1. IBM Flex System EN4023 10Gb Scalable Switch

Did you know?

The EN4023 switches feature Brocade VCS Fabric technology that enables organizations to build high-performance, cloud-optimized data centers while preserving existing network designs and cabling, and gaining active-active server connections. For scale-out fabric architectures, Brocade VCS Fabric technology allows organizations to flatten network designs, provide virtual machine (VM) mobility without network reconfiguration, and manage the entire fabric more efficiently.

With Dynamic Ports on Demand (DPOD), ports are licensed as they come online; therefore, the EN4023 allows you to buy only the ports that you need, when you need them. The base module includes 24 port licenses for 10 GbE connectivity that can be applied to the internal and external ports. You then have the flexibility of turning on more 10 GbE ports and 40 GbE uplinks when you need them by using IBM Features on Demand (FoD) licensing capabilities that provide "pay as you grow" scalability.

Part number information

The EN4023 module is initially licensed for 24 ports (internal or external 10 GbE connectivity). Further ports can be activated, including 16 additional 10 GbE ports and two 40 Gb external uplink ports with the FoD Upgrade 1 license option, and 16 more 10 GbE ports with the FoD Upgrade 2 license option. Upgrade 1 and Upgrade 2 can be applied independently of each other. The following table shows the part numbers for ordering the switches and the upgrades.

Table 1. Part numbers and feature codes for ordering

DescriptionPart numberFeature code
Switch module
IBM Flex System EN4023 10Gb Scalable Switch94Y5212ESWD
Features on Demand upgrades
IBM Flex System EN4023 10Gb Scalable Switch (FoD 1)94Y5158ESWE
IBM Flex System EN4023 10Gb Scalable Switch (FoD 2)94Y5159ESWF
IBM Flex System EN4023 10Gb Scalable Switch (FoD 3)47C9993ESWG

The part numbers for the switches include the following items:
  • One IBM Flex System EN4023 10Gb Scalable Switch
  • Important Notices Flyer
  • Warranty Flyer
  • Documentation CD-ROM

Note: Small form factor (SFP) and small form-factor pluggable plus (SFP+) transceivers are not included with the switch. They must be ordered separately (see Table 2).

The part numbers for the Features on Demand upgrades include the following items:
  • Features on Demand Activation Flyer
  • Upgrade authorization letter

The switch does not include a serial management cable; however, IBM Flex System Management Serial Access Cable, 90Y9338, is supported. It contains two cables, a mini-USB-to-RJ45 serial cable and a mini-USB-to-DB9 serial cable, either of which can be used to connect to the switch locally for configuration tasks.

The base switch and upgrades are specified:
  • 94Y5212 is the part number for the physical device, and it comes with 24x 10 GbE ports enabled. All external 10 Gb ports are connections that are based on SFP+.
  • 94Y5158 (FoD Upgrade 1) can be applied on the base switch or on top of the FoD Upgrade 2 to enable 16 additional 10 GbE ports (internal and external). The upgrade also enables two 40 Gb uplinks with quad small form factor pluggable plus (QSFP+) connectors.
  • 94Y5159 (FoD Upgrade 2) can be applied on the base switch or on top of the FoD Upgrade 1 to enable 16 additional 10 GbE ports (internal and external).
  • 47C9993 (FoD Upgrade 3) can be applied on the base switch or on top of the FoD Upgrade 1 & 2 to enable FCoE on all ports and Flex Ports on the external ports.

The following table lists supported port combinations on the switch and required upgrades.

Table 2. Supported port combinations
Supported port combinations
Quantity required
Base switch, 94Y5212Upgrade 1, 94Y5158Upgrade 2, 94Y5159
  • 24x 10 GbE ports (internal and external)
1
0
0
  • 40x 10 GbE ports (internal and external)
1
0
1
  • 40x 10 GbE ports (internal and external)
  • 2x external 40 GbE QSFP+ ports
1
1
0
  • 56x 10 GbE ports (internal and external)
  • 2x external 40 GbE QSFP+ ports
1
1
1

Supported cables and transceivers

The following table lists the supported transceivers and cables.

Table 3. Supported transceivers and cables

DescriptionPart numberFeature code
(x-config / e-config)
Serial console cables
IBM Flex System Management Serial Access Cable Kit90Y9338A2RR / None
SFP / SFP+ transceivers - 1 GbE and 10 GbE
IBM SFP 1000Base-T (RJ-45) Transceiver00FE333A5DL / None
IBM SFP SX Transceiver81Y16223269 / None
IBM SFP+ SR Transceiver46C34475053 / EB28
IBM SFP+ LR Transceiver00D6180A3NZ / ECB9
Brocade VDX SFP+ LR Transceiver95Y0540A3AB / EB37
Brocade 10Gb SFP+ SR Optical Transceiver49Y42160069 / EB3C
SFP+ direct-attach cables - 10 GbE
1m 10GE Twinax Act Copper SFP+81Y8295A18M / EN01
3m 10GE Twinax Act Copper SFP+81Y8296A18N / EN02
5m 10GE Twinax Act Copper SFP+81Y8297A18P / EN03
1m IBM Active DAC SFP+ Cable95Y0323A25A / None
3m IBM Active DAC SFP+ Cable95Y0326A25B / None
5m IBM Active DAC SFP+ Cable95Y0329A25C / None
1m IBM Passive DAC SFP+90Y9427A1PH / ECB4
3m IBM Passive DAC SFP+ 90Y9430A1PJ / ECB5
Fibre Channel Optics
16Gb FC SWL SFP98Y21762601
8Gb FC SWL SFP98Y21822802
8Gb FC LWL SFP45W12162821
8Gb FC ELWL SFP45W22832881
QSFP+ transceiver and cables - 40 GbE
IBM QSFP+ 40GBASE-SR Transceiver
(Requires either cable 90Y3519 or cable 90Y3521)
49Y7884A1DR / EB27
10m IBM MTP Fiber Optical Cable (requires transceiver 49Y7884)90Y3519A1MM / EB2J
30m IBM MTP Fiber Optical Cable (requires transceiver 49Y7884)90Y3521A1MN / EC2K
QSFP+ breakout cables - 40 GbE to 4x10 GbE
1m 40Gb QSFP+ to 4 x 10Gb SFP+ Cable49Y7886A1DL / EB24
3m 40Gb QSFP+ to 4 x 10Gb SFP+ Cable49Y7887A1DM / EB25
5m 40Gb QSFP+ to 4 x 10Gb SFP+ Cable49Y7888A1DN / EB26
QSFP+ direct-attach cables - 40 GbE
1m QSFP+ to QSFP+ DAC49Y7890A1DP / EB2B
3m QSFP+ to QSFP+ DAC49Y7891A1DQ / EB2H

Features and Benefits

Key features:

  • 24 x 10GbE Ports dynamically assignable
  • Full Layer 2 Switching
  • Full Layer 3 Routing
  • Brocade VCS Ethernet Fabric License
  • OpenFlow & VxLAN Gateway support
  • Active & Passive DAC Cable support

The EN4023 10Gb Scalable Switch is designed for easy integration into a Brocade Virtual Cluster Switching (VCS) environment. Brocade VCS Fabric technology simplifies network design and operations for a more automated and efficient network. The switch offers the following benefits:
  • Advanced Storage Support
    • The EN4023 10Gb Scalable Switch offers advanced storage support with multiple storage connectivity options, including FCoE, Fibre Channel, iSCSI, and NAS storage.
    • Data Center Bridging (DCB), enables the reliable exchange of storage traffic over the LAN network, eliminating packet loss when network congestion occurs and allocating bandwidth as needed to keep the network running efficiently.
    • Network-Attached Storage (NAS) Auto QoS intelligence prioritizes delay-sensitive IP storage traffic within the fabric and help ensure consistent performance while decreasing latency.
    • EN4023 10Gb Scalable Switch features 14 Flex Ports, which can take either a 10/1 GbE or 16/8/4/2Gb Fibre Channel personality.
    • In Fibre Channel mode, these Flex Ports can be used either to directly connect Fibre Channel storage to VCS fabrics, or to bridge FCoE traffic to Fibre Channel SANs, thus protecting existing SAN investments.
  • High performance and low latency:
    • With 10 Gb connections, the Flex System EN4023 switch can help your organization reduce network congestion, improve application performance, and meet the capacity required by 10 Gb servers. The 40 Gb uplinks can easily aggregate high-bandwidth traffic and reduce bottlenecks, helping your network operate at its peak performance.
    • Provides ultra-low any-to-any port latency of 850 nanoseconds.
  • Simplicity and automation:
    • Brocade VCS Logical Chassis: Provides the ability to manage an entire Brocade VCS Fabric (up to 48 switches) as a single switch, upgrade software across the fabric with one command, and centralize monitoring and troubleshooting to enhance the overall availability and reliability of the network.
    • Elastic, self-forming, and self-healing fabrics: Allows new switches added to the fabric to inherit the fabric configuration, providing a highly resilient, operationally scalable foundation.
    • Auto Fabric Provisioning: Automatically downloads the latest software image from the server, reducing configuration time.
  • Efficiency and resiliency:
    • Optimized east-west traffic: Embedded switching moves traffic through any active path and avoids the multiple hops found in tiered tree topologies.
    • Efficient multipathing across Layers 1 - 3: Delivers efficiently load-balanced multipathing at Layers 1 - 3, with multiple Layer 3 gateways for more effective link utilization.
  • Optimized for virtualization:
    • Brocade VM-Aware Network Automation: Eliminates the manual configuration of port profiles when a VM is added to the fabric or moved.
    • Automatic Migration of Port Profiles (AMPP): Applies VM and networking policies to a VM as it moves within the Brocade VCS Fabric.
    • Proactive Monitoring.
    • Supports Software-Defined Networking (SDN) technologies within data, control and management planes

Specifications

The IBM Flex System EN4023 10Gb Scalable Switch has the following specifications:

  • Internal ports:
    • Forty-two internal full-duplex 10 Gigabit ports.
    • Two internal full-duplex 1 GbE ports that are connected to the chassis management module.
  • External ports:
    • Fourteen ports for 10 Gb Ethernet SFP+ transceivers (support for 10GBASE-SR or 10GBASE-LR). SFP+ modules are not included and must be purchased separately.
    • Two ports for 40 Gb Ethernet QSFP+ transceivers (Ports are disabled, by default. An optional FoD license is required to activate them.) QSFP+ modules are not included and must be purchased separately.
    • One RS-232 serial port (mini-USB connector) that provides an additional means to configure the switch module.
  • Operating system - Brocade Network OS 5.0.1a
  • Layer 2 switching features:
    • Address Resolution Protocol (ARP) RFC 826
    • High availability/In-Service Software Upgrade—hardware-enabled
    • IGMP v1/v2 Snooping
    • MAC Learning and Aging
    • Link Aggregation Control Protocol (LACP) IEEE 802.3ad/802.1AX
    • Virtual Local Area Networks (VLANs)
    • VLAN Encapsulation 802.1Q
    • Private VLANs
    • Edge loop detection (ELD)
    • Per-VLAN Spanning Tree (PVST+/PVRST+)
    • Rapid Spanning Tree Protocol (RSTP) 802.1w
    • Multiple Spanning Tree Protocol (MSTP) 802.1s
    • STP PortFast, BPDU Guard, BPDU Filter
    • STP Root Guard
    • Layer 2 Access Control Lists (ACLs)
    • Pause Frames 802.3x
    • Uni-Directional Link Detection (UDLD)
  • Layer 3 switching features:
    • Border Gateway Protocol (BGP
    • DHCP Helper
    • Layer 3 ACLs
    • Multicast: PIM-SIM
    • OSPF
    • Static Routes
    • VRF Lite
    • VRF-aware OSPF, VRRP, static routes
    • VRRP-E
    • IPv4/IPv6 dual stack
    • IPv6 ACL packet filtering
    • IPv6 routing
  • Brocade VCS Fabric technology features:
    • Automatic Fabric Formation
    • DHCP Option 66/67 (Auto Fabric Provisioning)
    • Distributed Configuration Management
    • Distributed Fabric Services
    • Equal Cost Multi-Path (ECMP)
    • Switch Beaconing
    • Transparent Interconnection of Lots of Links (TRILL)
    • Transparent LAN Services
    • Virtual Link Aggregation Group (vLAG) spanning
    • VRRP-E
  • Fibre Channel/FCoE features(Requires FoD 3 - FCoE license)
    • Multihop Fibre Channel over Ethernet (FCoE); requires Brocade VCS Fabric technology
    • FC-BB5 compliant Fibre Channel Forwarder (FCF)
    • Native FCoE forwarding
    • FCoE to Fibre Channel Bridging
    • FCoE on IBM Flex System EN4023
    • Flex Ports, allowing direct and SAN connectivity of Fibre Channel targets
    • End-to-end FCoE (initiator to target)
    • FCoE Initialization Protocol (FIP) v1 support for FCoE device login and initialization
    • Name Server-based zoning
    • Supports connectivity to FIP Snooping Bridge (FSB) device
    • FCoE traffic over standard LAG
    • Interface Binding
  • Multitenancy and virtualization features:
    • TRILL FGL-based VCS Virtual Fabric feature
    • Brocade VCS Gateway for NSX with VMware NSX Orchestration
    • Automatic Migration of Port Profiles (AMPP)
    • VM-Aware Network Automation
  • IP storage:
    • Auto QoS (automatic prioritization of IP storage traffic)
  • Quality of Service (QoS):
    • ACL-based QoS
    • Eight priority levels for QoS
    • Class of Service (CoS) IEEE 802.1p
    • DSCP Trust
    • DSCP to Traffic Class Mutation
    • DSCP to CoS Mutation
    • DSCP to DSCP Mutation
    • Random Early Discard
    • Per-port QoS configuration
    • ACL-based Rate Limit
    • Dual-rate three color token bucket
    • ACL-based remarking of CoS/DSCP/Precedence
    • ACL-bases sFlow
    • Scheduling Strict Priority (SP), Deficit Weighted Round-Robin (DWRR), Hybrid Scheduling (Hybrid)
    • Queue-based Shaping
  • Management and control:
    • IPv4/IPv6 management
    • Industry standard Command Line Interface
    • Netconf API
    • REST API with YANG data model
    • Brocade VCS Plugin for OpenStack
    • Link Layer Discovery Protocol (LLDP) IEEE 802.1AB
    • Logical chassis management
    • MIB II RFC 1213 MIB
    • Switch Beaconing
    • Switch Port Analyzer (SPAN)
    • SNMPv3 (default) and SNMPv1
    • sFlow RFC 3176
    • Out-of-band management
    • Remote SPAN (RSPAN)
    • RMON-1, RMON-2
    • NTP
    • Management Access Control Lists (ACLs)
    • Role-Based Access Control (RBAC)
    • Range CLI support
    • UDLD
    • OpenStack Neutron ML2 plugin
  • Security
    • Port-based Network Access Control 802.1X
    • RADIUS
    • TACACS+
    • Secure Shell (SSHv2)
    • BPDU Drop
    • Lightweight Directory Access Protocol (LDAP)
    • Secure Copy Protocol (SCP)
    • Secure FTP (sFTP)
  • Switch health monitoring
    • Brocade Fabric Watch monitoring and notification
    • Command-line interface (CLI) through Telnet or Secure Shell V2 (SSHv2)
    • Terminal emulation program connection to the serial port interface
    • Brocade Network Advisor
    • Simple Network Management Protocol (SNMP) agent

Supported standards

The switches support the following standards:

  • IEEE 802.1AB Link Layer Discovery Protocol (LLDP)
  • IEEE 802.1p Class of Service (CoS) prioritization and Tagging
  • IEEE 802.1Q Tagged VLAN (frame tagging on all ports when VLANs are enabled)
  • IEEE 802.1Qbb Priority-Based Flow Control (PFC)
  • IEEE 802.1Qaz Enhanced Transmission Selection (ETS)
  • IEEE 802.1w Rapid Reconfiguration of Spanning Tree Protocol
  • IEEE 802.1D Spanning Tree Protocol
  • IEEE 802.1s Multiple Spanning Tree
  • IEEE 802.3 Ethernet
  • IEEE 802.3ab 1000BASE-T
  • IEEE 802.3ad Link Aggregation with LACP
  • IEEE 802.3ae 10 Gb Ethernet
  • IEEE 802.3ap 10GBASE-KR backplane 10 Gb Ethernet
  • IEEE 802.3ba 40GBASE-SR4 short range fiber optics 40 Gb Ethernet
  • IEEE 802.3ba 40GBASE-CR4 copper 40 Gb Ethernet
  • IEEE 802.3u 100BASE-TX Fast Ethernet
  • IEEE 802.3x Flow Control (Pause Frames)
  • IEEE 802.3z 1000BASE-X
  • SFF-8431 10GSFP+Cu SFP+ Direct Attach Cable

Supported chassis and adapters

The I/O modules are installed in switch bays in the rear of the IBM Flex System Enterprise Chassis, as shown in the following figure. I/O modules are normally installed in pairs because ports on the I/O adapters that are installed in the compute nodes are routed to two switch bays for redundancy and performance.

Location of the switch bays in the IBM Flex System Enterprise Chassis
Figure 2. Location of the switch bays in the IBM Flex System Enterprise Chassis

The connections between the adapters that are installed in the compute nodes to the EN4023 installed in the I/O bays in the chassis are shown in the following figure. The figure shows both half-wide servers, such as the x240 with two adapters, and full-wide servers, such as the x440 with four adapters.

Logical layout of the interconnects between I/O adapters and the EN4023 modules
Figure 3. Logical layout of the interconnects between I/O adapters and the EN4023 modules

The EN4023 10Gb Scalable Switches can be installed in bays 1, 2, 3, and 4 of the Enterprise Chassis. A supported adapter must be installed in the corresponding slot of the compute node (slot A1 when interconnect modules are installed in bays 1 and 2 or slot A2 when the modules are in bays 3 and 4). With four-port or eight-port adapters, an optional FoD Upgrade might be required for the switch to allow communications on all ports (However, with eight-port adapters, only six adapter ports can be used, and the two remaining ports are not used.). (See Table 2 for details.)

In compute nodes that have an integrated dual-port 10 GbE network interface controller (NIC), NIC ports are routed to bays 1 and 2 with a specialized periscope connector, and the adapter in slot A1 is not required. However, when needed, the periscope connector can be replaced with the adapter. In such a case, the integrated NIC is disabled.

The following table shows the connections between adapters that are installed in the compute nodes to the EN4023 modules installed in the I/O bays in the chassis.

Table 4. Adapter to I/O bay correspondence

I/O adapter slot
in the server
Port on the adapterCorresponding I/O bay in the chassis
Bay 1Bay 2Bay 3Bay 4
x222 integrated LOMPort 1Yes
Port 2Yes
Port 1Yes
Port 2Yes
Slot 1 (or integrated dual-port LOM - ports 1 and 2)Port 1Yes
Port 2Yes
Port 3*Yes
Port 4*Yes
Port 5*Yes
Port 6*Yes
Slot 2Port 1Yes
Port 2Yes
Port 3*Yes
Port 4*Yes
Port 5*Yes
Port 6*Yes
Slot 3
(full-wide compute nodes only)
Port 1Yes
Port 2Yes
Port 3*Yes
Port 4*Yes
Port 5*Yes
Port 6*Yes
Slot 4
(full-wide compute nodes only)
Port 1Yes
Port 2Yes
Port 3*Yes
Port 4*Yes
Port 5*Yes
Port 6*Yes
* Depending on the number of compute nodes and uplinks, Ports 3, 4, 5, and 6 might require an optional FoD port upgrade of the EN4023 (see Table 2).

The following table lists the I/O adapters that are supported by the EN4023 10Gb Scalable Switches.

Table 5. Supported network adapters

DescriptionPart
number
Feature code
(x-config / e-config)
Supported by
EN4023
40 Gb Ethernet
IBM Flex System EN6132 2-port 40Gb Ethernet Adapter90Y3482A3HK / A3HKNo
10 Gb Ethernet
Embedded 10Gb Virtual Fabric Adapter (x222; dual 2-port)NoneNone / NoneYes
Embedded 10Gb Virtual Fabric Adapter (x240, x440; 2-port)*NoneNone / NoneYes
IBM Flex System CN4022 2-port 10Gb Converged Adapter88Y5920A4K3 / A4K3Yes
IBM Flex System CN4052 2-port 10Gb Virtual Fabric Adapter00JY800A5RP / NoneYes
IBM Flex System CN4054 10Gb Virtual Fabric Adapter (4-port)90Y3554A1R1 / NoneYes
IBM Flex System CN4054R 10Gb Virtual Fabric Adapter (4-port)00Y3306A4K2 / A4K2Yes
IBM Flex System CN4058 8-port 10Gb Converged AdapterNoneNone / EC24Yes#
IBM Flex System CN4058S 8-port 10Gb Virtual Fabric Adapter94Y5160A4R6 / NoneYes#
IBM Flex System EN4054 4-port 10Gb Ethernet AdapterNoneNone / 1762Yes
IBM Flex System EN4132 2-port 10Gb Ethernet Adapter90Y3466A1QY / EC2DYes
IBM Flex System EN4132 2-port 10Gb RoCE AdapterNoneNone / EC26Yes
1 Gb Ethernet
Embedded 1 Gb Ethernet controller (x220; 2-port)NoneNone / NoneYes
IBM Flex System EN2024 4-port 1Gb Ethernet Adapter 49Y7900A10Y / 1763Yes
* The Embedded 10Gb Virtual Fabric Adapter is included in some models of the x240 and x440.
# With eight-port adapters, six adapter ports can be used with the EN4023; the remaining two ports are not used.

The adapters are installed in slots in each compute node. The following figure shows the locations of the slots in the x240 Compute Node. The positions of the adapters in the other supported servers are similar.

 Location of the I/O adapter slots in the IBM Flex System x240 Compute Node
Figure 4. Location of the I/O adapter slots in the IBM Flex System x240 Compute Node

Connectors and LEDs

The following figure shows the front panel of the EN4023 10Gb Scalable Switch.

Front panel of the IBM Flex System EN4023 10Gb Scalable Switch
Figure 5. Front panel of the IBM Flex System EN4023 10Gb Scalable Switch

The front panel contains the following components:

  • Light-emitting diodes (LEDs) that display the status of the interconnect module and the network:
    • The OK LED indicates that the interconnect module passed the power-on self-test (POST) with no critical faults and is operational.
    • Identify: This blue LED can be used to identify the module physically by illuminating it through the management software.
    • The error LED (switch module error) indicates that the module failed the POST or detected an operational fault.
  • One mini-USB RS-232 console port that provides an additional means to configure the interconnect module. This mini-USB-style connector enables the connection of a special serial cable. (The cable is optional, and it is not included with the interconnect module. For more information, see the "Part number information" section.)
  • One 1 Gb Ethernet RJ-45 port for switch configuration and management.
  • Fourteen external SFP+ ports for 10 Gb connections to external Ethernet devices.
  • Two external QSFP+ port connectors to attach QSFP+ modules for a single 40 Gb uplink per port.
  • An Ethernet link OK LED and an Ethernet Tx/Rx LED for each external port on the interconnect module.

Network cabling requirements

The network cables that can be used with the EN4023 are shown in the following table.

Table 6. EN4023 network cabling requirements

TransceiverStandardCableConnector
40 Gb Ethernet
IBM QSFP+ 40GBASE-SR Transceiver (49Y7884)40GBASE-SR410 m or 30 m IBM MTP fiber optics cables (see Table 3)MTP
IBM QSFP+ 40G Direct attach cable40GBASE-CR41m or 3m QSFP+ to QSFP+ DACs (see Table 3)QSFP+
10 Gb Ethernet
SFP+ SR Transceiver (46C3447, 49Y4216)10GBASE-SR850 nm multimode fiber cable (50 µ or 62.5 µ) up to 300 mLC
SFP+ LR Transceiver (90Y9412, 95Y0540)10GBASE-LR1310 nm single-mode fiber cable up to 10 kmLC
IBM SFP+ 10G Direct attach cable10GSFP+CuUp to 5m (Active) or 3m (Passive) SFP+ copper DACs (see Table 3)SFP+
Management ports
External 1 GbE management port1000BASE-TUTP Category 5, 5E, and 6 up to 100 metersRJ-45
External RS-232 management portRS-232DB-9-to-mini-USB or RJ-45-to-mini-USB console cable (comes with optional Management Serial Access Cable, 90Y9338)Mini-USB

Warranty

The IBM Flex System EN4023 carries a 1-year, customer-replaceable unit (CRU) limited warranty. When installed in a chassis, these I/O modules assume your system’s base warranty and any IBM ServicePac® upgrade.

Physical specifications

Dimensions and weight of the EN4023 (approximate):

  • Height: 30 mm (1.2 in.)
  • Width: 402 mm (15.8 in.)
  • Depth: 297 mm (11.7 in.)
  • Weight: 3.6 kg (7.9 lb)

Shipping dimensions and weight (approximate):
  • Height: 114 mm (4.5 in.)
  • Width: 508 mm (20.0 in.)
  • Depth: 432 mm (17.0 in.)
  • Weight: 4.1 kg (9.1 lb)

Agency approvals

The EN4023 conforms to the following regulations:

  • UL 60950
  • CAN/CSA-C22.2 No 60950
  • IEC/EN 60950
  • FCC 47 CFR Part 15, Class A
  • AS/NZS CISPR 22, Class A
  • CISPR 22, Class A
  • CE Mark (EN55022 Class A, EN55024, EN61000-3-2, EN61000-3-3)
  • ICES003, Class A
  • VCCI, Class A
  • KN22, Class A

Typical configurations

Inter-Switch Links

The connectivity topology, which can be used with the EN4023 switches and the Brocade VDX upstream network devices, is shown in the following figure.

EN4023 connectivity topology - Inter-Switch Links
Figure 6. EN4023 connectivity topology - Inter-Switch Links

In this topology, EN4023 switches are connected to the Brocade VDX switches via Inter-Switch Links (ISLs) forming a scale-out self-aggregated data center edge fabric that is based on Brocade VCS technology. Such a fabric, where STP is eliminated, is highly resilient and efficient, and it allows network administrators to manage the entire network as a single logical switch.

With both topologies, each port of the compute node's NIC can be divided into up to four virtual NICs (vNICs), adding more granularity in configuring virtualized network environments.

Direct connection without ToR switches

In this topology, shown in the following figure, the EN4023 switches are connected directly to the upstream network via vLAGs.


Figure 7. EN4023 connectivity topology – Direct connection without ToR switches

Such a topology allows network administrators to connect to the upstream network without the need for a Top of Rack (ToR) switch, allowing for a lower-cost solution and freeing up valuable rack space.

Virtual Link Aggregation

The connectivity topology, which can be used with the EN4023 switches and the Brocade MLX upstream network devices, is shown in the following figure.

EN4023 connectivity topology - Virtual Link Aggregation
Figure 8. EN4023 connectivity topology - Virtual Link Aggregation

In this access-layer topology, aggregation is split between two physical switches, and each EN4023 is connected to both ToR switches through static or Link Aggregation Control Protocol (LACP)-aggregated links. Compute Node's NICs are configured in an active/active NIC teaming. This topology can be used to insert Brocade VCS Fabric technology into existing infrastructure, because it fully interoperates with existing LAN protocols, services, and architectures.

The EN4023 can connect to the core and aggregations switches from IBM, Brocade, Cisco, Juniper, Arista, and any vendor that utilizes industry standard connectivity, is shown in the figure above.


Multiple connection types supported

The EN4023 supports multiple connection types to the upstream SAN or Ethernet network storage controllers. Some of these are shown in the following figure.


Figure 9. Sample of the connection types that are supported by the EN4023

Specifics about these supported configurations include:

  • Connecting to Brocade SAN by utilizing Access Gateway (NPIV) mode on the EN4023
  • Utilizing Fibre Channel Routing (FCR) to connect to the upstream Brocade SAN via Ex_Port
  • Direct-attach FCoE storage (IBM V7000 FCoE storage controller)
  • Direct-attach Ethernet (IP) storage - NAS or iSCSI

EN4023 in the VCS network

The following figure shows a sample scenario where the EN4023 is used as a part of the data center edge network built with VCS technology.


Figure 10. EN4023 in the VCS network with the Brocade VDX 6740 ToR

Table 7. EN4023 in the VCS network with the Brocade VDX 6740 ToR (Figure 10)
Diagram
reference
DescriptionPart
number
Quantity
1IBM Flex System VCS solution
IBM Flex System x240 Compute Node with Embedded Virtual Fabric AdapterVariesVaries
IBM Flex System EN4023 10Gb Scalable Switch94Y5212Two per chassis
2Brocade VDX 6740 Switch

Note: You also need SFP+ modules and optical cables (not shown in Table 7; see Table 3 for details) for the external 10 Gb Ethernet connectivity.

Related publications

For more information, see the following EN4023 10Gb Scalable Switch product publication, which is available from the IBM Flex System Information Center at
http://publib.boulder.ibm.com/infocenter/flexsys/information/index.jsp:

  • Installation and User Guide

The following documents are other useful references:



Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment. publib-b.boulder.ibm.com

Profile

Publish Date
08 October 2013

Last Update
01 December 2014


Rating: Not yet rated


Author(s)

IBM Form Number
TIPS1070