Lenovo Flex System EN4023 10Gb Scalable Switch

Product Guide

This document is being moved!
In September 2015, this document will be moved to lenovopress.com and will no longer be available on the IBM Redbooks site. Please bookmark the URL http://lenovopress.com/TIPS1070 to access this specific document both today and after the migration in September.

Learn more

Abstract

The Lenovo® Flex System™ EN4023 10Gb Scalable Switch is a high-performance 10 Gigabit Ethernet (GbE) embedded switch that supports the most demanding business applications. It offers advanced storage support with multiple storage connectivity options, including FCoE, Fibre Channel, iSCSI, and NAS storage. Clients deploying the new Brocade Virtual Cluster Switching (VCS) Fabric can now extend the features to the Flex System chassis using the EN4023 switch module. It is specifically designed to improve network utilization, maximize application availability, increase scalability, and dramatically simplify network architecture in virtualized data centers.

Changes in the August 20 update:
* Updated product descriptions
* Updated features and benefits
* Updated supported transceivers and cables
* Updated specifications

Contents


Table of contents


Introduction

The Lenovo® Flex System EN4023 10Gb Scalable Switch is a high-performance 10 Gigabit Ethernet (GbE) embedded switch that supports the most demanding business applications. It is designed for both LAN and SAN environments, offering Layer 2 and Layer 3 features for the LAN and support for connectivity to almost any kind of SAN including FCoE, Fibre Channel, iSCSI, and NAS storage.

The EN4023 is one of the most flexible switches in the blade market and also leverages Brocade’s Virtual Cluster Switching (VCS) Fabric which is designed to improve network utilization, maximize application availability, increase scalability, and dramatically simplify network deployment and management.

The following figure shows the Lenovo Flex System EN4023 10Gb Scalable Switch.

Lenovo Flex System EN4023 10Gb Scalable Switch

Figure 1. Lenovo Flex System EN4023 10Gb Scalable Switch

Did you know?

The EN4023 switch features Brocade VCS Fabric technology that enables organizations to build high-performance, cloud-optimized data centers while preserving existing network designs and cabling, and gaining active-active server connections. Brocade VCS Fabric technology provides a scale-out architecture which allows organizations to flatten network designs, provide virtual machine (VM) mobility without network reconfiguration, and manage the entire fabric more efficiently.

The EN4023 is the industry's first offering that supports 16 Gbps Fibre Channel connectivity on an embedded FCoE switch, which makes the EN4023 the best choice for flash storage, databases, or other environments that need more storage bandwidth.

The EN4023 is one of the lowest latency switches for Ethernet and FCoE traffic, and it utilizies a single ASIC design to achieve sub-microsecond port to port latencies, while also providing one of the largest aggregated throughput of 1.28 Tbps. Additionally, the EN4023 can support more FC uplinks than any other vendor’s converged switch by using an optional Flex Port technology license upgrade.

Part number information

The base EN4023 module is initially licensed for 24 ports (internal or external 10 GbE connectivity). Further ports can be activated, including 16 additional 10 GbE ports and two 40 Gb external uplink ports with the FoD Upgrade 1 license option, and 16 more 10 GbE ports with the FoD Upgrade 2 license option.

Clients can also leverage the 14 SFP+ ports as either FCoE or 8/16 Gb Fibre Channel, in addition to 1/10 Gb Ethernet with the FoD Upgrade 3 license option. Upgrade 1, Upgrade 2, and Upgrade 3 can be applied independently of each other.

The following table shows the part numbers for ordering the switches and the upgrades.

Table 1. Part numbers and feature codes for ordering

DescriptionPart numberFeature code
Switch module
Lenovo Flex System EN4023 10Gb Scalable Switch94Y5212ESWD
Features on Demand upgrades
Flex System EN4023 10Gb Scalable Switch (FoD 1)94Y5158ESWE
Flex System EN4023 10Gb Scalable Switch (FoD 2)94Y5159ESWF
Flex System EN4023 10Gb Scalable Switch (FoD 3)47C9993ESWG

The part numbers for the switches include the following items:
  • One Lenovo Flex System EN4023 10Gb Scalable Switch
    • Includes license for 24x 10 GbE dynamic ports
    • Includes Brocade VCS Ethernet Fabric license
  • Important Notices Flyer
  • Warranty Flyer
  • Documentation CD-ROM

Note: Small form factor (SFP) and small form-factor pluggable plus (SFP+) transceivers are not included with the switch. They must be ordered with the switch (for ordering information, see Table 2).

The part numbers for the Features on Demand upgrades include the following items:
  • Features on Demand Activation Flyer
  • Upgrade authorization letter

The switch does not include a serial management cable; however, Flex System Management Serial Access Cable, 90Y9338, is supported. It contains two cables, a mini-USB-to-RJ45 serial cable and a mini-USB-to-DB9 serial cable, either of which can be used to connect to the switch locally for configuration tasks.

The base switch and upgrades are as follows:
  • 94Y5212 is the part number for the physical device, and it comes with 24x 10 GbE ports enabled. All external 10 Gbps ports are connections that are based on SFP+.
  • 94Y5158 (FoD Upgrade 1) can be applied on the base switch or on top of the FoD Upgrade 2 to enable 16 additional 10 GbE ports (internal and external). The upgrade also enables two 40 Gbps uplinks with quad small form factor pluggable plus (QSFP+) connectors.
  • 94Y5159 (FoD Upgrade 2) can be applied on the base switch or on top of the FoD Upgrade 1 to enable 16 additional 10 GbE ports (internal and external).
  • 47C9993 (FoD Upgrade 3) can be applied on the base switch or on top of the FoD Upgrade 1, or Upgrade 2, or both, to enable FCoE on all internal ports and Flex Ports (support for 1/10 GbE, 10 Gb FCoE or 8/16 Gb FC) on the external ports.

The following table lists supported port combinations on the switch and required upgrades.

Table 2. Supported port combinations
Supported port combinations
Quantity required
Base switch, 94Y5212Upgrade 1, 94Y5158Upgrade 2, 94Y5159
  • 24x 10 GbE ports (internal and external)
1
0
0
  • 40x 10 GbE ports (internal and external)
1
0
1
  • 40x 10 GbE ports (internal and external)
  • 2x external 40 GbE QSFP+ ports
1
1
0
  • 56x 10 GbE ports (internal and external)
  • 2x external 40 GbE QSFP+ ports
1
1
1

Supported cables and transceivers

With the flexibility of the EN4023 switch, clients can use the technologies that they require for the following environments:

  • For 1 GbE links, clients can use RJ-45 SFP transceivers with UTP cables up to 100 meters. Clients that need longer distances can use a 1000BASE-SX transceiver, which can drive distances up to 220 meters by using 62.5 µ multi-mode fiber and up to 550 meters with 50 µ multi-mode fiber, or the 1000BASE-LX transceivers that support distances up to 10 kilometers by using single-mode fiber (1310 nm).
  • For 10 GbE (on external SFP+ ports), you can use SFP+ direct-attached copper (DAC) cables for in-rack cabling and distances up to 5 meters. These DAC cables have SFP+ connectors on each end, and they do not need separate transceivers. For longer distances the 10GBASE-SR transceiver can support distances up to 300 meters over OM3 multimode fiber or up to 400 meters over OM4 multimode fiber. The 10GBASE-LR transceivers can support distances up to 10 kilometers on single mode fiber.
  • For 40 GbE to 40 GbE connectivity, clients can use the affordable QSFP+ to QSFP+ DAC cables for distances up to 3 meters. For distances up to 100 m, the 40GBASE-SR4 QSFP+ transceivers can be used with OM3 multimode fiber or up to 150 m when using OM4 multimode fiber.
  • For 8 Gb FC links (supported on Flex Ports only), you can use 8 Gb FC SFP+ SW optical transceivers plus LC fiber optics cables for distances up to 150 m with OM3 multi-mode fiber. These transceivers can operate at 2/4/8 Gbps speeds.
  • For 16 Gb FC links (supported on Flex Ports only), you can use 16 Gb FC SFP+ SW optical transceivers plus LC fiber optics cables for distances up to 100 m with OM3 multi-mode fiber or up to 125 m with OM4 multi-mode fiber. These transceivers can operate at 4/8/16 Gbps speeds.

The following table lists the supported transceivers and cables.

Table 3. Supported transceivers and direct-attach cables
DescriptionPart numberFeature codeMaximum quantity supported
Serial console cables
Flex System Management Serial Access Cable Kit90Y9338A2RR1
SFP transceivers - 1 GbE
Lenovo 1000BASE-T SFP Transceiver (does not support 10/100 Mbps)00FE333A5DL14
Lenovo 1000BASE-SX SFP Transceiver81Y1622326914
Lenovo 1000BASE-LX SFP Transceiver90Y9424A1PN14
SFP+ transceivers - 10 GbE
Lenovo 10GBASE-SR SFP+ Transceiver46C3447505314
Lenovo 10GBASE-LR SFP+ Transceiver00D6180A3NZ14
Brocade 10Gb SFP+ SR Optical Transceiver49Y4216006914
Brocade 10Gb SFP+ LR Optical Transceiver95Y0540A3AB14
Optical cables for 1 GbE SX SFP, 10 GbE SR SFP+, and 8/16 Gb FC SW SFP+ transceivers
Lenovo 0.5m LC-LC OM3 MMF Cable00MN498ASR514
Lenovo 1m LC-LC OM3 MMF Cable00MN502ASR614
Lenovo 3m LC-LC OM3 MMF Cable00MN505ASR714
Lenovo 5m LC-LC OM3 MMF Cable00MN508ASR814
Lenovo 10m LC-LC OM3 MMF Cable00MN511ASR914
Lenovo 15m LC-LC OM3 MMF Cable00MN514ASRA14
Lenovo 25m LC-LC OM3 MMF Cable00MN517ASRB14
Lenovo 30m LC-LC OM3 MMF Cable00MN520ASRC14
SFP+ direct-attach cables - 10 GbE
Lenovo 1m Active SFP+ DAC Cable95Y0323A25A14
Lenovo 3m Active SFP+ DAC Cable95Y0326A25B14
Lenovo 5m Active SFP+ DAC Cable95Y0329A25C14
1m 10GE Twinax Act Copper SFP+81Y8295A18M14
3m 10GE Twinax Act Copper SFP+81Y8296A18N14
5m 10GE Twinax Act Copper SFP+81Y8297A18P14
QSFP+ transceiver and cables - 40 GbE
Lenovo 40GBASE-SR4 QSFP+ Transceiver49Y7884A1DR2
Optical cables for 40 GbE QSFP+ SR transceivers
Lenovo 10m QSFP+ MTP-MTP OM3 MMF Cable90Y3519A1MM2
Lenovo 30m QSFP+ MTP-MTP OM3 MMF Cable90Y3521A1MN2
Lenovo 10m QSFP+ MTP-MTP OM3 MMF Cable (replaces 90Y3519)00VX003AT2U2
Lenovo 30m QSFP+ MTP-MTP OM3 MMF Cable (replaces 90Y3521)00VX005AT2V2
QSFP+ direct-attach cables - 40 GbE
Lenovo 1m Passive QSFP+ DAC Cable49Y7890A1DP2
Lenovo 3m Passive QSFP+ DAC Cable49Y7891A1DQ2
SFP+ transceivers - 8 Gb FC (supported in Flex Ports)
Brocade 8Gb FC SFP+ Transceiver Module88Y6416A2B914
SFP+ transceivers - 16 Gb FC (supported in Flex Ports)
Brocade 16Gb FC SFP+ Transceiver Module88Y6393A22R14

Features and benefits

Data centers continue to evolve. This evolution creates a need for an infrastructure that can support growth in Virtual Machines (VMs), distributed applications, and data, as well as the transition to cloud-based computing — without compromising performance. However, traditional data centers typically use inflexible, three-tier network designs that cannot efficiently manage east-west traffic or deliver the bandwidth needed to support virtualization and new service delivery. In addition, with the movement toward cloud computing, the importance of deploying a flexible and responsive network infrastructure only grows.

To support the new direction of IT service delivery, data center operators need networks that are high performance, operationally efficient, automated, and elastic. The ideal network will be easy to manage and scale to meet demand as well as adapt to future requirements.

The EN4023 switch offers the following key features to meet raising networking requirements of data centers:

  • Extreme flexibility for LAN and storage networking

    The EN4023 supports Layer 2, Layer 3 networking in addition to providing advanced storage support with multiple storage connectivity options, including FCoE, Fibre Channel, iSCSI, and NAS storage. Data Center Bridging (DCB) enables the reliable exchange of storage traffic over the LAN network, eliminating packet loss when network congestion occurs and allocating bandwidth as needed to keep the network running efficiently. The switches offer Network Attached Storage (NAS) Auto QoS intelligence to prioritize delay-sensitive IP storage traffic within the fabric and help ensure consistent performance while decreasing latency.

  • Unmatched port flexibility and configuration simplicity

    The EN4023 switch can help reduce cost and complexity with Dynamic Ports on Demand (DPOD), which automatically activates ports as compute nodes, cables, or transceivers are plugged in. No manual settings required to turn ports on or off.

    The base license supports 24 ports and allows all 14 slots in the Flex System chassis to connect to an upstream switch with an oversubscription ratio better than 2:1 when using 2-port Ethernet adapters. Additional licenses can be purchased allowing "pay as you grow" scalability for additional bandwidth while still having an affordable price.

  • Unrivalled scalability and simplified management

    The EN4023 switch supports for up to 48 switches managed through one IP address. This allows up to 24 chassis and 336 compute nodes to be added to this single managed unit, making it easy for clients to deploy PODs or clusters with a single logical chassis requiring no networking beyond what is provided in a Flex System chassis.

  • Industry leading virtualization and cloud ready innovations with Brocade VM-aware Network Automation.

    The EN4023 supports Automatic Migration of Port Profiles where the Brocade VCS fabric directly communicates with VMware vCenter and its status, being the first blade embedded switch with VXLAN support. VNIC technology that is switch independent can also be run through the EN4023.

  • Unequalled SDN features and capabilities

    The EN4023 can help clients simplify administration through OpenFlow 1.3, OpenStack, Puppet, and Python while still including an industry standard CLI for easy transition of administrator skill sets.


Brocade VCS fabrics running on the Brocade VDX family of switches (like the EN4023) allow organizations to create data center networks that just work. Together, these technologies (Layer 2, Layer 3 and Storage) provide unmatched automation, efficiency, and elasticity in support of the most demanding workloads. The EN4023 switch offers the following benefits:
  • Aadvanced storage support:
    • The EN4023 10Gb Scalable Switch offers advanced storage support with multiple connectivity options, including FCoE, Fibre Channel, iSCSI, and NAS storage.
    • Data Center Bridging (DCB), enables the reliable exchange of storage traffic over the LAN, eliminating packet loss when network congestion occurs and allocating bandwidth as needed to keep the network running efficiently.
    • Network-Attached Storage (NAS) Auto QoS intelligence prioritizes delay-sensitive IP storage traffic within the fabric and help ensure consistent performance while decreasing latency.
    • EN4023 10Gb Scalable Switch features 14 Flex Ports, which can take either a 10/1 GbE or 16/8/4/2 Gb Fibre Channel personality.
    • In Fibre Channel mode, these Flex Ports can be used either to directly connect Fibre Channel storage to VCS fabrics, or to bridge FCoE traffic to Fibre Channel SANs as a Fibre Channel Forwarder (FCF), thus protecting existing SAN investments.
    • Enables IP storage networking for improved performance, management, security, and span of control.
  • High performance and reduced network congestion:
    • With 10 Gbps connections, the Flex System EN4023 switch can help your organization reduce network congestion, improve application performance, and meet the capacity required by 10 Gbps servers. The 40 Gbps uplinks can easily aggregate high-bandwidth traffic and reduce bottlenecks, helping your network operate at its peak performance.
    • Provides ultra-low any-to-any port latency of 850 nanoseconds (Ethernet or FCoE/Fibre Channel).
    • The EN4023 delivers an industry-leading 24 MB tunable deep buffers per switch. This provides the buffering capacity to handle increases in traffic, especially during peak times when ports are congested, without lowering performance or increasing latencies.
    • The EN4023 features a single ASIC design instead of the multiple ASIC designs commonly found on other switches. This further improves performance and reduces latency since all ports can communicate via one ASIC.
    • The EN4023 helps maximize network utilization with hardware-based Brocade ISL Trunking. Organizations can create an 80 GbE trunk by utilizing two 40 GbE ports, or a 160 GbE trunk with 16 10 GbE ports. The Brocade trunk is automatically formed between two EN4023 switches when they are linked together, allowing traffic to be equally distributed among all ports. This increases link efficiency and limits traffic disruptions, especially during high traffic times.
  • Simplicity and automation:
    • Brocade VCS Logical Chassis provides the ability to manage an entire Brocade VCS Fabric (up to 48 switches) as a single switch, upgrade software across the fabric with one command, and centralize monitoring and troubleshooting to enhance the overall availability and reliability of the network, with REST API support to allow higher-level management and orchestration frameworks.
    • Configuration is simplified with self-forming fabrics. As additional switches are added, they inherit the configuration of the fabric. This allows the network to scale out with ease. Configuration and device information is always shared among all switches allowing fabric nodes to be added or removed and physical or virtual servers to be relocated — without the fabric requiring manual reconfiguration. In addition, fabrics are self-healing which increases network resiliency. The fabric redirects traffic in case a link fails which helps to ensure uninterrupted traffic flow and prevent data loss.
    • Zero-touch provisioning enables simple, rapid deployment. This feature is provided natively in EN4023 switches through VCS Fabric technology, and it enables installation, automatic software download, and configuration without user intervention. VDX switches can be preconfigured so that newly deployed switches require only power and a network connection to become a part of the fabric. RBridge-ID, VCS-ID, and other VCS parameters are automatically assigned in this case. In addition, Inter-Switch Links (ISLs) are automatically formed between all new and existing switches in the fabric. By eliminating manual processes, this installation method greatly simplifies scale-out architecture.
    • Zero-touch scale-out is achieved with automatic configuration of VCS parameters, self-forming trunks, and logical chassis. Network engineers can add, move, and remove network Brocade VDX switches without having to add or delete network configurations. This helps organizations contain costs while increasing reliability and speed when deploying clouds and data centers.
    • Lenovo Flex System EN4023 switches can facilitate an optimal IP storage network by delivering a fabric-based network architecture to manage the network and storage holistically. At the same time, Brocade Network Advisor (Optional from Brocade)
  • Efficiency and resiliency:
    • Optimized east-west traffic: Embedded switching moves traffic through any active path and avoids the multiple hops found in tiered tree topologies.
    • The EN4023 delivers a highly efficient In-Service Software Upgrade (ISSU) by leveraging a software model that uses a dual-OS infrastructure on a multi-core CPU. This enables data center administrators to deliver enterprise-class business continuity on the switches during a software upgrade/downgrade process. This software change process is non-disruptive to Layer 2, Layer 3, Fibre Channel, and FCoE traffic.
    • The VCS Virtual Fabric feature of Brocade VCS Fabric technology is designed to address the scalability restrictions of traditional VLANs used for multi-tenant segmentation. It provides native secure multi-tenant support for both physical and virtual application deployments within and across VCS fabrics. Managed centrally through Brocade VCS Logical Chassis, the VCS Virtual Fabric feature simplifies and accelerates application deployment, and it ensures policy consistency for each tenant regardless of how application components are distributed across the data center. VXLAN and VRF Lite are other options for network segmentation.
    • The EN4023 provides the ability to have highly elastic domains with extremely efficient load balancing in Layers 1–3. Innovative Brocade ISL Trunking load balances traffic across all the links in a trunk for improved performance at Layer 2. Also in Layer 2, Equal Cost Multi-Path (ECMP) uses all available network bandwidth, allowing all links to be fully active and utilized. In the event of a failure, traffic is automatically routed to the closest path, providing higher resilience and greater application uptime. In Layer 3, the fabric automatically load balances all flows among a number of Layer 3 instances that collectively act as a single Layer 3 gateway. Multilayer multipathing helps improve network utilization, reduce latency, and increase overall network performance.
  • Optimized for virtualization and Software Defined Data Centers:
    • Zero-touch VM discovery is provided with Brocade VM-Aware Network Automation which eliminates the manual configuration of port profiles when a VM is added to the fabric or moved, offering an additional level of automation. The VCS fabric directly communicates with VMware vCenter, automatically downloading all port profile information and the associated MAC address, and distributes the VM-specific information to all switches within the fabric. When the VM moves, no additional configuration is required.
    • Automatic Migration of Port Profiles: During a VM migration, the destination network switch ports must be configured to ensure that the VM traffic experiences consistent policies and configurations. With the Brocade Automatic Migration of Port Profiles (AMPP) feature, the VM policies and networking policies follow the VM within the VCS fabric. As a VM migrates, the destination port in the fabric learns of the MAC address move and automatically activates the port profile configuration within a single fabric or across separate fabrics. AMPP is hypervisor-agnostic and can be used with various hypervisors.
    • Brocade VCS Gateway for NSX is a solution that unifies both virtual and physical infrastructure for a seamless transition to cloud environments. By unifying the best of both worlds — physical and virtual — Brocade VCS Gateway for NSX allows physical devices to connect to virtual devices. Brocade VCS Gateway is integrated with VMware NSX, enabling the entire VCS fabric to function as a VXLAN gateway and eliminating the need for specific network placement. As such, the NSX Controller sees the VCS fabric as a single logical gateway, thereby simplifying management and providing resiliency. Administrators can leverage existing infrastructure while gaining the benefits of VXLAN to support multi-tenancy and large-scale deployments of distributed applications.
    • Logical Chassis technology and northbound APIs with fabric- and node-level orchestration capabilities provide operationally scalable management and integration with data center orchestration frameworks such as OpenStack.
    • Brocade supports programmatic solutions and DevOps tools that allow a customized approach to deploying, operating, and interacting with the network. These solutions offer a new level of simplicity, agility, and rapid, automatic deployment, enabling data centers to evolve to meet new technology requirements.
      • OpenStack Neutron ML2 support and fabric-level, programmable REST APIs with a YANG data model to enable integration with third-party and in-house network automation and cloud management tools. Support for Puppet and Python scripting offers choice and more effective configuration management. These programmability options help automate, simplify, reduce human error, and streamline the process while reducing costs. The tools also help drive productivity by enabling rapid application deployment for enterprises, and increase profitability by streamlining the tenant provisioning process and making networks more intelligent and flexible for cloud providers.
      • VCS fabrics provide support for OpenFlow 1.3, an industry-standard SDN communications protocol, allowing operators to address complex network behavior, optimize performance, and leverage a richer set of capabilities.

System specifications

The Flex System EN4023 10Gb Scalable Switch has the following specifications:

  • Internal ports:
    • 42 internal full-duplex 10 Gigabit ports (optional Upgrade 1 and Upgrade 2 license upgrades are required to activate all ports).
    • Two internal full-duplex 1 GbE ports that are connected to the chassis management module.
  • External ports:
    • Fourteen SFP/SFP+ports (optional Upgrade 1 or Upgrade 2 license upgrades, or both, might be required to activate all external ports depending on the configuration; see Table 2 for details).
      • Base module supports 10 Gb Ethernet SFP+ transceivers (support for 10GBASE-SR or 10GBASE-LR) or SFP+ direct-attach copper (DAC) cables. SFP+ modules and DAC cables are not included and must be purchased with the switch (see Table 3).
      • Upgrade 3 adds support for Flex Ports (support for 8/16 Gb Fibre Channel SFP+ modules; supported FC port types include E_Port (connecting to EX_Port only), F_Port, and N_Port [Access Gateway mode]). FC SFP+ modules are not included and must be purchased with the switch, if needed (see Table 3).
    • Two ports for 40 Gb Ethernet QSFP+ transceivers or QSFP+ DAC cables (ports are disabled by default; an optional FoD license is required to activate them). QSFP+ modules and DAC cables are not included and must be purchased with the switch, if needed.
    • One RS-232 serial port (mini-USB connector) that provides an additional means to configure the switch module.
  • Operating system - Brocade Network OS 6.0.0
  • Scalability and performance
    • 40 Gbps Ethernet ports for extreme external bandwidth and performance (Optional)
    • Non-blocking architecture with wire-speed forwarding of traffic
    • Aggregated throughput of 1.28 Tbps
    • Support for up to 160,000 MAC addresses
    • Up to 4096 VLANs supported per switch
    • Up to 1,024 port profiles (AMPP)
    • Maximum members in a standard LAG: 64
    • Maximum LAG groups in a VCS Fabric: 512
    • Maximum members in which a vLAG can span: 8
    • Maximum members in a vLAG: 64
    • Support for jumbo frames (up to 9,216 bytes)
    • Queues per port: 8
    • Maximum ACLs: 13,000
    • Maximum ARP entries: 32,000
    • Maximum IPv4 unicast routes: 12,000
    • Maximum IPv6 unicast routes: 3,000 (Hardware support)
  • Layer 2 switching features:
    • Address Resolution Protocol (ARP) RFC 826
    • High availability/In-Service Software Upgrade—hardware-enabled
    • IGMP v1/v2 Snooping
    • MAC Learning and Aging
    • Link Aggregation Control Protocol (LACP) IEEE 802.3ad/802.1AX
    • Virtual Local Area Networks (VLANs)
    • VLAN Encapsulation 802.1Q
    • Private VLANs
    • Edge loop detection (ELD)
    • Per-VLAN Spanning Tree (PVST+/PVRST+)
    • Rapid Spanning Tree Protocol (RSTP) 802.1w
    • Multiple Spanning Tree Protocol (MSTP) 802.1s
    • STP PortFast, BPDU Guard, BPDU Filter
    • STP Root Guard
    • Layer 2 Access Control Lists (ACLs)
    • Pause Frames 802.3x
    • Uni-Directional Link Detection (UDLD)
  • Layer 3 switching features:
    • Border Gateway Protocol (BGP4+)
    • DHCP Helper
    • Layer 3 ACLs
    • Multicast: PIM-SIM, IGMPv2
    • OSPF
    • Static Routes
    • IPv4/v6 ACL
    • Policy-Based Routing (PBR)
    • Bidirectional Forwarding Detection (BFD)
    • 16-way ECMP
    • VRF Lite
    • VRF-aware OSPF, BGP, VRRP, static routes
    • VRRP v2 and v3
    • IPv4/IPv6 dual stack
    • IPv6 ACL packet filtering
    • IPv6 routing
    • Wire-speed routing for IPv4 and IPv6 using any routing protocol
  • Brocade VCS Fabric technology features:
    • Automatic Fabric Formation
    • DHCP Option 66/67 (Auto Fabric Provisioning)
    • Distributed Configuration Management
    • Distributed Fabric Services
    • 16-way Equal Cost Multi-Path (ECMP)
    • Zero-touch provisioning
    • Switch Beaconing
    • Transparent Interconnection of Lots of Links (TRILL)
    • Transparent LAN Services
    • Virtual Link Aggregation Group (vLAG) spanning
    • VRRP-E
    • Fabric virtual gateway
  • Fibre Channel/FCoE features (requires Upgrade 3 - FCoE license)
    • Priority-based Flow Control (PFC) 802.1Qbb
    • Enhanced transmission Selection (ETS) 802.1Qaz
    • Data Center Bridging eXchange (DCBX)
    • DCBX Application Type-Length-Value (TLV) for FCoE and iSCSI
    • Multihop Fibre Channel over Ethernet (FCoE); requires Brocade VCS Fabric technology
    • FC-BB5 compliant Fibre Channel Forwarder (FCF)
    • Native FCoE forwarding
    • FCoE to Fibre Channel Bridging
    • FCoE on QSFP+ ports
    • Flex Ports, allowing direct and SAN connectivity of Fibre Channel targets
    • Multi-hop Access Gateway support
    • End-to-end FCoE (initiator to target)
    • FCoE Initialization Protocol (FIP) v1 support for FCoE device login and initialization
    • Name Server-based zoning
    • Supports connectivity to FIP Snooping Bridge (FSB) device
    • FCoE traffic over standard LAG
    • Interface Binding
    • Logical SANs
  • Multitenancy and virtualization features:
    • TRILL FGL-based VCS Virtual Fabric feature
    • Brocade VCS Gateway for NSX with VMware NSX Orchestration
    • Virtual fabric extension
    • Automatic Migration of Port Profiles (AMPP)
    • VM-Aware Network Automation
    • BFD for virtual fabric extension
  • IP storage:
    • Auto QoS (automatic prioritization of IP storage traffic)
  • High Availability
    • ISSU L2 and L3
    • BFD
    • OSPF v2/v3 and OSPF3-NSR
    • BGP4 & BGP4-GR
  • Quality of Service (QoS):
    • ACL-based QoS
    • Eight priority levels for QoS
    • Class of Service (CoS) IEEE 802.1p
    • DSCP Trust
    • DSCP to Traffic Class Mutation
    • DSCP to CoS Mutation
    • DSCP to DSCP Mutation
    • Random Early Discard
    • Per-port QoS configuration
    • ACL-based Rate Limit
    • Dual-rate three color token bucket
    • ACL-based remarking of CoS/DSCP/Precedence
    • ACL-bases sFlow
    • Scheduling Strict Priority (SP), Deficit Weighted Round-Robin (DWRR), Hybrid Scheduling (Hybrid)
    • Queue-based Shaping
    • Flow-based QoS
  • Management and control:
    • IPv4/IPv6 management
    • Industry standard Command Line Interface
    • Netconf API
    • REST API with YANG data model
    • Brocade VCS Plugin for OpenStack
    • Link Layer Discovery Protocol (LLDP) IEEE 802.1AB
    • Logical chassis management
    • MIB II RFC 1213 MIB
    • Switch Beaconing
    • Management VRF
    • Switch Port Analyzer (SPAN)
    • Remote SPAN (RSPAN)
    • Telnet
    • SNMPv3 (default) and SNMPv1
    • sFlow RFC 3176
    • Out-of-band management
    • RMON-1, RMON-2
    • NTP
    • Management Access Control Lists (ACLs)
    • Role-Based Access Control (RBAC)
    • Range CLI support
    • UDLD
    • OpenStack Neutron ML2 plugin
    • Python
    • Puppet
  • Software-Defined Networking (SDN) and programmability:
    • OpenFlow 1.3
    • VXLAN Gateway
    • REST API with YANG data model
    • Puppet
    • Python
  • Security
    • Port-based Network Access Control 802.1X
    • RADIUS
    • TACACS+
    • Secure Shell (SSHv2)
    • BPDU Drop
    • Lightweight Directory Access Protocol (LDAP)
    • Secure Copy Protocol (SCP)
    • Secure FTP (sFTP)
  • Switch health monitoring
    • Brocade Fabric Watch monitoring and notification
    • Command-line interface (CLI) through Telnet or Secure Shell V2 (SSHv2)
    • Terminal emulation program connection to the serial port interface
    • Brocade Network Advisor (optional, can be purchased from Brocade)
    • Simple Network Management Protocol (SNMP) agent

Supported standards

The switches support the following standards:

  • IEEE 802.1AB Link Layer Discovery Protocol (LLDP)
  • IEEE 802.1p Class of Service (CoS) prioritization and Tagging
  • IEEE 802.1Q Tagged VLAN (frame tagging on all ports when VLANs are enabled)
  • IEEE 802.1Qbb Priority-Based Flow Control (PFC)
  • IEEE 802.1Qaz Enhanced Transmission Selection (ETS)
  • IEEE 802.1w Rapid Reconfiguration of Spanning Tree Protocol
  • IEEE 802.1D Spanning Tree Protocol
  • IEEE 802.1s Multiple Spanning Tree
  • IEEE 802.3 Ethernet
  • IEEE 802.3ab 1000BASE-T
  • IEEE 802.3ad Link Aggregation with LACP
  • IEEE 802.3ae 10 Gb Ethernet
  • IEEE 802.3ap 10GBASE-KR backplane 10 Gb Ethernet
  • IEEE 802.3ba 40GBASE-SR4 short range fiber optics 40 Gb Ethernet
  • IEEE 802.3ba 40GBASE-CR4 copper 40 Gb Ethernet
  • IEEE 802.3u 100BASE-TX Fast Ethernet
  • IEEE 802.3x Flow Control (Pause Frames)
  • IEEE 802.3z 1000BASE-X
  • SFF-8431 10GSFP+Cu SFP+ Direct Attach Cable

Supported chassis and adapters

The EN4093 switches are installed in I/O module bays in the rear of the Flex System chassis, as shown in the following figure. I/O modules are normally installed in pairs because ports on the I/O adapters that are installed in the compute nodes are routed to two I/O module bays for redundancy and performance.

Location of the switch bays in the Flex System chassis
Figure 2. Location of the switch bays in the Flex System chassis

The EN4023 10Gb Scalable Switches can be installed in bays 1, 2, 3, and 4 of the Flex System chassis. A supported adapter must be installed in the corresponding slot of the compute node.

With four-port or eight-port adapters, an optional FoD Upgrade might be required for the switch to allow communications on all ports (however, with eight-port adapters, only six adapter ports can be used, and the two remaining ports are not used.). (See Table 2 for details.)

With DPOD, there is no need to buy more switch upgrades for 4-port and 8-port adapters if the total number of port licenses on the switch does not exceed the number of external (upstream network ports) and internal (compute node network ports) connections that are used.

In compute nodes that have an integrated dual-port 10 GbE network interface controller (NIC), NIC ports are routed to bays 1 and 2 with a specialized periscope connector, and the adapter in slot A1 is not required. However, when needed, the periscope connector can be replaced with the adapter. In such a case, the integrated NIC is disabled.

The following table shows compatibility information for the EN4023 and Flex System chassis.

Table 4. Flex System chassis compatibility

DescriptionPart numberEnterprise Chassis with CMMEnterprise Chassis with CMM2Carrier-grade Chassis with CMM2
Lenovo Flex System EN4023 10Gb Scalable Switch94Y5212
Yes
Yes
No

The connections between the adapters that are installed in the compute nodes to the EN4023 installed in the I/O bays in the chassis are shown in the following figure. The figure shows both half-wide servers, such as the x240 with two adapters, and full-wide servers, such as the x440 with four adapters.

Logical layout of the interconnects between I/O adapters and the EN4023 modules
Figure 3. Logical layout of the interconnects between I/O adapters and the EN4023 modules

The following table shows the connections between adapters that are installed in the compute nodes to the EN4023 modules installed in the I/O bays in the chassis.

Table 5. Adapter to I/O bay correspondence
I/O adapter slot
in the server
Port on the adapterCorresponding I/O bay in the chassis
Bay 1Bay 2Bay 3Bay 4
x222 integrated LOMPort 1Yes
Port 2Yes
Port 1Yes
Port 2Yes
Slot 1 (or integrated dual-port LOM - ports 1 and 2)Port 1Yes
Port 2Yes
Port 3*Yes
Port 4*Yes
Port 5*Yes
Port 6*Yes
Slot 2Port 1Yes
Port 2Yes
Port 3*Yes
Port 4*Yes
Port 5*Yes
Port 6*Yes
Slot 3
(full-wide compute nodes only)
Port 1Yes
Port 2Yes
Port 3*Yes
Port 4*Yes
Port 5*Yes
Port 6*Yes
Slot 4
(full-wide compute nodes only)
Port 1Yes
Port 2Yes
Port 3*Yes
Port 4*Yes
Port 5*Yes
Port 6*Yes
* Depending on the number of compute nodes and uplinks, Ports 3, 4, 5, and 6 might require an optional FoD port upgrade of the EN4023 (see Table 2).

The following table lists the I/O adapters that are supported by the EN4023 10Gb Scalable Switches.

Table 6. Supported network adapters

DescriptionPart numberFeature code
10 Gb Ethernet
Embedded 10Gb Virtual Fabric Adapter (x222; dual 2-port)NoneNone
Embedded 10Gb Virtual Fabric Adapter (x240, x440; 2-port)*NoneNone
Flex System CN4022 2-port 10Gb Converged Adapter88Y5920A4K3
Flex System CN4052 2-port 10Gb Virtual Fabric Adapter00JY800A5RP
Flex System CN4054 10Gb Virtual Fabric Adapter (4-port)90Y3554A1R1
Flex System CN4054R 10Gb Virtual Fabric Adapter (4-port)00Y3306A4K2
Flex System CN4058S 8-port 10Gb Virtual Fabric Adapter#94Y5160A4R6
Flex System EN4132 2-port 10Gb Ethernet Adapter90Y3466A1QY
Flex System EN4172 2-port 10Gb Ethernet Adapter00AG530A5RN
1 Gb Ethernet
Embedded 1 Gb Ethernet controller (x220; 2-port)NoneNone
Flex System EN2024 4-port 1Gb Ethernet Adapter 49Y7900A10Y
* The Embedded 10Gb Virtual Fabric Adapter is included in some models of the x240 and x440.
# With eight-port adapters, six adapter ports can be used with the EN4023; the remaining two ports are not used.

The adapters are installed in slots in each compute node. The following figure shows the locations of the slots in the x240 Compute Node. The positions of the adapters in the other supported servers are similar.

 Location of the I/O adapter slots in the IBM Flex System x240 Compute Node
Figure 4. Location of the I/O adapter slots in the Flex System x240 Compute Node

Connectors and LEDs

The following figure shows the front panel of the EN4023 10Gb Scalable Switch.

Front panel of the Lenovo Flex System EN4023 10Gb Scalable Switch
Figure 5. Front panel of the Lenovo Flex System EN4023 10Gb Scalable Switch

The front panel contains the following components:

  • Light-emitting diodes (LEDs) that display the status of the interconnect module and the network:
    • The OK LED indicates that the interconnect module passed the power-on self-test (POST) with no critical faults and is operational.
    • Identify: This blue LED can be used to identify the module physically by illuminating it through the management software.
    • The error LED (switch module error) indicates that the module failed the POST or detected an operational fault.
  • One mini-USB RS-232 console port that provides an additional means to configure the interconnect module. This mini-USB-style connector enables the connection of a special serial cable. (The cable is optional, and it is not included with the interconnect module. For more information, see the "Part number information" section.)
  • One 1 Gb Ethernet RJ-45 port for switch configuration and management.
  • Fourteen external SFP+ ports for 1/10 Gb connections to external Ethernet devices. Upgrade 3 enables support for Flex Ports, allowing FCoE traffic and support for 8/16 Gb FC optical transceivers for FC SAN connectivity.
  • Two external QSFP+ port connectors to attach QSFP+ modules for a single 40 Gb uplink per port. Upgrade 3 enables support for FCoE traffic.
  • An Ethernet link OK LED and an Ethernet Tx/Rx LED for each external port on the interconnect module.

Network cabling requirements

The network cables that can be used with the EN4023 are shown in the following table.

Table 7. EN4023 network cabling requirements

TransceiverStandardCableConnector
40 Gb Ethernet
40GBASE-SR4 QSFP+ Transceiver (49Y7884)40GBASE-SR410 m or 30 m MTP fiber optics cables supplied by Lenovo (see Table 3); support for up to 100 m with OM3 multimode fiber or up to 150 m with OM4 multimode fiberMTP
Direct attach cable40GBASE-CR41m or 3m QSFP+ to QSFP+ DAC cables supplied by Lenovo (see Table 3)QSFP+
10 Gb Ethernet
10GBASE-SR SFP+ Transceiver (46C3447, 49Y4216)10GBASE-SRUp to 30 m with fiber cables supplied by Lenovo (see Table 3); 850 nm OM3 multimode fiber up to 300 m or up to 400 m with OM4 multimode fiberLC
10GBASE-LR SFP+ Transceiver (00D6180, 95Y0540)10GBASE-LR1310 nm single-mode fiber cable up to 10 kmLC
Direct attach cable10GSFP+CuUp to 5 m (Active) or 3 m (Passive) SFP+ copper DAC cables (see Table 3)SFP+
16 Gb Fibre Channel
16Gb FC SFP+ Transceiver Module (88Y6393)FC-PI-5 (16GFC)Up to 30 m with fiber optic cables supplied by Lenovo (see Table 3); 850 nm OM3 multimode fiber up to 100 m or OM4 multimode fiber up to 125 mLC
8 Gb Fibre Channel
8Gb FC SFP+ Transceiver Module (88Y6416)FC-PI-4 (8GFC)Up to 30 m with fiber optic cables supplied by Lenovo (see Table 3); 850 nm OM3 multimode fiber up to 150 mLC
Management ports
External 1 GbE management port1000BASE-TUTP Category 5, 5E, and 6 up to 100 metersRJ-45
External RS-232 management portRS-232DB-9-to-mini-USB or RJ-45-to-mini-USB console cable (comes with optional Management Serial Access Cable, 90Y9338)Mini-USB

Warranty

The Lenovo Flex System EN4023 carries a 1-year, customer-replaceable unit (CRU) limited warranty. When installed in a chassis, these I/O modules assume your system’s base warranty and any Lenovo Services warranty upgrades.

Physical specifications

Dimensions and weight of the EN4023 (approximate):

  • Height: 30 mm (1.2 in.)
  • Width: 402 mm (15.8 in.)
  • Depth: 297 mm (11.7 in.)
  • Weight: 3.6 kg (7.9 lb)

Shipping dimensions and weight (approximate):
  • Height: 114 mm (4.5 in.)
  • Width: 508 mm (20.0 in.)
  • Depth: 432 mm (17.0 in.)
  • Weight: 4.1 kg (9.1 lb)

Agency approvals

The EN4023 conforms to the following regulations:

  • UL 60950
  • CAN/CSA-C22.2 No 60950
  • IEC/EN 60950
  • FCC 47 CFR Part 15, Class A
  • AS/NZS CISPR 22, Class A
  • CISPR 22, Class A
  • CE Mark (EN55022 Class A, EN55024, EN61000-3-2, EN61000-3-3)
  • ICES003, Class A
  • VCCI, Class A
  • KN22, Class A

Typical configurations

The following typical configurations are described:

  • Inter-Switch Links
  • Direct connection without ToR switches
  • Virtual Link Aggregation
  • Multiple connection types
  • EN4023 in the VCS network

In all scenarios, each port of the compute node's NIC can be divided into up to four virtual NICs (vNICs), adding more granularity in configuring virtualized network environments, if needed.

Inter-Switch Links

In this topology, EN4023 switches are connected to the Brocade VDX switches via Inter-Switch Links (ISLs) forming a scale-out self-aggregated data center edge fabric that is based on Brocade VCS technology. Such a fabric, where STP is eliminated, is highly resilient and efficient, and it allows network administrators to manage the entire network as a single logical switch.

The connectivity topology, which can be used with the EN4023 switches and the Brocade VDX upstream network devices, is shown in the following figure.

EN4023 connectivity topology - Inter-Switch Links
Figure 6. EN4023 connectivity topology - Inter-Switch Links

This approach can also be used when building out a POD or cluster using EN4023 switches in multiple chassis. For example, a client can build a cluster of up to 24 chassis, using 48 EN4023 switches, which would support up to 336 compute nodes. This configuration is shown in the following figure.

EN4023 connectivity topology - Inter-Switch Links between chassis
Figure 7. EN4023 connectivity topology - Inter-Switch Links between chassis

Direct connection without ToR switches

In this topology, shown in the following figure, the EN4023 switches are connected directly to the upstream network via vLAGs.


Figure 8. EN4023 connectivity topology – Direct connection without ToR switches

Such a topology allows network administrators to connect to the upstream network without the need for a Top of Rack (ToR) switch, allowing for a lower-cost solution and freeing up valuable rack space.

Virtual Link Aggregation

The connectivity topology, which can be used with the EN4023 switches and the Brocade MLX upstream network devices, is shown in the following figure.

EN4023 connectivity topology - Virtual Link Aggregation
Figure 9. EN4023 connectivity topology - Virtual Link Aggregation

In this access-layer topology, aggregation is split between two physical switches, and each EN4023 is connected to both ToR switches through static or Link Aggregation Control Protocol (LACP)-aggregated links. Compute Node's NICs are configured in an active/active NIC teaming. This topology can be used to insert Brocade VCS Fabric technology into existing infrastructure, because it fully interoperates with existing LAN protocols, services, and architectures.

The EN4023 can connect to the core and aggregations switches from Lenovo, Brocade, Cisco, Juniper, Arista, and any vendor that utilizes industry standard connectivity, is shown in the figure above.

Multiple connection types

The EN4023 supports multiple connection types to the upstream SAN or Ethernet network storage controllers. Some of these are shown in the following figure. The EN4023 supports direct connections to storage or can connect through other switches or fabrics.


Figure 10. Sample of the connection types that are supported by the EN4023

Specifics about these supported configurations include:
  • Connecting to Brocade FCF (example VDX 6740) using the EN4023 as an FCoE Transit switch and breaking out the FC at the Top-of-Rack switch
  • Connecting to Brocade SAN by utilizing Access Gateway (NPIV) mode on the EN4023
  • Utilizing Fibre Channel Routing (FCR) to connect to the upstream Brocade SAN via EX_Port
  • Direct-attach storage
    • NAS IP (Ethernet) Storage
    • iSCSI IP (Ethernet) Storage (Lenovo Storage S2200/3200 or IBM Storwize V3700/V5000/V7000)
    • FC storage (Lenovo Storage S2200/3200 or IBM Storwize V3700/V5000/V7000)
    • FCoE or FC storage (IBM Storwize V3700/V5000/V7000)

EN4023 in the VCS network

The following figure shows a sample scenario where the EN4023 is used as a part of the data center edge network built with VCS technology.

EN4023 in the VCS network with the Brocade VDX 6740 ToR
Figure 11. EN4023 in the VCS network with the Brocade VDX 6740 ToR

Table 8. EN4023 in the VCS network with the Brocade VDX 6740 ToR (Figure 11)
Diagram
reference
DescriptionPart
number
Quantity
1Flex System VCS solution
Flex System x240 Compute Node with Embedded Virtual Fabric AdapterVariesVaries
Flex System EN4023 10Gb Scalable Switch94Y5212Two per chassis
2Brocade VDX 6740 Switch

Note: You also need SFP+ modules and optical cables (not shown in Table 8; see Table 3 for details) for the external 10 Gb Ethernet connectivity.

Related publications

For more information, see the following EN4023 10Gb Scalable Switch product publication, which is available from the Flex System Information Center at
http://publib.boulder.ibm.com/infocenter/flexsys/information/index.jsp:

  • Installation and User Guide

The following documents are other useful references:

Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.

Profile

Publish Date
08 October 2013

Last Update
20 August 2015


Rating: Not yet rated


Author(s)

IBM Form Number
TIPS1070