IBM System Networking RackSwitch G8332

IBM Redbooks Product Guide


Abstract

The IBM® System Networking RackSwitch™ G8332 (RackSwitch G8332) is a high-density 40 Gigabit Ethernet (GbE) switch that is designed for the data center, providing speed, intelligence, and interoperability on a proven platform. Most clients look at it as an ideal aggregation class switch for connecting multiple IBM System Networking RackSwitch G8264 class switches or multiple Flex Chassis by using the SI4093, EN4093R, or the CN4093 Ethernet modules. Some clients might consider the RackSwitch G8332 to build server clusters or PODs in combination with the SI4093 because it provides a low-cost alternative to expensive modular chassis-based solutions and uses technologies such as vLAG for multi-chassis link aggregation and L3 ECMP (for larger PODs of up to 2800 10G servers). Other clients might use the RackSwitch G8332 for High Performance Computing (HPC) of other applications running on servers that are using 40 Gigabit Ethernet adapters. The RackSwitch G8332 also supports Network Address Translation (NAT) through hardware and advanced features, such as PTP, DCBX, and OpenFlow. Designed with top performance in mind, the RackSwitch G8332 provides high-bandwidth switching, filtering, and traffic queuing without delaying data. Large data center grade buffers keep traffic moving. Hot-swappable, redundant power and fans, along with numerous high-availability features, enable the RackSwitch G8332 to be highly available for business-sensitive traffic.

Contents


This IBM® Redbooks® Product Guide describes the IBM System Networking RackSwitch™ G8332 switch (RackSwitch G8332). The RackSwitch G8332 provides low latency, lossless performance, and a feature-rich design with key virtualization features, such as Converged Enhanced Ethernet (CEE)/Data Center Bridging (DCB)¹, high availability, and enterprise class Layer 2 and Layer 3 functions. In addition, the RackSwitch G8332 also delivers excellent cost savings as you consider acquisition costs, energy costs, operational expenses, and ease of use and management for a 40 Gbps class switch. The RackSwitch G8332 has 32 QSFP+ ports and is suitable for clients using 10 Gigabit Ethernet or 40 Gigabit Ethernet connectivity (or both).

The RackSwitch G8332 enables customers to easily build an end-to-end flat 2-tier network that is based on industry standards using IBM switches and servers. For example, the RackSwitch G8332 is an ideal tier two switch to use when connecting a number of IBM switches, such as IBM System Networking RackSwitch G8264, IBM System Networking RackSwitch G8264T, or IBM System Networking RackSwitch G8264CS, or embedded Ethernet offerings for Flex Systems (SI4093, EN4093/R, or CN4093) with a 40 Gigabit Ethernet uplink ports at the access layer. Other clients like the RackSwitch G8332 for the investment protection. These clients can use it with their 10 Gigabit Ethernet environments today, but can also use it in the future as they move to 40 Gigabit Ethernet.

With latency below 600 nanoseconds, the RackSwitch G8332 is among industry leaders. This fact and the 2.56 Tbps throughput makes the RackSwitch G8332 an ideal offering for latency-sensitive applications, such as high-performance computing, financial applications, hosting environments, and cloud designs in Enterprise, Web 2.0, and VM Centric data centers. In addition, IBM supports IBM VMready® with Virtual Vision, which enables the network to be virtual machine (VM) aware, and provides the capability to have a virtualization environment that is simpler and less expensive with exceptional performance. Figure 1 shows the RackSwitch G8332.

IBM System Networking RackSwitch G8332 switch
Figure 1. IBM System Networking RackSwitch G8332


Did you know?

Here are the key features of the RackSwitch G8332:
  • Supports several types of configurations: 10 Gigabit Ethernet, 40 Gigabit Ethernet, Converged Enhanced Ethernet (CEE/DCB)1, and iSCSI.
  • With exceptional port density and flexibility with break-out cables, the RackSwitch G8332 can support up to 96x 10 Gigabit Ethernet connections and still have 8x 40 Gigabit Ethernet uplinks.
  • The RackSwitch G8332 is SDN ready.
    • The G8332 has support for OpenFlow. With OpenFlow, you can easily create user-controlled virtual networks, optimize performance dynamically, and minimize complexity when used with many industry available OpenFlow controllers including IBM Software Defined Network for Virtual Environments, OpenFlow Edition.
    • The RackSwitch G8332 supports network virtualization through overlays as a VXLAN hardware gateway.
  • The RackSwitch G8832 is Cloud ready.
    • NetBoot support offers automated network provisioning helping reduce operational costs.
    • Support for industry standards like VEPA and VEB 802.1Qbg using the VMready features in IBM Networking OS provide VM aware networking and automation for multiple hypervisor environments


Benefits

The RackSwitch G8332 offers the following benefits:
  • High performance: This 40 Gigabit Ethernet low latency switch with 2.56 of Tbps throughput provides the best combination of low latency, non-blocking switching, and ease of management. The RackSwitch G8332 is also a single ASIC design, which promises consistent lower port-to-port latency.
  • Lower power and better cooling: The RackSwitch G8332 uses as little as 270 W of power, which is a fraction of the power consumption of previous 40 Gigabit Ethernet offerings. The front-to-rear or rear-to-front cooling design reduces data center air conditioning costs by having airflow match the servers in the rack. In addition, variable speed fans help automatically reducing power consumption.
  • High availability: The RackSwitch G8332 also comes standard with hot-swap redundant power supplies and fans, making the switch highly reliable and easy to service in the unlikely event of a failure.
  • VM-Aware networking: VMready software on the switch simplifies configuration and improves security in virtualized environments. VMready automatically detects virtual machine movement between physical servers and instantly reconfigures each VM’s network policies across VLANs to keep the network running without interrupting traffic or impacting performance. VMready works with all leading VM providers, such as VMware, Citrix, Xen, Microsoft Hyper V, Red Hat KVM, and IBM PowerVM®.
  • Layer 3 function: The switch includes Layer 3 function, which provides security and performance benefits, as inter-VLAN traffic stays within the switch. This switch also provides the full range of Layer 3 protocols from static routes for technologies, such as Open Shortest Path First (OSPF) and Border Gateway Protocol (BGP) for enterprise customers.
  • Seamless interoperability: IBM switches interoperate seamlessly with other vendors' upstream switches.
  • Fault tolerance: IBM switches learn alternative routes automatically and perform faster convergence in the unlikely case of a link, switch, or power failure. The RackSwitch G8332 uses technologies such as L2 trunk failover, advanced VLAN-based failover, VRRP, and HotLink™.
  • OpenFlow enabled: The RackSwitch G8332 offers the benefits of OpenFlow. OpenFlow is the new open application programming interface (API) that enables the network administrator to easily configure and manage virtual networks that control traffic on a “per-flow” basis. It creates multiple independent virtual networks and related policies without dealing with the complexities of the underlying physical network and protocols.
  • Multicast: Multicast supports IGMP Snooping v1, v2, and v3 with 3K IGMP groups, and Protocol Independent Multicast, such as PIM Sparse Mode or PIM Dense Mode.
  • Management: Network Address Translation (NAT) allows a single device, such as a router, to act as an agent between the Internet (or "public network") and a local (or "private") network. This means that only a single, unique IP address is required to represent an entire group of local IP addresses, to the external network. In addition to reducing the number of public IP addresses that are required, NAT also provides security by acting as a firewall between internal and external networks.
  • Standards: The RackSwitch G8332 supports the IEEE 1588 Precision Time Protocol standard, which helps improve synchronization of the network while also improving performance, availability, and flexibility.


Features and specifications

This section lists the key features and specifications of the RackSwitch G8332.

Performance

Here are the performance features of the RackSwitch G8332:
  • Single-switch ASIC design
  • 2.56 Tbps non-blocking switching throughput (full duplex)
  • Sub-600 nanosecond latency (as low as 531 nanoseconds)
  • 1440 million packets per second (mpps)

Hardware features

Table 1 lists the hardware features.

Table 1. Hardware features
ItemDescription
Models
  • IBM System Networking RackSwitch G8332 Rear-to-Front: For ports in the rear of the rack to match traditional server designs such as IBM System x®, IBM Power Systems™, IBM BladeCenter®, and IBM Flex System™.
  • IBM System Networking RackSwitch G8332 Front-to-Rear: For ports in the front of the rack to match the airflow of IBM iDataPlex® and NeXtScale systems.
Interface options
  • 32x QSFP+ ports (40 Gbps).
    • Eight ports (1 and 26-32) are dedicated to only 40Gb Ethernet connectivity.
    • Support for optional break-out cables in ports 2 - 25, providing four 10 Gb connections per port.
  • One 10/100/1000 Ethernet RJ45 port for out-of-band management.
  • One USB port for mass storage device connection.
  • One fixed mini-USB Console port for configuration and code changes.
  • Server-like port orientations, enabling short and simple cabling.
Dimensions
  • 17.3 in. wide, 19 in. deep, and 1 RU high.
Weight
  • 12 kg (26.45 lb).
Rack Installation Kit
  • Generic Rack Mount Kit (2-post).
  • Optional versatile 4-post mounting options for 19-inch server rack or datacom rack.
  • Optional IBM System Networking Recessed 19" 4-post Rail Kit for NeXtScale clients to close the cabinet doors.
  • Can be mounted vertically or horizontally.
LEDs
  • One per QSFP+ port for Link Status and activity. The same LED is used for both 40G and 10G modes.
    • Blue “Service Required” LED:
      • Solid blue is a locator function.
      • Flashing blue means service is required.
    • Green Stacking LED; labeled as “S” on the front panel:
      • Solid green means stacking is enabled.
      • Off means stacking is disabled.
  • Fan module LED:
    • Solid green means functional.
    • Flashing green means that there is a problem.
  • Power Supply module LED:
    • Solid green means functional.
    • Flashing green means that there is a problem.
  • Dedicated Eight QSFP+ Module LED:
    • Solid green means functional.
    • Flashing green means that there is a problem.
Airflow
  • Front-to-rear or rear-to-front cooling.
  • Four redundant variable hot-swap speed fans for reduced power draw.
Power
  • Dual load-sharing hot-swap internal power modules, 50 - 60 Hz, 100 - 240 V AC auto-switching per module.
  • Typical power consumption of 270 watts.
Environmental specifications
  • Temperature: Ambient operating: 0 to 40 ºC.
  • Relative humidity: Non-condensing, operating 10 - 90%.
  • Altitude: Operating 2,000 m (6,550 feet).
  • Heat dissipation: 920 BTU/hour (typical).
  • Mean time between failures (MTBF): 227,781 hours at 30° C

Figure 2 shows the rear of the switch.

Rear view of the RackSwitch G8332
Figure 2. Rear view of the RackSwitch G8332

Software features

For the most current updates to software features, refer to the product guides. Table 2 lists the software features.

Table 2. Software features
ItemDescription
Security
  • RADIUS, TACACS+, and LDAP
  • SCP
  • Wire Speed Filtering: Allow and Deny
  • HTTPS Secure BBI
  • Enhanced security password and SSH v2
Management
  • MAC address move notification
  • SNMP v1, v2c, and v3
  • RMON
  • Dual NTP Support
  • Accept DHCP IP address assignment
  • DHCP Snooping, DHCP Option 82
  • Dual software images
  • Syslog Configuration tracking
  • Shift B Boot Menu (Password Recovery/Factory Default)
  • Management ACLs
  • IP Loopback
  • IPv6
  • IPv4
  • IPSec
  • IKE
  • IPv6 RFC support acceptable for NIST certification
  • Netboot
  • tftp/ftp/sFTP
  • Default IP, Duplicate IP Detection
VLANs
  • Port-based VLANs
  • 4096 VLAN IDs supported
Lossless Ethernet
  • 802.1 Data Center Bridging
  • Priority Based Flow Control (PFC)
  • Enhanced Transmission Selection (ETS)
  • Data Center Bridge Exchange protocol (DCBX)
  • FIP Snooping
  • Converged Enhanced Ethernet
Link Aggregation
  • Static Trunks (Etherchannel) and LACP (802.3ad)
  • Static LAG
  • vLAG
  • Configurable Trunk Hash algorithm
Spanning Tree
  • Multiple Spanning Tree (802.1 s)
  • Rapid Spanning Tree (802.1 w)
  • PVRST+
  • Auto assignment of VLANs to STG
  • BPDU Guard
  • Root guard
  • Loop guard
Quality of service
  • QoS 802.1p (priority queues)
  • DiffServ IPV4 and IPv6
  • ACL, IGMP Profiles
  • DSCP remarking
  • Metering
  • 12 MB shared packet buffer
  • Weighted random early detection with explicit congestion notification (WRED/ECN)
  • CPU priority policies (CoPP)
  • QoS Monitoring
  • Storm Control
Routing protocols
  • RIP v1/v2
  • OSPF v1, v2, v3, and v3 OSPF over IPSec
  • BGP – Debugging, NH-self, Route reflector
  • Static Routing
  • IPv6 Static Route
High availability
  • Layer 2 failover
  • HotLinks™
  • Virtual Router Redundancy support (VRRP)
Multicast
  • OpenFlow 1.0 and 1.3.1
  • MLDv2
  • IGMP Snooping v1/v2/v3
  • IGMP querier, relay, and filtering
  • Protocol Independent Multicast (PIM Sparse Mode/Dense Mode)
  • Static Multicast MAC entry
DCBX and other features
  • Microburst Detection
  • WRED
  • CoPP
  • FCoE – Fibre Channel over Ethernet Data Center Bridging
    • Priority Flow Control (PFC)
    • Enhanced Transmission Selection (ETS)
    • FIP Snooping
    • Data Center Bridging Capabilities Exchange Protocol (DCBX)
    • FC-BB-5 FCF with internal BB6 implementation
Monitoring
  • Port mirroring
  • ACL-based mirroring
  • ACL logging
  • Processor logging
  • sFlow
  • UDLD
Virtualization
  • VMready
Clients
  • System Networking Switch Center (SNSC)
  • isCLI (Cisco-like)
  • Browser-based client or Telnet
Standard protocols
  • DHCP Client
  • DHCP Relay
  • LLDP
  • 160K MAC Table
  • 9 K Jumbo Frames
  • 802.3X Flow Control
  • 802.1x
  • Precision Time Protocol (PTP)


Popular configurations

This section describes some configurations for the RackSwitch G8332.

40 G aggregation switch for east-west traffic

As more clients seek to flatten their networking to use more of an east-west traffic flow that is required by many of today’s applications and especially virtualized environments, the RackSwitch G8332 is an ideal tier-two switch for connecting multiple access switches or interconnect modules.

One example is when the RackSwitch G8332 is combined with IBM RackSwitch G8264 by using SFP+ or G8264T using 10Gbase-T to the compute nodes, where clients can easily support up to 1536 nodes on 10 Gigabit Ethernet network with as few as a 3:1 oversubscription. Alternatively, clients can also build a smaller network by using the G8264CS while using the Omni Port to connect to their FC/FCoE SAN, as shown in Figure 3 and Figure 4.

Aggregation switch for east-west traffic
Figure 3. Aggregation switch for east-west traffic

1536 Node 10Gigabit Ethernet Network (3:1 Oversubscribed)
Figure 4. 1536 Node 10 Gigabit Ethernet Network (3:1 Oversubscribed)

Another example is the exceptional flat network implementations that can be built with IBM embedded Ethernet offerings for Flex System (SI4093, EN4093/R, and CN4093), which can be used easily to support up to 15 chassis of only a pair of RackSwitch G8332 switches. Figure 5 shows an example with nine chassis.

Flat network design supporting multiple Flex System Chassis
Figure 5. Flat network design supporting multiple Flex System Chassis

Building a POD

To build even larger networks, the RackSwitch G8332 offering allows for the creation of a simple two-tier design that can support up to 2,816 10 GbE server ports with a low oversubscription ratio of 2.1 to 1. Figure 6 shows an example.

POD Design 2816 10 Gigabit Ethernet ports
Figure 6. POD Design 2816 10 Gigabit Ethernet ports

Using the RackSwitch G8332 switch as an edge switch for servers using 40 G Ethernet adapters

For environments such as HPC and others where I/O performance is important, 40 Gb starts to increase in popularity. As more 40 Gigabit Ethernet adapters appear and costs fall, plus applications start looking for the next step beyond 10 Gigabit, the RackSwitch G8332 becomes the ideal solution for connecting multiple servers per rack, as shown in Figure 7.

Aggregation switch for east-west traffic
Figure 7. Aggregation switch for east-west traffic


Ordering information

The RackSwitch G8332 includes a standard IBM 1-year hardware warranty, with Next Business Day (NBD), 9x5, Customer Replaceable Unit (CRU) warranty service. IBM also includes one year of software upgrades. IBM ServicePac® offerings are available (in select countries) for hardware warranty upgrade options.

Table 3 lists the part numbers (IBM System x) for ordering modules and additional options for them, including ServicePac offerings.

Table 3. IBM System Networking part numbers and feature codes
DescriptionSystem x part numbers
IBM System Networking RackSwitch G8332 (Rear-to-Front) 8036-BRX
IBM System Networking RackSwitch G8332 (Front-to-Rear)8036-BFX
10 Gb options (Break-put using 40Gb port)
1 m IBM QSFP+ DAC Break Out Cable*49Y7886
3 m IBM QSFP+ DAC Break Out Cable*49Y7887
5 m IBM QSFP+ DAC Break Out Cable*49Y7888
40 Gb options
IBM QSFP+ SR4 Transceiver49Y7884
IBM QSFP+ eSR4 Transceiver00FE325
IBM QSFP+ LR4 Transceiver**00D6222
1 m IBM QSFP+-to-QSFP+ Cable*49Y7890
3 m IBM QSFP+-to-QSFP+ Cable*49Y7891
5 m IBM QSFP+-to-QSFP+ Cable*00D5810
7 m IBM QSFP+-to-QSFP+ Cable*00D5813
10 m IBM QSFP+ MTP Optical Cable90Y3519
30 m IBM QSPF+ MTP Optical Cable90Y3521
1m IBM MTP-4xLC OM3 MMF Breakout Cable‡00FM412
3m IBM MTP-4xLC OM3 MMF Breakout Cable‡00FM413
5m IBM MTP-4xLC OM3 MMF Breakout Cable‡00FM414
Miscellaneous options
IBM System Networking Adjustable 19" 4 Post Rail Kit 00D6185
IBM System Networking Recessed 19" 4 Post Rail Kit (NeXtScale)00CG089
Air Inlet Duct for 483 mm RackSwitch†00D6060
Hot-swappable, Rear-to-Front Fan Assembly Spare00D6071
Hot-swappable, Front-to-Rear Fan Assembly Spare00D6073
Hot-swappable, Rear-to-Front 750W CFF Power Supply Spare00D5858
Hot-swappable, Front-to-Rear 550W CFF Power Supply Spare00D5961
Console Cable Kit Spare90Y9462
ServicePac offerings (available in select countries)
1 year onsite repair 24x7 2 hour response6756FCB
1 year onsite repair 24x7 4 hour response6756FCA
1 year onsite repair 9x5 4 hour response6756FC9
1 year onsite repair 9x5 next business day6756FC8
2 year onsite repair 24x7 2 hour response6756FCG
2 year onsite repair 24x7 4 hour response6756FCF
2 year onsite repair 9x5 4 hour response6756FCD
2 year onsite repair 9x5 next business day6756FCC
3 year onsite repair 24x7 2 hour response67568BO
3 year onsite repair 24x7 4 hour response67568BP
3 year onsite repair 9x5 4 hour response67568BN
3 year onsite repair 9x5 next business day67568BM
5 year onsite repair 24x7 2 hour response67568BS
5 year onsite repair 24x7 4 hour response67568BR
* QSFP+ DAC Break Out cables and QSFP+ to QSFP+ cables are not supported for Power Systems 10 Gigabit Ethernet NICs. These cables are used for switch to switch connectivity only. Also, QSFP+ DAC break out cables are only supported in ports 2 - 25.
** Because of power limitations, only 20 LR4 Transceivers can be supported, which is more than enough for what most clients require. The specific supported configuration is eight ports on the right of the switch, plus up to 12 of 24 ports on the left.
† Requires IBM System Networking Adjustable 19" 4 Post Rail Kit
‡ The QSFP+ eSR4 and SR4* transceivers can leverage these cables which breakout into four OM3 multimode fiber (MMF) cables with LC duplex connectors to support up to 100m (SR4), 300m eSR4). Valid configurations are:
1) MTP-4xLC <=== up to 100m (OM3) with SR4 or 300m with eSR4 ===> 4xLC-MTP
2) MTP-4xLC (connected to 4x SFP+ SR) up to 300m (OM3) using QSFP+ eSR4 only, * This configuration isn't supported with QSFP+ SR4.


With the flexibility of IBM RackSwitch switches, clients can take advantage of the technologies that they require for multiple environments:
  • For 10 Gigabit Ethernet connectivity, clients can use DAC break-out cables, which convert the QSFP+ ports in to four 10 Gigabit Ethernet ports and allows connection for distances up to 5 m.
  • For 40G to 40G connectivity, clients can use the affordable IBM QSFP+ to QSFP+ DAC cables for distances up to 7 meters. For distances up to 100m, the 40GBASE-SR4 QSFP+ transceiver can be used with OM3 multimode fiber with MTP connectors. For distances up to 10km, the 40GBASE-LR QSFP+ transceiver can be used with single mode fiber with LC connectors.

Figure 8 shows the QSFP+ 40 Gigabit Ethernet options.


Figure 8. QSFP+ 40 Gigabit Ethernet options

The module part numbers include the following items:
  • One IBM System Networking RackSwitch G8332
  • Generic Rail Mount Kit (2-post)
  • Mini-USB Console port for serial access

Figure 9 shows a 2-post Rack Mount Kit.

2-post Rack Mount Kit
Figure 9. 2-post Rack Mount Kit

Figure 10 shows the optional adjustable 4-post rail kit.

Optional adjustable 4-post rail kit
Figure 10. Optional adjustable 4-post rail kit

Make sure that you include power cords in your configuration. Table 4 lists the appropriate power cord, depending on country of purchase, for System x.

Table 4. IBM System x part numbers and descriptions for power cords
Part numberDescription
39Y7917Power Cord Europe AC plug 10A/250 V; OPT
39Y7918Power Cord Europe (Denmark) 10A/250 V; OPT
39Y7919Power Cord Europe (Switzerland) 10A/250 V; OPT
39Y7920Power Cord Europe (Israel) 10A/250 V; OPT
39Y7922Power Cord Europe (South Africa) 10A/250 V; OPT
39Y7923Power Cord UK 13A/250 V; OPT
39Y7924Power Cord Australia/New Zealand 10A/250 V; OPT
39Y7925Power Cord Korea 10A/250 V; OPT
39Y7927Power Cord India 10A/250 V; OPT
39Y7928Power Cord China 16A/250 V; OPT
39Y7929Power Cord Brazil 16A/250 V; OPT
39Y7930Power Cord Uruguay/Argentina 16A/250 V; OPT
00CG265Power Cord Taiwan AC plug 10A/250V, 2.8M; OPT
00CG267Power Cord Taiwan AC plug 15A/125V; 2.8M; OPT
46M25922.8 m, 10A/250 V, C13 to NEMA 6-15P (US/Canada)
46M2593Power Cord Japan 2.8M AC plug 12A/125V; OPT
39Y79324.3m, 10A/100-250V, C13 to IEC 320-C14 Rack Power Cable
39Y79371.5m, 10A/100-250V, C13 to IEC 320-C14 Rack Power Cable
39Y79382.8m, 10A/100-250V, C13 to IEC 320-C20 Rack Power Cable


Footnotes

1. Currently, testing has been completed only by using the IBM QSFP+ DAC Break Out Cable and 10 Gb Ethernet connectivity. 40 Gb Ethernet is not currently tested.


Related information

For more information, see the following resources:
  • IBM Offering Information page (to search on announcement letters, sales manuals, or both):
    http://www.ibm.com/common/ssi/index.wss?request_locale=en

    On this page, enter IBM System Networking RackSwitch G8332, select the information type, and then click Search. On the next page, narrow your search results by geography and language.
  • IBM System Networking RackSwitch G8332 product publications:
    http://ibm.com/support/entry/portal/documentation
    • Application Guide
    • Industry-Standard CLI Reference
    • Browser-Based Interface (BBI) Quick Guide
    • Menu-based CLI Command Reference

Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment. publib-b.boulder.ibm.com

Profile

Publish Date
18 February 2014

Last Update
18 September 2014


Rating: Not yet rated


Author(s)

IBM Form Number
TIPS1139