IBM Ethernet Switch B48Y

IBM Redbooks Product Guide

Note: This is publication is now archived. For reference only.

Published 11 November 2010

More options

Rate and comment

Authors: Jon Tate

Abstract

Organizations increasingly need to do more with less, which is driving more virtualization and automation inside the data center. The IBM Ethernet Switch B48Y can simplify connectivity between devices and easily extend a virtual machine’s sphere of mobility by using IronStack technology to create a single logical chassis out of up to eight discrete switches. Supporting up to four 10 GbE ports, the B48Y provides up to twice the uplink bandwidth of typical 1 GbE switches.

Note: This product has been withdrawn from marketing and is no longer available for ordering from IBM.

Contents

The IBM Ethernet Switch B48Y is a high-performance, feature-rich, easy-to-manage 1 GbE solution for network device access. Purpose-built for the data center with a wire-speed, non-blocking architecture, high-availability features including optional redundant and hot-swappable power supplies and fans, and rack-friendly airflow, the B48Y enables organizations to scale easily in virtualized and non-virtualized data center environments.

The switch comes fully featured with Layer 2 and Layer 3 capabilities, giving organizations ultimate flexibility in deployment options. With the optional 4-port 10 GbE interface module, a row of up to eight switches can be unified into a single logical chassis with 384 ports using IronStack technology, greatly simplifying management and increasing virtual machine mobility.

The switch runs a feature-rich and mature IronWare operating system with an industry-standard command-line interface (CLI), minimizing re-training and operational costs. Supporting a full range of RFC and IEEE standards for interoperability, along with compatibility for vendor-specific protocols such as Cisco Discovery Protocol (CDP) and Per-VLAN Rapid Spanning Tree Plus (PVRST+), the B48Y integrates seamlessly into existing network infrastructures.


Figure 1. IBM Ethernet Switch B48Y with optional 4-port 10 GbE SFP+ interface module (front view)



Figure 2. IBM Ethernet Switch B48Y (back view)


Did you know

As server utilization increases due to virtualization on more powerful hardware such as the IBM innovative eX5 architecture, so does the average and burst bandwidth utilization on the switches in the network. To alleviate these performance bottlenecks, the B48Y is capable of supporting four 10 GbE uplinks, double the bandwidth of most standard 1 GbE switches, to enable near 1:1 subscription during periods of heavy traffic flows with extremely low port-to-port latencies. Even with its outstanding performance, the B48Y is highly power efficient, using up to 36% less power than competing switches.

To ease manageability, up to eight switches can be stacked together to create a single logical chassis of 384 ports. All switches in the same stack share a single configuration file, IP address, and interface addressing scheme. This solution can be a cost-effective alternative to a design using an end-of-row, modular chassis. The IBM B48Y is complemented by a full Ethernet and IP product portfolio, allowing an organization to standardize on a network infrastructure running on the proven IronWare operating system.


Part number information

Table 1 lists the orderable part numbers for the B48Y and associated options through System x.

Table 1. IBM part numbers and feature codes for ordering

Description
IBM part number
IBM feature code
IBM Ethernet Switch B48Y
0563-022
0563-HCF
4-port 10 GbE Interface Module (SFP+) (enables stacking)
81Y1455
6687
210 W AC Power Supply (for 1+1 redundancy)
81Y1451
6685
10 GbE 300 m, SFP+, 10GBASE-SR (duplex LC, MMF, 850 nm, OM)
69Y0389
6416

The IBM Ethernet Switch B48Y (0563-022/0563-HCF) ships with the following:
  • One IBM Ethernet Switch B48Y, which includes:
      • Fourty-eight 10/100/1000 MbE RJ-45 ports
      • One open interface module slot for an optional 4-port 10 GbE (SFP+) module
      • One out-of-band 10/100/1000 MbE RJ-45 management port
      • One DB9 male serial console port
      • One hot-swappable quad-fan fan tray
      • One (of two) 210 W AC hot-swappable power supplys
  • One 1.5 m DB9 female-to-DB9 female straight-through serial console cable
  • Rack mount brackets for mounting into an EIA-310D compliant rack
  • IBM Ethernet Switch y-series Installation and User Guide
  • Statement of limited warranty
  • Brocade end user license agreement
  • Safety and regulatory notices
  • CD-ROM with manuals and environmental notices

Note: Qualified SFP+ transceivers are required for each SFP+ port on the optional module and are not included.

Power cords are not included and must be specified at the time of order. The B48Y has a C14 inlet to receive a C13 power plug. Table 2 lists the available power cable options.

Table 2. Power cord options
Description
IBM part number
Line cord – 2.8 m, 10A/230V, C13 to CEE7-VII (Europe)
39Y7917
Line cord – 2.8 m, 10A/250V, C13 to DK2-5a (Denmark)
39Y7918
Line cord – 2.8 m, 10A/250V, C13 to SEV 1011-S24507 (Swiss)
39Y7919
Line cord – 2.8 m, 10A/250V, C13 to SI 32 (Israel)
39Y7920
Line cord – 2.8 m, 10A/250V, C13 to SABS 164 (S. Africa)
39Y7922
Line cord – 2.8 m, 10A/250V, C13 to BS 1363/A (UK)
39Y7923
Line cord – 2.8 m, 10A/250V, C13 to AS/NZ 3112 (Australia/NZ)
39Y7924
Line cord – 2.8 m, 220-240V, C13 to KETI (S. Korea)
39Y7925
Line cord – 2.8 m, 10A/250V, C13 to (2P+Gnd) (India)
39Y7927
Line cord – 2.8 m, 220-240V, C13 to GB 2099.1 (China)
39Y7928
Line cord – 2.8 m, 125V, C13 to NBR 6147 (Brazil)
39Y7929
Line cord – 2.8 m, 10A/250V, C13 to IRAM 2073 (Australia)
39Y7930
Line cord – 4.3 m, 10A/125V, C13 to NEMA 5-15P (US)
39Y7931
Line cord – 4.3 m, 10A/100-250V, C13 to IEC 320-C14
39Y7932
Line cord – 1.5 m, 10A/100-250V, C13 to IEC 320-C14
39Y7937

Each SFP+ optical transceiver, such as the 10GBASE-SR SFP+, requires fiber optic cabling with duplex LC connectors to connect to another optical transceiver. See the “Network cabling requirements” section for additional details.

Each 10/100/1000 MbE RJ-45 port requires copper CAT 5e or higher cabling for connectivity. Table 3 lists the available copper cable options.

Table 3. Ethernet copper cable (RJ-45) options
Description
IBM part number
0.6 m Ethernet cable (blue)
40K5679
1.5 m Ethernet cable (blue)
40K8785
3 m Ethernet cable (blue)
40K5581
10 m Ethernet cable (blue)
40K8927
25 m Ethernet cable (blue)
40K8930

Table 4 lists the orderable part numbers for additional options available through IBM System Networking HVEC channels.

Table 4. IBM part numbers and feature codes for ordering (System Networking HVEC)
Description
IBM part number
IBM feature code
4-port 100/1000 MbE Interface Module (SFP, works as combination ports)
45W5943
1002
10 GbE 10 km, SFP+, 10GBASE-LR (duplex LC, SMF, 1310 nm, OM)
45W4264
2131
10 GbE Direct Attach SFP+, 1 m, Active TwinAx copper cable
45W2398
2711
10 GbE Direct Attach SFP+, 3 m, Active TwinAx copper cable
45W2408
2731
10 GbE Direct Attach SFP+, 5 m, Active TwinAx copper cable
45W3039
2751
1 GbE 550 m SFP, 1000BASE-SX (duplex LC, MMF, 850 nm, OM)
45W2815
2211
1 GbE 10 km SFP, 1000BASE-LX (duplex LC, SMF, 1310 nm, OM)
45W2816
2212
1 GbE 70 km SFP, 1000BASE-LHA (duplex LC, SMF, 1550 nm, OM)
45W2817
2213
100 MbE 2 km SFP, 100BASE-FX (duplex LC, MMF, 1310 nm, OM)
45W2817
2216
100 MbE 40 km SFP, 100BASE-FX-LR (duplex LC, SMF, 1310 nm, OM)
45W7552
2218
Note: OM = optical monitoring supported. SMF = single-mode fiber. MMF = multi-mode fiber. The MMF distance supported depends on the cabling type. Use OM3 50 ìm (2000 MHz*km) MMF to support the farthest distances.

No transceivers are required unless an optional interface module is ordered.

The optional 4-port 10 GbE SFP+ Interface Module (81Y1455) requires 10 GbE SFP+ transceivers for connectivity. While these higher-bandwidth ports are typically used for uplinks to an Aggregation or Core switch, they can also be connected to end devices such as servers. IronStack is supported over two of these 10 Gbps ports. Cee the “IronStack technology” section for additional details.

Note: The default configuration on the switch sets the first two ports on the optional 4-port 10 GbE SFP+ Interface Module as stack ports. To use all ports on the switch as regular Ethernet ports, the global CLI command stack disable can be configured to disable stacking. See the FastIron Configuration Guide for additional configuration details.

10 GbE SFP+ optical –SR (10GBASE-SR) transceivers support distances up to 300 m, ideal for connectivity within a data center. 10 GbE SFP+ optical –LR (10GBASE-LR) transceivers support extended distances up to 10 km.


Figure 3. SFP+/SFP optical transceiver, receives duplex LC connectors

For cost-effective server connectivity within a rack, 10 GbE copper direct-attach SFP+ active TwinAx cables are available. These cables have two 10 GbE SFP+ transceivers attached on either end of the cable and have been tested for compatibility with the Brocade 10Gb CNA, QLogic 10Gb CNA, and Emulex 10Gb Virtual Fabric Adapter for IBM System x. Compatibility with other 10 GbE server adapters and network devices is up to the user to determine.


Figure 4. 10 GbE Direct Attach SFP+ TwinAx Cable


IronStack technology

Table 5 lists several technology options available for the access layer of the network.

Table 5. Technology options available for the access layer of the network
Fixed port switches
  • Pros: low acquisition costs
  • Cons: switches need to be managed individually

Fi xed port, stackable switches
  • Pros: flexible, pay as you go, cost-effective management
  • Cons: performance compared with modular chassis
Modular switch chassis
  • Pros: high performance, port density, availability
  • Cons: initial acquisition cost, price per port

Organizations need a flexible network architecture that can be reconfigured easily as they grow, while keeping management complexity down. Stacking allows multiple, discrete switches to be aggregated into a single logical device, simplifying deployment and ongoing management. Fixed port, stackable switches provide chassis-like operations such as single IP management, cross-switch link aggregation, local switching of data within a stack, and port mirroring at near the economies of fixed port switches.

IronStack technology found on the IBM Ethernet Switch B48Y and matured over several product generations provides such capabilities along with resilient operations including automatic switch failover, hot switch insertion and removal, and automatic detection of shortest path when the network changes.

The optional 4-port 10 GbE SFP+ Interface Module (81Y1455) is required to build a stack. No additional licenses are required. Stacking can be done over one or two of the 10 Gbps ports on the switch, providing 20 Gbps of bandwidth between stack members. The remaining two or three 10 Gbps ports can be used to carry regular data traffic. Stacking is supported in both ring and linear topologies (Figure 5).

Figure03_IronStack (Update #1).gif
Figure 5. IronStack technology found on the B48Y enables a logical chassis of up to 384 ports

Consider using a ring or closed-loop topology to ensure un-interrupted stack operation in the event of a switch or link failure. Within the stack is an Active Controller, which handles stack management and configures all system and interface-level features, and a Standby Controller, which takes over if the Active Controller fails. Feature configuration in a stacked setup similar to a chassis whereby interfaces are identified in a <stack unit #>/<port region>/<port number> manner.

Regular 10GBASE-SR transceivers (69Y0389) and multi-mode fiber cable can be used to connect two stack members over distances of up to 300 meters. This enables splitting a stack across multiple rooms or floors for flexibility. In stacks of more than two switches, the distance between the other stacked units can be up to 100 meters, allowing for a total ringed cable loop of 1,000 meters between eight stacked units, ideal for most stack deployments.

More common within the data center is stacking across a row of servers, which can be done using lower-cost SFP+ TwinAx cables (Table 4) if the switches are less than 5 meters apart. To stay within the cabling distances of these Direct Attach Cables, a cabling method such as that illustrated in Figure 6 can be used.

Figure04_IronStack_Row (Update #1).gif
Figure 6. Using IronStack to unify the network layer across a row of servers into a single logical device

Any standards-based network device can connect to the switch stack. To these network devices, the multiple discrete switches in a stack look like a single switch. Network devices can utilize standards-based 802.3ad Link Aggregation Control Protocol (LACP) or static link aggregation connected across multiple switch stack members to support multi-device hardware redundancy. No special setup is required.

You can also support multi-device hardware redundancy within a rack using IronStack between the two access switches in the same rack. This allows servers to connect to the two access switches in an active-active configuration using standards-based link aggregation protocols, maximizing bandwidth while also simplifying configuration. Figure 7 illustrates this topology.

Figure05_IronStack_Rack (Update #1).gif
Figure 7. Using IronStack within the rack enables multi-device redundancy while increasing bandwidth utilization


Features and benefits

Features and benefits are:
  • Comprehensive standards support: Supporting a wide range of open IEEE and RFC standards, the B48Y is fully interoperable with existing network environments. In addition, the B48Y supports vendor-specific protocols such as Cisco Discovery Protocol (CDP), native support for Per-VLAN Rapid Spanning Tree Plus (PVRST+), and static trunks to EtherChannel for further supportability.
  • IronStack technology: IronStack enables multiple discrete switches to be unified together into a single logical chassis, enabling multi-device hardware redundancy, speeding up infrastructure rollouts, and simplifying management operations. Unifying the access layer can reduce the need for spanning tree protocol, which increases bandwidth utilization and helps flatten the network.
  • Double the uplink bandwidth: Supports up to four 10 Gbps uplinks, double the uplink bandwidth of common 1 Gbps switches. With full-line rate traffic on all ports, 1:1 internal subscription, and up to 176 Gbps of internal forwarding performance, the B48Y ensures that your network devices have the bandwidth that they require.
  • Built for the data center: The B48Y features back-to-front airflow, allowing installation into racks with a hot-cold aisle design without the need for complicated air baffles. All ports are found on the front side of the switch for easy serviceability.
  • Fully featured L2/L3 switch: Dynamic Layer 3 routing, advanced Layer 2 features such as Q-in-Q, protected links, and advanced quality of service (QoS) are all included in the base, eliminating complicated licensing schemes.
  • High availability: Hot-swappable and redundant power supplies and fan trays reduce downtime and allow you to meet your service level agreements by increasing switch availability.
  • Best-in-class power efficiency: At only 112 W, the low power consumption reduces operational costs tied to power and cooling and allows data centers running out of power to deploy more assets.
  • Industry-standard CLI: Minimize re-training and personnel costs while increasing operational efficiencies by utilizing a management interface that the network team is already familiar with.
  • Mature operating system: The feature-rich and time-tested IronWare operating system running on the B48Y and all IBM b-type system networking devices provide a consistent experience across the network.
  • Granular network visibility: Hardware-based sFlow v5 provides packet sampling for analysis, trending, and application of automated policies based on thresholds using industry applications such as Brocade Ironview Network Manager.
  • Cost competitive: The B48Y includes all software capabilities and a 3-year warranty on the base product, simplifying ordering while being priced extremely competitively compared with other vendors.


Specifications

The IBM Ethernet Switch B48Y supports the following when running IronWare R07.2.x and later (code download and documentation can be found on the IBM Systems support site in the Related links section):
  • Form-factor
      • One RU; 44.0 cm (17.3 in.) width x 4.4 cm (1.7 in.) height x 43.5 cm (17.2 in.) depth
      • Rack-mount into 19” EIA-310D compliant rack, Telco rack, or table-top
  • Ports (data)
      • Forty-eight 10/100/1000 MbE RJ-45 ports with auto-MDI/MDIX detection and auto-negotiation
      • One open interface module slot
  • Interface module options
      • 4-port 10 GbE SFP+ interface module (81Y1455)
  • Ports (management, out-of-band)
      • One 10/100/1000 MbE RJ-45 port
      • One DB9 male serial console port
  • Power supplies, fans, and airflow
      • Two 210 W AC hot-swappable power supplies supported for 1+1 redundancy (base switch comes with 1)
      • One hot-swappable fan tray with four 3+1 redundant fans
      • Back-to-front airflow supports data center hot-cold aisle deployments
  • Performance
      • 176 Gbps line speed full-duplex throughput
      • 132 Mpps forwarding capacity
  • Jumbo frames
      • Supported, up to 10,232-byte Ethernet frame sizes
  • Media (transceivers)
      • On 4-port 10 GbE SFP+ module (81Y1455)
          • 10 GbE SFP+ -SR transceivers and Direct Attach copper TwinAx cables available
  • VLANs
      • Up to 4096 maximum VLANs
      • 802.1Q tagging
      • 802.1Q-in-Q tagging
      • Dual-mode VLANs (tagged and untagged traffic across same port)
      • Port-based, protocol-based (AppleTalk, IPv4, IPv6, and IPX), and subnet-based VLANs
      • GARP VLAN Registration Protocol (GVRP) for distributed VLAN configuration
      • VLAN groups
      • Private VLANs
  • Link aggregation
      • Up to eight links per group, 32 link groups per switch
      • Static trunks compatible with Cisco EtherChannel
      • 802.3ad Link Aggregation Control Protocol (LACP) for dynamic trunk groups
      • Cross-device link aggregation for switches in the same stack
  • Quality of service and rate shaping
      • Eight queues per port with strict priority (SP), weighted round robin (WRR), and combined SP/WRR schedulers
      • 802.1p class of service (CoS)
      • IP DSCP/DiffServ (type of service) to priority mapping
      • ACL, port, and static MAC address to priority mapping
      • Layer 2 CoS and Layer 3 DSCP re-marking
      • Fixed and adaptive inbound rate limiting
      • Outbound rate shaping
      • Symmetric flow control supports 802.1x transmission and receipt of 802.1x PAUSE frames
  • Layer 2
      • Up to 32,000 MAC addresses.
      • Per-VLAN Rapid Spanning Tree Protocol Plus (PVST/PVST+/PVRST+) compatibility in default configuration supports smooth implementation into Cisco Layer 2 environments.
      • 802.1d Spanning Tree Protocol (STP).
      • 802.1w Rapid Spanning Tree Protocol (RSTP).
      • 802.1s Multiple Spanning Tree Protocol (MSTP) enables multiple STP instances and forwarding paths for more efficient traffic distribution.
      • Virtual Switch Redundancy Protocol (VSRP) is an alternative to STP and includes sub-second convergence and Layer 2 redundancy based on the VRRPE protocol.
      • Metro Ring Protocol (MRP) v1/v2 enables fast, sub-second convergence in ring topologies.
      • IGMPv1/v2/v3, PIM-SM, and MLDv1/v2 Snooping reduces multicast flooding in a Layer 2 network.
      • Root Guard prevents rogue devices from hijacking STP root.
      • BPDU Guard prevents rogue devices from participating in STP.
      • DHCP Assist helps DHCP servers respond correctly to devices in a different subnet than the DHCP server.
  • Layer 3
      • Up to 16,000 IPv4 routes
      • Routing Information Protocol (RIP) v1/v2
      • Open Shortest Path First (OSPF) v2
      • Internet Group Management Protocol (IGMP) v1/v2/v3
      • Protocol Independent Multicast Dense Mode (PIM-DM) v1/v2 and Sparse Mode (PIM-SM) v2
      • Equal Cost Multi Path (ECMP) for load balancing across up to eight paths
      • Virtual Router Redundancy Protocol (VRRP) and VRRP-E (Enhanced)
      • Policy-based routing (PBR) enables ACLs to be applied to routing decisions
  • Security
      • 802.1x authentication, accounting, port security, and dynamic assignment for ACLs, MAC filtering, and VLANs
      • Up to 1,534 inbound, rule-based (hardware-based) access control list (ACL) entries
      • Standard ACLs based on source IP address
      • Extended ACLs based on source and destination IP addresses or IP protocol information
      • Authorization, authentication, and accounting (AAA) support on RADIUS, TACACS, and TACACS+
      • Multi-device authentication validates and applies security policies based on MAC address information from a RADIUS server
      • AES encryption for SSHv2 and SNMPv3
      • Denial of service protection against Smurf (ICMP) and TCP SYN attacks
      • IP Source Guard permits traffic only after a valid IP address is learned on a port
      • DHCP Snooping prevents man-in-the-middle attacks against untrusted DHCP sources
      • Dynamic ARP inspection prevents man-in-the-middle attacks against invalid ARP bindings
  • Management
      • Industry-standard CLI via Telnet and SSHv2
      • Easy-to-use web-based GUI for device management over HTTP/HTTPS
      • Fault management integration with IBM Systems Director v6.2
      • Network-wide management of all IBM b-type system networking switches using Brocade IronView Network Manager (INM)
      • Simple Network Management Protocol (SNMP) v1/v2/v3
      • DHCP client-based auto-configuration for automatic retrieval of configuration files and firmware images
      • sFlow v5 for hardware-based packet sampling
          • Ties into industry tools such as Brocade INM and InMon products for trend analysis and automated policy application
      • Remote Monitoring (RMON) protocol
      • Link Layer Discovery Protocol (LLDP), LLDP-MED (Media Endpoint Detection), Cisco Discovery Protocol (CDP), and Foundry Discovery Protocol (FDP)
      • Simple Network Time Protocol (SNTP)
      • Firmware image update via TFTP and SCP
  • Monitoring
      • Syslog support of up to six Syslog servers for centralized log collection and analysis.
      • Port mirroring of inbound and outbound traffic.
      • Digital Optical Monitoring monitors power levels and temperatures of optical transceivers and sends an alert if operating out of specification.
      • Temperature monitoring sends alerts if the temperature exceeds a certain threshold and shuts down the device after an extended period of time.
      • Port flap dampening increases network resiliency by limiting the number of port state transitions on an interface.
      • Link Fault Signaling (LFS) detects and reports fault conditions on 10 GbE ports.
      • Uni-Directional Link Detection (UDLD) monitors the link between two devices and brings the port down if the link goes down on either end.

Connectivity options

The B48Y supports a wide variety of industry IEEE and RFC standards and should interoperate with any network device that supports industry standard Ethernet connectivity. This includes servers, other Ethernet switches, iSCSI Storage Area Network (SAN) arrays, Network-Attached Storage (NAS) arrays, and other network devices.

Figure06_B48Y_DC_labeled (Update #1).gif
Figure 8. IBM Ethernet Switch B48Y (front view with port labeling)

Table 6 lists the methods that the B48Y can use to connect to another device.

Table 6. Connectivity options
On the B48Y
On the device to be connected
10/100/1000 MbE RJ-45 port
  • Auto-sensing, auto-negotiating, auto-MDI/MDIX
Connect to any RJ-45 port or TX adapter supporting 10, 100, or 1000 Mbps Ethernet over CAT5 or higher cabling up to 100 meters.
On optional 4-port 10 GbE SFP+ interface module (81Y1455): SFP+ port using 10 GbE SFP+ optical transceiver:
  • 69Y0389: 10GBASE-SR

The above optical transceiver receives a duplex LC connector.
The optic on the device being connected can be another form-factor (that is, XFP transceiver, X2 module, or integrated into NIC), but it must have a matching optical transmission type.

For example, a 10GBASE-SR XFP optical transceiver on the device being connected must match a 10GBASE-SR SFP+ optical transceiver on the B48Y within the proper wavelength and power specifications.

Multi-mode or single-mode fiber as specified required for connectivity. See the "Network cabling requirements" section.
On optional 4-port 10 GbE SFP+ interface module (81Y1455): SFP+ port using 10 GbE Direct Attach SFP+ Active TwinAx cable:
  • 45W2398: 1 meter
  • 45W2408: 3 meter
  • 45W3039: 5 meter

Direct Attach Cables (DACs) are copper cables with two SFP+ transceivers attached to either end of the cable.
It is up to the user to decide whether the interface on the device being connected supports the IBM DAC used with the B48Y.

Below is a list of IBM devices that have been tested for compatibility with the part numbers in the left-hand column. If a product is not listed, it does not necessarily mean that it is incompatible. It simply has not been tested.

IBM server adapters:
  • 42C1820: Brocade 10Gb Converged Network Adapter for IBM System x
  • 49Y4250: Emulex 10Gb Virtual Fabric Adapter for IBM System x
  • 42C1800: QLogic 10 Gb CNA for IBM System x

IBM switches (SFP+ ports on the following):
  • 4003-B04R, 4003-B08R, 4003-B16R: IBM r-series Ethernet switches
  • 4002CY2: IBM y-series Ethernet switches
  • 4002AX2, 0563-012: IBM x-series Ethernet switches
  • 3758-L32: IBM Converged Switch B32
  • 3722-S51, 3722-S52: Cisco Nexus 5000 series


Connectivity to IBM System servers

The B48Y can be deployed as a Top of Rack (ToR) access switch to provide connectivity to servers within a rack. This includes servers from IBM System x, System p, and System i.

IBM System x servers typically come with on-board 1 Gbps Ethernet RJ-45 ports. Typical cabling on the 1 Gbps ports between the switch and the server NIC use Category 5 or higher copper cables with RJ-45 connectors supporting distances of up to 100 meters.

The 10 Gbps ports on the optional 4-port 10 GbE Interface Module (81Y1455) are typically used for uplinks to the rest of the Ethernet/IP network. These ports can be connected to servers as needed. 10 Gbps Ethernet connectivity is not standard in most System x servers, and a 10 GbE NIC or 10 GbE CNA must be installed. Only the IBM System x3850/3950 X5 series (except for the ARx model) come standard with a customized Emulex Virtual Fabric Adapter. Table 7 lists the 10 GbE adapters that are available for IBM System x servers.

Table 7. 10 Gbps adapters for IBM System x servers
System x rack and tower server adapter modelsx3200 M3x3250 M3x3400 M2x3400 M3x3500 M2x3500 M3x3550 M2x3550 M3x3620 M3x3650 M2x3650 M3x3850/3950 M2
    x3850/3950 X5
Brocade 10Gb CNA (42C1820)NNYNYNYYYYNYYYYYY
QLogic 10Gb CNA (42C1800)YYYNYNYYYYNYYYYYY
Emulex Virtual Fabric Adapter (49Y4250)YYYYYYYYYYYYYYYYY
NetXtreme II 10 GigE Express Fiber SR Adapter (42C1790)NNYYYYYYYYYYYYYYY

The Brocade 10Gb CNA [42C1820], QLogic 10Gb CNA [42C1800], and Emulex Virtual Fabric Adapter [49Y4250] each have two SFP+ ports and can be connected to the B48Y using:
  • The adapter’s supported 10GBASE-SR SFP+ transceiver connected to the B48Y 10GBASE-SR SFP+ transceiver [69Y0389] over multi-mode fiber cable
  • The Direct Attach SFP+ Active TwinAx copper cables listed in Table 4

The NetXtreme II 10 GigE Express Fiber SR Adapter [42C1790] has a single integrated 10GBASE-SR optic. This port can be connected to the B48Y 10GBASE-SR SFP+ transceiver [69Y0389] over multi-mode fiber cable.


Connectivity to IBM BladeCenter

IBM BladeCenter Ethernet Switch Modules are available with 1 Gbps and 10 Gbps Ethernet uplinks. The B48Y can be deployed as an aggregation switch to provide connectivity between down-stream 1 Gbps Ethernet Switch Module,s but would not typically be used to aggregate connections from 10 Gbps Ethernet Switch Modules.

Table 8 lists the various IBM BladeCenter Ethernet Switch Modules that can be used to connect to the B48Y.

Table 8. IBM BladeCenter Ethernet Switch Modules (ESM)
BladeCenter Ethernet Switch Module modelsPart #Ext. 1 GbE portsExt. 10 GbE ports
    BC H
    BC HT
    BC E
    BC T
    BC S
BNT Virtual Fabric 10Gb ESM
46C7191
10 dual-speed SFP+
Y
Y
N
N
N
Cisco Nexus 4001I ESM
46M6071
6 dual-speed SFP+
Y
Y
N
N
N
Cisco Catalyst 3110X ESM
41Y8522
0
1 X21
Y
Y
Y
Y
N
10Gb Ethernet Pass-Thru Module
46M6181
0
14 SFP+
Y
Y
N
N
N
BNT 1/10Gb Uplink ESM
44W4404
6 RJ-45
3 SFP+
Y
Y
Y
Y
Y
BNT Layer 2/3 Copper ESM
32R1860
6 RJ-45
0
Y
Y
Y
Y
Y
BNT Layer 2/3 Fiber ESM
32R1861
6 SFP
0
Y
Y
Y
Y
Y
BNT Layer 2-7 Gigabit ESM
32R1859
4 RJ-45
0
Y
Y
Y
Y
Y
Cisco Catalyst 3012 ESM
43W4395
4 RJ-45
0
Y
Y
Y
Y
Y
Cisco Catalyst 3110G ESM
41Y8523
4 RJ-45
0
Y
Y
Y
Y
N
Server Connectivity Module
39Y9324
6 RJ-45
0
Y
Y
Y
Y
Y
Intelligent Copper Pass-Thru Module
44W4483
14 RJ-45
0
Y
Y
Y
Y
Y
Note 1: X2 modules have duplex SC connectors. An SC-to-LC fiber cable is required to connect to a SFP+ transceiver with duplex LC connectors.

For Ethernet Switch Modules with SFP+ or X2 ports see their respective IBM Redbooks at-a-glance guides to locate the transceivers supported. Review Table 6 for guidance on how to connect the Ethernet Switch Module to the B48Y.

For optimal BladeCenter performance, the Intelligent Copper Pass-Thru Module (44W4483) can be used to directly connect the internal blade server NIC mezzanine cards to the B48Y, providing maximum, non-oversubscribed performance while eliminating a hop in the network and reducing latency.


Popular configurations

This section illustrates how the IBM Ethernet Switch B48Y can be used in various configurations.

Top of Rack switch

An access or edge switch is a switch that connects directly to the end devices requiring network connectivity, most commonly servers and storage arrays in the data center. A switch that is 1 or 2 RU in height that acts as the access switch for devices within a rack is commonly called a Top of Rack (ToR) switch.

The B48Y is ideal for deployment as an all-purpose Top of Rack switch in demanding network environments due to its high performance, resilient hardware characteristics, easy manageability, and comprehensive network features support. The B48Y can also be used to support NAS or iSCSI SAN storage in a shared or dedicated network environment (Figure 9).

Figure07_B48Y_Top_Of_Rack (Update #1).gif
Figure 9. IBM Ethernet Switch B48Y as Top of Rack switch with server and storage connections

Table 9 outlines the components used in Figure 9.

Table 9. Components used in reference to Figure 9
Diagram reference
Part number/ machine type
Description
Quantity
1
0563-022
IBM Ethernet Switch B48Y, with optional 4-port 10 GbE Interface Module (SFP+) (FC 6687).
2
2
Varies
IBM System x server with 1 GbE NICs.
Other vendor servers can be used.
Twenty 1 RU
Four 2 RU
3
Varies
NAS array.
Varies
4
Varies
Existing network infrastructure that can include IBM Systems Networking devices and other standards-compliant Ethernet/IP switches and routers.
-
5
0563-022
IBM Ethernet Switch B48Y, with optional 4-port 10 GbE Interface Module (SFP+) (FC 6687).
2
6
Varies
iSCSI SAN array.
Varies

Solution details:
  • Thirty-four rack units
  • 1:1 subscription ratio within rack
  • Up to 1.2:1 subscription ratio to rest of network
  • Connectivity to NAS and/or iSCSI SAN for mid-range scalable storage

In the solution shown in Figure 9 we have two IBM Ethernet Switch B48Ys providing connectivity to all devices within a single rack. The B48Ys can be in a stacked or non-stacked configuration:
  • In a stacked configuration, two of the four 10 GbE ports are used for stacking to provide a high-bandwidth backplane between the switches. Stacking simplifies manageability and allows servers to utilize Active-Active NIC teaming with standards-based link aggregation to each of the two switches in the rack. The remaining four 10 GbE ports in the dual-stacked switches connect to the rest of the network, providing 40 Gbps of bandwidth to the rest of the network. See Figure 7 for an example.
  • In a non-stacked configuration, up to four 10 GbE uplinks per switch can be used to connect to the rest of the network, allowing organizations to deploy highly utilized networks while avoiding congestion during peak usage.


The B48Y can also be used to connect to NAS arrays for scale-out file sharing and also to a separate iSCSI SAN for dedicated storage access as bandwidth and security needs require.


Network device connections

Figure 10 shows an example of the type of options that can be used to connect devices to the IBM Ethernet Switch B48Y.

Figure08_B48Y_Device_Connections (Update #1).gif

Figure 10. IBM Ethernet Switch B48Y device connections


Table 10 outlines the components shown in Figure 10.

Table 10. Components shown in Figure 10

Diagram reference
Part number/ machine type
Description
Quantity
1
0563-022
IBM Ethernet Switch B48Y
2
2
Varies
IBM x3850/x3960 X5 (except ARx model)
  • Comes with 2-port 10 GbE (SFP+) Emulex Virtual Fabric Adapter.
  • Connections to 10 GbE ports require optional 4-port 10 GbE Interface Module (81Y1455).
  • Use a single Direct Attach Cable (TwinAx) to connect to B48Y, available in 1, 3, or 5 meters. (See Table 4.)
-
3
Varies
NAS or iSCSI storage array.
Varies
4
41Y8522
Cisco Catalyst 3012 Ethernet Switch Module.
Connect to 1 GbE RJ-45 ports on controller using Category 5 or higher copper cabling.
2
5
Varies
IBM System x server.
Connect to on-board 1 GbE RJ-45 ports using Category 5 or higher copper cabling.
-
6
Varies
Other servers with 1 GbE NIC.
Connect to on-board 1 GbE RJ-45 ports using Category 5 or higher copper cabling.
-


Interoperating in existing network environments

The IBM Ethernet Switch B48Y interoperates seamlessly in existing network environments utilizing standards-based protocols. Figure 11 shows the B48Y used as an access and aggregation switch.

IBM Ethernet Switch B48Y connecting to the network infrastructure
Figure 11. IBM Ethernet Switch B48Y connecting to the network infrastructure

Table 11 outlines the components used in Figure 11.

Table 11. The components referenced in Figure 11
Diagram reference
Part number/ machine type
Description
Quantity
1
0563-022
IBM Ethernet Switch B48Y
1
2
4003-M16
IBM Ethernet Switch B16M
  • On B16M, use FC1711 (4-port 10 GbE XFP module) and FC2110 (10GBASE-SR XFP).
  • B48Y requires 81Y1455 (optional 4-port 10 GbE Interface Module (SFP+)) with 69Y0389 (10GBASE-SR SFP+).
  • Connect over multi-mode fiber.
1
3
N/A
Cisco Catalyst 6509-E
  • On 6509, use 8-port 10 GbE Fiber module with X2-10GB-SR (10GBASE-SR X2 module).
  • B48Y requires 81Y1455 (optional 4-port 10 GbE Interface Module (SFP+)) with 69Y0389 (10GBASE-SR SFP+).
  • Connect over multi-mode fiber. (Note that the X2 module has a SC connector, while the SFP+ has an LC connector.)
1

The IBM Ethernet Switch B48Y leverages the proven IronWare operating system with an industry-standard CLI so network administrators can utilize their existing skill sets.


Power supplies and fans

The IBM Ethernet Switch B48Y comes with one 210 W AC power supply and supports a second power supply (81Y1451) for 1+1 redundancy. The power supplies are hot-swappable, auto-sensing, auto-switching, and load sharing, with a universal input (90 VAC to 264 VAC). The power supply has an IEC-C14 inlet that connects to a C13 power connector. Power cords are not included with the switch and must be provided separately.

The IBM Ethernet Switch B48Y also comes with a single fan tray consisting of four variable-speed fans. If one fan within the fan tray fails it does not affect the operation of the other fans in the unit. The fan tray is hot-swappable, increasing device availability and serviceability.


Figure 12. IBM Ethernet Switch B48Y rear view (fans and power supply)

The airflow of the switch is non-port (back) to port (front) side. Air is drawn into the system from the fan tray in the back and expelled out vents on the front of the device. The cooling characteristics of the system allow it to be deployed in server racks featuring hot-cold aisle containment without the need of special baffling.



Figure 13. IBM Ethernet Switch B48Y air flow (back to front)


Installation

The IBM Ethernet Switch B48Y comes with two rack mount brackets affixed to each side for installation into an EIA-310D compliant rack. Screws for mounting the B48Y to the rack are not included with the switch. Allow at least 3 inches of space at the front, back, and sides of the device for proper cabling and cooling.

Figure12_B48Y_Rack.gif
Figure 14. IBM Ethernet Switch B48Y rack mount kit


Network cabling requirements

The network cables required for the IBM Ethernet Switch B48Y are 1000BASE-T (built-in 10/100/1000 MbE RJ-45 ports) (up to 100 m over UTP Category 5e or higher cabling, RJ-45 connector).

For the optional 4-port 10 GbE Interface Module (SFP+) [81Y1455]:
  • 10GBASE-SR SFP+ transceiver [69Y0389]
      • 840 to 860 nm wavelength using multi-mode fiber cable, LC duplex connector. Operating distance supported depends on multi-mode fiber cable used (Table 12).
  • DB9 male serial console port – DB9 female-to-DB9 female straight-through serial console cable that comes with the switch

Table 12. Multi-mode fiber cable distances supported
Multi-mode fiber cable
Distance
Loss
62.5 ìm (160 MHz*km)
26 m
1.6 dBm
62.5 ìm (200 MHz*km)
33 m
1.6 dBm
50 ìm (400 MHz*km)
66 m
1.7 dBm
50 ìm (500 MHz*km)
82 m
1.8 dBm
50 ìm (2000 MHz*km)
300 m
2.6 dBm


Related links

See the following websites for related information:

Others who read this publication also read



Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.

Follow IBM Redbooks

Follow IBM Redbooks