IBM System Networking SAN96B-5

IBM Redbooks Product Guide


Abstract

This IBM® Redbooks® Product Guide describes the IBM System Networking SAN96B-5 switch, a scalable enterprise-class SAN switch for highly virtualized cloud environments. The IBM System Networking SAN96B-5 switch is a high-density, purpose-built, foundational building block for large and growing storage area network (SAN) infrastructures. It is designed to provide highly resilient, scalable, and simplified network infrastructure for storage. By delivering market-leading, Gen 5 Fibre Channel technology and capabilities with 16 Gbps performance, SAN96B-5 meets the demands of growing, dynamic workloads; evolving, virtualized data-centers; and highly virtualized, private and hybrid cloud storage environments.

Contents


This IBM® Redbooks® Product Guide describes the IBM System Networking SAN96B-5 switch, a scalable enterprise-class SAN switch for highly virtualized cloud environments. The IBM® System Networking SAN96B-5 switch is a high-density, purpose-built, foundational building block for large and growing storage area network (SAN) infrastructures. It is designed to provide highly resilient, scalable, and simplified network infrastructure for storage. By delivering market-leading, Gen 5 Fibre Channel technology and capabilities with 16 Gbps performance, SAN96B-5 meets the demands of growing, dynamic workloads; evolving, virtualized data-centers; and highly virtualized, private and hybrid cloud storage environments.

SAN96B-5 provides industry-leading scalability, reliability and performance in a flexible, easy-to-deploy enterprise-class switch that enables greater data center consolidation, operational efficiency and business continuity. In addition to increased throughput, it helps improve bandwidth utilization, security, and network visibility and management through in-flight data compression, encryption, and advanced diagnostics. This is an ideal switch for bandwidth-intensive workloads that require a large number of ports. Figure 1 shows the IBM System Networking SAN96B-5 switch.

Figure 1. IBM System Networking SAN96B-5 switch
Figure 1. IBM System Networking SAN96B-5 switch


Did you know?
  • SAN96B-5 ships with 48 ports enabled; with 96 ports enabled, SAN96B-5 provides an aggregate 1,536 Gbps full-duplex throughput.
  • Fabric OS (FOS) v7.1 or later is included with each SAN96B-5 to provide all functions necessary to operate a base system.
Product highlights

The SAN96B-5 switch unleashes the full potential of high-density server virtualization, cloud architectures, and next-generation storage. Gen 5 Fibre Channel technology is designed to be the purpose-built, data center-proven network infrastructure for storage, delivering outstanding performance, reliability, and simplicity. By treating the network as a strategic part of a highly virtualized environment, organizations can increase optimization and efficiency even as they rapidly scale their environments.

SAN96B-5 has the following highlights:
  • Supports virtualized private and hybrid cloud storage environments and data center consolidation with high scalability in an ultra-dense, Gen 5 Fibre Channel 96-port switch
  • Enables “pay-as-you-grow” flexibility—from 48 to 96 ports—using the Ports on Demand (PoD) feature with speeds up to 16 Gbps
  • Simplifies management through Fabric Vision technology, which helps to reduce operational costs and optimize application performance
  • Provides data center-to-data center security and bandwidth savings with up to eight in-flight encryption and compression ports
  • Helps maximize application uptime and performance while reducing overall operational expenses with ClearLink diagnostic technology (D_Ports)


Architecture and key components

The architecture and components are described next.

Scalability

SAN96B-5 features 96 Fibre Channel ports in a 2U form factor, delivering industry-leading port density and space utilization for data center consolidation. Designed for maximum flexibility, this enterprise-class switch offers “pay-as-you-grow” scalability with the PoD feature. Organizations can quickly, easily, and cost-effectively scale from 48 to 96 ports in 24-port increments, each supporting 2, 4, 8, 10, or 16 Gbps. In addition, flexible, high-speed 16 Gbps and 8 Gbps optics allow organizations to deploy bandwidth on demand to meet growing data center needs. For maximum rack flexibility, two models with different airflow options are available to support the latest hot aisle/cold aisle configurations.

Fabric operating system and management software

Included with each SAN96B-5, the Fabric OS (FOS) v7.1 or later provides all functions necessary to operate a base system. The following features are included in the base FOS and do not require a license. Fabric Vision capabilities, like ClearLink diagnostics, bottleneck detection, Forward Error Correction, and Credit Loss Recovery, are also included in the base FOS.
  • Advanced Web Tools enable graphical user interface (GUI) based administration, configuration, and maintenance of fabric switches and SANs.
  • Advanced Zoning segments a fabric into virtual private SANs to restrict device communication and apply certain policies only to members within the same zone.
  • Virtual Fabrics allow a physical switch to be partitioned into independently managed logical switches, each with its own data, control, and management paths.
  • Full Fabric allows a switch to be connected to another switch. It is required to enable expansion ports (E_Ports).
  • Adaptive Networking service is a set of features that provides users with tools and capabilities for incorporating network policies to ensure optimal behavior in a large SAN. FOS v7.0 supports two types of quality-of-service (QoS) features with the 16 Gbps fabric backbones: ingress rate limiting and session ID (SID)/DID-based prioritization.
  • Server Application Optimization (SAO) enhances overall performance and virtual machine scalability by extending b-type data center fabric technologies to the server infrastructure. SAO enables individual traffic flows to be specifically configured, prioritized and optimized, from end to end, throughout the data center.
  • Enhanced Group Management (EGM) enables additional device-level management functionality for IBM b-type SAN products when added to the element management. It also allows large consolidated operations, such as firmware downloads and configuration uploads and downloads, for groups of devices.

The SAN96B-5 switch is designed to be easily upgradable with a low incremental cost by using its PoD capability of 48, 72 or 96 ports and selected optional features. Advanced capabilities can be enabled with the following optional license activation features:
  • Advanced Performance Monitoring helps identify end-to-end bandwidth usage by host/target pairs and is designed to provide for capacity planning.
  • Fabric Watch constantly monitors mission-critical switch operations for potential faults and automatically alerts administrators to problems before they become costly failures. This feature also includes port-fencing capabilities.
  • Inter-Switch Link (ISL) Trunking enables Fibre Channel packets to be distributed efficiently across multiple ISLs between two IBM b-type SAN fabric switches and directors while preserving in-order delivery. Both b-type SAN devices must have trunking activated. SAN96B-5 adds enhanced ISL Trunking support using 16 Gbps ports and enables Fibre Channel packets to be distributed across up to eight 16 Gbps-capable ISLs for a combined bandwidth of up to 128 Gbps.
  • Extended Fabrics extends SAN fabrics beyond the Fibre Channel standard of 10 km by optimizing internal switch buffers to maintain performance on ISLs connected at extended distances.
  • 24-port Activation provides the SAN96B-5 switch with two additional increments of 24 ports that can be enabled on demand. The SAN96B-5 switch ships with the first 48 ports activated in the base product.
  • Integrated Routing allows any 16 Gbps Fibre Channel port in SAN96B-5 to be configured as an EX_Port supporting Fibre Channel Routing.
  • Integrated 10 Gbps Fibre Channel Activation enables Fibre Channel ports on SAN96B-5 to operate at 10 Gbps; this 10 Gbps activation license must be installed before any port can run at 10 Gbps. The 10 Gbps Fibre Channel support is limited to any eight ports on the switch.
  • Enterprise Bundle offers a convenient set of optional features bundled into one orderable feature number. It includes one license for each of the following features: Fabric Watch, Extended Fabric, Advanced Performance Monitor, Trunking Activation, and Fabric Vision. Requires FOS 7.2 or later to activate all Fabric Vision license features.
  • Fabric Vision license offers a collection of breakthrough features and tools bundled into an optional licensed feature. It includes Flow Vision and Monitoring and Alerting Policy Suite (MAPS) advanced technologies and capabilities. Requires FOS 7.2 or later.

Switches installed with both Advanced Performance Monitoring and Fabric Watch licenses will automatically receive the Fabric Vision capabilities with FOS 7.2, and later, without having the Fabric Vision license installed. Customers who already have either Fabric Watch or Advanced Performance Monitoring installed (but not both) and want Fabric Vision technology capabilities (including MAPS and Flow Vision) can upgrade to Fabric Vision by purchasing and installing the other license.

When the switch is upgraded to FOS 7.2 or later, and the Fabric Vision license is enabled, Fabric Watch configuration and any Fabric Watch related features are no longer supported.

IBM Network Advisor software supports SAN96B-5 Gen 5 Fibre Channel technology by providing end-to-end management of data center fabrics from storage ports on networked storage systems to host bus adapters attached to physical or virtualized servers. IBM Network Advisor v12.0 or later is necessary to support transitions to cloud environments. IBM Network Advisor v12.1 or later is required to support Fabric Vision license capabilities.

Enterprise-class reliability, availability and serviceability

SAN96B-5 leverages proven enterprise-class Gen 5 Fibre Channel technology to deliver outstanding reliability to support non-stop operations for mission-critical workloads. SAN96B-5 uses Fabric Vision technology, which leverages hardware, FOS, and IBM Network Advisor integration to provide advanced functions. It features advanced monitoring, diagnostics, and reliability, availability and serviceability capabilities to minimize downtime, optimize performance, and simplify administration.

Industry-leading performance for growing workloads

SAN96B-5 delivers notable performance for growing and dynamic workloads through a combination of market-leading throughput and bandwidth utilization. With the unpredictability of virtualized workloads and cloud services, throughput becomes critical to ensuring that the network does not become the bottleneck. With 96 ports, SAN96B-5 provides an aggregate 1,536 Gbps full-duplex throughput. Up to eight 16 Gbps-optimized ISLs can be combined together in a 128 Gbps framed-based trunk. In addition, exchange-based Dynamic Path Selection (DPS) optimizes fabric-wide performance and load balancing by automatically routing data to the most efficient, available path in the fabric (see Figure 2). This augments ISL Trunking to provide more effective load balancing in certain configurations. Moreover, the enterprise-class capabilities of this switch yield a higher performance when compared to 10 Gigabit Ethernet (GbE) storage networking alternatives at a similar cost.

Figure 2. Exchange-based dynamic path selection optimizes fabric-wide performance and load balancing by automatically routing data to the most efficient and available path in the fabric.
Figure 2. Exchange-based dynamic path selection optimizes fabric-wide performance and load balancing by automatically routing data to the most efficient and available path in the fabric

Simplified deployment, ease of management, and robust network analytics

SAN96B-5 with Fabric Vision technology, an extension of Gen 5 Fibre Channel, delivers breakthrough technologies that dramatically simplify SAN deployment and management, drive down costs, and offer outstanding visibility and insight across the storage network. Key advantages include the following items:
  • Dramatically reduced costs
    • Automates the deployment of threshold-based rules and policies for proactive monitoring and management, reducing operational costs
    • Pre-tests and validates the SAN infrastructure to accelerate deployments and simplify ongoing support
    • Eliminates the need for expensive third-party monitoring, diagnostics, and test equipment through built-in flow monitoring, flow mirroring, and traffic generator tools
  • Maximum infrastructure uptime
    • Provides a customizable dashboard that displays the overall health of the SAN, helping to pinpoint problems faster and enabling trend analysis
    • Features critical diagnostic and monitoring capabilities, helping to ensure early problem detection and recovery
    • Validates the health, reliability, and performance of the network prior to deployment and for ongoing support, reducing downtime risk
  • Optimized application performance
    • Provides comprehensive visibility into flows in the fabric, including the ability to automatically learn (discover) flows and non-disruptively monitor flow performance
    • Instantly identifies congestion or abnormal levels of latency in the fabric, and identifies exactly which devices and hosts are impacted by the bottleneck
    • Provides a customizable performance dashboard with all critical information in one screen to easily identify hot spots and potential network congestion


Fabric Vision technology

Fabric Vision technology, an extension of Gen 5 Fibre Channel, introduces a breakthrough hardware and software solution that maximizes uptime, simplifies SAN management, and provides increased visibility and insight across the storage network. Offering innovative diagnostic, monitoring, and management capabilities, SAN96B-5 with Fabric Vision technology helps administrators avoid problems, maximize application performance, and reduce operational costs. Fabric Vision technology includes the following items:
  • Flow Vision enables administrators to identify, monitor, and analyze specific application and data flows in order to maximize performance, avoid congestion, and optimize resources. Flow Vision includes the following features:
    • Flow Monitor provides comprehensive visibility into flows in the fabric, including the ability to automatically learn (discover) flows and non-disruptively monitor flow performance. Organizations can monitor all flows from a specific host to multiple targets or LUNs, from multiple hosts to a specific target or LUN, or across a specific ISL. They also can perform LUN-level monitoring of specific frame types to identify resource contention or congestion that is affecting application performance.

      Users can monitor all flows from a specific host to multiple targets or LUNs or from multiple hosts to a specific target or LUN; monitor all flows across a specific ISL; or perform LUN-level monitoring of specific frame types to identify resource contention or congestion that is affecting application performance.

    • Flow Mirror provides the ability to non-disruptively create copies of specific application and data flows or frame types that can be captured for deeper analysis. Flow Mirror is used for in-depth analysis of flows of interest or specific frame types, such as analysis of SCSI Reservation frames, ABTS frames, or flows going to a bottlenecked device.

      Flow Mirror is used for in-depth analysis of flows of interest or specific frame types, such as analysis of SCSI Reservation frames, ABTS frames, or flows going to a bottlenecked device.

    • Flow Generator provides a built-in test traffic generator for pre-testing and validating the SAN infrastructure (including verification of routes and integrity of optics, cables, ports, and ISLs) for robustness before deploying applications. With Flow Generator users can do the following tasks:
      • Configure a Gen 5 Fibre Channel-capable port as a simulated device that can transmit frames at 16 Gbps line rate
      • Emulate a Gen 5 Fibre Channel SAN without actually having any 16 Gbps hosts or targets or SAN testers, and pre-test the entire SAN fabric
  • Monitoring and Alerting Policy Suite (MAPS) simplifies fabric-wide threshold configuration and monitoring. MAPS allows organizations to leverage pre-built rule and policy-based templates. The result is a simple, two-step process for applying thresholds and alerts to ports and switches. Organizations can configure the entire fabric (or multiple fabrics) at one time using common rules and policies, or customize policies for specific ports or switch elements, all through a single dialog. The integrated dashboard displays an overall switch health report, along with details on out-of-policy conditions, to help administrators quickly pinpoint potential issues and easily identify trends and other behaviors occurring on a switch or fabric.
  • ClearLink diagnostic technology (D_Port) ensures optical and signal integrity for Gen 5 Fibre Channel optics and cables, simplifying deployment and support of high-performance fabrics. It leverages ClearLink Diagnostic Port (D_Port) capabilities of Gen 5 Fibre Channel platforms.
  • Bottleneck Detection identifies and alerts administrators to device or ISL congestion as well as abnormal levels of latency in the fabric. This feature works in conjunction with IBM Network Advisor to automatically monitor and detect network congestion and latency in the fabric, providing visualization of bottlenecks in a connectivity map and product tree, and identifying exactly which devices and hosts are affected by a bottlenecked port.
  • Integration into IBM Network Advisor provides customizable health and performance dashboard views to pinpoint problems faster, simplify SAN configuration and management, and reduce operational costs.
  • Critical diagnostic and monitoring capabilities help ensure early problem detection and recovery.
  • Non-intrusive and non-disruptive monitoring on every port provides an end-to-end view of the entire fabric using capabilities integrated into the hardware. This allows sophisticated monitoring without imposing an additional burden on switches with frequent polling activity.
  • Forward Error Correction (FEC) enables recovery from bit errors in ISLs, enhancing transmission reliability and performance.
  • Credit Loss Recovery helps overcome performance degradation and congestion due to buffer credit loss.
  • Real-time bandwidth consumption by hosts or applications on ISLs helps easily identify hot spots and potential network congestion.


A building block for virtualized, private cloud storage

SAN96B-5 provides a critical building block for today’s highly virtualized, private cloud storage environments. It simplifies server virtualization and Virtual Desktop Infrastructure (VDI) management while meeting the high-throughput demands of solid-state disks. SAN96B-5 also supports multitenancy in cloud environments through Virtual Fabrics, QoS, and fabric-based zoning features.

In addition, SAN96B-5 enables secure Metro extension to virtual private or hybrid clouds with 10 Gbps dense wavelength division multiplexing link support, and also in-flight encryption and data compression to optimize bandwidth and minimize the risk of unauthorized access. With four times more in-flight encryption and compression ports than the IBM System Storage® SAN48B-5 switch, SAN96B-5 supports higher data volumes over long distances. This switch also features on-board data security and acceleration, minimizing the need for separate acceleration appliances to support distance extension. Internal fault-tolerant and enterprise-class reliability, availability and serviceability features help minimize downtime to support mission-critical cloud environments.


Specifications

Table 1 lists the IBM System Networking SAN96B-5 switch specifications.

Table 1. IBM System Networking SAN96B-5 switch specifications (part 1 of 2)
ItemDescription
Product number2498-F96 / 2498-N96
Base fabric switchComponents:
Two power distribution unit (PDU) jumpers, fixed rack-mount rail kit, install guide hard copy, EZSwitchSetup, CD-ROM (with manuals), service tools, RJ-45 wrap tools, wrist strap and small form-factor pluggable (SFP) extraction tool (no SFPs included in base switch), two integrated power supplies, and three redundant fan modules with reversible airflow options: non-port-side to port-side (2498-F96) and port-side to non-port-side exhaust (2498-N96)***
Functions:
Advanced Web Tools, Advanced Zoning, Enhanced Group Management, Full Fabric, Virtual Fabrics, Adaptive Networking and Server Application Optimization
Operating system:
Fabric OS (v7.1 or later)
Fibre Channel interfacesAuto-sensing of 2, 4, 8, or 16 Gbps port speeds; 10 Gbps and optionally programmable to fixed port speed diagnostic port (D_Port), E_Port, EX_Port, fabric port (F_Port), mirror port (M_Port); optional port type control
Transceivers
  • 16 Gbps: hot-pluggable SFP+, LC connector; 16 Gbps short-wavelength laser (SWL); 16 Gbps long-wavelength laser (LWL): 16 Gbps extended distance longwave (ELW)
  • 10 Gbps: hot-pluggable, 10 Gbps SFP, LC; 10 Gbps SWL; 10 Gbps LWL
  • 8 Gbps: hot-pluggable SFP+, LC; 8 Gbps SWL; 8 Gbps LWL; 8 Gbps ELW

ItemDescription
Hot-swap componentsPower supplies, fan modules, SFPs
Non-rack supportNon-rack installation is supported; country-specific power cords are required and must be ordered separately
Management softwareHTTP, SNMP v1/v3 (FE MIB, Fibre Channel Management MIB), Secure Shell (SSH) v2; Auditing, Syslog; Advanced Web Tools, Advanced Performance Monitoring, Fabric Watch; IBM Network Advisor v12.0 or later; CLI; SMI-S compliant
Servers supported*
  • IBM System z®
  • IBM Power Systems™, IBM System p®
  • IBM System x®
  • Other Intel processor-based servers with Linux, Microsoft Windows 2008 and Windows 2012
  • Select Sun and HP servers
Operating systems supported*
  • Windows 2008, Windows 2012
  • Red Hat Linux, Red Hat Linux Advanced Server
  • SUSE Linux, SUSE Linux Enterprise Server (SLES)
  • IBM AIX®
  • Other select operating systems
Storage products supported*
  • IBM XIV® Storage System
  • IBM System Storage DS8000® storage servers
  • IBM System Storage SAN Volume Controller (SVC)
  • IBM Storwize® V3700, V5000, and V7000
  • IBM System Storage DS5000 and DS3000
  • Other select storage systems
Fibre Channel switches supportedSystem Storage and IBM TotalStorage b-type and m-type SAN directors, switches and routers; other directors, switches and routers manufactured by Brocade
Fibre optic cableFibre optic cables with LCs are required and available in various lengths in single- and multi-mode formats
Power cordsJumper cables are included for installation; country-specific power cords must be ordered for desktop/standalone installation
WarrantyOne year; customer-replaceable unit (CRU) and onsite; next-business-day response, warranty service upgrades are available
Optional featuresSFPs; fiber optic cables; upgraded power supplies; 24-port Activation, Advanced Performance Monitor, Fabric Watch, Enterprise Bundle**, Extended Fabrics, Integrated Routing, Trunking Activation, Integrated 10 Gbps Fibre Channel Activation, Fabric Vision

* For the most current and complete details, see product details at the IBM System Storage Interoperation Center (SSIC): http://www.ibm.com/systems/support/storage/ssic/interoperability.wss
** The Enterprise Bundle includes one license for each of the following: Fabric Watch, Extended Fabric, Advanced Performance Monitor, Trunking Activation, and Fabric Vision.
*** The 2498-F96 model has the standard airflow (airflow from non-port side to the port side), while the 2498-N96 model has a reversed airflow (airflow from port side to the non-port side). There is no model conversion available to change a model F96 to a model N96.


Physical specifications

Table 2 lists the physical specifications for the IBM System Networking SAN96B-5 switch.

Table 2. Physical specifications
SpecificationDescription
SizeWidth: 42.93 cm (16.90 in.)
Height: 8.67 cm (3.42 in.)
Depth: 60.98 cm (24.01 in.)
Weight16.92 kg (37.3 lb) with two power supply field-replaceable units (FRUs), without transceivers


Operating environment

Table 3 describes the operating environment for the IBM System Networking SAN96B-5 switch.

Table 3. Operating environment
NameDescription
Temperature (operating)0°C – 40°C (32°F –104°F)
Humidity (operating)10% – 85% (non-condensing)
Altitude (operating)Up to 3,000 m (9,842 ft)


Electrical requirements

Table 4 lists the electrical requirements for the IBM System Networking SAN96B-5 switch.

Table 4. Electrical requirements
NameDescription
Input voltage85 – 264 V ac, single phase
Frequency47 – 63 Hz
Power consumption464 Watts with all 96 ports populated with 16 Gbps SWL optics


Related information

For more information, see the following resources:

Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.

Profile

Publish Date
03 December 2013

Last Update
16 January 2014


Rating: Not yet rated


Author(s)

IBM Form Number
TIPS1103