IBM FlashSystem in OLTP Database Environments

IBM Redbooks Solution Guide

Abstract

IBM® FlashSystem ™ family is an enterprise-class flash storage platform that is ideal for delivering extreme performance with low latency to OLTP (On-line Transaction Processing) environments. Transaction processing is typically characterized by a large volume of short transactions that are directly related to business processes. The primary goal of an OLTP system is very fast query processing while maintaining data integrity; its effectiveness is measured by the number of transactions per second that can be achieved. This IBM Rebooks Solution Guide describes how IBM flash storage systems drive extreme performance, cost efficiencies, and enterprise reliability to satisfy the demanding requirements of an OLTP implementation.

Changes in the latest update as of 07/10/14:
-Updated to include the latest FlashSystem product functionality

Special thanks to Ilya Krutov who wrote the first version of this Redbooks Solution Guide.

Contents


IBM® FlashSystem transforms the data center environment and enhances performance and resource consolidation to gain the most from business processes and critical applications. Extreme performance, IBM Microlatency, macro efficiency, and enterprise grade reliability make IBM FlashSystem a powerful and cost effective tool for accelerating Online Transaction Processing (OLTP) systems and gaining competitive advantage. Perhaps more importantly, the extraordinary capabilities and capacity of FlashSystem arrays enable commercial and governmental enterprises to address multiple compute challenges in current 24/7/365 operational environments while at the same time empowering growth and innovation into the future.

Figure 1 illustrates the value of the IBM FlashSystem storage infrastructure.


Figure 1. IBM FlashSystem storage infrastructure value

Did you know?

FlashSystem arrays are some of the highest density solutions on the market, offering dozens of terabytes of usable storage capacity in only a few rack units of space. While providing up to 1.1 million I/O per second (IOPS), they draw as low as 625 watts of power, making them extremely power efficient.

IBM has invested one billion dollars and established worldwide Flash Centers of Competency to help customers architect and implement flash-based solutions. FlashSystem arrays provide industry-leading performance, reliability, and MicroLatency. FlashSystem Enterprise Performance Solutions add the full spectrum of enterprise grade data management and feature rich storage services.

For the latest FlashSystem product details, see the IBM FlashSystem family product page at: http://www.ibm.com/storage/flash.

Business value
OLTP workloads are characterized by small, interactive transactions that generally require subsecond response times. The key performance indicator (KPI) of the transactional system is latency, because users expect to receive the requested product information and to place orders quickly. In OLTP environments, the inability to meet user expectations leads to customer dissatisfaction and revenue loss. IBM FlashSystem addresses these challenges by providing low latency, extreme performance, and efficient transaction management.

For most OLTP systems, the processor, memory, and I/O subsystem in a server are well balanced and are not considered performance bottlenecks. Instead, the major source of performance issues in OLTP environments is typically related to the storage I/O activity. The speed of traditional hard disk drive (HDD)-based storage systems does not match the processing capabilities of the servers. As a result, often a situation occurs where a powerful processor sits idle, waiting for the storage I/O requests to complete, negatively impacting user and business productivity. The negative impact on productivity extends the time to return on investments (ROI) and increases overall total cost of ownership (TCO). Therefore, storage IOPS performance and latency become strategic considerations for business. It is critical to ensure that the response time goals are met and that performance optimization is realized for other system resources (processor and memory).

In general, enterprises might experience the following challenges in OLTP environments:
  • Failure to meet user expectations and service levels because of slow application response time
  • Decreased user and business productivity
  • Application and data availability concerns (slow batch processing, long backup windows, and increased hardware failure rates)
  • Increased storage performance and capacity requirements
  • Scalability constraints because of data center space, power, and cooling limits
  • Increasing TCO
    • Rising data center power and cooling costs
    • Increasing software licensing fees
    • Rising server, network, and storage infrastructure management and support costs
  • Longer lead time to ROI because of inefficient utilization of the existing resources

FlashSystem storage addresses these challenges by:
  • Dramatically boosting the performance of existing applications and lowering cost per IOPS ratio without a need for rearchitecture
  • Increasing user productivity with better response times, improving business efficiency
  • Increasing data availability by using advanced system-level high availability and reliability technologies, reducing the number of solution components, and shortening batch processing and backup times
  • Increasing storage performance and capacity while decreasing power, cooling, and space requirements
  • Reducing TCO
    • Reducing energy costs because of lower power and cooling requirements
    • Reducing the number of systems, devices, and components that are required to build the solution by increasing usage of available resources
    • Reducing software license fees because fewer systems or processors are required
    • Reducing management and support costs because of fewer components to deploy and support
  • Faster ROI through better resource usage

Solution overview

An OLTP solution with IBM FlashSystem storage consists of the following components:
  • Database servers (IBM System x® or IBM Power Systems™) that run data management software such as IBM DB2®, Microsoft SQL Server, or Oracle databases
  • FlashSystem arrays that host the entire data set or subsets of data
  • Storage area network (SAN) that is used to provide connectivity across database servers and storage systems.
    Supported FlashSystem interfaces include Fibre Channel(FC), iSCSI, InfiniBand(IB), and Fibre Channel over Ethernet (FCoE). For the product details of supported platforms and interfaces, see the IBM System Storage® Interoperation Center (SSIC): http://ibm.com/systems/support/storage/ssic.

IBM DB2 for Linux, UNIX, and Windows is the database of choice for robust, enterprise-wide solutions that handle high-volume workloads. It is optimized to deliver industry-leading performance while lowering costs, and IBM servers that run DB2 are proven performance leaders. DB2 uses and optimizes multiple threads automatically, with no change to applications. The unique clustering design of DB2 provides near linear scalability, continuous availability, and simplified management.

IBM System X6 servers featuring exclusive IBM X-Architecture® innovations and Intel® Xeon® Processor E7 v2 families can help to meet business challenges with revolutionary levels of processor and storage performance, memory capacity, scalability, and reliability. In organizations in every industry, big data and analytics workloads deliver the actionable insight needed to drive faster decision making. Systems such as IBM System x3850 x6 can be confidently deployed to run business mission-critical applications, decrease operating costs, and support cloud computing plans.

Ideally suited for computing intensive workloads, IBM Power Systems deliver leadership performance and scalability in their class. An integrated approach to the design, development, and testing of each IBM POWER® server, blade, or compute node ensures the resiliency that is required for today’s IT infrastructure. All Power Systems server models include innovative reliability, availability, and serviceability features that help you avoid unplanned downtime. And, with capacity on demand, hot-node add, and IBM Active Memory™ Expansion, Power Systems enterprise servers ensure that you can keep your most important applications available, even as you add capacity to handle new business demands.

IBM FlashSystem solutions provide multiple options for addressing the low latency / high IOPS requirements of OLTP systems and increasing the effectiveness of customer experience-sensitive computing environments.

Four key differentiators set IBM FlashSystem apart from other flash storage platforms:
  • First, FlashSystem architecture is designed with IBM MicroLatency to speed response times, delivering data reads and writes in the 100 microsecond range.
  • Among the engineering objectives of FlashSystem is a focus on Extreme Performance. In addition to an obsession with low latency, IBM FlashSystem engineers also optimized IOPS and bandwidth.
    The resulting extreme performance ensures that as OLTP workloads increase, FlashSystem continues to scale performance without latency degradation. Whether supporting a single application that needs to handle high numbers of concurrent users or multiple applications with diverse workloads, FlashSystem extreme performance translates into performance scalability and better business results.
  • FlashSystem is optimized to provide Macro Efficiency through compact physical capacity, low energy consumption, and greater utilization of existing resources. The arrays are some of the highest density solutions on the market, offering dozens of terabytes of usable storage capacity in only a few rack units of space. While providing up to 1.1 million IOPS, they draw as low as 625 watts of power, making them extremely power efficient.
  • A key FlashSystem pillar is Enterprise Reliability. The system employs eMLC NAND flash plus two RAID dimensions. IBM’s patented Variable Stripe RAID™ at the flash module level as well as system-wide RAID, result in more data protection levels than are available from competing systems. FlashSystem has no single point of failure, plus its design enables rapid servicing because all hot swappable and redundant components (including flash modules, power supplies, fans, batteries, and canisters) are accessible from the front or back of the system. In addition, software and firmware updates can be completed with the system up and running.

Finally, FlashSystem Enterprise Performance Solutions offer a wide range of advanced storage services such as snapshots, data compression, and replication. And for business or governmental customers with data that requires an extra layer of protection for adherence to internal or regulatory requirements, FlashSystem supports AES 256 hardware-based encryption for data at rest. Figure 2 shows the latest FlashSystem family of products.


Figure 2. IBM FlashSystem Products

Solution architecture

In this solution, FlashSystem arrays are connected to the heterogeneous host platforms by using a Fibre Channel SAN. Host operating systems detect FlashSystem arrays as traditional block-level devices and applications transparently interact with them. FlashSystem supports multipathing to allow redundant storage connections through the SAN, including dual-port FlashSystem interface connections, dual-port host bus adapters (HBAs) on the host systems, and a redundant SAN switched fabric. Figure 3 shows the FlashSystem solution architecture.


Figure 3. FlashSystem solution architecture

A single 42U rack that is filled with FlashSystem arrays can provide up to a petabyte or more of protected main storage delivering millions of IOPS and nearly 200 GBps of throughput.


When database availability is required, you can achieve it by using either of the following ways:
  • Applying database vendor-specific availability solutions, such as log shipping, replication, or database mirroring
  • Using clustered systems such as Veritas Cluster Services, Microsoft Cluster Services, and Red Hat Cluster Services


Usage scenarios

FlashSystem storage within an OLTP environment can be used in the following ways:
  • Main data storage
  • Frequently accessed data storage
  • Software-defined storage with a full suite of storage services

Main data storage

When FlashSystem is used as the main OLTP data storage, entire database structures are placed onto FlashSystem logical volumes, as shown in Figure 4.


Figure 4. FlashSystem storage as main OLTP storage

This approach ensures maximum performance efficiency and server resource usage in heavily-loaded concurrent user access environments with frequent random access to all tables. However, if some tables are used less frequently, or rarely used, this approach might not be the most cost efficient option.

Frequently accessed data storage

Sometimes, putting only part of a database structure, such as the following examples, on high-speed FlashSystem storage can significantly improve the performance, while keeping the storage costs optimized:
  • Log files
  • Temporary table space
  • Frequently accessed tables
  • Table partitions
  • Indexes

Figure 5 shows FlashSystem deployed as frequently accessed data storage.


Figure 5. FlashSystem frequently accessed OLTP storage

This approach is manual and it is sometimes difficult to identify a subset of data that is frequently accessed. For these situations, automated tiered storage solutions might help.

Integration

IBM FlashSystem V840 can virtualize your entire storage ecosystem, bringing valuable storage services such as automated storage tiering to both IBM and non-IBM storage components. To further simplify management, FlashSystem V840 integrates with IBM ® Storage Productivity Center to create a single view of physical and virtual storage resources and their relationships with physical and virtual servers.

Software-defined storage

FlashSystem V840 provides storage virtualization through pooled management of diverse storage environments by using a simple, common interface that remains consistent regardless of storage type. Software-defined storage hides physical changes within the storage infrastructure to help improve availability and streamline the task of provisioning, making dynamic management of storage assets far faster and more efficient than manual processes.

IBM FlashSystem V840 Enterprise Performance Solution merges IBM software-defined storage with the scalable performance of IBM FlashSystem technology to help make your entire storage environment easier to manage while preserving your investments in storage. Software-defined storage services enable you to use the following features across all of your storage:
  • Thin provisioning: Enables dynamic growth; purchase only the storage you need when you need it.
  • Easy Tier flash storage management: Optimized performance at lower overall cost.
  • High availability configurations: Enable near-continuous data availability.
  • Copy Services: Space efficient backups
  • Practice disaster recovery techniques: Validate business continuity plans.
  • Simple GUIs: Allow storage to be quickly deployed and efficiently managed.
The following optional licensed features are offered with the FlashSystem V840 Software:
  • External storage virtualization: Enables FlashSystem V840 to manage capacity in other Fibre Capacity in external storage systems and inherits all the functional richness of FlashSystem V840.
  • Real-time Compression: Helps improve efficiency by compressing data by as much as 80%, enabling storage of up to 5x as much data in the same physical space.
  • Remote Mirroring: Provides storage system-based data replication by using either synchronous or asynchronous data transfers over Fibre Channel communication links.

For more details about IBM FlashSystem V840 Enterprise Performance Solution, refer to the IBM Redbooks Product Guide at: http://www.redbooks.ibm.com/abstracts/tips1158.html.

Automated tiered storage

Automated storage tiering can help to identify the most frequently used sets of data (frequently referred to as hot data). It provides continuous I/O monitoring and dynamic placement onto performance optimized storage (focus on cost per IOPS). Automated storage tiering keeps infrequently used data (known as cold data) on capacity optimized storage (focus on cost per GB).

IBM System Storage Easy Tier is a function that automatically and nondisruptively moves frequently accessed data from hard disk drives to flash storage, placing such data in a faster tier of storage. System Storage Easy Tier eliminates manual intervention when assigning highly active data on volumes to faster responding storage. In this dynamically tiered environment, data movement is seamless to the host application regardless of the storage tier in which the data resides.

IBM FlashSystem V840 integrates the IBM System Storage Easy Tier feature to provide automated storage tiering, dynamically optimizing storage performance as illustrated in Figure 6.


Figure 6. IBM FlashSystem V840 Enterprise Performance Solution and IBM Easy Tier


IBM Tivoli Storage Productivity Center

Further extending the automated tiering scenario, IBM Tivoli Storage Productivity Center can be deployed in an IBM FlashSystem V840 environment to simplify management of a heterogeneous storage infrastructure. IBM Tivoli Storage Productivity Center v5.2.x is a comprehensive solution designed to improve visibility, control, and automation for data and storage infrastructures, including storage systems, devices, and SAN fabrics, all from a single integrated console.

With Tivoli Storage Productivity Center V5.1 and greater, the IBM SmartCloud® Virtual Storage Center Storage(VSC) Analytics Engine license contains everything in the IBM Tivoli Storage Productivity Center license, and also includes advanced analytical functions. The Virtual Storage Center Storage Analytics Engine provides the advanced capabilities of Tivoli Storage Productivity Center, such as storage tier optimization and volume workload distribution. The Virtual Storage Center Storage Analytics Engine license is only available as part of the Virtual Storage Center solution.

The latest version of Tivoli Storage Productivity Center; V5.2.x includes the following key features:
  • Provides comprehensive visibility and helps centralize the management of heterogeneous storage infrastructure from a next-generation, web-based user interface by using role-based administration and single sign-on.
  • Provides cloud configuration and provisioning functions.
  • Delivers storage and device management to provide fast deployment with agent-less device management, while intelligent presets improve provisioning consistency and control.
  • Easily creates and integrates custom reports (based on IBM Cognos®) on capacity and performance.
  • Integrates performance management features end-to-end views, including devices, SAN fabrics and storage systems. The server-centric view of storage infrastructure enables fast troubleshooting.
  • Simplifies data replication for copy management and disaster recovery with advanced replication services (Global Mirror, Metro Mirror, and IBM FlashCopy®)
For Tivoli Storage Productivity Center product details refer to: http://www.ibm.com/software/products/en/tivostorprodcent/ and to the IBM Redbooks publication; Tivoli Storage Productivity Center V5.2 Release Guide, SG24-8204 at: http://www.redbooks.ibm.com/redpieces/abstracts/sg248204.html?Open.


Supported platforms

FlashSystem storage systems support a wide range of operating systems (Windows Server 2008 and 2012, Linux, and AIX), hardware platforms (System x, Power Systems, and x86 servers not from IBM), HBAs, and SAN fabrics. For specific information, see the System Storage Interoperation Center (SSIC): http://ibm.com/systems/support/storage/ssic


Ordering information

For FlashSystem ordering information, see the following IBM Redbooks® Product Guides:
http://www.redbooks.ibm.com/abstracts/tips1158.html


Related information

For more information, see the following documents:
http://www.redbooks.ibm.com/redpieces/abstracts/sg248189.html?Open

    Special Notices

    This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment. publib-b.boulder.ibm.com

    Profile

    Publish Date
    21 March 2013

    Last Update
    14 July 2014


    Rating:
    (based on 4 reviews)


    Author(s)

    IBM Form Number
    TIPS0973