Universal Messaging with IBM MQ V8

IBM Redbooks Solution Guide

Note: This is publication is now archived. For reference only.

Published 01 July 2014, updated 12 December 2014

More options

Rate and comment

Authors: Rufus Credle

Abstract

By implementing messaging technologies, businesses can use a consistent approach to connectivity, decoupling the business application from the complex tasks of handling failures, error recovery, transaction integrity, security, and scalability. IBM® MQ® delivers universal messaging. As explained in this IBM Redbooks® Solution Guide, IBM MQ provides simple, rapid, reliable, and secure transport of messages and data offering flexible connectivity for applications, web services, and other programming environments. IBM MQ is the market-leading, message-oriented middleware product. It delivers a reliable and proven universal messaging backbone for almost 10,000 organizations of different sizes, spanning many industries around the world.

Contents

By implementing messaging technologies, businesses can use a consistent approach to connectivity, decoupling the business application from the complex tasks of handling failures, error recovery, transaction integrity, security, and scalability. IBM® MQ® provides the universal messaging backbone for service-oriented architecture (SOA) connectivity (Figure 1). It connects virtually any commercial IT system, with support for more than 80 platforms.


Figure 1. Overview of the IBM MQ solution

IBM MQ includes a choice of APIs and supports the Java™ Message Service (JMS) API. IBM MQ is the market-leading messaging integration middleware product. Originally introduced in 1993 (under the IBM MQSeries® name), IBM MQ provides an available, reliable, scalable, secure, and high-performance transport mechanism to address businesses connectivity requirements.


Did you know?

IBM MQ has the following credentials and industry recognition:

  • It is the most widely deployed messaging backbone, with over 10,000 customers using the IBM messaging backbone:
    • Over 90% of Fortune 50 companies and of Fortune 10 companies use the IBM messaging backbone.
    • Over 80% of Global 25 companies and 70% of Global 10 companies use the IBM messaging backbone.

  • It is entrusted with tens of billions of messages each day:
    • A government client sends 675 million messages per day.
    • A banking client handles over 213 million messages per day on IBM z/OS® alone.

  • It is relied upon as the mission-critical backbone:
    • A financial markets client handles USD 1 trillion worth of traffic per day on one IBM MQ network.
    • A banking client sends USD 7 - USD 35 trillion worth of traffic per day on just one IBM MQ based Society for Worldwide Interbank Financial Telecommunication (SWIFT) gateway.


Business value

Business environments are constantly changing. Applications that were written 20 years ago must exchange data with applications that were written last week. With IBM MQ, businesses can respond quickly to the constantly changing environment by reducing the need to change existing applications to connect them to new ones. Businesses can use and extend the life of existing applications by enabling wider sharing of business data. They can also manage complexity through end-to-end visibility of data movement across all systems and applications.

By using IBM MQ, companies can create a robust IT infrastructure that supports SOAs, providing businesses with the reliability and flexibility that they require. Application developers can concentrate on solving business problems and adding value to the business, while relying on the messaging backbone that is provided by IBM MQ to handle the technical aspects of application connectivity and integration.

Most businesses today require services, such as managing payments or querying account information, to be made available through a range of channels. The core data can be held in a database on a mainframe, but a user of a browser requires a front-end web application server to interact with that database. As new delivery channels are created, such as smartphone applications, easy ways are needed for that new mechanism to interact. The smartphone application must communicate with the same applications and database without changes. IBM MQ Telemetry extends IBM MQ through clients by using IBM MQ Telemetry Transport (MQTT), which is a standard lightweight connectivity protocol. It enables connected physical devices and mobile devices, which might be restricted by bandwidth and power consumption, to exchange information with IBM MQ.


Solution overview

IBM MQ has always been in the business of connecting systems and applications together, regardless of platform or environment. Now, the range of environments in which IBM MQ can or must work has changed. Newer platforms, environments, requirements for qualities of service, and newer messaging patterns are available. Security has become more important as systems are made accessible to more users across an enterprise and beyond it. Performance and scalability requirements have increased. Regulators and auditors have imposed more controls on what can or must be done. Systems, which need access to enterprise data, have become both more powerful (faster and more processors) and much less powerful (sensors, tablets, and mobile phones). Therefore, IBM MQ has evolved.

IBM MQ includes the following features (illustrated in Figure 1) that enable business to connect all their assets in all possible ways:
  • With IBM MQ Advanced Message Security (AMS), customers can protect the security of their messages from application to application, without needing to change the application code itself.
  • IBM MQ Telemetry is a feature of IBM MQ that extends the universal messaging backbone with the MQTT protocol to a wide range of remote sensors, actuators, and telemetry devices. MQTT is a messaging protocol that is lightweight enough to be supported by the smallest devices, yet robust enough to ensure that important messages get to their destinations every time. With the MQTT protocol, such devices as smart energy meters, cars, trains, satellite receivers, and personal healthcare devices can communicate with each other and with other systems or applications.
  • With IBM MQ Managed File Transfer (MQMFT), customers can move their business data that is stored in files over the IBM MQ infrastructure, improving the reliability, security, and management of their file transfers.
  • Enterprise messaging is still the core function of IBM MQ. While IBM MQ is adding new interfaces, protocols, and environments, a large amount of IBM MQ workload continues to run in mainframe-based data centers. Efficient usage of, and integration with, the capabilities of the IBM System z® hardware and operating systems is critical. IBM MQ V8.0 for z/OS improves the performance, scalability, and functionality that can be achieved when running in a sysplex.

IBM MQ V8.0 has several new features. Features that are introduced in the new release are the result of client requirements. The following features have received much positive customer feedback:
  • Enhanced security
    Enhanced security with a new authentication capability that uses either an identity that is defined to the operating system instance in which the queue manager is running, or retrieves an identity from an external repository. This can be used in place of the existing IBM MQ authentication mechanism. This new authentication feature helps reduce the administration of defining and removing users, especially across multiple systems. An additional security enhancement in IBM MQ V8 allows CHLAUTH definitions to use DNS (domain) host names instead of being forced to use IP addresses. Usage of IP addresses continues to be preferred practice, but users now have the choice of either approach, which can simplify matters when machines move IP addresses. This enhancement enables the removal of user exits where clients have used them to deliver this capability in the past. Another security enhancement in IBM MQ V8 is a more flexible security configuration of a queue manager. A queue manager can now have multiple certificates, each signed by a different certificate authority. This enables a single queue manager to communicate with different partners, each of whom uses a different certificate authority, by controlling channel definitions.
  • Increased functionality in IBM MQ Telemetry
    IBM MQ Telemetry enables IBM MQ servers to connect to MQTT clients. Before IBM MQ V8, this function was available in two different licenses: Standard or Advanced. With Standard licensing, IBM MQ servers can connect to MQTT clients. With Advanced licensing, there was entitlement to deploy and connect to the IBM MQ Telemetry daemon for devices, which allowed buffering for MQTT messages and allowed the daemon to act as a client concentrator. With IBM MQ V8, all of these functions are available with the regular IBM MQ Telemetry license, offering improved flexibility of deployment options at no additional license cost. IBM MQ Advanced still includes licensing for IBM MQ Telemetry deployment for all IBM MQ servers with any number of IBM MQ Advanced licenses that are purchased.
  • Improved file transfer using IBM Sterling Connect:Direct
    IBM MQ has included Managed File Transfer capabilities for a number of years, allowing users to use the IBM MQ environment to send and receive data in and out of files. These capabilities enable file-to-file transfer, message-to-file transfer, and file-to-message transfer, with configuration and monitoring through IBM MQ Explorer. As announced in Software Announcement LP14-0047, dated March 11, 2014, the Managed File Transfer capabilities that are provided with IBM MQ have been enhanced through the additional entitlement to IBM Sterling Connect:Direct for peer-to-peer file transfer, and entitlement to IBM Sterling Control Center, which provides event-based monitoring and management for file transfers supporting both IBM MQ Managed File Transfer and IBM Sterling Connect:Direct. This increases the functions and capabilities that are available to users of IBM MQ wanting to reliably and securely move files or file contents. As part of this update and enhancement to previous releases, the IBM MQ Managed File Transfer Agent, now available as part of the WebSphere Managed File Transfer Managed Endpoint entitlement, is licensed per installation instead of per client device.
  • Enhancements to scalability
    IBM MQ V8 has made enhancements to how publish/subscribe can be deployed when using clustering. In this release, it is possible to assign roles to queue managers in a cluster. A subset of those queue managers in a cluster can be configured for more efficient routing of messages. This configuration assists in the propagation of publications, which can then help publish/subscribe configurations scale more effectively. Further enhancements in scalability in IBM MQ V8 include improved exploitation of the larger symmetric multiprocessing (SMP) machines that are available, with increased performance available for real-world deployments rather than performance being tuned for benchmarks.
  • Standards and currency
    With the availability of IBM MQ V8, all IBM MQ queue manager platforms now support 64-bit server implementations, subject to appropriate hardware support and operating system support. In this release, support is added for JMS 2.0, including new messaging features and API changes. This support enables businesses to move up to this new level of JMS as required. There are also enhancements in .NET support, with the IBM MQ .NET classes now supporting SSL without additional component installation required. Microsoft™ Windows™ native certificate stores are also supported by .NET applications. Additionally, the Windows Communication Foundation interface is extended to support non-SOAP, non-JMS messages. These changes all contribute to simplifying the usage of IBM MQ in .NET configurations.

Data is transferred between applications in messages (Figure 2).

Basic IBM MQ components to connect applications
Figure 2. Basic IBM MQ components to connect applications

A message is a container that consists of three parts:
  • An IBM MQ Message Descriptor (MQMD) identifies the message and contains additional control information. Examples of additional information are the type of message and the priority that is assigned to the message by the sending application.
  • Message properties are an optional set of user-definable elements that describe the message without being part of the payload. Applications that receive messages can choose whether to inspect these properties.
  • Message data contains the application data. The structure of the data is defined by the application programs that use it, and IBM MQ is unconcerned with its format or content.

Twelve types of objects can be configured in IBM MQ, in addition to many other resources (for example, security access controls or subscriptions) that are managed by IBM MQ. The most important objects from this set are the queue manager itself, queues, channels, and topics.

The nodes within an IBM MQ message queuing infrastructure are called queue managers. The queue manager is responsible for accepting and delivering messages. Multiple queue managers can run on a single physical server or on a wide network of servers across many various hardware and operating system platforms. Each queue manager provides facilities for reliable messaging.

The queue manager maintains queues of all messages that are waiting to be processed or routed. Queue managers are tolerant of failures and maintain the integrity of business-critical data that flows through the message queuing infrastructure. If a failure occurs (such as with the queue manager, machines, or power loss), the messages can be recovered.

The queue managers, within the infrastructure, are connected by logical channels over a communications network. Messages automatically flow across these channels from the initial producer of a message to the eventual usage of that message, which is based on the configuration of the queue managers in the infrastructure. Changes can be made to the configuration of queues and channels transparently for the applications. For example, a receiving application can be moved to a new machine, and a route to that machine can be defined without needing any changes to sending applications.


Solution architecture

The architecture for a solution that is based on IBM MQ depends on the business requirements and applications that it supports. One flexible approach is to join many queue managers in a dynamic logical network that is called a queue manager cluster. A cluster allows multiple instances of the same service to be hosted through multiple queue managers. Applications that request a particular service can connect to any queue manager within the queue manager cluster. When applications make requests for the service, the queue manager to which they are connected uses a workload balancing algorithm to spread these requests across all available queue managers that host an instance of that service.

Figure 3 illustrates workload balancing in an IBM MQ cluster.

Workload balancing in an IBM MQ cluster
Figure 3. Workload balancing in an IBM MQ cluster

The message that is sent from Application A can be processed by any instance of Application Z. The cluster chooses one of the routes based on various criteria, including knowledge of which route it previously chose and which routes are available.


Usage scenarios

The following scenarios show three ways in which IBM MQ is used to solve real-world problems:
  • Retail kiosk

    A retailer wanted to respond rapidly to online customer needs and have a consistent view of data that is shown at more than 25,000 self-service kiosks. Data changes rapidly and must be distributed rapidly to these kiosks.

    The network of kiosks was connected by a messaging service to the central databases, and changes are now pushed out rapidly. More than 14,000 transactions are processed each second.

  • Faster bank payments

    Government regulations that limit payment clearing times require faster payment processes through near-instant money transfers between banks.

    A high-performance payment solution was developed by using a messaging backbone to route message traffic between systems. Many existing components of the older payment system were adapted to work with the new messaging layer, preserving investment in these programs. The government requirement can clear payments in under two hours. However, this solution is technically capable of completing its processing in seconds. It handles millions of transactions every day.

  • Airport information

    At an airport, rapid changes in government regulations are coupled with increasing passenger volumes. This airport needed a flexible IT infrastructure that is populated with seamlessly integrated systems. The airport was supporting daily business operations with an inflexible heterogeneous infrastructure.

    A messaging-based solution was implemented that establishes a shared integration layer and enables the seamless flow of multiple data types between the client’s heterogeneous systems. For example, airport staff can now easily access important information, such as flight schedule changes from multiple systems. The airport can also flexibly integrate new systems to accommodate evolving government regulations.


Integration

IBM MQ is at the core of the SOA message backbone, and any-to-any connectivity is its core value. IBM MQ does not require the use of any other products. However, two products are frequently used alongside IBM MQ:
  • IBM Integration Bus
  • IBM WebSphere Application Server

Integration with IBM Integration Bus

Enterprise systems consist of many logical endpoints, such as standard applications, services, packaged applications, web applications, devices, appliances, and custom-built software. These endpoints use a set of inputs and outputs, such as the following ones:
  • Connection protocols, such as IBM MQ, TCP/IP, database, HTTP, files, FTP, SMTP, and POP3
  • Data formats, such as C or COBOL structures, XML, and industry-specific (SWIFT, EDI, HL7) and user-defined formats

An enterprise service bus (ESB) can be imagined as something that runs at the center of this wide range of connections, applications, and protocols. It performs such tasks as routing and data transformation so that any endpoint can communicate with any other endpoint. Traffic passes through an ESB without requiring a complex and unmaintainable mesh of individual point-to-point connections.

IBM Integration Bus is an ESB product that connects these endpoints in meaningful ways to simplify application and device integration. This connection requires IBM MQ to also be installed on systems where it runs. This requirement is partly because applications that are enabled for IBM MQ are an important class of connectivity that it must support. The requirement is also because it uses queue manager-provided services, such as transaction coordination, publish/subscribe, and the reliable storage of messages.

As an example of how IBM Integration Bus makes integration simple, consider a typical retail enterprise. Such an enterprise has many different endpoints, both inside and outside the organization, with data in multiple formats and protocols, such as TLOG, files, and JSON/HTTP. As new capabilities need to blend in (such as support for mobile applications and analytics), the flexibility of IBM Integration Bus makes it simple to add these functions without disrupting existing services.

Integration with WebSphere Application Server

The WebSphere Application Server product hosts and runs applications. At its core is a Java Platform, Enterprise Edition environment, which implements and supports a range of open standards. One such standard that Java Platform, Enterprise Edition applications often use is the Java Message Service (JMS).

WebSphere Application Server includes the runtime Java client code and the administration windows that make it easy to connect to an IBM MQ queue manager. This way, web applications can reliably send messages to, and get responses from, any other IBM MQ enabled application that might be running in the enterprise.


Supported platforms

IBM MQ helps to integrate virtually anything, with support for more than 80 platform configurations. For information about the supported platforms, see the system requirements for IBM MQ at:
http://ibm.com/software/integration/wmq/requirements


Ordering information

Table 1 shows the ordering information.

Table 1. Part numbers and feature codes
Program namePID numberCharge unit description
IBM MQ for Multiplatforms V8.05724-H72Per Processor Value Unit (PVU) for Linux on System z
Per PVU
PROCESSOR-Day


Related information

For more information, see the following documents:
  • IBM announcement letters and sales manuals
    http://www.ibm.com/common/ssi/index.wss?request_locale=en

    On this page, enter IBM MQ for z/OS V8.0, MQ for Multiplatforms, or ENUSLP14-0216, and click Search. On the next page, narrow your search results by information type, geography, language, or all three options.


Special Notices

This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. IBM assumes no responsibility for its accuracy or completeness. The use of this information or the implementation of any of these techniques is a client responsibility and depends upon the client's ability to evaluate and integrate them into the client's operational environment.

Follow IBM Redbooks

Follow IBM Redbooks