AI and Big Data on IBM Power Systems Servers

A draft IBM Redbooks publication

Updated 12 February 2019

Authors: Scott Vetter, Ivaylo B. Bozhinov, Anto A John, Rafael Freitas de Lima, Ahmed.(Mash) Mashhour, James Van Oosten, Fernando Vermelho, Allison White


As big data becomes more ubiquitous, businesses are wondering how they can best leverage it to gain insight into their most important business questions. The use of machine and deep learning on Big Data environments can identify historical patterns and build AI models that can help businesses to improve customer experience, add services and offerings, identify new revenue streams or lines of business, and optimize business or manufacturing operations. The power of AI for predictive analytics is being harnessed across all industries, so it is important that businesses familiarize themselves with all of the tools and techniques that are available for integration with their data lake environments.

In this IBM® Redbooks® Publication, we cover the preferred practices for deploying and integrating some of the best AI solutions on the market, including:

  • IBM Watson Machine Learning Accelerator (see note for product naming)
  • IBM Watson® Studio Local
  • IBM Power Systems™
  • IBM Spectrum™ Scale
  • IBM Data Science Experience
  • IBM Elastic Storage™ Server
  • Hortonworks Data Platform
  • Hortonworks DataFlow
  • H2O Driverless AI

We map out all the integrations that are possible with our different artificial intelligence (AI) solutions and how they can integrate with your existing or new data lake. We also walk you through some of our client use cases and show you how some of the industry leaders are using Hortonworks, PowerAI, and Watson Studio Local to drive decision making. We also advise you on your deployment options, when to use a GPU, and why you should use the Elastic Storage Server to improve storage management. Lastly, we describe how to integrate Watson Machine Learning Accelerator and Hortonworks with or without Watson Studio Local, how to access real-time data, and security.

Note: IBM Watson Machine Learning Accelerator is the new product name for IBM PowerAI Enterprise

Note: Hortonworks merged with Cloudera in January 2019 with the new company being called Cloudera. References to as a business entity in this publication are now referring to the merged company. Product names beginning with Hortonworks continue to be marketed and sold under their original names..

Table of contents

Chapter 1. Solution Overview
Chapter 2. Integration overview
Chapter 3. Integration of new data
Chapter 4. Accessing real-time data
Chapter 5. Integration details
Appendix A. Additional Information
Appendix B. Watson Machine Learning Accelerator Notebook installation

These pages are Web versions of IBM Redbooks- and Redpapers-in-progress. They are published here for those who need the information now and may contain spelling, layout and grammatical errors. This material has not been submitted to any formal IBM test and is published AS IS. It has not been the subject of rigorous review. Your feedback is welcomed to improve the usefulness of the material to others.

Follow IBM Redbooks

Follow IBM Redbooks