Building Big Data and Analytics Solutions in the Cloud
An IBM Redpaper publication
Published 08 December 2014
IBM Form #: REDP-5085-00
Authors: Wei-Dong Zhu, Manav Gupta, Ven Kumar, Sujatha Perepa, Arvind Sathi, Craig Statchuk
Big data is currently one of the most critical emerging technologies. Organizations around the world are looking to exploit the explosive growth of data to unlock previously hidden insights in the hope of creating new revenue streams, gaining operational efficiencies, and obtaining greater understanding of customer needs.
It is important to think of big data and analytics together. Big data is the term used to describe the recent explosion of different types of data from disparate sources. Analytics is about examining data to derive interesting and relevant trends and patterns, which can be used to inform decisions, optimize processes, and even drive new business models.
With today’s deluge of data comes the problems of processing that data, obtaining the correct skills to manage and analyze that data, and establishing rules to govern the data’s use and distribution. The big data technology stack is ever growing and sometimes confusing, even more so when we add the complexities of setting up big data environments with large up-front investments.
Cloud computing seems to be a perfect vehicle for hosting big data workloads. However, working on big data in the cloud brings its own challenge of reconciling two contradictory design principles. Cloud computing is based on the concepts of consolidation and resource pooling, but big data systems (such as Hadoop) are built on the shared nothing principle, where each node is independent and self-sufficient. A solution architecture that can allow these mutually exclusive principles to coexist is required to truly exploit the elasticity and ease-of-use of cloud computing for big data environments.
This IBM® Redpaper™ publication is aimed at chief architects, line-of-business executives, and CIOs to provide an understanding of the cloud-related challenges they face and give prescriptive guidance for how to realize the benefits of big data solutions quickly and cost-effectively.
Table of contents
1. Big data and analytics overview
2. Big data and analytics business drivers and technical challenges
3. Big data and analytics taxonomy
4. Big data and analytics architecture
5. Key big data and analytics use cases
6. Big data and analytics patterns
7. Picking a solution pattern
8. NoSQL technology
9. Big data and analytics implications for the data center
10. Big data and analytics on the cloud: Architectual factors and principles
Others who read this publication also read
Follow IBM Redbooks
Follow IBM Redbooks