CDP Private Cloud Fundamentals 200810

Educational Services
CDP Private Cloud

Fundamentals
The Enterprise Data Cloud Vision

The Enterprise Data Cloud Vision
■ A new kind of platform
■ Four key characteristics
Copyright © 2010-2020 Cloudera. All rights reserved. Not to be reproduced or shared without prior written consent from Cloudera.
Characteristics of the Enterprise Data Cloud
Hybrid & Secure &

Multi-Function Open
Multi-Cloud Governed
Data Lifecycle
DATA OPERATIONAL
ENGINEERING DATABASE
Collect Report Predict
02 04
01 03 05
Enrich Serve
STREAMING & DATA MACHINE
DATA FLOW WAREHOUSE LEARNING & AI
SECURITY | GOVERNANCE | LINEAGE | MANAGEMENT | AUTOMATION
Cloudera Data Platform Overview

Cloudera Data Platform: Recap
■ Control costs and manage resources

– Auto-scale
– Auto-suspend
■ Easy provisioning and support for multiple types of workloads
■ Consistent security and data governance across applications and datasets
■ Enables enterprise IT staff to quickly respond to business demands
CIO Magazine
How to Eliminate Shadow IT
“Enterprise IT doesn’t operate
at the speed of business.
So business users
build their own capabilities
through shadow IT purchases."
CIO Magazine
“Your IT group needs to perform better
than shadow IT.”
CIO Magazine
CDP Public Cloud
CDP Public Cloud
■ Runs on public cloud CDP Public Cloud
(platform-as-a-service)
infrastructure
CDP Control Plane
■ Uses cloud provider's AWS Azure GCP

object store
Virtual Private Self-Serve

Clusters Experiences
Data Hub DW, ML, DE, …
Cloud Storage
Cloudera Runtime
CDP Private Cloud Base
CDP Public Cloud
■ Similar architecture CDP Private Cloud Base
(installable software)
to CDH and HDP
Cloudera Manager
■ Formerly known as Bare Metal
CDP Data Center
Bare Metal Workloads
HDFS + Ozone
Cloudera Runtime
Introducing CDP Private Cloud

CDP Product Overview
CDP Public Cloud CDP Private Cloud Plus CDP Private Cloud Base
(platform-as-a-service) (installable software) (installable software)
CDP Control Plane CDP Control Plane Cloudera Manager
Bare Metal
AWS Azure GCP OpenShift
Virtual Private Self-Serve Self-Serve

Clusters Experiences Experiences Bare Metal Workloads
Data Hub DW, ML, DE, … DW, ML, DE, …
Cloud Storage HDFS / Ozone HDFS / Ozone
Cloudera Runtime Cloudera Runtime Cloudera Runtime
CDP Private Cloud Plus
■ Runs on private cloud CDP Private Cloud Plus

(installable software)
(OpenShift) CDP Private Cloud Base
CDP Control Plane
Cloudera Manager
■ Uses local storage OpenShift Bare Metal
(HDFS / Ozone)
■ Depends on a Private
Self-Serve
Cloud Base cluster Experiences
Bare Metal Workloads
DW, ML, DE, …

HDFS / Ozone
HDFS / Ozone
Cloudera Runtime
Cloudera Runtime
CDP Private Cloud Architecture

Important Trends
■ Rising cloud adoption increases the pace of business
■ This changed user's expectations for IT

– Instant provisioning
– Instant scalability
■ It has also led to innovations in software architecture
Traditional Cluster Architecture (Bare Metal)
■ This cluster might be running This node assigns tasks

and tracks where data is
– CDH stored
– HDP
– CDP Private Cloud Base
■ Nodes are connected via a network switch

– Gigabit ethernet was once common The remaining nodes do the
actual work of running
tasks and storing the data
■ Designed to conserve limited bandwidth

– Storage and compute are colocated
■ Specific services deployed to specific nodes
Limitations of the Traditional Cluster Architecture
■ Colocation of storage and compute

– Can't scale them independently
■ Optimized for large files

– Leads to the "small files" problem
■ Shared resource model for multitenancy

– Leads to "noisy neighbor" problem
■ Rigid mapping of services to nodes

– Distributes resources inefficiently
Key Aspects of the Cloud-Native Architecture
■ Fast networks enable separation of storage from compute

– This allows administrators to scale them independently
■ Object stores are the preferred way to store data

– This eliminates the "small files" problem
■ Containers decouple an application from the environment where it runs

– They provide isolation needed to solve the "noisy neighbor" problem
– They also enable more efficient distribution of resources
What is Kubernetes?
■ Often abbreviated as k8s
■ Software system used to deploy, scale, and manage containerized applications
■ Originally developed at Google, now open source
■ Supported by all major cloud providers and available in commercial products
■ A collection of machines running Kubernetes software is called a "cluster"
Kubernetes Overview
Control
Plane
Kubelet Kubelet Kubelet
C C C C C C C C C C C
Worker Node #1 Worker Node #2 Worker Node #3
CPU RAM Disk CPU RAM Disk CPU RAM Disk
Comparing CDP Public and Private Cloud
CDP Public Cloud - AWS CDP Public Cloud - Azure CDP Private Cloud
Self-Service Experiences Self-Service Experiences Self-Service Experiences
CDP Management Console CDP Management Console CDP Management Console
Elastic Kubernetes Service (EKS) Azure Kubernetes Service (AKS) OpenShift Container Platform (OCP)
Security · Governance Security · Governance Security · Governance

Schema · Metadata Schema · Metadata Schema · Metadata
EC2 Azure Virtual Machines CDP Private Cloud Base Cluster
S3 ADLS Gen2 HDFS / Ozone
CDP Private Cloud Architecture
Control Plane
Data Warehouse Machine Learning
Kubernetes (OpenShift OCP)
Security
Bare Metal HDFS
Metadata
Workloads Ozone
Governance
CDP Private Cloud: Initial Release
Control Plane
Data Warehouse Machine Learning
Security
Bare Metal HDFS
Metadata
Workloads Ozone
Governance
CDP Private Cloud: Future State
Control Plane
Data
Catalog
Workload
Manager
Data Machine Data Operational Data
Replication Warehouse Learning Engineering Database Flow
Manager
Security
Bare Metal HDFS
Metadata
Workloads Ozone
Governance
Installation
Installation Requirements
■ Check the documentation, as details may change
■ Red Hat OpenShift Cluster
■ CDP Private Cloud Base Edition cluster

– Serves as the data lake
– HDFS, Ozone, Hive metastore, Ranger, and Atlas services are required
– Can upgrade existing CDH/HDP cluster or perform new installation
– Must be configured for Kerberos (MIT or Active Directory)
– Must have Auto TLS enabled
■ Fast network connection between OpenShift and Data Lake clusters
Installation Demo
■ Your instructor will now demonstrate CDP Private Cloud installation
Conclusion
Essential Points
■ Simplified multitenancy
– Scale the workloads, not individual services
■ Infrastructure agility
– Manage shared resources efficiently
■ Upgrade agility
– Enable flexibility through containerization
■ Self-service provisioning
– Eliminate bottlenecks while maintaining oversight

CDP Private Cloud Fundamentals 200810

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CDP Private Cloud Fundamentals 200810

Uploaded by

Copyright:

Available Formats

Educational Services

CDP Private Cloud

The Enterprise Data Cloud Vision

■ A new kind of platform

■ Four key characteristics

Hybrid & Secure &

SECURITY | GOVERNANCE | LINEAGE | MANAGEMENT | AUTOMATION

Cloudera Data Platform Overview

■ Control costs and manage resources

■ Easy provisioning and support for multiple types of workloads

■ Consistent security and data governance across applications and datasets

■ Enables enterprise IT staff to quickly respond to business demands

How to Eliminate Shadow IT

■ Uses cloud provider's AWS Azure GCP

Virtual Private Self-Serve

Data Hub DW, ML, DE, …

■ Formerly known as Bare Metal

CDP Data Center

Bare Metal Workloads

Introducing CDP Private Cloud

CDP Control Plane CDP Control Plane Cloudera Manager

Virtual Private Self-Serve Self-Serve

Data Hub DW, ML, DE, … DW, ML, DE, …

Cloud Storage HDFS / Ozone HDFS / Ozone

Cloudera Runtime Cloudera Runtime Cloudera Runtime

■ Runs on private cloud CDP Private Cloud Plus

DW, ML, DE, …

CDP Private Cloud Architecture

■ Rising cloud adoption increases the pace of business

■ This changed user's expectations for IT

■ It has also led to innovations in software architecture

■ This cluster might be running This node assigns tasks

■ Nodes are connected via a network switch

■ Designed to conserve limited bandwidth

■ Speciﬁc services deployed to speciﬁc nodes

■ Colocation of storage and compute

■ Optimized for large ﬁles

■ Shared resource model for multitenancy

■ Rigid mapping of services to nodes

■ Fast networks enable separation of storage from compute

■ Object stores are the preferred way to store data

■ Containers decouple an application from the environment where it runs

■ Often abbreviated as k8s

■ Software system used to deploy, scale, and manage containerized applications

■ Originally developed at Google, now open source

■ Supported by all major cloud providers and available in commercial products

■ A collection of machines running Kubernetes software is called a "cluster"

Kubelet Kubelet Kubelet

Worker Node #1 Worker Node #2 Worker Node #3

CPU RAM Disk CPU RAM Disk CPU RAM Disk

Self-Service Experiences Self-Service Experiences Self-Service Experiences

CDP Management Console CDP Management Console CDP Management Console

Security · Governance Security · Governance Security · Governance

EC2 Azure Virtual Machines CDP Private Cloud Base Cluster

S3 ADLS Gen2 HDFS / Ozone

Data Warehouse Machine Learning

Kubernetes (OpenShift OCP)

Data Warehouse Machine Learning

Kubernetes (OpenShift OCP)

Kubernetes (OpenShift OCP)

■ Check the documentation, as details may change

■ Red Hat OpenShift Cluster

■ CDP Private Cloud Base Edition cluster