Containerized Storage for Containers

Kubernetes LA Meetup
July 18, 2017
@OpenEBS
Evan Powell - @epowell101
Who?

Evan Powell, CEO

Team of storage and DevOps engineers with Director of Ops & BD who started founding
a 5 year track record of delivering companies back in 2000 and is still at it. Also
containerized storage (built on jails) to NTT, an investor and advisor and friend to
eSilicon, TCS and other service providers. enterprise focused entrepreneurs.

CloudByte is the pre-existing SDS brand. Joined as CloudByte / Open EBS CEO 7/1/17
What if storage for
container native
applications was itself
container native?
 ✓ Intent inherited
✓ Scale w/o bottlenecks

✓ No special skills needed ✓ Granularity

✓ Resiliency ✓ Inate multitenancy

✓ Multi-cloud portability ✓ Smaller “blast radius”

Stateful still too difficult
Manifests express K8S used with special care for apps requiring persistence
intent because they require brittle tight coupling.

stateless stateful

Container 1 Container 2 Container 3

(Node1) (Node 2) (Node 3)
Container 1 Container 2 Container 3
Hard wired connections via plug-in
Container 1 Container 2 Container 3 “IO Blender”

NAS SAN S3 NAS Cloud

Any Server, Any Cloud

✓ Intent inherited
No changes to DevOps workflow even for containers requiring
persistence. Users manifest their intent and the storage and
Manifests express storage controllers adjust automatically as needed.
intent

stateless stateful

Container 1 Container 2 Container 3 Container 1 Container 2 Container 3

Container 1 Container 2 Container 3 Data Container Data Container Data Container

Any Server, Any Cloud Any Server, Any Cloud

Containerized storage for containers

✓ Same workflow, same tools, same skills
Dynamic Provisioning
Volumes

High Availability
Provisioners
stateful

PVC/PV Data Protection

Container 1 Container 2 Container 3

StorageClasses Storage Silos, Kubernetes

Data Container Data Container Data Container Managed

Attach/Detach Controllers
Vendor Lock-in, Cross cloud

CNCF/CSI
Monolithic!, Microservices
Any Server, Any Cloud
Slow Upgrades, Seamless
Containerized storage for containers Upgrades
Storage Orchestration - K8s Vs OpenEBS
Functionality K8s OpenEBS
Volume Life-cycle - provisioning, attaching/detach, de-provisioning X

Monitoring the volume data X

Scheduling the volume provisioning intelligently X

Smart space management X

Availability of volume X

Application aware backup/snapshots X

QoS guarantee (latency guarantee) X

Smart movement of the volumes across racks and regions X

Cataloging of storage requirements for different applications X

K8s takes care of connecting applications to stateful storage. OpenEBS answers the other persistent storage questions
Storage just fades away as a concern
How is OpenEBS built ?
is built using
Storage containerization

Storage orchestration is integrated with

(Control plane)
Deep storage operations Smart Capacity Management
is built into (OpenEBS “Maya”)
Volume Latency Monitoring and Auto-healing
Adapt to varying workloads - Application Aware Storage (Pods)
AWS EBS Like APIs
Storage Analytics

(Data Plane)
Containerized volumes
Thin and scalable block storage
Core Storage features is built into (OpenEBS) Enterprise HA
Intelligent QoS
Smart Tiering for higher performance
Architecture: Kubernetes
Minion Minion Minion
POD POD POD POD POD POD POD POD POD
Container Container Container Container Container Container Container Container Container

Container Container Container Container Container Container Container Container Container

Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet

etcd Cntrl
APIs
K8S Master Schld

Minions run on physical nodes

PODs group containers, share an IP address, and each include a Kubelet agent
K8S Master services include: etcd, APIs, the scheduler, the control manager & others
Architecture: Kubernetes + OpenEBS
Minion Minion Minion
POD POD POD POD POD POD POD POD POD
Container Container Container Container Container Container Container Container Container

Container Container Container Container Container Container Container Container Container

Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet Container
Kubelet

etcd Cntrl
APIs
K8S Master Schld

APIs OpenEBS Maya Schld

Data Containers run in PODs on physical machines

Data Containers group physical disks local or remote, manage replicas, provide QoS control and more
MAYA runs on the Master; delivers services such as: APIs, the storage scheduler, analytics & others
Why move the data & configs next to the app?
Today’s scale out storage systems are almost infinitely configurable. Matching their configurations to
static workloads is difficult. In a world of dynamic, ever evolving workloads due to micro-services and
multi-cloud deployments, it is not a tractable problem for humans.

Let’s say you want to benchmark for your workload:

Source: Alex Aizman’s blog: https://storagetarget.com/author/alexaizman/

Why move the data & configs next to the app?
Today’s scale out storage systems are almost infinitely configurable. Matching their configurations to
static workloads is difficult. In a world of dynamic, ever evolving workloads due to micro services and
multi-cloud deployments, it is not a tractable problem for humans.

Let’s say you want to benchmark for your workload:

Duration of run
~6,700 years
Source: Alex Aizman’s blog: https://storagetarget.com/author/alexaizman/
More benefits of Cloud Native approach
OpenEBS always available multi-cloud block services are delivered via containerized microservices

✓ Granularity: Each pod has its own block protocol stack, tiering engine, QoS engine and more. SLAs per POD.

✓ Performance & efficiency: access patterns maintained per-volume. Hot-data on NVMe-Flash or on 3DX-Memory. Cold-
data on slower disks / SAN/ Cloud-Storage/S3.

✓ Scalability: Metadata based on “size of the volume” & not on “number of volumes”. Can reduce metadata traversal 100x.

✓ Blast radius: Eliminates risk of scale out storage system corruption.

And much, much more such as cost savings due to resource efficiency, innate multitenancy, and cross cloud portability

The Conway’s Law benefits might be the most important - innovation unlocked
✓ Scalbility - no metadata bottleneck
Application

Application

IO Blocks
The volume IO processing has to
deal with the global metadata of The volume IO processing has to
IO Blocks
8TB deal with the volume metadata of
100GB
Lun

Lun

IO Chunks
Metadata of the volume is
Logical Metadata is managed at managed at chunk level
IO Blocks
volumes block level Volume
Logical
100 GB
meta Volumes in XFS files
Meta

8 TB
Global Metadata at a system level Meta
2 TB
XFS Meta data XFS Meta data 2 TB
Meta Meta

100TB
Raw 100TB
Raw
Cluster Admin - Enabling OpenEBS on K8s
K8s master K8s minions
Step 1: Use can existing K8s Cluster or Launch a new Cluster
(locally or in your fav cloud provider or use stackpoint.io to
create a cluster.)

Step 2: Configure OpenEBS on K8s.

kubectl apply -f openebs-operator.yaml

Step 3: Configure or Customize OpenEBS

kubectl apply -f openebs-config.yaml

kubectl apply -f openebs-storageclasses.yaml

https://github.com/openebs/openebs/tree/master/k8s
Developer - Using OpenEBS PVs
K8s master K8s minions
Step 1: Checkout the StorageClasses available

kubectl get sc

Step 2: Modify the Application YAML to use the openebs

storage class.

kubectl apply -f demo-jupyter-openebs.yaml

Step 3: Launch Jupyter UI

http://<external-ip>:<port>/

https://github.com/openebs/openebs/blob/master/k8s/demo/jupyter/demo-jupyter-openebs.yaml
Admin’s interaction with OpenEBS
3
1 Storage classes
kubectl apply -f openebs-storageclasses.yaml
Central Repo

Install OpenEBS

kubectl apply -f openebs-operator.yaml

K8s master K8s minions

kubectl apply -f openebs-config.yaml

2
Developer’s interaction with OpenEBS
- Data science developer
- Wants to set up a jupyter notebook (not an expert in storage)

- Configure the jupyter yaml file for

all the jupyter functionality
K8s master K8s minions - Read the available OpenEBS
storage classes and choose one

kubectl get sc

2 - Launch the jupyter application

kubectl apply -f demo-jupyter-openebs.yaml

In short….

Cloud Native Cloud Washing

Pod 1 VS Pod 1

OR
 Join the
community
Q&A #slack
Slack.openebs.io

blog.openebs.io

@openebs
 Join the
community
What do you think? #slack
Slack.openebs.io

https://github.com/openebs blog.openebs.io

@openebs
 Join Us and contribute to Kubernetes

- Prioritized Replication
Networking/Storage
- Federated Clusters
- Analytics and ML Services
- CNCF/CSI

https://www.openebs.io/careers/