vForum Online: May 13, 2020

Run and Manage

Kubernetes with
vSphere 7
Essential Services for the
Modern Hybrid Cloud

Himanshu Singh
Mike Foley
VMware Cloud Platform Marketing

Confidential │ ©2020 VMware, Inc.

 Between 2020 and 2024, enterprises
2020 was the first year
LOB IT spend (51%) exceeded will build 500 million apps
Infrastructure and Operations IT using cloud native tools and
spend (49%)1 methods2

2020 2024

VMware Portfolio Strength:

and more…

1: IDC WW Semiannual IT Spending Guide: Line of Business, 09 April 2020 (HW, SW and services; excludes Telecom) 2: IDC Futurescape,
Confidential │ ©2020 VMware, Inc. 2020 2
The Definition of an Application Has Changed

An application used Modern applications are

to consist of a few VMs distributed systems

App Database Kubernetes Cluster App

My Application
vGPU
Control Pane

Node GPU Node Node

Database

Node Serverless

Function 1 Function 2

Confidential │ ©2020 VMware, Inc. 3

Containers are the Future

Container use is skyrocketing

and is projected to grow at 64% By 2022, organizations will
64% CAGR through 2022. 77% deploy containers primarily in
the data center

The big stones we have to overcome are day two operations. It is not a big deal to deploy
Kubernetes clusters. The trouble starts when you want to start
updating, upgrading, adding nodes, all that kind of stuff.”
Stephan Massalt​, VP of Cloud Labs​, Swisscom​

Confidential │ ©2020 VMware, Inc. Source: Worldwide Container Infrastructure Software Forecast, 2020–2022, IDC, Dec 2020 4
“It’s important for IT operators
to closely partner with
developers on Kubernetes
deployments. Today, developers are
uprooted from previous environments and need
to understand the value of the other.”

Sharat Nellutla
Associate Director
Verizon

5
Confidential │ ©2020 VMware, Inc. 3 Kubernetes Secrets From Early Adopters, RADIUS, Jan 2020
Challenges for Modern Applications

Line-of-Business
Developer Leader VI Admin
More complicated to get Monolithic apps fail to meet modern Infrastructure silos make it challenging
modern apps into production requirements to provision resources

Ticket-based infrastructure Cannot quickly respond to changing market Security isolation of modern apps
slows development cycles demands and databases is difficult

Difficulties updating apps Weak customer experiences sacrifice Inconsistent operations and
can impact resiliency market share cross-functional workflows

Confidential │ ©2020 VMware, Inc. 6

What would an ideal solution look like?

Expanding market share

Faster time to revenue
Loyal customers
Self-service through Innovation Leverage skills I have
Kubernetes API
Manage multiple containers as
Sandbox to host multiple a unit
containers Development
Govern SLAs for security,
Open source conformant availability, and QoS
Interact with services, not Operations Fewer infrastructure
infrastructure stacks to maintain

Confidential │ ©2020 VMware, Inc. 7

Introducing vSphere 7

Confidential │ ©2020 VMware, Inc. 8

vSphere 7: Essential Services for the Modern Hybrid Cloud
Streamline Development
• Real-time infrastructure access through
Kubernetes APIs
• Integrated Tanzu Kubernetes Grid for
fully conformant Kubernetes
• vSphere Pod service delivers high
performance and enhanced security
Confidential │ ©2020 VMware, Inc.
Agile Operations
• Application-focused management
• Simplified lifecycle management and
intrinsic security
• Unified platform and consistent ops
across all cloud, data center, and edge
deployments
Accelerate Innovation
• Cost efficient AI/ML hardware pools
• Performance and resiliency for DB and
business-critical apps
• Predictable quality of service for time-
critical apps
9
vSphere with Kubernetes
Transform your infrastructure to run modern applications anywhere

VMware Cloud Foundation Services

Developer

TKG vSphere Pod Code Deploy

service service
Test Support

Streamline Development

Network
Collaboration Storage service Registry service
service

Performance Availability
Application
focused Security Lifecycle
management
Agile Operations
vSphere
VI Admin

Accelerate Innovation

Confidential │ ©2020 VMware, Inc. 10

 vSphere 7 with Kubernetes Powers VMware Cloud Foundation 4

Namespaces

DB & Analytics AI/ML Business Critical Time-critical

vCenter Server
VMware Cloud Foundation Services

Developer Tanzu Runtime Services Hybrid Infrastructure Services VI Admin

Tanzu Kubernetes Grid Service Storage Service Network Service
Registry Service vSphere Pod Service

Data Center | Edge | Service Provider | Public Cloud

Confidential │ ©2020 VMware, Inc. 11

vSphere 7, Cloud Infrastructure for Modern Applications
vSphere 7: Other major capabilities and enhancements

Simplified Application
Lifecycle Intrinsic Security
Acceleration
Management

 vSphere Lifecycle Manager for  Remote attestation with vSphere  Cost efficient AI/ML hardware
simplified upgrades Trust Authority pools

 Update planner to discover, manage  Identity federation with ADFS  Performance & resiliency
and upgrade (improved vSphere DRS)
 Simplified software patching
 Upgrade using Restful APIs  Predictable quality of service

Confidential │ ©2020 VMware, Inc. 12

 Key Capabilities in vSphere 7

vSphere with Kubernetes vSphere Lifecycle Manager

Cloud Foundation Services unify VM and container Advanced automation of vSphere updates, upgrades, and
Summary
management through Kubernetes API maintenance.

• Faster development and innovation • Greater operational efficiency

Business Outcome
• Greater operational efficiency and collaboration • Fewer failures due to configuration drift

vSphere Trust Authority Identity Federation with ADFS

Ensure sensitive apps only run on properly attested and Delegates authentication responsibilities to Microsoft Active
Summary
trusted hardware. Directory.

• Usernames & passwords are stored in ADFS

Business Outcome • Lower security risk via reduced attack surface
• Fewer security audits

Confidential │ ©2020 VMware, Inc. 13

Packaging

Confidential │ ©2020 VMware, Inc. 14

vSphere 7 Editions

MAIN EDITIONS* SPECIAL EDITIONS

Standard Enterprise Plus Scale Out ROBO

Enhanced Application
Performance,
Availability, and Intrinsic
Security

Optimized for Big Data

and High-Performance
Basic Server Basic Server Computing (HPC)
Consolidation and Consolidation and Workloads
Resilience Resilience Designed for remote or
branch offices

Confidential │ ©2020 VMware, Inc. *Note: vSphere Platinum reached EoA in April 2020 15
vSphere 7 with Kubernetes
Only available in VMware Cloud Foundation 4

VMware Cloud Foundation 4 vSphere Add-on for Kubernetes

VMware Cloud Foundation Services

vSphere Add-on for Kubernetes
• Tanzu Kubernetes Grid service
SDDC Manager
• vSphere Pod service
vRealize • Storage service

vSAN • Network service

NSX • Registry service

vSphere ENT+
Term License (1-year and 3-year options)

vSphere with Kubernetes is ONLY available with VMware Cloud Foundation 4

Confidential │ ©2020 VMware, Inc. 16

“vSphere with Kubernetes will allow
us to accelerate our application
development while adhering to our
organization’s security policies.”
Albert W. Alberts
Architect
KPN

Confidential │ ©2020 VMware, Inc. 17

vSphere 7 with Kubernetes
A Deeper Dive Into Features

Confidential │ ©2020 VMware, Inc. 18

What’s A Modern Application?

TKG Cluster VM App

k8s Native
Applications VM

Control Plane

Database
Node Node Node

VM

Serverless
VM

Function 1 Function 2
Function Function
VM

Confidential │ ©2020 VMware, Inc. 19

Challenges

TKG Cluster VM App

k8s Native
Applications
DevOps VM VI Admin

Control Plane

Database
Node Node Node

Deploy this app

Ensure availability
VM
Operate it on Day 2
Ensure security
Serverless
Tool choice VM
Deliver quality of service

Function 1 Function 2
Function Function Cost control
VM

Confidential │ ©2020 VMware, Inc. 20

Solve the Challenges

TKG Cluster VM App

k8s Native
Applications
DevOps VM VI Admin

Control Plane

Database
Node Node Node

Self-Service VM Governance
Serverless
VM

Function 1 Function 2
Function Function
VM

Confidential │ ©2020 VMware, Inc. 21

Self-Service for DevOps Using Kubernetes

kind: TanzuKubernetesCluster TKG Cluster VM App kind: VirtualMachine

apiVersion: tkg.vmware.com/v1 apiVersion: vms.vmware.com/v1

metadata: k8s Native metadata:

name: My Application Applications name: COTSapp

DevOps spec:
VM
spec:

topology: className: large

Control Plane
workers: imageName: my-app.ova

count: 3 powerState: poweredOn

Database
class: small Node Node Node policy:

distribution: v1.18.1 restartPolicy: OnFailure

Self-Service VM

kind: Pod
Serverless kind: HanaDatabase

apiVersion: v1 apiVersion: hana.sap.com/v1

metadata:
VM metadata:

name: Function 1 name: ERP database

Function 1 Function 2
spec: Function Function spec:

containers: VM nodes: 3

- name: func1 class: extra-large

image: func1

ports:

- containerPort: 80

Confidential │ ©2020 VMware, Inc. 22

Application Centric Management
Namespace

TKG Cluster VM App

k8s Native
Applications VM VI Admin

Control Plane

Database
Node Node Node

VM Governance
Serverless
VM

Function 1 Function 2
Function Function
VM

Confidential │ ©2020 VMware, Inc. 23

Application Centric Management
Namespace Quality of Service
• Priority: High
TKG Cluster VM App
• Reserved vCPUs: 128

k8s Native • Reserved Memory: 1 TB

Applications VM VI Admin
Security

Control Plane • Encrypt all persistent data

• Disallow all ports but 443
Database • Audit developer changes
Node Node Node

VM
•
Availability
Failures to tolerate: 2
Governance
• Disaster recovery site: us-east
Serverless
• Hourly snapshots to backup
VM

Function 1 Function 2 Access controls

Function Function
• Users in group app-admin: Write
VM
• Users in group ops: Read Only
• Disallow MySQL

Confidential │ ©2020 VMware, Inc. 24

Application Centric Management
Namespace D
Namespace C
Namespace B
Kubernetes Cluster VM App
Namespace A
K8s Native
Applications VM VI Admin
Application D
Control Plane

Database
Node Node Node Application C

VM Governance
Application B
Native Pods
VM
Application A
Function 1 Function 2
Function Function
VM

Confidential │ ©2020 VMware, Inc. 25

vSphere with Kubernetes
Components

Confidential │ ©2020 VMware, Inc. 26

 Tanzu Kubernetes Grid Service
Self-service manage Tanzu Kubernetes Grid clusters

DevOps DevOps
3
Deploy Apps Flexible TKG Cluster LCM

Upstream Conformant
kubectl
create TKG
Cluster
Full Control of TKG Clusters
2 VI Admin
VI Admin
Define template,
Tanzu Kubernetes versions, etc.
Grid service Define Template, version
1
TKG &
Resource Quota
VM Operator Supervisor Cluster Cluster API

SDDC

Confidential │ ©2020 VMware, Inc. 27

 Tanzu Kubernetes Grid Service Architecture

ESXi Cluster (Supervisor Kubernetes) vCenter

Auto Update
User Namespace
Lifecycle Service
vSphere Tanzu Kubernetes Grid
watch Namespace
TKG Object
TKG Controller
actuate Services UI

VKS Plugin
watch
Cluster Object Cluster API Namespace
watch Cluster API
Machine Object Cluster API Controller
Plugin
DevOps actuate vSphere Provider VI Admin
VM API Plugin

watch
VM Object VM API Namespace

VM Controller
VM Lifecycle
actuate

Confidential ││©2020
Confidential ©2018VMware,
VMware,Inc.
Inc. 28
 vSphere Pod Service
Advanced security and performance, without managing clusters

DevOps DevOps
Strong Security and Resource Isolation

Performance Advantages

Serverless Experience
Kubectl
create

VI Admin
deployment

1 VI Admin
Application Centric Management
Application
vSphere Pod Centric
service Management
Workload Visibility

VM Operator Supervisor Cluster

SDDC

Confidential │ ©2020 VMware, Inc. 29

 Virtual Machine Service
Self-service deploy VM based applications

DevOps DevOps
3
Deploy Apps Manage virtual machine with
Kubernetes interface

Kubectl
create
virtual

2
machine
VI Admin VI Admin

Virtual Machine
Define VM Class Define VM classes, machine sizes
service
1 Manage VM images
Supervisor Cluster

SDDC

Confidential │ ©2020 VMware, Inc. 30

 Network Service

DevOps DevOps
K8s API for Network Objects e.g.
Services and Ingress

Kubectl
create
Ingress

2 VI Admin
VI Admin
Define network Define Admin Policies for Security
security rules
Network service and etc.

NCP Supervisor Cluster

SDDC

Confidential │ ©2020 VMware, Inc. 31

 Storage Service

DevOps DevOps
• K8s API for Storage Objects e.g.
Persistent Volume Claim

Kubectl
create pvc
VI Admin
2 VI Admin • Resource Quota
• Visibility
Define storage
policy and quota
Storage service

CNS Supervisor Cluster

SDDC

Confidential │ ©2020 VMware, Inc. 32

 Registry Service

DevOps Embedded image registry

Push/pull
images Automated project lifecycle
Deploy apps
2 Consistent permission

VI Admin
Create Registry
Instance
Registry Service

Supervisor Cluster

SDDC

Confidential │ ©2020 VMware, Inc. 33

Registry Service

ESXi vCenter
Control vCenter Enable Harbor registry in the
K8s Master VM Plane APIs
Registry supervisor cluster
Registry Agent Service
• Integrated with vSphere SSO
VI Admin
Harbor APIs DevOps users push images directly
ESXi into the registry
• Container image signing and
System Namespace
PV Docker Registry scanning built-in
Harbor Pods APIs
Seamlessly deploy native pods
Image Pull and Kubernetes pods using
container images in the registry
User Namespace DevOps
PV User Pods

Confidential │ ©2020 VMware, Inc. 34

Platform Architecture

Confidential │ ©2020 VMware, Inc. 35

Enable Cloud Service Fabric in vSphere with vSphere Pod Service

VI Admin

vCenter
ESXi Cluster

ESXi ESXi ESXi

hostd hostd hostd

VM VM VM

VM VM VM

Confidential │ ©2020 VMware, Inc. 36

Enable Cloud Service Fabric in vSphere with vSphere Pod Service

VI Admin

vCenter
ESXi Cluster

ESXi ESXi ESXi

Spherelet hostd Spherelet hostd Spherelet hostd DevOps

k8s Control Plane k8s Control Plane k8s Control Plane

VM VM VM

Pod
CRX

Confidential │ ©2020 VMware, Inc. 37

Enable Kubernetes in vSphere with vSphere Pod Service

VI Admin

vCenter
ESXi Cluster

ESXi ESXi ESXi

Spherelet hostd Spherelet hostd Spherelet hostd DevOps

vSphere Pod Service
k8s Control Plane k8s Control Plane K8s Control Plane
VM VM VM

Pod Pod Pod

VM VM
CRX

Confidential │ ©2020 VMware, Inc. 38

Workload Platform Architecture

Pod Pod Pod

Namespace Namespace

Control Worker Worker Worker

Plane

VM VM VM VM
Pod Pod Pod Pod Pod VM Pod VM VM VM VM Pod Pod Pod
Tanzu Kubernetes Cluster

Namespace Namespace Namespace Namespace Namespace

Tanzu Tanzu
VM Operator vSphere Pod Service Cluster API Kubernete VM Operator vSphere Pod Service Cluster API Kubernete
s Grid s Grid

SDDC

Confidential │ ©2020 VMware, Inc. 39

Multi-tenancy with vSphere Pod Service namespaces

Each Namespace has its own

Resource Pool

Resource Isolation with Quota for

Namespace Namespace Namespace Namespace CPU/Memory/Storage

vSphere Pod Service vSphere Pod Service

All Workloads in a Namespace are
bounded by Namespace Quota
• Tanzu Kubernetes Clusters
SDDC
• Native Pods
• Virtual Machines

Confidential │ ©2020 VMware, Inc. 40

vSphere Pod Service Network Topology and Isolation
vSphere Pod Service Leverages NSX Network Capability

Physical Network Fabric

vSphere Pod Service are isolated with Edge

Uplinks
Firewall on Tier-1 Gateways and Distributed
Firewall on a per vNIC level
Tier-0 GW

Namespaces are isolated with DVPG(NSX

Logical Segment) and Distributed Firewall
NSX Edge Load Balancer for
Service of type LoadBalancer and Ingress

Inbound Traffic is denied for all namespaces

Tier-1 GW
by default
vSphere Pod Service vSphere Pod Service
NSX Edge LB is used for Service of type Load
Balancer and Ingress
DVPG (NSX Logical Segment)
Distributed Virtual Port Group NSX Distributed Load Balancer is used for
Service of type ClusterIP
Namespace Namespace Namespace Namespace
TKG Clusters can use other overlay (Calico by
NSX Distributed Load Balancer for k8s default)
service of type ClusterIP

Pod Pod VM VM Pod VM VM Pod

Confidential │ ©2020 VMware, Inc. 41

Cloud Native Storage

Unified management of cloud native

storage

Namespace Namespace vSphere SPBM policies become

Storage Classes in vSphere Pod Service

PV PV PV PV PV PV Works across local, hyperconverged

and shared storage

Leverage full vSphere ecosystem of

Cloud Native Storage Control Plane
storage partners

Storage Policy Exposes persistent volumes as

First Class Disks vSAN File Services paravirtualized drivers in Supervisor
Based Mgmt
Namespace

VMFS vSAN NFS vVOL

Confidential │ ©2020 VMware, Inc. 42

 TECHNOLOGY PREVIEW

VMware Changes the Game with Bitfusion

Works with Any Supported vSphere
Version

Embracing Hardware Acceleration for Modern Apps

Optimize vSphere for Machine Learning and AI

Workloads
Modern apps like ML and AI need compute acceleration to
handle large and complex computation. vSphere leverages
powerful accelerators for workloads in VMs or containers.
Infrastructure can also be used for some HPC workloads.

Machine Learning Identify, consolidate and share hardware

and AI Workloads
accelerators
Easily identify isolated and expensive resources that are
underutilized. Hardware accelerators can be shared
remotely (fully or partially) regardless of location.
Hardware Accelerators
Extend now and in the future

Leverage GPUs across an infrastructure plus integrate

evolving technologies such as FPGAs and custom ASICs using
the same infrastructure.
GPUs FPGAs ASICs

Confidential │ ©2020 VMware, Inc. 43

In Closing…

Confidential │ ©2020 VMware, Inc. 44

“vSphere with Kubernetes can help our IT
team to achieve consistent operations of
our existing system and rapid scale-up for
new applications. And the new
architecture offers flexibility between
private cloud and multiple public clouds.”
Yang Shen
Chief Information Officer
Digital China

Confidential │ ©2020 VMware, Inc. 45

“ VMware now delivers native Kubernetes support in
vSphere. Because vSphere's interface is familiar to
VMware’s SDDC stack, this innovation will help
simplify the process of deploying Kubernetes clusters
in VMware environments. I’m looking forward to how
it will enable us to scale out cloud native apps as
easily as we scale out our SDDC cluster.”
François Corfdir
IT Consultant
Syscom

Confidential │ ©2020 VMware, Inc. 46

 “Cloud isn’t where you operate,
it’s how you operate.”
Joe Beda
Principal Engineer, VMware and co-creator of Kubernetes

Confidential │ ©2020 VMware, Inc. 47

 Thank You

Confidential │ ©2020 VMware, Inc.