Microservices, Kubernetes & Istio - A great fit!

ISTIO ISTIO
Presenters: ! CONTROL DATA
PLANE PLANE
! SAMPLE BOOKINFO APP

Animesh Singh!
Anthony Amanse !
Tommy Li! ISTIO ENVOY
50%
! ENVOY PILOT

REVIEWS-V1

Panelists: !
! ENVOY

Chris Rosen ENVOY ENVOY
USER REVIEWS-V2 r
Nilesh Patel! MIXER
BOOK RATINGS
PAGE

ENVOY

50% REVIEWS-V3

ISTIO
AUTH

ENVOY
r
DETAILS
 Microservices, Kubernetes & Istio - A great fit!
Presenters: !
!
! Slides : http://ibm.biz/kube-istio!
Animesh Singh!
Anthony Amanse !
!
Tommy Li! Developer Journeys: http://ibm.biz/kube-journeys!
!
Panelists: !
!
!
Reach out to us at:!
Chris Rosen !
Nilesh Patel! @AnimeshSingh!
!
@AnthonyAmanse!
!
@Tomipli!
!
@ChrisRosen188!
!
@nileshandweb!
!
Evolution of Microservices
Microservices!

An engineering approach focused on decomposing

an application into single-function modules with
well defined interfaces which are independently
deployed and operated by a small team who owns
the entire lifecycle of the service.
Microservices accelerate delivery by minimizing
communication and coordination between people
while reducing the scope and risk of change.
Microservices
Application
Interactions
Microservices
Application
Scaling
Microservices
Application
Update
 Microservices, Containers
and Container Orchestrator
 IBM Bluemix Container Service

Typically microservices are encapsulated inside containers…

One:One relationship between a microservice and a container
Everyone’s container journey starts with one container….
IBM Bluemix Container Service

At first the growth is easy to handle….

IBM Bluemix Container Service

But soon it is overwhelming…we need container

and microservices management
IBM Bluemix Container Service

Enter Container Orchestrator

Container Orchestration = !

Scheduling, Cluster Mgmt, !

and Discovery!
Container Stack
Slide Title Goes Here

Layer 6 Development Workflow

Opinionated Containers

Layer 5 Orchestration/Scheduling
Service Model

Layer 4 Container Engine

Layer 3 Operating System

Layer 2 Virtual Infrastructure

Layer 1 Physical Infrastructure

Container Orchestration
Slide Title Goes Here

Source: devops.com
Kubernetes
 What is Kubernetes?
Slide Title Goes Here

•  Container orchestrator!

•  Runs and manages containers!

•  Supports multiple cloud and bare-metal environments!

•  Inspired and informed by Google's experiences and internal systems!

•  100% Open source, written in Go!

•  Manage applications, not machines!

•  Rich ecosystem of plug-ins for scheduling, storage, networking!

 IBM Bluemix Container Service

Intelligent Scheduling Self-healing Horizontal scaling

Service discovery & load balancing Automated rollouts and rollbacks Secret and configuration management
Kubernetes Architecture
Slide Title Goes Here

Worker Node 1!
UI!
Worker Node 2!
Kubernetes
API! Master!
Worker Node 3!

CLI! Worker Node n!

Registry
•  Etcd
•  API Server
•  Controller Manager
Server
•  Scheduler Server
 Kubernetes Architecture
Slide Title Goes Here

Nodes – hosts that run Worker Node 1!

UI! Kubernetes applications!

Worker Node 2!
Kubernetes Worker nodes:
API! Master! •  Hosts the K8s services
•  Worker Node
Kubelet (K8s3!agent that accepts
commands from the master)
Master nodes: •  Kubeproxy (network proxy service
•  CLI!
Controls and manages the cluster responsible
Worker Node for
n! routing activities for
•  Kubectl (command line) inbound or ingress traffic) Registry
•  REST API (communication•  with workers)
Etcd •  Docker host!
•  logic!
•  Scheduling and replication API Server
•  Controller Manager
Server
•  Scheduler Server
 Kubernetes Architecture
Slide Title Goes Here

Pods: Worker Node 1!

UI! •  Smallest deployment unit in K8s Labels:
•  Collection of containers that run on a •  Metadata assigned to K8s resources
worker node •  Key-value
Worker Node 2! pairs for identification
•  Each has its own IP •  Critical to K8s as it relies on querying the
•  Pod shares a PID Kubernetes
namespace, cluster for resources that have certain labels!
API! Master!
network, and hostname!
Worker Node 3!

Replication controller: Service:

CLI!
•  Ensures availability and scalability Worker Node n!
•  Collections of pods exposed as an
•  Maintains the number of pods as requested by Registry
endpoint
user •  Etcd
•  API Server •  Information stored in the K8s cluster
•  Uses a template that describes
•  specifically
Controller Manager state and networking info propagated
what each pod should contain! Server
•  Scheduler Server to all worker nodes!
IBM Bluemix Container Service
Slide Title Goes Here

•  Single tenant k8s master

Worker Node 1!
•  Running in IBM managed account
UI!
•  Key managed to maintain account separation

Worker Node 2!
Kubernetes
API! Master!
Worker Node 3!

•  Single tenant worker nodes

CLI! •  Dedicated
Workernodes
Nodearen! single
tenant top to bottom (worker
nodes, hypervisor, Registry
•  Etcd hardware)
•  API Server •  No core or RAM over
•  Controller Manager subscription to ensure Multi-tenant
Server services with per
•  Scheduler Server allocated resources and
ensure no noisy neighbors tenant isolation!
(
https://www.ibm.com/cloud-computing/bluemix/virtual-
servers)
Kubernetes
Developer
Journeys
 Developer Journeys:
Slide Title Goes Here
Scalable Wordpress on Kubernetes

In addition to running MySQL inside a container, we also show advanced capabilities like Bluemix service binding by leveraging
Compose for MySQL service!
!
Developer Works Code: https://developer.ibm.com/code/journey/scalable-wordpress-on-kubernetes!
Github: https://github.com/IBM/scalable-wordpress-deployment-on-kubernetes"
Developer Journeys:
Slide Title Goes Here
Deploy a Distributed GitLab on Kubernetes

This project shows how a common multi-component application can be deployed. GitLab represents a typical multi-tier app
and each component will have their own container(s).!
!
Developer Works Code: https://developer.ibm.com/code/journey/run-gitlab-kubernetes/ "
Github: https://github.com/IBM/kubernetes-container-service-gitlab-sample"
Developer Journeys:
Slide Title Goes Here
Scalable Apache Cassandra on Kubernetes

Leverages Kubernetes Pods, Service, Replication Controller, StatefulSets!

!
Developer Works Code: https://developer.ibm.com/code/journey/deploy-a-scalable-apache-cassandra-database-on-kubernetes"
Github: https://github.com/IBM/scalable-cassandra-deployment-on-kubernetes"
 Kubernetes &
Microservices



Developer Journeys
 Developer Journeys:
Slide Title Goes Here
Spring Boot Microservices on Kubernetes

This journey shows you how to create and deploy Spring Boot microservices within a polyglot application and then deploy the
app to a Kubernetes cluster.!
!
Developer Works Code: https://developer.ibm.com/code/journey/deploy-spring-boot-microservices-on-kubernetes/"
Github: https://github.com/IBM/spring-boot-microservices-on-kubernetes"
"
 Developer Journeys:
Slide Title Goes Here
Deploy Java microservices on Kubernetes within a polyglot ecosystem

With current application architectures, microservices need to co-exist in polyglot environments. In this developer journey, you’ll learn how to
deploy a Java microservices application that runs alongside other polyglot microservices, leveraging service discovery, registration, and routing.!
!
Developer Works Code: https://developer.ibm.com/code/journeys/deploy-java-microservices-on-kubernetes-with-polyglot-support/ "
Github:  https://github.com/IBM/GameOn-Java-Microservices-on-Kubernetes!
 Developer Journeys:
Slide Title Goes Here
Java MicroProfile Microservices on Kubernetes

Java based Microservices application using MicroProfile (baseline for Java Microservices architecture) and Microservices
Builder on Kubernetes!
!
Developer Works Code: https://developer.ibm.com/code/journey/deploy-microprofile-java-microservices-on-kubernetes!
Github: https://github.com/IBM/java-microprofile-on-kubernetes"
 Developer Journeys:
Slide Title Goes Here
Java MicroProfile Microservices on Kubernetes

Java based Microservices application using MicroProfile (baseline for Java Microservices architecture) and Microservices
Builder on Kubernetes!
!
Developer Works Code: https://developer.ibm.com/code/journey/deploy-microprofile-java-microservices-on-kubernetes!
Github: https://github.com/IBM/java-microprofile-on-kubernetes"

DEMO
 Kubernetes is great for
Microservices…
Why do we need a Service
mesh and what is it?
Container Orchestration = !

Scheduling, Cluster Mgmt, !

and Discovery!
What else do we need for"
Microservices?

●  Visibility!
●  Resiliency & Efficiency!
●  Traffic Control!
●  Security!
●  Policy Enforcement!
!

Enter Service Mesh"

What is a ‘Service Mesh’ ?

•  A network for services, not bytes!

●  Visibility!

●  Resiliency & Efficiency!

●  Traffic Control!

●  Security!

●  Policy Enforcement!
 Service Mesh Service Mesh
Data Plane Control Plane

How to build a Microservice-1 Sidecar

SERVICE
DISCOVER TELEMETR

‘Service Mesh’ ?
Y Y

•  Lightweight sidecars "

to manage traffic "
between services"
Microservice-2 Sidecar
SERVICE
REGISTRY ACCESS
CONTROL

•  Sidecars can do
much more
than just load
balancing!

RESILIENC
Microservice-3 Sidecar ROUTING Y
RULES FEATURES
Istio
 ISTIO ISTIO
CONTROL DATA
PLANE PLANE
•  Pilot - Configures Istio deployments
and propagate configuration to the
other components of the system.
ISTIO ENVOY
Routing and resiliency rules go here ENVOY PILOT

ROUTING MICROSERVICE
RULES
•  Mixer - Responsible for policy
decisions and aggregating telemetry
ENVOY
data from the other components in the
MIXER MICROSERVICE
system using a flexible plugin
GRAPHANA
Istio! architecture /ZIPKIN

Concepts! ENVOY

•  Proxy – Based on Envoy, mediates MICROSERVICE

ISTIO
inbound and outbound traffic for all AUTH

Istio-managed services. It enforces ENVOY

access control and usage policies, and MICROSERVICE

provides rich routing, load balancing,

and protocol conversion.
 ISTIO ISTIO
CONTROL DATA
PLANE PLANE
SAMPLE BOOKINFO APP

ISTIO ENVOY
ENVOY PILOT 50%
REVIEWS-V1
ROUTING
RULES

ENVOY

ENVOY ENVOY
MIXER USER REVIEWS-V2 r
BOOK RATINGS
PAGE
GRAPHANA

Istio! /ZIPKIN

ENVOY
Concepts!
50% REVIEWS-V3

ISTIO
AUTH

ENVOY
r
DETAILS
 Control Plane REST API
Prometheus
Istio ingress
Proxy Controller Metrics & reports
Istio
Istio Control
Control Plane
Plane from proxies
Istio Control Plane
(Pilot, Mixer,Auth)

Kube API Server

2. Proxy implements intelligent L7

routing, circuit breakers, enforces
policies and reports metrics to
control plane.
Pod

Istio Proxy Proxy

Architecture 1. All traffic entering and

leaving pod is transparently
routed via Proxy without
requiring any application
appA appB changes.

Service A Service B Kubernetes Cluster

Proxy. Based on Envoy, a high
performance L7 proxy from
Lyft, currently being used at
large scale in production.
https://github.com/lyft/envoy
Istio control plane traffic. User/application traffic. HTTP/
Request routing rules, 1.1, HTTP/2, gRPC, TCP with or
resilience configuration (circuit without TLS
breakers, timeouts, retries),
policies (ACLs, rate limits,
auth), and metrics/reports from
proxies.
 Kubernetes,
Microservices

and Istio
Developer Journeys
What is a ‘Service Mesh’ ?

A network for services, not bytes!

"  Resiliency & Efficiency

●  Traffic Control!

●  Visibility!

●  Security!

●  Policy Enforcement!
Resiliency

•  Istio adds fault tolerance to your application •  Resilience features

without any changes to code ❖  Timeouts
❖  Retries with timeout budget
•  // Circuit breakers ❖  Circuit breakers
•  destination: serviceB.example.cluster.local ❖  Health checks
policy:
- tags: ❖  AZ-aware load balancing w/
version: v1 automatic failover
circuitBreaker:
simpleCb:
❖  Control connection pool size and
maxConnections: 100 request load
httpMaxRequests: 1000 ❖  Systematic fault injection
httpMaxRequestsPerConnection: 10
httpConsecutiveErrors: 7
sleepWindow: 15m
httpDetectionInterval: 5m

 Resiliency Testing

•  Systematic fault injection to identify weaknesses in failure recovery policies

–  HTTP/gRPC error codes
–  Delay injection

Timeout: 100ms Timeout: 200ms

Retries: 3 Retries: 2
300ms 400ms
Envoy Envoy Envoy

svcA svcB svcC

Service Service Service

A B C
 Developer Journey:
Leverage Is3o to create resilient and fault tolerant Microservices
Slide Title Goes Here

Twelve-factor apps make a strong case for designing and implementing your microservices for failure. What that means is with the
proliferation of microservices, failure is inevitable, and applications should be fault-tolerant. Istio, a service mesh, can help make your
microservices resilient without changing application code.!
!
Developer Works Code: https://developer.ibm.com/code/journey/make-java-microservices-resilient-with-istio/"
Github: https://github.com/IBM/resilient-java-microservices-with-istio"
 Resilient Microservices with Istio
ISTIO
ISTIO
DATA
CONTROL "
PLANE
PLANE Load
ENVOY
ENVOY Balancing
Pool
schedule
ISTIO speaker
ENVOY PILOT
ENVOY

3 4 RESILIENC
Y & FAULT
TOLERANC ENVOY
E RULES
5 NOT
RSPONDING"
- CIRCUIT
BREAK
web-application 4

ENVOY

MIXER ENVOY
vote 3

ENVOY
MAX
CONNECTIONS
- CIRCUIT
ISTIO session BREAK
AUTH
5
DELAYED
1 RESPONSE
- TIME OUT

2
 User Input
N requests N requests
Is2o Ingress Envoy MS-A

Reached maximum pending

requests - eject all the incoming
requests.

Reached maximum connec2ons –

Administrator put the incoming requests in pending state
Set Des2na2on Policy
Circuit Breaker
Is2o Pilot ( X Max Conn, Envoy MS-B
Y Max Pending)
 User Input
N requests N requests
Is2o Ingress Envoy MS-A

1 2 3

Administrator Circuit Breaker

Set Des2na2on Policy 503

Is2o Pilot Envoy MS-B Envoy MS-B

MS-B Pod 2 MS-B Pod 1

(Broken) (Working)
Eject X minutes
Load Balancing Pool for MS-B
User Input 504 error
N requests
Is2o Ingress N requests Envoy MS-A

Timeout

Administrator Fault Injec2on

X seconds delay
Set Route Rule

Is2o Pilot
Envoy MS-B

MS-B Pod
What is a ‘Service Mesh’ ?

A network for services, not bytes

●  Resiliency & Efficiency

●  Traffic Control

●  Visibility
●  Security

●  Policy Enforcement
Traffic Splitting
serviceB.example.cluster.loca

Rules API Pod Labels:

•  // A simple traffic splitting rule version: v1.5
•  destination: Istio-Manager env: us-prod
serviceB.example.cluster.local
Traffic routing
•  match: rules Envoy
source: 99%
serviceA.example.cluster.local Pod
route:
- tags: Envoy svcB

Service B
version: v1.5
http://serviceB.example 1%
•  env: us-prod
svcA
•  weight: 99 Envoy
•  - tags: Service
version: v2.0-alpha A
•  env: us-staging svcB

•  weight: 1
Pod Labels:
version: v2.0-
alpha, env:us-
staging
Traffic Steering Service B
version: v1
Pod 1
Pod 2
•  // Content-based traffic steering rule svcA Pod 3

svcB
Service A
•  destination: serviceB.example.cluster.local
match:
httpHeaders:
user-agent:
Service B
regex: ^(.*?;)?(iPhone)(;.*)?$
version: v1
precedence: 2 Pod 1
route:
Pod 2
- tags:
Pod 3
version: canary
svcA svcB

Service A
Pod 4

Content-based traffic steering svcB’

version: canary

IBM Cloud I Internal Usage Only © IBM Corporation / Confidential

 Visibility

•  Monitoring
& tracing should not be an
afterthought in the infrastructure"
•  Goals!

Istio - Grafana dashboard w/ Prometheus backend

•  Metrics without instrumenting apps!

•  Consistent metrics across fleet!

•  Trace flow of requests across services!

•  Portable across metric backend Istio Zipkin tracing dashboard

providers!
 Metric Flow Prometheu
Prometheu
s InfluxDB
InfluxDB
Custom
Prometheu
s backend
s InfluxDB

Prometheus

Adapter
InfluxDB
Adapter

Adapter
Custom
•  Mixer collects metrics emitted by Envoys!

•  Adapters in the Mixer normalize and Mixer

forward to monitoring backends! API: /svcB

Latency: 10ms
Status Code: 503
Src: 10.0.0.1
Dst: 10.0.0.2
•  Metrics backend can be swapped at …...

runtime!
Pod
Envoy Envoy

svcA svcB

Service Service
A B
 Visibility : Tracing Zipkin
Prometheu Stackdriver
InfluxDB
Custom
Prometheu
s backend
s InfluxDB

Stackdriver
Adapter
Adapter
Application do not have to deal

Adapter
Custom
• 

Zipkin
with generating spans or
correlating causality!
Mixer
•  Envoys generate spans!

•  Applications need to *forward*

Span Span
context headers on outbound s s
calls!
Pod
•  Envoys send traces to Mixer!
Envoy Trace Headers

Envoy Envoy
X-B3-TraceId

X-B3-SpanId

•  Adapters at Mixer send traces to X-B3-ParentSpanId

X-B3-Sampled

respective backends! X-B3-Flags

svcA svcB svcC

Service Service Service

A B C
 Developer Journey:
Manage micro services
Slide Title Goes Here traffic using Istio on Kubernetes

Microservices and containers have changed application design and deployment patterns. They have also introduced new challenges, such as
service discovery, routing, failure handling, and visibility to microservices. Kubernetes can handle multiple container-based workloads, including
microservices, but when it comes to more sophisticated features like traffic management, failure handling, and resiliency, a microservices mesh
like Istio is required.!
!
Developer Works Code: https://developer.ibm.com/code/journey/manage-microservices-traffic-using-istio/ "
Github: https://github.com/IBM/microservices-traffic-management-using-istio "
 ISTIO ISTIO
EXTERNAL
CONTROL DATA
SERVICE
PLANE PLANE
SAMPLE BOOKINFO APP

ISTIO ENVOY
ENVOY PILOT 50%
REVIEWS-V1
ROUTING
3 RULES

ENVOY

ENVOY 6 ENVOY
MIXER USER REVIEWS-V2 r
4 BOOK RATINGS
PAGE
GRAPHANA
/ZIPKIN 7
ENVOY

5 50% REVIEWS-V3

ISTIO
AUTH

ENVOY
2 r
DETAILS

8
 Developer Journey:
Manage micro services
Slide Title Goes Here traffic using Istio on Kubernetes

Microservices and containers have changed application design and deployment patterns. They have also introduced new challenges, such as
service discovery, routing, failure handling, and visibility to microservices. Kubernetes can handle multiple container-based workloads, including
microservices, but when it comes to more sophisticated features like traffic management, failure handling, and resiliency, a microservices mesh
like Istio is required.!
!
Developer Works Code: https://developer.ibm.com/code/journey/manage-microservices-traffic-using-istio/ "
Github: https://github.com/IBM/microservices-traffic-management-using-istio "

DEMO
Part A

Pod
Traffic Route

Pod 50%

50%
Traffic Route

Pod user = jason

Traffic Route

Pod

100%
Access Policy

Pod

100%
Part B

Pod
Thank you!
ISTIO ISTIO
EXTERNAL
CONTROL DATA
SERVICE
PLANE PLANE
SAMPLE BOOKINFO APP

ISTIO ENVOY
ENVOY PILOT 50%
REVIEWS-V1
ROUTING
RULES

ENVOY

ENVOY ENVOY
MIXER USER REVIEWS-V2 r
BOOK RATINGS
PAGE
GRAPHANA
/ZIPKIN

ENVOY

50% REVIEWS-V3

ISTIO
AUTH

ENVOY
r
DETAILS