Continuous

deployment
with Kubernetes
ADVANCED STRATEGIES GUIDE

www.gocno.io
hello@gocno.io
This white paper is the result of a collaborative workshop
conducted with one of our customers to question the
organization’s IT performance with all stakeholders. As the
experiment was a success, this paper is now in your hands.

We’re going to discuss deployment with Kubernetes, starting

with the distinction between continuous delivery and continuous
deployment and the challenges these strategies impose! With
these fundamentals out of the way, we’ll continue with a
second part dedicated to understanding advanced deployment
strategies: Blue/Green, Canary and A/B Testing - How do they
work? Why and when should they be used? And let’s not forget
to share with you the magic CNO tool for implementing them...
in just a few clicks!
Continuous deployment
with Kubernetes

Deploying with
Kubernetes is not a panacea

1.1 Continuous deployment VS continuous delivery p04

1.2 Bottleneck of continuous deployment p05

Kubernetes advanced
deployment strategies

2.1 Warming up: deployment with Kubernetes p07

2.2 Blue/green to reduce stress p08

2.3 Canary to ensure your application stability p09

2.4 A/B Testing to validate our assumptions p10

2.5 How to choose the good strategy for the good use case ? p11

2.6 How to deploy with advanced strategy ? Use case : Canary with Flagger p12

Deploying with
Kubernetes is a panacea

3.1 Introduction to Cloud Native Onboarding p15

3.2 CNO deploy: the most effective CD solution p16

3.3 How to deploy with advanced strategy ... with CNO ? p17

3.3.1 Automatic Deployment p17

3.3.2 Blue/green Deployment p18

3.3.3 Canary Deployment p19

3.3.4 A/B testing deployment p20

Conclusion & areas of

improvement explored
 Deploying with
Kubernetes is not a panacea
If Kubernetes is the tool preferred by us - decision-makers and computer engineers of a large
portion of existing companies - that is continuous deployment, in fact we are not there yet! Rest
assured it’s normal, it’s up to us now to remedy it.

Where to start? In this first part, we will distinguish between continuous delivery and continuous
deployment. These concepts are important to assimilate in order to ask the right questions..
Questions that allow us to improve our decision-making power.

Continuous deployment
1.1
VS continuous delivery

Continuous delivery is the most widely used deployment strategy in the world. This involves manual
intervention in the CD process to bring the new image of the pushed application to market. Conversely,
continuous deployment implies that each new image pushed by developers is automatically launched on
the market.

Today, we see two things. The first is that continuous deployment has given way to continuous delivery
in most organizations. The second is that the companies that have dared to deploy continuously are now
the most efficient and profitable. Why ? We can almost instinctively grasp the losses associated with
continuous delivery and therefore the benefits associated with continuous deployment.

Kubernetes is complex Continuous deployment with K8s • www.gocno.io • p4

 Continuous delivery Continuous deployment

Each intervention has a cost in time and money You are already on the market

Each intervention slows down your time-to-market You are already on the market

Each intervention involves sprint review You are already on the market

Each intervention decreases teams motivation You are already on the market

Don’t worry, we’re not devaluing the usefulness of continuous delivery which in some use cases proves to
be the best choice, or the only one. However, it is fundamental to consider the benefits of what is actually
the much-cited CD. And by experience, taking the step from the first to the second is often based on
fears... proven but largely surmountable!

bottleneck
1.2
of continuous deployment

Let’s not throw the stone to this magnificent open-source tool that allowed us to make our monolithic
infrastructures an old memory by opening the era of microservices with its co-religionist Docker. Let’s
remember that Kubernetes is a recent technology that, like any novelty, takes with it its corollary of
challenges. These challenges must be aware of to complete your migration to Kubernetes. Here’s a list of
the issues we encounter most in the field as a kubernetes expert and certified trainer:

Distraction, illusory problem &

uncontrolled cost of exploitation
Answer given by Aliou Ba CEO of CNO to the following question:
what are the most important problems related to Kubernetes?

Kubernetes bottleneck Continuous deployment with K8s • www.gocno.io • p5

K8s native complexity
One of the greatest advantages of Kubernetes is also one of its greatest challenges: automation. Automation makes
operations complex for IT teams to implement, developers to use and managers to understand.

Too many tools

We’re all familiar with it. The CNCF landscape is incredibly vibrant, but more importantly, seemingly endless. These
tools introduce an extra level of intricacy to the already complex world of Kubernetes. How many months have we
devoted to selecting the perfect tool to address a specific issue? Well, quite a lot... perhaps too much. Yet, let’s refrain
from criticizing these solutions, as they enable us to effectively manage Kubernetes and deploy apps to it.

Increasingly heterogeneous ecosystems

How long does it take a DevOps engineer or a developer to simply access the desired environment? Well, a long
time… too long. As our Kubernetes ecosystem (number of pods and clusters) grows (and it’s growing all the time, we
hope), it becomes more complex to govern. Accessibility, security and observability are invading our teams’ sprints,
and our time to market continues to slow down.

Not enough skills

We all share this problem: DevOps resources are scarce on the market. How much time have we spent looking for
the consultant who will cure all our ills? This time, on a personal level, we can, because we’ve been those famous
consultants. The fact remains that, without this skill, the work of developers becomes more arduous, while training
costs increase (for the better!).

Denis, remind me of
the objectives, please.
Take heart!
Catching up...
I've got an idea.

Kubernetes bottleneck
 Overcoming your fears... progressively.

Remember, the first cause that prevents us from taking the step between continuous delivery
and continuous deployment is based on fear and in view of the list of challenges cited, it is totally
justified. Nevertheless, these challenges are surmountable and must be overcome to take full
advantage of the benefits that Kubernetes offers.

Here we are not only talking about the standard deployment - rolling release - but about
advanced strategies that bring both technical and strategic added value: Blue/Green, Canary,
A/B testing.

Kubernetes advanced
deployment strategies
Let’s get to the heart of the matter: the objective is to embrace the benefits of continuous
deployment. So, in this second part, we’re going to detail all the strategies available to us.

Warming up
2.1
Deployment with Kubernetes

A little theory to get us aligned! A Kubernetes deployment is a resource object providing declarative
updates for applications. A deployment object allows us to: deploy or update a set of replicas or a pod, roll
back to a previous version, scale a deployment or pause a deployment.

Suffice to say, if this task isn’t automated, deploying becomes a pain in the ass for all IT
teams and CFOs... and their budgets go through the roof!

To deploy with Kubernetes without an external tool, we need to write manifests that we then apply to
our «Kubectl apply» clusters, or use a declarative deployment model. We indicate the desired state to
Kubernetes, and the latter will work to respect our directives by applying a «Rolling Update» strategy
(progressive replacement of pods) or a «Recreate strategy» (deletion of existing pods to create new ones).
We consider deployment complete once all tasks have been carried out and our system is in the desired
state. Do we agree? We hope so, because this is a book!

Is it possible to deploy with advanced strategies only with Kubernetes only?

No.

Kubernetes deployment Continuous deployment with K8s • www.gocno.io • p7

 2.2 Blue/Green to reduce stress

Blue/green is part of a logic of continuous improvement of your services, moving from one version
to another with the advantage of instant rollback. Welcome to the wonderful world of continuous
deployment!

Blue-green is a technical strategy in which you deploy a new image of your application in a staging
environment - GREEN - environment while the previous image version remains operational in a production
environment - BLUE. GREEN is in a staging environment accessible only by your internal teams and
version BLUE is in the production environment accessible to your users online.

This allows your teams to test and correct the new version - GREEN - in a staging environment without
affecting the user experience. When the viability of this new version is assured, you simply switch it with
the old version. GREEN is then deployed in the production environment accessible to your users while
BLUE is taken offline.

Kubernetes deployment Continuous deployment with K8s • www.gocno.io • p8

 2.3 Canary to ensure your application stability

Canary is a secure deployment strategy that is very useful for application services requiring a high
degree of security, such as banking applications, identification functionalities, private messaging
functionalities... In short, critical services that threaten your users and your business.

Just like its A/B Testing counterpart, when you use Canary you deploy the old version and the new version
in a production environment accessible to your users. What differentiates them is the load balancer you
assign to them. In other words, the transition from one to the other is staggered: 10%, 20%, up to 100%
of the new application.

In the case of Canary, your choices to deploy the application progressively will be guided by the technical
metrics received. This allows you to ensure the stability of your new image in a production environment
throughout the transition. Once the new image is 100% deployed, the old version is taken offline.

Kubernetes deployment Continuous deployment with K8s • www.gocno.io • p9

 2.4 A/B Testing to validate your assumptions

A/B testing is part of a behavioral monitoring paradigm and not a purely technical process. Within
the behavioral spectrum, IT and marketing teams will be guided by the KPIs received to make the best
business decision.

B (New version) is deployed in a production environment just like A (old version) - which remains in a
production environment and are both accessible to your users. What differentiates them are the values
assigned to each. The values chosen will indicate that B will be accessible only to a targeted group of
consumers: Beta testers. A is accessible to all users not concerned by the values.

This allows your technical and marketing teams to test this new version directly with your consumers.
The decision to switch between the old and new versions will be based on behavioral data. B will be
accessible to all your users and A will be taken offline.

Kubernetes deployment Continuous deployment with K8s • www.gocno.io • p10

 How to choose the good strategy
2.5
for the good use case ?

There’s no such thing as a turnkey question. The factors to consider are your technical resources, your
available human resources and skills, the criticality of the application concerned, and your objectives. The
table below can help you to fine-tune your future deployments :

Blue/Green Canary A/B Testing

General services Critical services Business strategy

Goals

Release quickly and reduce Ensure your application Validate assumptions

stress during outages. stability over time against the real World.

No downtime Test by your customers Test by your customers

Instant rollbacks Deploying viable App in production Faster & reliable decision-making
(+)

Reliable deployment Quick problem detection & resolution Deploying viable App in production

Transactions may be lost during switchover Switching staggered in time Switching staggered in time
Double ressources needs (replication) Slow rollout speed Expensive and an intelligent load balancer
(-)

Test effective and done before the release Script-based processes not recommended The most complex process strategies

Less effort Less stress More value

Sheduling window maintenance not An excellent way to reconcile speed and An excellent way to test the new service
IT

required and cluster condition update only security for the deployment of critical stability in production environment on a
at once. applications sample of consumers
Business

The assurance of delivering a viable The assurance of maintaining consumer The oportunity Improve Marketing strategies
new image by applying backend testing confidence and experience choices directly on your consumers: operation
in a real environment without interruption launching, test new offer / feature...
End user

Secured & continuous renewal Secured & continuous renewal A better user experience
of the user experience of the user experience validated by them

Kubernetes deployment Continuous deployment with K8s • www.gocno.io • p11

 How to deploy with advanced strategy today ?
2.6
Use Case : Canary

From the point of view of difficulty of implementation, advanced deployment strategies are classified as
follows in ascending order: (1) Blue/Green, (2) Canary, (3) A/B testing. In this section, we’ll detail all the
operations required to deploy a service with Canary.

Use Case Canary with Flagger

Flagger is an open source Kubernetes operator that will gradually shift traffic to the new version while
measuring the success rate of requests and the average response time. To implement a Canary deployment
with Flagger, you need to configure a Canary custom resource to specify the list of metrics to be analyzed
and the mesh service to be used if you have several in the cluster.

Implementation :

(1) Deploy Istio with telemetry and prometheus support

istioctl manifest install --set profile=default

kubectl apply -f
https://raw.githubusercontent.com/istio/istio/release-1.8/samples/addons/prometheus.
yaml

(2) Install Flagger in the istio-system namespace

kubectl apply -k github.com/fluxcd/flagger//kustomize/istio

Kubernetes deployment Continuous deployment with K8s • www.gocno.io • p12

(3) Deploying a gateway

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: public-gateway
namespace: istio-system
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- «*»

(4) Create a test namespace to deploy our application

kubectl create ns test

kubectl label namespace test istio-injection=enabled
kubectl apply -k https://github.com/fluxcd/flagger//kustomize/podinfo?ref=main

(5) Deploy the customized Canary resource

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
name: podinfo
namespace: test
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: podinfo
autoscalerRef:
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
name: podinfo
service:
port: 9898
targetPort: 9898

Kubernetes deployment Continuous deployment with K8s • www.gocno.io • p13

 gateways:
- public-gateway.istio-system.svc.cluster.local
hosts:
- app.example.com
retries:
attempts: 3
perTryTimeout: 1s
retryOn: «gateway-error,connect-failure,refused-stream»
analysis:
interval: 1m
threshold: 5
maxWeight: 50
stepWeight: 10
metrics:
- name: request-success-rate
thresholdRange:
min: 99
interval: 1m
- name: request-duration
thresholdRange:
max: 500
interval: 30s

Congratulations, you’re on the market! The question now is, is it necessary to carry out all these steps? No.

Canary with Flagger Continuous deployment with K8s • www.gocno.io • p14

 Deploying with
Kubernetes is a panacea
As we saw earlier, continuous deployment is not a small job, even for the most skilled among
us. We’ve experienced these barriers first-hand as engineers, developers, administrators and
project owners.

To overcome them, we created our own deployment tool with the idea of standardizing
continuous deployment with Kubernetes, and more specifically the use of advanced strategies.
In this third and final part, we’ll show you how we did it. Because yes, we’ve done it: here, we
trade weeks of hard labor to get into production for a few days of easy operation.

Introduction to
3.1
Cloud Native Onboarding

CNO is a SaaS engineering platform that enables users to seamlessly deploy their Kubernetes services,
making all deployment strategies just a few clicks away. How does it work? Here’s an overview:

Basics setting

Continuous deployment Continuous deployment with K8s • www.gocno.io • p15

 CNO deploy
3.2
the most effective CD feature

CNO is a SaaS engineering platform that enables users to seamlessly deploy their Kubernetes services,
making all deployment strategies just a few clicks away. How does it work? Here’s an overview:

CNO API
The API module is the central component of the platform. It powers the CNO console, as well as the CNO
client. The APIs provided are RESTful, HTTP JSON-based, and secured using TLS.

CNO CD OPERATOR
CNO offers Continuous Deployment operations for users through the CD Operator. This operator handles
the Custom Resources related to the CD actions provided by CNO.

CNO AGENT
CNO offers Continuous Deployment operations for users through the CD Operator. This operator handles
the Custom Resources related to the CD actions provided by CNO.

CNO UI & CTL

The UI and CTL CNO offer exactly the same functionality. Privileges and roles are active on both surfaces.
The user just has to choose the preferred working environment.

Continuous deployment Continuous deployment with K8s • www.gocno.io • p16

 How to deploy with advanced strategy ...
3.3
with CNO ?

3.3.1 Manual to Auto for our developers

Before going into production, CNO includes the Default (rolling release) strategy. This makes it easy
to deploy a new version of your image, either manually or automatically. This strategy is perfect for CI
developers, who will be able to test their new build in no time.

In automatic mode, as soon as your developers push a new image to the artifact repository, it replaces the
previous version. If you remain in manual mode, you must select the version to be released each time. To
do this, find your applications in Deploy :

01) Go to deploy
02) Select your application
03) Update your application
04) Indicate the number of replicates
05) Select default strategy
06) Enter the version you wish to deploy
07) Press Automatic buton
08) Update

Continuous deployment Continuous deployment with K8s • www.gocno.io • p17

3.3.2 Blue/green with CNO

01) Go to deploy
02) Select your application
03) Update your application
04) Indicate the number of replicates
05) Select Blue/green strategy
06) Select the version to activate in the staging environment
It’s time to test your version internally. You’ll find the link on your application’s dashboard.
07) Choose the right time to switch from the old version to the new one

Continuous deployment Continuous deployment with K8s • www.gocno.io • p18

3.3.3 Canary with CNO

01) Go to deploy
02) Select your application
03) Update your application
04) Indicate the number of replicates
05) Select Canary strategy & Select the version to activate in the staging environment
a. Manual progression : unselect the Auto Progression checkbox and click on Save
Shift manually the load balancer in time according to the metrics received.
07) Choose the right time to switch from the old version to the new one
b. Automatic deployment : Customize the following depending on your use case

• Interval: the interval between two transitions,e.g., from 10% to 20%.

• Type: the type of Auto Progression, can be incremental or by step.
• Success rate: the success rate before deploying
• Latency: latency time during the progressive deployment

The gradual transition from the old to the new version will take place automatically until the new version is 100% producible.

Continuous deployment Continuous deployment with K8s • www.gocno.io • p19

3.3.4 A/B testing with CNO

01) Go to deploy
02) Select your application
03) Update your application
04) Indicate the number of replicates
05) Select A/B Testing strategy
06) Select the version B that you want to test on customer sample
07) Indicate the values (Key/value) that will allow you to redirect your sample on a version B
08) Choose the right time to switch from the A to the B version

Default Blue/green Canary A/B Testing

Continuous deployment Continuous deployment with K8s • www.gocno.io • p20

Conclusion & areas of
improvement explored
Remember, the main obstacle that prevents us from moving from continuous delivery to continuous
deployment is the fear of the underlying problems that this model implies: having the necessary skills,
ensuring the security of our users… So many challenges to meet in kubernetes ecosystems, every day it’s
a little more complex. This shared fear prevents organizations from taking full advantage of Kubernetes:
the famous CD (and not, of CD. Got it?) and benefiting from the strategic advantages offered by advanced
strategies. Consequences: Distraction, illusory problem & high operational cost. In other terms : the time-
to-market is lengthened,the expected returns on investment are not forthcoming.

The consequences are not only commercial but also

human. Manual interventions are multiplying.
The work environment for developers, operators and
security teams is not conducive to fulfillment.
by Mody Sow, Tech Lead of CNO

As we have seen, with CNO you can significantly reduce the amount of human intervention throughout
your CD process while benefiting from the most complex deployment strategies. Is this functionality
optimal?

Continuous deployment Continuous deployment with K8s • www.gocno.io • p21

Today, yes, it offers all the strategies described in this document. Tomorrow, no. Just like you, we’re working
to improve the CNO day by day. Behind the scenes, Maimouna Diagne (CTO) and James Gaglo (product
manager) are working with their team on the integration of a CI part so that the platform can handle the
entire process from development to production release.

Guide written and presented by :

Mor Dioum, lead Cloud & DevOps at CNO

Louisane Salagnac, CMO at CNO

www.gocno.io
hello@gocno.io