ECS781P

CLOUD COMPUTING

MICRO SERVICES

Lecturer: Dr. Sukhpal Singh Gill

School of Electronic Engineering and Computer Science
Contents
• Micro services
• Message-driven communications
• The reactive manifesto for cloud services

2
The history so far…

Scalability Consistency

Communications
Application

Data

Maintainability Reliability

3
Traditional design: monolithic applications

4
What has changed?
• Infrastructure
• Physical HW -> Cloud VMs -> Containers - > Functions
• Technology, middleware
• REST, NoSQL…
• Speed of change
• Continuous integration/devops
• Business priority
• Responsiveness

5
Netflix microsevices

6
Netflix microsevices
• Bucket testing
• to describe the method testing two versions of a website against one another to
test performance metrics (clicks, downloads or purchases)

• Zuul
• built to enable dynamic routing, monitoring, resiliency, and security.

• NCCP
• Netflix Content Control Plane

7
Microservices
“Approach to developing a single
application as a suite of small
services, each running in its own
process and communicating with
lightweight mechanisms, often an HTTP
resource API”
Martin Fowler

8
How to split into microservices?
• Componentization via Services
• Organized around Business Capabilities
• Products not Projects
• Decentralized Governance
• Smart endpoints and dumb pipes
• Evolutionary Design
• Decentralized Data Management
• Infrastructure Automation
• Design for failure

Sam Newman, “Building Microservices”. O Reilly 2015 9

Breaking down into microservices
• Componentisation via services
• Organised around business capabilities:
• strong boundaries
• "Any organization that designs a system (defined more
broadly here than just information systems) will
inevitably produce a design whose structure is a copy
of the organization's communication structure.“
Melvin Conway, 1976
• Size? Amazon: Split into 2-Pizza teams

10
https://www.theguardian.com/technology/2018/apr/24/the-two-pizza-rule-and-the-secret-of-amazons-success
Independently deployable
• Controlled by different
Customer
teams
Service
• Different technologies
Shipping
Returns
• Independent updates
• Release early, release
often
Inventory Invoicing

Accounts

12
Decentralised data management

https://martinfowler.com/articles/microservices.html 13
Scaling microservices

Monolithic scaling Microservices scaling

https://martinfowler.com/articles/microservices.html 14
Service discovery plus load balancing

Sam Newman, Building Microservices 15

Challenges of using micro services
• Distributed Systems: Harder to program,
distributed communications are unreliable.
• Operational Complexity: Application consists of
lots of services, updated regularly
• Eventual Consistency: Microservice instances can
be replicated, then consistency problems arise. No
transactions across microservices

16
Contents
• Micro services
• Message-driven communications
• Reactive services

17
Publish-subscribe model
• Decouple senders and receivers of messages
• Publishers send messages into queues
• Subscribers receive all messages from a queue

18
Request-driven vs message-driven design

19
The cost of blocking calls

20
Message-driven microservices

21
Message broker features
• Location transparency
• Message loss protection on failures
• Scalability -> Replication
Client Client

Client Broker Internet Broker Client

Client Client
Firewalls

Broker
Client

Client

Client

22
Contents
• Micro services
• Message-driven communications
• Reactive services

23
https://www.reactivemanifesto.org/
We believe that a coherent approach to systems
architecture is needed, and we believe that all
necessary aspects are already recognised individually:
we want systems that are Responsive, Resilient, Elastic
and Message Driven. We call these Reactive Systems.
Systems built as Reactive Systems are more flexible,
loosely-coupled and scalable. This makes them easier to
develop and amenable to change. They are significantly
more tolerant of failure and when failure does occur
they meet it with elegance rather than disaster.
Reactive Systems are highly responsive,
giving users effective interactive feedback.

24
https://www.reactivemanifesto.org/

25
Message driven
• Reactive systems rely on asynchronous message-
passing to establish a boundary between components that
ensures loose coupling, isolation and location
transparency.
• Employing explicit message-passing enables load
management, elasticity, and flow control by shaping and
monitoring the message queues in the system and
applying back-pressure when necessary.

26
Reactive programming
• Micro services are programmed as asynchronous
stream processors
• Operators are executed over observable streams
• Message-driven programming
• Functional programming, rather than procedural
programming
• Predefined set of transformations
• Reactive libraries for all major languages/tools
• E.g RxPy for Python, Akka for Scala
27
Reactive programming example

28
Elastic
• Reactive Systems are responsive by reacting to changes in
the workload by increasing or decreasing the
allocated resources.
• Ability to shard or replicate components and distribute inputs
among them.
• Predictive, as well as reactive scaling algorithms by
providing relevant live performance measures.

29
Resilient
• Reactive systems stay responsive in the face of failure.
• Failures are contained within each component, parts of the system
can fail and recover without compromising the system as a whole.
• Recovery of each component is delegated to another (external)
component
• High-availability is ensured by
replication where necessary.

30
Testing reliability: Netflix Chaos Monkey
• Every weekday between 9am and 5pm, an army of malicious programs,
affectionately known as “chaos monkeys,” are unleashed upon Netflix’s
information infrastructure.
• “Their sole purpose is to make sure that we’re failing in a consistent
and frequent enough way to make sure that we don’t drift into overall
failure,” Tseitlin said.

https://github.com/Netflix/chaosmonkey 31
Responsive
• Responsive systems focus on providing rapid and
consistent response times, establishing reliable upper
bounds so they deliver a consistent quality of service. This
consistent behaviour in turn simplifies error handling,
builds end user confidence, and encourages further
interaction.

32
Further reading
• Sam Newman, Building Microservices, Oreilly, 2015,
https://learning.oreilly.com/library/view/building-
microservices/9781491950340/
• Martin Fowler’s website,
https://martinfowler.com/articles/microservices.ht
ml