Bringing the State back to Cloud

Functions
Asterios Katsifodimos, Asst. Professor

Stateful Functions as a Service Online Meetup 27/08/2020, organized by Big Data Republic 1
 Canonical (micro)services architecture in 2020

Order Payment Stock

service service service

Data Data Data Database

base base base (state)layer

To checkout: stock & update stock -> verify payment -> checkout the cart. 2
 Microservices Architecture (1): Easiest Implem.

Perform an order iff there

REST Call
Order Stock
Business
Logic
Business
Logic is stock available and the
payment is cleared.
RP
C
RP

l
al
REST Call

al

C
C

C
Re

se
RP
sp

on Services are stateless

on

sp
se

Re

DB
Database does the heavy-
C
RP

DB
ll
Ca
lifting
Payment DB
C
Business RP
e
Logic o ns
s p
Re
C
RP
High latency, costly state
access
3
 Microservices Architecture (2): Embedded State/DB
Order Stock
Business Business
Logic REST Call Logic

Make state of each service

DB DB
local to the business logic.
Services now are stateful
Low-latency access to
REST Call

local state
Service calls still expensive
Payment Not obvious how to scale
this out (e.g., shard the
Business
Logic

state in some way)

DB Fault tolerance is hard
4
 Microservices Architecture (3): Event Sourcing
Order Stock
Business Business
Logic REST Call Logic
event-log
Each message
DB DB
exchange/change to the
state goes through an
event-log.
event-log
REST Call

Services are
asynchronous/reactive.

Payment
If we lose state, we replay
Business the log and rebuild it.
Logic
Time-travel debugging,
audits, etc. are trivial.
DB
5
Microservices Architecture (4): Scalable Deployment

Order 1 Stock 1
Business
event-log
DB
LogicSubscribe for Responses Business
DB
Logic

Stock 2
event-log
Business
Order 2 Logic
DB
Business
DB
Logic

event-log
Payment 1
Business
DB
OrderRPC
3 Calls Logic

Business
Logic DB

6
A tale of the cloud: first was the VMs

Machines in company
premises

Before the cloud In the cloud: VMs available to rent

7
After some years…

8
 AWS claimed to have some good news
Function-as-a-Service (FaaS) for achieving serverless

Managed
Infrastructure VM No State
(autoscaling!)
Fn
VM
Function-based VM Fn-to-fn
programming Fn Fn
Fn calls
model
Fn Fn
Fn Transactions &
Stateless orchestration
functions Fn
Fn among function
calls

Cloud database

9
Meanwhile at the TU Delft campus…
Assignment: implement Stock, Order, Payment
microservices with one tool from of each layer:

Service Layer: Flask/Spring, AWS Lambdas, Azure Functions. Akka

Persistence Layer: Postgres, CockroachDB, Mongo, Cassandra, Redis
Infra Layer: Docker+Kubernetes on Amazon or Google Cloud

Goal: maximum consistent order.checkout()

throughput in the Cloud

Quiz question: out of 40, 5-person teams, how many

managed to sustain a deployment of 10K order
chekcouts per second, and keep the data consistent? 10
 None!
State Management is hard!
And the current technology is primitive!
(or the students have learned nothing)

Ongoing research in Distributed Systems and Databases for decades.

How to make stateful computations fault tolerant?

How do we (or should we) guarantee message delivery?

How do we consistently query the global state of a full system?

What abstractions should people use?

11
 We live in the stone ages.

Building Cloud applications is like

programming assembly before
compilers were around. “Two-pizza” dev team in the year 2020.
The road ahead

Scalable and Consistent Services in the Cloud

13
 A Call to Arms: novel SFaaS abstractions & systems

Functions & communication

compiled to efficient Time-Travel Debugging Global Application State
implementations Capabilities Querying and Consolidation

Holistic Optimization Versioned Transactions &

of Service Compositions Deployments/State/Schema Service Orchestration
My Research: stateful functions as DAGs of operators
Together with:

Marios Fragkoulis
[1] Operational Stream Processing: Towards Scalable and
Consistent Event-Driven Applications
Asterios Katsifodimos, Marios Fragkoulis.
In EDBT 2019.
Adil Akhter (ING Bank)

[2] Stateful Functions as a Service in Action

Adil Akhter, Marios Fragkoulis, Asterios Katsifodimos.
In VLDB 2019.
Pedro Silvestre

15
 A programming model* for authoring cloud applications

Function definition Function arguments

Function state

Transaction
orchestration

Function call

Function state update

Transaction failure handling

*currently compiles to cyclic Flink+Kafka dataflows. 16

 Control Event (commit, prepare, snapshot marker, etc.) Message Managed Operator State

SVC1 SVC2 SVC3

Time-travel debugging using
checkpoints and message
broker

SVC5 SVC4
Guaranteed message delivery
Input Message Queues and exactly-once processing
Output Message Queue

Each operator executes a

(group of) microservices or

Outgoing Message Router

Logical Message Inboxes

functions that share the same

state
Business Logic

Operator-local state
partitioned on key input for
Managed State scalability and fault-tolerance

17
 Building Cloud applications is like
programming assembly before
compilers were around. We need to
fix this. How?

• Talk to your Cloud provider

• Support intitiatives like Cloudstate.io, Flink’s Stateful Functions, MS
Orleans, CloudBurst, etc.
• Talk to me if you want to know more about our plans!
18