S11 - System Architecture

System
Architecture
Memi Lavi
www.memilavi.com
System Architecture
• The Big Picture
• Answers the questions:

• How will the system work under heavy load?
• What will happen if the system will crash at this exact moment in the
business flow?
• How complicated can be the update process?
• And more…
System Architecture
• Includes:
• Defining the Software Components (Services)
• Defining the way these components communicate
• Designing the system’s capabilities (scalability, redundancy,
performance, etc.)
System Architecture
• Includes:
• Defining the Software Components (Services)
• Defining the way these components communicate
• Designing the system’s capabilities (scalability, redundancy,
performance, etc.)
Our Topics
• Loose Coupling
• Stateless
• Caching
• Messaging
• Logging & Monitoring

Loose Coupling
Making sure the services are not

strongly tied to other services
Loose Coupling
• Without:
Service A
Service B Service C
Loose Coupling
• Without:
Service A
Service B Service C
Loose Coupling
• Without:
Service A
Service B Service C
Loose Coupling
• With:
Service A
Service B Service C
Loose Coupling
• With:
Service A
Flexible, Easy to
Maintain
Service B Service C
Loose Coupling in Services
• Prevents platform coupling
• Prevents URL coupling

Stock Quotes Service

(Java)
Java RMI
Portfolio
PortfolioService
Service
(Java)
Stock Quotes Service Stock Quotes Service

(Java) (Java)
REST API REST API

http://server_55/api/stocks http://server_44/api/stocks
Recommendation
Service
(Python)
Loose Coupling
Service A
Service B Service F
The Spiderweb
Service C Service E
Service D
Loose Coupling
Service A
Service B Service F
The Spiderweb
Service C Service E
Service D
Loose Coupling
Service A
Service B Service F
The Spiderweb
Service C Service E
Service D
Loose Coupling - Directory
Query for URL Yellow
Service A
Pages
Service B Service F
Services only
Use
need to know
URL
the Directory’s
Service C Service E URL
Service D
Loose Coupling - Gateway
Service A
Service B Service F
Services only
Gateway need to know
the Gateway’s
Service C Service E URL
Service D
Stateless
The application’s state is stored

in only two places – the data
store and the user interface
State = Application’s
Stateless Example
User
Interface
Send user / password
Log In Service
Store user for future use

Data stored in code =
Stateful
Retrieve user details Check user / password
exists
Database
Scalability - A Reminder
• Grow and shrink as needed
• Scale Up vs Scale Out
• Scale Out is usually preferred

Redundancy - A Reminder
• Allows the system to function properly when resource
is not working
• Example:
• A system with more than one server
• When a server goes down, the other continue
working
Scalable & Redundant Architecture
User
Interface
Distributes Load Scalability

Load Balancer
Routes to live services only Redundancy
Is Alive?
Log In Service Log In Service Log In Service
Database
Stateful Example
User
Interface
Send user / password
Load Balancer
User / password sent to
Service #1
Log In Service Log In Service Log In Service
Store user for future use
1 2 3
Retrieve user details
Check user / password

exists
Database
Stateful Example
User
Interface
Add item to shopping cart
Please
Load Balancer
Log In
Request sent to
Service #2
Cart Service Cart Service Cart Service
Stored user details
1 2 3
Database
Stateless Example
User
Interface
Add item to shopping cart
Load Balancer
Action sent to
Service #2
Cart Service Cart Service Cart Service
1 2 Check for user details 3
Database
Stored user details
Stateless
• Always use stateless architecture
• Supports Scalability and Redundancy

Caching
Bring data closer to its consumer

so that its retrieval will be faster
Browser Cache
Browser Cache
Server
Service Cache
Service
DAL • Serialize to Objects
Cache
• Compile the SQL

Database • Retrieve the data
Cache Tradeoff
Reliability Performance
Single Source Database High Good
of Truth Data is saved to disk Data is retrieved from disk,
then serialized
Cache Poor Excellent

Data is stored in memory Data is retrieved from memory
What to Cache?
Cache should hold data that is

frequently accessed
and
rarely modified
What to Cache?

frequently accessed Retrieval should be fast and easy
for:
and - Optimal user experience
- Minimum load
rarely modified Using cache:
- Retrieval is fast (in-memory)
- UX is optimal
What to Cache?

frequently accessed
and
rarely modified - Syncing cache and DB is a
challenge
- When not in sync, leads to data
corruption and bad user
experience
Example
Stock Quotes
Service
Cache
Database
Example
Load Balancer
Request
Update Stock API Update Stock API Update Stock API
Stock Quotes Stock Quotes Stock Quotes

Service Service Service
Cache Cache Cache
Stale
Database
Cache Types

In-Memory, In-
Process Cache
Distributed Cache
• Existing libraries • External product

• Can be easily implemented using • Data is stored in separate process
static concurrent collection • Provides interface for accessing the data
• Great performance • Size virtually unlimited
• Size is limited to the process’s • Auto nodes syncing
memory • Not the best performance
• Stores only primitive types
Example
Load Balancer
Request

In Memory, Process Cache In Memory, Process Cache In Memory, Process Cache
• Develop Sync Mechanism

• Hammer the Database
Database
Example
Load Balancer
Request

Distributed Cache
• Data immediately synced

with all nodes
• All servers have the same
Database
data
Choosing Cache Type
Distributed Cache
Distribution among servers
Failover capabilities
Large Cache storage
Choosing Cache Type
Distributed Cache In-Memory, In-Process Cache

Distribution among servers Best performance possible
Failover capabilities Store complex objects
Large Cache storage
Choosing Cache Type
Distributed Cache In-Memory, In-Process Cache

Distribution among servers Best performance possible
Failover capabilities Store complex objects
Large Cache storage Very easy to use
Requires training and setup
Messaging
Means of communication
between the various services
Messaging
• Not just REST API
• Not Exclusive
Messaging Criteria
• Performance
• Message Size
• Execution Model
• Feedback & Reliability
• Complexity
REST API
• De-Facto Standard for HTTP-based systems

GET http://server/api/orders/17
POST http://server/api/orders
{order data…}
HTTP API
Service
REST API
Performance Very Fast

REST API

Message Size Same as HTTP protocol limitations
(Usually Get -> 8KB, POST & PUT -> dozens MB)
REST API

Execution Model Request / Response
Great for quick, short actions, not suitable for long
processes
REST API

processes
Feedback & Immediate feedback via Response
Reliability Codes
REST API

processes
Reliability Codes
Complexity Extremely easy to implement
REST API

processes
Reliability Codes
Useful For Traditional Web Apps
HTTP Push Notifications
Client Real-Time Communication:
Subscribe Notify
Service
HTTP Push Notifications
• Uses advanced web techniques (ie. Web Sockets)
• Very popular in chats

HTTP Push Notification
Performance Excellent
Message Size Limited
Usually no more than a few KB
Execution Model Web Socket connection / Long Polling
Feedback & None (Fire & Forget)
Reliability (Can be implemented, quite complex)


Useful For Chat, Monitoring
Queue
Place Pull message

message in… from…
Service #1 Queue Service #2
• Messages will be handled once and only once
• Messages will be handled in order

Queue
Performance Not so good

(Push / Poll, DB Persistence)
Queue

Message Size Technically almost not limited
But use small messages
Queue

Execution Model Polling
Queue

Feedback & Very reliable
Reliability
Queue

Reliability
Complexity Requires training and setup
Queue

Message Size Technically almost unlimited
Reliability
Useful For Complex system with lots of data, when
order and reliability are top priority
File-based & Database-based
Place Pull message

message in… File Folder from…
Service #1 Service #2
Database

Message Size Unlimited
Reliability
Useful For Complex system with lots of data.
Better use queues
Service #2
Instance #1
Problems:
Service #2
1. File locked File Folder Service #2
Instance #2
2. Duplicate processing
Service #2
Instance #3
Messaging Summary
REST API HTTP Push Queue File- & DB-based
Performance Very fast Excellent Not so good Not so good
Message Size Same as Limited Technically Unlimited
HTTP unlimited
limitations
Execution Request / Web Socket / Long Polling Polling
Model Response Polling
Feedback & Immediate None Very reliable Very reliable
Reliability feedback
Complexity Extremely Extremely easy Requires training Requires training
easy and setup and setup
Useful For Traditional Chat, Monitoring Complex system Complex system
web apps with lots of data, with lots of data.
where order and Better use queues
reliability are top
priority
Logging & Monitoring
Central Logging Service
Service #1 Service #2 Service #3
Log Engine #1 Log Engine #2 Log Engine #3
Database NoSQL
Log Engine #1 Log Engine #2 Log Engine #3
Logging Service
Database
• Implementation:
• API
• Watch folders
Correlation ID

Correlation ID
Recommendation
Log
X
Stock Quotes
Log
Recommendation Stock Quotes
Log Log
CorID 17 CorID 17
CorID 18 CorID 18
System Architecture- Summary
• Use these concepts to design a fast ,secure, reliable
and easy to maintain system
• Make the choice as early as possible
• They are not exclusive…
• But they are the most important ones

S11 - System Architecture

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

S11 - System Architecture

Uploaded by

Copyright:

Available Formats

System

• The Big Picture

• Answers the questions:

• Defining the Software Components (Services)

• Defining the way these components communicate

• Designing the system’s capabilities (scalability, redundancy,

• Defining the Software Components (Services)

• Defining the way these components communicate

• Designing the system’s capabilities (scalability, redundancy,

• Logging & Monitoring

Making sure the services are not

• Prevents platform coupling

• Prevents URL coupling

Stock Quotes Service

Stock Quotes Service Stock Quotes Service

REST API REST API

The application’s state is stored

Send user / password

Store user for future use

• Grow and shrink as needed

• Scale Up vs Scale Out

• Scale Out is usually preferred

• Allows the system to function properly when resource

• A system with more than one server

• When a server goes down, the other continue

Distributes Load Scalability

Log In Service Log In Service Log In Service

Store user for future use

Check user / password

Stored user details

1 2 Check for user details 3

• Always use stateless architecture

• Supports Scalability and Redundancy

Bring data closer to its consumer

DAL • Serialize to Objects

• Compile the SQL

Cache Poor Excellent

Cache should hold data that is

Cache should hold data that is

Cache should hold data that is

Update Stock API Update Stock API Update Stock API

Stock Quotes Stock Quotes Stock Quotes

Cache Cache Cache

Stock Quotes Stock Quotes Stock Quotes

• Existing libraries • External product

Update Stock API Update Stock API Update Stock API

Stock Quotes Stock Quotes Stock Quotes

• Develop Sync Mechanism

Update Stock API Update Stock API Update Stock API

Stock Quotes Stock Quotes Stock Quotes

• Data immediately synced

Distributed Cache In-Memory, In-Process Cache

Distributed Cache In-Memory, In-Process Cache

• Not just REST API

• Feedback & Reliability

• De-Facto Standard for HTTP-based systems

Performance Very Fast

Performance Very Fast

Performance Very Fast

Performance Very Fast

Performance Very Fast

Performance Very Fast

Client Real-Time Communication: