AMAZON SQS

Guided By Presented By
NANDINI GOWDA P MANASA SB
 Contents
1.Introduction
2.Basic architecture
3. Types of amazon SQS
4. Standard Queue
5. Fifo queue
6. Examples of Implementations of SQS
7. Benefits of SQS
8. Decoupling of SQS
9. Conclusion
 INTRODUCTION
 Amazon Sqs is a web service that gives you acess to a message
queue that can be used to store messages while waiting for a
computer to process

 Amazon SQS is distributed queue system that enables web

service applications to quickly and reliable messages that one
component in the application generates to be consumed by
another component .
 Basic Amazon SQS Architecture
Distributed Queues
 There are three main parts in a distributed messaging system: the
components of your distributed system, your queue (distributed on
Amazon SQS servers), and the messages in the queue.

 In the following scenario, your system has

several producers (components that send messages to the queue)
and consumers (components that receive messages from the queue).
Message Lifecycle
The following scenario describes the lifecycle of an Amazon SQS message in a
queue, from creation to deletion.
 - A producer (component 1) sends message A to a queue, and the message is
distributed across the Amazon SQS servers redundantly.

 - When a consumer (component 2) is ready to process messages, it

consumes messages from the queue, and message A is returned. While
message A is being processed, it remains in the queue and isn't returned to
subsequent receive requests for the duration of the visibility timeout.

 - The consumer (component 2) deletes message A from the queue to prevent

the message from being received and processed again when the visibility
timeout expires.
 Types of Amazon SQS
 The queue acts as a buffer between the components
producing and saving data and the components
receive the data for processing.

 This means the queue resolves issues that arises if the

producer is producing work faster than the consumer
can process it.
 There are two types of queues :

 Standard Queues (Default)

 2. FIFO Queues
 Standard Queues

 Standard Queues support nearly unlimited number of

transsactions per second (TPS) per API action (send
message, delete message or delete message).

 Standard queues support atleast once message

delivery

 Occasionally, because of highly distribute

architecture that allows unlimited through put.
 We get to know that more than one copy of a message
might be delievered out of order.

 Standard Queues provide best – effort ordering which

ensures that messages are generally delievered in the
same order they are sent.

 These queues can be implemented in many scenarios,

as long as your application can process messages that
arrive more than once in out of order.
Examples:
 - Decouple live user requests from intensive background work –
Let users upload media while resizing or encoding it.

 - Allocate tasks to multiple worker nodes – Process a high number

of credit card validation requests.

 - Batch messages for future processing – Schedule multiple

entries to be added to a database.
 Summary of Standard Queue
 Unlimited Throughput:
Standard queues support a nearly unlimited number of
transactions per second (TPS) per API action.

 At-Least-Once Delivery:
A message is delivered at least once, but occasionally more
than one copy of a message is delivered.

 Best-Effort Ordering:
Occasionally, messages might be delivered in an order
different from which they were sent.(PTO FOR THE FIG)
 Amazon SQS FIFO (First-In-First-Out) Queues

 FIFO queues have all the capabilities of the 

standard queue.

 By default, with batching, FIFO queues support up to

3,000 messages per second (TPS), per API action
(SendMessageBatch, ReceiveMessage,
or DeleteMessageBatch).
 Sending message and receiving it are strictly
preserved and are delivered once, they remain
available to the consumer and then it gets deleted.

 Duplicates are not allowed in the queue. They

support a group of messages that inturn allow
multiple ordered message in a single queue.
FIFO (First-In-First-Out) queues are designed to enhance messaging between
applications when the order of operations and events is critical, or where
duplicates can't be tolerated, for example:

 Ensure that user-entered commands are executed in the right order.

 Display the correct product price by sending price modifications in the right
order.

 Prevent a student from enrolling in a course before registering for an

account.

 FIFO queues also provide exactly-once processing but have a limited

number of transactions per second (TPS)
 Summary of FIFO
 High Throughput:
By default, FIFO queues support up to 300 messages per second (300 send,
receive, or delete operations per second). When you batch 10 messages per
operation (maximum), FIFO queues can support up to 3,000 messages per
second. To request a quota increase, file a support request.

 Exactly-Once Processing:
A message is delivered once and remains available until a consumer processes
and deletes it. Duplicates aren't introduced into the queue.
Examples of
Implementations
of SQS
 Priority Queue Pattern
Implementation
 Use SQS to prepare multiple queues for the individual priority levels.

 Place those processes to be executed immediately (job requests) in the

high priority queue.

 Prepare numbers of batch servers, for processing the job requests of the
queues, depending on the priority levels.

 Queues have a message "Delayed Send" function. You can use this to
delay the time for starting a process.
 SQS Job Server Pattern
Implementation
 Enqueue job requests as SQS messages.

 Have the batch server dequeue and process messages from

SQS.

 Set up Auto Scaling to automatically increase or decrease the

number of batch servers, using the number of SQS messages
(CloudWatch) as the trigger to do so.
 Processing high volume big data
concurrently with no duplicates using
AWS SQS
Implementations

 A simple use case for the below architecture could be building an end-
end messaging service, or sending out transactional emails.

 In both the above use cases, a highly concurrent processing with no

duplicates is needed.
 We have a Lambda function that writes messages to the
Standard queue in batches.
 This writer function is invoked when a file is posted to S3.
 While there are messages in the queue, Lambda polls the
queue, reads messages in batches and invokes the Processor
function synchronously with an event that contains queue
messages.
 The processing function is invoked once for each batch.
 When the function successfully processes a batch, Lambda
deletes its messages from the queue. 
 SQS vs SNS for Lambda Dead Letter Queues

There are a huge number of different patterns (and anti-patterns) out

there for implementing SQS and SNS, as well as Lamba and event-
driven patterns in general.  The two above are just a basic representation
that work well in certain scenarios.  I’d be really interested to hear from
other people who have worked with serverless/event-driven on AWS and
what your opinions are, as well as any patterns you’ve found to be a
good way of managing DLQs.
First-In-First-Out Delivery:
The order in which messages are sent and received is strictly preserved (i.e. First-In-
First-Out).
 Benefits of SQS
- Eliminate administrative overhead

 AWS manages all ongoing operations and underlying infrastructure

needed to provide a highly available and scalable message queuing
service.

 SQS queues are dynamically created and scale automatically so you

can build and grow applications quickly and efficiently.
- Reliably deliver messages

 Use Amazon SQS to transmit any volume of data, at any level of

throughput, without losing messages or requiring other services to be
available.

 Multiple copies of every message are stored redundantly across

multiple availability zones so that they are available whenever needed.
- Keep sensitive data secure

 You can use Amazon SQS to exchange sensitive data between

applications using server-side encryption (SSE) to encrypt each
message body.

 AWS KMS logs every use of your encryption keys to AWS CloudTrail
to help meet your regulatory and compliance needs.
 Synchronous and Asynchronous AWS Decoupling Solutions

 Coupling is a measure of how closely connected two

functions, modules or applications are.

 Low coupling is often a good sign of a well architected

system and good design practice since it allows you to
lower cost of maintenance and high readability of code.
Synchronous
 In order to achieve synchronous decoupling an Elastic Load
Balancing (ELB) could be used.

 An ELB distributes incoming application traffic across

multiple EC2 instances, in multiple Availability Zones.

 You can add and remove instances from your load balancer
as your needs change, without disrupting the overall flow of
requests to your application.
There are two types of ELBs:
 Application load balancers: work in between your web
servers and the application servers, generally doing content-
based routing.
-The nodes of an internal load balancer have only private IP addresses.
-Therefore, internal load balancers can only route requests from clients
with access to the VPC for the load balancer.

 Classic Load balancer: work in between the public Internet

and your web servers.
Asynchronous

 For asynchronous decoupling, AWS offers a Simple Queue

Service (SQS) that provides a messaging queue infrastructure.

 By using Amazon SQS, you can move data between

distributed components of your applications that perform
different tasks without losing messages or requiring each
component to be always available.
 CONCLUSION
 Dead letter queues refers to the queues which can used to debug application or
messaging system since they give user the ability to separate messages that cause
problems and determine the cause behind this.

 Cost-allocation tags can ne used to organise and identify the queues purpose, its
owner and its environment.This comes in handy when there are many queues
which need to be handeled.

 Hence we understood how SQS can be used to efficiently manage messages that
flow through queues.
 THANK YOU
FOR YOUR
COOPERATION