This tutorial covers how the AWS Simple Queue Service (SQS) service works, how to create messages on the standard queue, process the messages, how to set up a Dead Letter Queue (DLQ) and how to use a First In First Out (FIFO) queue.
The idea is simple – there’s a queue with a producer and consumer. Producers produce messages that go into the queue and the consumers consume those messages. Messages go into the queue and consumers poll the queue to receive those messages. Every time the consumers polls the queue you have to pay AWS.
Long Polling
The idea of long polling is that a consumer can poll the queue and if it doesn’t find a message you could still sit at the queue for up to 20 seconds and still receive a message instantaneously if it comes into the queue. That will reduce costs because you’re not continuously polling for messages and getting nothing. The Receive Message Time parameter is where you set Long Polling when setting up the queue.
Visibility Timeout
Visibility timeout is when a consumer is processing a message there’s this idea that other consumers will not see that message because we don’t want the same message processed more than once by other consumers. So when a consumer is processing that message we can create a visibility timeout to make the message invisible to other consumers.
We don’t want to set it too high because if there’s a problem consuming that message then there’s going to be a long delay before other consumers can see that message. We don’t want to set it too low because if there’s not enough time for the consumer to process the message then that message will become visible to other consumers and the message can be processed more than once.
Dead Letter Queue (DLQ)
The Dead Letter Queue is a secondary queue that receives messages from the first queue after a certain number of times the message wasn’t processed on the main queue. It’s a way to store problematic messages in a separate queue for further analysis.
Standard Queue
In your SQS dashboard, you can create a Standard or FIFO queue. In a standard queue, you are not guaranteed that the order that the messages were produced by the producers are going to be consumed by the consumers. The messages are mostly in order but not guaranteed to be in order (the way a FIFO queue does). When creating a standard queue, you can specify a dead letter queue to use.
FIFO Queue
A FIFO (First In First Out) Queue adds the ability to guarantee the order of messages on the queue and a way to de-duplicate messages.
References