AWS SQS

AWS SQS is the default queue/message broker in AWS. Many aspects of WebSemaphore are inspired by SQS. WebSemaphore shares the philosophy of SQS and other services tailored to a class of use cases without attempting to serve all needs at once.

Pros: easy setup and usage, fully serverless, high availablility.

Cons: SQS is defined as a Simple Queueing Service and as such is limited in its functionality when compared to larger message broker systems. It's debatable whether it's a con since SQS is perfect for usecases that don't go beyond its capabilities.

Concurrency features: Unlike some tools compared here, SQS provides concurrency support in the same sense as WebSemaphore. Namely, when using SQS to invoke lambdas automatically as messages arrive, it is possible to specify the exact concurrency limit and AWS infrastructure will observe that the number of concurrent invocations never exceed the specified number. It will also observe that the next message is processed immediately when concurrent capacity allows, i.e. when one of the currently processing lambda instances terminates, thereby ensuring no capacity sits idle when there is work to do.

The limitations of this approach surface when attempting to execute jobs that take longer than 15 minutes to complete, since 15m is the max execution time limit for AWS Lambda in general. Another subeset of this case is when the actual job is performed outside the lambda and all the lambda does is make a request to a remote system only to wait for a response. When the time to deliver the response is in seconds the SQS + Lambda approach is reasonable. However, as task execution time grows it becomes unreasonably costly to keep the lambda running while the actual job is executed elsewhere.

In both cases above, passing the control to another runtime and terminating the lambda terminates SQS's concurrency guarantee.

WebSemaphore helps you in such cases with its untimed, runtime-independent distributed locking capbaility.

See the diagram below for more clarity. Scenario #1 corresponds to a use case where tasks never go over 15 minutes. Scenario #2 provides an alternative design that accounts for longer tasks using WebSemaphore.

References: