Systems Integration Scenarios
1. Intro
WebSemaphore helps achieve lossless amortization of traffic to comply with requirements such as concurrent request/execution limit or exclusive access to a resource. The concept of amortization should be understandable from the chart below. Please see the operating modes section for a complete comparison of async and sync operating modes.
Legend:
Fig 2.1 WebSemaphore's impact on traffic in async mode. Note how the lockValue (red) extends far beyond the traffic spikes (purple) consuming the waiting requests (yellow). See more charts and explanations in the operating modes section.
In this section we focus on scenarios using the asynchronous access mode.
2. Assumptions
We hope WebSemaphore fits as many use cases as possible, but integrations that have the following traits will more likely benefit from the features:
- The integration is expected to be lossless. In cases when response time is preferred over 100% quality such as media or telephony, this requirement may be redundant for the core application.
- The order of requests will be maintained. In other words, requests will be processed on a FIFO/FCFS basis.
- The communication is expected to be near-realtime. When similar flows are performed in batch (e.g. scheduled ETL) mode it may be easier to control the number of processes performing a task since task invocation is tightly controlled by the scheduling mechanism. Not so in realtime where the integration traffic depends on business activity such as users performing tasks in business applications or inbound requests from external systems. For periods when demand exceeds capacity the delivery becomes eventual.
- Eventual delivery/consistency is assumed to be acceptable. During peak times the solution will prefer to not "fail fast" but rather deliver eventually.
To summarize the above: the integration requirements are assumed to favor guaranteed delivery over near-realtime delivery over immediate data consistency.
3. Usage Scenarios
While the variety of use cases WebSemaphore can support is potentially endless, it boils down to the following high-level scenarios.
3.1 Provider is external to consumer
This is the most common scenario. The consumer is invoking an API that has a limit on the number of concurrent connections. When the consumer's traffic is significantly lower than the limit there is no direct benefit in queueing the requests. However if the traffic increases occasionally or regularly, the consumer will need to implement a queueing solution that will need to consider resource allocation and release. WebSemaphore reduces this to an acquire and release calls where the queueing and resource allocation is performed internally by WebSemaphore.
The following represents the solution using WebSemaphore
- The Consumer requests a lock from WebSemaphore
- WebSemaphore queues the request
- When the resource is available, WebSemaphore grants the lock to the client by invoking a callback endpoint implemented by the Consumer
- In the process invoked by the callback the Consumer uses the external resource, processes the job.
- The client releases the semaphore.
Fig 2. Provider is external to customer
3.2 Provider is internal to consumer
This scenario is similar to scenario 1, but involves a system internal to the business, such as a high-cost computation resource, a physical device or a legacy system. In such scenario WebSemphore will serve as a queueing mechanism and keep access count to prevent simulatenous access to the limited resource. Since the limited resource is internal to the business it is easy to wrap the resource API in a WebSemaphore and avoid the callback phase (step 4 in scenario 1), instead providing the results to the consumer in an asynchronous fashion. Note: If communication of results from the service is required, it will take the shape of another callback. The only real difference from scenario 1 is that the responsibility for releasing the lock is shifted to the scope (=dev team) of the limited resource.
The flow for this scenario:
- The Consumer requests the Provider service encompassing the limited resource masked by WebSemaphore
- WebSemaphore queues the request
- When the resource is available, WebSemaphore grants the lock and invokes a callback endpoint implemented by the Provider
- In the process invoked by the callback the Provider performes the task over the limited resource, processes the job.
- The Provider communicates the result back to the consumer.
- The Provider releases the semaphore.
Fig 3. Provider is internal to customer
3.3 Provider implements WebSemaphore
This scenario is similar to scenarios 1 and 2 but caters more to the provider side. By implementing WebSemaphore in front of their API, providers gain a straightforward solution to avoid overwhelming their core system while ensuring no requests are lost, even during peak times. This approach requires less resource provisioning, resulting in reduced undercapacity issues and no loss of business opportunities for providers. On the other hand, consumers benefit from not having to deal with the implementation intricacies of WebSemaphore as these details are handled by the provider.
Note, since this enforces an asynchronous communication pattern, the Provider will need a way to communicate the response back to the Consumer, which similarly to the note in scenario 2, means that the essence of this scenario is that the responsibility for managing resource allocation and release is shifted to the Provider.
The steps for the flow are exactly as for flow 2. The difference is in the split of responsibility/ownership.
- The Consumer requests the Provider service encompassing the limited resource masked by WebSemaphore
- WebSemaphore queues the request
- When the resource is available, WebSemaphore grants the lock and invokes a callback endpoint implemented by the Provider
- In the process invoked by the callback the Provider performes the task over the limited resource, processes the job.
- The Provider communicates the result back to the consumer.
- The Provider releases the semaphore.
Fig 4. Provider implements WebSemaphore
3.4 Conclusion
The async WebSemaphore is no silver bullet that can magically increase the capacity of anything. However under matching circumstances it will contribute to more efficient packing of processes in time to maximize resource usage while staying within the allocated resource limits.
In the bottom line this means that when planning the provider capacity, using async WebSemaphore it is sufficient to support enough capacity to handle median load rather than always plan for the maximum. This holds true both time-globally when no autoscaling is present and time-locally when there is autoscaling in place.
