CDP:Job Observer Pattern

Job Monitoring and Adding/Deleting Servers

Problem to Be Solved

There is a technique for load distribution in batch processing where you can control job requests using queues, where the job requests that are queued are processed in parallel by multiple batch servers. However, because, if Cloud technology is not used, the number of batch servers that are provided must be up to handling the peak requirements, there will be excessive batch server resources in time bands other than the peak, which is harmful in terms of cost effectiveness. On top of this, the responsiveness may suffer if there is an unanticipatedly large load on the batch system.

Explanation of the Cloud Solution/Pattern

Because, in the past, the server resources could not the increased or decreased dynamically, the batch servers were provided considering the peak demand and the allowable cost. This resulted in problems with poor cost effectiveness and the inability to handle any unanticipated load. In the AWS Cloud, however, a system is provided where you can increase or decrease the number of virtual servers automatically by monitoring the load. You can use this system to increase or decrease the number of batch servers depending on the load, resulting in excellent cost effectiveness and enabling handling of unanticipated loads. Specifically, the volume of job requests (queued messages) is monitored, and batch servers are added or deleted automatically as required.

Implementation

AWS has a system known as "Auto Scaling" that is able to increase or decrease the number of EC2 instances automatically, and which works in cooperation with a resource monitoring tool known as "CloudWatch" to increase or decrease the EC2 instances in accordance with the value that is monitored. One thing that can be monitored by CloudWatch is the numbers of messages in the queues of the Simple Queue System (SQS) provided by AWS. You can use SQS to manage job requests and use Auto Scaling and CloudWatch to structure a system able to increase or decrease the number of batch servers automatically depending on the number of messages (job requests) within a queue.

• Enqueue job requests as SQS messages.
• Have the batch server dequeue and process messages from SQS.
• Set up Auto Scaling to automatically increase or decrease the number of batch servers, using the number of SQS messages (CloudWatch) as the trigger to do so.

Configuration

Benefits

• This lets you coordinate the number of EC2 instances for job servers with the number of jobs, improving cost effectiveness.
• This lets you reduce the overall time for executing jobs through parallel processing.
• Even if a job server EC2 instance were to fail, the SQS messages (job requests) would remain, enabling processing to be continued immediately upon recovery of the EC2 instance, producing a system that is robust to failure.

Cautions

• EC2 instances are charged by unit time, with a one-hour minimum, so one hour of time will be billed if an EC2 instance is launched and then shut down in a short period of time. Because of this, you need to be aware of the timing of launching and termination of instances.

Other

Queuing Chain Pattern and the Priority Queue Pattern can be combined.