CDP:Ondemand Disk Pattern

Dynamically Increasing/Decreasing Disk Capacity

Problem to Be Solved

It is difficult to forecast in advance the disk capacity that will be used by the system. As a result, usually, at the time of system deployment, the designer provides a disk capacity that envisions what the need will be several years out, multiplied by a safety margin. Not only is that disk not fully used right away, but, in reality, might not be used for several years. Nevertheless, that disk still must be paid for.

Disk striping is effective when you need to improve the performance of the disk I/O. However, it is also difficult to estimate the number of disks that will satisfy the requirements when testing the striping, and an up-front investment in hardware is required when performing the testing.

Explanation of the Cloud Solution/Pattern

The AWS Cloud lets you use virtual disks. A virtual disk provides you with whatever capacity you need, whenever you need it.

When you use a virtual disk you will no longer need to make elaborate estimates in advance. Once the system is running, you can secure a disk with the required capacity on-demand while monitoring capacity usage.

Because there is a tendency for the cost of storage to fall year-by-year, securing capacity only as it becomes necessary is extremely beneficial in terms of cost. In the past, a situation where there is a temporary need for increased capacity would require you to buy a high-capacity disk, but with a virtual disk, you can add the disk can when necessary and then eliminate it after use.

Implementation

The virtual disk Elastic Block Store (EBS) in AWS allows you to secure the required capacity at any time, with whatever timing you need.

(Procedure)

• When using EBS, first secure a volume of the minimum required size.
• If you need more disk capacity, take a snapshot of the EBS and create a new EBS based on that snapshot.
• When creating the new EBS, specify a volume of a larger size than the original volume.
• Attach the new EBS to an EC2 instance.
• After attaching, use the Resize command in the file system that is being used (for example, resize2fs) to expand the region to the new capacity.
• When striping, attach multiple EBS volumes and use mdadm or operating system functions to create a software RAID configuration.

Implementation

The virtual disk Elastic Block Store (EBS) in AWS makes it possible to secure the required capacity at any time, with just the right timing.

(Procedure)

• When using EBS, first secure a volume of the minimum required size.
• If more disk capacity is required, take a snapshot of the EBS and create a new EBS based on that snapshot.
• When creating the new EBS, specify a volume of a larger size than the original volume.
• Attach the new EBS to an EC2 instance.
• After attaching, use the Resize command in the file system that is being used (for example, resize2fs) to expand the region to the new capacity.
• When striping, attach multiple EBS volumes and use mdadm or operating system functions to create a software RAID configuration.

Configuration

Benefits

• You can change the volume size later, freeing you from the work involved in making estimations.
• Being able to secure disk capacity when required has benefits in terms of cost.
• Striping can improve disk I/O performance.

Cautions

• Unlike S3, in EBS there is a charge for the disk capacity secured. For example, if a 100 GB region is secured, you will be charged for 100 GB, even if only 5 GB is used.
• The upper limit for disk capacity that the set in a single EBS (as of April 2012) is 1 TB. When capacity in excess of 1 TB is needed in a single partition, attach multiple EBSs and use software such as mdadm to combine them together into a single volume for use.

Other

An EBS is a disk volume that can be used over a network, so using an EC2 instance with a larger size increases the I/O performance. When striping, in particular, you should be aware of the EC2 instance size.