Skip to content

Storage (Elastic Block Store)#

EC2 Instances have several options for storing data in a persistent (and non-persistent) manner via what we call EBS Volumes. We'll talk about these options here.

Types#

With Amazon Elastic Block Store (EBS), you get several storage types that you can choose from. You have to select carefully to ensure you're picking the right one for the workload you intend to run on the storage. These storage types are broken down into two storage "classes": SSDs and HDDs.

There are four storage types we can choose from based on these two technologies:

  • General Purpose SSD
  • Provisioned IOPS SSD
  • Throughput Optimised HDD
  • Cold HDD
  • Magnetic (standard)

Each of these has a type "code", just like EC2 Instances do:

Human Name Code
General Purpose SSD gp2, gp3
Provisioned IOPS SSD io1, io2
Throughput Optimised HDD st1
Cold HDD sc1

Picking the right one depends on the work load. If you need a general purpose, faster disk then gp2 and gp3 are going to be fine. If you need an extremely fast disk, then upgrade to io1 or io2. If you need a lot of space, then st1 is perfect. The sc1 is ideal for low through-put, large data sets you don't access too frequently.

Instance Stores#

Another type of storage that comes with some instance types is the "instance store". It's a piece of storage that is a directly attached to the Instance that does not persist beyond stops, reboots, hardware failure (inside of AWS), or termination of the Instance.

Instance stores do not support Volue encryption. If you need to encrypt the data on the Volume, then you have to use a standard EBS Volume, not an instance store.

The other EBS Volume types do persist rebooting, shutting down (and starting up again) Instances, and beyond, but they can also be configured to terminate (delete) when the Instance they're attached to is also deleted.

Scability and Lateral Movement#

If you're using the latest generation EBS Volume types, attached to a current genertion EC2 Instance, you can scale the Volume in size dynamically, up to higher storage requirements (but not downwards due to how filesystems store information on a block device.) You can also change the provisioned IOPS (Input/Output Operations Per Second) dynamically as the need for faster performance becomes apparant based on traffic Volume.

When you create an EBS Volume, it can actually live well beyond the life cycle of an EC2 Instance. Multiple Instances, actually. This is because you can detach a Volume from one Instance and attach it to another (or not.)

This kind of functionality can be useful if, for example, you want to replicate a Volume to another region and then attach it to an Instance in that region. Or perhaps you want to delete the Instance and replace it with another one without data loss, in which case you can detach the Volume, delete the Instance, create the new Instance, and then attach the Volume to it.

This is one of the powerful features EBS Volumes provide, but keep in mind that an EBS Volume is AZ bound, not VPC or account bound. You have to replicate a Volume to go beyong AZ, VPC or region boundaries.

Encryption#

All EBS Volumes can be encrypted using a AWS Key Management Service (KMS) key. When you encrypt an EBS volume, the following security advantages are granted to you (this is taken directly from AWS' documentation):

  • Data at rest inside the volume
  • All data moving between the volume and the instance
  • All snapshots created from the volume
  • All volumes created from those snapshots

All encryption uses AES-256. Latency isn't really a thing at all, so it's safe to use the encryption functionality without it reducing performance of the EBS Volume itself.

Encryption isn't available on all Volume types, nor is it available on all Instance types. Make sure you're reviewing what Instance type you intend on using before you plan your encryption strategy.

When you encrypt an EBS Volume, all future snapshots of that Volume will be encrypted too (and with the same KMS Key.)

Note

We have worked with AWS KMS yet, and thankfully there is no reason to at this point in time. AWS EBS will automatically create an AWS managed KMS key for you when you start using EBS resources in a particular region. This (default) KMS key is used for all encryption operations when you request them. You can, of course, create and manage your own KMS keys, but there's really no need to in most cases.

Snapshots#

After you've got an EBS Volume working for you, it's possible to take snapshots of it. These offer us a few key benefits:

  • Backups of the Volume itself at a specific point in time
  • Cross region replication
  • AMI creation

These tools make EBS a powerful tool. Not only can it store your data for you in a consistent manner, but it can back it up, restore it, copy it to entirely different regions, and more.

And again, if you encrypt a Volume, the Snapshot will be encrypted as well.

We'll explore snapshots in more details when we come to create AMIs later on.

Creating an EBS Volume#

That's enough theory for now. Let's create a simple, small EBS Volume (and then pretend to create some other ones later on).

Login to the EC2 console and head to "Elastic Block Store", and click "Volumes":

Create EBS

Create EBS

  1. Click this option to get to your EBS Volumes
  2. And here you'll see any and all EBS Volumes that exist in your account

You'll likely have zero EBS Volumes (unless you've been experimenting, which is great!). I have one at the moment. You can see it's a gp2, which is an SSD based Volume. It's 8 GiB in size and provides 100 IOPS. It's a very generic, basic Volume.

Click "Create volume". We'll step through the options:

Create EBS

Create EBS

  1. We can define the type we want here - gp2 is the baseline, basic SSD
  2. The size in GiB (GBs; gigabytes) - notice the minimum and maxiumum sizes
  3. IOPS for a gp2 is fixed in place, but gp3 allows you to configure it (we'll check this out)
  4. Throughout isn't applicable here as we're provising a gp2 Volume
  5. Remember, Volumes are AZ specific
  6. You can create a Volume from a snapshot, allowing you to recover a backup and create an Instance from it
  7. And you can define if this Volume is encrypted or not

Note

Go ahead and define the tags (which can't be seen in the screenshot, but they're obvious in the UI): Name and EducationOnly.

When I say that a gp2 Volume's IOPS is fixed in place, I don't mean at 100 IOPS. AWS scales a gp2 Volume's IOPS per-GB provisioned. For every GB of space provisioned, you're given 3 IOPS. If you provision 10GB, then you (mathematically) get 30 IOPS, but AWS has a minimum of 100 IOPS, so it's rounded up until you provision around 34GB or more. Change the 10GB to 100GB and you'll get 300 / 3000 IOPS. Also note that 100 / 3000 IOPS in the UI means the disk can burst to 3,000 IOPS for short intervals. Ideal for spikes in traffic. If you need 3,000 IOPS constantly, you're best going with a gp3 or io1/io2 Volume.

Once created, click on the "Volume ID" to bring up its details:

Create EBS

Create EBS

  1. Our Volume ID, as expected (almost everything in AWS has an ID)
  2. The KMS Key ID, had we used one (we didn't opt for encryption)
  3. The AZ we provisioned to
  4. The size we requested
  5. The IOPS we get for a gp2 SSD (fixed IOPS)
  6. And the type, a gp2

Go ahead and goto the "Create volume" UI again, but this time we're just going to look at the options.

Change the type to gp3. Look how the various options change for us:

Create EBS

Create EBS

I've changed a few options (for no particular reason other than you can.) The IOPS option now starts at 3,000 IOPS, even if you provision a 10 GiB disk. You can go as high as 16,000 IOPS. You'll notice there is no bursting with a gp3 - you basically provision exactly what you need between 3,000 and 16,000 IOPS.

The throughout option also comes to life and is made available to us. IOPS doesn't equate speed because you can perform a lot of I/O operations on a Volume without transferring any data at all. Throughput is about actual data transfer rates, and with a gp3 you can go as high as 1,000 MiB per-second.

Put another way, IOPS is the amount of read/write operations per second you perform, but throughput is the actual amount of bits that are read/write per second and transferred from the Volume to the Instance.

Change the type again to io2. Notice the warning that pops up:

Provisioned IOPS SSD (io2) volumes with a size greater than 16 TiB, IOPS greater than 64,000, or IOPS:GiB ratio greater than 500:1 are supported only with instance types that support io2 Block Express.

At that point, you're dealing with seriously big and fast demands on the underlaying hardware. 64,000 IOPS? That's insane, but definitely understandable for some workloads.

With the io2 type, we can scale the IOPS directly to the GiB we provision. In the case of io1 and io2 Volume types, we're chiefly concerned with the IOPS only performance, and not throughput, which is why that option is now not avialable again.

Depending on workloads, you'll have to decide on the right Volume type to use.

Right now you know enough about EBS Volumes to move on. We'll most certainly revisit EBS Volumes in more details in later, more advanced courses.