AWS Cloud Storage: Which one is right for my data?

While AWS provides various choices to store data cost effectively, it is important to know the differences between various storage services & their ideal usage pattern. Knowing these AWS cloud storage services & their usage pattern will help AWS solution architects recommend right storage choice to their customers.

This blog will serve as quick read for those preparing for AWS certification.

Let’s categorize various storage choices as follows:

· Object Storage: AWS S3

· Block Storage: AWS EBS

· File Storage: AWS EFS, AWS FSx for windows file server & AWS FSx for Lustre

· Archival Storage: AWS S3 Glacier

Simple Storage Service (S3) is storage for the internet. You can use S3 to store and retrieve any amount of data at any time, from anywhere on the web.

Usage Pattern:

· Storage and distribution of static web content and media.

· Host entire static websites.

· Used as a data store (data lakes) for computation and large-scale analytics, such as analyzing financial transactions, clickstream analytics, and media transcoding.

Elastic Block Store (EBS) provides block level storage volumes for use with EC2 instances. EBS volumes behave like raw, unformatted block devices.

Usage Pattern:

· Used by EC2 instances for data that changes relatively frequently and requires long-term persistence.

· Used as the primary storage for a database or file system, or for any applications that require access to raw block-level storage.

· EBS Provisioned IOPS volumes are particularly well-suited for use with databases applications that require a high and consistent rate of random disk reads and writes.

Elastic File System (EFS) provides a simple, scalable, fully managed elastic NFS file system for use with AWS Cloud services and on-premises resources.

Usage Pattern:

· Multiple EC2 instances can access an EFS file system at the same time, providing a common data source for workloads and applications running on more than one instance or server.

· EFS file systems can grow to petabyte scale, drive high levels of throughput, and allow massively parallel access from EC2 instances to your data.

FSx provides fully managed third-party file systems with the native compatibility and feature sets for workloads such as Microsoft Windows–based storage, and high-performance computing.

FSx for Windows File Server provides fully managed Microsoft Windows file servers, backed by a fully native Windows file system.

Usage Pattern:

· Windows applications and workloads ideal for FSx include business applications, home directories, web serving, content management, data analytics, software build setups, and media processing workloads.

FSx for Lustre makes it easy and cost-effective to launch and run the popular, high-performance Lustre file system. The open-source Lustre file system is designed for applications that require fast storage that can keep up with your compute. It provides submillisecond latencies, up to hundreds of GBps of throughput, and up to millions of IOPS.

Usage Pattern:

· FSx for Lustre is POSIX-compliant, so you can use your current Linux-based applications without having to make any changes. FSx for Lustre provides a native file system interface and works as any file system does with your Linux operating system.

· You use Lustre for workloads where speed matters, such as machine learning, high performance computing (HPC), video processing, and financial modeling.

S3 Glacier (S3 Glacier), is a storage service optimized for infrequently used data, or cold data. S3 Glacier is an extremely low-cost storage service that provides durable storage with security features for data archiving and backup.

Usage Pattern:

· Archiving offsite enterprise information, media assets, research and scientific data, digital preservation and magnetic tape replacement.

Narendra Dharm is a founder of Cloud++(www.cloud-plusplus.com) that serves customers to build capabilities in AWS cloud through AWS consulting, AWS trainings & AWS certification courses.