Data Migration on your mind, get the right service to move data in AWS.

Most of us know that humungous data is being processed & analyzed in AWS, far more efficiently & cost effectively than on-premises. But what about the data that already exists in private data centers that is being used by legacy applications. How can you unlock the potential of on-premises data? The answer is simple, move/migrate the data to cloud by using the right AWS service.

In my previous blog (AWS Cloud Storage: Which one is right for my data?), I talked about various storage options & the right one for your data.

This blog articulates various options (AWS services) that can help you to move data in AWS. It goes further to help you identify which is the right service for your data. Finally, this will certainly benefit those who are preparing for AWS certification.

There are two broad category for data migration options:

a) online data transfer

b) offline data transfer

Usable network bandwidth is a key consideration when choosing between using online or offline transfer mechanisms. Below table helps take that decision.

Reference: AWS Data Migration Tech Talks

Online Transfer:

AWS DataSync: AWS DataSync fully automates and accelerates moving large active datasets to AWS, up to 10 times faster than command line tools. It is natively integrated with Amazon S3, Amazon EFS, Amazon FSx for Windows File Server, Amazon CloudWatch, and AWS CloudTrail, which provides seamless and secure access to your storage services, as well as detailed monitoring of the transfer.

DataSync uses a purpose-built network protocol and scale-out architecture to transfer data.

DataSync is ideal for customers who need online migrations for active data sets, timely transfers for continuously generated data, or replication for business continuity.

If you want an accelerated and automated data transfer between existing on-premises storage systems (NFS servers or SMB file shares) and Amazon S3, Amazon EFS, and Amazon FSx for Windows File Server, use AWS DataSync.

AWS Transfer for SFTP: If you currently use SFTP to exchange data with third parties, AWS Transfer for SFTP provides a fully managed SFTP transfer directly into and out of Amazon S3, while reducing your operational burden.

With AWS Transfer for SFTP, you get access to a file transfer protocol-enabled server in AWS without the need to run any server infrastructure. You can use this service to migrate your file transfer-based workflows to AWS while maintaining your end users' clients and configurations as is.

S3 Transfer Acceleration: S3 Transfer Acceleration enables fast, easy, and secure transfers of files over long distances between your client and an S3 bucket. Transfer Acceleration takes advantage of Amazon CloudFront’s globally distributed edge locations. As the data arrives at an edge location, data is routed to Amazon S3 over an optimized network path.

If your applications are already integrated with the Amazon S3 API, and you want higher throughput for transferring large files to S3, you can use S3 Transfer Acceleration.

Offline Transfer:

AWS Snowball: AWS Snowball service uses physical storage devices to transfer large amounts (80 TB and 50 TB) of data between Amazon S3 and your onsite data storage location at faster-than-internet speeds. By working with AWS Snowball, you can save time and money.

AWS Snowball Edge: AWS Snowball Edge is a type of Snowball device with on-board storage and compute power for select AWS capabilities. Snowball Edge can undertake local processing and edge-computing workloads in addition to transferring data between your local environment and the AWS Cloud.

Each Snowball Edge device can transport data (up to 100 TB) at speeds faster than the internet. This transport is done by shipping the data in the appliances through a regional carrier. The appliances are rugged shipping containers, complete with E Ink shipping labels.

The AWS Snowball Edge device differs from the standard Snowball because it can bring the power of the AWS Cloud to your on-premises location, with local storage and compute functionality.

AWS Snowmobile: AWS Snowmobile is an Exabyte-scale data transfer service used to move extremely large amounts of data to AWS. You can transfer up to 100PB per Snowmobile, a 45-foot long ruggedized shipping container, pulled by a semi-trailer truck. Snowmobile makes it easy to move massive volumes of data to the cloud, including video libraries, image repositories, or even a complete data center migration.

Narendra Dharm is a founder of Cloud++(www.cloud-plusplus.com) that serves customers to build capabilities in AWS cloud through AWS consulting, AWS training & AWS certification training.