AWS DataSync is Fast, Reliable, and Secure

Here’s what you should know

When transferring large amounts of data between on-premise infrastructure and cloud with existing tools and methods, one needs to maintain the following:

• Data encryption
• Data integrity
• Quick movement of data
• Data monitoring
• Maintaining the file’s metadata and ACL at the destination would become difficult and take more time to copy data successfully, and vice versa.

The typical use cases of moving large amounts of data are:

AWS has introduced the Managed DataSync service to overcome problems when fulfilling the above use cases. DataSync is an online data transfer service which automates movement of data between on-premise and cloud storage such as S3 or EFS and pay for the data that you transfer. It takes care of the common data transfer tasks automatically such as data integrity, encryption during transfer, maintaining security information and fast transfer by optimizing network bandwidth. DataSync service will also enables one-time, recurring data transfer, replication backup and recovery.

DataSync service is an agent-based solution that makes use of existing storage or file system or Network File share protocol.

The benefits of DataSync service are:

• Simplifying the data transfer securely along with data integrity
• Faster movement of data via multi-part, parallel uploads
• Reducing operational costs of data transfer

How AWS DataSync works:

DataSync is an agent-based solution that requires you to install an agent in the on-premise infrastructure that connects to existing on-premise storage or Network File Share protocol.

Here are the steps for enabling data transfer from on-premise to cloud.

1. Log in to the AWS Console, navigate to DataSync and get started with the data transfer tasks.

2. As part of this, a DataSync agent needs to be installed for the use case such as on-premise to AWS or AWS to on-premise.

3. Create an Agent and use the agent key to get registered with DataSync service after installation at the on-premise side.

4. Once the agent is enabled, a task should be created for the data transfer. For the source, select the location at the on-premise side and the storage system.

5. Once the source location is selected, AWS destination needs to be selected such as S3 or EFS. For S3, the destination bucket needs to be selected along with the IAM Role for the S3 access.

6. DataSync service enables you to select options such as copy permissions, metadata and Network bandwidth to be used which will control the data transfer. Once the options are selected, it creates tasks to start the copy.

7. Once the task is created, start the task that initiates the data transfer.

8. The following is the life cycle for the task that transfers the data.

9. The tasks status can be monitored from the history dashboard.
10. DataSync can be integrated with CloudWatch which monitors the data transfers and notifies the status.

Conclusion:

If you’re looking to enhance your online data transfer by 10 times, faster than any other open-source tools, AWS DataSync is the best option. Also, getting started with DataSync is simple — just deploy the DataSync agent on-premise, connect it to a file system, select your AWS storage, and initiate the data transfer.