Which solution will meet these requirements with the LEAST operational overhead?
Create an AWS DataSync location for both the destination S3 bucket and the EFS file system. Create a task for the destination S3 bucket and the EFS file system. Set the transfer mode to transfer only data that has changed.
Create an AWS Lambda function. Mount the file system to the function. Set up an S3 event notification to invoke the function when files are created and changed in Amazon S3. Configure the function to copy files to the file system and the destination S3 bucket.
Create an AWS DataSync location for both the destination S3 bucket and the EFS file system. Create a task for the destination S3 bucket and the EFS file system. Set the transfer mode to transfer all data.
Launch an Amazon EC2 instance in the same VPC as the file system. Mount the file system. Create a script to routinely synchronize all objects that changed in the origin S3 bucket to the destination S3 bucket and the mounted file system.
Explanations:
AWS DataSync is designed for transferring data between S3 and EFS efficiently. By creating locations for both the destination S3 bucket and the EFS file system and setting the transfer mode to only transfer changed data, it minimizes operational overhead and ensures that only updated files are copied, meeting the requirement for continuous synchronization with minimal manual intervention.
While an AWS Lambda function can respond to S3 events and copy files, it may introduce higher operational overhead due to the complexity of managing the function, handling potential errors, and ensuring that file copying happens correctly on each event. Additionally, Lambda has a timeout limit that might not be suitable for large file transfers.
Setting the transfer mode to transfer all data in DataSync will result in unnecessary overhead as it will copy all files every time, regardless of whether they have changed. This does not align with the requirement to overwrite only if the source file changes and increases operational overhead.
Launching an EC2 instance to run a script for synchronizing files requires more management, monitoring, and maintenance compared to AWS DataSync. This option also does not automatically handle file changes and would require a scheduled job, increasing operational complexity.
I calculate that the answer is:
Create an AWS DataSync location for both the destination S3 bucket and the EFS file system. Create a task for the destination S3 bucket and the EFS file system. Set the transfer mode to transfer only data that has changed.
It seems to me that the answer is:
Create an AWS DataSync location for both the destination S3 bucket and the EFS file system. Create a task for the destination S3 bucket and the EFS file system. Set the transfer mode to transfer only data that has changed.