Which solution will provide the LARGEST overall cost reduction while meeting these requirements?
Migrate the data from the existing shared file system to an Amazon S3 bucket that uses the S3 Intelligent-Tiering storage class. Before the job runs each month, use Amazon FSx for Lustre to create a new file system with the data from Amazon S3 by using lazy loading. Use the new file system as the shared storage for the duration of the job. Delete the file system when the job is complete.
Migrate the data from the existing shared file system to a large Amazon Elastic Block Store (Amazon EBS) volume with Multi-Attach enabled. Attach the EBS volume to each of the instances by using a user data script in the Auto Scaling group launch template. Use the EBS volume as the shared storage for the duration of the job. Detach the EBS volume when the job is complete.
Migrate the data from the existing shared file system to an Amazon S3 bucket that uses the S3 Standard storage class. Before the job runs each month, use Amazon FSx for Lustre to create a new file system with the data from Amazon S3 by using batch loading. Use the new file system as the shared storage for the duration of the job. Delete the file system when the job is complete.
Migrate the data from the existing shared file system to an Amazon S3 bucket. Before the job runs each month, use AWS Storage Gateway to create a file gateway with the data from Amazon S3. Use the file gateway as the shared storage for the job. Delete the file gateway when the job is complete.
Explanations:
Amazon FSx for Lustre is a high-performance, cost-effective file system suitable for short-term, high-performance workloads. S3 Intelligent-Tiering minimizes storage costs. Lazy loading ensures that only needed data is retrieved, reducing costs.
Amazon EBS with Multi-Attach is not optimized for high-performance parallel access and would incur high costs, as EBS volumes with Multi-Attach are expensive, and there is no automated way to scale the storage cost effectively.
Batch loading from Amazon S3 to FSx for Lustre would increase the time and costs due to the need to load the entire dataset upfront. This approach would not leverage lazy loading, making it less cost-efficient than Option A.
AWS Storage Gateway with file gateway is less performant than FSx for Lustre and is not optimized for high-performance access, which is required in this scenario, making it unsuitable for reducing costs during the high-performance job run.