Which solution will run the batch job within 15 minutes with the LEAST operational overhead?
Use AWS Lambda with functional scaling.
Use Amazon Elastic Container Service (Amazon ECS) with AWS Fargate.
Use Amazon Lightsail with AWS Auto Scaling.
Use AWS Batch on Amazon EC2.
Explanations:
AWS Lambda is not suitable for this use case because it has a maximum execution time of 15 minutes. Since the batch job runs every hour and is CPU intensive, it exceeds Lambda’s limitations and would require a different service.
Amazon ECS with AWS Fargate could handle the batch job, but it may involve more operational overhead in terms of container management and scaling policies. While it can run the job effectively, it may not be the simplest or most efficient option for ensuring the job completes in the desired timeframe.
Amazon Lightsail is designed for simpler workloads and may not provide the necessary resources or capabilities to match the performance of the existing on-premises server (64 vCPUs and 512 GiB of memory). Additionally, it lacks the scalability and flexibility needed for CPU-intensive batch processing.
AWS Batch on Amazon EC2 is specifically designed for running batch jobs. It can provision the necessary EC2 instances with the required vCPU and memory to match the performance of the on-premises server. It automates job scheduling and scaling, minimizing operational overhead while ensuring the job completes within the desired time frame.
It appears that the answer is:
Use AWS Batch on Amazon EC2.