What services could be used to reduce the elaboration time and improve the availability of the solution?

By:
In: SAP-C01
Tagged:
With: 2 Comments
You have a periodic image analysis application that gets some files in input, analyzes them and tor each file writes some data in output to a ten file the number of files in input per day is high and concentrated in a few hours of the day.Currently you have a server on EC2 with a large EBS volume that hosts the input data and the results. It takes almost 20 hours per day to complete the process.

What services could be used to reduce the elaboration time and improve the availability of the solution?

S3 to store I/O files. SQS to distribute elaboration commands to a group of hosts working in parallel. Auto scaling to dynamically size the group of hosts depending on the length of the SQS queue

EBS with Provisioned IOPS (PIOPS) to store I/O files. SNS to distribute elaboration commands to a group of hosts working in parallel Auto Scaling to dynamically size the group of hosts depending on the number of SNS notifications

S3 to store I/O files, SNS to distribute evaporation commands to a group of hosts working in parallel. Auto scaling to dynamically size the group of hosts depending on the number of SNS notifications

EBS with Provisioned IOPS (PIOPS) to store I/O files SQS to distribute elaboration commands to a group of hosts working in parallel Auto Scaling to dynamically size the group ot hosts depending on the length of the SQS queue.

Explanations:

S3 is suitable for storing large I/O files as it’s highly scalable and durable. SQS can distribute tasks efficiently, while Auto Scaling dynamically adjusts resources based on queue length, reducing processing time.

EBS with Provisioned IOPS provides fast storage but is more limited in scalability and cost-effective high-throughput storage than S3. SNS is more suitable for pub/sub scenarios and not designed for handling task queues.

S3 is suitable for storage, but SNS is primarily a notification service, not a task queue like SQS, so it may not ensure task distribution for parallel processing.

While EBS with Provisioned IOPS is high-performance, it lacks the scalability of S3. SQS is correctly chosen for task distribution, but S3 is a better choice than EBS for input/output files given the high volume.

2026-04-06

2 Comments

  1. Adam
    Author

    I opine that the answer is:
    S3 to store I/O files. SQS to distribute elaboration commands to a group of hosts working in parallel. Auto scaling to dynamically size the group of hosts depending on the length of the SQS queue

  2. Thomas
    Author

    From my perspective, the answer is:
    S3 to store I/O files. SQS to distribute elaboration commands to a group of hosts working in parallel. Auto scaling to dynamically size the group of hosts depending on the length of the SQS queue

Leave a Reply

Your email address will not be published. Required fields are marked *

10 − seven =