Which solution meets these requirements with the LEAST operational overhead?
Configure Amazon EMR to read text files from Amazon S3. Run processing scripts to transform the data. Store the resulting JSON file in an Amazon Aurora DB cluster.
Configure Amazon S3 to send an event notification to an Amazon Simple Queue Service (Amazon SQS) queue. Use Amazon EC2 instances to read from the queue and process the data. Store the resulting JSON file in Amazon DynamoDB.
Configure Amazon S3 to send an event notification to an Amazon Simple Queue Service (Amazon SQS) queue. Use an AWS Lambda function to read from the queue and process the data. Store the resulting JSON file in Amazon DynamoDB.
Configure Amazon EventBridge (Amazon CloudWatch Events) to send an event to Amazon Kinesis Data Streams when a new file is uploaded. Use an AWS Lambda function to consume the event from the stream and process the data. Store the resulting JSON file in Amazon Aurora DB cluster.
Explanations:
Amazon EMR is an overkill for small, simple processing tasks and requires significant operational overhead. It is designed for large-scale processing, not for lightweight tasks. Storing the results in Amazon Aurora adds complexity and operational overhead compared to a serverless solution.
While using SQS and EC2 is a valid approach, it introduces more operational overhead due to managing EC2 instances. Additionally, EC2 scaling is less efficient for varying demand compared to serverless solutions like AWS Lambda.
This solution is serverless and scales automatically with the number of file uploads. Using S3 event notifications to trigger an AWS Lambda function is a highly efficient way to process files in near real time with minimal operational overhead. Storing the JSON in DynamoDB is appropriate for quick access and analysis.
Using Kinesis Data Streams adds unnecessary complexity for simple file processing tasks. While Lambda can process events from Kinesis, it’s not the optimal solution compared to directly using S3 events. Additionally, storing the results in Aurora DB is over-complicated for the use case.
I arrange that the answer is:
Configure Amazon S3 to send an event notification to an Amazon Simple Queue Service (Amazon SQS) queue. Use an AWS Lambda function to read from the queue and process the data. Store the resulting JSON file in Amazon DynamoDB.