Which change will create the required transformed records with the LEAST operational overhead?
Create an AWS Lambda function that can transform the incoming records. Enable data transformation on the ingestion Kinesis Data Firehose delivery stream. Use the Lambda function as the invocation target.
Deploy an Amazon EMR cluster that runs Apache Spark and includes the transformation logic. Use Amazon EventBridge (Amazon CloudWatch Events) to schedule an AWS Lambda function to launch the cluster each day and transform the records that accumulate in Amazon S3. Deliver the transformed records to Amazon S3.
Deploy an Amazon S3 File Gateway in the stores. Update the in-store software to deliver data to the S3 File Gateway. Use a scheduled daily AWS Glue job to transform the data that the S3 File Gateway delivers to Amazon S3.
Launch a fleet of Amazon EC2 instances that include the transformation logic. Configure the EC2 instances with a daily cron job to transform the records that accumulate in Amazon S3. Deliver the transformed records to Amazon S3.
Explanations:
Using a Lambda function with Kinesis Data Firehose for data transformation allows the records to be transformed as they are ingested, minimizing operational overhead.
Using Amazon EMR and EventBridge adds unnecessary complexity and operational overhead for transforming records after they are already stored in S3.
Using an S3 File Gateway and AWS Glue adds complexity and is not as efficient as transforming data at the ingestion stage. This would increase latency and overhead.
Launching EC2 instances with cron jobs to process the data introduces additional infrastructure management and scaling challenges compared to Lambda and Firehose.