Which solution will meet these requirements with the LEAST operational overhead?
Send activity data to an Amazon Kinesis data stream. Configure the stream to deliver the data to an Amazon S3 bucket.
Send activity data to an Amazon Kinesis Data Firehose delivery stream. Configure the stream to deliver the data to an Amazon Redshift cluster.
Place activity data in an Amazon S3 bucket. Configure Amazon S3 to run an AWS Lambda function on the data as the data arrives in the S3 bucket.
Create an ingestion service on Amazon EC2 instances that are spread across multiple Availability Zones. Configure the service to forward data to an Amazon RDS Multi-AZ database.
Explanations:
While Kinesis Data Streams can ingest data in real time, it requires manual management of data processing and delivery to S3, leading to higher operational overhead. Additionally, S3 does not provide direct SQL analytics capabilities without additional services.
Kinesis Data Firehose can automatically deliver data to Amazon Redshift, which allows for on-demand SQL analytics on the ingested data. Firehose handles scaling and data transformation, resulting in minimal operational overhead while providing high availability and performance.
Although this option allows data to be processed as it arrives in S3, it relies on AWS Lambda for processing, which could introduce latency and operational complexity. It also does not provide a direct SQL analytics capability without integrating with a service like Athena or Redshift.
Creating an ingestion service on EC2 introduces significant operational overhead, including management of EC2 instances, scaling, and maintenance. Additionally, using RDS may not be optimal for handling petabyte-scale data due to storage and performa
I would say the answer is:
Send activity data to an Amazon Kinesis Data Firehose delivery stream. Configure the stream to deliver the data to an Amazon Redshift cluster.