Which solution would be MOST efficient?
Use Amazon Kinesis Data Streams for real-time events with a partition for each equipment asset. Use Amazon Kinesis Data Firehose to save data to Amazon S3.
Use Amazon Kinesis Data Streams for real-time events with a shard for each equipment asset. Use Amazon Kinesis Data Firehose to save data to Amazon Elastic Block Store (Amazon EBS).
Use an Amazon SQS FIFO queue for real-time events with one queue for each equipment asset. Trigger an AWS Lambda function for the SQS queue to save data to Amazon Elastic File System (Amazon EFS).
Use an Amazon SQS standard queue for real-time events with one queue for each equipment asset. Trigger an AWS Lambda function from the SQS queue to save data to Amazon S3.
Explanations:
Amazon Kinesis Data Streams can efficiently handle real-time data ingestion with ordered delivery through partitions. Using Kinesis Data Firehose to store data in S3 is also effective for further processing.
While Kinesis Data Streams can use shards, using a shard for each equipment asset may lead to scalability issues and complexity. Additionally, storing data in EBS is less suited for long-term storage compared to S3.
An SQS FIFO queue can maintain order, but having one queue per asset can lead to management overhead. Furthermore, EFS is not optimized for saving event data compared to S3.
An SQS standard queue does not guarantee message order, which is critical for real-time events. Triggering a Lambda function to save data to S3 is valid, but the lack of ordering makes this option less effective.
From what I’ve seen, the answer is:
Use Amazon Kinesis Data Streams for real-time events with a partition for each equipment asset. Use Amazon Kinesis Data Firehose to save data to Amazon S3.
I categorize that the answer is:
Use Amazon Kinesis Data Streams for real-time events with a partition for each equipment asset. Use Amazon Kinesis Data Firehose to save data to Amazon S3.