What else should the solutions architect recommend to meet these requirements?
Send the sensor data to Amazon Kinesis Data Firehose. Use an AWS Lambda function to read the Kinesis Data Firehose data, convert it to .csv format, and insert it into an Amazon Aurora MySQL DB instance. Instruct the data analysts to query the data directly from the DB instance.
Send the sensor data to Amazon Kinesis Data Firehose. Use an AWS Lambda function to read the Kinesis Data Firehose data, convert it to Apache Parquet format, and save it to an Amazon S3 bucket. Instruct the data analysts to query the data by using Amazon Athena.
Send the sensor data to an Amazon Kinesis Data Analytics application to convert the data to .csv format and store it in an Amazon S3 bucket. Import the data into an Amazon Aurora MySQL DB instance. Instruct the data analysts to query the data directly from the DB instance.
Send the sensor data to an Amazon Kinesis Data Analytics application to convert the data to Apache Parquet format and store it in an Amazon S3 bucket. Instruct the data analysts to query the data by using Amazon Athena.
Explanations:
While using Amazon Kinesis Data Firehose and AWS Lambda for converting data to .csv format is feasible, this approach may not optimize costs and operational overhead effectively. The requirement to have a relational database server (Amazon Aurora) for SQL queries creates potential downtime during maintenance. Direct querying of the DB instance by data analysts may still lead to data loss if servers are down.
This solution effectively uses Amazon Kinesis Data Firehose and AWS Lambda to handle sensor data, converting it to Apache Parquet format, which is optimized for query performance and storage efficiency. Storing the data in Amazon S3 allows for durability and scalability. Data analysts can use Amazon Athena to query the data without requiring a separate relational database, thus minimizing operational overhead and reducing costs related to maintenance and downtime.
This option uses Kinesis Data Analytics to convert the data to .csv format and then stores it in Amazon S3 before importing it into an Amazon Aurora MySQL DB instance. Although it allows querying via SQL, the intermediate step of storing data in S3 and later importing it to a relational database introduces unnecessary complexity and may still lead to downtime during maintenance. Additionally, it does not fully optimize for cost as it involves maintaining a relational database.
While converting data to Apache Parquet format is beneficial for performance and storage, using Kinesis Data Analytics to store data directly in S3 and then querying with Amazon Athena bypasses the relational database. However, this approach does not provide a relational model for data analysis and can limit complex queries that may be required by the analysts. It also may not fully align with the goal of reducing operational overhead if additional steps are needed to manage the S3 data structure.