Which solution will meet these requirements?

By:
On:
In: SAP-C01
Tagged:
With: 0 Comments
A company is collecting a large amount of data from a fleet of IoT devices.Data is stored as Optimized Row Columnar (ORC) files in the Hadoop Distributed FileSystem (HDFS) on a persistent Amazon EMR cluster.The company’s data analytics team queries the data by using SQL in Apache Presto deployed on the sameEMR cluster.Queries scan large amounts of data, always run for less than 15 minutes, and run only between 5 PM and 10 PM.The company is concerned about the high cost associated with the current solution.A solutions architect must propose the most cost-effective solution that will allow SQL data queries.

Which solution will meet these requirements?

Store data in Amazon S3. Use Amazon Redshift Spectrum to query data.

Store data in Amazon S3. Use the AWS Glue Data Catalog and Amazon Athena to query data.

Store data in EMR File System (EMRFS). Use Presto in Amazon EMR to query data.

Store data in Amazon Redshift. Use Amazon Redshift to query data.

Explanations:

While Amazon Redshift Spectrum allows querying data in S3, it incurs additional costs for data scanned and is generally more expensive than other options for frequent queries.

Storing data in Amazon S3 and using AWS Glue Data Catalog with Amazon Athena provides a serverless architecture, reducing costs significantly as you only pay for the queries run, and it scales well for varying query loads.

While using EMR File System (EMRFS) and Presto allows querying data efficiently, maintaining a persistent EMR cluster incurs higher operational costs compared to serverless options like Athena.

Storing data in Amazon Redshift may not be cost-effective for this scenario due to the fixed cost of maintaining a Redshift cluster, especially if queries only occur during a limited timeframe.

Leave a Reply

Your email address will not be published. Required fields are marked *

1 × four =