Which solution will meet these requirements with the LARGEST performance improvement?

By:
In: SAP-C01
Tagged:
With: 1 Comment
A company deploys workloads in multiple AWS accounts.Each account has a VPC with VPC flow logs published in text log format to a centralized Amazon S3 bucket.Each log file is compressed with gzip compression.The company must retain the log files indefinitely.A security engineer occasionally analyzes the logs by using Amazon Athena to query the VPC flow logs.The query performance is degrading over time as the number of ingested logs is growing.A solutions architect must improve the performance of the log analysis and reduce the storage space that the VPC flow logs use.

Which solution will meet these requirements with the LARGEST performance improvement?

Create an AWS Lambda function to decompress the gzip files and to compress the files with bzip2 compression. Subscribe the Lambda function to an s3:ObjectCreated:Put S3 event notification for the S3 bucket.

Enable S3 Transfer Acceleration for the S3 bucket. Create an S3 Lifecycle configuration to move files to the S3 Intelligent-Tiering storage class as soon as the files are uploaded.

Update the VPC flow log configuration to store the files in Apache Parquet format. Specify hourly partitions for the log files.

Create a new Athena workgroup without data usage control limits. Use Athena engine version 2.

Explanations:

While using bzip2 compression may reduce the size of the logs, it does not inherently improve query performance in Athena significantly. Decompressing and recompressing logs is also a resource-intensive task that may not yield substantial benefits for querying.

Enabling S3 Transfer Acceleration and moving files to the S3 Intelligent-Tiering storage class helps with the speed of uploads and cost management but does not directly enhance Athena query performance or address the size of the log files for analysis.

Updating the VPC flow log configuration to store logs in Apache Parquet format and specifying hourly partitions can dramatically improve query performance. Parquet is a columnar storage format optimized for large datasets and analytical queries, which allows Athena to read only the necessary data. Additionally, partitioning helps reduce the amount of data scanned during queries, further enhancing performance.

While creating a new Athena workgroup and using the latest engine version can improve performance, it does not address the underlying issue of the data storage format and size. It may offer some performance improvements, but not as significant as switching to Parquet format.

2025-09-28

1 Comment

  1. Zachary
    Author

    I plot that the answer is:
    Update the VPC flow log configuration to store the files in Apache Parquet format. Specify hourly partitions for the log files.

Leave a Reply

Your email address will not be published. Required fields are marked *

fourteen − 13 =