What is the MOST operationally efficient way for the SysOps administrator to analyze the log files?
Use S3 Select to write a query to search for errors. Run the query across all log groups of interest.
Create an AWS Glue processing job to index the logs of interest. Run a query in Amazon Athena to search for errors.
Use Amazon CloudWatch Logs Insights to write a query to search for errors. Run the query across all log groups of interest.
Use Amazon CloudWatch Contributor Insights to create a rule. Apply the rule across all log groups of interest.
Explanations:
S3 Select is suitable for querying individual objects within S3 using SQL-like queries but is not designed for large-scale, multi-object analysis or log aggregation tasks across numerous Lambda logs in a way that is efficient for finding patterns across multiple log files.
AWS Glue can catalog JSON log files and prepare data for querying in Amazon Athena, which is optimized for large-scale analytics and historical log analysis in S3. Using Athena allows querying across all logs and fields flexibly, making it operationally efficient for searching error strings regardless of field.
CloudWatch Logs Insights is efficient for real-time log analysis directly in CloudWatch but does not support searching logs stored in S3. It’s unsuitable here because the logs are already in S3, not in CloudWatch.
CloudWatch Contributor Insights focuses on identifying top contributors and trends, not querying for specific strings in JSON logs. It’s not suitable for analyzing historical errors across JSON logs stored in S3.
I figure that the answer is:
Create an AWS Glue processing job to index the logs of interest. Run a query in Amazon Athena to search for errors.