What should the developer do to accomplish this goal with the LEAST operational overhead?
Set up Amazon CloudWatch Logs log groups to filter and store the messages in an Amazon S3 bucket. Import the messages in Lambda. Run the Lambda function again.
Configure Amazon EventBridge (Amazon CloudWatch Events) to send the messages to Amazon Simple Notification Service (Amazon SNS) to initiate the Lambda function again.
Implement a dead-letter queue for discarded messages. Set the dead-letter queue as an event source for the Lambda function.
Send Amazon EventBridge (Amazon CloudWatch Events) events to an Amazon Simple Queue Service (Amazon SQS) queue. Configure the Lambda function to pull messages from the SQS queue. Run the Lambda function again.
Explanations:
Using Amazon CloudWatch Logs to filter and store messages in S3 adds unnecessary complexity and operational overhead. It requires manual import of messages into Lambda for retries, which is not efficient.
Sending messages to Amazon SNS does not provide a reliable mechanism for handling failed invocations directly. SNS is primarily for notifications, not for tracking failures or retrying message processing without additional logic.
Implementing a dead-letter queue (DLQ) allows messages that fail processing in the Lambda function to be stored for later retrieval and processing. This is a straightforward method to handle errors with minimal operational overhead.
While using SQS to manage messages can work, it requires setting up an additional queue and pulling messages from it, which may introduce more operational complexity compared to the simplicity of a dead-letter queue directly linked to the Lambda function.
In my experience, the answer is:
Implement a dead-letter queue for discarded messages. Set the dead-letter queue as an event source for the Lambda function.