Which solution meets these requirements and is the MOST operationally efficient?
Set up an Amazon EC2 instance running a Redis database. Configure both applications to use the instance. Store, process, and delete the messages, respectively.
Use an Amazon Kinesis data stream to receive the messages from the sender application. Integrate the processing application with the Kinesis Client Library (KCL).
Integrate the sender and processor applications with an Amazon Simple Queue Service (Amazon SQS) queue. Configure a dead-letter queue to collect the messages that failed to process.
Subscribe the processing application to an Amazon Simple Notification Service (Amazon SNS) topic to receive notifications to process. Integrate the sender application to write to the SNS topic.
Explanations:
Using an EC2 instance with Redis introduces operational overhead for managing the instance, scaling, and reliability. It also does not provide built-in message retention or dead-letter capabilities, which are critical for handling failed message processing.
Amazon Kinesis is designed for real-time data streaming and may not be optimal for handling messages that can take up to 2 days to process. It lacks built-in support for message retention and error handling like dead-letter queues, making it less operationally efficient for this use case.
Amazon SQS provides a fully managed message queuing service with built-in support for message retention and dead-letter queues. This solution allows for reliable message processing and retention of failed messages, making it the most operationally efficient option.
Amazon SNS is designed for pub/sub messaging and does not provide message storage or processing guarantees. It does not have built-in capabilities for handling message failures, making it less suitable for this scenario where message retention is necessary.
I conclude that the answer is:
Integrate the sender and processor applications with an Amazon Simple Queue Service (Amazon SQS) queue. Configure a dead-letter queue to collect the messages that failed to process.