Which solution will meet these requirements?
Configure AWS Lambda to poll the SQS queue and invoke a Lambda function to check whether the queue messages are valid. If validation fails, send a copy of the data that is not valid to an Amazon S3 bucket so that the scientists can review and correct the data. When the data is corrected, amend the message in the SQS queue by using a replay Lambda function with the corrected data.
Convert the SQS standard queue to an SQS FIFO queue. Configure AWS Lambda to poll the SQS queue every 10 minutes by using an Amazon EventBridge schedule. Invoke the Lambda function to identify any messages with a SentTimestamp value that is older than 5 minutes, push the data to the same location as the application’s output location, and remove the messages from the queue.
Create an SQS dead-letter queue. Modify the existing queue by including a redrive policy that sets the Maximum Receives setting to 1 and sets the dead-letter queue ARN to the ARN of the newly created queue. Instruct the scientists to use the dead-letter queue to review the data that is not valid. Reprocess this data at a later time.
Configure API Gateway to send messages to different SQS virtual queues that are named for each of the satellites. Update the application to use a new virtual queue for any data that it cannot transform, and send the message to the new virtual queue. Instruct the scientists to use the virtual queue to review the data that is not valid. Reprocess this data at a later time.
Explanations:
While this solution captures invalid messages and stores them in S3 for review, it does not utilize SQS’s built-in dead-letter queue functionality, which is designed specifically for handling message failures and ensures messages are retried before being sent to an alternative storage. Additionally, modifying messages in SQS is not a typical practice, and this method may lead to more complexity.
This option proposes converting to a FIFO queue and processing messages based on their timestamp, but it does not address the requirement to retain failed messages for future review and processing. Instead, it removes the messages from the queue without a proper mechanism for scientists to review the failed data.
This solution effectively creates a dead-letter queue to handle failed messages after they have been received a set number of times (in this case, once). This ensures that any messages that cannot be processed by the application are retained for scientists to review and reprocess later, meeting the requirements of retaining and reviewing failed data.
This approach uses virtual queues based on satellite data but introduces unnecessary complexity. It does not leverage the SQS dead-letter queue feature, which is specifically designed for handling message failures. This may lead to difficulties in managing multiple virtual queues and does not provide a straightforward method for scientists to review and reprocess the invalid messages.
My best guess is:
Create an SQS dead-letter queue. Modify the existing queue by including a redrive policy that sets the Maximum Receives setting to 1 and sets the dead-letter queue ARN to the ARN of the newly created queue. Instruct the scientists to use the dead-letter queue to review the data that is not valid. Reprocess this data at a later time.