What is the MOST operationally efficient way to refactor the serverless application to accommodate this change?
Use an AWS Step Functions state machine to monitor API failures. Use the Wait state to delay calling the Lambda function.
Use an Amazon Simple Queue Service (Amazon SQS) queue to hold the API calls. Configure the Lambda function to poll the queue within the API threshold limits.
Use an Amazon CloudWatch Logs metric to count the number of API calls. Configure an Amazon CloudWatch alarm that stops the currently running instance of the Lambda function when the metric exceeds the API threshold limits.
Use Amazon Kinesis Data Firehose to batch the API calls and deliver them to an Amazon S3 bucket with an event notification to invoke the Lambda function.
Explanations:
While AWS Step Functions can manage complex workflows and handle retries, using a Wait state may not be the most efficient approach for managing API call limits, as it could lead to delays in processing without optimizing the number of calls made within allowed limits. This option lacks a robust way to manage the actual rate of API calls.
Using Amazon SQS to hold API calls allows for throttling the requests based on the API rate limits. The Lambda function can poll the queue at a controlled rate that adheres to the minute and daily limits of the third-party service. This method effectively manages the API call limits while ensuring that the processing can continue efficiently.
Counting API calls with CloudWatch Logs metrics and stopping the Lambda function when limits are exceeded does not efficiently manage the flow of API calls. This approach could result in the Lambda function being stopped, leading to incomplete processing of data and potentially missing reporting deadlines. It does not provide a mechanism to schedule or batch requests within the allowed limits.
Amazon Kinesis Data Firehose is designed for streaming data and batch processing, but it is not specifically intended for managing API call limits. This option could complicate the architecture and does not provide a straightforward method to ensure that API calls stay within the set thresholds. The event notifications might not effectively throttle the calls according to the third-party service’s restrictions.
I deduce that the answer is:
Use an Amazon Simple Queue Service (Amazon SQS) queue to hold the API calls. Configure the Lambda function to poll the queue within the API threshold limits.