Which solution should the company use to evaluate and improve the performance of the models?
Use Amazon CloudWatch algorithm metrics for visibility into the SageMaker training weights, gradients, biases, and activation outputs. Compute the filter ranks based on this information. Apply pruning to remove the low-ranking filters. Set the new weights. Run a new training job with the pruned model.
Use SageMaker Debugger for visibility into the training weights, gradients, biases, and activation outputs. Adjust the model hyperparameters, and look for lower inference times. Run a new training job.
Use SageMaker Debugger for visibility into the training weights, gradients, biases, and activation outputs. Compute the filter ranks based on this information. Apply pruning to remove the low-ranking filters. Set the new weights. Run a new training job with the pruned model.
Use SageMaker Model Monitor for visibility into the ModelLatency metric and OverheadLatency metric of the model after the model is deployed. Adjust the model hyperparameters, and look for lower inference times. Run a new training job.
Explanations:
Amazon CloudWatch provides visibility into logs and metrics, but it does not offer direct insights for model optimization such as pruning based on weights or activation outputs. This solution does not focus on improving model inference performance.
SageMaker Debugger can provide insights into training metrics, but adjusting hyperparameters for inference time improvement during training is not directly effective. It is better suited for debugging and monitoring training processes, not optimizing inference.
SageMaker Debugger offers visibility into model metrics during training, and it is suitable for pruning models based on activation outputs. Pruning the model to remove low-ranking filters and retraining can effectively reduce inference time.
SageMaker Model Monitor is designed for monitoring deployed models’ performance, such as drift or accuracy over time, but not for improving inference speed through hyperparameter tuning. Inference optimization requires model-level changes, not just monitoring metrics.
I weigh that the answer is:
Use SageMaker Debugger for visibility into the training weights, gradients, biases, and activation outputs. Compute the filter ranks based on this information. Apply pruning to remove the low-ranking filters. Set the new weights. Run a new training job with the pruned model.