Which technique will meet these requirements with LEAST computational time?
Grid search
Random search
Bayesian optimization
Hyperband
Explanations:
Grid search evaluates every possible combination of hyperparameters exhaustively, which can be extremely time-consuming and inefficient, especially with a large search space. This approach does not allow for early stopping of underperforming jobs, leading to wasted computational resources.
Random search samples random combinations of hyperparameters, which can be more efficient than grid search. However, it still does not incorporate a systematic way to stop underperforming jobs early and may require more trials to find optimal configurations compared to other techniques.
Bayesian optimization builds a probabilistic model of the function mapping hyperparameters to performance. While it is more efficient than grid and random searches, it generally does not allow for simultaneous evaluations of many configurations, which can slow down the process of stopping underperforming jobs.
Hyperband is a bandit-based approach that efficiently allocates resources to promising configurations and stops underperforming ones. It allows for parallel evaluations of hyperparameters while dynamically adjusting the number of configurations based on their performance, minimizing wasted computational time and resources.