Baby introduction to Bayesian Hyperparameter Optimization for Machine Learning

Hyperparameters, in contrast to model parameters, are set by the ML engineer before training. Hyperparameter optimization is about finding the hyperparameters of an algorithm that give the best performance. It is represented in equation form as:

image by Aaweg I

Now, evaluating the objective function, specially when we consider K-fold validation as…