Algorithms for Hyper-Parameter Optimization

← Back to document page

Algorithms for Hyper-Parameter Optimization James Bergstra Re mi Bardenet The Rowland Institute Laboratoire de Recherche en Informatique Harvard University Universite Paris-Sud Yoshua Bengio Bala zs Ke gl De pt. d'Informatique et Recherche Ope rationelle Linear Accelerator Laboratory Universite de Montre al Universite Paris-Sud, CNRS. Abstract Several recent advances to the state of the art in image classification benchmarks have come from better configurations of existing techniques rather than novel ap- proaches to feature learning. Traditionally, Hyper-Parameter Optimization has been the job of humans because they can be very efficient in regimes where only a few trials are possible. Presently, computer clusters and GPU processors make it pos- sible to run more trials and we show that algorithmic approaches can find better results.

optimization of DBNs in [1], and 2) Automatic sequential optimization outperforms both manual and random search. Section 2 covers sequential model-based optimization, and the expected improvement criterion. Sec-tion 3 introduces a Gaussian Process based hyper-parameter optimization algorithm. Section 4 in-

  Parameters, Algorithm, Optimization, Hyper, Algorithms for hyper parameter optimization