Develop a Lightning Hyperparameter Optimization (HPO) App¶
A bit of background¶
Traditionally, developing machine learning (ML) products requires choosing among a large space of hyperparameters while creating and training the ML models. Hyperparameter optimization (HPO) aims to find a well-performing hyperparameter configuration for a given ML model on a dataset at hand, including the ML model, its hyperparameters, and other data processing steps.
HPOs free the human expert from a tedious and error-prone, manual hyperparameter tuning process.
It is possible and recommended to search the hyperparameter space for the best validation score.
An HPO search consists of:
an objective method
a defined parameter space
a method for searching or sampling candidates
A naive method for sampling candidates is grid search, which exhaustively considers all hyperparameter combinations from a user-specified grid.
Fortunately, HPO is an active area of research, and many methods have been developed to optimize the time required to get strong candidates.
In the following tutorial, you will learn how to use Lightning together with Optuna.
Optuna is an open source HPO framework to automate hyperparameter search. Out-of-the-box, it provides efficient algorithms to search large spaces and prune unpromising trials for faster results.
First, you will learn about the best practices on how to implement HPO without the Lightning Framework. Secondly, we will dive into a working HPO application with Lightning, and finally create a neat HiPlot UI for our application.