Machine Learning Research
Optimizer Without Hyperparameters: VeLO, the system that eliminates the need for optimizer hyperparameters
During training, a neural network usually updates its weights according to an optimizer that’s tuned using hand-picked hyperparameters. New work eliminates the need for optimizer hyperparameters.