LM101-068: How to Design Automatic Learning Rate Selection for Gradient Descent Type Machine Learning Algorithms

This 68th episode of Learning Machines 101 discusses a broad class of unsupervised, supervised, and reinforcement machine learning algorithms which iteratively update their parameter vector by adding a perturbation based upon all of the training data. This process is repeated, making a perturbation of the parameter vector based upon all of the training data until a parameter vector is generated which exhibits improved predictive performance. The magnitude of the perturbation at each learning iteration is called the "stepsize" or "learning rate" and the identity of the perturbation vector is called the "search direction". Simple mathematical formulas are presented based upon research from the late 1960s by Philip Wolfe and G. Zoutendijk that ensure convergence of the generated sequence of parameter vectors. These formulas may be used as the basis for the design of artificially intelligent smart automatic learning rate selection algorithms. For more information, please visit the official website:

You can listen to LM101-068: How to Design Automatic Learning Rate Selection for Gradient Descent Type Machine Learning Algorithms online on Radio and Podcast. Open the player on this page to stream the available audio.

LM101-068: How to Design Automatic Learning Rate Selection for Gradient Descent Type Machine Learning Algorithms is an episode from Learning Machines 101 by Richard M. Golden.

This episode is 21:49 long.

This episode was published on Sep 26, 2017.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from Learning Machines 101 when more episodes are available from the podcast feed.