Theses and Dissertations Collection

The Nesterov gradient descent algorithm serves as a performance benchmark forconvex optimization problems. Like many other gradient-based methods, the Nesterov algorithm requires choosing a constant step size before optimization begins, and the performance of the algorithm heavily depends on the step size. Here, we propose three novel adaptive algorithms which adaptively determine the step size based on the searching history. The new adaptive methods were tested alongside the original Nesterov algorithm on a list of commonly-used optimization test functions in a range of dimensions. The experimental results showed that they consistently outperformed the Nesterov algorithm by a wide margin. We also discuss ways that the adaptive methods could be improved.