Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/springer-journals/adaptive-stochastic-approximation-algorithm-urICJaIOXZ
References (22)
-
H Robbins, S Monro (1951)
A stochastic approximation methodAnn. Math. Stat., 22
-
RH Byrd, GM Chin, J Nocedal, Y Wu (2012)
Sample size selection in optimization methods for machine learningMath. Program., 134
-
Z Xu, X Xu (2013)
A new hybrid stochastic approximation algorithmOptim. Lett., 7
-
K Knight (2000)
Mathematical Statistics -
F Yousefian, A Nedic, UV Shanbhag (2012)
On stochastic gradient and subgradient methods with adaptive steplength sequencesAutomatica, 48
-
JJ Moré, BS Garbow, KE Hillstrom (1981)
Testing unconstrained optimization softwareTOMS, 7
-
N Krejic, Z Luzanin, Z Ovcin, I Stojkovska (2015)
Descent direction method with line search for unconstrained optimization in noisy environmentOptim Methods Softw, 30
-
Z Xu, YH Dai (2008)
A stochastic approximation frame algorithm with adaptive directionsNumer. Math. Theor. Meth. Appl., 1
-
B Delyon, A Juditsky (1993)
Accelerated stochastic approximationSIAM J. Optim., 3
-
R Durrett (1995)
Probability: Theory and Examples -
M Raydan (1997)
The Barzilai and Borwein gradient method for the large scale unconstrained minimization problemSIAM J. Optim., 7
-
HF Chen (2002)
Stochastic Approximation and Its Application -
Z Xu, YH Dai (2012)
New stochastic approximation algorithms with adaptive step sizesOptim. Lett., 6
-
RH Byrd, GM Chin, W Neveitt, J Nocedal (2011)
On the use of stochastic Hessian information in optimization methods for machine learningSIAM J. Optim., 21
-
JC Spall (2003)
Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control -
H Kesten (1958)
Accelerated stochastic approximationAnn. Math. Stat., 29
-
DP Bertsekas, JN Tsitsiklis (2000)
Gradient convergence in gradient methods with errorsSIAM J. Optim., 10
-
N Krejic, Z Luzanin, I Stojkovska (2013)
A gradient method for unconstrained optimization in noisy environmentAppl. Numer. Math., 70
-
JC Spall (2000)
Adaptive stochastic approximation by the simultaneous perturbation methodIEEE AC, 45
-
RH Byrd, SL Hansen, J Nocedal, Y Singer (2016)
A stochastic quasi-Newton method for large-scale optimizationSIAM J. Optim., 26
-
WB Powell (2007)
Approximate Dynamic Programming: Solving the Curses of Dimensionality Chapter, vol. 6 -
Z Xu (2010)
A combined direction stochastic approximation algorithm, OptimLett., 4
- Publisher
- Springer Journals
- Copyright
- Copyright © 2017 by Springer Science+Business Media New York
- Subject
- Computer Science; Numeric Computing; Algorithms; Algebra; Theory of Computation; Numerical Analysis
- ISSN
- 1017-1398
- eISSN
- 1572-9265
- DOI
- 10.1007/s11075-017-0290-4
- Publisher site
- See Article on Publisher Site
Abstract
In this paper, stochastic approximation (SA) algorithm with a new adaptive step size scheme is proposed. New adaptive step size scheme uses a fixed number of previous noisy function values to adjust steps at every iteration. The algorithm is formulated for a general descent direction and almost sure convergence is established. The case when negative gradient is chosen as a search direction is also considered. The algorithm is tested on a set of standard test problems. Numerical results show good performance and verify efficiency of the algorithm compared to some of existing algorithms with adaptive step sizes.
Journal
Numerical Algorithms – Springer Journals
Published: Feb 27, 2017