Access the full text.
Sign up today, get DeepDyve free for 14 days.
S. Bonnabel (2011)
Stochastic Gradient Descent on Riemannian ManifoldsIEEE Transactions on Automatic Control, 58
F. Yger, Maxime Bérar, G. Gasso, A. Rakotomamonjy (2012)
Adaptive Canonical Correlation Analysis Based On Matrix Manifolds
Traian Abrudan, J. Eriksson, V. Koivunen (2008)
Steepest Descent Algorithms for Optimization Under Unitary Matrix ConstraintIEEE Transactions on Signal Processing, 56
Hiroyuki Sato, Hiroyuki Kasai, Bamdev Mishra (2016)
Riemannian stochastic variance reduced gradientArXiv, abs/1702.05594
J. Bolte, Trong Nguyen, J. Peypouquet, B. Suter (2015)
From error bounds to the complexity of first-order descent methods for convex functionsMathematical Programming, 165
Yiqiao Zhong, Nicolas Boumal (2017)
Near-Optimal Bounds for Phase SynchronizationSIAM J. Optim., 28
A. So (2013)
Non-Asymptotic Convergence Analysis of Inexact Gradient Methods for Machine Learning Without Strong ConvexityArXiv, abs/1309.0113
D. Sorensen (2002)
Numerical methods for large eigenvalue problemsActa Numerica, 11
O. Shamir (2014)
A Stochastic PCA and SVD Algorithm with an Exponential Convergence Rate
L. Tunçel (2009)
Optimization algorithms on matrix manifoldsMath. Comput., 78
(2014)
Provable tensor factorizationwithmissing data
R. Schneider, André Uschmajew (2014)
Convergence Results for Projected Line-Search Methods on Varieties of Low-Rank Matrices Via Łojasiewicz InequalitySIAM J. Optim., 25
Z. Luo, J. Sturm (1999)
Error Bounds for Quadratic Systems
Guoyin Li, Ting Pong (2016)
Calculus of the Exponent of Kurdyka–Łojasiewicz Inequality and Its Applications to Linear Convergence of First-Order MethodsFoundations of Computational Mathematics, 18
W. Sun, Junwei Lu, Han Liu, Guang Cheng (2015)
Provable sparse tensor decompositionJournal of the Royal Statistical Society: Series B (Statistical Methodology), 79
André Uschmajew (2014)
A new convergence proof for the higher-order power method and generalizationsarXiv: Optimization and Control
C. Udrişte (1994)
Convex Functions and Optimization Methods on Riemannian Manifolds
Guoyin Li, B. Mordukhovich, T. Pham (2015)
New fractional error bounds for polynomial systems with applications to Hölderian stability in optimization and spectral theory of tensorsMathematical Programming, 153
B. Merlet, Thanh-Nhan Nguyen (2013)
Convergence to equilibrium for discretizations of gradient-like flows on Riemannian manifoldsDifferential and Integral Equations
Z. Luo (2000)
New error bounds and their applications to convergence analysis of iterative algorithmsMathematical Programming, 88
Ju Sun, Qing Qu, John Wright (2015)
Complete dictionary recovery over the sphere2015 International Conference on Sampling Theory and Applications (SampTA)
Zirui Zhou, A. So (2015)
A unified approach to error bounds for structured convex optimization problemsMathematical Programming, 165
A. So (2011)
Moment inequalities for sums of random matrices and their applications in optimizationMathematical Programming, 130
J Sun, Q Qu, J Wright (2017)
Complete dictionary recovery over the sphere I: overview and the geometric pictureIEEE Trans. Inf. Theory, 63
B. Jiang, Yuhong Dai (2013)
A framework of constraint preserving update schemes for optimization on Stiefel manifoldMathematical Programming, 153
(1996)
Matrix Computations, 3rd edn
M. Forti, P. Nistri, M. Quincampoix (2006)
Convergence of Neural Networks for Programming Problems via a Nonsmooth Łojasiewicz InequalityIEEE Transactions on Neural Networks, 17
Y. Nesterov (2014)
Introductory Lectures on Convex Optimization - A Basic Course, 87
Alekh Agarwal, Anima Anandkumar, Prateek Jain, Praneeth Netrapalli (2013)
Learning Sparsely Used Overcomplete Dictionaries via Alternating MinimizationArXiv, abs/1310.7991
Zaiwen Wen, W. Yin (2012)
A feasible method for optimization with orthogonality constraintsMathematical Programming, 142
GH Golub, CF Loan (1996)
Matrix Computations
K. Hou, Zirui Zhou, A. So, Z. Luo (2013)
On the Linear Convergence of the Proximal Gradient Method for Trace Norm Regularization
Huikang Liu, Weijie Wu, A. So (2015)
Quadratic Optimization with Orthogonality Constraints: Explicit Lojasiewicz Exponent and Linear Convergence of Line-Search MethodsArXiv, abs/1510.01025
Moritz Hardt (2013)
Understanding Alternating Minimization for Matrix Completion2014 IEEE 55th Annual Symposium on Foundations of Computer Science
L. Dieci, T. Eirola (1999)
On Smooth Decompositions of MatricesSIAM J. Matrix Anal. Appl., 20
X. Chang, C. Paige, G. Stewart (1997)
Perturbation Analyses for the QR FactorizationSIAM J. Matrix Anal. Appl., 18
Z-Q Luo, JF Sturm (2000)
High Performance Optimization, Volume 33 of Applied Optimization
Ji-guang Sun (1995)
On perturbation bounds for the QR factorizationLinear Algebra and its Applications, 215
ST Smith (1994)
Hamiltonian and Gradient Flows, Algorithms and Control. Fields Institue Communications
Huikang Liu, Man-Chung Yue, A. So (2016)
On the Estimation Performance and Convergence Rate of the Generalized Power Method for Phase SynchronizationSIAM J. Optim., 27
P. Schönemann (1966)
A generalized solution of the orthogonal procrustes problemPsychometrika, 31
(2015)
Pinning down theŁojasiewicz exponent: towards understanding the convergence behavior of first-order methods for structured non-convex optimization problems
Rie Johnson, Tong Zhang (2013)
Accelerating Stochastic Gradient Descent using Predictive Variance Reduction
Ju Sun, Qing Qu, John Wright (2016)
A Geometric Analysis of Phase RetrievalFoundations of Computational Mathematics, 18
P. Absil, J. Malick (2012)
Projection-like Retractions on Matrix ManifoldsSIAM J. Optim., 22
P. Feehan (2014)
Global existence and convergence of solutions to gradient systems and applications to Yang-Mills gradient flowarXiv: Differential Geometry
A. Bloch (1995)
Hamiltonian and Gradient Flows, Algorithms and Control, 3
Qinqing Zheng, J. Lafferty (2015)
A Convergent Gradient Descent Algorithm for Rank Minimization and Semidefinite Programming from Random Linear Measurements
Hiroyuki Sato, T. Iwai (2013)
A Riemannian Optimization Approach to the Matrix Singular Value DecompositionSIAM J. Optim., 23
Hongyi Zhang, S. Sra (2016)
First-order Methods for Geodesically Convex Optimization
Hongyi Zhang, Sashank Reddi, S. Sra (2016)
Fast stochastic optimization on Riemannian manifoldsArXiv, abs/1605.07147
Zirui Zhou, Qi Zhang, A. So (2015)
\(\ell_{1, p}\)-Norm Regularization: Error Bounds and Convergence Rate Analysis of First-Order Methods
R Sun, Z-Q Luo (2016)
Guaranteed matrix completion via non-convex factorizationIEEE Trans. Inf. Theory, 62
E. Candès, Xiaodong Li, M. Soltanolkotabi (2014)
Phase Retrieval via Wirtinger Flow: Theory and AlgorithmsIEEE Transactions on Information Theory, 61
O. Shamir (2015)
Fast Stochastic Algorithms for SVD and PCA: Convergence Properties and ConvexityArXiv, abs/1507.08788
C Udrişte (1994)
Convex Functions and Optimization Methods on Riemannian Manifolds, Volume 297 of Mathematics and Its Applications
Ruoyu Sun, Z. Luo (2014)
Guaranteed Matrix Completion via Nonconvex Factorization2015 IEEE 56th Annual Symposium on Foundations of Computer Science
Praneeth Netrapalli, Prateek Jain, S. Sanghavi (2013)
Phase Retrieval Using Alternating MinimizationIEEE Transactions on Signal Processing, 63
J. Bolte, A. Daniilidis, Olivier Ley, L. Mazet (2009)
Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexityTransactions of the American Mathematical Society, 362
P. Schönemann (1968)
On two-sided orthogonal procrustes problemsPsychometrika, 33
Ju Sun, Qing Qu, John Wright (2015)
Complete Dictionary Recovery Over the Sphere II: Recovery by Riemannian Trust-Region MethodIEEE Transactions on Information Theory, 63
Yaguang Yang (2007)
Globally Convergent Optimization Algorithms on Riemannian Manifolds: Uniform Framework for Unconstrained and Constrained OptimizationJournal of Optimization Theory and Applications, 132
S.T. Smith (2014)
Optimization Techniques on Riemannian ManifoldsArXiv, abs/1407.5965
J. Manton (2002)
Optimization algorithms exploiting unitary constraintsIEEE Trans. Signal Process., 50
E. Kokiopoulou, Jie Chen, Y. Saad (2011)
Trace optimization and eigenproblems in dimension reduction methodsNumerical Linear Algebra with Applications, 18
P. Absil, R. Mahony, B. Andrews (2005)
Convergence of the Iterates of Descent Methods for Analytic Cost FunctionsSIAM J. Optim., 16
Z. Luo, J. Pang (1994)
Error bounds for analytic systems and their applicationsMathematical Programming, 67
Tetsuya Kaneko, S. Fiori, Toshihisa Tanaka (2013)
Empirical Arithmetic Averaging Over the Compact Stiefel ManifoldIEEE Transactions on Signal Processing, 61
M. Bolla, G. Michaletzky, G. Tusnády, M. Ziermann (1998)
Extrema of sums of heterogeneous quadratic formsLinear Algebra and its Applications, 269
Z. Luo, P. Tseng (1993)
Error bounds and convergence analysis of feasible descent methods: a general approachAnnals of Operations Research, 46-47
The problem of optimizing a quadratic form over an orthogonality constraint (QP-OC for short) is one of the most fundamental matrix optimization problems and arises in many applications. In this paper, we characterize the growth behavior of the objective function around the critical points of the QP-OC problem and demonstrate how such characterization can be used to obtain strong convergence rate results for iterative methods that exploit the manifold structure of the orthogonality constraint (i.e., the Stiefel manifold) to find a critical point of the problem. Specifically, our primary contribution is to show that the Łojasiewicz exponent at any critical point of the QP-OC problem is 1 / 2. Such a result is significant, as it expands the currently very limited repertoire of optimization problems for which the Łojasiewicz exponent is explicitly known. Moreover, it allows us to show, in a unified manner and for the first time, that a large family of retraction-based line-search methods will converge linearly to a critical point of the QP-OC problem. Then, as our secondary contribution, we propose a stochastic variance-reduced gradient (SVRG) method called Stiefel-SVRG for solving the QP-OC problem and present a novel Łojasiewicz inequality-based linear convergence analysis of the method. An important feature of Stiefel-SVRG is that it allows for general retractions and does not require the computation of any vector transport on the Stiefel manifold. As such, it is computationally more advantageous than other recently-proposed SVRG-type algorithms for manifold optimization.
Mathematical Programming – Springer Journals
Published: Jun 1, 2018
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.