Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
Attouch, Hedy; Chbani, Zaki; Peypouquet, Juan; Redont, Patrick
20160324 00:00:00
In a Hilbert space setting
$${{\mathcal {H}}}$$
H
, we study the fast convergence properties as
$$t \rightarrow + \infty $$
t
→
+
∞
of the trajectories of the secondorder differential equation
$$\begin{aligned} \ddot{x}(t) + \frac{\alpha }{t} \dot{x}(t) + \nabla \Phi (x(t)) = g(t), \end{aligned}$$
x
¨
(
t
)
+
α
t
x
˙
(
t
)
+
∇
Φ
(
x
(
t
)
)
=
g
(
t
)
,
where
$$\nabla \Phi $$
∇
Φ
is the gradient of a convex continuously differentiable function
$$\Phi : {{\mathcal {H}}} \rightarrow {{\mathbb {R}}}, \alpha $$
Φ
:
H
→
R
,
α
is a positive parameter, and
$$g: [t_0, + \infty [ \rightarrow {{\mathcal {H}}}$$
g
:
[
t
0
,
+
∞
[
→
H
is a small perturbation term. In this inertial system, the viscous damping coefficient
$$\frac{\alpha }{t}$$
α
t
vanishes asymptotically, but not too rapidly. For
$$\alpha \ge 3$$
α
≥
3
, and
$$\int _{t_0}^{+\infty } t \Vert g(t)\Vert dt < + \infty $$
∫
t
0
+
∞
t
‖
g
(
t
)
‖
d
t
<
+
∞
, just assuming that
$${{\mathrm{argmin\,}}}\Phi \ne \emptyset $$
argmin
Φ
≠
∅
, we show that any trajectory of the above system satisfies the fast convergence property
$$\begin{aligned} \Phi (x(t)) \min _{{{\mathcal {H}}}}\Phi \le \frac{C}{t^2}. \end{aligned}$$
Φ
(
x
(
t
)
)

min
H
Φ
≤
C
t
2
.
Moreover, for
$$\alpha > 3$$
α
>
3
, any trajectory converges weakly to a minimizer of
$$\Phi $$
Φ
. The strong convergence is established in various practical situations. These results complement the
$${{\mathcal {O}}}(t^{2})$$
O
(
t

2
)
rate of convergence for the values obtained by Su, Boyd and Candès in the unperturbed case
$$g=0$$
g
=
0
. Time discretization of this system, and some of its variants, provides new fast converging algorithms, expanding the field of rapid methods for structured convex minimization introduced by Nesterov, and further developed by Beck and Teboulle with FISTA. This study also complements recent advances due to Chambolle and Dossal.
http://www.deepdyve.com/assets/images/DeepDyveLogolg.pngMathematical ProgrammingSpringer Journalshttp://www.deepdyve.com/lp/springerjournals/fastconvergenceofinertialdynamicsandalgorithmswithasymptoticMBvwkUa90o
Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
Mathematics; Calculus of Variations and Optimal Control; Optimization; Mathematics of Computing; Numerical Analysis; Combinatorics; Theoretical, Mathematical and Computational Physics; Mathematical Methods in Physics
In a Hilbert space setting
$${{\mathcal {H}}}$$
H
, we study the fast convergence properties as
$$t \rightarrow + \infty $$
t
→
+
∞
of the trajectories of the secondorder differential equation
$$\begin{aligned} \ddot{x}(t) + \frac{\alpha }{t} \dot{x}(t) + \nabla \Phi (x(t)) = g(t), \end{aligned}$$
x
¨
(
t
)
+
α
t
x
˙
(
t
)
+
∇
Φ
(
x
(
t
)
)
=
g
(
t
)
,
where
$$\nabla \Phi $$
∇
Φ
is the gradient of a convex continuously differentiable function
$$\Phi : {{\mathcal {H}}} \rightarrow {{\mathbb {R}}}, \alpha $$
Φ
:
H
→
R
,
α
is a positive parameter, and
$$g: [t_0, + \infty [ \rightarrow {{\mathcal {H}}}$$
g
:
[
t
0
,
+
∞
[
→
H
is a small perturbation term. In this inertial system, the viscous damping coefficient
$$\frac{\alpha }{t}$$
α
t
vanishes asymptotically, but not too rapidly. For
$$\alpha \ge 3$$
α
≥
3
, and
$$\int _{t_0}^{+\infty } t \Vert g(t)\Vert dt < + \infty $$
∫
t
0
+
∞
t
‖
g
(
t
)
‖
d
t
<
+
∞
, just assuming that
$${{\mathrm{argmin\,}}}\Phi \ne \emptyset $$
argmin
Φ
≠
∅
, we show that any trajectory of the above system satisfies the fast convergence property
$$\begin{aligned} \Phi (x(t)) \min _{{{\mathcal {H}}}}\Phi \le \frac{C}{t^2}. \end{aligned}$$
Φ
(
x
(
t
)
)

min
H
Φ
≤
C
t
2
.
Moreover, for
$$\alpha > 3$$
α
>
3
, any trajectory converges weakly to a minimizer of
$$\Phi $$
Φ
. The strong convergence is established in various practical situations. These results complement the
$${{\mathcal {O}}}(t^{2})$$
O
(
t

2
)
rate of convergence for the values obtained by Su, Boyd and Candès in the unperturbed case
$$g=0$$
g
=
0
. Time discretization of this system, and some of its variants, provides new fast converging algorithms, expanding the field of rapid methods for structured convex minimization introduced by Nesterov, and further developed by Beck and Teboulle with FISTA. This study also complements recent advances due to Chambolle and Dossal.
Journal
Mathematical Programming
– Springer Journals
Published: Mar 24, 2016
Recommended Articles
Loading...
References
Newtonlike dynamics and forward–backward methods for structured monotone inclusions in Hilbert spaces
Abbas, B; Attouch, H; Svaiter, BF
On the minimizing property of a secondorder dissipative system in Hilbert spaces
Alvarez, F
An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping
Alvarez, F; Attouch, H
A secondorder gradientlike dissipative dynamical system with Hessiandriven damping. Application to optimization and mechanics
Alvarez, F; Attouch, H; Bolte, J; Redont, P
Asymptotic almostequivalence of Lipschitz evolution systems in Banach spaces
Alvarez, F; Peypouquet, J
A unified approach to the asymptotic almostequivalence of evolution systems without Lipschitz conditions
Alvarez, F; Peypouquet, J
The dynamics of elastic shocks via epigraphical regularization of a differential inclusion
Attouch, H; Cabot, A; Redont, P
A dynamical approach to an inertial forwardbackward algorithm for convex minimization
Attouch, H; Peypouquet, J; Redont, P
Un exemple concernant le comportement asymptotique de la solution du problème
$$\frac{du}{dt} + \partial \phi (u) \ni 0$$
d
u
d
t
+
∂
ϕ
(
u
)
∋
0
Baillon, JB
Quelques propriétés des opérateurs anglebornés et ncycliquement monotones
Baillon, JB; Haddad, G
Convex Analysis and Monotone Operator Theory in Hilbert spaces
Bauschke, H; Combettes, P
A fast iterative shrinkagethresholding algorithm for linear inverse problems
Beck, A; Teboulle, M
Asymptotic convergence of nonlinear contraction semigroups in Hilbert spaces
Bruck, RE
On the long time behavior of second order differential equations with asymptotically small dissipation
Cabot, A; Engler, H; Gadat, S
Second order differential equations with asymptotically small dissipation and piecewise flat potentials
Cabot, A; Engler, H; Gadat, S
New proximal point algorithms for convex minimization
Güler, O
Theory and Application of Infinite Series
Knopp, K
An inertial forward–backward algorithm for monotone inclusions
Lorenz, DA; Pock, T
Convergence of a splitting inertial proximal method for monotone operators
Moudafi, A; Oliny, M
A method of solving a convex programming problem with convergence rate O(1/k2)
Nesterov, Y
Introductory Lectures on Convex Optimization: A Basic Course, Volume 87 of Applied Optimization
Nesterov, Y
Smooth minimization of nonsmooth functions
Nesterov, Y
Weak convergence of the sequence of successive approximations for nonexpansive mappings
Opial, Z
Proximal algorithms
Parikh, N; Boyd, S
Convex Optimization in Normed Spaces: Theory, Methods and Examples
Peypouquet, J
Evolution equations for maximal monotone operators: asymptotic analysis in continuous and discrete time
Peypouquet, J; Sorin, S
A differential equation for modeling Nesterov’s accelerated gradient method: theory and insights
Su, W; Boyd, S; Candès, EJ
Accelerated and inexact forward–backward
Villa, S; Salzo, S; Baldassarres, L; Verri, A
You’re reading a free preview. Subscribe to read the entire article.
“Hi guys, I cannot tell you how much I love this resource. Incredible. I really believe you've hit the nail on the head with this site in regards to solving the researchpurchase issue.”
Daniel C.
“Whoa! It’s like Spotify but for academic articles.”
@Phil_Robichaud
“I must say, @deepdyve is a fabulous solution to the independent researcher's problem of #access to #information.”
@deepthiw
“My last article couldn't be possible without the platform @deepdyve that makes journal papers cheaper.”
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Subscribe to Journal Email Alerts
To subscribe to email alerts, please log in first, or sign up for a DeepDyve account if you don’t already have one.
Follow a Journal
To get new article updates from a journal on your personalized homepage, please log in first, or sign up for a DeepDyve account if you don’t already have one.