Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

A stochastic Gauss–Newton algorithm for regularized semi-discrete optimal transport

A stochastic Gauss–Newton algorithm for regularized semi-discrete optimal transport We introduce a new second order stochastic algorithm to estimate the entropically regularized optimal transport (OT) cost between two probability measures. The source measure can be arbitrary chosen, either absolutely continuous or discrete, whereas the target measure is assumed to be discrete. To solve the semi-dual formulation of such a regularized and semi-discrete optimal transportation problem, we propose to consider a stochastic Gauss–Newton (SGN) algorithm that uses a sequence of data sampled from the source measure. This algorithm is shown to be adaptive to the geometry of the underlying convex optimization problem with no important hyperparameter to be accurately tuned. We establish the almost sure convergence and the asymptotic normality of various estimators of interest that are constructed from this SGN algorithm. We also analyze their non-asymptotic rates of convergence for the expected quadratic risk in the absence of strong convexity of the underlying objective function. The results of numerical experiments from simulated data are also reported to illustrate the finite sample properties of this Gauss–Newton algorithm for stochastic regularized OT and to show its advantages over the use of the stochastic gradient descent, stochastic Newton and ADAM algorithms. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Information and Inference A Journal of the IMA Oxford University Press

A stochastic Gauss–Newton algorithm for regularized semi-discrete optimal transport

Bercu, Bernard; Bigot, Jérémie; Gadat, Sébastien; Siviero, Emilia
Information and Inference A Journal of the IMA , Volume 12 (1): 58 – May 19, 2022
Download PDF
58 pages

Loading next page...
 
/lp/oxford-university-press/a-stochastic-gauss-newton-algorithm-for-regularized-semi-discrete-8sA86izYaP
Publisher
Oxford University Press
Copyright
© The Author(s) 2022. Published by Oxford University Press on behalf of the Institute of Mathematics and its Applications. All rights reserved.
ISSN
2049-8764
eISSN
2049-8772
DOI
10.1093/imaiai/iaac014
Publisher site
See Article on Publisher Site

Abstract

We introduce a new second order stochastic algorithm to estimate the entropically regularized optimal transport (OT) cost between two probability measures. The source measure can be arbitrary chosen, either absolutely continuous or discrete, whereas the target measure is assumed to be discrete. To solve the semi-dual formulation of such a regularized and semi-discrete optimal transportation problem, we propose to consider a stochastic Gauss–Newton (SGN) algorithm that uses a sequence of data sampled from the source measure. This algorithm is shown to be adaptive to the geometry of the underlying convex optimization problem with no important hyperparameter to be accurately tuned. We establish the almost sure convergence and the asymptotic normality of various estimators of interest that are constructed from this SGN algorithm. We also analyze their non-asymptotic rates of convergence for the expected quadratic risk in the absence of strong convexity of the underlying objective function. The results of numerical experiments from simulated data are also reported to illustrate the finite sample properties of this Gauss–Newton algorithm for stochastic regularized OT and to show its advantages over the use of the stochastic gradient descent, stochastic Newton and ADAM algorithms.

Journal

Information and Inference A Journal of the IMA – Oxford University Press

Published: May 19, 2022

Keywords: stochastic optimization; stochastic Gauss–Newton algorithm; optimal transport; entropic regularization; convergence of random variables

References

  • Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression
    Bach, F. R.
  • Asymptotic pseudotrajectories and chain recurrent flows, with applications
    Benaïm, M.; Hirsch, M.
  • Iterative Bregman projections for regularized transportation problems
    Benamou, J.-D.; Carlier, G.; Cuturi, M.; Nenna, L.; Peyré, G.
  • Asymptotic distribution and convergence rates of stochastic algorithms for entropic optimal transportation between probability measures
    Bercu, B.; Bigot, J.
  • An efficient stochastic Newton algorithm for parameter estimation in logistic regressions
    Bercu, B.; Godichon, A.; Portier, B.
  • Data-driven regularization of Wasserstein barycenters with an application to multivariate density registration
    Bigot, J.; Cazelles, E.; Papadakis, N.
  • Geodesic PCA in the Wasserstein space by convex PCA
    Bigot, J.; Gouet, R.; Klein, T.; López, A.
  • Statistical data analysis in the Wasserstein space
    Bigot, J.
  • The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems
    Bolte, J.; Daniilidis, A.; Lewis, A.
  • From error bounds to the complexity of first-order descent methods for convex functions
    Bolte, J.; Nguyen, P.; Peypouquet, J.; Suter, B. W.
  • Sliced and radon Wasserstein barycenters of measures
    Bonneel, N.; Rabin, J.; Peyré, G.; Pfister, H.
  • Log-PCA versus geodesic PCA of histograms in the Wasserstein space
    Cazelles, E.; Seguy, V.; Bigot, J.; Cuturi, M.; Papadakis, N.
  • An efficient averaged stochastic Gauss-Newton algorithm for estimating parameters of non linear regressions models
    Cénac, P.; Godichon-Baggioni, A.; Portier, B.
  • Semidual regularized optimal transport
    Cuturi, M.; Peyré, G.
  • A generalization of the Sherman-Morrison-Woodbury formula
    Deng, C. Y.
  • Regularized discrete optimal transport
    Ferradans, S.; Papadakis, N.; Peyré, G.; Aujol, J.-F.
  • Wasserstein discriminant analysis
    Flamary, R.; Cuturi, M.; Courty, N.; Rakotomamonjy, A.
  • Optimal non-asymptotic bound of the Ruppert-Polyak averaging without strong convexity
    Gadat, S.; Panloup, F.
  • Updating the inverse of a matrix
    Hager, W. W.
  • Convergence of a Newton algorithm for semi-discrete optimal transport
    Kitagawa, J.; Mérigot, Q.; Thibert, B.
  • Empirical regularized optimal transport: statistical theory and applications
    Klatt, M.; Tameling, C.; Munk, A.
  • On gradients of functions definable in o-minimal structures
    Kurdyka, K.
  • A multiscale approach to optimal transport
    Mérigot, Q.
  • An algorithm for optimal transport between a simplex soup and a point cloud
    Mérigot, Q.; Meyron, J.; Thibert, B.
  • Amplitude and phase variation of point processes
    Panaretos, V. M.; Zemel, Y.
  • Statistical aspects of Wasserstein distances
    Panaretos, V. M.; Zemel, Y.
  • Asymptotic almost sure efficiency of averaged stochastic algorithms
    Pelletier, M.
  • Computational optimal transport
    Peyré, G.; Cuturi, M.
  • Entropic optimal transport is maximum-likelihood deconvolution
    Rigollet, P.; Weed, J.
  • Convolutional Wasserstein distances: efficient optimal transportation on geometric domains
    Solomon, J.; Goes, F.; Peyré, G.; Cuturi, M.; Butscher, A.; Nguyen, A.; Du, T.; Guibas, L.
  • Inference for empirical Wasserstein distances on finite spaces
    Sommerfeld, M.; Munk, A.
  • Properties of the matrix A-XY
    Steerneman, A.; Perlo-ten Kleij, F.
  • Fréchet means and Procrustes analysis in Wasserstein space
    Zemel, Y.; Panaretos, V. M.
  • Central limit theorems of a recursive stochastic algorithm with applications to adaptive designs
    Zhang, L.-X.