Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Stochastic Backward Euler: An Implicit Gradient Descent Algorithm for k-Means Clustering

Stochastic Backward Euler: An Implicit Gradient Descent Algorithm for k-Means Clustering In this paper, we propose an implicit gradient descent algorithm for the classic k-means problem. The implicit gradient step or backward Euler is solved via stochastic fixed-point iteration, in which we randomly sample a mini-batch gradient in every iteration. It is the average of the fixed-point trajectory that is carried over to the next gradient step. We draw connections between the proposed stochastic backward Euler and the recent entropy stochastic gradient descent for improving the training of deep neural networks. Numerical experiments on various synthetic and real datasets show that the proposed algorithm provides better clustering results compared to k-means algorithms in the sense that it decreased the objective function (the cluster) and is much more robust to initialization. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Scientific Computing Springer Journals

Stochastic Backward Euler: An Implicit Gradient Descent Algorithm for k-Means Clustering

Download PDF
14 pages

Loading next page...
 
/lp/springer_journal/stochastic-backward-euler-an-implicit-gradient-descent-algorithm-for-k-KH505woBm1
Publisher
Springer Journals
Copyright
Copyright © 2018 by Springer Science+Business Media, LLC, part of Springer Nature
Subject
Mathematics; Algorithms; Computational Mathematics and Numerical Analysis; Mathematical and Computational Engineering; Theoretical, Mathematical and Computational Physics
ISSN
0885-7474
eISSN
1573-7691
DOI
10.1007/s10915-018-0744-4
Publisher site
See Article on Publisher Site

Abstract

In this paper, we propose an implicit gradient descent algorithm for the classic k-means problem. The implicit gradient step or backward Euler is solved via stochastic fixed-point iteration, in which we randomly sample a mini-batch gradient in every iteration. It is the average of the fixed-point trajectory that is carried over to the next gradient step. We draw connections between the proposed stochastic backward Euler and the recent entropy stochastic gradient descent for improving the training of deep neural networks. Numerical experiments on various synthetic and real datasets show that the proposed algorithm provides better clustering results compared to k-means algorithms in the sense that it decreased the objective function (the cluster) and is much more robust to initialization.

Journal

Journal of Scientific Computing – Springer Journals

Published: May 31, 2018

References

  • Linearly constrained nonsmooth and nonconvex minimization
    Artina, M; Fornasier, M; Solombrino, F
  • Subdominant dense clusters allow for simple learning and high computational performance in neural networks with discrete synapses
    Baldassi, C; Ingrosso, A; Lucibello, C; Saglietti, L; Zecchina, R
  • Nonlinear Programming
    Bertsekas, DP
  • Convergence properties of the $$k$$ k -means algorithms
    Bottou, L; Bengio, Y
  • Proximal point method and nonconvex optimization
    Kaplan, A; Tichatschke, R
  • Deep learning
    LeCun, Y; Bengio, Y; Hinton, G
  • Gradient-based learning applied to document recognition
    LeCun, Y; Bottou, L; Bengio, Y; Haffner, P
  • Least squares quantization in PCM
    Lloyd, S
  • Proximité et dualité dans un espace hilbertien
    Moreau, J-J
  • Monotone operators and the proximal point algorithm
    Rockafellar, R