Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Bounds on the rate of convergence of learning processes based on random sets and set‐valued probability

Bounds on the rate of convergence of learning processes based on random sets and set‐valued... Purpose – Bounds on the rate of convergence of learning processes based on random samples and probability are one of the essential components of statistical learning theory (SLT). The constructive distribution‐independent bounds on generalization are the cornerstone of constructing support vector machines. Random sets and set‐valued probability are important extensions of random variables and probability, respectively. The paper aims to address these issues. Design/methodology/approach – In this study, the bounds on the rate of convergence of learning processes based on random sets and set‐valued probability are discussed. First, the Hoeffding inequality is enhanced based on random sets, and then making use of the key theorem the non‐constructive distribution‐dependent bounds of learning machines based on random sets in set‐valued probability space are revisited. Second, some properties of random sets and set‐valued probability are discussed. Findings – In the sequel, the concepts of the annealed entropy, the growth function, and VC dimension of a set of random sets are presented. Finally, the paper establishes the VC dimension theory of SLT based on random sets and set‐valued probability, and then develops the constructive distribution‐independent bounds on the rate of uniform convergence of learning processes. It shows that such bounds are important to the analysis of the generalization abilities of learning machines. Originality/value – SLT is considered at present as one of the fundamental theories about small statistical learning. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Kybernetes Emerald Publishing

Bounds on the rate of convergence of learning processes based on random sets and set‐valued probability

Minghu Ha; Jiqiang Chen; Witold Pedrycz; Lu Sun
Kybernetes , Volume 40 (9/10): 27 – Oct 18, 2011
Download PDF
27 pages

Loading next page...
 
/lp/emerald-publishing/bounds-on-the-rate-of-convergence-of-learning-processes-based-on-nctORmYw3D
Publisher
Emerald Publishing
Copyright
Copyright © 2011 Emerald Group Publishing Limited. All rights reserved.
ISSN
0368-492X
DOI
10.1108/03684921111169486
Publisher site
See Article on Publisher Site

Abstract

Purpose – Bounds on the rate of convergence of learning processes based on random samples and probability are one of the essential components of statistical learning theory (SLT). The constructive distribution‐independent bounds on generalization are the cornerstone of constructing support vector machines. Random sets and set‐valued probability are important extensions of random variables and probability, respectively. The paper aims to address these issues. Design/methodology/approach – In this study, the bounds on the rate of convergence of learning processes based on random sets and set‐valued probability are discussed. First, the Hoeffding inequality is enhanced based on random sets, and then making use of the key theorem the non‐constructive distribution‐dependent bounds of learning machines based on random sets in set‐valued probability space are revisited. Second, some properties of random sets and set‐valued probability are discussed. Findings – In the sequel, the concepts of the annealed entropy, the growth function, and VC dimension of a set of random sets are presented. Finally, the paper establishes the VC dimension theory of SLT based on random sets and set‐valued probability, and then develops the constructive distribution‐independent bounds on the rate of uniform convergence of learning processes. It shows that such bounds are important to the analysis of the generalization abilities of learning machines. Originality/value – SLT is considered at present as one of the fundamental theories about small statistical learning.

Journal

Kybernetes – Emerald Publishing

Published: Oct 18, 2011

Keywords: Random sets; Set‐valued probability; The key theorem; VC dimension; Rate of uniform convergence; Learning processes; Probability theory

References

  • Set‐valued measures
    Artstein, Z.
  • A strong law of large numbers for random compact sets
    Artstein, Z.; Vitale, R.A.
  • Set‐valued Analysis
    Aubin, J.P.; Frankowska, H.
  • Integrals of set‐valued functions
    Aumann, R.J.
  • Support Vector Machine and Its Application in Mixed Gas Infrared Spectrum Analysis
    Bai, P.; Zhang, X.B.; Zhang, B.; Li, Y.; Xie, W.J.; Liu, J.H.
  • Pattern Recognition
    Bian, Z.Q.; Zhang, X.G.
  • Machine learning with data dependent hypothesis classes
    Cannon, A.; Ettinger, J.M.; Hush, D.; Scovel, C.
  • Laws of Large Numbers for Random Sets and Fuzzy Random Sets
    Chen, Y.P.
  • Probabilistic characterization of random decision trees
    Dhurandhar, A.; Dobra, A.
  • Theory of selection operators on hyperspaces and set‐valued stochastic processes
    Gao, Y.; Zhang, W.X.
  • VC theory of large margin multi‐category classifiers
    Guermeur, Y.
  • The theoretical foundations of statistical learning theory based on fuzzy number samples
    Ha, M.H.; Tian, J.
  • The key theorem and bounds on the rate of convergence of statistical learning theory on the fuzzy samples
    Ha, M.H.; Tian, J.; Zhou, C.L.
  • The key theorem and bounds on the rate of uniform convergence of learning theory in quasi‐probability measure space
    Ha, M.H.; Feng, Z.F.; Song, S.J.; Gao, L.Q.
  • The key theorem and bounds on the rate of uniform convergence of learning theory in Sugeno measure space
    Ha, M.H.; Li, Y.; Li, J.; Tian, D.Z.
  • Some theoretical results of learning theory based on random sets in set‐valued probability space
    Ha, M.H.; Pedrycz, W.; Chen, J.Q.; Zheng, L.F.
  • Generalization bounds and complexities based on sparsity and clustering for convex combinations of functions from random classes
    Jaeger, S.A.
  • Support vector machines with genetic fuzzy feature transformation for biomedical data classification
    Jin, B.; Tang, Y.C.; Zhang, Y.Q.
  • Foundation of a theory of random sets
    Kendall, D.G.
  • Comparison between error correcting output codes and fuzzy support vector machines
    Kikuchi, T.; Abe, S.
  • Topology
    Kuratowski, K.; Jaworowski, J.; Kirkor, A.
  • Strong laws of large numbers for independent fuzzy set‐valued random variables
    Li, S.M.; Ogura, Y.
  • Fuzzy support vector machines
    Lin, C.F.; Wang, S.D.
  • Random Sets and Integral Geometry
    Matheron, G.
  • On the convergence in distribution of measurable multifunctions (random sets), normal integrands, stochastic progresses and stochastic infima
    Salinetti, G.; Wets, R.J.B.
  • Application of support vector machines in financial time series forecasting
    Tay, F.E.H.; Cao, L.J.
  • Hybrid approach of selecting hyper‐parameters of support vector machine for regression
    Tsong, J.J.
  • The Nature of Statistical Learning Theory
    Vapnik, V.N.
  • Statistical Learning Theory
    Vapnik, V.N.
  • An overview of statistical learning theory
    Vapnik, V.N.
  • Edgeworth allocations in an exchange economy with many traders
    Vind, K.
  • On efficient large margin semisupervised learning: method and theory
    Wang, J.H.; Shen, X.T.; Pan, W.
  • A new fuzzy support vector machine to evaluate credit risk
    Wang, Y.Q.; Wang, S.Y.; Lai, K.K.
  • Set‐valued Measure and Random Sets
    Zhang, W.X.
  • Set‐valued probability and its properties
    Zhang, W.X.; Li, H.G.