Access the full text.
Sign up today, get DeepDyve free for 14 days.
A. Ehrenfeucht, D. Haussler, M. Kearns, L. Valiant (1988)
A general lower bound on the number of examples needed for learningInf. Comput., 82
M. Kearns, Ming Li (1993)
Learning in the presence of malicious errorsSIAM J. Comput., 22
J. Littlewood (1969)
On the probability in the tail of a binomial distributionAdvances in Applied Probability, 1
R. Bahadur (1960)
Some Approximations to the Binomial Distribution FunctionAnnals of Mathematical Statistics, 31
R. Schapire (1992)
Toward Eecient Agnostic Learning
M. Kearns, R. Schapire (1990)
Efficient distribution-free learning of probabilistic conceptsProceedings [1990] 31st Annual Symposium on Foundations of Computer Science
P. Laird (1988)
Learning from Good and Bad Data
S. Shelah (1972)
A combinatorial problem; stability and order for models and theories in infinitary languages.Pacific Journal of Mathematics, 41
R. Bahadur, R. Rao (1960)
On Deviations of the Sample MeanAnnals of Mathematical Statistics, 31
C. Gentile, D. Helmbold (1998)
Improved lower bounds for learning from noisy examples: an information-theoretic approachInf. Comput., 166
K. Jogdeo, S. Samuels (1968)
Monotone Convergence of Binomial Probabilities and a Generalization of Ramanujan's EquationAnnals of Mathematical Statistics, 39
H. Simon (1993)
General bounds on the number of examples needed for learning probabilistic conceptsJ. Comput. Syst. Sci., 52
David Editor
Artificial Intelligence and Language Processing a Theory of the Learnable
V. Vapnik (1982)
Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics)
A. Blumer, A. Ehrenfeucht, D. Haussler, Manfred Warmuth (1989)
Learnability and the Vapnik-Chervonenkis dimensionJ. ACM, 36
N. Sauer (1972)
On the Density of Families of SetsJ. Comb. Theory, Ser. A, 13
Hu Jin (2021)
Probability TheoryFoundations of Constructive Probability Theory
M. Kearns, R. Schapire, Linda Sellie (1992)
Toward efficient agnostic learningMachine Learning, 17
In this paper, we prove various results about PAC learning in the presence of malicious noise. Our main interest is the sample size behavior of learning algorithms. We prove the first nontrivial sample complexity lower bound in this model by showing that order of ॉ/ख 2 + d /ख (up to logarithmic factors) examples are necessary for PAC learning any target class of {0,1}-valued functions of VC dimension d , where ॉ is the desired accuracy and ॑ = ॉ/(1 + ॉ) - ख the malicious noise rate (it is well known that any nontrivial target class cannot be PAC learned with accuracy ॉ and malicious noise rate ॑ ≥ ॉ/(1 + ॉ), this irrespective to sample complexity). We also show that this result cannot be significantly improved in general by presenting efficient learning algorithms for the class of all subsets of d elements and the class of unions of at most d intervals on the real line. This is especialy interesting as we can also show that the popular minimum disagreement strategy needs samples of size d ॉ/ख 2 , hence is not optimal with respect to sample size. We then discuss the use of randomized hypotheses. For these the bound ॉ/(1 + ॉ) on the noise rate is no longer true and is replaced by 2ॉ/(1 + 2ॉ). In fact, we present a generic algorithm using randomized hypotheses that can tolerate noise rates slightly larger than ॉ/(1 + ॉ) while using samples of size d /ॉ as in the noise-free case. Again one observes a quadratic powerlaw (in this case d ॉ/ख 2 , ख = 2ॉ/(1 + 2ॉ) - ॑) as ख goes to zero. We show upper and lower bounds of this order.
Journal of the ACM (JACM) – Association for Computing Machinery
Published: Sep 1, 1999
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.