Problems of Information Transmission, Vol. 39, No. 1, 2003, pp. 119–147. Translated from Problemy Peredachi Informatsii, No. 1, 2003, pp. 134–165.
Original Russian Text Copyright
2003 by Muchnik, Semenov.
On the Role of the Law of Large Numbers
in the Theory of Randomness
An. A. Muchnik and A. L. Semenov
Institute of New Technologies, Nizhnyaya Radishchevskaya 10, Moscow, 109004 Russia
Abstract—In the ﬁrst part of this article, we answer Kolmogorov’s question (stated in 1963
in ) about exact conditions for the existence of random generators. Kolmogorov theory
of complexity permits of a precise deﬁnition of the notion of randomness for an individual
sequence. For inﬁnite sequences, the property of randomness is a binary property, a sequence
can be random or not. For ﬁnite sequences, we can solely speak about a continuous property,
a measure of randomness. Is it possible to measure randomness of a sequence t by the extent to
which the law of large numbers is satisﬁed in all subsequences of t obtained in an “admissible
way”? The case of inﬁnite sequences was studied in . As a measure of randomness (or,
more exactly, of nonrandomness) of a ﬁnite sequence, we consider the speciﬁc deﬁciency of
randomness δ (Deﬁnition 5). In the second part of this paper, we prove that the function
δ/ ln(1/δ) characterizes the connections between randomness of a ﬁnite sequence and the extent
to which the law of large numbers is satisﬁed.
In 1930-s Andrei Kolmogorov founded probability theory on the base of measure theory. In 
he writes: “The set theoretic axioms of the calculus of probability . . . had solved the majority of
formal diﬃculties in the construction of a mathematical apparatus which is useful for a very large
number of applications of probabilistic methods so successfully that the problem of ﬁnding the basis
of real applications of the results of the mathematical theory of probability became of secondary
importance to many investigators.”
However, Kolmogorov himself regarded the question about the basis as a principal one. In 1962,
during his visit to India, he began to develop a new approach to it,
the so-called theory of
descriptive complexity. Now the research area initiated by Kolmogorov has grown into a rich
theory, which has important connections not only with probability theory but also with theory of
algorithms, theory of coding, theory of matroids, and other ﬁelds of mathematics.
As for practical applications, the main results are to appear in future. To get them, it is
required to take into account not only the descriptive complexity of a program but also an amount of
resources used by it. In many cases, this task is connected with unsolved problems of computational
Consider a sequence of independent trials with two equiprobable outcomes, 0 and 1. The simplest
and at the same time the most signiﬁcant condition of randomness for a sequence of outcomes is an
approximate equality of the number of zeros and the number of ones. Obviously, this requirement
alone is not suﬃcient (for instance, the sequence 0101010101 ... does not seem to be random).
But if this condition holds for all subsequences obtained from the original one with the help of
Supported in part by the Russian Foundation for Basic Research, project nos. 01-01-00505, 02-01-10904,
and 02-01-22001, and the Council on Grants for Scientiﬁc Schools.
The ﬁrst publication appeared in 1963, see .
2003 MAIK “Nauka/Interperiodica”