Appl Math Optim 40:377–392 (1999)
1999 Springer-Verlag New York Inc.
Envelopes of Sets of Measures, Tightness, and
Markov Control Processes
and O. Hern´andez-Lerma
Departamento de Probabilidad y Estad´ıstica, IIMAS-UNAM,
Apartado Postal 20-726, M´exico D.F. 01000, M´exico
Departamento de Matem´aticas, CINVESTAV-IPN,
Apartado Postal 14-740, M´exico D.F. 07000, M´exico
Abstract. We introduce upper and lower envelopes for sets of measures on an
arbitrary topological space, which are then used to give a tightness criterion. These
concepts are applied to show the existence of optimal policies for a class of Markov
Key Words. Envelopes of measures, Tightness criteria, (Discrete-time) Markov
AMS Classiﬁcation. 93E20, 28C15.
A well-known fact in control theory is that a large class of optimal control problems
(deterministic or stochastic, in discrete or continuous time—see , , –, ,
and [21–) can be transformed into minimization problems over sets of measures. In
this case, the control problem is typically reduced to the form:
cdµ subject to µ ∈ M(X), (1.1)
where M(X) is a set of measures on some space X, and c denotes the control problem’s
running cost. Moreover, under mild assumptions on c and X, and endowing M(X ) with
This research was partially supported by the Consejo Nacional de Ciencia y Tecnolog´ıa (CONACYT)
Grant 3115P-E9608. The work of the ﬁrst author was also supported by a CONACYT scholarship.