Appl Math Optim 38:261–281 (1998)
1998 Springer-Verlag New York Inc.
Ergodic Control of a Singularly Perturbed Markov Process in
Discrete Time with General State and Compact Action Spaces
T. R. Bielecki
and L. Stettner
Department of Mathematics, Statistics, and Computer Science,
University of Illinois at Chicago,
Chicago, IL 60607, USA
Institute of Mathematics, Polish Academy of Sciences,
00-950 Warszawa, Poland
Communicated by D. Ocone
Abstract. Ergodic control of singularly perturbed Markov chains with general
state and compact action spaces is considered. A new method is given for character-
ization of the limit of invariantmeasures, for perturbed chains, when the perturbation
parameter goes to zero. It is also demonstrated that the limit control principle is sat-
isﬁed under natural ergodicity assumptions about controlled Markov chains. These
assumptions allow for the presence of transient states, a situation that has not been
considered in the literature before in the context of control of singularly perturbed
Markov processes with long-run-average cost functionals.
Key Words. Markov process, Invariant measure, Singular perturbation, Ergodic
control, Limit control principle.
AMS Classiﬁcation. Primary 60J05, 47A55, Secondary 90C40.
Numerous dynamic optimization problems are distinguished by the presence of so-
called strong and weak interactions characterizing the dynamics of the problems (see,
Present address: Department of Mathematics, Northeastern Illinois University, Chicago, IL 60625,