Appl Math Optim 37:269–293 (1998)
1998 Springer-Verlag New York Inc.
Adaptive Control of a Partially Observed
Discrete Time Markov Process
T. E. Duncan,
and L. Stettner
Department of Mathematics, University of Kansas,
Lawrence, KS 66045, USA
Institute of Mathematics, Polish Academy of Sciences,
00-950 Warsaw, Poland
Communicated by D. Ocone
Abstract. An adaptive control problem of a discrete time Markov process that is
completely observed in a ﬁxed recurrent domain and is partially observed elsewhere
is formulated and a solution is given by constructing an approximately self-optimal
strategy. The state space of the Markov process is either a closed subset of Euclidean
space or a countable set. Another adaptive control problem is solved where the
process is always only partially observed but there is a family of random times
when the process evaluated at these times is a family of independent, identically
distributed random variables.
Key Words. Stochastic adaptive control, Partially observed stochastic systems,
Discrete time Markov processes.
AMS Classiﬁcation. 93E35, 93E12, 60J05.
The results that are available for the adaptive control of a partially observed stochastic
system are quite limited. For the adaptive control of a discrete time, partially observed
Markov process there are an elementary maintenance model in  and a description of
a general methodology in . In this paper an almost self-optimal control for an ergodic
This research was partially supported by NSF Grant DMS-9305936.