Problems of Information Transmission, Vol. 39, No. 3, 2003, pp. 266–293. Translated from Problemy Peredachi Informatsii, No. 3, 2003, pp. 40–71.
Original Russian Text Copyright
2003 by Golubtsov, Lyubetsky.
Stochastic Dynamic Games
with Various Types of Information
P. V. Golubtsov
M.V. Lomonosov Moscow State University
Institute for Information Transmission Problems, RAS, Moscow
Received April 29, 2002
Abstract—Dynamic discrete-time games are generalized to a stochastic environment, in order
to examine the inﬂuence of various types of information structures on the course of a game.
It is shown that the information structure of a game, i.e., type and amount of information
available to players and, in particular, asymmetry of information, may lead to unexpected and
sometimes counter-intuitive eﬀects on the game result, i.e., the players’ payoﬀs. The paper also
develops algorithms for obtaining the Nash equilibrium strategies in such games. These involve
reducing optimal reaction policies to the corresponding dynamic programming algorithms and
generalizing the classical optimal control technique. Results of computer simulations for a
variant of ﬁshery harvesting game are presented.
The paper studies a class of stochastic dynamic discrete-time games with explicit account of
information available to the players.
We explore a wide range of information structures in our games in order to study the role of
information available to players in constructing optimal strategies (in other words, we want to study
eﬀects of contextual use of information in the behavior of players). Thus, we are dealing with the
semantic aspect of information (i.e., semantic information).
Two independent players perform control of a common discrete-time dynamic system, which is
also aﬀected by random disturbing eﬀects. At every time step, the players make decisions based
on some information about these eﬀects. Speciﬁcally, each player selects his strategy in order to
maximize his total discounted payoﬀ for a long enough time period provided that the other player
follows some ﬁxed strategy. This means that the course of the game is described by the dynamic
Random eﬀects mentioned above are represented by a random Markov parameter (which may
We shall explore a wide range of information structures  in our games. In particular, players
may possess various levels of knowledge about realizations of a random parameter, e.g., current or
delayed information [2, 3], or even information obtained from imperfect observation of a random
parameter. Moreover, information structure may be asymmetric; e.g., one player may possess
full current information while the other has only delayed or imperfect information. We shall also
consider cooperative games with diﬀerent types of information structures, etc.
For each variant of the game, we propose an algorithm that reduces the original problem to a
multiple construction of the Nash equilibrium in a certain ﬁnite-dimensional space. Moreover, in
the most complicated case considered in Section 5.3, we construct an algorithm that reduces the
2003 MAIK “Nauka/Interperiodica”