Stochastic Models

Eisenberg, Julia

doi: 10.1080/15326349.2017.1392867pmid: 30532661

We consider an individual or household endowed with an initial capital and an income, modeled as a linear function of time. Assuming that the discount rate evolves as an Ornstein–Uhlenbeck process, we target to find an unrestricted consumption strategy such that the value of the expected discounted consumption is maximized. Differently than in the case with restricted consumption rates, we can determine the optimal strategy and the value function.

Kubicka, Ewa M.; Kubicki, Grzegorz; Kuchta, Małgorzata; Morayne, Michał

doi: 10.1080/15326349.2017.1402347pmid: N/A

A gambler buys N tokens that enable him to play N rounds of the following game. A symmetric random walk on a discrete interval { − r, …, r} starts from the point 0. The gambler knows only the number of steps made so far, but is unaware of the current position of the walk. Once the walk hits one of the barriers − r or r for the first time in the current round, the round ends with no payoff. The gambler can start a new round by inserting a new token, if there are any tokens left. The gambler can end the game at any time getting the payoff equal to the number of steps made in the current round. We find the optimal stopping strategy for this game and calculate the expected payoff once the optimal strategy is applied.

Lefèvre, Claude; Simon, Matthieu

doi: 10.1080/15326349.2017.1411818pmid: N/A

A block-structured transient Markov process is introduced to describe an epidemic spreading within two linked populations, of carriers and susceptibles. The epidemic terminates as soon as there are no more carriers or susceptibles present in the population. Our purpose is to determine the distribution of the final susceptible and carrier states, and of any integral path for the susceptible process. The transient epidemic state is also briefly discussed. Then, the model is extended to allow the recovery of infected individuals. Finally, several particular models, some known, are used for illustration.

Ko, Young Myoung; Pender, Jamol

doi: 10.1080/15326349.2018.1425886pmid: N/A

In real stochastic systems, the arrival and service processes may not be renewal processes. For example, in many telecommunication systems such as internet traffic where data traffic is bursty, the sequence of inter-arrival times and service times are often correlated and dependent. One way to model this non-renewal behavior is to use Markovian Arrival Processes (MAPs) and Markovian Service Processes (MSPs). MAPs and MSPs allow for inter-arrival and service times to be dependent, while providing the analytical tractability of simple Markov processes. To this end, we prove fluid and diffusion limits for MAPt/MSPt/∞ queues by constructing a new Poisson process representation for the queueing dynamics and leveraging strong approximations for Poisson processes. As a result, the fluid and diffusion limit theorems illuminate how the dependence structure of the arrival or service processes can affect the sample path behavior of the queueing process. Finally, our Poisson representation for MAPs and MSPs is useful for simulation purposes and may be of independent interest.

Kimura, Masatoshi; Takine, Tetsuya

doi: 10.1080/15326349.2018.1451753pmid: N/A

This paper considers the computation of the conditional stationary distribution in Markov chains of level-dependent M/G/1-type, given that the level is not greater than a predefined threshold. This problem has been studied recently and a computational algorithm is proposed under the assumption that matrices representing downward jumps are nonsingular. We first show that this assumption can be eliminated in a general setting of Markov chains of level-dependent G/G/1-type. Next we develop a computational algorithm for the conditional stationary distribution in Markov chains of level-dependent M/G/1-type, by modifying the above-mentioned algorithm slightly. In principle, our algorithm is applicable to any Markov chain of level-dependent M/G/1-type, if the Markov chain is irreducible and positive-recurrent. Furthermore, as an input to the algorithm, we can set an error bound for the computed conditional distribution, which is a notable feature of our algorithm. Some numerical examples are also provided.

Kuhn, Julia; Mandjes, Michel

doi: 10.1080/15326349.2018.1458629pmid: N/A

This paper considers a multi-server queue with Markov-modulated Poisson input and server-dependent phase-type service times. We develop an efficient rare-event simulation technique to estimate the probability that the number of customers in this system reaches a high value. Relying on explicit bounds on the probability under consideration as well as the associated likelihood ratio, we succeed in proving that the proposed estimator is of bounded relative error. Simulation experiments illustrate the significant speed-up that can be achieved by the proposed algorithm.