Computation of absorption probability distributions of continuous-time Markov chains using regenerative randomization

Abstract

Randomization is a popular method for the transient solution of continuous-time Markov models. Its primary advantages over other methods (i.e., ODE solvers) are robustness and ease of implementation. It is however well-known that the performance of the method deteriorates with the “stiffness” of the model: the number of required steps to solve the model up to time t tends to \Ld t for \Ld t \rightarrow \infty, where \Ld as the maximum output rate. For measures like the unreliability \Ld t can be very large for the t of interest, making the randomization method very inefficient. In this paper we consider such measures and propose a new solution method called regenerative randomization which exploits the regenerative structure of the model and can be far more efficient. Regarding the number of steps required in regenerative randomizaizon we prove that:1) it is smaller than the number of steps required in standard randomization when the initial distribution is concentrated in a single state, 2) for \Ld t \rightarrow \infty, it is upper bounded by a function O(log(\Ld t/\eps)), where \eps is the desired approximation error hound. Using a reliability example we analyze the performance and stability of the method.