Two new algorithms to compute steady-state bounds for Markov models with slow forward and fast backward transitions

Abstract

Two new algorithms are proposed for the computation of bounds for the steady-state reward rate of irreducible finite Markov models with slow forward and fast backward transitions. The algorithms use detailed knowledge of the model in a subset of generated states G and partial information about the model in the non-generated portion U of the state space. U is assumed partitioned into subsets U_k,1\leq k\leq N with a “nearest neighbor” structure. The algorithms involve the solution of, respectively, |M| + 2 and 4 linear systems of size |G|, where M is the set of values of k corresponding to the subsets U_k through which the model can jump from G to U. Previously proposed algorithms for the same type of models required the solution of |S| linear systems of size |G| + N , where S is the subset of G through which the model can enter G from U, to achieve the same bounds as our algorithms, or gave less tighter bounds if state cloning techniques were used to reduce the number of solved linear systems. An availability model with system state dependent repair rates is used to illustrate the application and performance of the algorithms.