Approximation of the exit probability of a stable Markov modulated constrained random walk

Creative Commons License

Kabran F. B. , SEZER A. D.

ANNALS OF OPERATIONS RESEARCH, 2020 (Journal Indexed in SCI) identifier identifier

  • Publication Type: Article / Article
  • Volume:
  • Publication Date: 2020
  • Doi Number: 10.1007/s10479-020-03693-7


Let X be the constrained randomwalk on Z(+)(2) having increments (1, 0), (- 1, 1), (0,- 1) with jump probabilities lambda(M-k), mu(1)(M-k), and mu(2)(M-k) where M is an irreducible aperiodic finite state Markov chain. The process X represents the lengths of two tandem queues with arrival rate lambda(M-k), and service rates mu(1)(M-k), and mu(2)(M-k); the process M represents the random environment within which the system operates. We assume that the average arrival rate with respect to the stationary measure of M is less than the average service rates, i.e., X is assumed stable. Let tau(n) be the first time when the sum of the components of X equals n for the first time. Let Y be the random walk on ZxZ(+) having increments (- 1, 0), (1, 1), (0,- 1) with probabilities lambda(M-k), mu(1)(M-k), and mu(2)(M-k). Supposing that the queues share a joint buffer of size n, p(n) = P(xn(,)m)(tau(n) < tau(0)) is the probability that this buffer overflows during a busy cycle of the system. To the best of our knowledge, the only methods currently available for the approximation of p(n) are classical large deviations analysis giving the exponential decay rate of pn and rare event simulation. Let tau be the first time the components of Y are equal. For x epsilon R-+(2), x(1)+ x(2) < 1, x(1) > 0, and x(n) = left perpendicular nx right perpendicular , we show that P-(n-xn (1),P-xn(2),P-m)(tau < infinity) approximates P(x(n),m)(tau(n) < tau(0)) with exponentially vanishing relative error as n -> infinity. For the analysis we define a characteristic matrix in terms of the jump probabilities of (X, M). The 0-level set of the characteristic polynomial of this matrix defines the characteristic surface; conjugate points on this surface and the associated eigenvectors of the characteristic matrix are used to define (sub/super) harmonic functions which play a fundamental role both in our analysis and the computation/approximation of P-(y,P-m)(tau < infinity).