eCite Digital Repository

Second-order Markov reward models driven by QBD processes


Bean, NG and O'Reilly, MM and Ren, Yong, Second-order Markov reward models driven by QBD processes, Performance Evaluation, 69, (9) pp. 440-455. ISSN 0166-5316 (2012) [Refereed Article]

Copyright Statement

Crown Copyright 2012

DOI: doi:10.1016/j.peva.2012.05.002


Second-order reward models are an important class of models for evaluating the performance of real-life systems in which the reward measure fluctuates according to some underlying noise. These models consist of a Markov chain driving the evolution of the system, and a continuous reward variable representing its performance. Thus far, only models with a finite number of states have been studied. We consider second-order reward models driven by Quasi-birth-and-death processes, a class of block-structured Markov chains with infinitely many states. We derive the expressions for the Laplace-Stieltjes transforms of the accumulated reward and demonstrate how they can be efficiently evaluated. We use our results to analyse a simple example and, in doing so, show that the second-order feature can make a significant difference to the accumulated reward. The inclusion of the second-order feature also creates new difficulties which require the development of new conditions in the analysis.

Item Details

Item Type:Refereed Article
Keywords:reward model, quasi-birth-and-death (QBD) process, RG-factorization, Brownian motion
Research Division:Mathematical Sciences
Research Group:Statistics
Research Field:Stochastic analysis and modelling
Objective Division:Expanding Knowledge
Objective Group:Expanding knowledge
Objective Field:Expanding knowledge in the mathematical sciences
UTAS Author:O'Reilly, MM (Associate Professor Malgorzata O'Reilly)
UTAS Author:Ren, Yong (Dr Yong Ren)
ID Code:80964
Year Published:2012
Funding Support:Australian Research Council (DP1111663)
Web of Science® Times Cited:1
Deposited By:Mathematics and Physics
Deposited On:2012-11-19
Last Modified:2015-01-27

Repository Staff Only: item control page