University of Tasmania
Browse

File(s) under permanent embargo

Second-order Markov reward models driven by QBD processes

journal contribution
posted on 2023-05-17, 14:25 authored by Bean, NG, Malgorzata O'ReillyMalgorzata O'Reilly, Ren, Yong
Second-order reward models are an important class of models for evaluating the performance of real-life systems in which the reward measure fluctuates according to some underlying noise. These models consist of a Markov chain driving the evolution of the system, and a continuous reward variable representing its performance. Thus far, only models with a finite number of states have been studied. We consider second-order reward models driven by Quasi-birth-and-death processes, a class of block-structured Markov chains with infinitely many states. We derive the expressions for the Laplace-Stieltjes transforms of the accumulated reward and demonstrate how they can be efficiently evaluated. We use our results to analyse a simple example and, in doing so, show that the second-order feature can make a significant difference to the accumulated reward. The inclusion of the second-order feature also creates new difficulties which require the development of new conditions in the analysis.

Funding

Australian Research Council

History

Publication title

Performance Evaluation

Volume

69

Issue

9

Pagination

440-455

ISSN

0166-5316

Department/School

School of Natural Sciences

Publisher

Elsevier BV * North-Holland

Place of publication

Netherlands

Rights statement

Crown Copyright 2012

Repository Status

  • Restricted

Socio-economic Objectives

Expanding knowledge in the mathematical sciences

Usage metrics

    University Of Tasmania

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC