The censored Markov chain and the best augmentation
Computationally, when we solve for the stationary
probabilities for a countable-state Markov chain, the transition
probability
matrix of the Markov chain has to be truncated, in some way, into
a finite matrix.
Different augmentation methods might be valid such that the
stationary probability distribution for the truncated Markov chain approaches
that for the countable Markov chain as the truncation size gets large.
In this paper, we prove that
the censored (watched) Markov chain provides the best approximation
in the sense that for a given truncation size the sum of errors
is the minimum and show, by examples, that
the method of augmenting the last column only is not always the best.
Back to my
home page.