Author ORCID Identifier
https://orcid.org/0000-0001-8502-6549 : Yiping Zhang
Keywords
Markov Chain, K-th Passage Probability, Mean Passage Steps, Collective Marks Method, Laplace Transforms
Abstract
In this paper, we consider the extension of first passage probability. First, we present the first, second, third, and generally k-th passage probability of a Markov Chain moving from one state to another state through step-by-step calculation and two other matrix-version methods. Similarly, we compute the first passage probability of a Markov Chain moving from one state to multiple states. In all discussions, we take into account the situations that one state moves to a different state and returns to itself. Also, we find the mean number of steps needed from one state to another state in a Markov Chain for the first, second, third, and generally k-th passage. Besides, we find the probability generating function for the number of steps. This makes the calculation of passage probabilities, mean and variance of passage steps, easier. Additionally, if we extend a discrete Markov Chain to its corresponding continuous Markov Process with the same transition probabilities and transition time in the form of an exponential distribution with parameter 1 between two states, we can obtain the mean time needed from one state to another state by Laplace Transforms, which is the same as with the discrete situation. Subsequently, we can calculate the variance of the time needed from one state to another state in the same way.
Primary Advisor
Myron Hlynka
Co-Advisor
Percy Brill
Program Reader
Mohamed Belalia
Degree Name
Master of Science
Department
Mathematics and Statistics
Document Type
Major Research Paper
Convocation Year
2019