Markov Chain and Linear Algebra — Calculation of Stationary Distribution using Python

In Our Last Chapter, We have discussed about Random Walk in Markov Chain using python. Now, We shall dive into more detail into Markov Chain and its relation with linear algebra more theoretically.

Representing Markov Chain Using Linear Algebra

So, We are approaching this Model with Linear Algebra. As this is a directed graph, We can just replace this with Adjacent Matrix.

If You don’t what it is, We have already seen its usage in our last discussion with our approach of designing Transition Matrix using Pandas.

So, the Markov Chain above will translate into –

The Transition Matrix

The elements of the matrix just denote the weight of the edge connecting the two corresponding vertices. But, We have already seen the example in our last discussion. So,

• The element of the second row and first column denotes the weight or the Transition Probability from a Downside state to an Upside state.
• If an element is 0, it means there is no edge between two vertices.

This matrix is called as “Transition Matrix”. Let’s denote it as A.

Forecasting Futures Probabilities of States

Remember that, Our goal is to find the probabilities of each state. So…