Gaussian Elimination

Gaussian Elimination is used to solve a system of linear equations. It transforms the matrix into a reduced row-echelon form.

The transformation is completed with the row operations. We can swap rows, multiply rows by a nonzero number, and adding one row to another. In such a way, the matrix will become an upper triangular matrix whose all leading coefficients are 1.

for example, the following equations can be transform to the matrix: $$ \begin{align} \nonumber r_0 + r_1 = 26 \
\nonumber r_1 + r_2 = 20 \
\nonumber r_0 = 2r_2 \
\end{align} $$

$$ \begin{vmatrix} 1 && 1 && 0 && 26 \
0 && 1 && 1 && 20 \
1 && 0 && -2 && 0 \
\end{vmatrix} $$

The reduced row chelon form will be: $$ \begin{vmatrix} 1 && 0 && 0 && -14 \
0 && 1 && 0 && 40 \
0 && 0 && 1 && -20 \
\end{vmatrix} $$ Using bakwward substitute, we can have:

$$ \begin{align} \nonumber r_0 = -14 \
\nonumber r_1 = 40 \
\nonumber r_2 = -20 \end{align} $$

Matrix Inversion

2x2 matrix inversion can be calculated with the fixed equation: $$ \begin {vmatrix} a && b \
c && d \
\end {vmatrix} ^ {-1}

\frac{1}{ad-bc} \begin {vmatrix} d && -b \
-c && a \
\end {vmatrix} $$

3x3 matrix inversion is calculated using Minors, Cofactors, and Adjugate, but I want to show a more general approach, the Gauss-Jordan method.

Suppose we have a matrix A:

$$ \begin {vmatrix} 3 && 0 && 2 \
2 && 0 && -2 \
0 && 1 && 1 \
\end {vmatrix} $$ To calculate the inverse, we will add an identity matrix on the right side to create a new augmented matrix:

$$ \begin {vmatrix} 3 && 0 && 2 && 1 && 0 && 0\
2 && 0 && -2 && 0 && 1 && 0\
0 && 2 && 1 && 0 && 0 && 1\
\end {vmatrix} $$ Then we use row operations to transfrom the left 3x3 matrix to identity matrix.

$$ \begin {vmatrix} 1 && 0 && 0 && 0.2 && 0.2 && 0\
0 && 1 && 0 && -0.2 && 0.3 && 1\
0 && 0 && 1 && 0.2 && -0.3 && 0\
\end {vmatrix} $$ And the 3x3 matrix in the right is the inverse.

Markov Chain

patrickJMT has a series of nice videos explaining Markov Chain. Consider a company A initially has a 10% market share, other companies (A') hold the 90% share. By using an advertising campaign, the rate of keeping choosing A is 80%, the rate of changing to A is 60%. In other words, from A to A is 80%, A' to A is 60%, and A to A' is 20%, A' to A' is 40%. So we have this Transition Matrix:

$$ \begin{vmatrix} 0.8 && 0.2 \
0.6 && 0.4 \end{vmatrix} $$ Suppose the Initial State of the rate of choosing A is 0.1.

$$ S_{0} = \begin {vmatrix} 0.1 && 0.9 \end {vmatrix} $$ After multiplying $S_0$ and Transition matrix, we’ll have $$ S_{1} = \begin {vmatrix} 0.42 && 0.38 \end {vmatrix} $$

That means, after one advertising campaign, the rate of choosing A increase to 42%! Keep multipling, we’ll realize that $S$ is approaching to $\begin {vmatrix} 0.75 && 0.25 \end {vmatrix}$. And this is the Stationary Matrix, and the system is at Steady State.

Absorbing Markov Chain

If the possibility of entering another state is zero then the state is an Absorbing State. For example: $$ \begin{vmatrix} 1 && 0 \
0.3 && 0.7 \end{vmatrix} $$

The first row is an absorbing state.

A standard form of absorbing markov chain is:

$$ \begin{vmatrix} I && O \
R && Q \end{vmatrix} $$

I is the identity matrix, O is a zero matrix. R and Q are two submatrices. The limiting matrix form is: $$ \begin{vmatrix} I && O \
FR && O \end{vmatrix} \
F = (I - Q)^{-1} $$