A.Kaw矩阵代数初步学习笔记 10. Eigenvalues and Eigenvectors

“矩阵代数初步”(Introduction to MATRIX ALGEBRA)课程由Prof. A.K.Kaw(University of South Florida)设计并讲授。
PDF格式学习笔记下载（Academia.edu）
第10章课程讲义下载（PDF）

Summary

• Definition
  If $[A]$ is a $n\times n$ matrix, then $[X]\neq \vec0$ is an eigenvector of $[A]$ if $$[A][X] = \lambda[X]$$ where $\lambda$ is a scalar and $[X]\neq0$.
  The scalar $\lambda$ is called the eigenvalue of $[A]$ and $[X]$ is called the eigenvector corresponding to the eigenvalue $\lambda$.
• Finding eigenvalue and eigenvector
  • To find the eigenvalues of a $n\times n$ matrix $[A]$, we have $$AX=\lambda X$$ $$\Rightarrow AX-\lambda X=0$$ $$\Rightarrow (A-\lambda I)X=0$$ For the above set of equations to have a non-zero solution $$\det(A-\lambda I) = 0$$ The above equation is called the characteristic equation of $[A]$, which gives $$\lambda^n + c_1\lambda^{n-1} + \cdots + c_n=0$$ Hence this polynomial has $n$ roots.
  • For example, finding the eigenvalues of the matrix $$[A] = \begin{bmatrix}3& -1.5\\ -0.75& 0.75 \end{bmatrix}$$ We have $$A-\lambda I = \begin{bmatrix}3 - \lambda & -1.5\\ -0.75& 0.75 - \lambda \end{bmatrix}$$ $$\det(A - \lambda I) = (3-\lambda)(0.75-\lambda) - (-0.75)(-1.5)$$ $$\Rightarrow \lambda^2-3.75\lambda + 1.125 =0$$ $$\Rightarrow \lambda = {3.75\pm\sqrt{{3.75}^{2} - 4.5}\over2} = 3.421165,\ 0.3288354$$ That is, the eigenvalues are 3.421165 and 0.3288354.
  • To find the eigenvectors of the above matrix $[A]$. Let $[X] = \begin{bmatrix}x_1 \\ x_2\end{bmatrix}$ and we already have $\lambda_1 = 3.421165$ and $\lambda_2 = 0.3288354$.
    When $\lambda = 3.421165$, from the definition we have $$(A-\lambda I)X=0$$ $$\Rightarrow \left(\begin{bmatrix}3& -1.5\\ -0.75& 0.75 \end{bmatrix} - \begin{bmatrix}3.421165& 0\\ 0& 3.421165 \end{bmatrix} \right) \begin{bmatrix}x_1 \\ x_2\end{bmatrix} = 0$$ $$\Rightarrow \begin{bmatrix}-0.421165& -1.5\\ -0.75& -2.671165 \end{bmatrix} \begin{bmatrix}x_1 \\ x_2\end{bmatrix} = \begin{bmatrix}0 \\ 0\end{bmatrix}$$ $$\Rightarrow -0.421165x_1 -1.5x_2 = 0 \Rightarrow x_2 = -0.2807767x_1$$ that is, $$[X] = \begin{bmatrix}x_1\\ -0.2807767x_1\end{bmatrix} = x_1\begin{bmatrix}1 \\ -0.2807767\end{bmatrix}$$ Hence the eigenvector corresponding to $\lambda_1 = 3.421165$ is $$\begin{bmatrix}1 \\ -0.2807767\end{bmatrix}$$ Similarly, we have calculate the eigenvector corresponding to $\lambda_2 = 0.3288354$: $$ \left(\begin{bmatrix}3& -1.5\\ -0.75& 0.75 \end{bmatrix} - \begin{bmatrix}0.3288354& 0\\ 0& 0.3288354 \end{bmatrix} \right) \begin{bmatrix}x_1 \\ x_2\end{bmatrix} = 0$$ $$\Rightarrow \begin{bmatrix}2.671165& -1.5\\ -0.75& 0.4211646 \end{bmatrix} \begin{bmatrix}x_1 \\ x_2\end{bmatrix} = \begin{bmatrix}0 \\ 0\end{bmatrix}$$ $$\Rightarrow 2.671165x_1 -1.5x_2 = 0 \Rightarrow x_2 = 1.780776x_1$$ that is, $$[X] = \begin{bmatrix}x_1\\ 1.780776x_1\end{bmatrix} = x_1\begin{bmatrix}1 \\ 1.780776\end{bmatrix}$$ Hence the eigenvector corresponding to $\lambda_1 = 0.3288354$ is $$\begin{bmatrix}1 \\ 1.780776\end{bmatrix}$$
• Some related theorems
  • If $[A]$ is a $n\times n$ triangular matrix - upper triangular, lower triangular and diagonal, the eigenvalues of $[A]$ are the diagonal entries of $[A]$.
  • $\lambda = 0$ is an eigenvalue of $[A]$ if $[A]$ is a singular (non-invertible) matrix.
  • $[A]$ and $[A]^{T}$ have the same eigenvalues.
  • Eigenvalues of a symmetric matrix are real.
  • Eigenvectors of a symmetric matrix are orthogonal, but only for distinct eigenvalues.
  • $|\det(A)|$ is the product of the absolute values of the eigenvalues of $[A]$.
• Power Method
  • One of the most common methods used for finding eigenvalues and eigenvectors is the power method. It is used to find the largest eigenvalue in an absolute sense. Note that if this largest eigenvalues is repeated, this method will not work. Also this eigenvalue needs to be distinct.
  • The method is as follows:
    1. Assume a guess $X^{(0)}$ for the eigenvector in $$AX=\lambda X$$ equation. One of the entries of $X^{(0)}$ needs to be unity.
    2. Find $$Y^{(1)} = AX^{(0)}$$
    3. Scale $Y^{(1)}$ so that the chosen unity component remains unity. $$Y^{(1)} = \lambda^{(1)}X^{(1)}$$
    4. Repeat steps 2 and 3 with $X=X^{(1)}$ to get $X^{(2)}$.
    5. Repeat steps 2 and 3 until the value of the eigenvalue converges.
  • For example, using the power method, find the largest eigenvalue and the corresponding eigenvectors of $$[A] = \begin{bmatrix}1.5& 0& 1\\ -0.5& 0.5& -0.5\\ -0.5& 0& 0 \end{bmatrix}$$ given with the initial guess $\begin{bmatrix}1\\ 1\\ 1 \end{bmatrix}$.
    From the algorithm, we have $$AX^{(0)} = \begin{bmatrix}1.5& 0& 1\\ -0.5& 0.5& -0.5\\ -0.5& 0& 0 \end{bmatrix} \begin{bmatrix}1\\ 1\\ 1 \end{bmatrix} = \begin{bmatrix}2.5\\ -0.5\\ -0.5 \end{bmatrix}$$ $$\Rightarrow Y^{(1)} = 2.5\begin{bmatrix}1\\ -0.2\\ -0.2 \end{bmatrix} $$ so $\lambda^{(1)} = 2.5$ and $X^{(1)} = \begin{bmatrix}1\\ -0.2\\ -0.2 \end{bmatrix}$. Note that we choose the first element of $X^{(0)}$ to be unity. Then $$AX^{(1)} = \begin{bmatrix}1.5& 0& 1\\ -0.5& 0.5& -0.5\\ -0.5& 0& 0 \end{bmatrix} \begin{bmatrix}1\\ -0.2\\ -0.2 \end{bmatrix} = \begin{bmatrix}1.3\\ -0.5\\ -0.5 \end{bmatrix}$$ $$\Rightarrow Y^{(2)} = 1.3\begin{bmatrix}1\\ -0.3846\\ -0.3846 \end{bmatrix}$$ so $\lambda^{(2)} = 1.3$ and $X^{(2)} = \begin{bmatrix}1\\ -0.3846\\ -0.3846 \end{bmatrix}$.
    Thus far, the absolute relative approximate error in the eigenvalues is $$|\varepsilon| = \left|{\lambda^{(2)}-\lambda^{(1)}\over \lambda^{(2)}}\right| = \left|{1.3-2.5\over1.3}\right| = 0.9230769$$ Conducting further iterations, the eigenvalue after 5 iterations is 1.02459 and its absolute relative approximate error is 0.012441.
    The exact value of the eigenvalue is $\lambda = 1$ and the corresponding eigenvector is $$X=\begin{bmatrix}1\\-0.5\\-0.5 \end{bmatrix}$$
  • R code

    This function includes 4 parameters:

    • A is the target matrix;
    • x0 is the initial guess which is a vector;
    • eps is the tolerance of the error which can be modified;
    • maxit is the maximum number of iterations in the process.

    We can calculate the previous example by using this script:

    A = matrix(c(1.5, -0.5, -0.5, 0, 0.5, 0, 1, -0.5, 0), ncol = 3)
    PowerEigen(A, x0 = c(1, 1, 1))
    Converged after  23 iterations
    $value
    [1] 1
    $vector
         [,1]
    [1,]  1.0
    [2,] -0.5
    [3,] -0.5

Selected Problems

1. The eigenvalues $\lambda$ of matrix $[A]$ are found by solving the equation ( ).

Solution: $$|A-\lambda I| = 0$$

2. Find the eigenvalues and eigenvectors of $$[A] = \begin{bmatrix} 10& 9\\ 2& 3\end{bmatrix}$$ using the determinant method.

Solution: $$|A-\lambda I| = 0$$ $$\Rightarrow \det\left(\begin{bmatrix} 10-\lambda & 9\\ 2 & 3-\lambda\end{bmatrix}\right) = 0$$ $$\Rightarrow (10-\lambda)(3-\lambda) - 18=0$$ $$\Rightarrow \lambda^2 - 13\lambda +12 =0$$ $$\Rightarrow \lambda_1=1,\ \lambda_2=12$$ For $\lambda_1=1$, we have $$\begin{bmatrix} 10-\lambda & 9\\ 2 & 3-\lambda\end{bmatrix} \begin{bmatrix}x_1 \\ x_2\end{bmatrix} =\begin{bmatrix} 0\\ 0\end{bmatrix} $$ $$\Rightarrow \begin{bmatrix} 9 & 9\\ 2 & 2\end{bmatrix}\begin{bmatrix}x_1 \\ x_2\end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}$$ $$\Rightarrow x_2 =-x_1$$ $$\Rightarrow X=\begin{bmatrix}x_1\\ -x_1 \end{bmatrix} = x_1\begin{bmatrix} 1\\ -1\end{bmatrix}$$ Thus the eigenvector corresponding to $\lambda_1=1$ is $\begin{bmatrix} 1\\ -1\end{bmatrix}$. Similarly, we can find the second eigenvector corresponding to $\lambda_2=12$: $$\begin{bmatrix} 10-\lambda & 9\\ 2 & 3-\lambda\end{bmatrix} \begin{bmatrix}x_1 \\ x_2\end{bmatrix} =\begin{bmatrix} 0\\ 0\end{bmatrix} $$ $$\Rightarrow \begin{bmatrix} -2 & 9\\ 2 & -9\end{bmatrix}\begin{bmatrix}x_1 \\ x_2\end{bmatrix} = \begin{bmatrix} 0\\ 0 \end{bmatrix}$$ $$\Rightarrow -2x_1+9x_2 = 0 \Rightarrow x_2 = {2\over9}x_1$$ $$\Rightarrow X=\begin{bmatrix}x_1\\ {2\over9}x_1 \end{bmatrix} = x_1\begin{bmatrix} 1\\ {2\over9}\end{bmatrix} \Rightarrow \begin{bmatrix} 9\\ 2\end{bmatrix}$$ Thus the eigenvector corresponding to $\lambda_2=12$ is $\begin{bmatrix} 9\\ 2\end{bmatrix}$.

3. Find the eigenvalues and eigenvectors of $$[A] = \begin{bmatrix}4& 0& 1\\ -2& 0& 1\\ 2& 0& 1\end{bmatrix}$$ using the determinant method.

Solution:

First of all, we can read off that $\lambda = 0$ is an eigenvalue of this matrix since it is singular. Then from the definition we have $$|A-\lambda I| = 0$$ $$\Rightarrow \det\left( \begin{bmatrix} 4-\lambda & 0 & 1\\ -2 & -\lambda & 1\\ 2 & 0& 1-\lambda \end{bmatrix}\right) = 0$$ $$\Rightarrow (4-\lambda)\left[(-\lambda)(1-\lambda) - 0\right]+\left[1\cdot(0+2\lambda)\right] =0$$ $$\Rightarrow (4-\lambda)(\lambda^2-\lambda) +2\lambda= 0$$ $$\Rightarrow \lambda(-\lambda^2+5\lambda-4+2) =0$$ $$\Rightarrow \lambda(\lambda^2-5\lambda+2) =0$$ $$\Rightarrow \lambda_1=0,\ \lambda_2 = 4.561553,\ \lambda_3=0.4384472.$$ For $\lambda_1 =0$, we have $$\begin{bmatrix} 4-\lambda & 0 & 1\\ -2 & -\lambda & 1\\ 2 & 0& 1-\lambda \end{bmatrix}\begin{bmatrix}x_1\\ x_2\\ x_3 \end{bmatrix}=\begin{bmatrix}0\\ 0\\ 0 \end{bmatrix}$$ $$\Rightarrow \begin{bmatrix} 4 & 0 & 1\\ -2 & 0 & 1\\ 2 & 0& 1 \end{bmatrix}\begin{bmatrix}x_1\\ x_2\\ x_3 \end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0 \end{bmatrix}$$ The coefficient matrix is $$\begin{bmatrix} 4& 0& 1\\ -2& 0& 1\\ 2& 0& 1 \end{bmatrix} \Rightarrow \begin{bmatrix} 0& 0& 3\\ -2& 0& 1\\ 0& 0& 2 \end{bmatrix} \Rightarrow \begin{bmatrix} 0& 0& 0\\ -2& 0& 0\\ 0& 0& 2 \end{bmatrix}$$ that is, $x_1=x_3=0$ and $x_2$ is arbitrary. Hence the eigenvector corresponding to $\lambda_1=0$ is $\begin{bmatrix}0 \\ 1\\ 0 \end{bmatrix}$. For $\lambda_2= 4.561553$, we have $$\begin{bmatrix} 4-\lambda & 0 & 1\\ -2 & -\lambda & 1\\ 2 & 0& 1-\lambda \end{bmatrix} \begin{bmatrix}x_1\\ x_2\\ x_3 \end{bmatrix}=\begin{bmatrix}0\\ 0\\ 0 \end{bmatrix}$$ $$\Rightarrow \begin{bmatrix} -0.561553 & 0 & 1\\ -2 & -4.561553 & 1\\ 2 & 0& -3.561553 \end{bmatrix}\begin{bmatrix}x_1\\ x_2\\ x_3 \end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0 \end{bmatrix}$$ The coefficient matrix is $$\begin{bmatrix} -0.561553 & 0 & 1\\ -2 & -4.561553 & 1\\ 2 & 0& -3.561553 \end{bmatrix} \Rightarrow \begin{bmatrix} -0.561553 & 0 & 1\\ 0 & -4.561553 & -2.561553\\ 0 & 0& 0 \end{bmatrix}$$ $$\Rightarrow \begin{cases}x_1= 1.780776x_3\\ x_2 = -0.5615528x_3\end{cases}$$ where $x_3$ is arbitrary. Thus the eigenvector corresponding to $\lambda_2=4.561553$ is $\begin{bmatrix}1.780776\\ -0.5615528\\ 1 \end{bmatrix}$. For $\lambda_3= 0.4384472$, we have $$\begin{bmatrix} 4-\lambda & 0 & 1\\ -2 & -\lambda & 1\\ 2 & 0& 1-\lambda \end{bmatrix} \begin{bmatrix}x_1\\ x_2\\ x_3 \end{bmatrix}=\begin{bmatrix}0\\ 0\\ 0 \end{bmatrix}$$ $$\Rightarrow \begin{bmatrix} 3.561553 & 0 & 1\\ -2 & -0.4384472 & 1\\ 2 & 0& 0.5615528 \end{bmatrix} \begin{bmatrix} x_1\\ x_2\\ x_3 \end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0 \end{bmatrix}$$ The coefficient matrix is $$\begin{bmatrix} 3.561553 & 0 & 1\\ -2 & -0.4384472 & 1\\ 2 & 0& 0.5615528 \end{bmatrix} \Rightarrow \begin{bmatrix} 3.561553 & 0 & 1\\ 0 & -0.4384472 & 1.561553\\ 0 & 0& 0 \end{bmatrix}$$ $$\Rightarrow \begin{cases}x_1= -0.2807764x_3\\ x_2 = 3.561553x_3\end{cases}$$ where $x_3$ is arbitrary. Thus the eigenvector corresponding to $\lambda_3= 0.4384472$ is $\begin{bmatrix}-0.2807764\\ 3.561553 \\ 1 \end{bmatrix}$.

4. Find the eigenvalues of these matrices by inspection: (A) $\begin{bmatrix}2& 0& 0\\ 0& -3& 0\\ 0& 0& 6\end{bmatrix}$; (B) $\begin{bmatrix}3& 5& 7\\ 0& -2& 1\\ 0& 0& 0\end{bmatrix}$; (C) $\begin{bmatrix}2& 0& 0\\ 3& 5& 0\\ 2& 1& 6\end{bmatrix}$.

Solution:

The eigenvalues of a triangular matrix are the diagonal entries of the matrix. Thus, (A) $\lambda_1=2,\ \lambda_2=-3,\ \lambda_3=6$. (B) $\lambda_1=3,\ \lambda_2=-2,\ \lambda_3=0$. (C) $\lambda_1=2,\ \lambda_2=5,\ \lambda_3=6$.

5. Find the largest eigenvalue in magnitude and its corresponding vector by using the power method $$[A] = \begin{bmatrix}4& 0& 1\\ -2& 0& 1\\ 2& 0& 1 \end{bmatrix}$$ Start with an initial guess of the eigenvector as $\begin{bmatrix}1\\ -0.5\\ 0.5 \end{bmatrix}$.

Solution:

We will use the R script directly,

A = matrix(c(4, -2, 2, 0, 0, 0, 1, 1, 1), ncol = 3)
PowerEigen(A, x0 = c(1, -0.5, -0.5))

Converged after  9 iterations
$value
[1] 4.561553

$vector
           [,1]
[1,]  1.0000000
[2,] -0.3153416
[3,]  0.5615528

6. Prove if $\lambda$ is an eigenvalue of $[A]$, then ${1\over\lambda}$ is an eigenvalue of $[A]^{-1}$.

Solution:

We hope to prove that $A^{-1}X={1\over\lambda}X$ where $AX=\lambda X$. $$A^{-1}X=A^{-1}(\lambda \cdot {1\over\lambda}) X = {1\over\lambda} A^{-1}\lambda X = {1\over\lambda} A^{-1}A X = {1\over\lambda}X$$

7. Prove that square matrices $[A]$ and $[A]^{T}$ have the same eigenvalues.

Solution:

We hope to prove that $\det(A-\lambda I) = \det(A^{T}-\lambda I)$, and an important result is $\det(A) = \det\left(A^{T}\right)$ for $A$ is a square matrix. $$\det(A-\lambda I) = \det\left((A-\lambda I)^{T}\right)$$ $$=\det\left(A^{T}-(\lambda I)^{T}\right)$$ $$=\det\left(A^{T}-\lambda I\right)$$

8. Show that $|\det(A)|$ is the product of the absolute values of the eigenvalues of $[A]$.

Solution:

We hope to prove that $$|\det(A)| =\prod_{i=1}^{n}|\lambda_i|$$ where $\lambda_i$ is the eigenvalues of matrix $A$. By the definition we have $$|\det(A-\lambda I)| = |f(\lambda)| =|(\lambda_1-\lambda)(\lambda_2-\lambda)\cdots(\lambda_n-\lambda)| $$ Set $\lambda=0$ (since it is a variable), we have $$|\det(A)| = |\lambda_1\lambda_2\cdots\lambda_n|= \prod_{i=1}^{n}|\lambda_i|$$

9. What are the eigenvalues of the following matrix? $$\begin{bmatrix}5& 6& 17\\ 0& -19& 23\\ 0& 0& 37 \end{bmatrix}$$

Solution:

This is an upper triangular matrix, hence its eigenvalues are the diagonal elements, that is, 5, -19, and 37.

10. If $\begin{bmatrix}-4.5\\ -4\\ 1 \end{bmatrix}$ is an eigenvector of $\begin{bmatrix}8& -4& 2\\ 4& 0& 2\\ 0& -2& -4 \end{bmatrix}$, what is the eigenvalue corresponding to the eigenvector?

Solution:

From the definition we have $AX=\lambda X$, that is $$\begin{bmatrix}8& -4& 2\\ 4& 0& 2\\ 0& -2& -4 \end{bmatrix} \begin{bmatrix}-4.5\\ -4\\ 1 \end{bmatrix} =\lambda \begin{bmatrix}-4.5\\ -4\\ 1 \end{bmatrix}$$ $$\Rightarrow \begin{bmatrix}-18\\ -16\\ 4 \end{bmatrix} = \lambda \begin{bmatrix}-4.5\\ -4\\ 1 \end{bmatrix}$$ Hence $\lambda = 4$.

11. The eigenvalues of the following matrix $$\begin{bmatrix}3& 2& 9\\ 7& 5& 13\\ 6& 17& 19\end{bmatrix}$$ are given by solving the cubic equation ( ).

Solution: $$|A-\lambda I| =\det\left( \begin{bmatrix}3-\lambda& 2& 9\\ 7& 5-\lambda& 13\\ 6& 17& 19-\lambda\end{bmatrix}\right)$$ $$= (3-\lambda)\begin{vmatrix}5-\lambda & 13\\ 17 & 19-\lambda\end{vmatrix} - 2\begin{vmatrix}7 & 13\\ 6 & 19-\lambda\end{vmatrix} + 9\begin{vmatrix}7 & 5-\lambda\\ 6 & 17\end{vmatrix}$$ $$= (3-\lambda)\left((5-\lambda)(19-\lambda) - 13\times17\right) - 2\times \left(7(19 - \lambda) - 6 \times 13 \right) + 9 \left(7\times17-6(5-\lambda)\right)$$ $$=\lambda^3 - 27\lambda^2 -122\lambda -313$$

12. The eigenvalues of a $4\times4$ matrix $[A]$ are given as 2, -3, 13, and 7. What is the $|\det(A)|$?

Solution:

Since for a $n\times n$ matrix $$|\det(A)| = \prod_{i=1}^{n}|\lambda_i|$$ Hence we have $$|\det(A)| = |2\times(-3)\times13\times7| = 546$$

13. If one of the eigenvalues of $[A]_{n\times n}$ is zero, it implies ( ).

Solution:

If an eigenvalue is zero, then its determinant must be zero. Furthermore, this means it is a singular matrix (i.e. non-invertible).

14. Given that matrix $$[A] = \begin{bmatrix}8& -4& 2\\ 4& 0& 2\\ 0& -2& -3 \end{bmatrix}$$ has an eigenvalue value of 4 with the corresponding eigenvectors of $[x]=\begin{bmatrix}-4.5\\ -4\\ 1\end{bmatrix}$, then what is the value of $[A]^{5}[X]$?

Solution:

Firstly, we show that $A^{m}X=\lambda^{m}X$, where $\lambda$ is an eigenvalue of $[A]$. By Mathematical Induction, we can read off that $n=1$ is correct.\\ Then suppose that $n=m-1$ is correct, that is, $A^{m-1}X = \lambda^{m-1}X$ holds. For $n=m$, we have $$A^{m}X = AA^{m-1}X = A\lambda^{m-1}X =\lambda^{m-1}AX = \lambda^{m-1}\lambda X =\lambda^{m}X$$ as desired. From this result, we have $$A^5X=\lambda^{5}X = 4^5\begin{bmatrix}-4.5\\ -4\\ 1\end{bmatrix} = \begin{bmatrix}-4608\\ -4096\\ 1024\end{bmatrix}$$