Eigenvalues and Eigenvectors

← Back to MATH1012

Introduction

• If $A$ is an $n\times n$ matrix, then a non-zero vector $v$ is called an eigenvector of $A$ with eigenvalue $\lambda$, if $Av=\lambda v$
  • i.e. multiplying by $A$ results in a scalar multiplication of $v$
• Eigenvalues can be found/proven through row-reduction
  • A system has non-zero solutions when $\det (A-\lambda I)=0$
    • $Av=\lambda v$ → $(A-\lambda I)v=0$
    • $I$ is identity matrix
  • Solving the determinant for $\lambda$ can give us all eigenvalues

Systematic Approach to Finding Eigenvalues

• If $Av=\lambda v$ then: $(A-\lambda I)v=0$
  • $I$ is the identity matrix
  • When does $\det (A-\lambda I)=0$ have non-zero solutions?
    • These are the only eigenvalues that will work for $A$
    • Remember, $\lambda$ is the eigenvalue
• We will solve this equation and then find all values for $\lambda$ rather than manually checking if each value of $\lambda$ works
• Remember, $v$ is an eigenvector
  • i.e. a solution to the equation $Av=\lambda v$ for a given $A$ and $\lambda$

You do not need to know $v$, this process takes a vector $A$, finds all possible eigenvalues $\lambda$ it could produce and then allows you to solve for any possible $v$. Once you have $\lambda$, create a homogenous system of equations (by moving the variables to the right-hand side) and solve for $v$ (do this for each potential $\lambda$). Worked solution in the lecture recording.

Using Eigenvalues to Solve High Powers

• Eigenvalues makes high power linear algebra analysis easy
• Suppose $Av=\lambda v$:
  • $AAv=\lambda \lambda v={\lambda }^{2}v$
  • $A^{3}v={\lambda }^{3}v$
• Typically, you’ll express a vector in the form of two eigenvectors
  • $A^{500}[\genfrac{}{}{0ex}{}{1}{3}]$ → $[\genfrac{}{}{0ex}{}{1}{3}]=\alpha [\genfrac{}{}{0ex}{}{1}{1}]+\beta [\genfrac{}{}{0ex}{}{1}{-1}]$ → $[\genfrac{}{}{0ex}{}{1}{3}]=2[\genfrac{}{}{0ex}{}{1}{1}]-1[\genfrac{}{}{0ex}{}{1}{-1}]$
    • The eigenvectors are given from solution to lecture example
  • $A^{500}[\genfrac{}{}{0ex}{}{1}{3}]=[\genfrac{}{}{0ex}{}{1}{3}]\to A^{500}[\genfrac{}{}{0ex}{}{1}{3}]=2A^{500}[\genfrac{}{}{0ex}{}{1}{1}]-A^{500}[\genfrac{}{}{0ex}{}{1}{-1}]$
  • Full example in lecture recording

Characteristic Polynomial

• If $A$ is an $n\times n$ matrix, then $\det (A-\lambda I)$ is a polynomial of degree $n$ in $\lambda$, called the characteristic polynomial of $A$
• $\det (A-\lambda I)=0$ is also known as the characteristic equation
  • Its solutions are the eigenvalues of $A$
  • There are always $n$ complex solutions
    • But they may not be real and may be repeated
• See: The Determinant

Eigenspaces

• For a given eigenvalue $\lambda$, the eigenspace is the set of all eigenvectors corresponding to $\lambda$, together with the zero vector ($v$ cannot be $\varnothing$)
  • Represented by $E_{\lambda }$
• $E_{\lambda }=\{v\in {\mathbb{R}}^n|Av=\lambda v\}$ which also equals $\ker (A-\lambda I)$
  • When describing an eigenspace, you should find a basis for it

Multiplicity

• There can be repeated solutions for a characteristic equation
  • e.g. $(\lambda -\alpha {)}^n=0$ has $n$ solutions with the same $\lambda =\alpha$
• The algebraic multiplicity of a particular eigenvalue $k$ is $m$, if: $\det (A-\lambda I)=...(k-\lambda {)}^m...$
  Remember, this is referring to an eigenvalue, not eigenspace!
• The geometric multiplicity of a particular eigenvalue $k$ is the dimension of the eigenspace $E_k$
  • i.e. how many vectors is in the basis of the eigenspace
  • It will always be less than or equal to the algebraic multiplicity $(m)$

Algebraic vs Geometric Multiplicity

$1\le \text{geometric multiplicity}\le \text{algebraic multiplicity}$

⭐ The algebra tells us the maximum possible multiplicity
⭐ The geometry tells us the actual multiplicity

Properties of Eigenvalues

• The sum of all eigenvalues for a matrix is equal to its trace
  • The trace of a matrix is the sum of the diagonals
  • For upper-triangular matrices, the diagonal values are eigenvalues
• The product of all eigenvalues is equal to the determinant

$A$ and $A^T$ have the same eigenvalues
$kA$ has eigenvalues $k\lambda$
$A^k$ has eigenvalues ${\lambda }^k$

• Cayley-Hamilton Theorem: $A$ satisfies its own characteristic equation
  • i.e. if the polynomial is $(\lambda -1)(\lambda -2)$, then $(A-I)(A-2I)=0$
• If $A$ is invertible:
  • $\lambda \ne 0$
    • Invertible matrices have trivial null space (only 0 maps to 0)
    • If $\lambda =0$, then $A\mathbf{v}=\lambda \mathbf{v}=0:\mathbf{v}\in \text{null space}$ of $A$
      $\Rightarrow$ $A$ is not invertible
  • $A^{-1}\mathbf{v}=\frac{1}{\lambda }\mathbf{v}$
    • Start with $A\mathbf{v}=\lambda \mathbf{v}$
      $\to A^{-1}A\mathbf{v}=A^{-1}\lambda \mathbf{v}$
      $\to \mathbf{v}=A^{-1}\lambda \mathbf{v}$
      $\to \frac{1}{\lambda }\mathbf{v}=A^{-1}\mathbf{v}$

Diagonalisation

• Diagonal matrices are square matrices where all entries outside the main diagonal (top left to bottom right) are zero
  • Diagonal matrices are convenient for calculations
  • We want to find a basis $B$ that will put a matrix, of a linear transformation, into a diagonal form
• Easy to find high powers of diagonal matrices as you just raise each diagonal entry to that power

Diagonalisability Test

• A matrix $A$ is diagonalisable if and only if:
  • It has all real eigenvalues
  • Every eigenvalue has the maximum possible geometric multiplicity
    • i.e. geometric = algebraic
  • Special Case: the matrix has $n$ distinct eigenvalues
    • i.e. each eigenvalue automatically has max multiplicity

A basis vector for a particular eigenvalue is always linearly independent of another eigenvectors’ basis

Special Case

• A matrix is symmetric if $A=A^T$
• If $A$ is symmetric:
  • Each eigenvalue of $A$ is real
  • Each eigenvalue has maximum multiplicity
  • Eigenvectors from distinct eigenspaces are orthogonal
• Thus, symmetric matrices are diagonalisable

Creating a Diagonal Matrix

• For a given matrix $A$, we need to find $P$, $D$ and $P^{-1}$ to utilise the diagonal matrix
  • $D$ is the new diagonal matrix
    • The diagonals are made up of $A$‘s eigenvalues
    • $D=P^{-1}AP$
  • $P$ is the eigenvector basis
    • $P$ is a matrix where each column is the eigenvectors of $A$
    • $P=[v_1{v}_2...v_n]$
• Lets say you have the vector $\mathbf{x}$ in the original coordinate system
  • $\mathbf{y}=P^{-1}\mathbf{x}$ where $\mathbf{y}$ is in the eigenvector coordinate system
  • We need $P$ and $P^{-1}$ to move between the original and eigenvector coordinate system
• This means we can simplify: $A^n\vec{v}\to P{D}^n{P}^{-1}\vec{v}$
  • $D$ is a diagonal matrix, whose power can be easily solved
  • This helps us solve what would otherwise be a difficult problem
• More info: Slides