Eigenvalues and Eigenvectors - MIT Mathematics

1 Chapter 6 Eigenvalues and Introduction to EigenvaluesLinear equationsAxDbcome from steady state problems. Eigenvalues have their greatestimportance indynamic problems. The solution ofdu=dtDAuis changing with time growing or decaying or oscillating. We can t find it by elimination. This chapter enters anew part of linear algebra , based onAxD x. All matrices in this chapter are good model comes from the powersA; A2;A3;:::of a matrix. Suppose you need thehundredth powerA100. The starting matrixAbecomes unrecognizable after a few steps,andA100is very close to :6 :6I:4 :4 : :8 :3:2 :7 :70 :45:30 :55 :650 :525:350 :475 :6000 :6000:4000 :4000 AA2A3A100A100was found by using theeigenvaluesofA, not by multiplying 100 matrices. Thoseeigenvalues (here they are1and1=2) are a new way to see into the heart of a explain Eigenvalues , we first explain Eigenvectors .

2 Almost all vectors change di-rection, when they are multiplied exceptional vectorsxare in the samedirection are the Eigenvectors . Multiply an eigenvector byA, and thevectorAxis a number times the basic equation isAxD number is an eigenvalue eigenvalue tells whether the special vectorxis stretched or shrunk or reversed or leftunchanged when it is multiplied byA. We may find D2or12or 1or1. The eigen-value could be zero! ThenAxD0xmeans that this eigenvectorxis in the the identity matrix, every vector hasAxDx. All vectors are Eigenvectors Eigenvalues lambda are D1. This is unusual to say the least. Most2by2matriceshavetwoeigenvector directions andtwoeigenvalues. We will show that I 6. Eigenvalues and EigenvectorsThis section will explain how to compute thex s and s. It can come early in the coursebecause we only need the determinant of a2by2matrix.

3 Let me use I /D0tofind the Eigenvalues for this first example, and then derive it properly in equation (3).Example 1 The matrixAhas two Eigenvalues D1and D1=2. Look at I /:AD :8 :3:2 :7 det :8 :3:2:7 D 2 32 C12D. 1/ 12 :I factored the quadratic into 1times 12, to see the two Eigenvalues D1and D12. For those numbers, the matrixA Ibecomessingular(zero determinant). Theeigenvectorsx1andx2are in the nullspaces ofA IandA I/x1D0isAx1Dx1and the first eigenvector is.:6; :4/..A 12I/x2D0isAx2D12x2and the second eigenvector ; 1/:x1D :6:4 andAx1D :8 :3:2 :7 :6:4 Dx1(AxDxmeans that 1D1)x2D 1 1 andAx2D :8 :3:2 :7 1 1 D :5 :5 (this is12x2so 2D12).Ifx1is multiplied again byA, we still getx1. Every power ofAwill , and if we multiply again we squared, the Eigenvectors stay the same. The Eigenvalues are pattern keeps going, because the Eigenvectors stay in their own directions (Figure )and never get mixed.

4 The Eigenvectors ofA100are the samex1andx2. The small number. D1Ax1Dx1D :6:4 Ax2D 2x2D :5 :5 D:5x2D 1 1 :5/2x2D :25 :25 AxD xA2xD 2x 2D:25 2D1 Figure : The Eigenvectors keep their eigenvalues12and.:5 vectors do change direction. But all other vectors are combinations of the twoeigenvectors. The first column ofAis the combinationx1C.:2/x2:Separate into Eigenvectors :8:2 Dx1C.:2/x2D :6:4 C :2 :2 :(1) Introduction to Eigenvalues285 Multiplying byAgives.:7; :3/, the first column ofA2. Do it separately forx1and.:2 courseAx1Dx1. AndAmultipliesx2by its eigenvalue12:Multiply eachxiby iA :8:2 D :7:3 isx1C12.:2/x2D :6:4 C :1 :1 :Each eigenvector is multiplied by its eigenvalue , when we multiply byA. We didn t needthese Eigenvectors to findA2. But it is the good way to do99multiplications. At every stepx1is unchanged andx2is multiplied , :A99 :8:2 is reallyx1C.

5 :2/.12/99x2D :6:4 C24verysmallvector35:This is the first column ofA100. The number we originally wrote as:6000was not left out.:2/.12/99which wouldn t show up for30decimal eigenvectorx1is a steady state that doesn t change (because 1D1/. Theeigenvectorx2is a decaying mode that virtually disappears (because 2D:5/. Thehigher the power ofA, the closer its columns approach the steady mention that this particularAis aMarkov matrix. Its entries are positive andevery column adds to1. Those facts guarantee that the largest eigenvalue is D1(as wefound). Its eigenvectorx1D.:6; :4/is thesteady state which all columns ofAkwillapproach. Section shows how Markov matrices appear in applications like projections we can spot the steady state. D1/and the nullspace. D0/.Example 2 The projection matrixPD :5 :5:5 :5 has Eigenvalues D1and Eigenvectors ; 1 ; 1/.))

6 For those vectors,Px1Dx1(steadystate) andPx2D0(nullspace). This example illustrates Markov matrices and singularmatrices and (most important) symmetric matrices. All have special s andx column ofPD :5 :5:5 :5 adds to1,so D1is an ,so D0is an , so its ; 1 ; 1/are only Eigenvalues of a projection matrix are0and1. The Eigenvectors for D0(which meansPxD0x/fill up the nullspace. The Eigenvectors for D1(which meansPxDx/fill up the column space. The nullspace is projected to zero. The column spaceprojects onto itself. The projection keeps the column space and destroys the nullspace:Project each partvD 1 1 C 22 projects ontoPvD 00 C 22 :Special properties of a matrix lead to special Eigenvalues and is a major theme of this chapter (it is captured in a table at the very end).286 Chapter 6. Eigenvalues and EigenvectorsProjections have D0and 1.))

7 Permutations have allj jD1. The next matrixR(areflection and at the same time a permutation) is also 3 The reflection matrixRD 0110 has Eigenvalues 1 and ; 1/is unchanged byR. The second eigenvector ; 1/ its signsare reversed byR. A matrix with no negative entries can still have a negative eigenvalue !The Eigenvectors forRare the same as forP, I:RD2P I 0110 D2 :5 :5:5 :5 1001 :(2)Here is the point. IfPxD xthen2 PxD2 x. The Eigenvalues are doubled whenthe matrix is doubled. Now subtractIxDx. The result 1 a matrix is shifted byI, each is shifted by1. No change in : ProjectionsPhave eigenvalues1and0. ReflectionsRhave D1and typicalxchanges direction, but not the idea: The Eigenvalues ofRandPare related exactly as the matrices are related:The Eigenvalues ofRD2P 1D Eigenvalues ofR2are 2. In this caseR2DI.

8 1 Equation for the EigenvaluesFor projections and reflections we found s andx s by geometry :PxDx;PxD0;RxD x. Now we use determinants and linear is the key calculation inthe chapter almost every application starts by solvingAxD move xto the left side. Write the equationAxD I /xD0. ThematrixA Itimes the eigenvectorxis the zero Eigenvectors make up thenullspace ofA I. When we know an eigenvalue , we find an eigenvector by I first. I /xD0has a nonzero solution,A Iis not determinant ofA Imust be is how to recognize an eigenvalue Introduction to Eigenvalues287 EigenvaluesThe number is an eigenvalue ofAif and only ifA Iis I /D0:(3)This characteristic equation I /D0involves only , notx. WhenAisnbyn,the equation has degreen. ThenAhasneigenvalues and each leads tox:For each I /xD0orAxD xto find an eigenvectorx:Example 4AD 1224 is already singular (zero determinant).

9 Find its s andx singular, D0is one of the Eigenvalues . The equationAxD0xhassolutions. They are the Eigenvectors for D0. But I /D0is the way to findall s andx s. Always subtract IfromA:Subtract from the diagonal to findA ID 1 224 :(4)Take the determinant ad bc of times4 ,the ad part is 2 5 C4. The bc part, not containing , 1 224 /.4 / .2/.2/D 2 5 :(5)Set this determinant 2 5 to zero. One solution is D0(as expected, sinceAissingular). Factoring into times 5, the other root is D5 I /D 2 5 D0yields the Eigenvalues 1D0and 2D5:Now find the Eigenvectors . I /xD0separately for 1D0and 2D5:.A 0I /xD 1224 yz D 00 yields an eigenvector yz D 2 1 for 5I /xD 422 1 yz D 00 yields an eigenvector yz D 12 for 2D5:The matricesA 0 IandA 5 Iare singular (because 0 and 5 are Eigenvalues ). ; 1 ; 2/are in the nullspaces:.A I /xD0isAxD need to emphasize:There is nothing exceptional about every othernumber, zero might be an eigenvalue and it might not.

10 IfAis singular, it is. The eigenvec-tors fill the invertible, zero is not an eigenvalue . We shiftAby a multiple ofItomake it the example, the shifted matrixA 5 Iis singular and5is the other 6. Eigenvalues and EigenvectorsSummaryTo solve the eigenvalue problem for annbynmatrix, follow these the determinant ofA I. With subtracted along the diagonal, thisdeterminant starts with nor n. It is a polynomial in of the roots of this polynomial, by solving I /D0. Thenroots aretheneigenvalues ofA. They makeA each eigenvalue , I /xD0to find an note on the Eigenvectors of2by2matrices. WhenA Iis singular, both rows aremultiples of a ; b/.The eigenvector is any multiple ; a/. The example had D0and D5: D0 Wrows ofA 0 Iin the ; 2/; eigenvector in the ; 1/ D5 Wrows ofA 5 Iin the direction. 4; 2/; eigenvector in the ; 4/:Previously we wrote that last eigenvector ; 2/.