Math 225 Linear Algebra II Lecture Notes - ualberta.ca

1 Math 225 Linear Algebra II Lecture NotesJohn C. BowmanUniversity of AlbertaEdmonton, CanadaMay 6, 2022 2010 John C. BowmanALL RIGHTS RESERVEDR eproduction of these Lecture Notes in any form, in whole or in part, is permitted only fornonprofit, educational Vectors42 Linear Equations63 Matrix Algebra84 Determinants115 Eigenvalues and Eigenvectors136 Linear Transformations167 Dimension178 Similarity and Diagonalizability189 Complex Numbers2310 Projection Theorem2811 Gram-Schmidt Orthonormalization2912 QRFactorization3113 Least Squares Approximation3214 Orthogonal (Unitary) Diagonalizability3415 Systems of Differential Equations4016 Quadratic Forms4317 Vector Spaces:4618 Inner Product Spaces4819 General Linear transformations5120 Singular Value Decomposition5321 The Pseudoinverse57 Index581 Vectors VectorsinRn:u= ,v= ,0= . Parallelogram law:u+v= u1+ +vn.

2 Multiplication by scalarc R:cv= . Dot(inner) product:u v=u1v1+ +unvn. Length(norm):|v|= v v=qv21+ +v2n. Unit vector:1|v|v. Distancebetween (endpoints of)uandv(positioned at0):d(u,v) =|u v|. Law of cosines:|u v|2=|u|2+|v|2 2|u||v|cos . Angle betweenuandv:|u||v|cos =12(|u|2+|v|2 |u v|2) =12(|u|2+|v|2 [u v] [u v])=12(|u|2+|v|2 [u u 2u v+v v])=u Orthogonal vectorsuandv:u v=0. Pythagoras theoremfor orthogonal vectorsuandv:|u+v|2=|u|2+|v|2. Cauchy-Schwarz inequality(|cos | 1):|u v| |u||v|. Triangle inequality:|u+v|=p(u+v) (u+v) =p|u|2+2u v+|v|2 p|u|2+2|u v|+|v|2 p|u|2+2|u||v|+|v|2=p(|u|+|v|)2=|u|+|v|. Componentofvin directionu:v u|u|=|v|cos . Equation of line:Ax+By=C. Parametric equation of linethroughpandq:v= (1 t)p+tq. Equation of planewith unit normal (A,B,C) a distanceDfrom origin:Ax+By+Cz=D. Equation of planewith normal (A,B,C) through (x0,y0,z0):A(x x0) +B(y y0) +C(z z0) = 0.

3 Parametric equation of planespanned byuandwthroughv0:v=v0+tu+ Linear Equations Linear equation:a1x1+a2x2+ +anxn=b. Homogeneous Linear equation:a1x1+a2x2+ +anxn= 0. System of Linear equations:a11x1+a12x2+ +a1nxn=b1a21x1+a22x2+ +a2nxn= +am2x2+ +amnxn=bm. Systems of Linear equations may have 0, 1, or an infinite number of solutions [Anton& Busby, p. 41]. Matrixformulation: a11a12 a1na21a22 amn |{z}n mcoefficient matrix = . Augmented matrix: a11a12 a1nb1a21a22 amnbm.. Elementary row operations: Multiply a row by a nonzero constant. Interchange two rows. Add a multiple of one row to : Elementary row operations do not change the solution! Row echelon form(Gaussian elimination): 1 0 01 ..0 00 1 . Reduced row echelon form(Gauss Jordan elimination): 1 0 00 01 ..0 00 1 . For example:2x1+x2= 5,7x1+ 4x2= 17. Adiagonal matrixis a square matrix whose nonzero values appear only as entriesaiialong the diagonal.

4 For example, the following matrix is diagonal: 1 0 0 00 1 0 00 0 0 00 0 0 1 . Anupper triangular matrixhas zero entries everywhere below the diagonal (aij= 0fori > j). Alower triangular matrixhas zero entries everywhere above the diagonal (aij= 0fori < j).Problem : Show that the product of two upper triangular matrices of the samesize is an upper triangle : IfLis a lower triangular matrix, andUis an upper triangular matrix,show thatLUis a lower triangular matrix andULis an upper triangular Matrix Algebra Elementaijappears in rowiand columnjof them nmatrixA= [aij]m n. Addition of matricesof the same size:[aij]m n+ [bij]m n= [aij+bij]m n. Multiplication of a matrix by a scalarc R:c[aij]m n= [caij]m n. Multiplication of matrices:[aij]m n[bjk]n ="nXj=1aijbjk#m . Matrix multiplication islinear:A( B+ C) = AB+ ACfor all scalars and . Two matricesAandBare said tocommuteifAB= : Give examples of 2 2 matrices that commute and ones that don t.

5 Transposeof a matrix:[aij] m n= [aji]n m. Asymmetric matrixAis equal to its : Does a matrix necessarily compute with its transpose? Prove orprovide a : LetAbe anm nmatrix andBbe ann matrix. Prove that(AB) =B A ..8 Problem : Show that the dot product can be expressed as a matrix multiplication:u v=u v. Also, sinceu v=v u, we see equivalently thatu v=v u. Traceof a square matrix:Tr [aij]n n=nXi= : LetAandBbe square matrices of the same size. Prove thatTr(AB) = Tr(BA).. Identitymatrix:In= [ ij]n n. InverseA 1of aninvertiblematrixA:AA 1=A 1A= : IfAx=bis a Linear system ofnequations, and the coefficientmatrixAis invertible, prove that the system has the unique solutionx=A : Prove that(AB) 1=B 1A 1. Inverse of a 2 2 matrix: a bc d d b c a = (ad bc) 1 00 1 a bc d 1=1ad bc d b c a exists if and only ifad bc = 0. Determinant of a 2 2 matrix:det a bc d = a bc d =ad Anelementary matrixis obtained on applying asingleelementary row operationto the identity matrix.

6 Two matrices arerow equivalentif one can be obtained from the other by elementaryrow operations. An elementary matrix is row equivalent to the identity matrix. Asubspaceis closed under scalar multiplication and addition. span{v1,v2,..,vn}is the subspace formed by the set of Linear combinationsc1v1+c2v2+ +cnvngenerated by all possible scalar multipliersc1,c2, .. ,cn. The subspaces inR3are the origin, lines, planes, and all ofR3. The solution space of the homogeneous Linear systemAx=0withnunknowns isalways a subspace ofRn. A set of vectors{v1,v2,..,vn}inRnarelinearly independentif the equationc1v1+c2v2+ +cnvn=0has only the trivial solutionc1=c2= =cn= 0. If this were not the case, wesay that the set is linearly dependent; forn 2, this means we can express at leastone of the vectors in the set as a Linear combination of the others. The following are equivalent for ann nmatrixA:(a)Ais invertible;(b) The reduced row echelon form ofAisIn;(c)Acan be expressed as a product of elementary matrices;(d)Ais row equivalent toIn;(e)Ax=0has only the trivial solution;(f)Ax=bhas exactly one solution for allb Rn;(g) The rows ofAare linearly independent;(h) The columns ofAare linearly independent;(i) detA = 0;(j) 0 is not an eigenvalue The elementary row operations that reduce an invertible matrixAto the identitymatrix can be applied directly to the identity matrix to yieldA : IfBA=Ithen the system of equationsAx=0has the unique solutionx=Ix=BAx=B(Ax) =B0=0.

7 HenceAis invertible andB=A Determinants Determinant of a 2 2 matrix: a bc d =ad bc. Determinant of a 3 3 matrix: a b cd e fg h i =a e fh i b d fg i +c d eg h . Determinantof a square matrixA= [aij]n ncan be formally defined as:detA=Xpermutations(j1,j2,..,jn) of (1,2,..,n)sgn(j1,j2,..,jn) Properties of Determinants: Multiplying a single row or column by a scalarcscales the determinant byc. Exchanging two rows or columns swaps the sign of the determinant. Adding a multiple of one row (column) to another row (column) leaves thedeterminant invariant. Given ann nmatrixA= [aij], theminorMijof the elementaijis the determinantof the submatrix obtained by deleting theith row andjth column fromA. Thesigned minor ( 1)i+jMijis called thecofactorof the elementaij. Determinants can be evaluated by cofactor expansion (signed minors), either alongsome rowior some columnj:det[aij]n n=nXk=1aikMik=nXk= In general, evaluating a determinant by cofactor expansion is inefficient.

8 However,cofactor expansion allows us to see easily that the determinant of a triangularmatrix is simply the product of the diagonal elements!Problem : Prove that detA = : LetAandBbe square matrices of the same size. Prove thatdet(AB) = det(A) det(B)..Problem : Prove for an invertible matrixAthat det(A 1) =1detA. Determinants can be used to solve Linear systems of equations like a bc d xy = uv . Cramer s rule:x= u bv d a bc d , y= a uc v a bc d . For matrices larger than 3 3, row reduction is more efficient than Cramer s rule. An upper (or lower)triangular systemof equations can be solved directly by backsubstitution:2x1+x2= 5,4x2= 17. The transpose of the matrix of cofactors, adjA= [( 1)i+jMji], is called theadjugate(formerly sometimes known as theadjoint) ofA. The inverseA 1of a matrixAmay be expressed compactly in terms of its adjugatematrix:A 1= , this is an inefficient way to compute the inverse of matrices larger than3 3.

9 A better way to compute the inverse of a matrix is to row reduce [A|I] to[I|A 1].12 Thecross productof two vectorsu= (ux,uy,uz) andv= (vx,vy,vz) inR3can beexpressed as the determinantu v= i j kuxuyuzvxvyvz = (uyvz uzvy)i+ (uzvx uxvz)j+ (uxvy uyvx)k. The magnitude|u v|=|u||v|sin (where is the angle between the vectorsuandv) represents the area of the parallelogram formed Eigenvalues and Eigenvectors Afixed pointxof a matrixAsatisfiesAx=x. The zero vector is a fixed point of any matrix. Aneigenvectorxof a matrixAis anonzerovectorxthat satisfiesAx= xfora particular value of , known as aneigenvalue(characteristic value). An eigenvectorxis a nontrivial solution to (A I)x=Ax x= 0. This requiresthat (A I) be noninvertible; that is, must be a root of thecharacteristicpolynomialdet( I A): it must satisfy thecharacteristic equationdet( I A) = 0. If [aij]n nis a triangular matrix, each of thendiagonal entriesaiiare eigenvaluesofA:det( I A) = ( a11)( a22).

10 ( ann). IfAhas eigenvalues 1, 2,.., nthen det(A I) = ( 1 )( 2 )..( n ).On setting = 0, we see that detAis the product 1 nof the eigenvaluesofA. Given ann nmatrixA, the terms in det( I A) that involve nand n 1mustcome from the main diagonal:( a11)( a22)..( ann) = n (a11+a22+..+ann) n 1+..Butdet( I A) = ( 1)( 2)..( n) = n ( 1+ 2+..+ n) n 1+..+( 1)n 1 , TrAis the sum 1+ 2+..+ nof the eigenvalues ofAand the characteristicpolynomial ofAalways has the form n TrA n 1+..+ ( 1) : Show that the eigenvalues and corresponding eigenvectors of thematrixA= 1 23 2 are 1, with eigenvector [1, 1], and 4, with eigenvector [2,3]. Note that the trace(3) equals the sum of the eigenvalues and the determinant ( 4) equals the productof eigenvalues. (Any nonzero multiple of the given eigenvectors are also acceptableeigenvectors.)Problem : ComputeAvwhereAis the matrix in Problem andv= [7,3] ,both directly and by expressingvas a Linear combination of the eigenvectors ofA:[7,3] = 3[1, 1] + 2[2,3].