Matrices and Linear Algebra

1 Chapter 2 Matrices and Linear BasicsDefinition anm narray of scalars from a givenfieldF. The individual values in the matrix are ^213 124 B=^1234 Thesizeof the array is written asm n,wherem ncAnumber of rows number of columnsNotationA= amn A rowstAAccolumnsA:= uppercase denotes a matrixa:= lower case denotes an entry of a matrixa matrices3334 CHAPTER 2. Matrices AND Linear Algebra (1) Ifm=n, the matrix is (1a) A matrixAis said to bediagonalifaij=0iW=j.(1b) A diagonal matrixAmay be denoted by diag(d1,d2,.. ,dn)whereaii=diaij=0jW= diagonal matrix diag(1,1,.. ,1) is called theidentitymatrixand is usually denoted byIn= or simplyI,whennis assumed to be known. 0 = diag(0,.. ,0)is called thezero matrix .(1c) A square matrixLis said to belower triangulariffij=0i<j.(1d) A square matrixUis said to be upper triangular ifuij=0i>j.(1e) A square matrixAis calledsymmetricifaij=aji.(1f) A square matrixAis calledHermitianifaij= aji( z:= complex conjugate ofz).

2 (1g)Eijhas a 1 in the (i, j) position and zeros in all other positions.(2) A rectangular matrixAis callednonnegativeifaij 0alli, is calledpositiveifaij>0alli, of these Matrices has some special properties, which we will studyduring this BASICS35 Definition set of allm nmatrices is denoted byMm,n(F),whereFis the underlyingfield (usuallyRorC). In the case wherem=nwe writeMn(F) to denote the Matrices of sizen ,nis a vector space with basis given byEij,1 i m,1 j , Addition, MultiplicationDefinition matricesAandBare equal if and only if they havethesamesizeandaij=bijalli, any matrix and Fthen the scalar multipli-cationB= Ais defined bybij= aijalli, Matrices of the same size then thesumAandBis defined byC=A+B,wherecij=aij+bijalli, jWe can also compute thedifferenceD=A Bby summingAand ( 1)BD=A B=A+( 1) addition inherits many properties from , B, C Mm,n(F)and , F,then(1)A+B=B+Acommutivity(2)A+(B+C)=(A +B)+Cassociativity(3) (A+B)= A+ Bdistributivity of a scalar(4) IfB=0(a matrix of all zeros) thenA+B=A+0=A(4)( + )A= A+ A36 CHAPTER 2.

3 Matrices AND Linear Algebra (5) ( A)= A(6)0A=0(7) 0= Rn,x=( )y=( ).Then the scalaror dotproduct ofxandyis given by x, yX=n3i= (i) Alternate notation for the scalar product: x, yX=x y.(ii) The dot product is defined only for vectors of the same (1,0,3, 1) andy=(0,2, 1,2) then x, yX=1(0) + 0(2) + 3( 1) 1(2) = nandBisn (A) denote the vectorwith entries given by theithrow ofA,andletcj(B) denote the vector withentries given by thejthrow ofB. The productC=ABis them pmatrixdefined bycij= ri(A),cj(B)Xwhereri(A) is the vector inRnconsisting of theithrow ofAand similarlycj(B) is the vector formed from thejthcolumn ofB. Other notation forC=ABcij=n k=1aikbkj1 i m1 j }101321]andB= 2130 11 .ThenAB=}1211 4]. BASICS37 Properties of matrix multiplication(1) IfABexists, does it happen thatBAexists andAB=BA?Theanswer is usually no. FirstABandBAexist if and only ifA Mm,n(F)andB Mn,m(F). Even if this is so the sizes ofABandBAare different (ABism mandBAisn n) unlessm= even ifm=nwe may haveABW= They may be different sizes and if they are the same size ( )theentriesmaybedifferentA=[1,2]B=} 11]AB=[1]BA=} 1 212]A=}1234]B=} 1101]AB=} 13 37]BA=}2234](2) IfAis square we defineA1=A, A2=AA, A3=A2A=AAAAn=An 1A=A A(nfactors).

4 (3)I= diag(1,.. ,1). IfA Mm,n(F)thenAIn=AandImA= ( matrix Multiplication Rules).AssumeA, B,andCare Matrices for which all products below make sense. Then(1)A(BC)=(AB)C(2)A(B C)=AB ACand(A B)C=AC BC(3)AI=AandIA=A(4)c(AB)=(cA)B(5)A0=0and 0B=038 CHAPTER 2. Matrices AND Linear Algebra (6) ForAsquareArAs=AsArfor all integersr, s :IfACandBCare equal, it does not follow thatA=B. See use an alternate notation for matrix entries. For anymatrixBdenote the (i, j)-entry by (B) Mm,n(F).(i) Define thetransposeofA, denoted byAT,tobethen mmatrixwith entries(AT)ij=aji.(ii) Define theadjointofA, denoted byA ,tobethen mmatrix withentries(A )ij= ajicomplex conjugateExample }123541]AT= 152431 In The rows ofAbecome the columns ofAT, taken in the sameorder. The following results are easy to (Laws of transposes).(1)(AT)T=Aand(A ) =A(2)(A B)T=AT BT(and for )(3)(cA)T=cAT(cA) = cA (4)(AB)T=BTAT(5) IfAis symmetricA= BASICS39(6) IfAis HermitianA=A.

5 More facts about (1) We know (AT)ij= ((AT)T)ij= (AT)T=A.(2) (A B)T=aji (A B)T=AT (1)Ais symmetric if and only ifATis symmetric.(1) Ais Hermitian if and only ifA is Hermitian.(2) IfAis symmetric, thenA2is also symmetric.(3) IfAis symmetric, thenAnis also symmetric for matrix is called skew-symmetric ifAT= matrixA= 012 10 3 23 0 is (1) IfAis skew symmetric, thenAis a square matrixandaii=0,i=1,.. ,n.(2) For any matrixA Mn(F)A ATis skew-symmetric whileA+ATis symmetric.(3) Every matrixA Mn(F)can be uniquelywritten as the sum of askew-symmetric and symmetric (1) IfA Mm,n(F), thenAT Mn,m(F). So, ifAT= Awemust havem= aiifori=1,.. , 2. Matrices AND Linear Algebra (2) Since (A AT)T=AT A= (A AT), it follows thatA ATisskew-symmetric.(3) LetA=B+Cbe a second such decomposition. Subtraction gives12(A+AT) B=C 12(A AT).The left matrix is symmetric while the right matrix is both are the zero (A+AT)+12(A AT). 110ois skew-symmetric. LetB=}12 14]BT=}1 124]B BT=}03 30]B+BT=}2118].

6 ThenB=12(B BT)+12(B+BT).An important observation about matrix multiplication is related to ideasfrom vector spaces. Indeed, two very important vector spaces are associatedwith Mm,n(C).(i)Denote bycj(A):=jthcolumn ofAcj(A) Cm. We call the subspace ofCmspanned by the columns ofAthecolumn (A),..,cn(A) denoting the columns BASICS41the column space isS(c1(A),..,cn(A)).(ii) Similarly, we call the subspace ofCnspanned by the rows (A),..,rm(A) denoting the rows ofAthe row spaceis thereforeS(r1(A),..,rm(A)).Letx Cn,whichweviewasthen 1matrixx=[ ] defined andAx=n3j=1xjcj(A).That is to say,Ax S(c1(A),.. ,cn(A)) =columnspace Mn(F). The matrixAis said to beinvertibleif there is a matrixB Mn(F) such thatAB=BA= this caseBis called theinverseofA, and the notation for the inverse isA (i) LetA=}13 12]ThenA 1=15}2 311].(ii) Forn=3wehaveA= 12 1 13 1 23 1 A 1= 01 1 13 2 37 5 A square matrix need not have an inverse, as will be discussed in thenext section.

7 As examples, the two Matrices below do not have inversesA=}1 2 12]B= 101021122 42 CHAPTER 2. Matrices AND Linear Linear SystemsThe solutions of Linear systems is likely the single largest application of ma-trix theory. Indeed, most reasonable problems of the sciences and economicsthat have the need to solve problems of several variable almost without ex-ception are reduced to component parts where one of them is the solutionof a Linear system. Of course the entire solution process may have the linearsystem solver as a relatively small component, but an essential one. Eventhe solution of nonlinear problems, especially, employ Linear systems to greatand crucial be precise, we suppose that the coefficientsaij,1 i mand 1 j nand the databj,1 j thelinear systemfor thenunknownsx1,..,xnto bea11x1+a12x2+ +a1nxn=b1a21x1+a22x2+ +a2nxn=b2( )am1x1+am2x2+ +amnxn=bmThesolution setis defined to be the subset ofRnof vectors (x1,..,xn)thatsatisfy each of themequations of the system.

8 The question of how to solvea Linear system includes a vast literature of theoretical and computationmethods. Certain systems form the model of what to do. In the systemsbelow we note that thefirst one has three highly coupled (interrelated) 2x2+4x3=7x1 6x2 2x3=0 x1+3x2+6x3= 2 The second system is more tractable because there appears even to theuntrained eye a clear and direct method of 2x2 x3=7x2 2x3=12x3= 2 Indeed,wecanseerightoffthatx3= this value into thesecond equation we obtainx2=1 2= bothx2andx3into thefirst equation, we obtain 2x1 2( 1) ( 1) = 7,givesx1= Linear SYSTEMS43solution set is the vector (2, 1, 1).The virtue of the second system isthat the unknowns can be determined one-by-one, back substituting thosealready found into the next equation until all unknowns are determined. Soif we can convert the given system of thefirst kind to one of the second kind,we can determine the procedure for solving Linear systems is therefore the applications ofoperations to effect the gradual elimination of unknowns from the equationsuntil a new system results that can be solved by direct means.

9 The oper-ations allowed in this process must have precisely one important property:They must not change the solution set by either adding to it or subtractingfrom are exactly three such operations needed to reduce any setof Linear equations so that it can be solved directly.(E1) Interchange two equations.(E2) Multiply any equation by a nonzero constant.(E3) Add a multiple of one equation to can be summarized in the following theoremTheorem the Linear system (*). The set of equation opera-tions E1, E2, and E3 on the equations of (*) does not alter the solution setof the system (*).We leave this result to the exercises. Our main intent is to convert theseoperations into corresponding operations for Matrices . Before we do thiswe clarify which Linear systems can have a soltution. First, the system canbe converted to matrix form by settingAequal to them nmatrix ofcoefficients,bequal to them 1 vector of data, andxequal to then 1vector of unknowns.

10 Then the system (*) can be written asAx=bIn this way we see that withci(A)denotingtheithcolumn ofA,the systemis expressible asx1c1(A)+ +xncn(A)=bFrom this equation it is clear that the system has a solution if and only ifthe vectorbis inS(c1(A), ,cn(A)). This is summarized in the 2. Matrices AND Linear ALGEBRAT heorem condition thatAx=bhas asolution is thatb S(c1(A)..cn(A)).In the general matrix productC=AB, we note that the column space ofC column space we regard the matrixAas a function acting upon vectors in one vector space with range in anothervector space. This is entirely similar to the domain-range idea of {Ax|x Rn(orCn)}.It follows directly from our discussion above that the range ofAequalsS(c1(A),.. ,cn(A)).Row operations:To solveAx=bwe use a process calledGaussianelimination, which is based on row 1:Interchange two rows. (Notation:Ri Rj)Type 2:Multiply a row by a nonzero constant. (Notation:cRi Ri)Type 3:Add a multiple of one row to another row.