Introduction to Linear Algebra, 5th Edition

1 22 Chapter 1. Introduction to matrices '&$%1A= 1 23 45 6 is a3by2matrix :m= 3rows andn= 1 23 45 6 x1x2 is acombination of the columnsAx=x1 135 +x2 246 .3 The3components ofAxare dot products of the3rows ofAwith the vectorx:Row at a time 1 23 45 6 78 = 1 7 + 2 83 7 + 4 85 7 + 6 8 = 235383 .4 equations in matrix formAx=b: 2 53 7 x1x2 = b1b2 replaces2x1+ 5x2=b13x1+ 7x2= solution toAx=bcan be written asx=A 1b. But some matrices don t allowA section starts with three vectorsu,v,w. I will combine them vectorsu= 1 10 v= 01 1 w= 001 .Their Linear combinations in three-dimensional space arex1u+x2v+x3w:Combinationof the vectorsx1 1 10 +x2 01 1 +x3 001 = x1x2 x1x3 x2 .(1)Now something important:Rewrite that combination using a matrix. The vectorsu,v,wgo into the columns of the matrixA. That matrix multiplies the vector(x1, x2, x3) :Matrix times vectorCombination of columnsAx= 1 0 0 1 1 00 1 1 x1x2x3 = x1x2 x1x3 x2.

2 (2)The numbersx1, x2, x3are the components of a vectorx. The matrixAtimes the vectorxis thesameas the combinationx1u+x2v+x3wof the three columns in equation (1).This is more than a definition ofAx, because the rewriting brings a crucial changein viewpoint. At first, the numbersx1, x2, x3were multiplying the vectors. Now Matrices23matrix is multiplying those matrixAacts on the vectorx. The outputAxis acombinationbof the columns see that action, I will writeb1, b2, b3for the components ofAx:Ax= 1 0 0 1 1 00 1 1 x1x2x3 = x1x2 x1x3 x2 = b1b2b3 =b.(3)The input isxand the output isb=Ax. ThisAis a difference matrix becausebcontains differences of the input vectorx. The top difference isx1 x0=x1 is an example to show differences ofx= (1,4,9): squares inx, odd numbers 149 =squaresAx= 1 04 19 4 = 135 =b.

3 (4)That pattern would continue for a4by4difference matrix. The next square would bex4= 16. The next difference would bex4 x3= 16 9 = 7(the next odd number).The matrix finds all the differences1,3,5,7at Note: Multiplication a row at a may already have learned aboutmultiplyingAx, a matrix times a vector. Probably it was explained differently, using therows instead of the columns. The usual way takes the dot product of each row withx:Axis alsodot productswith rowsAx= 1 0 0 1 1 00 1 1 x1x2x3 = (1,0,0) (x1, x2, x3)( 1,1,0) (x1, x2, x3)(0, 1,1) (x1, x2, x3) .(5)Those dot products are the samex1andx2 x1andx3 x2that we wrote in equation (3).The new way is to work withAxa column at a time. Linear combinations are the key tolinear algebra , and the outputAxis a Linear combination of numbers, you can multiplyAxby rows.

4 With letters, columns are the good 2 will repeat these rules of matrix multiplication,and explain the EquationsOne more change in viewpoint is crucial. Up to now, the numbersx1, x2, x3were right hand sidebwas not known. We found that vector of differences by we think ofbas known and we look question: Compute the Linear combinationx1u+x2v+x3wto question: Which combination ofu,v,wproduces a particular vectorb?This is theinverse problem to find the inputxthat gives the desired outputb= have seen this before, as a system of Linear equations forx1, x2, x3. The right handsides of the equations areb1, b2, b3. I will now solve that systemAx=bto findx1, x2, x3:24 Chapter 1. Introduction to VectorsEquationsAx=bx1=b1 x1+x2=b2 x2+x3=b3 Solutionx=A 1bx1=b1x2=b1+b2x3=b1+b2+b3.(6)Let me admit right away most Linear systems are not so easy tosolve.

5 In this example,the first equation decidedx1=b1. Then the second equation producedx2=b1+ equations can be solved in order(top to bottom)becauseAis a triangular at two specific choices0,0,0and1,3,5of the right sidesb1, b2, b3:b= 000 givesx= 000 b= 135 givesx= 11 + 31 + 3 + 5 = 149 .The first solution (all zeros) is more important than it looks. In words:If the output isb=0,then the input must bex=0. That statement is true for this matrixA. It is not truefor all matrices . Our second example will show (for a different matrixC) how we can haveCx=0whenC6= 0andx6= matrixAis invertible . Frombwe can recoverx. We writexasA Inverse MatrixLet me repeat the solutionxin equation (6). A sum matrix will appear!Ax=bis solved by x1x2x3 = b1b1+b2b1+b2+b3 = 1 0 01 1 01 1 1 b1b2b3 .(7)If the differences of thex s are theb s, the sums of theb s are thex s.

6 That was true forthe odd numbersb= (1,3,5)and the squaresx= (1,4,9). It is true for all sum matrix in equation(7)is the inverseA 1of the difference : The differences ofx= (1,2,3)areb= (1,1,1). Sob=Axandx=A 1b:Ax= 1 0 0 1 1 00 1 1 123 = 111 A 1b= 1 0 01 1 01 1 1 111 = 123 Equation (7) for the solution vectorx= (x1, x2, x3)tells us two important everybthere is one solution toAx= matrixA 1producesx=A next chapters ask about other equationsAx=b. Is there a solution? How to find it?Note on calculus. Let me connect these special matrices to calculus. The vectorxchangesto a functionx(t). The differencesAxbecome thederivativedx/dt=b(t). In theinverse direction, the sumsA 1bbecome theintegralofb(t).Sums of differences are likeintegrals of Matrices25 The Fundamental Theorem of Calculus says :integration is the inverse of 1bdxdt=bandx(t) =Zt0b dt.

7 (8)The differences of squares0,1,4,9are odd numbers1,3,5. The derivative ofx(t) =t2is2t. A perfect analogy would have produced the even numbersb= 2,4,6at timest= 1,2,3. But differences are not the same as derivatives, and our matrixAproduces not2tbut2t 1:Backwardx(t) x(t 1) =t2 (t 1)2=t2 (t2 2t+ 1) = 2t 1.(9)The Problem Set will follow up to show that forward differences produce2t+ best choice (not always seen in calculus courses) is acentered differencethat usesx(t+ 1) x(t 1). Divide that xby the distance tfromt 1tot+ 1, which is2:Centered difference ofx(t) =t2(t+ 1)2 (t 1)22= 2texactly.(10)Difference matrices are great. Centered is the best. Our second example isnot DifferencesThis example keeps the same columnsuandvbut changeswto a new vectorw :Second exampleu= 1 10 v= 01 1 w = 101 .Now the Linear combinations ofu,v,w lead to acyclic difference matrixC:CyclicCx= 1 0 1 1 1 00 1 1 x1x2x3 = x1 x3x2 x1x3 x2 =b.

8 (11)This matrixCis not triangular. It is not so simple to solve forxwhen we are it is impossible to findthesolution toCx=b, because the three equations eitherhaveinfinitely many solutions(sometimes) or elseno solution(usually) :Cx=0 Infinitelymanyx x1 x3x2 x1x3 x2 = 000 is solved by all vectors x1x2x3 = ccc .(12)Every constant vector likex= (3,3,3)has zero differences when we go cyclically. Theundetermined constantcis exactly like the+Cthat we add to integrals. The cyclic dif-ferences cycle around tox1 x3in the first component, instead of starting fromx0= 1. Introduction to VectorsThe more likely possibility forCx=bisno solutionxat all:Cx=b x1 x3x2 x1x3 x2 = 135 Left sides add to0 Right sides add to9No solutionx1, x2, x3(13)Look at this example geometrically. No combination ofu,v,andw will produce thevectorb= (1,3,5).

9 The combinations don t fill the whole three-dimensional right sides must haveb1+b2+b3= 0to allow a solution toCx=b, becausethe left sidesx1 x3,x2 x1, andx3 x2always add to zero. Put that in different words :All Linear combinationsx1u+x2v+x3w lie on the plane given byb1+b2+b3= subject is suddenly connecting algebra with combinations can fill allof space, or only a plane. We need a picture to show the crucialdifference betweenu,v,w(the first example) andu,v,w (all in the same plane).Figure : Independent vectorsu,v,w. Dependent vectorsu,v,w in a and DependenceFigure shows those column vectors, first of the matrixAand then ofC. The first twocolumnsuandvare the same in both pictures. If we only look at the combinations ofthose two vectors, we will get a two-dimensional key question is whether thethird vector is in that plane:Independencewis not in the plane isin the plane important point is that the new vectorw is a Linear combination ofuandv:u+v+w = 0w = 101 = u v.

10 (14) Matrices27 All three vectorsu,v,w have components adding to zero. Then all their combinationswill haveb1+b2+b3= 0(as we saw above, by adding the three equations ). This is theequation for the plane containing all combinations ofuandv. By includingw we getno new vectorsbecausew is already on that originalw= (0,0,1)is not on the plane:0 + 0 + 16= 0. The combinations ofu,v,wfill the whole three-dimensional space. We know this already, because the solutionx=A 1bin equation (6) gave the right combination to produce two matricesAandC, with third columnswandw , allowed me to mention twokey words of Linear algebra : independence and dependence. The first half of the course willdevelop these ideas much further I am happy if you see them early in the two examples:u,v,wareindependent. No combination except0u+ 0v+ 0w=0givesb= ,v,w aredependent.