Linear Algebra and Its Applications

1 Linear Algebra and Its ApplicationsFourth EditionGilbert Strangyx y z zAx b b0Ay b 0Az 0 ContentsPrefaceiv1 Matrices and Gaussian .. Geometry of Linear Equations .. Example of Gaussian Elimination .. Notation and Matrix Multiplication .. Factors and Row Exchanges .. and Transposes .. Matrices and Applications ..66 Review Exercises ..722 Vector Spaces and Subspaces .. Independence, Basis, and Dimension .. Four Fundamental Subspaces .. and Networks .. Transformations ..140 Review Exercises ..1543 Vectors and Subspaces .. and Projections onto Lines .. and Least Squares .. Bases and Gram-Schmidt .. Fast Fourier Transform ..211 Review Exercises ..221iiiCONTENTS4 .. of the Determinant.

2 For the Determinant .. of Determinants ..247 Review Exercises ..2585 Eigenvalues and .. of a Matrix .. Equations and PowersAk.. Equations andeAt.. Matrices .. Transformations ..325 Review Exercises ..3416 Positive Definite , Maxima, and Saddle Points .. for Positive Definiteness .. Value Decomposition .. Principles .. Finite Element Method ..3847 Computations with .. Norm and Condition Number .. of Eigenvalues .. Methods forAx=b..4078 Linear Programming and Game Inequalities .. Simplex Method .. Dual Problem .. Models .. Theory ..451A Intersection, Sum, and Product of The Intersection of Two Vector Spaces .. The Sum of Two Vector Spaces .. The Cartesian Product of Two Vector Spaces.

3 The Tensor Product of Two Vector Spaces .. The Kronecker ProductA Bof Two Matrices ..462 CONTENTSiiiB The Jordan Form466C Matrix Factorizations473D Glossary: A Dictionary for Linear Algebra475E MATLAB Teaching Codes484F Linear Algebra in a Nutshell486AT~y=~0A~x=~0~0~0 RnRmRow SpaceColumn SpaceallAT~yallA~xNullSpaceLeftNull SpaceA~x=~bAT~y=~cC(AT)dimrC(A)dimrN(A)d imn rN(AT)dimm r PrefaceRevising this textbook has been a special challenge, for a very nice reason. So manypeople have read this book, and taught from it, and even loved it. The spirit of the bookcould never change. This text was written to help our teaching of Linear Algebra keep upwith the enormous importance of this subject which just continues to step was certainly possible and desirable to add new problems.

4 Teaching for allthese years required hundreds of new exam questions (especially with quizzes going ontothe web). I think you will approve of the extended choice of problems. The questions arestill a mixture ofexplain and compute the two complementary approaches to learningthis beautiful personally believe that many more people need Linear Algebra than calculus. IsaacNewton might not agree! But he isn t teaching mathematics in the 21st century (andmaybe he wasn t a great teacher, but we will give him the benefit of the doubt). Cer-tainly the laws of physics are well expressed by differential equations. Newton neededcalculus quite right. But the scope of science and engineering and management (andlife) is now so much wider, and Linear Algebra has moved into a central I say a little more, because many universities have not yet adjusted the balancetoward Linear Algebra .

5 Working with curved lines and curved surfaces, the first step isalways tolinearize. Replace the curve by its tangent line, fit the surface by a plane,and the problem becomes Linear . The power of this subject comes when you have tenvariables, or 1000 variables, instead of might think I am exaggerating to use the word beautiful for a basic coursein mathematics. Not at all. This subject begins with two vectorsvandw, pointing indifferent directions. The key step is totake their Linear combinations. We multiply toget3vand4w, and we add to get the particular combination3v+4w. That new vectoris in thesame planeasvandw. When we take all combinations, we are filling in thewhole plane. If I drawvandwon this page, their combinationscv+dwfill the page(and beyond), but theydon t go upfrom the the language of Linear equations, I can solvecv+dw=bexactly when the vectorblies in the same plane will keep going a little more to convert combinations of three-dimensional vectors intolinear Algebra .

6 If the vectors arev= (1,2,3)andw= (1,3,4), put them into thecolumnsof a matrix:matrix= 1 12 33 4 .To find combinations of those columns, multiply the matrix by a vector(c,d): Linear combinationscv+dw 1 12 33 4 [cd]=c 123 +d 134 .Those combinations fill avector space. We call it thecolumn spaceof the matrix. (Forthese two columns, that space is a plane.) To decide ifb= (2,5,7)is on that plane, wehave three components to get right. So we have three equations to solve: 1 12 33 4 [cd]= 257 meansc+d=22c+3d=53c+4d= leave the solution to you. The vectorb= (2,5,7)does lie in the plane the 7 changes to any other number, thenbwon t lie in the plane it willnotbe acombination ofvandw, and the three equations will have no I can describe the first part of the book, about Linear equationsAx=b.

7 ThematrixAhasncolumns Algebra moves steadily to n vectors in m-dimensional space. We still want combinations of the columns (in the column space).We still getmequations to produceb(one for each row). Those equations may or maynot have a solution. They always have a least-squares interplay of columns and rows is the heart of Linear Algebra . It s not totally easy,but it s not too hard. Here are four of the central space(all combinations of the columns). space(all combinations of the rows). (the number of independent columns) (or rows). (the good way to find the rank of a matrix).I will stop here, so you can start the PagesIt may be helpful to mention the web pages connected to this book. So many messagescome back with suggestions and encouragement, and I hope you will make free useof everything.

8 You can directly , which is continuallyupdated for the course that is taught every semester. Linear Algebra is also on MIT sOpenCourseWare , where became exceptional by includingvideos of the lectures (which you definitely don t have to ). Here is a part ofwhat is available on the schedule and current homeworks and exams with goals of the course, and conceptual Java demos (audio is now included for eigenvalues). Algebra Teaching Codes of the complete course (taught in a real classroom).The course page has become a valuable link to the class, and a resource for the am very optimistic about the potential for graphics with sound. The bandwidth forvoiceover is low, and FlashPlayer is freely available. This offers aquick review(withactive experiment), and the full lectures can be downloaded.

9 I hope professors andstudents worldwide will find these web pages helpful. My goal is to make this book asuseful as possible with all the course material I can Supporting MaterialsStudent Solutions Manual 0-495-01325-0 The Student Solutions Manual providessolutions to the odd-numbered problems in the s Solutions Manual 0-030-10588-4 The Instructor s Solutions Man-ual has teaching notes for each chapter and solutions to all of the problems in the of the CourseThe two fundamental problems areAx=bandAx= xfor square matricesA. The firstproblemAx=bhas a solution whenAhasindependent columns. The second problemAx= xlooks forindependent eigenvectors. A crucial part of this course is to learnwhat independence believe that most of us learn first from examples.

10 You can see thatA= 1 1 21 2 31 3 4 doesnothave independent 1 plus column 2 equals column 3. A wonderful theorem of Linear Algebra saysthat the three rows are not independent either. The third row must lie in the same planeas the first two rows. Some combination of rows 1 and 2 will produce row 3. You mightfind that combination quickly (I didn t). In the end I had to use elimination to discoverthat the right combination uses 2 times row 2, minus row is the simple and natural way to understand a matrix by producing a lotof zero entries. So the course starts there. But don t stay there too long! You have to getfrom combinations of the rows, to independence of the rows, to dimension of the rowspace. That is a key goal, to see whole spaces of vectors: therow spaceand thecolumnspaceand further goal is to understand how the matrixacts.