Lecture notes Math 4377/6308 { Advanced Linear …

1 Lecture notesMath 4377/6308 Advanced Linear algebra IVaughn ClimenhagaOctober 7, 20132 The primary text for this course is Linear algebra and its Applications ,second edition, by Peter D. Lax (hereinafter referred to as[Lax]). Thelectures will follow the presentation in this book, and many of the homeworkexercises will be taken from may occasionally find it helpful to have access to other resourcesthat give a more expanded and detailed presentation of various topics thanis available in Lax s book or in the Lecture notes . To this end I suggest thefollowing list of external references, which are freely available online.(Bee) A First Course in Linear algebra , by Robert A. Beezer, Universityof Puget Sound. Long and comprehensive (1027 pages). Starts fromthe very beginning: vectors and matrices as arrays of numbers, systemsof equations, row reduction. Organisation of book is a little non-standard: chapters and sections are given abbreviations instead (CDW) Linear algebra , by David Cherney, Tom Denton, and AndrewWaldron, UC Davis.

2 308 pages. Covers similar material to[Bee]. ~ Linear /(Hef ) Linear algebra , by Jim Hefferon, Saint Michael s College. 465pages. Again, starts from the very (LNS) Linear algebra as an Introduction to Abstract Mathematics , byIsaiah Lankham, Bruno Nachtergaele, and Anne Schilling, UC pages. More focused on abstraction than the previous three ref-erences, and hence somewhat more in line with the present ~anne/linear_algebra/(Tre) Linear algebra Done Wrong ,1by Sergei Treil, Brown pages. Starts from the beginning but also takes a more ~treil/papers/ books listed above can all be obtained freely via the links provided.(These links are also on the website for this course.) Another potentiallyuseful resource is the series of video lectures by Gilbert Strang from MIT sOpen CourseWare project: the title seems strange, it may help to be aware that there is a relatively famoustextbook by Sheldon Axler called Linear algebra Done Right , which takes a differentapproach to Linear algebra than do many other books, including the ones 1 Monday, Aug.

3 26 Motivation, Linear spaces, and isomorphismsFurther reading:[Lax]Ch. 1 (p. 1 4). See also[Bee]p. 317 333;[CDW]Ch. 5 (p. 79 87);[Hef ]Ch. 2 (p. 76 87);[LNS]Ch. 4 (p. 36 40);[Tre]Ch. 1 (p. 1 5) General motivationWe begin by mentioning a few examples that on the surface may not appearto have anything to do with Linear algebra , but which turn out to involveapplications of the machinery we will develop in this course. These (andother similar examples) serve as a motivation for many of the things thatwe Fibonacci sequence is the sequence ofnumbers 1,1,2,3,5,8,13,.., where each number is the sum of theprevious two. We can use Linear algebra to find an exact formula forthenth term. Somewhat surprisingly, it has the odd-looking form1 5((1 + 52)n (1 52)n).We will discuss this example when we talk about eigenvalues, eigen-vectors, and algebra and Markov chain methods are at the heartof the PageRank algorithm that was central to Google s early successas an internet search engine.

4 We will discuss this near the end of single-variable calculus, the derivative isa number, while in multivariable calculus it is a matrix. The properway to understand this is that in both cases, the derivative is a lineartransformation. We will reinforce this point of view throughout value is an important tool that hasapplications to image compression, suggestion algorithms such as thoseused by Netflix, and many other areas. We will mention these nearthe end of the course, time 1. MONDAY, AUG. I start with a sphere, and rotate it first aroundone axis (through whatever angle I like) and then around a differentaxis (again through whatever angle I like). How does the final positionof the sphere relate to the initial one? Could I have gotten from start tofinish by doing a single rotation around a single axis? How would thataxis relate to the axes I actually performed rotations around? Thisand other questions in three-dimensional geometry can be answeredusing Linear algebra , as we will see differential important problems in ap-plied mathematics and engineering can be formulated as partial dif-ferential equations; the heat equation and the wave equation are twofundamental examples.

5 A complete theory of PDEs requires functionalanalysis, which considers vector spaces whose elements are not arraysof numbers (as inRn), but rather functions with certain differentiabil-ity are many other examples: to chemistry (vibrations of molecules interms of their symmetries), integration techniques in calculus (partial frac-tions), magic squares, error-correcting codes, Background: general mathematical notationand terminologyThroughout this course we will assume a working familiarity with standardmathematical notation and terminology. Some of the key pieces of back-ground are reviewed on the first assignment, which is due at the beginningof the next example, recall that the symbolRstands for the set of real numbers;Cstands for the set of complex numbers;Zstands for the integers (bothpositive and negative); andNstands for the natural numbers 1,2,3,.. Ofparticular importance will be the use of the quantifiers ( there exists ) and ( for all ), which will appear in many definitions and theorems throughoutthe The statement x Rsuch thatx+ 2 = 7 is true,because we can choosex= The statement x+ 2 = 7 x R is false, becausex+ 26= 7 whenx6= VECTOR SPACES53.

6 The statement x R y Rsuch thatx+y= 4 is true, becauseno matter what value ofxis chosen, we can choosey= 4 xand thenwe havex+y=x+ (4 x) = last example hasnestedquantifiers: the quantifier occurs insidethe statement to which applies. You may find it helpful to interpret suchnested statements as a game between two players. In this example, PlayerA has the goal of making the statementx+y= 4 (the innermost statement)be true, and the game proceeds as follows: first Player B chooses a numberx R, and then Player A choosesy R. If Player A s choice makes it sothatx+y= 4, then Player A wins. The statement in the example is truebecause Player A can always statement y Rsuch that x R,x+y= 4 isfalse. In the language of the game played just above, Player A is forced tochoosey Rfirst, and then Player B can choose anyx R. Because PlayerB gets to chooseafterPlayer A, he can make it so thatx+y6= 4, so PlayerA parse such statements it may also help to use parentheses: the state-ment in Example would become y R(such that x R(x+y= 4)).

7 Playing the game described above corresponds to parsing the statement fromthe outside in. This is also helpful when finding the negation of the state-ment (formally, itscontrapositive informally, its opposite).Example negations of the three statements in Example are1. x Rwe havex+ 26= x Rsuch thatx+ 26= x Rsuch that y Rwe havex+y6= the pattern here: working from the outside in, each is replacedwith , each is replaced with , and the innermost statement is negated (so= becomes6=, for example). You should think through this to understandwhy this is the Vector spacesIn your first Linear algebra course you studied vectors as rows or columnsof numbers that is, elements ofRn. This is the most important example6 Lecture 1. MONDAY, AUG. 26of a vector space, and is sufficient for many applications, but there are alsomany other applications where it is important to take the lessons from thatfirst course and re-learn them in a more abstract do we mean by a more abstract setting ?

8 The idea is that weshould look at vectors inRnand the things we did with them, and seeexactly what properties we needed in order to use the various definitions,theorems, techniques, and algorithms we learned in that for the moment, think of a vector as an element ofRn. What can wedo with these vectors? A moment s thought recalls several things:1. we can add vectors together;2. we can multiply vectors by real numbers (scalars) to get another vector,which in some sense points in the same direction ;3. we can multiply vectors by matrices;4. we can find the length of a vector;5. we can find the angle between two list could be extended, but this will do for now. Indeed, for the timebeing we will focus only on the first two items on the last. The others willenter : vectors are things that we can add together, and that we can multiplyby scalars. This motivates the following space(orlinear space) overRis a setXon whichtwo operations are defined: addition, so that given anyx,y Xwe can consider their sumx+y X; scalar multiplication, so that given anyx Xandc Rwe canconsider their productcx operations of addition and scalar multiplication are required to satisfycertain properties:1.

9 Commutativity:x+y=y+xfor everyx,y X;2. associativity of addition:x+ (y+z) = (x+y) +zfor everyx,y,z X;3. identity element: there exists an element0 Xsuch thatx+0=xfor allx X; VECTOR SPACES74. additive inverses: for everyx Xthere exists ( x) Xsuch thatx+ ( x) =0;5. associativity of multiplication:a(bx) = (ab)xfor alla,b Randx X;6. distributivity:a(x+y) =ax+ayand (a+b)x=ax+bxfor alla,b Randx,y X;7. multiplication by the unit: 1x=xfor allx properties in the list above are theaxiomsof a vector space. Theyhold forRnwith the usual definition of addition and scalar , this is in some sense the motivation for this list of axioms: they for-malise the properties that we know and love for the example of row/columnvectors inRn. We will see that these properties are in fact enough to let usdo a great deal of work, and that there are plenty of other things besidesRnthat satisfy textbooks use different font styles or some other typo-graphic device to indicate that a particular symbol refers to a vector, insteadof a scalar.

10 For example, one may writexor~xinstead ofxto indicate anelement of a vector space. By and large we will not do this; rather, plainlowercase letters will be used to denote both scalars and vectors (althoughwe will write0for the zero vector, and 0 for the zero scalar). It will alwaysbe clear from context which type of object a letter represents: for example,in Definition it is always specified whether a letter represents a vector(as inx X) or a scalar (as ina R). You should be very careful whenreading and writing mathematical expressions in this course that you arealways aware of whether a particular symbol stands for a scalar, a vector,or something moving on to some examples, we point out that one may alsoconsider vector spaces overC, the set of complex numbers; here the scalarsmay be any complex numbers. In fact, one may consider anyfieldKanddo Linear algebra with vector spaces overK. This has many interestingapplications, particularly ifKis taken to be a finite field, but these exampleslie beyond the scope of this course, and while we will often say LetXbea vector space over the fieldK , it will always be the case in our examplesthatKis eitherRorC.