Linear algebra explained in four pages

1 Linear algebra explained in four pagesExcerpt from the NO BULLSHIT GUIDE TO Linear ALGEBRAby Ivan SavovAbstract This document will review the fundamental ideas of Linear will learn about matrices, matrix operations, Linear transformations anddiscuss both the theoretical and computational aspects of Linear algebra . Thetools of Linear algebra open the gateway to the study of more advancedmathematics. A lot ofknowledge buzzawaits you if you choose to follow thepath ofunderstanding, instead of trying to memorize a bunch of INTRODUCTIONL inear algebra is the math of vectors and matrices. Letnbe a positiveinteger and letRdenote the set of real numbers, thenRnis the set of alln-tuples of real numbers. A vector~v Rnis ann-tuple of real notation S is read element ofS. For example, consider a vectorthat has three components:~v= (v1,v2,v3) (R,R,R) matrixA Rm nis a rectangular array of real numbers withmrowsandncolumns.

2 For example, a3 2matrix looks like this:A= a11a12a21a22a31a32 R RR RR R R3 purpose of this document is to introduce you to the mathematicaloperations that we can perform on vectors and matrices and to give you afeel of the power of Linear algebra . Many problems in science, business,and technology can be described in terms of vectors and matrices so it isimportant that you understand how to work with only prerequisite for this tutorial is a basic understanding of high schoolmath concepts1like numbers, variables, equations, and the fundamentalarithmetic operations on real numbers: addition (denoted+), subtraction(denoted ), multiplication (denoted implicitly), and division (fractions).You should also be familiar withfunctionsthat take real numbers asinputs and give real numbers as outputs,f:R R.

3 Recall that, bydefinition, theinverse functionf 1undoesthe effect off. If you aregivenf(x)and you want to findx, you can use the inverse function asfollows:f 1(f(x)) =x. For example, the functionf(x) = ln(x)has theinversef 1(x) =ex, and the inverse ofg(x) = xisg 1(x) = DEFINITIONSA. Vector operationsWe now define the math operations for vectors. The operations we canperform on vectors~u= (u1,u2,u3)and~v= (v1,v2,v3)are: addition,subtraction, scaling, norm (length), dot product, and cross product:~u+~v= (u1+v1,u2+v2,u3+v3)~u ~v= (u1 v1,u2 v2,u3 v3) ~u= ( u1, u2, u3)||~u||= u21+u22+u23~u ~v=u1v1+u2v2+u3v3~u ~v= (u2v3 u3v2, u3v1 u1v3, u1v2 u2v1)The dot product and the cross product of two vectors can also be describedin terms of the angle between the two vectors. The formula for the dotproduct of the vectors is~u ~v= ~u ~v cos.

4 We say two vectors~uand~vareorthogonalif the angle between them is90 . The dot product oforthogonal vectors is zero:~u ~v= ~u ~v cos(90 ) = the cross product is given by ~u ~v = ~u ~v sin . Thecross product is not commutative:~u ~v6=~v ~u, in fact~u ~v= ~v ~ good textbook to (re)learn high school math Matrix operationsWe denote byAthe matrix as a whole and refer to its entries mathematical operations defined for matrices are the following: addition (denoted+)C=A+B cij=aij+bij. subtraction (the inverse of addition) matrix product. The product of matricesA Rm nandB Rn `is another matrixC Rm `given by the formulaC=AB cij=n k=1aikbkj, a11a12a21a22a31a32 [b11b12b21b22]= a11b11+a12b21a11b12+a12b22a21b11+a22b21a 21b12+a22b22a31b11+a32b21a31b12+a32b22 matrix inverse (denotedA 1) matrix transpose (denotedT):[ 1 2 3 1 2 3]T= 1 1 2 2 3 3.

5 Matrix trace: Tr[A] ni=1aii determinant (denoted det(A)or|A|)Note that the matrix product is not a commutative operation:AB6= Matrix-vector productThe matrix-vector product is an important special case of the matrix-matrix product. The product of a3 2matrixAand the2 1columnvector~xresults in a3 1vector~ygiven by:~y=A~x y1y2y3 = a11a12a21a22a31a32 [x1x2]= a11x1+a12x2a21x1+a22x2a31x1+a32x2 =x1 a11a21a31 +x2 a12a22a32 (C)= (a11,a12) ~x(a21,a22) ~x(a31,a32) ~x .(R)There are two2fundamentally different yet equivalent ways to interpret thematrix-vector product. In the column picture,(C), the multiplication of thematrixAby the vector~xproduces alinear combination of the columnsof the matrix:~y=A~x=x1A[:,1]+x2A[:,2], whereA[:,1]andA[:,2]arethe first and second columns of the the row picture,(R), multiplication of the matrixAby the vector~xproduces a column vector with coefficients equal to thedot products ofrows of the matrixwith the vector~ Linear transformationsThe matrix-vector product is used to define the notion of alineartransformation, which is one of the key notions in the study of linearalgebra.

6 Multiplication by a matrixA Rm ncan be thought of ascomputing alinear transformationTAthat takesn-vectors as inputs andproducesm-vectors as outputs:TA:Rn more info see the video of Prof. Strang s MIT of writing~y=TA(~x)for the Linear transformationTAapplied tothe vector~x, we simply write~y=A~x. Applying the Linear transformationTAto the vector~xcorresponds to the product of the matrixAand thecolumn vector~x. We sayTAisrepresented bythe can think of Linear transformations as vector functions and describetheir properties in analogy with the regular functions you are familiar with:functionf:R R Linear transformationTA:Rn Rminputx R input~x Rnoutputf(x) outputTA(~x) =A~x Rmg f=g(f(x)) TB(TA(~x)) =BA~xfunction inversef 1 matrix inverseA 1zeros off N(A) null space ofArange off C(A) column space ofA=range ofTANote that the combined effect of applying the transformationTAfollowedbyTBon the input vector~xis equivalent to the matrix productBA~ Fundamental vector spacesAvector spaceconsists of a set of vectors and all Linear combinations ofthese vectors.

7 For example the vector spaceS=span{~v1,~v2}consists ofall vectors of the form~v= ~v1+ ~v2, where and are real now define three fundamental vector spaces associated with a spaceof a matrixAis the set of vectors that can be producedas Linear combinations of the columns of the matrixA:C(A) {~y Rm|~y=A~xfor some~x Rn}.The column space is therangeof the Linear transformationTA(the setof possible outputs). You can convince yourself of this fact by reviewingthe definition of the matrix-vector product in the column picture(C). ThevectorA~xcontainsx1times the 1stcolumn ofA,x2times the 2ndcolumnofA, etc. Varying over all possible inputs~x, we obtain all possible linearcombinations of the columns ofA, hence the name column space. Thenull spaceN(A)of a matrixA Rm nconsists of all the vectorsthat the matrixAsends to the zero vector:N(A) {~x Rn|A~x=~0}.

8 The vectors in the null space areorthogonalto all the rows of the can see this from the row picture(R): the output vectors is~0if andonly if the input vector~xis orthogonal to all the rows spaceof a matrixA, denotedR(A), is the set of linearcombinations of the rows ofA. The row spaceR(A)is the orthogonalcomplement of the null spaceN(A). This means that for all vectors~v R(A)and all vectors~w N(A), we have~v ~w= 0. Together, thenull space and the row space form the domain of the transformationTA,Rn=N(A) R(A), where stands fororthogonal direct Matrix inverseBy definition, the inverse matrixA 1undoesthe effects of the cumulative effect of applyingA 1afterAis the identity matrix1:A 1A=1 .The identity matrix (ones on the diagonal and zeros everywhere else)corresponds to the identity transformation:T1(~x) =1~x=~x, for all~ matrix inverse is useful for solving matrix equations.

9 Whenever wewant to get rid of the matrixAin some matrix equation, we can hit Awith its inverseA 1to make it disappear. For example, to solve for thematrixXin the equationXA=B, multiply both sides of the equationbyA 1from the right:X=BA 1. To solve forXinABCXD=E,multiply both sides of the equation byD 1on the right and byA 1,B 1andC 1(in that order) from the left:X=C 1B 1A 1ED COMPUTATIONAL Linear ALGEBRAOkay, I hear what you are saying Dude, enough with the theory talk, let ssee some calculations. In this section we ll look at one of the fundamentalalgorithms of Linear algebra called Gauss Jordan Solving systems of equationsSuppose we re asked to solve the following system of equations:1x1+ 2x2= 5,3x1+ 9x2= 21.(1)Without a knowledge of Linear algebra , we could use substitution, elimina-tion, or subtraction to find the values of the two Jordan elimination is a systematic procedure for solving systemsof equations based the followingrow operations: ) Adding a multiple of one row to another row ) Swapping two rows ) Multiplying a row by a constantThese row operations allow us to simplify the system of equations withoutchanging their illustrate the Gauss Jordan elimination procedure, we ll now show thesequence of row operations required to solve the system of Linear equationsdescribed above.

10 We start by constructing anaugmented matrixas follows:[1253921].The first column in the augmented matrix corresponds to the coefficients ofthe variablex1, the second column corresponds to the coefficients ofx2,and the third column contains the constants from the right-hand Gauss-Jordan elimination procedure consists of two phases. Duringthe first phase, we proceed left-to-right by choosing a row with a leadingone in the leftmost column (called apivot) and systematically subtractingthat row from all rows below it to get zeros below in the entire column. Inthe second phase, we start with the rightmost pivot and use it to eliminateall the numbers above it in the same column. Let s see this in ) The first step is to use the pivot in the first column to eliminate thevariablex1in the second row. We do this by subtracting three timesthe first row from the second row, denotedR2 R2 3R1,[125036].