Linear algebra explained in four pages

1 Linear algebra explained in four pagesExcerpt from the NO BULLSHIT GUIDE TO Linear ALGEBRAby Ivan SavovAbstract This document will review the fundamental ideas of Linear will learn about matrices, matrix operations, Linear transformations anddiscuss both the theoretical and computational aspects of Linear algebra . Thetools of Linear algebra open the gateway to the study of more advancedmathematics. A lot ofknowledge buzzawaits you if you choose to follow thepath ofunderstanding, instead of trying to memorize a bunch of INTRODUCTIONL inear algebra is the math of vectors and matrices.

2 Letnbe a positiveinteger and letRdenote the set of real numbers, thenRnis the set of alln-tuples of real numbers. A vector~v Rnis ann-tuple of real notation S is read element ofS. For example, consider a vectorthat has three components:~v= (v1,v2,v3) (R,R,R) matrixA Rm nis a rectangular array of real numbers withmrowsandncolumns. For example, a3 2matrix looks like this:A= a11a12a21a22a31a32 R RR RR R R3 purpose of this document is to introduce you to the mathematicaloperations that we can perform on vectors and matrices and to give you afeel of the power of Linear algebra .

3 Many problems in science, business,and technology can be described in terms of vectors and matrices so it isimportant that you understand how to work with only prerequisite for this tutorial is a basic understanding of high schoolmath concepts1like numbers, variables, equations, and the fundamentalarithmetic operations on real numbers: addition (denoted+), subtraction(denoted ), multiplication (denoted implicitly), and division (fractions).You should also be familiar withfunctionsthat take real numbers asinputs and give real numbers as outputs,f:R R.

4 Recall that, bydefinition, theinverse functionf 1undoesthe effect off. If you aregivenf(x)and you want to findx, you can use the inverse function asfollows:f 1(f(x)) =x. For example, the functionf(x) = ln(x)has theinversef 1(x) =ex, and the inverse ofg(x) = xisg 1(x) = DEFINITIONSA. Vector operationsWe now define the math operations for vectors. The operations we canperform on vectors~u= (u1,u2,u3)and~v= (v1,v2,v3)are: addition,subtraction, scaling, norm (length), dot product, and cross product:~u+~v= (u1+v1,u2+v2,u3+v3)~u ~v= (u1 v1,u2 v2,u3 v3) ~u= ( u1, u2, u3)||~u||= u21+u22+u23~u ~v=u1v1+u2v2+u3v3~u ~v= (u2v3 u3v2, u3v1 u1v3, u1v2 u2v1)The dot product and the cross product of two vectors can also be describedin terms of the angle between the two vectors.

5 The formula for the dotproduct of the vectors is~u ~v= ~u ~v cos . We say two vectors~uand~vareorthogonalif the angle between them is90 . The dot product oforthogonal vectors is zero:~u ~v= ~u ~v cos(90 ) = the cross product is given by ~u ~v = ~u ~v sin . Thecross product is not commutative:~u ~v6=~v ~u, in fact~u ~v= ~v ~ good textbook to (re)learn high school math Matrix operationsWe denote byAthe matrix as a whole and refer to its entries mathematical operations defined for matrices are the following: addition (denoted+)C=A+B cij=aij+bij. subtraction (the inverse of addition) matrix product.

6 The product of matricesA Rm nandB Rn `is another matrixC Rm `given by the formulaC=AB cij=n k=1aikbkj, a11a12a21a22a31a32 [b11b12b21b22]= a11b11+a12b21a11b12+a12b22a21b11+a22b21a 21b12+a22b22a31b11+a32b21a31b12+a32b22 matrix inverse (denotedA 1) matrix transpose (denotedT):[ 1 2 3 1 2 3]T= 1 1 2 2 3 3 . matrix trace: Tr[A] ni=1aii determinant (denoted det(A)or|A|)Note that the matrix product is not a commutative operation:AB6= Matrix-vector productThe matrix-vector product is an important special case of the matrix-matrix product. The product of a3 2matrixAand the2 1columnvector~xresults in a3 1vector~ygiven by:~y=A~x y1y2y3 = a11a12a21a22a31a32 [x1x2]= a11x1+a12x2a21x1+a22x2a31x1+a32x2 =x1 a11a21a31 +x2 a12a22a32 (C)= (a11,a12) ~x(a21,a22) ~x(a31,a32) ~x.

7 (R)There are two2fundamentally different yet equivalent ways to interpret thematrix-vector product. In the column picture,(C), the multiplication of thematrixAby the vector~xproduces alinear combination of the columnsof the matrix:~y=A~x=x1A[:,1]+x2A[:,2], whereA[:,1]andA[:,2]arethe first and second columns of the the row picture,(R), multiplication of the matrixAby the vector~xproduces a column vector with coefficients equal to thedot products ofrows of the matrixwith the vector~ Linear transformationsThe matrix-vector product is used to define the notion of alineartransformation, which is one of the key notions in the study of linearalgebra.

8 Multiplication by a matrixA Rm ncan be thought of ascomputing alinear transformationTAthat takesn-vectors as inputs andproducesm-vectors as outputs:TA:Rn more info see the video of Prof. Strang s MIT of writing~y=TA(~x)for the Linear transformationTAapplied tothe vector~x, we simply write~y=A~x. Applying the Linear transformationTAto the vector~xcorresponds to the product of the matrixAand thecolumn vector~x. We sayTAisrepresented bythe can think of Linear transformations as vector functions and describetheir properties in analogy with the regular functions you are familiar with:functionf:R R Linear transformationTA.

9 Rn Rminputx R input~x Rnoutputf(x) outputTA(~x) =A~x Rmg f=g(f(x)) TB(TA(~x)) =BA~xfunction inversef 1 matrix inverseA 1zeros off N(A) null space ofArange off C(A) column space ofA=range ofTANote that the combined effect of applying the transformationTAfollowedbyTBon the input vector~xis equivalent to the matrix productBA~ Fundamental vector spacesAvector spaceconsists of a set of vectors and all Linear combinations ofthese vectors. For example the vector spaceS=span{~v1,~v2}consists ofall vectors of the form~v= ~v1+ ~v2, where and are real now define three fundamental vector spaces associated with a spaceof a matrixAis the set of vectors that can be producedas Linear combinations of the columns of the matrixA:C(A) {~y Rm|~y=A~xfor some~x Rn}.

10 The column space is therangeof the Linear transformationTA(the setof possible outputs). You can convince yourself of this fact by reviewingthe definition of the matrix-vector product in the column picture(C). ThevectorA~xcontainsx1times the 1stcolumn ofA,x2times the 2ndcolumnofA, etc. Varying over all possible inputs~x, we obtain all possible linearcombinations of the columns ofA, hence the name column space. Thenull spaceN(A)of a matrixA Rm nconsists of all the vectorsthat the matrixAsends to the zero vector:N(A) {~x Rn|A~x=~0}.The vectors in the null space areorthogonalto all the rows of the can see this from the row picture(R): the output vectors is~0if andonly if the input vector~xis orthogonal to all the rows spaceof a matrixA, denotedR(A), is the set of linearcombinations of the rows ofA.