Linear algebra in R - UH

1 Linear algebra in RS ren H jsgaardFebruary 15, 2005 Contents1 Introduction12 Vectors .. Transpose of vectors .. Multiplying a vector by a number .. Sum of vectors .. (Inner) product of vectors .. The length (norm) of a vector .. The 0 vector and 1 vector .. Orthogonal (perpendicular) vectors ..53 Matrices .. Multiplying a matrix with a number .. Transpose of matrices .. Sum of matrices .. Multiplication of a matrix and a vector .. Multiplication of matrices .. Vectors as matrices .. Some special matrices .. Inverse of matrices .. Solving systems of Linear equations .. Trace .. Determinant .. Some additional rules for matrix operations .. Details on inverse matrices* .. Inverse of a 2 2 matrix* .. Inverse of diagonal matrices*.

2 Generalized inverse* .. Inverting ann nmatrix* ..134 Least squares155 A neat little exercise from a bird s perspective161 IntroductionThis note has two goal: 1) Introducing Linear algebra (vectors and matrices) and 2)showing how to work with these concepts in VectorsA column vector is a list of numbers stacked on top of each other, 213 A row vector is a list of numbers written one after the other, (2,1,3)In both cases, the list is ordered, (2,1,3)6= (1,2,3).We make the following convention: In what follows all vectors are column vectors unless otherwise stated. However, writing column vectors takes up more space than row vectors. There-fore we shall frequently write vectors as row vectors, but with the understand-ing that it really is a column generaln vector has the forma= where theais are numbers, and this vector shall be writtena= (a1.

3 , an).A graphical representation of 2 vectors is shown Figure 1. Note that row and 10123 10123a1=(2,2)a2=(1, )Figure 1: Two 2-vectorscolumn vectors are drawn the same way.> a <- c(1, 3, 2)> a[1] 1 3 2 The vectorais in R printed in row format but can really be regarded as acolumn vector, cfr. the convention Transpose of vectorsTransposing a vector means turning a column (row) vector into a row (column)vector. The transpose is denoted by > .Example 12413235>= [1,3,2]og[1,3,2]>=2413235 Hence transposing twice takes us back to where we started:a= (a>)>> t(a)[,1] [,2] [,3][1,] 1 3 Multiplying a vector by a numberIfais a vector and is a number then ais the vector a= a1 an See Figure 27 132 = 72114 > 7 * a[1] 7 21 143 10123 10123a1=(2,2)a2=(1, )2a2=(2, 1) a2=( 1, )Figure 2: Multiplication of a vector by a Sum of vectorsLetaandbben vectors.

4 The suma+bis then vectora+b= + = a1+b1a2+ +bn =b+aSee Figure 3 and 4. Only vectors of the same dimension can be 32413235+2428935=241 + 23 + 82 + 935=243111135 10123 10123a1=(2,2)a2=(1, )a1+a2=(3, )Figure 3: Addition of vectors4 10123 10123a1=(2,2)a2=(1, )a1+a2=(3, ) a2=( 1, )a1+( a2)=(1, )Figure 4: Addition of vectors and multiplication by a number> a <- c(1, 3, 2)> b <- c(2, 8, 9)> a + b[1] 3 11 (Inner) product of vectorsLeta= (a1, .. , an) andb= (b1, .. , bn). The (inner) product ofaandbisa b=a1b1+ +anbnNote, that the product is a number not a vector.> sum(a * b)[1] The length (norm) of a vectorThe length (or norm) of a vectorais||a||= a a= n i=1a2i> sqrt(sum(a * a))[1] The 0 vector and 1 vectorThe 0-vector (1 vector) is a vector with 0 (1) on all entries.

5 The 0 vector(1 vector) is frequently written simply as 0 (1) or as 0n(1n) to emphasizethat its lengthn.> rep(0, 5)[1] 0 0 0 0 0> rep(1, 5)[1] 1 1 1 1 Orthogonal (perpendicular) vectorsTwo vectorsv1andv2are orthogonal if their inner product is zero, writtenv1 v2 v1 v2= 0> v1 <- c(1, 1)> v2 <- c(-1, 1)> sum(v1 * v2)[1] 03 MatricesAnr cmatrixA(reads anrtimescmatrix ) is a table withr rows ogc columnsA= a11a12.. a1ca21a22.. arc Note that one can regardAas consisting ofccolumns vectors put after each other:A= [a1:a2: :ac]> A <- matrix(c(1, 3, 2, 2, 8, 9), ncol = 3)> A[,1] [,2] [,3][1,] 1 2 8[2,] 3 2 96 Note that the numbers 1,3,2,2,8,9 are read into the matrix column by column. To get the numbers read in row by row do> A2 <- matrix(c(1, 3, 2, 2, 8, 9), ncol = 3, byrow = T)> A2[,1] [,2] [,3][1,] 1 3 2[2,] 2 8 Multiplying a matrix with a numberFor a number and a matrixA, the product Ais the matrix obtained bymultiplying each element inAby.

6 Example 47 1 23 82 9 = 7 1421 5614 63 > 7 * A[,1] [,2] [,3][1,] 7 14 56[2,] 21 14 Transpose of matricesA matrix is transposed by interchanging rows and columns and is denoted by > .Example 5 1 23 82 9 >=[1 3 22 8 9] Note that ifAis anr cmatrix thenA>is ac rmatrix.> t(A)[,1] [,2][1,] 1 3[2,] 2 2[3,] 8 Sum of matricesLetAandBber cmatrices. The sumA+Bis ther cmatrix obtainedby matrices with the same dimensions can be 6 1 23 82 9 + 5 48 23 7 = 6 611 105 16 > B <- matrix(c(5, 8, 3, 4, 2, 7), ncol = 3, byrow = T)> A + B[,1] [,2] [,3][1,] 6 10 11[2,] 7 4 Multiplication of a matrix and a vectorLetAbe anr cmatrix and letbbe ac-dimensional column vector. TheproductAbis ther 1 matrixAb= a11a12.. a1ca21a22.

7 Arc = a11b1+a12b2+ +a1cbca21b1+a22b2+ + +ar2b2+ +arcbc Example 7 1 23 82 9 [58]= 1 5 + 2 83 5 + 8 82 5 + 9 8 = 217982 > A %*% a[,1][1,] 23[2,] 27 Note the difference to> A * a[,1] [,2] [,3][1,] 1 4 24[2,] 9 2 18 Figure out yourself what goes on! Multiplication of matricesLetAbe anr cmatrix andBac tmatrix, [b1:b2: :bt]. TheproductABis ther tmatrix given by:AB=A[b1:b2: :bt] = [Ab1:Ab2: :Abt]Example 8[1 23 82 9][5 48 2]= 1 23 82 9 [58]: 1 23 82 9 [42] = 1 5 + 2 81 4 + 2 23 5 + 8 83 4 + 8 22 5 + 9 82 4 + 9 2 = 21 879 2882 26 Note that the productABcan only be formed if the number of rows inBandthe number of columns inAare the same. In that case,AandBare said tobe generalABandBAare not for matrix multiplicationis : 1 23 82 9 [5 48 2]=54821 21 5 + 2 8 1 4 + 2 23 83 5 + 8 8 3 4 + 8 22 92 5 + 9 8 2 4 + 9 2= 21 879 2882 26 > A <- matrix(c(1, 3, 2, 2, 8, 9), ncol = 2)> B <- matrix(c(5, 8, 4, 2), ncol = 2)> A %*% B[,1] [,2][1,] 21 8[2,] 79 28[3,] 82 Vectors as matricesOne can regard a column vector of lengthras anr 1 matrix and a rowvector of lengthcas a 1 Some special matrices Ann nmatrix is asquare matrix A matrixAissymmetricifA=A>.

8 A matrix with 0 on all entries is the0 matrixand is often written simplyas A matrix consisting of 1s in all entries is of writtenJ. A square matrix with 0 on all off diagonal entries and elementsd1, d2, .. , dnon the diagonal adiagonal matrixand is often writtendiag{d1, d2, .. , dn} A diagonal matrix with 1s on the diagonal is called theidentity matrixand is denotedI. The identity matrix satisfies thatIA=AI=A. 0-matrix and 1-matrix> matrix(0, nrow = 2, ncol = 3)[,1] [,2] [,3][1,] 0 0 0[2,] 0 0 0> matrix(1, nrow = 2, ncol = 3)[,1] [,2] [,3][1,] 1 1 1[2,] 1 1 1 Diagonal matrix and identity matrix> diag(c(1, 2, 3))[,1] [,2] [,3][1,] 1 0 0[2,] 0 2 0[3,] 0 0 3> diag(1, 3)[,1] [,2] [,3][1,] 1 0 0[2,] 0 1 0[3,] 0 0 1 Note what happens whendiagis applied to a matrix:> diag(diag(c(1, 2, 3)))[1] 1 2 3> diag(A)[1] 1 Inverse of matricesIn general, the inverse of ann nmatrixAis the matrixB(which is alson n) which when multiplied withAgives the identity matrixI.

9 That is,AB=BA= says thatBisA s inverse and writesB=A 1. Likewise,AisBs 9 LetA=[1 32 4]B=[ 2 ]NowAB=BA=IsoB=A 1. Example 10 IfAis a 1 1 matrix, a number, for exampleA= 4, thenA 1= 1/4. Some facts about inverse matrices are: Only square matrices can have an inverse, but not all square matriceshave an inverse. When the inverse exists, it is unique. Finding the inverse of a large matrixAis numerically complicated (butcomputers do it for us).In Section??the issue of matrix inversion is discussed in more the inverse of a matrix in R is done using thesolve()function:> A <- matrix(c(1, 3, 2, 4), ncol = 2, byrow = T)> A[,1] [,2][1,] 1 3[2,] 2 4> B <- solve(A)> B[,1] [,2][1,] -2 [2,] 1 > A %*% B[,1] [,2][1,] 1 0[2,] 0 Solving systems of Linear equationsExample 11 Matrices are closely related to systems of Linear equations.

10 Con-sider the two equationsx1+ 3x2= 72x1+ 4x2= 10 The system can be written in matrix form[1 32 4][x1x2]=[710] 1A=Iand sinceIx=xwe havex=A 1b=[ 2 ][710]=[12]A geometrical approach to solving these equations is as follows: Isolatex2inthe equations:x2=73 13x1x2=104 24x1 These two lines are shown in Figure 5 from which it can be seen that thesolution isx1= 1, x2= 2. 10123 10123x12 Figure 5: Solving two equations with two the Figure it follows that there are 3 possible cases of solutions to one solution when the lines intersect in one solutions when the lines are parallel but not many solutions when the lines coincide. > A <- matrix(c(1, 2, 3, 4), ncol = 2)> b <- c(7, 10)> x <- solve(A) %*% b> x[,1][1,] 1[2,] Some additional rules for matrix operationsFor matricesA, BandCwhose dimension match appropriately: the following rulesapply(A+B)>=A>+B>(AB)>=B>A>A(B+C) =AB+ACAB=AC6 B=CIn generealAB6=BAAI=IA=AIf is a number then AB=A( B) Details on inverse matrices* Inverse of a2 2matrix*It is easy find the inverse for a 2 2 matrix.