Introduction to Matrix Algebra - Statpower

1 4 Introduction to MatrixAlgebraIn the previous chapter, we learned the algebraic results that form the founda-tion for the study of factor analysis and structural equation modeling. Theseresults, powerful as they are, are somewhat cumbersome to apply in morecomplicated systems involving large numbers of variables. Matrix algebraprovides us with a new mathematical notation that is ideally suited for de-veloping results involving linear combinations and transformations. Once wehave developed a few basic results and learned how to use them, we will beready to derive the fundamental equations of factor analysis and BASIC TERMINOLOGYD efinition ( Matrix )Amatrixis defined as an ordered array of numbers, ofdimensionsp, standard notation for a matrixAof orderp,qwill be:pAqThere are numerous other notations. For example, one might indicate amatrix of orderp,qasA(p q).

2 Frequently, we shall refer to such a matrixas ap qmatrixA. 5354 Introduction TO Matrix ALGEBRAOn occasion, we shall refer explicitly to theelementsof a Matrix ( , thenumbers or random variables in the array). In this case, we use the followingnotation to indicate that Ais a Matrix with elementsaij .A={aij}When we refer to elementaij,thefirstsubscript will refer to therowpositionof the elements in the array. Thesecondsubscript (regardless ofwhich letter is used in this position) will refer to the column position. Hence,a typical matrixpAqwill be of the form:A= a11a12a13 a1qa21a22a23 a2qa31a32a33 apq We shall generally use bold capital letters to indicate matrices, and employlower case letters to signify elements of an array, except where clarity particular, we may find it convenient, when referring to matrices of randomvariables, to refer to the elements with capital letters, so we can distinguishnotationally between constants and random (Column Vector)Acolumn vectorof numbers or randomvariables will be a Matrix of orderp 1.

3 We will, in general, indicate column vectorswith the following notation:px1 Definition (Row Vector)Arow vectorof numbers or random variableswill be a Matrix of order 1 q. We will, in general, indicate row vectors with thefollowing notation:1x qA column vector with all elements equal to one will be symbolized as how we reserve the use of boldface for matrices and Special MatricesWe will refer occasionally to special types of matrices by name. For anypAq,1. Ifp =q,Ais Ifp=q,Ais TERMINOLOGY553. In asquarematrix, the elementsaii,i=1,pdefine thediagonalof A square Matrix islower triangularifaij=0fori< A square Matrix isupper triangularifaij=0fori> A square Matrix is adiagonal matrixifaij=0fori =j7. A square Matrix is ascalar matrixif it is a diagonal Matrix and alldiagonal elements are Anidentity matrixis a scalar Matrix with diagonal elements equal toone.

4 We use the notationIpto denote ap pidentity , a Matrix composed entirely of zeros, is called anull A square matrixAissymmetricifaij=ajifor alli, A 1 1 Matrix is (Special Matrices)Some examples follow:1. A rectangular Matrix 12345678 2. A square Matrix 1234 3. A lower triangular Matrix 100 0230 0456 078910 4. An upper triangular Matrix 123 4056 7008 900010 5. A diagonal Matrix 100020007 56 Introduction TO Matrix ALGEBRA6. A scalar Matrix 200020002 7. A symmetric Matrix 123224342 Partitioning of MatricesIn many theoretical discussions of matrices, it will be useful to conceive of amatrix as being composed of sub-matrices. When we do this, we will parti-tion the Matrix symbolically by breaking it down into its components. Thecomponents can be either matrices or scalars. Here is a simple (A Simple Partitioned Matrix )In discussions of simplemultiple regression, where there is one criterion variableYandppredictor variablesthat can be placed in a vectorx, it is common to refer to the correlation Matrix ofthe entire set of variables using partitioned notation, as follows:R= 1r YxrxYRxx In the above notation, the scalar 1 refers to the correlation of theYvariablewith itself.

5 The row vectorr Yxrefers to the set of correlations between thevariableYand the set ofprandom variables thep pmatrixof correlations of the predictor variables. We will refer to the order of the partitioned form as the number of rows and columns in the partitioning,which is distinct from the number of rows and columns in the Matrix beingrepresented. For example, suppose there werep= 5 predictor variables inExample Then the matrixRis a 6 6 Matrix , but the example shows a 2 2 partitioned form. When matrices are partitionedproperly, it is understood that pieces thatappear to the left or right of other pieces have the same number of rows, andpieces that appear above or below other pieces have the same number ofcolumns. So, in the above example,Rxx, appearing to the right of thep 1column vectorrxY, must haveprows, and since it appears below the 1 prow vectorr Yx,itmusthavepcolumns.

6 Hence, it must be ap Matrix SOME Matrix OPERATIONSIn this section, we review the fundamental operations on Matrix (and Vector) Addition and SubtractionFor the addition and subtraction operations to be defined for two matricesA,B, they must (Conformability for Addition and Subtraction)Twomatrices areconformable for addition and subtractionif and only if they are of thesame ( Matrix Addition)LetA={aij}andB={bij}.LetAandBbe conformable. The sumA+B=Cis defined as:C=A+B={cij}={aij+bij}Definition ( Matrix Subtraction)LetA={aij}andB={bij}.LetAand Bbe conformable. The sumA B=Cis defined as:C=A B={cij}={aij bij} addition and subtraction are natural, intuitive exten-sions to scalar addition and subtraction. One simply adds elements in thesame ( Matrix Addition)LetA= 145234440 ,andB= 321231132 .FindC=A+BandD=A 466465572 D= 22 400 331 2 Definition ( Matrix Equality)Two matrices are equal if and only if theyare of the same row and column order, and have all elements addition has some important mathematical properties, which, for-tunately, mimic those of scalar addition and subtraction.

7 Consequently, thereis little negative transfer involved in generalizing from the scalar to thematrix TO Matrix ALGEBRAR esult (Properties of Matrix Addition)For matricesA,B,andC,properties include:1. AssociativityA+(B+C)=(A+B)+C2. CommutativityA+B=B+A3. There exists a neutral element for addition, , the null matrix0,such thatA+0= There exist inverse elements for addition, in the sense that for any ma-trixA, there exists a Matrix A, such thatA+ A= Scalar Multiples and Scalar ProductsIn the previous section, we examined some Matrix operations, addition andsubtraction, that operate very much like their scalar algebraic this section, we begin to see a divergence between Matrix Algebra and (Scalar Multiplication)Given a matrixA={aij}, {caij}is called ascalar multiples are not to be confused withscalar products,which will be defined subsequently.

8 Scalar multiplication is a simple idea multiply a Matrix by a scalar, and you simply multiply every element of thematrix by the (Scalar Multiple)LetA= 2 134 .Then2A= 4 268 .Result (Properties of Scalar Multiplication)For matricesAandB, and scalarsaandb, scalar multiplication has the following mathemati-cal properties:1.(a+b)A=aA+ (A+B)=aA+ (bA)=(ab) Matrix OPERATIONS59 Definition (Scalar Product)Given row vector1a ={ai}andb={bi}.Thescalar producta bis defined asa b= iaibiNote:This is simply the sum of cross products of the elements of the two (Scalar Product)Leta = 123 .Letb= 421 .Thena b= Matrix MultiplicationMatrix multiplication is an operation with properties quite different from itsscalar counterpart. To begin with,order mattersin Matrix is, the Matrix productABneed not be the same as the Matrix productBA.

9 Indeed, the Matrix productABmight be well-defined, while the productBAmight not exist. To begin with, we establish the fundamental (Conformability for Matrix Multiplication)pAqandrBsareconformable for Matrix multiplicationif and only ifq= Matrix multiplication operation is defined as ( Matrix Multiplication)LetpAq={aij}andqBs={bjk}. ThenpCs=AB={cik}where:cik=q j=1aijbjkExample (The Row by Column Method)The meaning of the formaldefinition of Matrix multiplication might not be obvious at first glance. Indeed,there are several ways of thinking about Matrix multiplication. The first way, whichIcalltherow by column approach, works as follows. VisualizepAqasasetofprowvectorsandqBsas a set ofscolumn ,elementcikofCis the scalar product ( , the sum of cross products) of theith row ofAwith thekth column ofB. For example, letA= 246571235 , and letB= 410251.

10 60 Introduction TO Matrix ALGEBRAThenC=AB= 38 1625 2033 13 . Consider elementc21, which has a value of 25. Thiselement, which is in the second row and first column ofC, is computed by takingthe sum of cross products of the elements of the second row ofAwith the firstcolumn ,(5 4) + (7 0) + (1 5) = (Properties of Matrix Multiplication)The following are somekey properties of Matrix multiplication:1. Associativity.(AB)C=A(BC)2. Not generally commutative. That is, oftenAB = Distributive over addition and (A+B)=CA+CB4. Assuming it is conformable, the identity matrixIfunctions like the num-ber 1, that ispAqIq=A,andpIpAq= not necessarily imply that eitherA=0orB= of the above results are surprising, and result in negative transferfor beginning students as they attempt to reduce Matrix Algebra (A Null Matrix Product)The following example shows thatone can, indeed, obtain a null Matrix as the product of two non-null matrices.