Chapter 5 Symmetric and Hermitian Matrices

1 Notes2 2013/4/9page 73iiiiChapter 5 Symmetric and HermitianMatricesIn this Chapter , we discuss the special classes of Symmetric and Hermitian will conclude the Chapter with a few words about so-called we begin, we mention one consequence of the last Chapter that will beuseful in a proof of the unitary diagonalization of Hermitian anm nmatrix withm n, and assume (for the moment) thatAhas linearly independent columns. Then if the Gram-Schmidt processis appliedto the columns ofA, the result can be expressed in terms of a matrix factorizationA= Q R, where the orthogonal vectors are the columns in Q, and Ris unit uppertriangularn nwith the inner products as the entries in example, consider thatqi:=ai i 1k=1qHkaiqHkqkqk. Rearranging the equa-tion,ai=qi+i 1 k=1qHkaiqHkqkqk+n k=i+10qkwhere the right hand side is a linear combination of the columns of the firsti(only!)columns of the matrix Q. So this equation can be rewrittenai= Q qH1aiqH1q1qH2aiqH2q2.

2 QHi 1aiqHi 1qi .Putting such equations together fori= 1, .. , n, we arrive at the desired result,A= Q notes2 2013/4/9page 74iiii74 Chapter 5. Symmetric and Hermitian MatricesNow, wereallywould prefer that Qhave orthonormal columns. That meanscolumn scalingeach column of Qby 1 divided by its length in the 2-norm. Thiscorresponds topostmultiplicationof Qby a diagonal matrix Dthat contains1/ qi ,A= Q R= Q D D 1 R=QR, where nowQhas orthonormal columnsandRis still upper triangular, but it s rows have been scaled by the entriesin thediagonal matrix D factorizationA=QRis called theQRfactorization ofA. Although weassumed in the preceding for ease of discussion thatAhad linearly independentcolumns, in fact such a factorization exists for any matrixA, the fine details Diagonalization of Hermitian MatricesDefinition matrix is said to beHermitianifAH=A, where theHsuper-script means Hermitian ( conjugate) transpose. Some texts may use an asteriskfor conjugate transpose, that is,A means the same asA.

3 IfAis Hermitian , itmeans thataij= ajifor everyi, jpair. Thus, the diagonal of a Hermitian matrixmust be matrix is said to besymmetricifAT= ,ifAis real, thenAH=AT, so a real-valued Hermitian matrixis Symmetric . However, ifAhas complex entries, Symmetric and Hermitian havedifferent meanings. There is such a thing as a complex- Symmetric matrix (aij=aji)- a complex Symmetric matrix need not have real diagonal are a few Matrices :A=(2 11 4);A= 0 2 4 2 7 54 5 8 ;A=(2 i3 + 4i3 + 4i8 7i). Hermitian Matrices :A=(68 + 4i8 4i9);A= 1 2 + 3i8 2 3i46 7i86 + 7i5 ;A=(3 55 8).As the examples show, the set of allreal Symmetric matricesis included withinthe set of all Hermitian Matrices , since in the case thatAis real-valued,AH= the other hand, one example illustrates that complex-symmetricmatrices arenot thatAis Hermitian . Then all the eigenvalues ofAarereal. notes2 2013/4/9page Diagonalization of Hermitian Suppose thatAx= xfor ( ,x) an eigenpair ofA.

4 Multiply both sidesof the eigen-equation byxH. (Recall that by definition of an eigenvector,x6=0.)Then we havexHAx=xH x= xHxxHAHx= x 22(Ax)Hx= x 22( x)Hx= x 22 x 2= x 22 = and the last equality implies that must be thatAis diagonalizable (overCn) only whenAhas a set ofnlin-early independent eigenvectors. We will show that Hermitian Matrices are alwaysdiagonalizable, and that furthermore, that the eigenvectors have a very special Hermitian , then any two eigenvectors from different eigenspacesare orthogonal in the standard inner-product forCn(Rn, ifAis real Symmetric ). ,v2be two eigenvectors that belong to two distinct eigenvalues,say 1, 2, respectively. We need to show thatvH1v2= 0. Since this is true iffvH1( 2v2) = 2(vH1)v2= 0, let us start there:vH1( 2v2) =vH1(Av2)=vH1 AHv2= (Av1)Hv2= ( 1v1)Hv2= 1vH1v2= 1vH1v2,where the last equality follows using the previous theorem. It followsthat ( 2 1)vH1v2= 0. But we assumed that 26= 1, it must be the casevH1v2= 0, real, square matrixAis said to beorthogonally diagonaliz-ableif there exists an orthogonal matrixQand diagonal matrixDsuch thatA= matrixA Cn nis calledunitarily diagonalizableif there notes2 2013/4/9page 76iiii76 Chapter 5.

5 Symmetric and Hermitian Matricesexists a unitary matrixUand diagonal matrixDsuch thatA= (Spectral Theorem).LetAbe Hermitian . ThenAis Jordan decompositionA=WJW 1. SinceWis square, wecan factor (see beginning of this Chapter )W=QRwhereQis unitary andRisupper triangular. Thus,A=QRJR 1QH=QTQH whereTis upper triangular because it is the product of upper triangular matrices13,andQis similarto an upper triangular matrixT, and we maypre-multiply byQHand post-multiply byQto obtainQHAQ= the conjugate transpose of both sides,QHAHQ=THHowever,A=AHand so we getT=TH. ButTwas upper triangular, and thiscan only happen ifTis diagonal. ThusA=QDQHas summary, ifAisn nHermitian, it has the following properties: Ahasnreal eigenvalues, counting multiplicities. The algebraic and geometric mulitplicites of each distincteigenvalue match. The eigenspaces are mutually orthogonal in teh sense that eigenvectors corre-spoinidng to different eignevalues are orthogonal. Ais unitarily be orthogonally diagonalizable real Matrices .

6 A)ShowAandBare Symmetric b) Show that ifAB=BA, thenABis mentioned elsewhere in the text, it is straightforward toshow the product of upper trian-gular Matrices is upper triangular. It is likewise straightforward to show that the inverse of anupper triangular matrix is upper triangular, so the expressionRJR 1is the product of 3 uppertriangular Matrices and is upper triangular14 This is in fact the Schur factorization. notes2 2013/4/9page Diagonalization of Hermitian Spectral DecompositionDefinition set of eigenvalues of a matrixAis sometimes called thespectrumofA. Thespectral radiusis the largest magnitude eigenvalue know that ifAis Hermitian ,A=QDQH, so let us write triple matrixproduct out explictly:A=(q1, .. ,qn) 10 00 0 n = [ 1q1, , nqn =n i=1 iqiqHiThis expression forAis called thespectral decompositionofA. Note thateachqiqHiis arank-onematrix AND that eachqiqHiis an orthogonal projectionmatrix onto Span(qi). Positive Definite, Negative Definitie, IndefiniteDefinition a real Symmetric matrix.]

7 We say thatAis alsopositive definiteif for everynon-zero x Rn,xTAx> similar result holds for Hermitian matricesDefinition a complex Hermitian matrix. We say thatAis alsopositive definiteif for everynon-zero x CN,xHAx> useful consequence of HPD (SPD) Matrices is that their eigenvalues (whichwe already know are real due to the Hermitian property) must be , HPD (SPD) Matrices MUST BE INVERTIBLE!Theorem Hermitian ( Symmetric ) matrix with all positive eigenvaluesmust be positive from the previous section we knowA=QDQH exists, from what weare given, the entries ofDmust be positive. Letx6= 0. ThenxHAx=xHQDQHx notes2 2013/4/9page 78iiii78 Chapter 5. Symmetric and Hermitian Matrices = (QHx)HD(QHx)=zHDz=n i=1 i|zi| the i>0 andz:=QHxcannot be zero (why?), the result (4 11 2). Compute the eigenvalues, observe they areboth positive. By the previous theorem, this matrix is HPD. Show<q,z>:=zHAqdefines a valid inner close cousin is thepositive Hermitian ( Symmetric ) matrix is semi-definite if for everynon-zerox Cn(x Rn),xHAx also have the concept of negative-definite Hermitian , then it isnegative definiteif for everynon-zerox Cn,xHAx< negative definite Hermitian ( Symmetric ) matrix must have all strictly neg-ative eigenvalues.

8 So it, too, is Symmetric ( Hermitian )indefintematrix is one that has some positive andsome negative (and possibly zero) Quadratic FormsA motivating quote from David Lay s Third Ed., Linear Algebra and Its Applica-tions:quadratic forms, occur frequently in applications of linear algebra toengineering (in design criteria and optimization) and signal processing(as output noise power). They also arise, for example, in physics (aspotential and kinetic energy) differential geometry (as normal curvatureof surfaces), economics (as utility functions), and statistics (in confidenceellipsoids). notes2 2013/4/9page Quadratic Forms79In fact, you saw a quadratic form already in the definitions of the formonRn(Cn) is a functionQ:Rn R(Q:Cn R) defined as follows:Q(x) =xTAx(Q(x) =xHAx)whereAis ann nsymmetric matrix ( Hermitian matrix). Here,Ais called thematrix of the quadratic A=(5 00 4). Compute the quadratic 5x21+ A=(5 1 1 4). If we compute the quadratic form here, thereare cross terms due to the presence of non-zero off diagonals:xTAx= 5x21 2x1x2+ the 2nd example, it s difficult to see whether or not this quadratic term willalways give something that s positive.

9 However, there is an easy wayto investigatethe possibility (well, easy provided someone has handed you the eigendecomposi-tion!):Theorem ann nHermitian ( Symmetric ). Then there is a unitary(orthogonal) change of variable of the formx=Qythat transforms the quadraticformxHAx(xTAx) into a quadratic formyHDy(xHDx) where the latter has nocross product Geometry and Principal AxesInR2, we can get some geometric intuition of the quadratic form for a symmetricAby looking at its level (x) =xTAxas a map considerWc={x R2|xTAx=c}for a fixed, real, constantc. The setWcis called thec-level setof the quadratic functionf(x). One of the followingwill occur: The c-level set will be an ellipse (or circle, if both semi-axes have the samelength) The c-level set will be a hyperbola The c-level set will be one or two lines, or a single point, or contain no pointsat all notes2 2013/4/9page 80iiii80 Chapter 5. Symmetric and Hermitian MatricesTo see this, we start withAbeing a 2 2 real, diagonal matrix.

10 ThenxTAx=a11x21+a22x22. Let s assume thatAis invertible (no 0 s on the diagonal). Thena11x21+a22x22=c a11cx21+a22cx22= 1.( )Consider first the case thata11>0, a22>0. Note that in this caseAissymmetric positive definite. Then ifc >0,a11c=1 2for some positive anda22c=1 2for some positive . For example, ifc= 1, =1 a11. Thus the rightmostequation in ( ) is in fact the equation for anellipse centered at the origin, with being the length of the semi-axis oriented on thex1axis, and be being the lengthof the semi-axis oriented in the verticalx2component. On the other hand, ifc= 0anda11, a22are both positive, there are no solutions to the equation; the -2level set would contain no points at all for negative definite,a11<0, a22<0. If you look at a level set for whichc <0, the same analysis as above goes through - the picture will be an , without loss of generality, assume thata11>0 buta22<0. This meansthatAisindefinite. Looking back at ( ), it can be rewrittenx21 2 x22 2= 1, >0, >0for as above and 2=c|a22|.