Complex inner products (6.7 supplement) u 6= 0 and

1 Complex inner products ( supplement) The definition of inner product given in section of Lay is not useful for Complex vector spacesbecause no nonzero Complex vector space has such an inner product. If it did, pick any vectoru6=0andthen 0< u,u . But also0< iu, iu =i u, iu =i2 u,u = u,u <0which is a contradiction. Following is an altered definition which will work for Complex vector spaces. Recallthat ifz=a+bithenz=a biis the Complex conjugate ofz. We alter the definition of inner product bytaking Complex conjugate inner producton a Complex vector spaceVis a function that, to each pair ofvectorsuandvinV, associates a Complex number u,v and satisfies the following axioms, for allu,v,winVand all scalarsc:1. u,v = v,u .2. u+v,w = u,w + v,w and u,v+w = u,v + u,w .3. cu,v =c u,v and u, cv =c u,v .14. u,u is a nonnegative real number and u,u = 0 if and only ifu= vector spaceCnhas a standard inner product, u,v =u v.

2 Recallu =uTso another formula is u,v =uTv. So for example [1 +i,2 i],[3 2i,1 +i] = (1 i)(3 2i) + (2 +i)(1 +i) = 1 5i+ 1 + 3i= 2 can compute this in Matlab asdot([1+i, 2-i], [3-2i, 1+i]).Another example is the Complex vector spaceCC[a, b] of Complex valued continuous functions withdomain [a, b]. So any vector inCC[a, b] is of the formf(t) +ig(t) wherefandgare inC[a, b]. Examplesaret2+t3iande5ti= cos(5t) + sin(5t)i. We can define a Hermitian inner product onCC[a, b] by u, v = bau(t)v(t)dt. So for example inCC[0,2 ] ekti, e`ti = 2 0ektie`tidt= 2 0e(` k)tidt=e(` k)ti(` k)i]2 0= 0if`6=k. (Note thateai=e ai.)Just as for inner products , the length of a vectoruis defined as||u||= u,u . The angle betweentwo vectorsvanduis defined by||u|| ||v||cos =<( u,v ), where<( u,v ) denotes the real part of theHermitian inner product. For example, to find the angle between [3i]Tand [2 +i1 i]TinC2, we have:||[3i]T||= 3 3 + ( i) i= 10||[2 +i1 i]T||= (2 i) (2 +i) + (1 +i) (1 i) = 22+ 12+ 12+ 12= 7 [3i]T,[2 +i1 i]T = 3 (2 +i) + ( i) (1 i) = 6 + 3i i 1 = 5 + 2i = cos 1(5/( 10 7)).

3 93 radiansI promised to prove Cauchy-Schwarz for a Hermitian inner product for anyone interested. Here +biis a Complex number we define|z|= zz= a2+b2. First, we show that| v,u | 1 for allunit vectorsvandu. To see this suppose,vanduare unit vectors, so||v||= 1 and||u||= 1. Then for anycomplex numbertwe have:0 ||u tv||2= u tv,u tv = u,u + u, tv + tv,u + tv, tv 1 There is some disagreement on this axiom. Some authors say cu,v =c u,v and u, cv =c u,v .We choose our form somewhat arbitrarily. But, for example, it is the definition used by Matlab where theHermitian inner product is calculated bydot(u,v).1= u,u t u,v t v,u +tt v,v = 1 t u,v t v,u +ttLett= v,u . Then by property 1 of the Hermitian inner product, we know thatt= u,v . Then0 1 t u,v t v,u +tt= 1 tt= 1 |t|2So 1 |t|=| v,u |as we desired. Now supposevanduare any vectors. Thenv/||v||andu/||u||are unitvectors, so we just showed that1 | v/||v||,u/||u|| |=| v,u |/(||v|| ||u||)Consequently,||v|| ||u|| | v,u |.

4 Hermitian and Unitary matrices (supplement to )There is a Complex version of orthogonal matrices. A Complex square matrixUis calledunitaryifU =U 1. Equivalently, the columns ofUform an orthonormal set (using the standard Hermitian innerproduct onCn). Any orthogonal matrix is , there is a Complex version of symmetric matrices. A Complex square matrixAis calledHermitianifA =A. What this boils down to is that the diagonal entries ofAare all real, and the abovediagonal entries are the conjugates of the corresponding below diagonal entries. Any symmetric matrix isHermitian. The spectral theorem applies to Hermitian matrices and indeed it is most easily proven forHermitian Lay does not provide a proof of the spectral theorem I will sketch a proof :V Vis a linear transformation of a nonzero finite dimensional Complex vector spaceVthenThas an :By taking coordinates with respect to a basis ofV, we may as well suppose thatV=CnandTismultiplication by a Complex matrixA.

5 Alson >0 sinceVis nonzero. Then the characteristic polynomialofAhas at least one root, soAhas at least one eigenvector, which corresponds to an eigenvector product of two unitary matrices is :SupposeQandSare unitary, soQ 1=Q andS 1=S . Then (QS) =S Q =S 1Q 1=(QS) 1soQSis unitaryTheorem 3.(Schur Lemma) IfAis any square Complex matrix then there is an upper triangular complexmatrixUand a unitary matrixSso thatA=SU S =SU S :Letq1be an eigenvector ofA, which we may suppose has unit length. By the Gram-Schmidt processwe may chooseq iso that{q1, q 2, .. , q n}is an orthonormal basis. LetQ0= [q1q 2 q n], thenQ0is unitaryandQ 0AQ0=[ 1 0A1]for some (n 1) (n 1) matrixA1. Likewise, we may find a unitary (n 1) (n 1)matrixQ1so thatQ 1A1Q1=[ 2 0A2]. Then ifS1=Q0[1 00Q1]we haveS 1AS1= 1 0 2 0 0A2 .Note thatS1is unitary by Theorem 2. Now continue in this fashion, lettingSk=Sk 1[Ik00Qk], and wesee thatU=S nASnis upper triangular.

6 LettingS=Snwe see thatA=SU S .We are now prepared to prove the Spectral theorem. Note that ifAis a real symmetric matrix then itis Hermitian so the following result implies Theorem 3 on page 452 of 4.(Spectral theorem) IfAis ann nHermitian matrix then:a)Ahasnreal eigenvalues, counting ) The dimension of the eigenspace for each eigenvalue equals the multiplicity of as a root of thecharacteristic polynomial ) The eigenspaces are mutually orthogonal, in the sense that eigenvectors corresponding to differenteigenvalues are ) There is a unitary matrixSand a real diagonal matrixDso thatA=SDS =SDS ) IfAis real then we may chooseSin d) to be real (orthogonal).2 Proof:By Theorem 3 there is a unitary matrixSand an upper triangularUso thatA=SU S . ButA=A soU=S AS=S A S= (S AS) =U but thenUmust be diagonal sinceUandU are both upper triangular.

7 Moreover, if is a diagonal entry ofU, then is the corresponding entry ofU which means = , which means is real. So settingD=Uwehave shown d). SinceSdiagonalizesAwe know that the diagonal entries ofDare the eigenvalues ofAandthe columns ofSare the eigenvectors. So we have shown a) and b) (see theorem 5 page 320 and Theorem7 page 324). Note c) follows since the columns ofSare orthogonal. IfAis real we might need to choosea newSsince theSwe constructed in Theorem 3 might not be real, unless we chose it very carefully. Foreach eigenvalue jofA, choose an orthonormal (real) basisBjof the eigenspace corresponding to j. Letthe columns ofSbe all these basis vectors inB1, .. ,Bk. By b) this will makencolumns. ThenS ASisdiagonal and by part c),Sis will now give an alternative proof which does not use the Schur Lemma. As above, letBjbe anorthonormal basis of the jeigenspace.

8 By Theorem 1, page 450 we know thatB1,B2, .. ,Bkform anorthonormal set{q1, .. , q`}. Complete it to an orthonormal basis{q1, .. , qn}ofCn. Note that if`=nweare done since we have an orthonormal basis of eigenvectors, so suppose that` < n. LetWbe the span of{q`+1, .. , qn}. Note that vectors inWare perpendicular to all eigenspaces ofA. Ifi > `andj `then Aqi, qj = (Aqi) qj=q iA qj=q iAqj=q i qj= qi, qj = 0consequentlyAqiis perpendicular to allqjforj `which meansAqiis inW. Consequently,A:W by Theorem 1,Ahas an eigenvector inW. But this contradicts the fact that all vectors inWareperpendicular to all eigenspaces ofW, a nonzero vector can t be perpendicular to itself. So we conclude thatin fact`=nandW={0}so we are we characterize which matrices can be diagonalized by a unitary matrix. We say a matrixAisnormalifAA =A matrixAis diagonalizable with a unitary matrix if and only ifAis normal.

9 In other words:a) IfAis normal there is a unitary matrixSso thatS ASis ) If there is a unitary matrixSso thatS ASis diagonal thenAis :SupposeAis normal. By Theorem 3 there is a unitary matrixSand an upper triangularUso thatA=SU S . ThenU U =S AS(S AS) =S ASS A S=S AA S=S A AS=S A SS AS=U UBut if we letuijdenote theij-th entry ofUthen the upper left entry ofU Uisu11u11=|u11|2but theupper left entry ofU U isu11u11+u12u12+ +u1nu1n=|u11|2+|u12|2+ +|u1n|2 Since this equals|u11|2and all summands are nonnegative real numbers we must haveu12=u13= =u1n= 0. Similarly, looking at the second diagonal entry we see thatu2j= 0 for allj >2. Continuing inthis way we see thatUmust be diagonal. So we have shown that ifAis normal, then it is diagonalizablewith a unitary suppose thatAis any matrix so that there is a unitary matrixSso thatS AS=Dis =D D. ThenAA =SDS (SDS ) =SDS SD S =SDD S =SD DS =SD S SDS =A AConsequently,Ais of normal matrices are Hermitian matrices (A=A ), skew Hermitian matrices (A= A )and unitary matrices (A =A 1) so all such matrices are