Transcription of 17. Inner product spaces - MIT Mathematics
1 product spacesDefinition a real vector space . Aninner productonVis a function , :V V R,which is symmetric, that is u,v = v,u . bilinear, that is linear (in both factors): u,v = u,v ,for all scalars and u1+u2,v = u1,v + u2,v ,for all vectorsu1,u2andv. positivethat is v,v 0. non-degeneratethat is if u,v = 0for everyv Vthenu= say thatVis areal Inner product space . Theassociatedquadratic formis the functionQ:V R,defined byQ(v) = v,v .Example Mn,n(R)be a real matrix. We can define afunction , :Rn Rn R,by the rule u,v = basic rules of matrix multiplication imply that this function is bi-linear.
2 Note that the entries ofAare given byaij=etiAej= ei,ej .In particular, it is symmetric if and only ifAissymmetricthat isAt=A. It is non-degenerate if and only ifAis invertible, that isAhas rankn. Positivity is a little harder to the canonical example is to takeA=In. In this case ifu= (r1,r2,..,rn)andv= (s1,s2,..,sn)thenutInv= risi. Note1that if we takeu=vthen we get r2i. The square root of this is theEuclidean a real vector space . AnormonVis afunction . :V R,what has the following properties kv =|k| v ,for all vectorsvand scalarsk.
3 Positivethat is v 0. non-degeneratethat is if v = 0thenv= 0. satisfies thetriangle inequality, that is u+v u + v .Lemma a real Inner product . :V R,defined by v = v,v ,is a norm is clear that the norm satisfies the first property and that itis positive. Suppose thatu V. By assumption there is a vectorvsuch that u,v 6= u+tv,u+tv = u,u + 2t u,v +t2 v,v .If v,v = 0 then certainly u,u >0. Otherwise putt= u,v v,v > u,u + 2t u,v +t2 v,v = u,u +t u,v .2 Once again u,u >( u,v )2 v,v > the norm is suppose thatuandv V. Then u+v,u+v = u,u + 2 u,v + v,v u 2+ 2 u v + v 2= ( u + v ) square roots gives the triangle inequality.
4 Note that one can recover the Inner product from the norm, usingthe formula2 u,v =Q(u+v) Q(u) Q(v),whereQis the associated quadratic form. Note the annoying ap-pearence of the factor of also that on the way we proved:Lemma (Cauchy-Schwarz-Bunjakowski).LetVbe a real innerproduct Vthen u,v u v .Definition a real vector space with an Inner say that two vectorsvandwareorthogonalif u,v = say that a basisv1,v2,..,vnis anorthogonal basisif the vectorsv1,v2,..,vnare pairwise orthogonal. If in addition the vectorsvihavelength one, we say thatv1,v2,..,vnis anorthonormal a real Inner product space .
5 (1)If the vectorsv1,v2,..,vmare pairwise orthogonal then theyare independent. In particular ifm= dimVthenv1,v2,..,vmare an orthogonal basis ofV.(2)Ifv1,v2,..,vnare an orthonormal basis ofVandv Vthenv= rivi,whereri= v,vi . first prove (1). Suppose thatr1v1+r2v2+ +rmvm= the Inner product of both sides withvjgives0 = r1v1+r2v2+ +rmvm,vj =m i=1ri vi,vj =rj vj,vj .As vj,vj 6= 0,it follows thatrj= 0. This is (1).The proof of (2) is similar. So how does one find an orthonormal basis?Algorithm (Gram-Schmidt).Letv1,v2,..,vnbe independentvectors in a real Inner product spaceV.
6 (1)Let1 k nbe the largest index such thatv1,v2,..,vkareorthonormal.(2)Ifk=mth enstop.(3)Otherwise letuk+1=vk+1 r1v1 r2v2 rmvm,whereri= vk+1,vi . Replacevk+1byuk+1 uk+1 ,and return to (1).In practice the algorithm works as follows. First we replacev1byv1 v1 ,so thatv1has unit length. Then we considerv2. We have to subtractsome ofv1to ensure that it is orthogonal tov1. So consider a vectorof the formu=v2+ v1,where is chosen to makeuorthogonal tov1. We have0 = u,v1 = v2,v1 + v1,v1 ,so that = v2,v1 .4 Then we rescale to get a vector of unit length. At the next stage, wecan choose and so thatv3+ v1+ v2,is orthogonalv1andv2.
7 The key thing is that sincev1andv2areorthogonal, our choice of and are independent of each example, consider the vectorsv1= (1, 1,1), v2= (1,0,1)andv3= (1,1,2),inR3with the usual Inner product . The first step is to replacev1byv1=1 3(1, 1,1).Now letu= (1,0,1) 23(1, 1,1)=13(1,2,1).Then we replaceuby a vector parallel touof unit lengthv2=1 6(1,2,1).Finally we putu= (1,1,2) 23(1, 1,1) 56(1,2,1)=12( 1,0,1).Finally we replaceuby a vector of unit length,v3=1 2( 1,0,1).Thusv1=1 3(1, 1,1)v2=1 6(1,2,1)andv3=1 2( 1,0,1),is the orthonormal basis produced by very useful property of Inner products is that we get canonicallydefined complimentary linear subspaces:Lemma a finite dimensional real Inner product Vis a linear subspace, then letU ={w V| w,u = 0, u U},the set of all vectors orthogonal to every element ofU.
8 Then5 U is a linear subspace ofV. U U ={0}. UandU particularVis isomorphic toU U . first prove (1). Suppose thatw1andw2 U . Picku w1+w2,u = w1,u + w2,u = 0 + 0 = follows thatw1+w2 U and soU is closed under addition. Nowsuppose thatw U and is a scalar. Then w,u = w,u = 0 = w U and soU is closed under scalar multiplication. ThusU is a linear subspace ofV. This is (1).Suppose thatw U U . Then w,w = thenw= 0. This is (2).Suppose thatv V. Ifv Uthere is nothing to prove. Otherwisepick an orthonormal basisu1,u2,..,ukofU. Then the vectorsv,u1,u2,..,ukare independent.
9 By Gram-Schmidt we may find scalarsr1,r2,..,rksuch thatw=v riui,is orthogonal tou1,u2,..,uk(in factri= v,ui ). But thenwisorthogonal toU, that isw U . Letu= riui U. Thenv=u+ is (3). 6
