MATH 304 Linear Algebra

1 MATH 304 Linear AlgebraLecture 20:Inner product notion ofnormgeneralizes the notion of lengthof a vector a vector space. A function :V Ris called anormonVif it has thefollowing properties:(i) (x) 0, (x) = 0 only forx=0(positivity)(ii) (rx) =|r| (x) for allr R(homogeneity)(iii) (x+y) (x) + (y) (triangle inequality) norm of a vectorx Vis usuallydenotedkxk. Different norms onVaredistinguished by subscripts, , V=Rn,x= (x1,x2, .. ,xn) Rn. kxk = max(|x1|,|x2|, .. ,|xn|). kxkp=(|x1|p+|x2|p+ +|xn|p)1/p,p V=C[a,b],f: [a,b] R. kfk = maxa x b|f(x)|. kfkp=( ba|f(x)|pdx)1/p,p vector vector spaceis a vectorspace endowed with a norm defines a distance function on the normedvector space:dist(x,y) =kx we say that a sequencex1,x2, ..convergesto a vectorxifdist(x,xn) 0 asn .Also, we say that a vectorxis a goodapproximationof a vectorx0ifdist(x,x0) is productThe notion ofinner productgeneralizes the notionof dot product of vectors a vector space.

2 A function :V V R, usually denoted (x,y) =hx,yi,is called aninner productonVif it is positive,symmetric, and bilinear. That is, if(i)hx,xi 0,hx,xi= 0 only forx=0(positivity)(ii)hx,yi=hy,xi(symmet ry)(iii)hrx,yi=rhx,yi(homogeneity)(iv)hx +y,zi=hx,zi+hy,zi(distributive law)Aninner product spaceis a vector space endowedwith an inner V=Rn. hx,yi=x y=x1y1+x2y2+ +xnyn. hx,yi=d1x1y1+d2x2y2+ +dnxnyn,whered1,d2, .. ,dn>0. hx,yi= (Dx) (Dy),whereDis an invertiblen an inner product onR2such thathe1,e1i= 2,he2,e2i= 3, andhe1,e2i= 1,wheree1= (1,0),e2= (0,1).Letx= (x1,x2),y= (y1,y2) +x2e2,y=y1e1+ bilinearity, we obtainhx,yi=hx1e1+x2e2,y1e1+y2e2i=x1he1, y1e1+y2e2i+x2he2,y1e1+y2e2i=x1y1he1,e1i+ x1y2he1,e2i+x2y1he2,e1i+x2y2he2,e2i= 2x1y1 x1y2 x2y1+ remains to check thathx,xi>0 forx6= ,xi= 2x21 2x1x2+ 3x22= (x1 x2)2+x21+ V=Mm,n(R), space ofm nmatrices. hA,Bi=trace(ABT).IfA= (aij) andB= (bij), thenhA,Bi=m i=1n j= V=C[a,b].

3 Hf,gi= baf(x)g(x)dx. hf,gi= baf(x)g(x)w(x)dx,wherewis bounded , piecewise continuous, andw>0 everywhere on [a,b].wis called ,yiis an inner product on avector spaceV. Thenhx,yi2 hx,xihy,yifor allx,y :For anyt Rletvt=x+ty. Thenhvt,vti=hx,xi+ 2thx,yi+t2hy, right-hand side is a quadratic polynomial int(provided thaty6=0). Sincehvt,vti 0 for allt,the discriminantDis nonpositive. ButD= 4hx,yi2 4hx,xihy, Inequality:|hx,yi| hx,xi hy, Inequality:|hx,yi| hx,xi hy, 1|x y| |x||y|for allx,y , for allxi,yi R,(x1y1+ +xnyn)2 (x21+ +x2n)(y21+ +y2n).Corollary 2 For anyf,g C[a,b],( baf(x)g(x)dx)2 ba|f(x)|2dx ba|g(x)| induced by inner productsTheoremSupposehx,yiis an inner product on avector spaceV. Thenkxk= hx,xiis a :Positivity is obvious. Homogeneity:krxk= hrx,rxi= r2hx,xi=|r| hx, inequality (follows from Cauchy-Schwarz s):kx+yk2=hx+y,x+yi=hx,xi+hx,yi+hy,xi+hy ,yi hx,xi+|hx,yi|+|hy,xi|+hy,yi kxk2+ 2kxkkyk+kyk2= (kxk+kyk) The length of a vector inRn,|x|= x21+x22+ +x2n,is the norm induced by the dot productx y=x1y1+x2y2+ +xnyn.

4 The normkfk2=( ba|f(x)|2dx)1/2on thevector spaceC[a,b] is induced by the inner producthf,gi= baf(x)g(x) |hx,yi| kxkkyk, we can define theanglebetween nonzero vectors in any vector space withan inner product (and induced norm): (x,y) = arccoshx, ,yi=kxkkykcos (x,y).In particular, vectorsxandyareorthogonal(denotedx y) ifhx,yi= +yyPythagorean Law:x y= kx+yk2=kxk2+kyk2 Proof:kx+yk2=hx+y,x+yi=hx,xi+hx,yi+hy,xi +hy,yi=hx,xi+hy,yi=kxk2+ +yx yyyParallelogram Identity:kx+yk2+kx yk2= 2kxk2+ 2kyk2 Proof:kx+yk2=hx+y,x+yi=hx,xi+hx,yi+hy,xi +hy, ,kx yk2=hx,xi hx,yi hy,xi+hy, +yk2+kx yk2= 2hx,xi+ 2hy,yi= 2kxk2+ setsLetVbe an inner product space with an innerproducth , iand the induced normk nonempty setS Vof nonzerovectors is called anorthogonal setif all vectors inSare mutually orthogonal. That is,0/ Sandhx,yi= 0 for anyx,y S,x6= orthogonal setS Vis calledorthonormalifkxk= 1 for anyx ,v2, .. ,vk Vform anorthonormal set if and only ifhvi,vji={1 ifi=j0 ifi6=jExamples.}

5 V=Rn,hx,yi=x standard basise1= (1,0,0, .. ,0),e2= (0,1,0, .. ,0), .. ,en= (0,0,0, .. ,1).It is an orthonormal set. V=R3,hx,yi=x (3,5,4),v2= (3, 5,4),v3= (4,0, 3).v1 v2= 0,v1 v3= 0,v2 v3= 0,v1 v1= 50,v2 v2= 50,v3 v3= the set{v1,v2,v3}is orthogonal but notorthonormal. An orthonormal set is formed bynormalized vectorsw1=v1kv1k,w2=v2kv2k,w3=v3kv3k. V=C[ , ],hf,gi= f(x)g(x) (x) = sinx,f2(x) = sin 2x, .. ,fn(x) = sinnx, ..hfm,fni= sin(mx) sin(nx)dx={ ifm=n0 ifm6=nThus the set{f1,f2,f3, ..}is orthogonal but is orthonormal with respect to a scaled innerproducthhf,gii=1 f(x)g(x) Linear independenceTheoremSupposev1,v2, .. ,vkare nonzerovectors that form an orthogonal set. Thenv1,v2, .. ,vkare linearly :Supposet1v1+t2v2+ +tkvk=0for somet1,t2, .. ,tk for any index 1 i kwe haveht1v1+t2v2+ +tkvk,vii=h0,vii= t1hv1,vii+t2hv2,vii+ +tkhvk,vii= 0By orthogonality,tihvi,vii= 0 = ti= basesLetv1,v2.}

6 ,vnbe an orthonormal basis for aninner product +x2v2+ +xnvnandy=y1v1+y2v2+ +ynvn, wherexi,yj R. Then(i)hx,yi=x1y1+x2y2+ +xnyn,(ii)kxk= x21+x22+ + :(ii) follows from (i) wheny= ,yi= n i=1xivi,n j=1yjvj =n i=1xi vi,n j=1yjvj =n i=1n j=1xiyjhvi,vji=n i= projectionTheoremLetVbe an inner product space andV0be a finite-dimensional subspace ofV. Then anyvectorx Vis uniquely represented asx=p+o,wherep V0ando componentpis theorthogonal projectionofthe vectorxonto the subspaceV0. We havekok=kx pk= minv V0kx is, the distance fromxto the an inner product space. Letpbe theorthogonal projection of a vectorx Vonto afinite-dimensional a one-dimensional subspace spanned by avectorvthenp=hx,vihv, ,v2, .. ,vnis an orthogonal basis forV0thenp=hx,v1ihv1,v1iv1+hx,v2ihv2,v2i v2+ +hx,vnihvn, ,hp,vii=n j=1hx,vjihvj,vjihvj,vii=hx,viihvi,viihvi ,vii=hx,vii= hx p,vii= 0 = x p vi= x p V0.