Transcription of The formula for the orthogonal projection
1 The formula for the orthogonal projectionLetVbe a subspace ofRn. To find the matrix of the orthogonal projection ontoV, the way wefirst discussed, takes three steps:(1) Find a basis~v1,~v2, .. ,~vmforV.(2) Turn the basis~viinto an orthonormal basis~ui, using the Gram-Schmidt algorithm.(3) Your answer isP= ~ui~uTi. Note that this is ann nmatrix, we are multiplying a columnvector by a row vector instead of the other way is often better to combine steps (2) and (3). (Note that you still need to find a basis!) Hereis the result: LetAbe the matrix with columns~vi. ThenP=A(ATA) 1 ATYour textbook states this formula without proof in Section , so I thought I d write up urge you to also understand the other ways of dealing with orthogonal projec-tion that our book discusses, and not simply memorize the the span of the vectors (1 2 3 4)Tand (5 6 7 8)T.
2 These two vectors are linearlyindependent (since they are not proportional), soA= 1 52 63 74 8 .ThenATA=(307070 174)(ATA) 1=(87160 732 732332).Note thatATandAare not square, but the productATAis, so (ATA) 1makes we haveA(ATA) 1AT= 1 52 63 74 8 (87160 732 732332)(1 2 3 45 6 7 8)= 71025110 1525310151101101531025 1511025710 You can have fun checking that this is, indeed, the matrix of orthogonal projection isATAinvertible?This formula only makes sense ifATAis invertible. So let s prove that it is. Suppose, for thesake of contradiction, that~c= (c1, c2, , cm) is a nonzero vector in the kernel ofATA. ThenATA~c= 0 and so0 =~cTATA~c= (A~c)T(A~c) =|A~c| the only vector with length 0 is~0, this shows thatA~c= ~c=c1~v1+c2~v2+ +cm~vmand we assumed that~v1,~v2.
3 ,~vmwas a basis forV, sothere are no linear relations between the~vi. So we can t havec1~v1+c2~v2+ +cm~vm=~0. Thisis our contradiction and we deduce thatATAdidn t have a kernel after proofLet s first check thatP ~wis the right thing in two particular 1:~wis inV. In this case,~w=A~cfor some~c. ThenP ~w=A(ATA) 1AT(A~c) =A(ATA) 1(ATA)~c=A~c=~wwhich is what we 2:~wis inV . SinceV= Image(A), we haveV = Image(A) = Ker(AT) and we seethatAT~w= 0. ThenP ~w=A(ATA) 1AT~w=A(ATA) 1~0 =~0which is, again, what we a general~w, write~w=~w1+~w2where~w1is inVand~w2is inV . ThenP ~w=P ~w1+P ~w2=~w1+~0 =~w1where we used Case 1 to computeP ~w1and Case 2 to computeP ~w2.
4 Taking~wto~w1is, indeed,exactly what orthogonal projection is suppose to using orthogonal basesOur vectors~v1,~v2, .. ,~vmare a basis forV, but not an orthonormal basis. However,Vdoeshave an orthonormal basis. Let~u1,~u2, .. ,~umbe this basis. Let s writeQfor the matrix whosecolumns are the~ui. The condition that~uiare orthonormal is the same asQTQ= Idm. Let~vj= iRij~ui. We can rewrite this using matrices,A= claim thatRis invertible. Notice thatRis square (it sm m). IfR~x= 0, thenA~x=QR~x= 0. But the columns ofAare linearly independent, soAis injective, a contradiction. SinceRank(R) = Rank(RT), this also shows thatRTis we plug in:P=A(ATA) 1AT= (QR)((QR)TQR) 1(QR)T=QR(RTQTQR) 1 RTQT=QR(RTR) 1 RTQT becauseQTQ= Id=QRR 1(RT) 1 RTQTwe already argued thatRandRTare invertible=QQTAs your textbook explains (Theorem ), when the columns ofQare an orthonormal basisofV, thenQQTis the matrix of orthogonal projection that we needed to argue thatRandRTwere invertible before using the formula (RTR) 1=R 1(RT) 1.
5 By contrast,AandATare not invertible (they re not even square) so it doesn t makesense to write (ATA) 1=A 1(AT) 1.
