Singular Value Decomposition & Independent …

1 HST582 ,2005 IntroductionInthischapterwewillexamineho wwecangeneralizetheideaoftransforminga timeseriesinanalternativerepresentation, suchastheFourier(frequency)domain,tofaci li-tatesystematicmethodsofeitherremoving ( ltering)oradding(interpolating) , wewillexaminethetechniquesofPrincipalCom ponentAnalysis(PCA)usingSingularValueDec omposition(SVD),andIndependentComponentA nalysis(ICA).Bothofthesetechniquesutiliz ea representationofthedataina statisticaldomainratherthana ,thedatais projectedontoa newsetofaxesthatful llsomestatisticalcriterion,whichimplyind ependence,ratherthana thattheFouriercomponentsontowhicha datasegmentis projectedare xed, If thestructureofthedatachangesovertime,the ntheaxesontowhichthedatais essentiallya methodforseparatingthedataoutintoseparat esourceswhichwillhopefullyallowustoseeim portantstructureina ,bycalculatingthepowerspectrumofa segmentofdata, (amplitudesquared)

2 Alongcertainfrequencyvectorsis thereforehigh,meaningwehavea strongcomponentinthesignal1atthatfrequen cy. Bydiscardingtheprojectionsthatcorrespond totheunwantedsources(suchasthenoiseorart ifactsources)andinvertingthetransformati on,weeffectivelyper-forma trueforbothICA andPCA , oneimportantdifferencebetweenthesetechni quesis andICA weattemptto nda assumetherearea setofindependentsourcesin thedata,butdonotassumetheirexactproperti es.(Therefore,theymayoverlapinthefrequen cydomainincontrasttoFouriertechniques.)W e thende , ratherthande nethetheaxes,thisprocessis knownasblindsourcesepara-tion. themeasureweusetodiscovertheaxesisvarian ceandleadstoa setoforthogonalaxes(becausethedatais decorrelatedin a secondordersenseandthedotproductof anyof theaxesis zero).

3 ForICA thismeasureis basedonnon-Gaussianity(suchaskurtosis1- see ) thefourthmoment(mean,variance,andskewnes sarethe rstthree)andis a measureofhownon-Gaussiana thatif wemaximizethenon-Gaussianityofa setofsignals,thentheyaremaximallyindepen dent.(Thiscomesfromthecentrallimittheore m;if wekeepaddingindependentsignalstogether, wewilleventuallyarriveata Gaussiandistribution.)If webreaka Gaussian-likeobservationdownintoa setof non-Gaussianmixtures,eachwithdis-tributi onsthatareasnon-Gaussianaspossible, ,kurtosisallowsustoseparatenon-Gaussiani ndependentsources, ,if formulatedinthecorrectmanner, canleadtosomesurprisingresults,asyouwill discoverintheapplicationssectionlaterint hesenotesandintheaccompa-nyinglaboratory .

4 However, weshall noiseseparationIngeneral,anobserved(reco rded) datareductionstep( ltering)followedbya datareconstructiontechnique(suchasinterp olation).However, thesuccessofthedatareductionandreconstru ctionstepsis nition,noiseis thepartoftheobservationthatmaskstheunder lyingsignalwewish1a measureofhowwiderornarrowera distributionis thana Gaussian2toanalyze2, , fora noisesignaltocarrynoinformation,it mustbewhitewitha atspectrumandanautocorrelationfunction(A CF)equaltoanimpulse3. Mostrealnoiseis notreallywhite, ,thetermnoiseisoftenusedratherlooselyand is ,muscularactivityrecordedontheelectrocar diogram(ECG)is , increasedmuscleartifactontheECGactuallyt ellsusthatthesubjectis moreactivethanwhenlittleornomusclenoisei s thereforea sourceofinformationaboutactivity, althoughit nitionsarethereforetask-relatedandchange dependingonthenatureof illustratestherangeofsignalcontaminantsf ortheECG4.

5 We shallalsoexam-inethestatisticalqualities ofthesecontaminantsintermsoftheirprobabi litydistributionfunctions(PDFs)sincethep owerspectrumofa signalis notalwayssuf cienttocharac-terizea PDFcanbedescribedintermsofitsGaussianity , orrather,departuresfromthisidealizedform (whicharethereforecalledsuper-orsub-Gaus sian).ThefactthatthesesignalsarenotGauss ianturnsouttobeanextremelyimportantqual- ity, whichis closelyconnectedtotheconceptofindependen ce,whichweshallexploittoseparatecontamin antsformthesignalAlthoughnoiseis oftenmodeledasGaussianwhitenoise5, thisis oftencorrelated(withitselforsometimesthe signal), ,50 Hzor60 Hzmainsnoisecontaminationis sinusoidal,a waveformthatspendsmostofitstimeattheextr emevalues(nearitsturningpoints).

6 ByconsideringdeparturesfromtheidealGauss iannoisemodelwewillseehowconventionaltec hniquescanunder-performandhowmoresophist icated(statistical-based)techniquescanpr ovideimproved willnowexplorehowthisissimplyanotherform ofdatareduction(or ltering)throughprojectionontoa lterthedata(bydiscardingtheprojectionson toaxesthatarebelievedtocorrespondtonoise ).Byprojectingfroma reducedsetofbasisfunctions(ontowhichthed atahasbeencompressed)backtotheoriginalsp ace,weperforma typeofinterpolation(byaddinginformationf roma modelthatencodessomeofourpriorbeliefsabo uttheunderlyingnatureofthesignaloris derivedfroma dataset).

7 2it lowerstheSNR!3 Therefore,noone-steppredictionis descriptiveexamplebecauseit haseasilyrecognizable(andde nable) 3 Qualities FrequencyTimeContaminant Rangeduration50or60 HZPowerlineNarrowband50 Continuousor HzMovementBaselineNarrowbandTransientorW ander( Hz)ContinuousMuscleNoiseBroadbandTransie ntElectricalInterferenceNarrowbandTransi entorContinuousElectrodepopNarrowbandTra nsientObservationnoiseBroadbandContinuou sQuantizationnoiseBroadbandContinuousTab le1 :10secondsof3 rsttwosecondsandthe (S) ltersThesimplest lteringofa timeseriesinvolvesthetransformationofa discreteonedimen-sional( ) timeseries , consistingof pointssuchthat ,intoa newrepresentation,!

8 " # %$& '$( ) '$( *$( + . If , -. / 0 1 2 is a columnvec-tor6thatrepresentsa channelofECG,thenwecangeneralizethisrepr esentationsothat channelsofECG3, andtheirtransformedrepresentation4aregiv enby35 67778 9 + 9 % : : : 9 <; + : : : =;.. > ? @ : : :A B;CEDDDF 4G 67778$& + $& % : : :$ $( $( + : : :$( =;..$/ > H$/ @ : : :I$/ B;CEDDDF(1)Notethatwewilladopttheconvent ionthroughoutthischapter(andintheaccompa nyinglaboratoryexercises)thatallvectorsa rewrittenin lower-caseboldandarecolumnvectors, pointsofeachofthe signalchannelsform JK matrices( -dimensionalwith samplesforeachvector).An( JL ) transformationmatrixMcanthenbeappliedto3 tocreatethetransformedmatrix4suchthat4 KNK OM 3PN (2)Thepurposeofa transformationis tomap(orproject) lterthedatawediscardthenoise,or`unintere sting'partsofthesignal(whicharemaskingth einformationweareinterestedin).)))))

9 Thisamountstoa dimensionalityreduction,aswearediscardin gthedimensions(orsubspace) , ,thetransformedsignalis samelength( ) astheoriginalandtheenergyofthedatais theDiscreteFouriertrans-form(DFT)whereth esamesignalis measuredalonga newsetofperpendicularaxescorrespondingto thecoef cientsoftheFourierseries(seechapter4).In thecaseoftheDFTwithQR S frequencyvectors, as4 UTV XW;Y Z M TY3 YwhereM TY O[]\]^ _`TY a;, orequivalentlyM 67778[\]^ _[\]^cb_: : :[\]^ _ ;[]\]^=b_[]\]^ed_: : :[]\]^=b_ ;..[]\]^ _` []\]^=b_ : : :X[]\]^ _ B;CDDDF (3)Forbiorthogonaltransforms,theanglesbe tweentheaxesmaychangeandthenewaxesarenot necessarilyperpendicular.

10 However, noinformationis lostandperfectreconstructionoftheorigina lsignalis stillpossible(using3 OM\gf4 ).6 Inh jilk nmthecommandophUq`r`s]t uwv`x <y givesa dimensionofz#{}|anda lengthequalto~ (sothatthetransformationcanbereversedand theoriginaldatarestoredexactly)oraslossy . Whena signalis lteredorcompressed(throughdownsamplingfo rinstance),informationis oftenlostandthetransformationis ,lossytransformationsinvolvea non-invertibletransformationofthedatausi nga transformationmatrixthathasatleastonecol umnsetto anirreversibleremovalof someof thedataandthiscorrespondstoa mappingtoa (PCA)andIndependentComponentAnalysis(ICA ).