Coding Facial Expressions with Gabor W avelets

1 Codedusinga multi-orientation,multi-resolutionsetofG aborfilterswhichare is significantdegreeofpsychologicalplausibi lity, a designfeature IntroductionIncontrastwithcurrenthuman-c omputerinteraction,face-to-facehumancomm unicationusesa varietyofmodes, a :relativedisplacementsoffeatures(opening themouth);quasi-texturalchangesintheskin sur-face(furrowingthebrow);changesinskin hue(blushing); basedontopographicallyordered, multi-resolution, usedintheautomaticfacerecognitionsystemd evelopedbythevonderMalsburg group[7].Previousworkonautomaticfacialex pressionprocess-ingincludesstudiesusingr epresentationsbasedon:opticalflowestimat ionfromimagesequences[10, 15, 1]; principalcomponentsanalysisofsingleimage s[2, 1]; andphysically-basedmodels[5].

2 Is possibleto builda automaticfacialex-pressionrecognitionsys tembasedona Gaborwaveletcodewhichhasa supportedbytherecentworkofZhangetal.[16] demonstratingexpressionclassificationusi ngGaborcodinganda GaborcodingoffacialexpressionsTo extractinformationaboutfacialexpression, each256by256pixelimage, , wasconvolvedwitha multiplespatialresolution,multipleorient ationsetofGaborfilters( ), and . Thesignsubscriptindicatesfiltersofevenan doddphase,while , thefilterwave-vector, determinesthespatialfrequencyandorientat iontuningofthefilter. Adescriptionofthecomplex-valuedtwodimens ionalGabortransformis givenbyDaugman[3].

3 Responsesofthefilterstotheimagewerecombi nedintoa vector, , withcompo-nentsgivenby: where, !"!$# % '&)( * *,+(-&/.!10&325476 98 ;: ) , $:# 0&/.! 5 <>= ?>@$ACB DFEHG$IKJML NFO B PRQTS;UVJMW>Q AYXZO[\BTA;ACB P N>Q>]TP'B P9[\Q[,^VQ_S`JTa =bJMOcB ITNTACB P'P3=dQ$]_=eLfJK?MB PgS`QTA [3hFACB BiQTS [3hTBXZO [1B ACPj@TP'BKklE c !"!# m '&)( * *+(-&1.!10& 63n-o p8 q: / 'rTheintegralofthecosineGaborfilter,# 0&/.!, wassub-tractedfromthefiltertorenderit tsmu! uvZ setathalftheNyquistsamplingfrequency, withfrequencylevelsspacedat octaves;" zywasusedin allcalculations,givinga filterbandwidthofaboutanoctave, ,withanglesequallyspacedat intervalsofu{from| , , aredefinedastheamplitudeofthecombinedeve nandoddfilterresponses ~} !)]]]]

4 ! . Theresponseamplitudeis lesssen-sitiveto studythesimilarityspaceofGaborcodedfacia lim-ages,responsesoffiltershavingthesame spatialfrequencyandorientationpreference werecomparedat correspondingpointsin neighbor-ingpixelsarehighlycorrelatedand redundant,it is sufficienttocalculatetheaverageona sparsegridcoveringtheface( ).Thissimilaritymeasureisusedintheautoma ticfacerecognitionsystemdevelopedbythevo nderMalsburggroup[7]. <$= ?>@$ACB mE h B M ] QmkFB ?>A`=dk @ P'B k ['Q ACB NKA-B P'BT] [S-JTa =dJ O$?FB Q$LfB [3AC E<$= ?>@$ACBg mE NFNMJ A-JK[3@TP @TP'BKk ['Q N$h Q ['Q ?>A`J N$h S`JKa =bJMOB ITNTACB P'P3=bQ$] P Eandregisteringa graphapproximatelywiththefeaturesofthefa cehavebeendemonstratedpreviously[7, 14].]]]

5 Inthisstudy, forhigherprecision,facialgraphswereposit ionedmanuallyonimagesofa or4 examplesofeachofthesixba-sicfacialexpres sions(happiness,sadness,surprise,anger,d isgust,fear)[4]anda neutralfacefora < <$= ?>@$ACB %E G$IKJ L NFO B P QTSR=eLfJK?MB PqS A-Q>L [3hTB S-JTa =dJ OFB ITNTACB P'P3=dQ$] k$JK[\JMWMJ P'B$Eselfwhilelookingthrougha boxen-closedtheregionbetweencameraandpla sticsheetto flatbedscanner. , , , (31subjects)rated108picturesonsixbasicfa cialexpressions(happiness,sadness,surpri se,anger, disgustandfear). (31subjects)ratedthecomplementarysetof11 1 pictures(outofthe219total) (includingfearim-ages).]

6 (15subjects)rated94picturesonfiveofthesi xbasicfacialexpressions(fearwasexcluded) . (15subjects)rateda differentsetof93imagesonthefivebasicfaci alexpressions(fearexcluded). 5 or 6 is conve-nientto HJMWFO B D$ER YJM]$ a7 QTA,ACBTObJ [5=dQ$] W$B [,^cB BT] LfQmkMB O JM] kP'BTLfJM] ['=da A-JK[5=e]T? P3= L =eObJ A`=b['=bB P Eperformanceasthisavoidstheproblemthatpo sedexpres-sionsarenotalwayspureexampleso fa ,asa ( )fromthepointatthenosetipformedthecompon entsofa ( ) (expressorTM)to (expres-sorNA) , (expressorUY)to (expressorKM) ( ) YJMWFO B mEt YJM]F a7Q A ACB O JK[5=bQ$] P W$B [ ^VB BT] L Q>kFBTOJM]Mk P'BTLfJM] ['=da A`J [5= ]T?

7 P3=eL = O J A = [5= B P S-Q AYB I N$BTA`= L B ]T[\P^ hM=da h B IKa Od@MkMB k SCBKJ AVP'[5=eL @FOd=$JM]Mk A-JK[5=e]T?FP (expressorTM) (expressorKA), , (ex-pressorUY)to (expressorKM) s s (nMDS)usingtheALSCAL algorithm[13].nMDSembedspointsina eu-clideanspacein sucha waythatthedistancesbetweenpointspreserve stherankorderofthedissimilarityvaluesbet weenthosepoints. Stress and Rsq ,it , 7 ,7,thefollowingabbreviationsareused:NE-N eutral,HA- Happiness,SA- Sadness,SU-Surprise,AN- Anger, DI- showssamplenMDSsolutionin whichimageshavebeenpositionedat , thepointshavebeenrotated,translated, ,7havenotbeentreatedinthisway.]

8 Highlysignificantdegreeofcor-relation, , perceptronclassifiera facialexpressionrecognitionhasbeenbuilt[ 16].Twosetsofexperimentswererun, consid-eredto bea , thelow-dimensionalspacesforratingsdataan dGabor-codearesimilar. Oneaxis( )correspondstothedegreeofpleasantness( ) roughlyorthogonaldimensioncorrespondstot helevelofarousalshownbytheface( ).Thisconfigurationwasseenforalloftheexp ressorsstudied(exceptNM,wherethedatais er-ratic). , higherdegreeofcorrela-tionwiththedatatha ndida , anexplicitandprecisefunctionoffacialde-f ormationduetoexpression, geometrymeasure,butatthepriceofgreatlyin creasedcomputationalcomplexity.

9 Locationofgridpointsis themostexpensivepartofa fullyautomaticsystem[7, 14]. MoreovertheGabormeasureputslessstrin-gen tdemandsontheprecisionofthegridpositioni ng, +Geometrysystemcouldhavestillhigherperfo rmance,howeverresultsof[16] <$=b?>@FACB %EY]m P5Q$Oe@T[5=bQ$] PRS`QTA9 UVJMW>QTAcJM] kgP'BTLfJM] ['=da A-JK[5=e]T? P3= L =eObJ A`=b['=bB Pp /kFJK['JfS,A-Q$L NF=eObQ [9P'[3@ kF T )E<>= ?>@$ACB mE ]m P5Q>Od@T[5=dQ$]TP S`QTA U J W>Q A9J ]Mk P'BTL J ] [5=ba A`J [5= ]T? P3=eL = O J A = [5= B Pf P @FW3 ,BKa [f V )E mB B [\B I [jS-Q A )B ['QB ITNTACB P'P3=bQ$] JMW$WKA-B K=dJK['=dQ$]TP E<>= ?]]]]

10 >@$ACB mE;]m P5Q$Od@ [5=bQ$]gP N JTa B PjS`QTAjUVJMW>QTA JM] k P'B LfJM]T[5=ba A-JK[5=e] ? P3= L =eObJ A`=b['=bB P ,P @FW5 /BKa [ q )E mB B [\B I [pS`QTA B [\QgB ITNTACB P'P3=bQ$] J W$WTACB T=bJK[5=bQ$] P ,suchchangesarebestrepresentedusingthesp atiallylocal-izedreceptivefieldstypicalo fprimaryvisualcortex(V1) [3, 6, 11]. Lyonset al.[8] , it is (beginningwithSchlosberg [12]) suggesta cir-cumplex conjecturethathigh-level(evensemanticlev el) grantfromtheAnnenberg [1] ,P. Viola,T. Sejnowski, , , & P. :Ad-vancesin NeuralInformationProcessingSystems8, ,MITP ress,Cambridge,MA,1996.]]