Chapter 4 FACE RECOGNITION AND ITS APPLICATIONS

1 Chapter 4 FACE RECOGNITION AND ITS APPLICATIONSA ndrew W. Senior and Ruud M. BolleIBM Research Center, Box 704,Yorktown Heights,NY 10598, USA.{aws, RECOGNITION has long been a goal of computer vision, butonly in recent yearsreliable automated face RECOGNITION has become a realistictarget of biometricsresearch. New algorithms, and developments spurred by falling costs of camerasand by the increasing availability processing power have led to practical facerecognition systems. These systems are increasingly beingdeployed in a widerange of practical APPLICATIONS , and future improvements promise to spread theuse of face RECOGNITION further still.}

2 In this Chapter , we review the field of facerecognition, analysing its strengths and weaknesses and describe the applicationswhere the technology is currently being deployed and where it shows future po-tential. We describe the IBM face RECOGNITION system and some of its :Face RECOGNITION , robust faces is something that people usually do effortlessly and with-out much conscious thought, yet it has remained a difficult problem in thearea of computer vision, where some 20 years of research is just beginningto yield useful technological solutions.

3 As a biometric technology, automatedface RECOGNITION has a number of desirable properties that are driving researchinto practical problem of face RECOGNITION can be stated as identifying an individualfrom images of the face and encompasses a number of variations other thanthe most familiar application of mug shot identification. One notable aspectof face RECOGNITION is the broad interdisciplinary nature of the interest in it:101102 Chapter 4within computer RECOGNITION and pattern RECOGNITION ; biometrics and security;multimedia processing; psychology and neuroscience.

4 It isa field of researchnotable for the necessity and the richness of interaction between computerscientists and automatic RECOGNITION of human faces spans a variety of different tech-nologies. At a highest level, the technologies are best distinguished by the inputmedium that is used, whether visible light, infra-red [29, 31] or 3-dimensionaldata [7] from stereo or other range-finding technologies. Thus far, the fieldhas concentrated on still, visible-light, photographic images, often black andwhite, though much interest is now beginning to be shown in the recognitionof faces in colour video.

5 Each input medium that is used for face recognitionbrings robustness to certain conditions, face imaging is practicallyinvariant to lighting conditions while 3-dimensional datain theory is invariantto head pose. Imaging in the visible light spectrum, however, will remain thepreeminent domain for research and application of face RECOGNITION because ofthe vast quantity of legacy data and the ubiquity and cheapness of photographiccapture as a BiometricFace RECOGNITION (see [6, 33] for recent surveys) has a number of strengthsto recommend it over other biometric modalities in certain circumstances, andcorresponding weaknesses that make it an inappropriate choice of biometricfor other APPLICATIONS .

6 Face RECOGNITION as a biometric derives a number ofadvantages from being the primary biometric that humans useto recognizeone another. Some of the earliest identification tokens, , use thisbiometric as an authentication pattern. Furthermore it is well-accepted andeasily understood by people, and it is easy for a human operator to arbitratemachine decisions in fact face images are often used as a human-verifiablebackup to automated fingerprint RECOGNITION of its prevalence as an institutionalized and accepted guarantor ofidentity since the advent of photography, there are large legacy systems basedon face images such as police records, passports and driving licences thatare currently being automated.

7 Video indexing is another example of legacydata for which face RECOGNITION , in conjunction with speaker identification [19],is a valuable RECOGNITION has the advantage of ubiquity and of being universal overother major biometrics, in that everyone has a face and everyone readily displaysthe face. (Whereas, for instance, fingerprints are capturedwith much moredifficulty and a significant proportion of the population hasfingerprints that cannot be captured with quality sufficient for RECOGNITION .) Uniqueness, anotherdesirable characteristic for a biometric, is hard to claim at current levels ofAchievements and Challenges in Fingerprint Recognition103accuracy.

8 Since face shape, especially when young, is heavily influenced bygenotype, identical twins are very hard to tell apart with this some configuration and co-ordination of one or more cameras, it is bemore or less possible to acquire face images without active participation of thesubject. Such passive identification might be desirable forcustomization ofuser services and consumer devices, whether that be openinga house door asthe owner walks up to it, or adjusting mirrors and car seats tothe driver s presetswhen sitting down in their systems rely on passive acquisition by capturing the face im-age without the cooperation or knowledge of the person beingimaged.

9 Facerecognition also has the advantage that the acquisition devices are cheap andare becoming a commodity (though this is not true for non-visible wavelengthdevices and some of the more sophisticated face recognitiontechnologies basedon 3-dimensional data).The main drawbacks to face RECOGNITION are its current relatively low accuracy(compared to the proven performance of fingerprint and iris RECOGNITION ) and therelative ease with which many systems can be defeated (Section ). Finally,there are many attributes leading to the variability of images of a single facethat add to the complexity of the RECOGNITION problem if theycan not be avoidedby careful design of the capture situation.

10 Inadequate constraint or handling ofsuch variability inevitably leads to failures in include:Physical changes:facial expression change; aging; personal appearance(make-up, glasses, facial hair, hairstyle, disguise).Acquisition geometry changes:change in scale, location and in-planerotation of the face (facing the camera) as well as rotation in depth (facingthe camera obliquely, or presentation of a profile, not full-frontal face).Imaging changes:lighting variation; camera variations; channel char-acteristics (especially in broadcast, or compressed images).