Chapter 4 FACE RECOGNITION AND ITS …

1 Chapter 4 FACE RECOGNITION AND ITS APPLICATIONSA ndrew W. Senior and Ruud M. BolleIBM Research Center, Box 704,Yorktown Heights,NY 10598, USA.{aws, RECOGNITION has long been a goal of computer vision, butonly in recent yearsreliable automated face RECOGNITION has become a realistictarget of biometricsresearch. New algorithms, and developments spurred by falling costs of camerasand by the increasing availability processing power have led to practical facerecognition systems. These systems are increasingly beingdeployed in a widerange of practical applications , and future improvements promise to spread theuse of face RECOGNITION further still. In this Chapter , we review the field of facerecognition, analysing its strengths and weaknesses and describe the applicationswhere the technology is currently being deployed and where it shows future po-tential.}

2 We describe the IBM face RECOGNITION system and some of its :Face RECOGNITION , robust faces is something that people usually do effortlessly and with-out much conscious thought, yet it has remained a difficult problem in thearea of computer vision, where some 20 years of research is just beginningto yield useful technological solutions. As a biometric technology, automatedface RECOGNITION has a number of desirable properties that are driving researchinto practical problem of face RECOGNITION can be stated as identifying an individualfrom images of the face and encompasses a number of variations other thanthe most familiar application of mug shot identification. One notable aspectof face RECOGNITION is the broad interdisciplinary nature of the interest in it:101102 Chapter 4within computer RECOGNITION and pattern RECOGNITION ; biometrics and security;multimedia processing; psychology and neuroscience.

3 It isa field of researchnotable for the necessity and the richness of interaction between computerscientists and automatic RECOGNITION of human faces spans a variety of different tech-nologies. At a highest level, the technologies are best distinguished by the inputmedium that is used, whether visible light, infra-red [29, 31] or 3-dimensionaldata [7] from stereo or other range-finding technologies. Thus far, the fieldhas concentrated on still, visible-light, photographic images, often black andwhite, though much interest is now beginning to be shown in the recognitionof faces in colour video. Each input medium that is used for face recognitionbrings robustness to certain conditions, face imaging is practicallyinvariant to lighting conditions while 3-dimensional datain theory is invariantto head pose.

4 Imaging in the visible light spectrum, however, will remain thepreeminent domain for research and application of face RECOGNITION because ofthe vast quantity of legacy data and the ubiquity and cheapness of photographiccapture as a BiometricFace RECOGNITION (see [6, 33] for recent surveys) has a number of strengthsto recommend it over other biometric modalities in certain circumstances, andcorresponding weaknesses that make it an inappropriate choice of biometricfor other applications . Face RECOGNITION as a biometric derives a number ofadvantages from being the primary biometric that humans useto recognizeone another. Some of the earliest identification tokens, , use thisbiometric as an authentication pattern. Furthermore it is well-accepted andeasily understood by people, and it is easy for a human operator to arbitratemachine decisions in fact face images are often used as a human-verifiablebackup to automated fingerprint RECOGNITION of its prevalence as an institutionalized and accepted guarantor ofidentity since the advent of photography, there are large legacy systems basedon face images such as police records, passports and driving licences thatare currently being automated.

5 Video indexing is another example of legacydata for which face RECOGNITION , in conjunction with speaker identification [19],is a valuable RECOGNITION has the advantage of ubiquity and of being universal overother major biometrics, in that everyone has a face and everyone readily displaysthe face. (Whereas, for instance, fingerprints are capturedwith much moredifficulty and a significant proportion of the population hasfingerprints that cannot be captured with quality sufficient for RECOGNITION .) Uniqueness, anotherdesirable characteristic for a biometric, is hard to claim at current levels ofAchievements and Challenges in Fingerprint Recognition103accuracy. Since face shape, especially when young, is heavily influenced bygenotype, identical twins are very hard to tell apart with this some configuration and co-ordination of one or more cameras, it is bemore or less possible to acquire face images without active participation of thesubject.

6 Such passive identification might be desirable forcustomization ofuser services and consumer devices, whether that be openinga house door asthe owner walks up to it, or adjusting mirrors and car seats tothe driver s presetswhen sitting down in their systems rely on passive acquisition by capturing the face im-age without the cooperation or knowledge of the person beingimaged. Facerecognition also has the advantage that the acquisition devices are cheap andare becoming a commodity (though this is not true for non-visible wavelengthdevices and some of the more sophisticated face recognitiontechnologies basedon 3-dimensional data).The main drawbacks to face RECOGNITION are its current relatively low accuracy(compared to the proven performance of fingerprint and iris RECOGNITION ) and therelative ease with which many systems can be defeated (Section ).

7 Finally,there are many attributes leading to the variability of images of a single facethat add to the complexity of the RECOGNITION problem if theycan not be avoidedby careful design of the capture situation. Inadequate constraint or handling ofsuch variability inevitably leads to failures in include:Physical changes:facial expression change; aging; personal appearance(make-up, glasses, facial hair, hairstyle, disguise).Acquisition geometry changes:change in scale, location and in-planerotation of the face (facing the camera) as well as rotation in depth (facingthe camera obliquely, or presentation of a profile, not full-frontal face).Imaging changes:lighting variation; camera variations; channel char-acteristics (especially in broadcast, or compressed images).Figure variations of a single face: in pose, facial appearance, age, lighting current system can claim to handle all of these problems well.

8 In particularthere has been little research on making face RECOGNITION robust to the effects of104 Chapter 4aging the faces . In general, constraints on the applicationscenario and capturesituation are used to limit the amount of invariance of face image sample thatneeds to be afforded main challenges of face RECOGNITION today are handling rotation in depthand broad lighting changes, together with personal appearance changes. Evenunder good conditions, however, accuracy needs to be and FraudAll biometric RECOGNITION systems are susceptible to accidental errors of twotypes which both must be minimized: False Accept (FA) errorswhere a randomimpostor is accepted as a legitimate users and False Reject (FR) errors wherea legitimate user is denied access. Designers of biometric systems must alsobe very conscious of how the system will behave when deliberately much of biometric system design falls into the more traditional cate-gories of physical, procedural and electronic security preventing an attackerfrom circumventing the RECOGNITION system or preventing false enrollment ofbiometric identities into a system s database, for example.

9 That is, purposefuland successful attempts at creating a false accept error by general means ofsecurity attacks. Nevertheless, there are a number of security attack types thatare specific to is very easy to change one s facial appearance to make one look verydifferent, and so to prevent identification, a false rejection. This isparticularly important in a non-cooperative application where the biometricis being used to prevent a single person from obtaining a privilege (such as avote or driving licence) more than once. While underlying bone structure isextremely difficult to change, it is also hard to measure, andall face recognitionsystems rely on more superficial, changeable characteristics (Section )making them defeasible for determined is also possible for some people to impersonate others with a high degree ofsimilarity (an important vulnerability in cooperative applications like physicalaccess control).

10 Photographs, rubber masks, video replay all allow impostorattacks the deliberate engineering of a false acceptance error. Detection ofsuch fake biometrics data is only superficially handled by commercial systems,though this is improving. A couple of years ago, few systems had a test todetect authenticity (rejecting objects that looked too flatto be faces rather thanphotographs), but a recent PC Magazine test [21] found that both systems testedcould distinguish a real person from a photograph. More sophisticated shapealgorithms could be devised, and elastic deformation can beused to preventsimple photograph replay attacks. (One system allows the option of requiring achange in facial expression during verification.) With computing power moreabundant, the technology for detecting fake biometrics will keep and Challenges in Fingerprint Recognition105 The combination with other biometrics particularly lip motion verificationor speaker ID [23] reduces the exposure to impersonation attacks, but furthermeasures are necessary to prevent video replay attacks where a pre-recordedsequence of the authorized individual is somehow injected into the established in speaker identification literature [2],prompted-text or text-independent verification can avoid a simple replay attack, at the cost of a moreintrusive, complex and expensive system, but the advances in trainable speechand face synthesis algorithms [11, 15]