COVER FEATURE An Introduction to Evaluating Biometric …

1 0018-9162/00/$ 2000 IEEE56 ComputerAn Introduction toEvaluating Biometric SystemsOn the basis of media hype alone, you mightconclude that Biometric passwords will soonreplace their alphanumeric counterpartswith versions that cannot be stolen, forgot-ten, lost, or given to another person. Butwhat if the performance estimates of these systems arefar more impressive than their actual performance?To measure the real-life performance of biometricsystems and to understand their strengths and weak-nesses better we must understand the elements thatcomprise an ideal Biometric system.

2 In an ideal system all members of the population possess the char-acteristic that the Biometric identifies, like irisesor fingerprints; each Biometric signature differs from all others inthe controlled population; the Biometric signatures don t vary under the con-ditions in which they are collected; and the system resists evaluation quantifies how well bio-metric systems accommodate these , Biometric evaluations require that an inde-pendent party design the evaluation, collect the testdata, administer the test, and analyze the designed this article to provide you with suffi-cient information to know what questions to ask whenevaluating a Biometric system, and to assist you indetermining if performance levels meet the require-ments of your application.

3 For example, if you plan touse a Biometric to reduce as opposed to eliminate fraud, then a low-performance Biometric system maybe sufficient. On the other hand, completely replacingan existing security system with a Biometric -based onemay require a high-performance Biometric system, orthe required performance may be beyond what cur-rent technology can we focus on Biometric applications that givethe user some control over data acquisition. Theseapplications recognize subjects from mug shots, pass-port photos, and scanned fingerprints. Examples notcovered include recognition from surveillance photosor from latent fingerprints left at a crime the biometrics that meet these constraints, voice,face, and fingerprint systems have undergone the moststudy and testing and therefore occupy the bulk ofour discussion.

4 While iris recognition has receivedmuch attention in the media lately, few independentevaluations of its effectiveness have been STATISTICST here are two kinds of Biometric systems: identifi-cationand identification systems, a Biometric signature ofan unknown person is presented to a system. The sys-tem compares the new Biometric signature with a data-base of Biometric signatures of known individuals. Onthe basis of the comparison, the system reports (or esti-mates) the identity of the unknown person from thisdatabase. Systems that rely on identification includethose that the police use to identify people from fin-gerprints and mug shots.

5 Civilian applications includethose that check for multiple applications by the sameperson for welfare benefits and driver s verification systems, a user presents a biometricsignature and a claim that a particular identity belongsto the Biometric signature. The algorithm either acceptsHow and where Biometric systems are deployed will depend on their performance. Knowing what to ask and how to decipher the answers canhelp you evaluate the performance of these emerging JonathonPhillipsAlvin WilsonMarkPrzybockiNationalInstitute ofStandards andTechnologyCOVER FEATUREor rejects the claim.

6 Alternatively, the algorithm canreturn a confidence measurement of the claim s valid-ity. Verification applications include those that authen-ticate identity during point-of-sale transactions or thatcontrol access to computers or secure statistics for verification applicationsdiffer substantially from those for identification sys-tems. The main performance measure for identificationsystems is the system s ability to identify a biometricsignature s owner. More specifically, the performancemeasure equals the percentage of queries in which thecorrect answer can be found in the top few example, law enforcement officers often use anelectronic mug book to identify a suspect.

7 The inputto an electronic mug book is a mug shot of a suspect,and the output is a list of the top matches. Officers maybe willing to examine only the top twenty such an application, the important performancemeasure is the percentage of queries in which the cor-rect answer resides in the top twenty performance of a verification system, on theother hand, is traditionally characterized by two errorstatistics: false-reject rate and false-alarm rate. Theseerror rates come in pairs; for each false-reject rate thereis a corresponding false alarm. A false reject occurswhen a system rejects a valid identity; a false alarmoccurs when a system incorrectly accepts an a perfect Biometric system, both error rateswould be zero.

8 Unfortunately, Biometric systemsaren t perfect, so you must determine what trade-offsFebruary 200057 Biometric OrganizationsKirk L. Kroeker, ComputerAlthough poised for substantial growth as the marketplace beginsto accept biometrics, recent events have demonstrated that thefledgling industry s growth could be severely constricted by mis-information and a lack of public particular, concerns about privacy can lead to ill-informedregulations that unreasonably restrict biometrics use. The lack ofcommon and clearly articulated industry positions on issues suchas safety, privacy, and standards further increase odds that gov-ernments will react inappropriately to uninformed and evenunfounded assertions regarding Biometric technology s functionand organizations, the International Biometric IndustryAssociation and the Biometric Consortium, aim to improve thissituation.

9 International Biometric Industry AssociationA Washington, trade association, the IBIA seeks togive the young industry a seat at the table in the growing publicdebate on the use of Biometric technology. The IBIA focuses oneducating lawmakers and regulators about how biometrics canhelp deter identity theft and increase personal addition to helping provide a lobbying voice for biometriccompanies, the IBIA s board of directors has taken steps to estab-lish a strong code of ethics for its members. In addition to cer-tifying that the consortium will adhere to standards for productperformance, each member must recognize the protection ofpersonal privacy as a fundamental obligation of the promoting a position on member ethics, the IBIA rec-ommends safeguards to ensure that Biometric data is not misused tocompromise any information; policies that clearly set forth how Biometric data will be col-lected, stored, accessed, and used; limited conditions under which agencies of national securityand law enforcement may acquire, access, store, and use bio-metric data.

10 And controls to protect the confidentiality and integrity of data-bases containing Biometric IBIA is open to Biometric manufacturers, integrators, andend users ( ). Biometric ConsortiumOn 7 December 1995, the Facilities Protection Committee (acommittee of the Security Policy Board established by US PresidentBill Clinton) chartered the Biometric Consortium. With more than500 members from government, industry, and academia, the BCserves as one of the US government s focal points for research,development, testing, evaluation, and application of Biometric -based systems.