17EVALUATION

TECHNICAL TESTING AND

OF BIOMETRIC
IDENTIFICATION DEVICES
James L. Wayman
National Biometric Test Center
San Jose State University
San Jose, CA
biomet@email.sjsu.edu
Abstract Although the technical evaluation of biometric
identification devices has a history spanning over two decades, it is
only now that a general consensus on test and reporting measures
and methodologies is developing in the scientific community. By
''technical evaluation", we mean the measurement of the five
parameters generally of interest to engineers and physical
scientists: false match and false non-match rates, binning error
rate, penetration coefficient and transaction times. Additional
measures, such as 'failure to enroll" or 'failure to acquire",
indicative of the percentage of the general population unable to use
any particular biometric method, are also important. We have not
included in this chapter measures of more interest to social
scientists, such as user perception and acceptability. Most
researchers now accept the "Receiver Operating Characteristic"
(ROC) curve as the appropriate measure of the application-
dependent technical performance of any biometric identification
device. Further, we now agree that the error rates illustrated in the
ROC must be normalized to be independent of the database size
and other "accept/reject" decision parameters of the test. This
chapter discusses the general approach to application-dependent,
decision-policy independent testing and reporting of technical
device performance and gives an example of one practical test.
System performance prediction based on test results is also
discussed.
Keywords: Biometric identification, testing, receiver operating
characteristic curve.
 346 Wayman

1. Introduction

We can say, somewhat imprecisely, that there are two distinct functions for biometric
identification devices: 1) to prove you are who you say you are, and 2) to prove you
are not who you say you are not. In the first function, the user of the system makes a
"positive" claim of identity. In the second function, the user makes the "negative"
claim that she is not anyone already known to the system.
Biometric systems attempt to use measures that are both distinctive between
members of the population and repeatable over each member. To the extent that
measures are not distinctive or not repeatable, errors can occur. In discussing system
errors, the terms "false acceptance" and "false rejection" always refer to the claim of
the user. So a user of a positive identification system, claiming to match an enrolled
record, is "falsely accepted" if incorrectly matched to a truly non-matching biometric
measure, and "falsely rejected" if incorrectly not matched to a truly matching
biometric measure. In a negative identification system, the converse is true: "false
rejection" occurring if two truly non-matching measures are matched, and "false
acceptance" occurring if two truly matching measures are not matched. Most systems
have a policy allowing use of multiple biometric samples to identify a user. The
probability that a user is ultimately accepted or rejected depends upon the accuracy of
the comparisons made and the accept/reject decision policy adopted by the system
management. This decision policy is determined by the system manager to reflect the
operational requirements of acceptable error rates and transaction times and, thus, is
not a function of the biometric device itself
Consequently, in this chapter we refer to "false matches" and "false non-matches"
resulting from the comparison of single presented biometric measure to a single
record previously enrolled. These measures can be translated into "false accepf' and
"false rejecf under a variety of system decision policies.
In addition to the decision policy, the system "false rejection" and "false
acceptance" rates are a function of five inter-related parameters: single comparison
false match and false non-match rates, binning error rate, penetration coefficient, and
transaction speed. In this chapter, we will focus on testing of these basic parameters
and predicting system performance based on their resulting values and the system
decision policy.
Regardless of system function, the system administrator ultimately has three
questions: What will be the rate of occurrence of false rejections, requiring
intervention by trained staff?; Will the probability of false acceptance be low enough
to deter fraud?; Will the throughput rate of the system keep up with demand? The
first question might further include an estimate of how many customers might be
unable to enroll in or use the system. The focus of this chapter will be on developing
predictive tools to allow "real-world" estimates of these numbers from small-scale
tests.

2. Classifying Applications

Technology performance is highly application dependent. Both the repeatability and

distinctiveness of any biometric measure will depend upon difficulty of the