Report Template –

Biometric Matching
Algorithm

Trusted Digital Identity Framework

Release 4.6 – March 2022

PUBLISHED VERSION
Department of Finance (Finance)

This work is copyright. Apart from any use as permitted under the Copyright Act 1968
and the rights explicitly granted below, all rights are reserved.

Licence

With the exception of the Commonwealth Coat of Arms and where otherwise noted,
this product is provided under a Creative Commons Attribution 4.0 International
Licence. (http://creativecommons.org/licenses/by/4.0/legalcode)

This licence lets you distribute, remix, tweak and build upon this work, even
commercially, as long as they credit the Finance for the original creation. Except
where otherwise noted, any reference to, reuse or distribution of part or all of this
work must include the following attribution:

Trusted Digital Identity Framework (TDIF)™ © Commonwealth of Australia

(Department of Finance) 2022

Use of the Coat of Arms

The terms under which the Commonwealth Coat of Arms can be used are detailed
on the It’s an Honour website (http://www.itsanhonour.gov.au) Conventions
References to TDIF documents, abbreviations and key terms (including the words
MUST, MUST NOT, and MAY) are denoted in italics are to be interpreted as
described in the current published version of the TDIF: 01 – Glossary of
Abbreviations and Terms.

TDIF requirements and references to Applicants are to be read as also meaning

Accredited Participants, and vice versa. The scope of TDIF requirements are to be
read as applying to the identity system under Accreditation and not to the
organisation’s broader operating environment.

Digital Identity | Biometric Testing: Technical Matching Algorithm Report iii

Contact us

Finance is committed to providing web accessible content wherever possible. This

document has undergone an accessibility check however, if you are having
difficulties with accessing the document, or have questions or comments regarding
the document please email the Director, Digital Identity Policy at
digitalid@finance.gov.au

Document management

Finance has endorsed this document for release.

Change log

Version Date Author Description of the changes

1.0 March 2022 JK Published Version

Digital Identity | Biometric Testing: Technical Matching Algorithm Report iv

Contents

1 Executive Summary.............................................................................................................6

2 Test Objectives....................................................................................................................7

3 Characteristics of Dataset...................................................................................................8

4 System Under Test..............................................................................................................9

4.1 System Information....................................................................................................................... 9

4.2 Concept of Operations.................................................................................................................. 9

4.3 Configuration Audit....................................................................................................................... 9

4.4 Test Observations/Problem Log.................................................................................................. 10

4.5 Expected Outputs....................................................................................................................... 10

5 Test Overview.....................................................................................................................11

5.1 Specific Test Processes.............................................................................................................. 11

5.2 Data Collection............................................................................................................................ 11

6 Data Analysis.....................................................................................................................12

6.1 Verification Metrics...................................................................................................................... 12

6.2 Error Rates.................................................................................................................................. 12

6.3 Throughput Performance Reporting............................................................................................ 12

7 Record Keeping.................................................................................................................13

8 Performance Results.........................................................................................................14

9 Full Test Plan......................................................................................................................15

Digital Identity | Biometric Testing: Technical Matching Algorithm Report v

Report Template: Biometric Matching Algorithm

1 Executive Summary
This section should include a summary of the evaluation that was performed and the
key evaluation outcomes.

Digital Identity | Biometric Testing: Technical Matching Algorithm Report 6

Report Template: Biometric Matching Algorithm

2 Test Objectives
This section should provide an overview of:
 the goals of the evaluation
 the algorithm under test, and
 the context of the evaluation.

This would include information about the benchmarks the algorithm is being tested
against, relevant ISO testing standards, and the TDIF requirements this evaluation
seeks to meet.

To satisfy the biometric testing requirements related to Technical Verification, the

goals of the evaluation should correspond to the targets found in TDIF Req: IDP-03-
08-18 to TDIF Req: IDP-03-08-18d.

Digital Identity | Biometric Testing: Technical Matching Algorithm Report 7

Report Template: Biometric Matching Algorithm

3 Characteristics of Dataset
This section should give an overview of the biometric dataset used for the
evaluation, including:
 A description of the dataset profile
 Information about how ‘ground truth’ information regarding subject identities
was generated.
 A description of the included non-biometric metadata
 Information about how the dataset was collected

Requirements for the composition of the dataset used in the evaluation can be found
in TDIF Req: IDP-03-08-18b. Additionally, when selecting the size of their dataset,
testing entities may consider IDP-03-08-18c to ensure the required 90% confidence
interval can be met.

Digital Identity | Biometric Testing: Technical Matching Algorithm Report 8

Report Template: Biometric Matching Algorithm

4 System Under Test

This section should give a detailed description of the biometric matching algorithm
under evaluation, with information addressing the following topics.

4.1 System Information

Information about the biometric matching algorithm under test:

 Hardware/software name
 Specifications
 OS version/model/type/style
 Serial number
 Manufacturer/developer
 Function description
 Setup processes prior to testing

4.2 Concept of Operations

A description of the operating environment of the system under test, including the
risk environment, levels of user supervision, and high-level threat assessment.

4.3 Configuration Audit

A log of the biometric matching algorithm configuration, including the following

information.

 Hardware/software name
 Specifications
 Model/type/style
 Serial number
 Manufacturer
 Identified anomalies
 Date and time performed

Digital Identity | Biometric Testing: Technical Matching Algorithm Report 9

Report Template: Biometric Matching Algorithm

4.4 Test Observations/Problem Log

A test observation log to log any abnormal test results that are observed by
personnel, which includes the following information.
 Event details
 Hardware/software details
 Time to resolve/repair
 Name of the individual who observed the event
 Any known effects on the test outcome
 Times and dates of all significant maintenance and repair actions
 Link to problem log if relevant

4.5 Expected Outputs

A description of the expected (claimed) outputs of biometric matching algorithm

under test, and a description of the expected performance results, including relevant
test pass/fail criteria.

This may include, but no be limited to:

 System operational or failure data (if any); and

 Comparison results (similarity scores or match/no match decision output).

Digital Identity | Biometric Testing: Technical Matching Algorithm Report 10

Report Template: Biometric Matching Algorithm

5 Test Overview

5.1 Specific Test Processes

A description of how the evaluation was carried out, with linkages to the evaluation
goals, and noting any differences between the test plan and the actual executed test.

5.2 Data Collection

A description of how the data generated by the test processes was collected and
validated.

Digital Identity | Biometric Testing: Technical Matching Algorithm Report 11

Report Template: Biometric Matching Algorithm

6 Data Analysis
This section should give a description of the statistical techniques used in the
evaluation, including a description of any data that may be included/excluded from
the analysis.

6.1 Verification Metrics

Generally, this should include at minimum as per ISO/IEC 19795-2:

 False non-match (reject) rate

 Generalised false non-match (reject) rate
 False match (accept) rate
 Generalised false non-match (accept) rate

6.2 Error Rates

This section should give generalised error rates, including failure to enrol and failure
to enquire rates, as per Section 6.3.5 of ISO/IEC 19795-2.

These results should be contextualised in terms of statistical significance, including

confidence levels and associated confidence intervals. Examples include using the
Rule of 3 or Rule of 30, as defined in ISO/IEC 19795-1.

6.3 Throughput Performance Reporting

This section should provide throughput transaction performance reporting as per

Section 6.3.6 of ISO/IEC 19795-2.

Digital Identity | Biometric Testing: Technical Matching Algorithm Report 12

Report Template: Biometric Matching Algorithm

7 Record Keeping
This section should provide a summary of the completed tests making specific note
of any deviations from the Specific Test Processes section. This could be presented
in a tabular format.

Digital Identity | Biometric Testing: Technical Matching Algorithm Report 13

Report Template: Biometric Matching Algorithm

8 Performance Results
This section should provide the results of data analysis activities performed on the
relevant data.

This should be reported in the following tabular format from ISO/IEC 19795-5,
reporting the listed metrics at the target False Accept Rate (FAR). TDIF Req: IDP-
03-08-18c prescribes a target FMR of 0.01%. A higher and lower FMR could be
reported at to provide additional context to the results – these may be based upon
client requirements, recommendations from the provider of the biometric matching
algorithm, or on the knowledge of personnel involved.

Lower FMR Target FMR Higher FMR

(0.01%)

False Non-Match Rate

(FNMR)

Threshold

Failure to Enrol Rate

Failure to Acquire Rate

Mean Transaction Time

Depending on the test objectives, it may be relevant to record performance results

that are separated by demographics within the test corpus. When this is performed,
the statistical significance must be noted as this refers to only a portion of the total
transactions performed under which the statistical significance is different.

Digital Identity | Biometric Testing: Technical Matching Algorithm Report 14

Report Template: Biometric Matching Algorithm

9 Full Test Plan

The full test plan, as determined prior to the evaluation, should be provided as a
component of the Test Report.

Digital Identity | Biometric Testing: Technical Matching Algorithm Report 15