d i g i t a l i n v e s t i g a t i o n 8 ( 2 0 1 1 ) 1 4 1 e1 4 4
Available online at www.sciencedirect.com
journal homepage: www.elsevier.com/locate/diin
Standards, regulation & quality in digital investigations:
The state we are in
Angus M. Marshall*
n-gate Ltd, Oak House, Market Place, Bedale, North Yorkshire DL18 1AQ, United Kingdom
1. Introduction
In the world of forensic science there is an ongoing debate
about the use of quality standards as a means of demon-
strating the suitability of scientific methods for the production
of material which can be used within the criminal justice
system. To a large extent, the position in the UK can be
summed up by the following statements from the “Forensic
Science on Trial” report1
“171. Establishing the validity of new scientific techniques or
theories, and the basis for their interpretation, is essential before
evidence derived from them can be used in court.”
“173. The absence of an agreed protocol for the validation of
scientific techniques prior to their being admitted in court is
entirely unsatisfactory.
. purpose”. That is not to say that the evidence is inherently
and should build on the US Daubert test.”
“172.The Daubert principles require expert testimony to be
tested against four criteria:
- Whether the theory or technique can be (and has been)
tested
- Whether the theory or technique has been subjected to peer
review and publication
- In the case of a particular technique, what the known or
potential rate of error is or has been; and
- Whether the evidence has gained widespread acceptance
within the scientific community.”
The process leading to this report was, itself, instigated
partly as a result of the controversy and confusion
surrounding the use of Low Template DNA following the
Omagh bombing case. In the final judgment of R. v. Hoey2,
* Tel.: þ44 1325 722602; fax: þ44 7092 372395.
E-mail address: angus@n-gate.net.
1
Forensic Science on Trial, Seventh Report of Session 2004e2005,
House of Commons Science & Technology Committee.
2
[2007] NICC 49.
1742-2876/$ e see front matter ª 2011 Published by Elsevier Ltd.
doi:10.1016/j.diin.2011.11.001
Justice Weir cited the opinion of Lord Lowry in R. v. Steenson &
Others3 that
“Justice ‘according to law’ demands proper evidence. By that we
mean not merely evidence which might be true and to a consider-
able extent probably is true, but, as the learned trial judge put it,
“evidence which is so convincing in truth and manifestly reliable
that it reaches the standard of proof beyond reasonable doubt.”
These, and other reports, suggest that there is perhaps
a desire to adopt scientific methods too rapidly in the field of
criminal investigations. A side-effect of this rapid adoption is
a perception that proper review and validation of methods
and processes is, to some extent, bypassed and this creates
a situation where evidence produced by new methods can be
challenged on the ground of lack of evidence of “fitness for
bad, but primarily that the evidence that it is good has not
been produced.
The problem is not unique to the UK. “Strengthening
Forensic Science in the United States”4 reports, in the execu-
tive summary that
“. in some cases, substantive information and testimony based
on faulty forensic science analyses may have contributed to
wrongful convictions of innocent people”
“There is no unformity in the certification of forensic practi-
tioners, or in the accreditation of crime laboratories.”
“There are often no standard protocols governing forensic practice
in a given discipline.”
Once again, this report expresses uncertainty about the validity
of many methods, noting a lack of agreement on what constitutes
a “licence to practice” as well as “best practice” across the US.
3
[1986] NIJB 17.
4
Strengthening Forensic Science in the United States : a path forward,
Committee on Identifying the Needs of the Forensic Sciences
Community & National Research Council, 2009, National Acade-
mies Press, (online athttp://www.nap.edu/openbook.php?record_
id¼12589 last checked 12th July 2011).
142 d i g i t a l i n v e s t i g a t i o n 8 ( 2 0 1 1 ) 1 4 1 e1 4 4
2. “Traditional” forensic science
The approach taken in some traditional forensic sciences (e.g.
DNA, biology chemistry) even before the publication of these
reports was to adopt ISO/IEC 17025 “General requirements for
the competence of testing and calibration laboratories” in
conjunction with ILAC-G19 “Guidelines for Forensic Science
Laboratories”, which provides guidance on the application of
ISO/IEC 17025, as the most appropriate practical standard for
the implementation and management of quality systems.
Indeed, accreditation to ISO/IEC 17025 is required in order for
a DNA laboratory to be able to submit samples to the UK
National DNA Database.
Underpinning ISO/IEC 17025 is a requirement for a service
provider to provide evidence of three key areas in addition to
common quality management systems:
 Competence of staff
i.e. an ability to show that are not only properly qualified at the
point of recruitment, but that their skills and knowledge are
maintained and updated appropriately.
 Validation of processes
all processes should be subjected to proper validation to show
that they are fit for purpose. This validation should be based
on requirements agreed with the customer.
 Proficiency of the provider
service providers should engage in proficiency testing to
demonstrate that, given identical samples, their results
should match those of other providers offering the same
service regardless of the process used.
Throughout all of this, there is a requirement to main-
tain proper records of work undertaken including details of
any non-conformances (i.e. unexpected results, exceptions
to normal working etc.) and the actions taken to deal with
them. Providers who intend to use a process that has not
been validated against a particular set of requirements
should acknowledge that the work is “out of scope” or
carry out a validation exercise to extend the scope of the
process.
3. Forensic science regulator
5  “Guidelines for Analysis & Interpretation of Digital
The regulator was appointed in2008 as an agency within the
Home Office, to examine the issue of how best to manage
quality standards across forensic science providers to the
criminal justice system in England & Wales. The office of the
regulator is run by a small team with several advisory groups
dealing with overall quality matters and subject specific
issues.
5
http://www.homeoffice.gov.uk/agencies-public-bodies/fsr/
(last viewed 13th July 2011).
6
Codes of Practice & Conduct, Forensic Science Regulator, Feb.
2011 (available from http://www.homeoffice.gov.uk/publications/
agencies-public-bodies/fsr/codes-conduct-practice last viewed
13th July 2011).
The regulator has published several drafts of a “Manual of
Regulation” for forensic science but this has been superceded
by the “Codes of Practice and Conduct.”6
These codes build on the provisions of ISO/IEC 17025 and
ILAC-G19 for laboratory-based work and provide further
guidance on their application to scientific processes used for
the production of evidence for the criminal justice system.
The intention is for all service providers to be compliant by the
end of 2013, with the exception of digital evidence providers
who are expected to be compliant by the end of 2014.
It is also worth noting that the Regulator’s Codes include
provision for the adoption of ISO/IEC 17020 “General criteria
for the operation of various types of bodies performing
inspection” as a standard for processes carried out at crime
scenes. As is the case with ISO/IEC 17025, this is a generic
standard whose application to “forensic” processes is clarified
by interpretive guidance given in the codes.
4. The information security perspective7
ISO/IEC JTC1 SC27 Working Group 4 is the international
standards committee which deals with security controls and
services. Within its work it is, at the time of writing, devel-
oping at least two standards which have relevance to the
“digital forensic” practitioner.
 ISO/IEC 27035 “Information security incident management”
is currently in the final stages of preparation for publication
and, within its consideration of incident response, includes
the investigation of information security incidents. It is ex-
pected that the final version will also include material on
information security incident investigation readiness.
 ISO/IEC 27037 “Guidelines for identification, collection,
acquisition, and preservation of digital evidence”, in second
committee draft at the time of writing, deals with steps
which should be taken immediately following an incident in
order to maximise potential evidence gathered for investi-
gative purposes.
Allied to these, proposal for three new standards are being
considered for ISO/IEC approval. These are joint projects
between the UK and South Africa and provisionally titled:
 “Investigation Principles & Processes”
Evidence”
 “Guidance on assuring suitability and adequacy of investi-
gation methods”
The goal of this group of standards is to produce mecha-
nisms by which information security incident investigations
can be carried out effectively across national boundaries while
retaining the ability to demonstrate suitability or “fitness for
purposes” of the processes used, equivalence of different
7
All standards referred to in this section should have the text
“Information Technology - Security Techniques - “ as a prefix to
their titles.
 d i g i t a l i n v e s t i g a t i o n 8 ( 2 0 1 1 ) 1 4 1 e1 4 4
Fig. 1 e Relationship between ISO/IEC 27xxx standards.
processes and appropriate standards of process quality for the
context in which the investigation is being conducted.
Ideally, an organisation which implements all five stan-
dards will be equipped to deal efficiently and effectively with
any information security incident that may occur.
Fig. 1 shows the relationship between the family of 5
standards which may result.
 ISO/IEC 27035 deals with all aspects of incident response,
including pre-incident preparation for evidence gathering.
 “Investigation Principles and Processes” will define common
concepts and models for investigation.
 ISO/IEC 27037 deals with immediate response to an incident
in order to gather and preserve as much potential evidence
as is required.
 ISO/IEC 27042 deals with post-incident processes used for
investigation.
 ISO/IEC 27041 deals, particularly, with issues such as veri-
fication and validation of tools, methods and processes.
The editorial groups responsible for ISO/IEC 20735 and ISO/
IEC 27037 have been careful not to produce a homogenized,
standardised procedure which presupposes judicial require-
ments. Rather, they have sought to lay down a fundamental
set of principles with guidance on how they can be applied in
common scenarios. A similar approach will be used for the
three proposed new standards. The groups involved in the
production of these standards are keen to see them progress
quickly, but not at the expense of correctness.
The existing work on all five standards makes reference to
competence, proficiency and validation, as described above,
and ISO/IEC 27037 makes clear that the implementer of
a process is responsible for ensuring validation in the context
in which they intend to apply that process.
5. Conclusion
There are two distinct, but closely related, approaches to the
production of standards and/or regulation of digital investi-
gations. One is based on existing practice within the estab-
lished forensic science community, and the other is based on
a perceived need for improvements in information security
143
practices. Both approaches have their basis in common
requirements of proficiency, competence and validation but
do not, as yet, directly address how best to demonstrate that
these requirements have been satisfied. At their most
prescriptive, the existing standards define these three
concepts in terms of requirements agreed with customers.
The underlying principles based on proficiency and
competence have been shown to be sound in other disciplines
and should be applicable to digital investigations. The issue of
validation is viewed by many who do not have experience of
ISO 17025 and equivalent quality systems as a major stum-
bling block. This seems to arise primarily through an under-
standing of the concepts of verification and validation derived
from software engineering, where these terms apply to
confirmation that a product conforms to specification and
meets the needs of the user. The problem seems to be that
there is an assumption that every feature of the product must
be tested in every conceivable condition and configuration in
order for it to be considered validated. Close examination of
the wording of the relevant standards and guidance make it
clear that this is not the case and that what is really required is
the production of evidence that the process applied to
potential evidence (whether it involves the use of tools or not)
is "fit for purpose". i.e. the process must satisfy the require-
ments for its role in a particular investigation.
The proposed ISO/IEC 27041 draft available in October 2011
proposes a three stage process using the terms Verification,
Validation and Acceptance. Verification is a confirmation that
a product (tool, etc.) conforms to its specification and may be
conducted by the producer of that product. Validation is the
confirmation that a process is fit for purpose as described
above. ISO/IEC 27041 proposes that evidence produced for
verification may be used to simplify the validation process
where there is clear mapping from the producer’s require-
ments & specification to the requirements for the intended
use within the process under validation. Finally, acceptance is
the formal process of confirming that a previously validated
process may be re-used in different circumstances because
the requirements are identical. Where the requirements are
substantially the same, ISO/IEC 27041 proposes that a process
may be validated by a combination of acceptance (for the
requirements which match the new requirements) and vali-
dation (for the new requirements only).
144 d i g i t a l i n v e s t i g a t i o n 8 ( 2 0 1 1 ) 1 4 1 e1 4 4

The process outlined in the draft ISO/IEC 27041 available at terminology used is somewhat different. It should be noted,
the time of writing seems to be compatible with that currently however, that only the concept of validation is defined in ISO/
used in Digital Forensics laboratories accredited to ISO/IEC IEC 17025 and thus the ISO Concepts Database prior to the
17025 (with the adoption of ILAC-G19) although the drafting of ISO/IEC 27041.