You are on page 1of 7

Piper, et al.

Forensic Audio Standards

FORENSIC AUDIO STANDARDS

MICHAEL PIPER1, 2, DAVID HALLIMORE1,3, AND JOHN POWELL1, 4

1
Scientific Working Group for Digital Evidence, USA
audio@swgde.us
2
United States Secret Service, Washington, D.C., USA
michael.piper@usss.dhs.gov
3
Houston Police Department, Houston, TX, USA
forensicaudio@gmail.com
4
Los Angeles County Sheriff’s Department, USA
jbpowell@lasd.org

In 2009 the U.S. National Research Council (NRC) released a report entitled “Strengthening Forensic Science in the
United States: A Path Forward” [1] (hereafter, “the Report”) to the Congress of the United States. It contained a broad
overview of the status of forensic science in the United States along with several recommendations for improvement.
The Report has generated significant discussion, and its recommendations, if adopted, would have broad reaching
effects within the forensic science community, including the forensic audio community. In September 2009, the
Scientific Working Group on Digital Evidence (SWGDE) published a position paper [2] on the Report, responding to
each of the recommendations made therein. An overview of the Report, its effects, SWGDE’s response, and the
SWGDE Audio Committee’s current projects will be presented.

AES to submit comments on all its documents so that


INTRODUCTION they may accurately reflect the views of the community.
The Report triggered a call to action for SWGDE to
strengthen the digital evidence discipline (which 1 THE NAS REPORT
includes forensic audio) through collaboration with The National Academy of Sciences (NAS) is a private,
research institutions, accrediting bodies, standards non-profit body chartered by the U.S. government in
organizations, and forensic science organizations. 1863 to be an adviser on scientific and technological
Additionally, an effort is underway to promote matters, and more recently was tasked to study the
minimum standards within the digital evidence needs of the forensic science community. This action
discipline. The SWGDE Audio Committee is drafting was promoted by the Consortium of Forensic Science
comprehensive recommendations for training and Organizations to bring attention to the lack of proper
education in forensic audio and continues to develop a resources devoted to forensic science [3][4][5]. The
discussion paper on the means to validate ENF analysis NAS created a Committee on Identifying the Needs of
as an authentication test method. One of our primary the Forensic Sciences Community within the National
concerns is that the methodologies employed in forensic Research Council to fulfil its Congressional charter.
audio analysis be scientifically valid in order to be This Committee held several open meetings where they
relied upon. heard presentations from forensic science organizations,
standards organizations, medical examiners,
In each of the topics presented (best practices, minimum government and private forensic science laboratories,
standards, accreditation, certification, training, ENF, legal scholars, cognitive scientists, and other interested
etc.), there will be many points suitable for audience parties [1].
input and discussion. One of the key purposes of this
presentation is to solicit input from the forensic audio In response to the Report, the U.S. National Science and
community for the work we are undertaking at Technology Council chartered the Subcommittee on
SWGDE. All documents that SWGDE publishes are Forensic Science to “advise and assist … the Executive
made available for public comment at www.swgde.org, Office of the President on policies, procedures and
and all comments are considered. SWGDE encourages plans related to forensic science” with emphasis on
the participants of this conference and the members of “enhancing the validity and reliability” of forensic
science, the adoption of best practices, and

AES 39th International Conference, Hillerød, Denmark, 2010 June 17–19


Piper, et al. Forensic Audio Standards

“improv[ing] research, training and education, at the previous AES forensic audio conference [14]) and
accreditation and certification, protocol testing, methods continuously seeks comments for their improvement.
development, scientific validation, uncertainty
measurement, standards, uniformity of forensic science 2 THE RECOMMENDATIONS
terminology and reporting” [6]. Readers are encouraged to read the whole Report
wherein detailed justifications for its thirteen
1.1 SWGDE’s Response recommendations are presented. Those most directly
SWGDE is one of many Scientific Working Groups impacting forensic audio are summarized below along
(SWGs) established by the FBI Laboratory [7] whose with our ideas to address each issue from an audio
members are comprised primarily of U.S. law community perspective. These topics are intended to
enforcement laboratories. The mission of SWGDE is to seed an open discussion at this conference.
bring together organizations actively engaged in the
field of digital and multimedia evidence to ensure 2.1 New Federal Oversight Agency
quality and consistency throughout the digital forensic The primary recommendation is to form an independent
community [8]. federal level organization to establish and enforce best
practices; establish standards for mandatory
In his testimony before the Senate Judiciary Committee accreditation of laboratories and mandatory certification
[9], Judge Harry Edwards, co-chair of the NRC of forensic scientists; promote scholarly, competitive
Committee that assembled the Report, specifically peer-reviewed research; oversee education standards
criticised the Scientific Working Groups as a whole for and the accreditation of forensic science programs in
producing guidelines that are too vague to be of colleges and universities to advance the credibility and
practical use, for their standards not being enforced by reliability of the discipline; improve understanding of
any law or regulation, failing to measure the impact of the discipline and its limitations within legal systems;
their guidelines on or endorsement by the community. and assess the development and introduction of new
The Report identified three holdover challenges for technologies in forensic investigations, including a
“digital and multimedia evidence:” [1] comparison of new technologies with former ones.
1. The community does not have an agreed
certification program or list of qualifications for While this may be specific to the U.S., SWGDE would
digital forensic examiners. be interested to learn the pros and cons of the different
2. Some agencies still treat the examination of oversight structures in other countries, for example, the
digital evidence as an investigative rather than a role of ENFSI in Europe and the role of NIFS in
forensic activity, Australia, and how they apply to government and
3. There is wide variability in and uncertainty about private laboratories.
the education, experience, and training of those
practicing this discipline. 2.2 Standardized Terminology and Reporting
At this time it is not clear whether policy action will The Report states, “[D]espite the lack of a statistical
take place first within the executive branch or legislative foundation, examiners make probabilistic claims based
branch of government, but action is likely coming on their experience. A statistical framework that allows
[10][11]. Already, the Report has been cited within the quantification of these claims is greatly needed. These
judiciary [12]. Regardless, SWGDE took this less as a disciplines also critically need to standardize and clarify
criticism, but more as a call to action. the terminology used in reporting and testifying about
the results and in providing more information.” It also
After the Report was released, SWGDE created an ad calls for “rigorous protocols for performing subjective
hoc committee to discuss and formulate a position on interpretations.” Judge Edwards testified to the
each recommendation that reflected both the current misleading uses of the word “match” in forensic science
state of the digital and multimedia evidence discipline testimony. To standardize this terminology, the Report
and what could be done to improve the discipline as a recommends that model reports specifying the
whole [2]. Subsequent to forming this position, minimum information to be included be established and
SWGDE is reviewing all aspects of the discipline to standardized for each discipline and that their use be a
determine where minimum community standards should requirement for laboratory accreditation.
be established and where best practices can be defined.
These minimum requirements will be proposed as Terminology used in forensic audio derives from
standards to be adopted by forensic laboratory physical acoustics, audio engineering, signal
accrediting bodies. The Audio Committee within processing, general forensic science, and testing
SWGDE has already published best practices [13] for laboratories. Published dictionaries and glossaries can
general forensic audio laboratory operations (presented be found within AES [15], SWGDE [16], IEEE

AES 39th International Conference, Hillerød, Denmark, 2010 June 17–19


Piper, et al. Forensic Audio Standards

[17][18], ANSI [19], and ASTM [20]. A common AES43 [26], like many current SWG guidelines,
vernacular among forensic audio practitioners would currently lacks the desired specificity described in the
likely draw upon all of these sources. Convergence of Report. The same is true for the Koenig paper [27]
terms toward those commonly accepted in industry commonly cited within U.S. law enforcement as
could make forensic audio problems more readily describing the generally accepted operating procedure.
accessible to the general research community. What This same paper, published twenty years ago, is the
would be required of a standardized framework for most current informative reference within AES43.
interpreting and reporting the results of authentication Even the seminal work of the “Watergate tapes”
testing and the opinions developed there from? Would investigators [28], cited in both papers, was limited to a
standard descriptions of signal quality and intelligibility closed set of three recorders, all of which were available
levels, such as the speech intelligibility index [21] be for testing and allowing for deductive reasoning.
useful for customers to evaluate enhancement exams? Typical forensic cases lack complete knowledge and
require inductive reasoning [29] on an open set.
2.3 Validation
The Report insists upon research to establish “the One step required to begin to deal with this situation is
scientific bases demonstrating the validity of forensic to dispel the notion that court acceptance is an indicator
methods” and to establish quantifiable measures and of scientific validation [1][30][31] and bring “forensic”
limits of accuracy, reliability, and uncertainty and the science practices in line with those in the general
conditions under which those measures and limits apply. scientific community. Some points to consider
The Report notes a “noticeable dearth” of such peer- concerning standards of validation in forensic audio:
reviewed research studies. As others have noted [22], • Is the existing body of research sufficient to justify
these methods are not necessarily unreliable, but the conclusions being made in the field?
reliability has not been verified either by statistical • Of the research being cited to justify current
models or by consistent data on error rates. The Report methodologies, how much of it has been
devotes all of Chapter 4 (“The Principles of Science”) to reproduced by independent labs?
describing proper scientific validation studies. It • Should validation require testing by independent
defines the “most important outcomes from such a laboratories, i.e., ones outside of forensics that have
validation study are (1) information about whether or no interest in using the tool operationally?
not the method can discriminate the hypothesis from an • Have techniques that predate the validation criteria
alternative, and (2) assessments of the sources of errors in ISO/IEC 17025, ILAC, and ENFSI been
and their consequences on the decisions returned by the validated to these criteria?
method.” It goes on to cite the validation criteria in • Can the reliability of current techniques be
ISO/IEC 17025 [23] and defines as a critical step quantifiably measured? Can the uncertainty of
publication in a peer-reviewed journal with detail current techniques be quantifiably measured? Are
sufficient that an independent laboratory can replicate examiners reporting that reliability and uncertainty
the results. The second section of the chapter describes with their results?
the importance of defining uncertainty and error, and • Should opinions and interpretations in forensic
modelling the sensitivity, specificity, and positive and audio move away from “expert” opinions subject to
negative predictive values of a technique. The final bias and toward statistical reporting of results, such
section describes how sources of bias manifest as likelihood ratios [29][30][32]? Obviously
themselves in human judgment, such as ignoring computing a likelihood ratio requires the evaluation
typicality data, and that “[t]he goal is to make scientific of competing hypotheses. This could require large-
investigations as objective as possible so the results do scale typicality studies testing variables on known
not depend on the investigator.” These practices data sets. If those studies are lacking, how should
parallel those in the guidelines put forth by ILAC [24] we as a community publicly acknowledge the
and ENFSI [25]. limitations of our accepted practices?
• Do we need to create or identify standardized
The first questions to ask are what methods are being known reference collections to make validation test
employed within forensic audio, and have those results comparable?
methods been validated to the standards set in the • Like the paternity testing market with DNA, is
Report? If so, where is the scientific data published? there a commercial interest that can be leveraged to
There is a distinction between engineering practices advance the state of forensic audio?
used to enhance an audio recording, where no opinions
or interpretations of the data are offered, and more SWGDE is considering compiling a “sourcebook” of
rigorous scientific practices that are required to form a relevant material in the existing literature upon which to
reliable opinion as to the authenticity of a recording. base best practices and minimum “standards.” This can

AES 39th International Conference, Hillerød, Denmark, 2010 June 17–19


Piper, et al. Forensic Audio Standards

also serve to identify research needed to resolve Collaboration is at the heart of this presentation. With
unanswered questions. We propose to begin with basic this being the third AES conference dedicated to
test methods to see if they can be codified and forensic audio, clearly the idea of collaboration is not
standardized, and from there, develop standard limited to the authors of the Report. If any
protocols for choosing test methods and interpreting recommendation SWGDE puts forward is sufficient to
results. gain traction as a U.S. standard, it will likely reflect the
values of the international forensic audio community as
A good example for discussing validation is electric well. We’ve discussed validation above, which is an
network frequency (ENF) analysis. There have been a integral part of developing tools that advance the state
couple papers suggesting ENF test methods [33][34], of technology relevant to the discipline. Another key
but they differ in their approach and are applied to a point mentioned in this recommendation is proficiency
single grid. North America has four major testing.
interconnected grids each potentially with its own
performance characteristics. SWGDE currently Presumably a forensic audio examiner should be
maintains a discussion paper on ENF [35] cataloguing periodically tested on the tasks being performed.
these issues to benefit potential researchers. Could a Certainly those tasks that result in an expert opinion for
standard methodology be developed that could be a legal proceeding should be tested, but what about
applied to any grid? Could standard test methods be enhancement? How should proficiency be tested? How
developed and agreed upon for the extraction of ENF should the content of the test be determined? How
from a recording, the recording of reference databases, should the structure and evaluation of the test be
and the comparison of one to the other? Would the determined? How shall tests themselves be validated –
adoption of standard methods accelerate research to by what principles and to what standard? ENFSI offers
better the technique because the data would be quality general guidance on administering proficiency tests
controlled and therefore easier to cross-reference? For [36], and has administered two proficiency tests for
the development of acceptable practices and standards, authentication [37], but these tests were evaluative and
how much testing is enough? And should minimum ENFSI has not yet issued minimum testing guidelines
standards be established for subjective processes like suitable for standardization.
enhancement?
2.6 Mandatory Accreditation, Certification and
2.4 Context Bias Quality Assurance
The Report encourages research into sources and The Report states, “Forensic laboratories should
quantifiable levels of human bias and error in forensic establish routine quality assurance and quality control
examinations including to what extent results of procedures to ensure the accuracy of forensic analyses
analysis are influenced by knowledge of case details and the work of forensic practitioners.” It also suggests
unrelated to the measurements being made. It that accreditation of laboratories and certification of
recommends standard operating procedures be examiners be mandatory, and that standards for such
developed to minimize potentially biasing sources of should be enforced through disciplinary procedures. It
information. states “No person (public or private) should be allowed
to practice in a forensic science discipline or testify as a
Has any research been done on these effects in forensic forensic science professional without certification.
audio analysis? Does anyone currently have these Certification requirements should include, at a
provisions written in their procedures? minimum, written examinations, supervised practice,
proficiency testing, continuing education, recertification
2.5 Collaboration, Tool and Protocol Development procedures, adherence to a code of ethics, and effective
The Report calls for collaboration between NIST (the disciplinary procedures.”
U.S national standards body), the SWGs, government
labs, private labs, and universities to “develop tools for SWGDE has expressed caution [2] at making
advancing measurement, validation, reliability, accreditation mandatory due to the personnel and
information sharing, and proficiency testing” and “to financial burdens it imposes, but endorses a robust,
establish protocols for forensic examinations, methods, written quality assurance program [38]. To date, other
and practices. Standards should reflect best practices than general laboratory accreditation under ISO/IEC
and serve as accreditation tools for laboratories and as 17025, is there an accreditation program available
guides for the education, training, and certification of suitable for forensic audio examination? To what
professionals.” standards should labs be accredited and to what
categories of testing should it apply? What would the

AES 39th International Conference, Hillerød, Denmark, 2010 June 17–19


Piper, et al. Forensic Audio Standards

impact of mandatory accreditation be on smaller toward fulfilling the recommendations of the Report,
laboratories? but the competition for such funding will be great.
SWGDE is attempting to position itself to influence
Similar questions can be asked about mandatory funding decisions toward problems recognized as most
certification. If certification is mandatory, does that not valuable to the forensic audio community as whole. To
imply it be consistent for all examiners? Where should achieve this, we intend to create comprehensive
this be developed and how should it be administered? recommendations based on international collaboration
Should the certification program itself be accredited to among SWGDE, AES, ENFSI, and other interested
some testing standard? SWGDE offers some basic parties. While some of the recommendations in the
criteria that any such certification should meet [2], but Report focus on law and policy issues specific to the
the content of such a program should be consistent with U.S., others apply to the forensic community as a
education standards for the discipline which are also not whole.
yet defined. How should accreditation and certification
be enforced (if at all), through licensing or through the We look to our colleagues at this conference for
courts? assistance in this task and have posed many questions to
further our field. Do you agree with some or all of the
2.7 Ethics recommendations? If funding were available to realize
The Report also calls for a unification of codes of ethics them, would you support (some of) them? If so, why?
across all disciplines and forensic science societies, If not, why not? SWGDE encourages comments on this
recommending enforcement through the certification presentation as well as our “Best Practices for Forensic
process. Audio” document. Ideas and remarks can be sent to
audio@swgde.us.
2.8 Education
4 REFERENCES
To generate university graduates suitable for forensic
sciences, the Report urges the development of [1] Committee on Identifying the Needs of the
interdisciplinary graduate programs focused on forensic Forensic Sciences Community, National
science related fields with emphasis on developing and Research Council (U.S.), “Strengthening
improving research methods and methodologies Forensic Science in the United States: A Path
applicable to forensic science practice. Forward”, The National Academies Press
(2009),
The Audio Committee within SWGDE is currently http://www.nap.edu/catalog.php?record_id=125
drafting comprehensive recommendations for training 89. Also available from the U.S. Department of
and education in forensic audio by identifying the core Justice at
competencies needed for each process in a forensic http://www.ncjrs.gov/pdffiles1/nij/grants/22809
audio lab. Once identified, these core competencies will 1.pdf. The Executive Summary of the report is
serve as the backbone for any recommended training available for free at http://www.nap.edu/nap-
program, proficiency testing program, and certification cgi/report.cgi?record_id=12589&type=pdfxsum
process.
[2] Scientific Working Group on Digital Evidence,
3 DISCUSSION “Scientific Working Group on Digital Evidence
(SWGDE) Position on the National Research
The Report calls for standardization of operating Council Report to Congress Strengthening
procedures to make measurable and maximize reliability Forensic Science in the United States: A Path
and minimize both bias and sources of human error. Forward” (2009),
Some aspects of forensic audio analysis may be best http://www.swgde.org/documents/swgde2009/P
served by adopting uniform test methods and uniform osition on the National Research Council
interpretation protocols to guarantee reliable, Report to Congress.pdf.
comparable results. For others there may be a set of
multiple acceptable practices each with its own
[3] Consortium of Forensic Sciences Organizations
measurable reliability under defined circumstances.
(CFSO), Press release, 2009 February 17,
Still others might not be suitable for standardization at
http://www.thecfso.org/news/press_releases/20
all. One of our primary concerns is that the
090217_nsa_study.pdf.
methodologies employed in forensic audio analysis be
scientifically valid. SWGDE intends to identify
[4] CFSO Advocacy web page, accessed 2010
shortcomings within the forensic audio community and
March 11, http://www.thecfso.org/advocacy.
facilitate their solutions. As the U.S. government
response takes form, research funding may be allocated

AES 39th International Conference, Hillerød, Denmark, 2010 June 17–19


Piper, et al. Forensic Audio Standards

[5] R. J. Garrett, President of the International [13] Scientific Working Group on Digital Evidence,
Association for Identification (IAI), Letter to “SWGDE Best Practices for Forensic Audio,”
IAI members, 2009 February 19. http://www.swgde.org/documents/swgde2008/S
http://www.theiai.org/current_affairs/nas_memo WGDEBestPracticesforForensicAudioV1.0.pdf
_20090219.pdf. (2008).

[6] Charter of the Subcommittee on Forensic [14] M. Piper and D. Hallimore, “SWGDE Best
Science, Committee on Science, National Practices for Forensic Audio”, presented at the
Science and Technology Council, Office of AES 33rd International Conference: Audio
Science and Technology Policy, Executive Forensics—Theory and Practice, Denver, CO,
Office of the President, USA, 2008 June 5–7.
http://www.whitehouse.gov/sites/default/files/m
icrosites/ostp/forensic-science-subcommittee- [15] AES27-1996 (r2007), “AES recommended
charter.pdf. practice for forensic purposes – Managing
recorded audio materials intended for
[7] D. E. Adams and K. L. Lothridge, “Scientific examination.”
Working Groups”, Forensic Science
Communications, Vol. 3, No. 3 (2000). [16] Scientific Working Groups on Digital Evidence
and Imaging Technology, “SWGDE and
[8] Scientific Working Group on Digital Evidence SWGIT Digital & Multimedia Evidence
By-Laws, Article I, Section 2 (2008). Glossary,”
http://www.swgde.org/documents/swgde2008/S
[9] Statement of the Hon. Harry T. Edwards, Senior WGDE_SWGITGlossaryV2.2.pdf (2007).
Circuit Judge and Chief Judge Emeritus, United
States Court of Appeals for the D.C. Circuit and [17] IEEE Standards Information Network, IEEE
Co-Chair, Committee on Identifying the Needs Standards Dictionary: Glossary of Terms &
of the Forensic Science Community, The Definitions, ISBN 978-0-7381-5739-9 (CD-
Research Council of the National Academies, ROM).
before the United States Senate Committee on
the Judiciary, March 18, 2009, [18] IEEE 100, The Authoritative Dictionary of
http://judiciary.senate.gov/pdf/09-03- IEEE Standards Terms, Seventh Ed. (CD-
18EdwardsTestimony.pdf. Webcast at ROM), ISBN 978-0-7381-4903-5 (2006).
http://judiciary.senate.gov/webcast/judiciary031
82009-1000.ram. Full hearing webpage at [19] ANSI S1.1-1994 (R2004) “Acoustical
http://judiciary.senate.gov/hearings/hearing.cfm Terminology.”
?id=3714.
[20] ASTM E 1732-96a (Reapproved 2005)
[10] Hearing: “The Need to Strengthen Forensic “Standard Terminology Relating to Forensic
Science in the United States: The National Science”.
Academy of Science's Report on a Path
Forward”, United States Senate, Committee on [21] ANSI S3.5-1997 (R2007) “Methods for the
the Judiciary, 2009 March 18, Calculation of the Speech Intelligibility Index.”
http://judiciary.senate.gov/hearings/hearing.cfm
?id=3714. [22] D. Kennedy, “Forensic Science: Oxymoron?”
editorial, Science, vol. 302, 5 December 2003,
[11] Hearing: “Strengthening Forensic Science in the p. 1625.
United States”, United States Senate,
Committee on the Judiciary, 2009 September 9, [23] ISO/IEC 17025:2005 “Accreditation of testing
http://judiciary.senate.gov/hearings/hearing.cfm and measurement laboratories.”
?id=4038.
[24] ILAC G-19:2002, “Guidelines for Forensic
[12] Melendez-Diaz v. Massachusetts, 129 S.Ct. Science Laboratories.”
2527 (2009).
http://www.law.cornell.edu/supct/html/07- [25] ENFSI QCC-VAL-001, “Validation and
591.ZO.html Implementation of (New) Methods” (2006).

AES 39th International Conference, Hillerød, Denmark, 2010 June 17–19


Piper, et al. Forensic Audio Standards

[26] AES43-2000 (r2005), “AES standard for [33] C. Grigoras, “Applications of ENF Analysis in
forensic purposes – Criteria for the Forensic Authentication of Digital Audio and
authentication of analog audio tape recordings.” Video Recordings,” Journal of the Audio
Engineering Society, vol. 57, no. 9, pp.6-660
[27] B. E. Koenig, “Authentication of Forensic (2009).
Audio Recordings,” Journal of the Audio
Engineering Society vol. 38, no. 1/2, pp. 3-33 [34] A. J. Cooper, “The Electric Network Frequency
(1990). (ENF) as an Aid to Authenticating Forensic
Digital Audio Recordings—An Automated
[28] R. H. Bolt, F. S. Cooper, J. L. Flanagan, J. G. Approach,” presented at the AES 33rd
McKnight, T. G. Stockham, Jr., and M. R. International Conference: Audio Forensics—
Weiss, “Report on a Technical Investigation Theory and Practice, Denver, CO, USA, 2008
Conducted for the U.S. District Court for the June 5–7.
District of Columbia by the Advisory Panel on
White House Tapes," U. S. Government [35] Scientific Working Group on Digital Evidence,
Printing Office (1974). “ENF Discussion Paper,”
http://www.swgde.org/documents/swgde2009/S
[29] G. Jackson, S. Jones, G. Booth, C. Champod, WGDE_ENF_Discussion_Paper_v1.1.pdf
and I. W. Evett, “The Nature of Forensic (2009).
Science Opinion,” Science & Justice, vol.46,
no. 1, pp. 35-44 (2006). [36] ENFSI QCC-PT-001, “Guidance on the
Conduct of Proficiency Tests and Collaborative
[30] I. Evett, “Evaluation and Professionalism,” Exercises Within ENFSI,” 2005 February 18.
Science and Justice, vol. 49, pp. 159-160
(2009). [37] “Final report of the first Proficiency Test for
Authentication of Analogue Audio Tape
[31] D. M. Risinger, “The NAS/NRC Report on Recordings” (draft), September/December
Forensic Science: A Path Forward Fraught with 2003. Provided by ENFSI FSAAWG.
Pitfalls,” Utah Law Review (forthcoming),
January 15, 2010. Available at SSRN: [38] Scientific Working Group on Digital Evidence,
http://ssrn.com/abstract=1537038. “Recommended Guidelines for Developing a
Quality Management System,”
[32] G. S. Morrison, “Forensic Voice Comparison http://www.swgde.org/documents/swgde2006/S
and the Paradigm Shift,” Science and Justice, WGDE%20Recommended%20Guidelines%20f
vol. 49, pp.298-308 (2009). or%20Developing%20a%20Quality%20Manag
ement%20System.pdf (2006).

AES 39th International Conference, Hillerød, Denmark, 2010 June 17–19

You might also like