DATE DOWNLOADED: Wed May 17 03:01:02 2023

SOURCE: Content Downloaded from HeinOnline

Citations:
Please note: citations are provided as a general guideline. Users should consult their preferred
citation format's style manual for proper citation formatting.

Bluebook 21st ed.

Esther Nir & Siyu Liu, What Do the Gatekeepers See? Perceptions and Evaluations of
Scientific Evidence among State Court Judges, 22 CRIMINOLOGY, CRIM. Just. L & Soc'y 1
(2021).

ALWD 7th ed.

Esther Nir & Siyu Liu, What Do the Gatekeepers See? Perceptions and Evaluations of
Scientific Evidence among State Court Judges, 22 Criminology, Crim. Just. L & Soc'y 1
(2021).

APA 7th ed.

Nir, E., & Liu, S. (2021). What do the gatekeepers see? perceptions and evaluations
of scientific evidence among state court judges. Criminology, Criminal Justice, Law &
Society, 22(1), 1-16.

Chicago 17th ed.

Esther Nir; Siyu Liu, "What Do the Gatekeepers See? Perceptions and Evaluations of
Scientific Evidence among State Court Judges," Criminology, Criminal Justice, Law &
Society 22, no. 1 (April 2021): 1-16

McGill Guide 9th ed.

Esther Nir & Siyu Liu, "What Do the Gatekeepers See? Perceptions and Evaluations of
Scientific Evidence among State Court Judges" (2021) 22:1 Criminology, Crim Just L &
Soc'y 1.

AGLC 4th ed.

Esther Nir and Siyu Liu, 'What Do the Gatekeepers See? Perceptions and Evaluations of
Scientific Evidence among State Court Judges' (2021) 22(1) Criminology, Criminal
Justice, Law & Society 1

MLA 9th ed.

Nir, Esther, and Siyu Liu. "What Do the Gatekeepers See? Perceptions and Evaluations
of Scientific Evidence among State Court Judges." Criminology, Criminal Justice, Law
& Society, vol. 22, no. 1, April 2021, pp. 1-16. HeinOnline.

OSCOLA 4th ed.

Esther Nir & Siyu Liu, 'What Do the Gatekeepers See? Perceptions and Evaluations of
Scientific Evidence among State Court Judges' (2021) 22 Criminology, Crim Just L &
Soc'y 1 Please note: citations are provided as a general guideline.
Users should consult their preferred citation format's style manual for proper
citation formatting.

Provided by:
International Islamic University Malaysia
 VOLUME 22, ISSUE 1, PAGES 20-35 (2021)

Criminology, Criminal Justice, Law & Society

CRIMINOLOGY, CRIMINAL JUSTICE, LAW & SOCIETY

E-ISSN 2332-886X
Available online at
https://scholasticah.com/criminology-criminal-iustice-law-societv/

What Do the Gatekeepers See? Perceptions and Evaluations of Scientific

Evidence Among State Court Judges

Esther Nir,a Siyu Liub

a New Jersey City University
b Penn State Harrisburg

ABSTRACT AND ARTICLE INFORMATION

Judicial gatekeeping decisions impact criminal case processing (e.g., verdicts, pleas) in significant ways. Despite this
influential role thatjudges play, little is known about the key factors that contribute to judicial perceptions of evidence and
the evaluative processes thatjudges employ in making evidentiary rulings. Using qualitative interviews with 41 state court
judges presiding in a U.S. state employing the Frye standard, we explore judicial perceptions of a variety of scientific
evidentiary forms and the processes by which judges reach reliability and credibility determinations in performing their
gatekeeping duties. Our analyses reveal that judges are highly influenced by the general reputation of forensic evidence,
often lack understanding of the scientific methodologies involved, and tend to focus on factors external to reliability (e.g.,
expert's communication skills and showmanship, weight of the evidence) in their evaluative processes. Further, credibility
assessments of witnesses are fraught with subjective interpretations, potentially leading to disparate evidentiary rulings.

Article History: Keywords:

Received January 1 5th, 2021 evidence, judicial discretion, forensic, law, procedural justice, qualitative methods,
Received in revised form March 16 th, 2021 courts
Accepted March 16 th 2021

© 2021 Criminology, Criminal Justice, Law & Society and The Western Society of Criminology
Hosting by Scholastica. All rights reserved.

Correspondingauthor: Esther Nir, Department of Criminal Justice, New Jersey City University - Jersey City, Professional
Studies Building, Room 226, 2039 John F. Kennedy Boulevard, Jersey City, NJ, 07305, USA.
Email: enir@njcu.edu
2 NIR
"Evidence is a currency in which legal cases are
transacted" (Cooney, 1994, p. 851).
The judicial role as "gatekeeper of the
evidence" is a prominent and critical decision-making
responsibility that is highly consequential to case
outcomes (Daubertv. Merrell Dow Pharmaceuticals,
1993; Schweitzer & Saks, 2009). Indeed, judges'
decisions to admit incriminating pieces of evidence at
trial may contribute to the establishment of the
defendant's guilt, while rulings to allow favorable
defense evidence or exclude damaging prosecution
evidence may lead to an acquittal. Judicial
gatekeeping decisions may also influence plea offers
by the prosecution, as well as defense decisions to
accept or reject offers (Kramer et al., 2007; Rossman
et al., 1980); specifically, plea negotiations are often
guided by prosecution and defense perceptions of the
likelihood of prevailing at trial (based, in part, on their
perceptions of the strength of the admissible
evidence), sometimes referred to as "bargaining in the
shadow of the trial" (Bushway et al., 2014; Dezember
& Redlich, 2019; Kramer & Ulmer 2002; Landes,
1971; Redlich et al., 2016; Yan, 2019). In fact, mere
projections that a particular judge may exclude or
admit a specific piece of evidence (e.g., due to
reliability or constitutional challenges) could
profoundly impact the manner by which prosecutors
and defense attorneys exercise their discretion in
resolving cases within the court community (Nardulli,
1986; Nir & Liu, in press; Wilson, 2016). Lastly, the
admission of faulty evidence has been linked with the
occurrence of wrongful convictions (Derwin, 2018).
Beyond gatekeeping functions, judicial
perceptions of evidence may influence other stages of
criminal case processing. For example, in bench trials,
judges play the role of triers of fact and determine
whether the defendant is guilty based on their
perceptions of the reliability, credibility, and overall
strength of the admissible evidence (Blanck, 1992).
Further, despite the theoretical understanding that
evidence used to establish guilt should have little
bearing in sentencing (since questions of guilt have
been resolved by this point of case processing), recent
empirical studies establish that judicial perceptions of
evidentiary strength (as manifested by type and
quantity) are significantly related to the length of
custodial sentences judges impose (Nir & Griffiths,
2018, 2019; Peterson et al., 2010, 2013) and that
judicial confidence level in guilt (based on perceptions
of evidentiary strength) partially guides sentencing
decisions (Nir & Griffiths, 2019).
The documented influence of evidentiary
considerations in driving case outcomes underlines the
Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1
& Liu
determinative force of admissible evidence in criminal
case processing. Among the various types of evidence,
judges are uniquely challenged by the task of
evaluating scientific evidence. As Beecher-Monas
(1999) stated, "judges, traditionally triers of law,
occasionally pressed into service as triers of fact, now
must also be triers of science in cases where experts
proffer scientific evidence" (p. 1047). These
challenges prompt important questions: Do judges
have sufficient knowledge and guidance to perform
their gatekeeping function effectively? How do judges
perceive the strength and reliability of different types
of scientific evidence? What evaluative processes do
judges employ in assessing the reliability of various
forms of scientific evidence? Gaining insight on
judicial perceptions of evidence and evaluative
processes in determining admissibility is a crucial step
to better understanding criminal case processing as a
whole. Yet, despite the imperative role that judicial
perceptions and evaluations of evidence play, only a
handful of existing empirical studies specifically focus
on this topic (Brodsky et al., 2010; Jurs, 2009;
O'Brien, 2018; for studies using non-American data,
see Canela et al., 2019; Connolly & Gordon, 2011;
Moulin et al., 2018; Porter& Brinke, 2009). Using in-
depth qualitative interviews of 41 state court judges
presiding in a state in the United States that employs
the Frye standard, we seek to help fill this gap by
exploring judicial knowledge, perceptions, and
opinions of the most common forms of scientific
evidence used in criminal cases and the processes by
which judges analyze evidentiary reliability,
credibility, and value in performing their gatekeeping
functions.
Literature Review
Types of Evidence
A broad range and scope of evidence may be
introduced during criminal prosecutions (Grimm,
2018) and can be classified in a variety of ways
(Findley, 2013; Greenstein, 2009; Heller, 2006;
Thompson, 2012). In this article, we discuss specific
scientific evidentiary forms as points of focus to gain
insight into the manner by which judges approach the
task of evaluating the admissibility of scientific
evidence, as well as their general perceptions of the
evidence evaluated. We discuss DNA, latent prints,
ballistic evidence, and chemist evidence, commonly
introduced in criminal cases (for details, see Table 1).
Our respondents most commonly referenced these
types of evidence during our interviews, and they have
been tasked with reaching admissibility decisions on
 NIR & Li 3

Table 1: Evidentiary Types in the Current Study

By comparing the analytical result

Considered "the most Forensic evidence
from the evidence sample to reference
discriminating" evidence, garners a great deal
samples collected from individuals
deoxyribonucleic acid (DNA) of confidence from
with known identities through
DNA Expert testing facilitates identifications the general public
polymerase chain reaction (PCR)
of individuals; it is contained in and the courts. In
Short Tandem repeat (STR)
body fluids, skin samples, bones, fact, "the trust that is
technology on matching 20 loci (FBI,
and hair (FBI, 2019). laid upon the forensic
2020).
sciences generally
falls somewhere
between uncritical
faith and
manufactured myth"
(Saks & Faigman,
2008, p. 150; see also
Edmond & Roque,
2012).

Judges are generally

This refers to any evidence
related to the functionality and
Ballistic
Expert physical characteristics of
evidence
suspect firearm or firearm-
related items (FBI, 2019).
very receptive to
expert opinions
presented by the
prosecution on behalf
of the state (Edmond
"The observed physical & Roque, 2012).
characteristics of the questioned items
are compared to reference firearms or
other information sources to
determine if they are consistent"
(FBI, 2019, p. 47). Often, using "a
traditional microscopic comparison of
the actual evidence items," the digital
image of a test-fired cartridge
case/shotshell casing or evidence
cartridge case/shotshell casing can be
compared with records in the
database (FBI, 2019, p. 70).

these forms of scientific evidence. We further explore Judicial Gatekeeping Decisions

judicial perceptions/approaches to credibility
determinations applicable to scientific expert The Standards and Processes of Gatekeeping
witnesses. Below, we describe the processes and legal
While jurors consider evidentiary weight in a
standards that guide and direct judges in their
case, judges are responsible for deciding the
gatekeeping tasks. We then review challenges drawn
admissibility of evidence (Faigman et al., 2016);
from the literature on these evaluative processes.
evidence determined to be inadmissible by the trial

Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1

4 NIR
judge is not heard by the jury.1 During hearings or
trials, expert witnesses may provide explanations on
procedures and scientific contexts for forensic
evidence. Before an expert witness is permitted to
present their opinion in court, the judge decides
whether or not the witness possesses sufficient
expertise on the subject matter in question. In contrast
to lay witnesses, whose admissibility could rely
largely on judicial common sense (Burns, 2016) and
legal training, expert witnesses must possess the
knowledge, experience, education, and skill set to
reliably inform the factfinder about the evidence in the
relevant and currently established field of scientific or
professional practice. Since judges are in the
"gatekeeping" position, criteria have been established
to help them reach determinations on the admissibility
of expert witness testimony (Shelton, 2009).
Currently, two legal standards are widely
practiced in the United States. The Frye Rule, also
known as the "general acceptance" test, was first
established in Frye v. United States (1923, D.C.
Circuit) and requires trial judges to determine whether
the scientific methods employed are generally
accepted in the scientific community. The newer
standard is based on Daubert v. Merrell Dow
Pharmaceuticals (1993) where the Supreme Court
solidified the "gatekeeper" role of the trial courts and
recommended four factors to facilitate the evaluative
process: whether the methods used are generally
accepted within the scientific community, whether
they have been published and subjected to peer review,
whether the methods have been or can be tested (i.e.,
falsifiability), and whether there is a known error rate.
The Daubertstandard replaces the prior Frye standard
in federal courts and is aligned with the amended
Federal Rules of Evidence 702 (i.e., expert's
specialized knowledge, reliance on sufficient facts and
data, application of "reliable principles and methods,"
and reliable application of the methods to case facts;
FederalRules of Evidence 702, 2021; Grimm, 2018).
Over the past two decades, a majority of state
jurisdictions became followers of the Daubert test or
similar model, and only a handful of states remain in
the Frye camp (Cheng & Yoon, 2005; DeMatteo et al.,
2019; see also the Frye-Mack standard in Minnesota
discussed by Alter, 2017).2 Fradella and colleagues
(2004) reported that judges perform well under
Daubert in excluding unqualified experts yet
demonstrate inconsistencies in allowing or excluding
certain types of evidence (e.g., those of forensic
identification).
Both the Frye and Daubert standards are not
without flaws. The Frye test is a single- prong criterion
that lacks the flexibility and comprehensiveness to
allow for constant scientific advancement (Allen
&
Nafisi, 2010). Beecher-Monas (1998) argued that
Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1
& Liu
many types of "junk" science evidence were admitted
under Frye (e.g., hair comparison, bitemark analysis,
and voice spectrography) even though there is a lack
of sufficient support to establish that these evidentiary
types could be of identification value. On the other
hand, many commentators highlight that the Daubert
standard shifted the authority of gatekeeping decisions
from the relevant scientific community under Frye
(i.e., fingerprint examiners in cases involving
fingerprint evidence) to trial judges, who do not
routinely receive scientific training and thus may not
be the most appropriate gatekeepers (Allen & Nafisi,
2010; Cheng & Yoon, 2005; Grimm, 2018; Shelton,
2009). Nevertheless, some evidence show that judges
have a heightened awareness of "junk" science and
have adopted a more careful approach to evaluating
expert testimonies since Daubert (Cheng & Yoon,
2005; Krafka et al., 2002; see Dickson, 2018, for a
critique that Daubert falls short in guiding judges to
rule out "junk" science, using future dangerousness
evidence as an example). Further, legal scholars have
also pointed out instances where judges' tolerated lab
reports involving misconduct (Giannelli, 2007, on
Joyce Gilchrist, forensic chemist in Oklahoma City
Police Department and her abuse of scientific
evidence).
In addition to reliability determinations,
evidentiary gatekeeping requires credibility
evaluations by judges. In evaluating credibility, judges
make a host of assessments including whether the
testimony sounds credible, whether the witness has a
motive to lie, the witness's demeanor as nonverbal
"leakage cues" (Ekman & Friesen, 1969; O'Regan,
2017), surrounding circumstances, the witness's facial
expressions (Brodsky et al., 2010; Connolly
&
Gordon, 2011; Porter & Brinke, 2009; Wessel et al.,
2006), and the use of "uptalk"-rising final intonation
(lower confidence rating by listeners in the expert
witness context, Levon & Ye, 2020).
The EvaluativeDifficulties in Gatekeeping
The ability of judges to ably perform their
gatekeeping function is fraught with hardships
(Derwin, 2018). First, judges must make evidentiary
rulings on subjects that are outside of their legal
expertise and general knowledge base. For example,
judges have little training in medicine, methodology,
and statistics, yet gatekeeping decisions often rely on
the application of knowledge from these fields (Canela
et al., 2019; McQuiston-Surrett & Saks, 2009; Saks
&
Faigman, 2008; Wojcikiewicz, 2013). Even guiding
standards (i.e., Daubert or Frye) can only provide
limited assistance to judges; in fact, scholars have
found that it remains questionable whether the
implementation of either the Frye or Daubertstandard
results in any practical difference in how judges reach
 NIR & Liu

determinations (Cheng & Yoon, 2005; Damon-Moore, (2010), Damon-Moore (2017) provides a deep and
2017; Groscup et al., 2002; Jensen, 2003; Moreno, scathing explanation for this judicial failure, focusing
2003). on the lack of scientific training (see also Kovera

&
Further, instead of following the applicable McAuliff, 2000), the lack of defense challenges to
standard of review, prior scholarship suggests that prosecutorial evidence (see also Neal et al., 2019),
when the scientific processes are difficult to reliance on heuristics (i.e., mental shortcuts relying on
understand, judges resort to what they do understand the fallacy of tradition; see also Liden et al., 2019),
(Beecher-Monas, 1998). For example, Gatowski and judicial instincts about the defense and prosecution,
her colleagues (2001) found that judges have trouble and the unspoken alliance between the bench and the
understanding the concepts of falsifiability and error prosecution (see Nir & Liu, in press), all of which
rates, and attach more weight to "general acceptance." disadvantage the criminal defendant's case.
Saks and Faigman (2008) found that in jurisdictions An additional challenge to judicial
that apply the Daubert standard, judges have "bent gatekeeping decisions involves the subjectivity
over backwards to evade the application of Daubert inherent in these determinations (Walsh, 1999). Given
when conscientious application would lead to the the differences in attitudes and philosophies among
exclusion of any of the nonscience forensic sciences" judges toward this role, Judge Walsh (1999) raised
(p. 163). Other factors employed by judges to evaluate doubt that the result of evidence earning entry in one
expert evidence include relevance and qualification of courtroom would be able to do so in another. Despite
the expert, or factors external to their expertise- Judge Walsh's concern, little empirical evidence on
including likability, believability, trustworthiness, and judicial disparity in evidentiary rulings is available.
intelligence (Brodsky et al., 2010).
The shortfalls in judicial scientific Method
knowledge are exacerbated by several factors. First,
Data for this study were drawn from
criminal defense attorneys are generally unable to
qualitative interviews with 41 state court judges
identify methodological weaknesses or to access
presiding in a Frye state. Interviewees preside over
expert witnesses to testify on behalf of their clients
felony cases; among other functions, they make
(Edmond & Roque 2012; Saks & Faigman, 2008).
evidentiary rulings on a myriad of pre-trial motions
Second, there is a vulnerability to bias in the
and hearings, determine the admissibility of expert
production of scientific evidence (Butt, 2013; Dror,
testimony at trial, preside over jury trials, conduct
2013; Dror & Cole 2010; Elwell, 2020; Kassin et al.,
bench trials, and sentence convicted defendants.
2013; National Academy of Science [NAS], 2009;
Judicial experience among our respondents ranges
Smalarz et al., 2016; Stevenage & Bennett, 2017),
from less than 2 years and over 30 years (average =
even though forensic evidence is considered to be the
17.6 years), with 36 judges having at minimum of 9
most influential in guilt determinations (Smith et al.,
years of experience on the bench. Moreover, the vast
2011).
majority of our subjects have prior legal experience in
Another major challenge to gatekeeping is
criminal law (e.g., about 95% of our sample are former
the troubling vulnerability of perception to judicial
prosecutors, defense attorneys, or both). Seven judges
bias (Burns, 2016). Using a sample of civil rights cases
in our sample are female, 3 and judges preside in urban
in U. S. district courts, O'Brien (2018) found that male
(53.7%), suburban (19.5%), or rural (26.8%) locations
judges are more likely than their female counterparts
and are either elected (over 60%) or appointed to the
to exclude expert evidence. Other factors, such as
bench.
judicial experience, may also be influential (Moulin et
After receiving approval from the Internal
al., 2018). Judicial bias is also observed unevenly in
Review Board (IRB), data collection took place during
civil and criminal cases. While defendants in civil
2014 and 2015,4 and include judges from counties all
cases are often able to successfully challenge
over the state. Efforts to recruit judges to participate in
plaintiffs' evidence, criminal defendants face
our study involved various methods (Nir, 2018),
insurmountable challenges (Dioso-Villa, 2015;
including reaching out to personal contacts (n = 8),
Groscup et al., 2002; Risinger, 2000; Rozelle, 2007;
referrals from other respondents (n 13), cold-calling
Shniderman, 2017; see Merlino et al., 2007 for a study
methods (n 15), and cold calling with reference to a
of admissibility decisions under Daubert, comparing
general email that one of our interviewees sent to
those of criminal and civil cases). Commentators like
fellow judges in his courthouse introducing our study
Judge Donald E. Shelton (2010) attribute the failure of
(n 5). Our interviews were conducted in judges'
judges as gatekeepers in criminal cases to precedent-
chambers and ranged from 45 minutes to 3 hours (90
based rationalizations and pro-prosecution bias,
minutes on average). An in-depth semi-structured
resulting in the admission of sub-par science into the
interview design was used to collect the data to
courtroom, regardless of validity. Echoing Shelton

Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1

6 NIR
"ensure that the basic lines of inquiry [were] pursued
with each person" (Patton, 2002, p. 343). As this study
was part of a larger project evaluating the influence of
evidentiary weight on sentencing determinations, our
interview guide included questions related to a myriad
of topics surrounding judicial decision-making, with
an emphasis on judicial perceptions of different
evidentiary types and processes. Other areas of focus
include judicial perceptions of sentencing disparities,
sentencing factors, juries, mandatory minimums, and
perceptions of judicial discretion, among others.
Regarding judicial perceptions of evidence, we asked
judges to discuss their perceptions of different forms
of evidence, including DNA, fingerprints, chemists,
ballistic evidence, and other non-scientific evidence
(eyewitnesses, character witnesses, alibi witnesses),
among others. Judges were probed to discuss the
strengths and weaknesses of each evidentiary form and
to compare different evidentiary types to one another;
they were also asked to discuss their evaluative
processes in assessing reliability and credibility of
evidence in their own courtrooms. We utilized an
inductive grounded theory approach during data
collection and analysis to develop themes that "place
the data into a more general or abstract framework"
(Maxwell, 2005, p. 97). Observed themes include
judicial confidence in various forms of forensic
evidence, judicial focus on the presentation skills of
expert witnesses, and concerns regarding the
reliability and credibility of witnesses, among others.
We compared data from each new interview with prior
interviews and refined developing themes to
accurately reflect the complete body of data collected
(Maxwell, 2005). In this paper, we focus on the
gatekeeping of scientific evidence widely used in
criminal cases.
Results
Overall, our respondents value scientific
evidence and perceive it to be stronger and more
reliable than non-scientific evidence. As one judge
noted: "Science doesn't lie. People make mistakes.
People lie. People can deceive. The science doesn't"
(Judge 34). A second judge echoed a similar
sentiment: "Well, scientific evidence, most of it, either
it is or it isn't whereas witness testimony, no matter
what they say, there is a possibility that they are
wrong, right, half right, half wrong" (Judge 40). As a
result, judges generally feel more confident
adjudicating cases when the evidentiary package
includes scientific evidence (Nir & Griffiths, 2019).
Yet, judicial perceptions of reliability vary
considerably among scientific evidentiary types, with
DNA ranking the most reliable, followed by
fingerprint evidence (2), ballistics evidence (3), and
Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1
& Liu
forensic chemistry (4). In this section, we discuss these
general impressions, followed by an analysis of the
evaluative processes that judges employ in assessing
reliability. We then discuss prevalent judicial concerns
regarding scientific evidence and judicial credibility
assessments. We conclude the results with a
discussion of how judges' personal experiences
influence their evaluative processes.
General Impressions, By Evidentiary Type
DNA Evidence
The vast majority of our respondents
perceive DNA evidence as the strongest, most
objective form of evidence available: "DNA is 100%
reliable . . . . It comes back so many billion to one"
(Judge 9). Among a host of positive terms used by
judges to portray their perceptions of DNA evidence,
descriptors include "a rock crusher" (Judge 26),
"infallible" (Judge 11), "bulletproof' (Judge 34), and
"the golden child" (Judge 7). Given its perceived
value, most judges expressed a strong preference to
have DNA as part of the evidentiary package, when it
is relevant and available: "I always prefer to have
certainty, and you have that with DNA evidence. For
truth seeking courts, it is always best to be sure"
(Judge 11). Anotherjudge described the value of DNA
in criminal case processing this way: "When you get
an expert up there who says that 'with 99.7% certainty
they can rule out A, B, and C,' you have to give it
weight" (Judge 18). Consistent with existing
scholarship, the majority of our respondents not only
praise DNA for its evidentiary strength, but also accept
the applicable science and testing procedures without
question (see Saks & Faigman, 2008), a few citing its
strong reputation.
FingerprintEvidence
Though generally ranked right below DNA in
evidentiary strength, most judges view fingerprint
evidence as a highly probative and reliable form of
evidence (see Smith et al., 2011):
Fingerprint evidence, when presented by
responsible witnesses who are experienced
and it is something where the jury can
actually see how the prints match up and the
number of points of comparison and all of
that, is very strong evidence. (Judge 26)
In describing their perceptions of its strength, many
judges noted that fingerprint evidence is highly
instrumental in resolving identification issues
(particularly in circumstantial cases), is visually
approachable and understandable (to juries and
judges), and usually generates considerable juror
interest.
 NIR & Li 7

Ballistics Evidence on their own experiences, judges routinely referenced

the strong reputation of DNA evidence in resolving
Ranked third of all forensic evidence, most
identification issues, external to their experiences on
judges perceive ballistics evidence to be a relatively
the bench. Likewise, judges connected the discussion
reliable evidentiary form. Judges tend to divide expert
of chemist evidence with the dated scandal-tainted
ballistic testimony into two general categories-gun
reputation of the work in the jurisdiction. In fact, while
operability and ammunitions/weapons matching.
judges expressed outrage at past mistakes made by
While the vast majority of judges perceive gun
chemists, virtually all of these comments were based
operability evidence to be highly reliable and objective
on information judges received outside of their
(a couple describing it as "kosher" [Judge 2] or "based
courtrooms, even though they were reminded to
on some physical reality as opposed to statistical
comment on their direct experience.
analysis" [Judge 36]), judges expressed far less
Second, evidentiary rankings were
confidence in the scientific procedures involved in
influenced by the (perceived) degree that the evidence
matching shell casings to guns: "In terms of gun
relies on human evaluations, with more required
operability, ballistics evidence is purely scientific and
human assessments leading to a decrease in judicial
reliable. But if you are trying to match gyrations on
confidence in the evidentiary form. For example,
shells, it is somewhat subjective" (Judge 11).
DNA was touted as the most reliable evidentiary type
Forensic Chemistry due partially to judicial perceptions that DNA
evidence is largely based on computer generated
Judicial opinions regarding the reliability of
results and, therefore, relatively free of the perils of
chemist testimony run a wide gamut. On one end of
human assessments. In contrast, most judges
the spectrum, forensic chemistry is subject to far more
expressed less confidence in fingerprint evidence than
criticism than DNA, fingerprint, or ballistic evidence,
DNA, partially because it requires subjective
with about a quarter of the judges arguing that it is
assessments by human experts (i.e., regarding whether
unreliable. Interestingly, perceptions of the general
there is a match; see Dror, 2013, for a related
reliability of forensic chemistry evidence are more
discussion). As one judge stated,
focused on the work quality of the particular expert
than the other three evidentiary forms: the DNA goes in the computer and it's a
match by the computer and it makes the
So many of the chemists are screwy. They
match very accurately. With fingerprints, the
have had issues in a lot of the labs. They
computer says it is a match and a human
haven't met standards. Some of the chemists
being has to look with comparison
have fudged the numbers. It gets a little hairy.
microscopes and make a determination so
You have no way of knowing as you sit here
there is more room for human error. (Judge
that this is reliable or unreliable. I have seen
9)
them effectively impeached. (Judge 5)
Notably, while several judges explained that a
Presenting a very different perspective, several judges
computer's ability to make accurate matches far
in the jurisdiction expressed confidence in the overall
exceeds that of human experts, these same judges
scientific procedures employed by chemists appearing
conceded that they do not understand the computer
in their courtrooms:
processes involved.
The chemist comes in and they talk about the
Evaluative Processes
presumptive test and then about the gas test
and how they confirm that it is cocaine or it's In assessing the reliability of different
heroin or whatever. That is very routine stuff evidentiary forms, judges were encouraged and
and I have never seen anybody effectively afforded the opportunity to freely discuss their
challenge one of the police lab chemists on a evaluative processes. As our analyses reveal, certain
simple drug analysis. (Judge 27) evaluative patterns emerged among judges. First, we
observed that increased understanding of the scientific
Common Themes methods commonly utilized for a given type of
Two prevalent themes (regarding these evidence was associated with judges being more
general perceptions) warrant mention. First, judicial critical of these processes, whereas a lack of
notions of evidentiary reliability are heavily dependent understanding was often coupled with a higher degree
on judges' perceptions of the general reputation of the of acceptance. For example, judges were more focused
evidence, as opposed to their own courtroom on the scientific procedures employed in fingerprint
experiences. For example, despite our request that analyses than DNA; judges explained the various
judges discuss their evaluation of the evidence based points of comparison and quality of the prints, were

Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1

8 NIR & Liu
aware of the comparative methods employed by
experts in the field and noted the susceptibility of the
evidence to subjective and potentially unreliable
assessments. Likewise, judges were somewhat
focused on the methodologies (e.g., visual
comparisons) used to match shell casings to guns in
evaluating ballistics evidence; this increased
understanding was accompanied by greater critiques
regarding the methodologies and conclusions rendered
by ballistics experts. In contrast, despite their high
regard for DNA evidence, almost a quarter of the
judges openly admitted that they did not fully
understand how DNA evidence works or the specific
methodologies involved (for similar discussions, see
McQuiston-Surrett & Saks, 2009; Saks & Faigman,
2008; Wojcikiewicz, 2013); while discussions
regarding the reliability of DNA evidence were filled
with praise, they involved virtually no critical analysis
or even reference to scientific methods.
Second, in responding to questions regarding
the reliability of different forms of evidence,
assessments often centered around the showmanship,
communication skills, and likeability of testifying
experts rather than on the scientific basis of the
testimony (see Brodsky et al., 2010). For example, in
response to our question "what is your perception of
DNA evidence?" one judge noted: "Some explain it
better than others. Some just have a little bit more flair
to their testimony because it is pretty dry stuff' (Judge
34). Another judge described the importance of
reaching the jury: "I think that if the expert is hyper
technical it goes right over everyone's head. The best
experts are those that make it comprehensible to the
ordinary juror" (Judge 9). A third judge assessed the
overall communication skills and protocols of most
DNA experts appearing in his courtroom:
Most of them that I have seen have been from
the Office of the Chief Medical Examiner.
They have very clear English-speaking
witnesses that can articulate what this is, how
it works and what it means. . . . They've
developed a script that is capable of
explaining this very complicated stuff to the
jury in a very simple way. (Judge 19)
Notably, none of our questions prompted our
respondents to comment on the expert's
communication skills or showmanship; our inquiries
were focused on reliability and credibility
determinations applicable to admissibility decisions.
Similarly, assessments regarding fingerprint
testimony often led with critiques about the
presentation skills of the expert witness; a few judges
openly explained that this focus is due to their lack of
knowledge about the scientific processes:
Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1
There is skill in the laboratory and there is
skill in testifying. I can only judge their skill
in testifying. There are some people who can
communicate more effectively than others,
but I don't know if that means they're less
skilled at achieving the right result. (Judge
24)
Likewise, conversations pertaining to ballistics
evidence focused on the testifying skill of the expert.
In general, judges were highly complementary of
ballistics experts, several noting less variation in
quality among ballistics experts than fingerprint
experts: "The same witnesses come in all of the time.
It's almost like you press play and they say what they
say" (Judge 5). A second judge stated: "They testify
very well because that's all that they do. So, I think
that their testimony is clean and succinct and
persuasive" (Judge 28). This focus on presentation
skills was also prevalent in analyses of the reliability
of forensic chemistry, where judges were focused on
the inability of expert chemists to effectively
communicate with the jury.
We observed a third pattern in judges'
evaluative processes. Our respondents often focused
on the weight of the evidence (a task for the factfinder)
rather than its admissibility (a task for the gatekeeper).
For example, some judges assessed the substantive
limitations of DNA in criminal trials, extrinsic to the
reliability of the scientific evidence itself. Specifically,
about one third of the judges explained that DNA
evidence is often only one piece of the puzzle and
cannot establish all of the elements of the crime:
DNA doesn't necessarily determine the case;
it only determines defendant's presence.
What happened and everything else is still up
for grabs. Sometimes you get DNA from a
woman who says that she was raped; you still
have to show that it's not consensual. That's
not the end of the case. (Judge 9)
Similarly, judges noted the evidentiary limitations of
fingerprint analyses in establishing the required
elements of the charged crime. In particular, while
judges perceive that fingerprint evidence is highly
probative for identifying perpetrators in circumstantial
cases (e.g., it can place a defendant at the scene of the
crime), it doesn't necessarily determine guilt: "Just
because you find somebody's fingerprints, that in and
of itself doesn't necessarily mean anything unless it
ties in with other evidence" (Judge 40; i.e., there may
be another explanation for defendant's presence at the
scene). Likewise, judges emphasized that ballistics
evidence is only a small piece of the evidentiary
puzzle. Specifically, while evidence relating to gun
operability is usually highly reliable, weapon
operability is rarely an issue at trial: "I have never
 NIR & Liu 9

found it to determine guilt or innocence" (Judge 7). Variations in Quality of Experts

Finally, judges emphasized that chemist testimony is
With heightened concern regarding the
usually not dispositive of case outcome.
subjective aspects of fingerprint evidence, judges
Finally, a fourth area of emphasis in
focused on the quality of the fingerprint expert: "It all
evaluating the reliability of scientific evidence
depends on the quality of the expert, which varies. I
centered on chain of custody issues. For example, one
really think that there's a strong bearing in who
judge noted how chain of custody concerns can
collected it, how they collected it, what their
decrease the reliability of DNA evidence:
background and training is, and how they come across
You cannot challenge the science of DNA; as a witness" (Judge 30). Several judges criticized
everyone knows how reliable DNA is. I have certain police technicians who present fingerprint
seen cases where the evidence was evidence in the jurisdiction due to lack of qualified
mishandled or mixed with other fluids and experience:
then that type of issue comes up and then you
I have seen such slip shot police officers that
get a mixed sample. (Judge 9)
should never have been police officers let
Similarly, discussions about properly preserving the alone technicians taking evidence and then
chain of custody of fingerprints, ballistics evidence, trying to present it as matching or something.
and samples of drugs (e.g., by police and scientists) When they have only done 5 or 6 or 7, that
were noted during discussions about evidentiary doesn't mean that they are an expert; that just
reliability. means they happened to be there and
somebody said, 'here, you do this.' (Judge
Reference to the Frye Standard
39)
Only a minority of judges referenced the Frye
In contrast, about a third of the judges perceive that the
standard when discussing how they evaluate the
quality of fingerprint experts appearing in their
reliability of scientific evidence. Even among those
courtrooms as relatively uniform: "Distinctions in the
judges who acknowledged that Frye is the applicable
quality of these experts are minimal" (Judge 8).
standard which guides reliability determinations of
scientific evidence in the jurisdiction, their discussions Inability to Communicate Effectively
regarding reliability were dominated by other areas of
The overwhelming area of focused concern
concern.
in response to questions regarding the reliability of
Prevalent Areas of Concern chemist evidence is the inability of chemists to
effectively communicate with the jury in English: "As
Areas of expressed concerns varied by
far as the testimony is concerned, sometimes they have
evidentiary type. As discussed below, judges focused
trouble being understood because English isn't their
on the inability of attorneys to challenge DNA experts,
first language" (Judge 24). Apart from language
the variations in quality among fingerprint experts,
barriers, judges were critical of the overall ability of
and the poor communication skills of expert chemists.
chemists to grasp the attention of jurors: "There is an
These concerns are expressed below.
art to testifying that some people do not get, like the
Incompetency ofAttorneys ability to hold the jury's attention and being up on the
field" (Judge 17). Another judge simply stated:
Judges were highly critical of the inability of
"Chemists don't make the best witnesses" (Judge 6).
attorneys to challenge the reliability of DNA experts
and lamented that this widespread deficiency limits the Credibility Assessments
power of the court system to achieve its truth finding
Judges also discussed the factors they
mission: "The adversarial system fails its purpose
consider in evaluating witness credibility; consistent
when the attorney is not skilled enough to challenge
with the scholarly literature (see Brodsky et al., 2010;
the expert" (Judge 14; see Damon-Moore, 2017;
Connolly & Gordon, 2011; Porter & Brinke, 2009;
Edmond & Roque, 2012; Kovera & McAuliff, 2000;
Wessel et al., 2006), assessment tools and
Saks & Faigman, 2008). Many judges attributed this
determinants vary widely and similar facts were
inability to attorneys' lack of scientific knowledge (see
interpreted differently by various judges. In fact, a
Edmond & Roque, 2012; Saks & Faigman, 2008):
high degree of subjectivity is evident at the core of
"DNA people almost always get a free pass because
judicial credibility assessments, highlighting the
lawyers are intimidated by DNA and do not know how
potential for vastly disparate conclusions. Factors
to attack it" (Judge 16).
considered to determine credibility include motive,
body language, attitude, witness background ("you

Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1

10
don't get your players from central casting" [Judge
17]), relationship between the parties, and consistency
in testimony, among other considerations. In assessing
credibility, judges use a host of observational
techniques and similar facts may lead different judges
to opposite conclusions. For example, one judge
explained how he evaluates credibility based on the
witness's demeanor: "As a rule of thumb, if somebody
is too calm, I feel they were probably rehearsed and
maybe they weren't telling the truth" (Judge 21). In
contrast, other judges perceive nervousness negatively
and calmness as a sign of truthfulness. A few judges
described how body language can reveal dishonesty:
"I look at how they are moving their eyes. Which way
are their eyes going as they are thinking? There is a
whole science behind left bending eyes; which is recall
and which is thinking up a lie" (Judge 4). Several
judges found witnesses to lack credibility based on the
vagueness of their answers to attorney questions:
"When witnesses start being vague, instead of just
answering the question, when they beat around the
bush like politicians do, that's a red flag to me" (Judge
14). While some judges are confident in their own
ability to assess credibility, others are far less
confident: "It's a very difficult area, it's a very
subjective area and I don't claim to have any superior
ability to determine credibility in a close case. And I
have been wrong in the past" (Judge 37).
Influence of Personal Experiences
At times, judges referred to their own out-of-
court encounters in discussing their perceptions of
evidence. For some, their personal life experiences
might have influenced the manner in which they
evaluated case evidence. One judge considered his
own experience as a witness in evaluating the
credibility of an "anxious looking" witness appearing
in his courtroom: "If they were nervous, they probably
had a good reason to be. Not because they were
lying...because you should be uncomfortable, I was
once called as a witness, I was uncomfortable" (Judge
21). A second judge was not disturbed by the
prosecution's failure to produce fingerprint evidence
in a weapons case, based on his own experiences with
guns: "People don't understand that certain surfaces
do not leave fingerprints so that's a difficult hurdle for
prosecutors... I... have handled guns so I know
sometimes you leave one, sometimes you don't"
(Judge 1). Finally, a third judge was unconvinced by
the defense argument that a boxcutter could not have
caused an injury, based on his own past injury:
"Having been cut by one in the past I know it can cause
serious damage" (Judge 2). These examples
demonstrate how judicial life experiences sometimes
influence judicial assessments of evidence.
Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1
NIR & Liu
Discussion & Conclusions
Given the importance of judicial gatekeeping
decisions, reliable and streamlined evaluative
processes are critical to ensure that evidentiary
decision-making proceeds fairly and that evidence
admitted "through the gate" is reliable, relevant, and
trustworthy. Despite the importance of fair and
consistent gatekeeping processes, our study reveals
that evidentiary rulings and evaluations are sometimes
fraught with subjectivity and bias and made by
uninformed judges using inadequate criteria.
First, consistent with existing scholarship
(Allen & Nafisi, 2010; Canela et al., 2019;
McQuiston-Surrett & Saks, 2009; Neal et al., 2019;
Saks & Faigman, 2008), our data reveal that judges
often do not understand the scientific processes
involved in analyzing forensic evidence. While it is
understandable that judicial expertise lies in legal
matters and not science, those tasked to evaluate the
admissibility of scientific evidence must be able to
understand the processes that they are evaluating in
order to effectively carry out their gatekeeping
mission. This issue is exacerbated by current
documented validity and reliability concerns regarding
main feature-comparison methods in forensic science
(e.g., fallibility of DNA evidence, lack of foundational
validity of ballistic evidence, and the higher-than-
believed false positive rates of fingerprint analysis;
President's Council of Advisors on Science and
Technology, 2016). In Frye jurisdictions, how can
judges possibly evaluate whether or not the scientific
processes used are generally accepted within the
scientific community if they cannot understand the
methodologies employed by the testifying expert?
This could be more alarming in Daubertjurisdictions
where judges are further tasked with determining
reliability using evaluative criteria foreign to their area
of expertise. This lack of knowledge would be
problematic on its own, even if defense attorneys were
fully schooled in applicable scientific processes and
able to competently challenge testifying experts.
However, prior studies show-and our subjects
confirm-that this is not the case (Edmond & Roque,
2012; Saks & Faigman, 2008). In fact, while the stakes
involved in criminal cases (life, liberty) are greater
than those in civil cases (predominantly financial),
criminal cases are far less likely to have competing
expert testimony, and defense attorneys are often less
proficient in their ability to challenge scientific experts
(Dioso-Villa, 2015; Groscup et al., 2002; Risinger,
2000; Shniderman, 2017) than civil court actors.
Further, the ability of criminal defendants to obtain
competing experts may depend on their
socioeconomic positions; variances in financial
resources may increase racial disparities in
 NIR & Li
gatekeeping decisions. In the end, the combination of
judges' lack of scientific knowledge, the inability of
many defense attorneys to challenge expert witnesses,
and financial constraints preventing some defendants
from securing competing experts, may lead judges to
one-sidedly rely on prosecution experts in reaching
gatekeeping decisions.
While legislative attempts and (jurisdiction-
dependent) case precedents strive to provide guidance
by delineating criteria and standards (such as the Frye
and Daubert standards) for judges to follow,
scholarship reveals that this guidance does not
effectively direct judicial gatekeeping processes
(Cheng & Yoon, 2005; Damon-Moore, 2017; Groscup
et al., 2002; Jensen, 2003; Moreno, 2003; Neal et al.,
2019; see Beecher-Monas, 1999 for a combined use of
multiple mechanisms to facilitate judges' decision
making). In fact, only a few of our respondents even
referenced the Frye standard in discussing reliability
determinations, reinforcing prior scholarship that the
applicable gatekeeping standards (i.e. Frye or
Daubert, or other adapted variations) are not
particularly influential in reliability assessments.
Indeed, our respondents discussed many
considerations they employ in evaluating the
reliability of forensic evidence extrinsic to relevant
factors, including the showmanship and
communication skills of the testifying expert (see
Brodsky et al., 2010; O'Regan, 2017). A few judges
clearly expressed the reason for this focus: "There is
skill in the laboratory and there is skill in testifying. I
can only judge their skill in testifying" (Judge 24).
While evaluations regarding showmanship and
communication skills may touch on the witness's
effectiveness on the stand, it speaks little to the
reliability of the substance of that testimony
(O'Regan, 2017).
Extending beyond reliability assessments,
credibility determinations play a role in gatekeeping
decisions. While credibility assessments do not
generally require specific expertise beyond judicial
common sense, our data reveal that the determination
process follows a highly subjective path and that
different judges viewing similar witness aspects (e.g.,
body language) sometimes reach opposing
conclusions regarding trustworthiness. Further, our
data reveal that judges draw on their personal
experiences in assessing credibility and reliability.
Indeed, our respondents described a host of influential
life experiences that guide their determinations. The
influence of a judge's life experiences on their
perceptions and interpretations is natural; one might
even argue that judges are elected or appointed to the
bench because of their rich backgrounds that influence
their world and courtroom views. Yet, distinct life
experiences can influence judicial decisions in ways
Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1
1
11
that lead to stark differences in the application of
gatekeeping considerations. These differences
highlight monumental challenges to equitable
gatekeeping processes similar to those echoed by
Judge Marvin Frankel and others in discussing
sentencing disparities (Frankel, 1972; Lynch, 2009).
Existing scholarship is relatively silent regarding
unwarranted disparities in evidentiary rulings (in
contrast to the abundance of scholarship on sentencing
disparities). Nevertheless, evidentiary rulings may
lead to unfair outcomes that not only influence the
severity of punishment imposed post-conviction, but
also affect plea offers, verdicts, and case strategies at
earlier influential stages. Without effective advocates
or knowledgeable gatekeepers, criminal defendants
are vulnerable to damaging unreliable evidence that
unfairly weaken their position during the adversarial
process.
This study has several methodological
limitations. First, our study directly explores judicial
perceptions of four types of forensic evidence (DNA,
fingerprints, ballistics, and forensic chemistry) and
judicial credibility assessments of witnesses. Future
studies should expand the types of forensic evidence
and also study non-scientific evidence including
confessions, eyewitnesses, and other physical and
documentary evidence. Second, our respondents all
preside in a single state that employs the Frye standard
for admissibility of expert scientific evidence. Since
the "general acceptance" test in Frye is considerably
more general than the criteria established in Daubert,
it would be helpful to sample judges from Daubert(or
its variant) jurisdictions to further explore judicial
understanding and application of the relevant
standards in their jurisdictions. However, this
limitation is mitigated based on existing literature that
establishes that, regardless of the adoption of Frye or
Daubert standards, "the practical results are
essentially the same" (Cheng & Yoon, 2005: 474; see
also Faigman et al., 2002; Giannelli, 2003). Finally,
our study sample is limited to judges; future studies
should explore the perceptions of other court actors
regarding judicial gatekeeping decisions.
References
Allen, R. J., & Nafisi, E. (2010). Daubert and its
discontents. Brooklyn Law Review, 76(1), 131-
166.
Alter, Z. (2017). Unpacking Frye-Mack: A critical
analysis of Minnesota's Frye-Mack Standard for
admitting scientific evidence. Mitchell Hamline
Law Review, 43, 626-667.
12
Beecher-Monas, E. (1998). Blinded by science: How
judges avoid the science in scientific evidence.
Temple Law Review, 71(1), 55-102.
Beecher-Monas, E. (1999). A ray of light for judges
blinded by science: Triers of science and
intellectual due process. Georgia Law Review,
33(4), 1047-1112.
Blanck, P. D. (1992). Calibrating the scales of justice:
Studying judges' behavior in bench trials. Indiana
Law Journal, 68(4), 1119-1198.
Brodsky, S. L., Griffin, M. P., & Cramer, R. J. (2010).
The witness credibility scale: An outcome
measure for expert witness research. Behavioral
Sciences and the Law, 28, 892-907.
https://doi.org/0. 1002/bsl.9 17
Burns, K. (2016). Judges, 'common sense' and judicial
cognition. Griffith Law Review, 25(3), 319-351.
https://doi.org/10.108010383441.2016.1259536
Bushway, S. D., Redlich, A. D., & Norris, R. J. (2014).
An explicit test of plea bargaining in the "shadow
of the trial". Criminology, 52(4), 723-754.
https://doi.org/10.1111/1745-9125.12054
Butt, L. (2013). The forensic confirmation bias:
Problems, perspectives, and proposed solutions-
Commentary by a forensic examiner. Journal of
Applied Research in Memory and Cognition, 2,
59-60.
https://doi.org/10.1016/i.ia c.2013.01.012
Canela, C., Buadze, A., Dube, A., Jackowski, C.,
Pude, I., Nellen, R., Signorini, P., & Liebrenz, M.
(2019). How do legal experts cope with medical
reports and forensic evidence? The experiences,
perceptions, and narratives of Swiss judges and
other legal experts. Frontiers in Psychiatry,
10(18), 1-13.
https://doi.org/10.3389/fpsat.2019.00018
Cheng, E. K., & Yoon, A. H. (2005). Does Frye or
Daubert matter? A study of scientific
admissibility standards. Virginia Law Review,
91(2), 471-513.
Cole, S. A. (2009). Forensics without uniqueness,
conclusions without individualization: The new
epistemology of forensic identification. Law,
Probability and Risk, 8(3), 233- 255.
https:H/doi.org/10. 1093/lpr/mgpo 16
Connolly, D. A., & Gordon, H. M. (2011). He-said-
she-said: Contrast effects in credibility
assessments and possible threats to fundamental
principles of criminal law. Legal and
Criminological Psychology, 16, 227-241.
https://doi.org/10.1348/135532510X498185
Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1
NIR & Liu
Damon-Moore, S. L. (2017). Trial judges and the
forensic science problem. New York University
Law Review, 92(5), 1532-1570.
Daubert v. Merrell Dow Pharmaceuticals, 509 U.S.
579 (1993).
DeMatteo, D., Fishel, S., & Tansey, A. (2019). Expert
evidence: The (unfulfilled) promise of Daubert.
Psychological Science in the Public Interest,
20(3), 129-134.
htps://doi.org/10. 1177/1529100619894336
Derwin, A. (2018). The judicial admission of flawed
expert opinion evidence informing wrongful
convictions. Western Journal of Legal Studies,
8(2), 1-19.
Dezember, A., & Redlich, A. D. (2019). Plea
bargaining in the shadow of the trial. In C. Spohn
& P. Brennan (Eds.), Handbook on Sentencing
policies and practices in the 21st century (pp.
168-187). Routledge.
Dickson, J. (2019). Daubert won't do: Why expert
testimony regarding future dangerousness
requires new rule of evidence. Georgetown Law
Journal, 107(2), 481-499.
Dioso-Villa, R. (2016). Is the expert admissibility
game fixed? Judicial gatekeeping of fire and arson
evidence. Law & Policy, 38(1), 54-80.
https://doi.org/10. 111 1/lapo.12047
Dror, I. (2013). The ambition to be scientific: Human
expert performance and objectivity. Science and
Justice, 53, 81-82.
littps://doi.org/10. 1016/j scijus 20 13.03.002
Dror, I. E., & Cole, S. A. (2010). The vision in "blind"
justice: Expert perception, judgment, and visual
cognition in forensic pattern recognition.
Psychonomic Bulletin & Review, 17(2), 161-167.
https://doi.org/10.3758/PBR.172.161
Dror, I. E., Champod, C., Langenburg, G., Charlton,
D., Hunt, H., & Rosenthal, R. (2011). Cognitive
issues in fingerprint analysis: Inter-and intra-
expert consistency and the effect of a 'target'
comparison. ForensicScience International,208,
10-17.
https://doi.org/10. 1016/j.forsciint.2010. 10.013
Edmond, G., & Roque, M. S. (2012-2013). The cool
crucible: Forensic science and the frailty of the
criminal trial. CurrentIssues in CriminalJustice,
24, 51-68.
https://doi.org/10.1080/10345329.2012.1203594
4
 NIR & Liu 13

Ekman, P., & Friesen, W. V (1969). Nonverbal Giannelli, P. C. (2003). Admissibility of scientific
leakage and clues to deception. Psychiatry: evidence. Oklahoma City University Law
Journalfor the Study ofInterpersonalProcesses, Review, 28, 1-16.
32(1), 88-106.
Giannelli, P. C. (2007). Wrongful convictions and
https://doi.org/1.1080/0033274, 1969.1102357
forensic science: The need to regulate crime labs.
5
North CarolinaLaw Review, 86(1), 163-236.
Elwell, D. A. (2020). Industry-influenced evidence:
Greenstein, R. K. (2009). Determining facts: The myth
Bias, conflict, and manipulation in scientific
of diect evidence. Houston Law Review, 45(5),
evidence. Boston College Law Review, 61(6),
1801-1830.
2155-2190.
Grimm, P. W. (2018). Challenges facing judges
Faigman, D. L., Kaye, D. H., Saks, M. J., & Sanders,
regarding expert evidence in criminal cases.
J. (2002). Modern scientific evidence: The law
Fordham Law Review, 86(4), 1601-1616.
and science of expert testimony (Vol. 1). West
Group. Groscup, J. L., Penrod, S. D., Studebaker, C. A., Huss,
M. T., & O'Neil, K. M. (2002). The effects of
Faigman, D. L., Slobogin, C., & Monahan, J. (2016).
Daubert on the admissibility of expert testimony
Gatekeeping science: using the structure of
in state and federal criminal cases. Psychology,
scientific research to distinguish between
Public Policy, and Law, 8(4), 339-372.
admissibility and weight in expert testimony.
httips:H/doi.org/10. 1037//1076-8971.8.4.339
Northwestern University Law Review, 110(4),
859-904. Heller, K. (2006). The cognitive psychology of
circumstantial evidence. Michigan Law Review,
Federal Bureau of Investigation (FBI). (2019).
105(2), 241-306.
Handbook of forensic services.
http://www.f bi.gov/about-us/lab/a ndbook-of- Jensen, P. J. (2003). Frye versus Daubert: Practically
forensic-services-pdf the same. Minnesota Law Review, 87(5), 1579-
1620.
Federal Bureau of Investigation (FBI). (2020).
Frequently asked questions on CODIS and NDIS. Jurs, A. (2009). Judicial Analysis of Complex

httips://www .fbi.pov/services/laboratory/biometri
c-analysis/codis/codis-and-ndis-fact-shect
Federal Rules of Evidence (2021). Rule 702
-
&
Cutting-Edge Science in the Daubert Era:
Epidemiologic Risk Assessment as a Test for
Case for Reform Strategies. Connecticut Law

Review, 42(1), 49-100.

Testimony by Expert Witnesses.
htts://www.rulesofevidence.org/article-vii/rule- Kassin, S. M., Dror, I. E., & Kukucka, J. (2013). The
702/ forensic confirmation bias: Problems,
perspectives, and proposed solutions. Journal of
Findley, K. A. (2013). Judicial gatekeeping of suspect
Applied Research in Memory and Cognition, 2(1),
evidence: Due process and evidentiary rules in the
42-52.
age of innocence. GeorgiaLaw Review, 47, 723-
https://doi.org/10.1016/i.iarmac.2013.01.001
774.
Kovera, M. B., & McAuliff, B. D. (2000). The effects
Fradella, H. F., O'Neill, L., & Fogary, A. (2004). The
of peer review and evidence quality on judge
impact of Daubert on forensic science.
evaluations of psychological science: Are judges
PepperdineLaw Review, 31(2), 323-362.
effective gatekeepers? Journal of Applied
Frankel, M. E. (1972). Lawlessness in sentencing. Psychology, 85(4), 574-586.
University of CincinnatiLaw Review, 41(1), 1-54. https:doi.org/10. 1037/002-9010.85.4.574
Frye v. United States, 293 F. 1013 (D.C. Cir, 1923). Krafka, C., Dunn, M. A., Johnson, M. T., Cecil, J. S.,
& Miletich, D. (2002). Judge and attorney
Gatowski, S. I., Dobbin, S. A., Richardson, J. T.,
experiences, practices, and concerns regarding
Ginsburg, G. P., Merlino, M. L., & Dahir, V.
expert testimony in federal civil trials.
(2001). Asking the gatekeepers: A national survey
Psychology, Public Policy, and Law, 8(3), 309-
of judges on judging expert evidence in a post-
332. https://doi.org/10.1037/1076-8971.8.3.309
Daubertworld. Law and Human Behavior, 25(5),
433-458.
https://doi.org/10.1023/A:1012899030937
General Electric Co. v. Joiner, 522 U.S. 136 (1997).

Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1

14
Kramer, J. H., & Ulmer, J. T. (2002). Downward
departures for serious violent offenders: Local
court "corrections" to Pennsylvania's sentencing
guidelines. Criminology, 40(4), 897-932.
https://doi.org/10.1111/i.1745-
9125.2002.tb00977.x
Kramer, G., Wolbransky, M., & Heilbrun, K. (2007).
Plea bargaining recommendations by criminal
defense attorneys: Evidence strength, potential
sentence and defendant preferences. Behavioral
Sciences and the Law, 25, 573-585.
https://doi.org/10.1002/bsl.759
Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999).
Landes, W. (1971). An economic analysis of the
courts. Journal of Labor Economics, 14(1), 61-
107. http://dx.doi.org/10.1086/466704
Levon, E., & Ye, Y. (2020). Language, indexicality
and gender ideologies: contextual effects on the
perceived credibility of women. Gender and
Language, 14(2), 123-151.
https://doi.org/10.1558/gen1.39235
Liden, M., Grans, M., & Juslin, P. (2019). 'Guilty, no
doubt': detention provoking confirmation bias in
judges' guilt assessments and debiasing
techniques. Psychology, Crime & Law, 25(3),
2 19-247.
Lynch, G. E. (2009). Marvin Frankel: A reformer
reassessed. Federal Sentencing Reporter, 21(4),
235-241.
https://doi.org/10.1525/fsr.2009.21.4.235
Maxwell, J. A. (2005). Qualitative research design:
An interactive approach (2 "a ed.). Sage
Publications, Inc.
McQuiston-Surrett, D., & Saks, M. J. (2009). The
testimony of forensic identification science: What
expert witnesses say and what factfinders hear.
Law and Human Behavior, 33, 436-453.
https://doi.org/10.1007/s10979-008-9169-1
Moreno, J. (2003). Einstein on the bench: Exposing
what judges do not know about science and using
child abuse cases to improve how courts evaluate
scientific evidence. Ohio State Law Journal,
64(2), 531-584.
Moulin, V., Mouchet, C., Pillonel, T., Gkotsi, G. M.,
Baertschi, B., Gasser, J., & Teste, B. (2018).
Judges' perceptions of expert reports: The effect
of neuroscience evidence. InternationalJournal
of Law and Psychiatry, 61, 22-29.
https://doi.org/10.1016/i.ijlp.2018.09.008
Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1
NIR & Liu
Nardulli, P. F. (1986). Insider justice: Defense
attorneys and the handling of felony cases.
Journal of Criminal Law & Criminology, 77,
379-417. https://doi.org/10.2307/1143338
National Academy of Sciences (NAS). (2009).
Strengthening forensic science in the United
States: A path forward.
https://doi.org/10. 17226/12589
Neal, T. M., Slobogin, C., Saks, M. J., Faigman, D. L.,
& Geisinger, K. F. (2019). Psychological
assessments in legal contexts: Are courts keeping
"junk science" out of the courtroom?
Psychological Science in the Public Interest,
20(3), 135-164.
https://doi.org/10.1177/1529100619888860
Nir, E. (2018). Approaching the bench: accessing
elites on the judiciary for qualitative
interviews. International Journal of Social
Research Methodology, 21(1), 77-
89. https://doi.org/10.1080/136455792017.1324
669
Nir, E., & Griffiths, E. (2018). Sentencing on the
evidence. CriminalJustice Policy Review,
29(4), 365-390.
https: //doi. org/ 10. 1177/088740341663 5248
Nir, E., & Griffiths, E. (2019). The thirteenth
juror. The British Journalof Criminology, 59(2),
315-333. https:doiorg/10.1093/bic/azyO29
Nir, E., & Liu, S. (in press). Defending constitutional
rights in imbalanced courtrooms. Journal of
CriminalLaw and Criminology.
O'Brien, T. L. (2018). Gender, expert advice, and
judicial gatekeeping in the United States. Social
Science Research, 72, 134-145.
https:H/doi.org/10. 10 16/j. ssresearch.20 18.02.011I
O'Regan, D. (2017). Eying the body: The impact of
classical rules for demeanor credibility, bias, and
the need to blind legal decision makers. PaceLaw
Review, 37(2), 379-454.
Patton, M. Q. (2002). Qualitative research and
evaluation methods (3 rd ed.). Sage Publications.
Peterson, J. L., Sommers, I., Baskin, D., & Johnson,
D. (2010). The role and impact of forensic
evidence in the criminal justice process. U.S.
Department of Justice.
https://www.ojp.gov/pdffilesl/rdj/grants/231977.
pdf
 NIR & Liu
Peterson, J. L., Hickman, M. J., Strom, K. J.,
&
Johnson, D. J. (2013). Effect of forensic evidence
on criminal justice case processing. Journal of
Forensic Science, 58, S78-S90.
https://doi.org/10. 1111/1556-4029.12020
Porter, S., & Brinke, L. T. (2009). Dangerous
decisions: A theoretical framework for
understanding how judges assess credibility in the
courtroom. Legal and Criminological
Psychology, 14, 119-134.
https://doi.org/10.1348/135532508X281520
President's Council of Advisors on Science and
Technology. (2016). Report to the President,
Forensic Science in Criminal Courts: Ensuring
Scientific Validity of Feature-comparison
Methods. Executive Office of the President of the
United States.
https://obamawhitehouse.archives. gov/sites/defa
ult/files/rnicrosites/ost2/PCAST/pcast forensic s
cience report finalgpdf
Redlich, A. D., Bushway, S. D., & Norris, R. J. (2016).
Plea decision-making by attorneys and judges.
JournalofExperimentalCriminology, 12(4), 537-
561. https://doi.org/10.1007/si1292-016-9264-0
Risinger, D. (2000). Navigating expert reliability: Are
criminal standards of certainty being left on the
dock. Albany Law Review, 64(1), 99-152.
Rossman, H. H., McDonald, W. F., & Cramer, J. A.
(1980). Some patterns and determinants of plea-
bargaining decisions: A simulation and quasi-
experiment. In W. F. McDonald & J. A. Cramer
(Eds.), Plea bargaining(pp. 77-114). Lexington
Books.
Rozelle, S. D. (2007). Daubert, Schmaubert: Criminal
defendants and the short end of the science stick.
Tulsa Law Review, 43(2), 597-608.
Saks, M. J., & Faigman, D. L. (2008). Failed forensics:
How forensic science lost its way and how it
might yet find it. Annual Review of Law and
Social Science, 4, 149-171.
https://doi.org/10.1146/annurev.lawsocsci.4.110
707.172303
Schweitzer, N. J., & Saks, M. J. (2009). The
gatekeeper effect: The impact of judges'
admissibility decisions on the persuasiveness of
expert testimony. Psychology, PublicPolicy, and
Law, 15(1), 1-18.
https://doi.org/10.1146/annurev.lawsocsci.4.110
707172303
Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1
15
Shelton, D. E. (2009). Twenty-First Century forensic
science challenges for trial judges in criminal
cases: Where the polybutadiene meets the
bitumen. Widener Law Journal, 18, 309-396.
Shelton, D. E. (2010). Forensic science evidence and
judicial bias in criminal cases. The Judges'
Journal,49(3), 18-24.
Shniderman, A. B. (2017). Prosecutors respond to
calls for forensic science reform: More sharks in
dirty water. Yale Law Journal Forum, 126, 348-
361.
Smalarz, L., Madon, S., Yang, Y., Guyll, M., & Buck,
S. (2016). The perfect match: Do criminal
stereotypes bias forensic evidence analysis? Law
and Human Behavior, 40(4), 420-429.
https://doi.org/10.1037/lhb0000190
Smith, S. M., Stinson, V., & Patry, M. W. (2011). Fact
of fiction? The myth and reality of the CSI effect.
CourtReview, 47, 4-7.
Stevenage, S. V., & Bennett, A. (2017). A biased
opinion: Demonstration of cognitive bias on a
fingerprint matching task through knowledge of
DNA test results. ForensicScience International,
276, 93-106.
https://doi.org/10. 1016/jforsciint.2017.04.009
Thompson, S. G. (2012). Judicial gatekeeping of
police-generated witness testimony. Journal of
CriminalLaw & Criminology, 102(2), 329-396.
Trozzi, T. A., Schwartz, R. L., Hollars, M. L.,
Leighton, L. D., Trozzi, Y. E., & Wade, C. (2001).
Processing guide for developing latent prints.
U.S. Department of Justice, Federal Bureau of
Investigation.
hittps ://archives fbi. gov/archives/aboutt-
us/lab/forensic-science-
cominunications/fsc/ an2001/lpu.pdf
Yan, S. (2019) Estimating the size of plea discounts:
Why does it matter? In C. Spohn & P. Brennan
(Eds.), Handbook on sentencing policies and
practices in the 21st century (pp. 188-207).
Routledge.
U. S. Congress Committee on the Judiciary. (2014).
Federal Rules of Evidence (Rule 702, p. 15).
URL:
https://www.uscourts.gov/sites/default/files/Rule
sO 2Oof o20Evidence. Retrieved on Mar. 24,
2021.
Walsh, J. T. (1999). Keeping the gate: The evolving
role of the judiciary in admitting scientific
evidence. Judicature, 83(3), 140-143.
16 NIR & Liu

Wessel, E., Drevland, G. C., Eilertsen, D. E., About the Authors

&
Magnussen, S. (2006). Credibility of the
emotional witness: A study of ratings by court
judges. Law Human Behavior, 30, 221-230.
https:Hdoi.org/10. 1007/s1O979-006-9024-
Wilson, M. J. W. (2016). Defense attorney bias and the
rush to the plea. University of Kansas Law
Review, 65(2), 271-326.
https://doiorg/1O. 1716 1/1808.25553
Wojcikiewicz, J. (2013). Judges' attitude towards
scientific evidence. Revija za Kriminalistiko in
Kriminologijo, 64, 249-255.
Esther Nir is an Associate Professor in the
Department of Criminal Justice at New Jersey
City University. She received her doctoral degree
from Rutgers University School of Criminal
Justice and her law degree from Fordham Law
School. She previously served as a prosecutor in
Queens County, New York. Her research interests
include sentencing disparities, police misconduct,
qualitative methods, and community engaged
learning initiatives. Her research can be found in
various criminal justice and research
methodology journals including The British
Journal of Criminology, CriminalJustice Policy
Review, The International Journal of Social
Research Methodology, International Journal of
Police Science and Management, and others.
Siyu Liu is an Assistant Professor of Criminal Justice
in the School of Public Affairs at Penn State
Harrisburg. Her research is currently focusing on
police legitimacy, juvenile delinquency, and the
Chinese death penalty. Her work has appeared in
Journal of Quantitative Criminology, Criminal
Justice and Behavior, Psychology, Crime and
Law, Asian Journal of Criminology, and others.
She received her doctoral degree in 2014 from the
School of Criminal Justice, State University of
New York at Albany.

Endnotes

In contrast to jury trials, judges in bench trials are both gatekeepers of the evidence and triers of the fact.
2 Subsequently, the Supreme Court underlined the limited scope of appellate review in judicial gatekeeping
decisions and emphasized the wide discretionary power of trialjudges in the evaluation of methodology, as well
as the expert's reasoning (General Electric Co. v. Joiner, 1997); the Court further expanded the trial judge's
authority based on Daubert beyond scientific testimonies to all expert testimonies (Kumho Tire Co v. Carmichael,
1999).
3 While only 7 of our 41 respondents are female, this number roughly reflects the percentage of female judges
practicing in the state at that time, by district.
4 Since our data were collected, the applicable rules and laws of evidence admission are substantially the same.

Criminology, CriminalJustice, Law & Society - Volume 22, Issue 1