Original Article

Evolutionary Psychology
January-March 2018: 1–10
Exploring the Great Schism in the Social ª The Author(s) 2018
Reprints and permissions:
sagepub.com/journalsPermissions.nav
Sciences: Confirmation Bias and the DOI: 10.1177/1474704917752691
journals.sagepub.com/home/evp
Interpretation of Results Relating to Biological
Influences on Human Behavior and Psychology

Jeffrey Winking1

Abstract
The nature–nurture debate is one that biologists often dismiss as a false dichotomy, as all phenotypic traits are the results of
complex processes of gene and environment interactions. However, such dismissiveness belies the ongoing debate that is
unmistakable throughout the biological and social sciences concerning the role of biological influences in the development of
psychological and behavioral traits in humans. Many have proposed that this debate is due to ideologically driven biases in the
interpretation of results. Those favoring biological approaches have been accused of a greater willingness to accept biological
explanations so as to rationalize or justify the status quo of inequality. Those rejecting biological approaches have been accused of
an unwillingness to accept biological explanations so as to attribute inequalities solely to social and institutional factors, ultimately
allowing for the possibility of social equality. While it is important to continue to investigate this topic through further research
and debate, another approach is to examine the degree to which the allegations of bias are indeed valid. To accomplish this, a
convenience sample of individuals with relevant postgraduate degrees was recruited from Mechanical Turk and social media.
Participants were asked to rate the inferential power of different research designs and of mock results that varied in the degree to
which they supported different ideologies. Results were suggestive that researchers harbor sincere differences of opinion con-
cerning the inferential value of relevant research. There was no suggestion that ideological confirmation biases drive these dif-
ferences. However, challenges associated with recruiting a large enough sample of experts as well as identifying believable mock
scenarios limit the study’s inferential scope.

Keywords
confirmation bias, academia, evolutionary studies, cognitive bias, MTurk

Date received: April 13, 2017; Accepted: December 13, 2017

The nature–nurture debate is one that biologists often dismiss
as a false dichotomy, as all phenotypic traits are the results of
complex processes of gene and environment interactions
playing out across numerous dimensions. Therefore, all traits
might be considered to be due to 100% nature and 100%
nurture. However, such dismissiveness belies the long-
standing debate that is unmistakable throughout the biological
and social sciences concerning the role of biological influ-
ences in the development of psychological and behavioral
traits in humans (e.g., Buller, 2006; Horowitz & Yaworsky,
2014; Marks, 2012; McKinnon & Silverman, 2005b; Pinker,
2002; Rose & Rose, 2001). Indeed, this debate divides
many disciplines and has at times proven quite contentious
(Segerstrale, 2000).
Much of the theory and methods employed by biologically
oriented social scientists flows from the animal literature.
When human studies suggest homologous or analogous fea-
tures, however, explanations invoking biological pathways or
adaptive functions tend to be less readily accepted for human
traits than for animal traits (e.g., for incest avoidance, see
1
Department of Anthropology, Texas A&M University, College Station,
TX, USA
Corresponding Author:
Jeffrey Winking, Department of Anthropology, Texas A&M University,
4352 TAMU, College Station, TX 77843, USA.
Email: jwinking@tamu.edu

Creative Commons Non Commercial CC BY-NC: This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 License
(http://www.creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial use, reproduction and distribution of the work without further permission
provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage).
2 Evolutionary Psychology
Chapais, 2008; El-Guindi & Read, 2012; Kushnick & Fessler,
2011; Leavitt, 2007). This is reasonable, given that humans
exhibit an unprecedented level of behavioral plasticity, driven
by uniquely elevated capacities for fluid reasoning and social
learning (i.e., culture). Many have argued that such character-
istics make it impossible to disentangle the roles of culture and
biology in the phylogenetic story of humans and the ontoge-
netic stories of individuals (Fuentes, 2012; Marks, 2012;
McKinnon & Silverman, 2005a). Under this argument, it is not
simply that such a goal is logistically unattainable, but that such
endeavors are meaningless: In humans, the biological is inex-
tricably cultural and the cultural biological. In other academic
fields, extreme forms of social constructionism have eliminated
the consideration of biological factors entirely (Sayer, 1997).
Obviously, complex phylogenetic and ontogenetic pro-
cesses cannot be captured by the single dimension of nature
versus nurture, genetic versus environment, and so on. For
many phenomena, such a dimension would prove nonsensical
in its mutually exclusive, linear form. However, the debate that
divides the social sciences is more specific.
When biologists refer to gene–environment interactions,
many types of environmental factors can influence outcomes:
intrauterine hormonal levels, salinity of interstitial fluids,
energy availability, and so on. However, as the debate has
played out in academic journals and popular publications,
many of these factors (such as intrauterine hormonal levels)
would likely be conceptually placed on the biological side of
the ledger. The environmental factors that are relevant are
typically those relating to experiential influences—those
derived from what an individual experiences through his or her
senses through time. This includes one’s upbringing, schooling,
and cultural environment. Even when limiting the discussion to
these factors, one’s experiences and biological processes con-
tinue to interact to define an intertwined developmental trajec-
tory. However, the overarching spectrum of possibilities
becomes limited, and we can more meaningfully characterize
the debate in a more limited question: To what degree, do
biological processes constrain humans’ remarkable cultural
and behavioral responsiveness to experiential variance? And
the question explored here, therefore, is what explains the
remarkable variation in academics’ interpretations of data
relating to this question?
For those who still question the meaningfulness of suggest-
ing that a trait is more or less biologically constrained (i.e.,
environmentally canalized) than another, perhaps a look at the
tails would be helpful. There are a number of traits for which
there is less doubt that selection played a role in shaping their
expression and that genetically determined biologically pro-
cesses play a role in constraining their development. These
include the aversion to the smell of vomit on one’s food, pre-
ferring to be exposed to room temperatures rather than arctic
temperatures, perceiving the act of sex to feel more pleasant
than being stabbed, and the consequent motivation to pursue
sex more than being stabbed. On the other hand, there are many
traits for which there is little doubt that biological processes do
not play a large role in constraining their development such as
the sound used to represent the concept of “tree,” the aesthetic
preference for a certain hair length on men, and the feeling of
embarrassment when one’s bare legs are seen by others. Obvi-
ously, there are some biological constraints such as the fact that
humans must be able to produce and hear the sound represent-
ing “tree,” grow hair to a certain length, and be able to inter-
nalize norms and feel shame when they are broken. But within
those broad constraints, the possibility that any particular pre-
ference might emerge through a cultural process of historical
happenstance is indeed real.
Inferential Power (IP) of Different Study Designs
If there is fair agreement at the tails, this would suggest that
there is some degree of evidence and/or cogency of argument
that satisfies most parties and that the debate likely rests on
variance in the inferential thresholds demanded before infer-
ring biological constraints. This raises the question of how we
infer a biological influence (Andrews, Gangestad, & Matthews,
2002). A preference to not be exposed to the arctic cold is
cross-culturally universal; it appears early in development, can
be seen in other animals, and serves a clear adaptive purpose.
None of those things can be said for long hair on men. Thus,
despite not knowing the genetic or biological etiology of a
cognitive motivation to seek warmth—a common critique of
biological explanations—most are comfortable with assigning
a biological foundation to a preference for warmth over freez-
ing. There are a number of research designs (RDs) that are used
to examine the role of biology in the development of a partic-
ular trait. Referenced above, these include explorations of
cross-cultural universality (Buss, 1989; Eagly & Wood,
1999; Henrich et al., 2004), early developmental onset (Cam-
pos, Bertenthal, & Kermoian, 1992; Langlois et al., 1987;
Thompson & Newton, 2013), animal homologs/analogs (Plav-
can, 2012; Proctor, Williamson, de Waal, & Brosnan, 2013;
Sussman, 2013; Wrangham & Glowacki, 2012), and adaptive
design (Gould & Lewontin, 1979; Oaten, Stevenson, & Case,
2009; Winking, Gurven, Kaplan, & Stieglitz, 2009) but also
allelic associations (Hamer, Hu, Magnuson, Hu, & Pattatucci,
1993; Munafò, Yalcin, Willis-Owen, & Flint, 2008), hormonal
associations (Gettler, McDade, Feranil, & Kuzawa, 2011;
Mehta, Jones, & Josephs, 2008), and heritability (Polderman
et al., 2015).
There likely exists disagreement concerning the IP of these
different designs and this might explain some of the variance in
the degree to which researchers believe biology plays a role in
the development of human traits. Unfortunately, no study to my
knowledge has explored how researchers differ in their inter-
pretations according to their outlook on biological influences.
This is one of the goals of this study.
Confirmation Bias
In addition to epistemological differences, researchers on both
sides of the debate have alleged that the disagreements stem
from ideological biases. Those who believe biological
Winking
influences to be unimportant often allege that the other side is
motivated to rationalize at best, and justify at worst, current and
historical patterns of hegemony, social dominance, and
inequality. This argument harkens back to the beginning of the
sociobiology debates, summarized in the collective response of
an early Harvard study group which evaluated the merits of
sociobiology, and which included intellectual giants like
Stephen J. Gould and Richard Lewontin, “[T]hey consistently
tend to provide a genetic justification of the status quo and of
existing privileges for certain groups according to class, race,
or sex” (Allen et al., 1975). The argument continues today and
is evident in writings across multiple disciplines:
Many researchers use [evolutionary approaches to the study of
international relations] to justify the status quo in the guise of
science. (Lebow, 2013, p. 2)
The totality [of sociobiology and evolutionary psychology]
is a myth of origin that is compelling precisely because it reso-
nates strongly with Euro American presuppositions about the
nature of the world. (McKinnon & Silverman, 2005a, p. 4)
. . . in the social sciences (with the exception of primatology
and psychology) sociobiology appeals most to right-wing social
scientists. (Pavelka, 2002, p. 42)
Conversely, those who favor biological approaches have
argued that those on the other side are rendered incapable of
objective assessment by their ideological promotion of equal-
ity. They are alleged to erroneously reject evidence of biologi-
cal influences because such evidence suggests that social
outcomes are partially explained by biology, and this might
inhibit the realization of equality. Their critiques of biological
approaches are therefore often blithely dismissed as examples
of the moralistic/naturalistic fallacy. This line of reason is
exemplified in the quote by biologist Jerry Coyne
If you can read the [major Evolutionary Psychology review
paper] and still dismiss the entire field as worthless, or as a
mere attempt to justify scientists’ social prejudices, then I’d
suggest your opinions are based more on ideology than judi-
cious scientific inquiry. (Coyne, 2012)
Again, such claims are quite common:
For critics, the heart of the intellectual problem remains an
ideological adherence to the increasingly implausible view that
human behavior is strictly determined by socialization. . . . -
Should [social] hierarchies result strictly from culture, then the
possibilities for an egalitarian future were seen to be as open
and boundless as our ever-malleable brains might imagine.
(Horowitz & Yaworsky, 2014, p. 490)
Like the Church, a number of contemporary thinkers have
also grounded their moral and political views in scientific
assumptions about . . . human nature, specifically that there isn’t
one. (Hagen, 2005, p. 166)
Although we academics like to believe that we approach
debates with an open mind, that we objectively weigh the
3
evidential support for competing arguments, and that we are
always prepared to change our beliefs upon the presentation of
sufficient evidence, a great deal of research suggests that the
cognitive game is fixed (Nickerson, 1998; Snyder & Swann,
1978; Stanovich, West, & Toplak, 2013). Our reasoning is
often clouded by a confirmation bias—in its broad definition,
confirmation bias is the tendency to advantage preexisting
beliefs through many forms of inferential acrobatics
(Nickerson, 1998). We actively seek confirmatory evidence
and avoid disconfirmatory evidence (Jonas, Schulz-Hardt,
Frey, & Thelen, 2001; Taber & Lodge, 2006), and we recall
this confirmatory evidence more readily (Stanovich et al.,
2013). More importantly here, we accept confirmatory evi-
dence more readily and weight its inferential value more
heavily, while being overly critical or dismissive of discon-
firmatory evidence (often referred to as a “disconfirmation
bias”; Lehner, Adelman, Cheikes, & Brown, 2008; Nickerson,
1998; Taber & Lodge, 2006).
While it is important to engage in research and debate that
advances our understanding of how biological factors influence
the development of behavioral and psychological outcomes, it
is also important to examine the validity of these alleged biases.
Previous research has shown that those who self-identify as
political radicals or feminists are more likely to reject biologi-
cal influences on human traits (Horowitz & Yaworsky, 2014;
Sanderson & Ellis, 1992), while others have found academic
proponents of biological approaches to be no less liberal on
political issues (with both groups being far more liberal than
the population at large; Lyle & Smith, 2012; Tybur, Miller, &
Gangestad, 2007). However, while a finding of an association
with political outlook might suggest the possibility of an ideo-
logical bias in interpretations, it does not shed light on which
side, if either or if both, is indeed biased. To answer this ques-
tion, we must examine the interpretations of results that differ
solely on the degree to which they support or fail to support the
preexisting ideological beliefs that allegedly drive these biases.
I attempt this here.
Hypotheses
It is entirely possible that confirmation bias plays no role in
driving this disagreement and that the overarching debate in
academia is driven by sincere disagreements concerning the
inferential value of the RDs informing the debate. Although
academics might believe that inferential logic is universal and
abundantly clear, biological systems are inconceivably com-
plex, and it is unlikely that we will ever reach a consensus on
the meaningfulness of large bodies of research. While no pre-
dictions can be made concerning which RDs should be per-
ceived as holding more or less IP, we can at least predict that
those who believe biology to play a more important role (for
convenience, we will dichotomize this belief and label this
group “bioists”) should report that positive results, across a
range of RDs, would carry more inferential weight than that
reported by participants who believe biology to be less relevant
(“skeptics”; Hypothesis 1a; Table 1). Similarly, skeptics should
4 Evolutionary Psychology
Table 1. Hypotheses.
Research Design Description
Hypothesis 1a Bioists will assign a higher influential value than
skeptics to positive results of studies exploring
biological influences
Hypothesis 1b Skeptics will assign a higher influential value than
bioists to null results of studies exploring
biological influences
Hypothesis 2 Bioists will exhibit a more conservative political
orientation than skeptics
Hypothesis 3a Bioists will rate results suggesting a biological
antecedent to a socially undesirable trait higher
when that biological antecedent is reported to
exist in a higher frequency in a nonprivileged
group than those that do not mention frequencies
Hypothesis 3b Bioists will rate results suggesting a biological
antecedent to a socially undesirable trait higher
when that biological antecedent is reported to
exist in a higher frequency in a nonprivileged
group than those that report the frequency
higher in a privileged group
Hypothesis 4a Skeptics will rate results suggesting a biological
antecedent to a socially undesirable trait lower
when that biological antecedent is reported to
exist in a higher frequency in a nonprivileged
group than those that do not mention frequencies
Hypothesis 4b Skeptics will rate results suggesting a biological
antecedent to a socially undesirable trait lower
when that biological antecedent is reported to
exist in a higher frequency in a nonprivileged
group than those that report the frequency
higher in a privileged group
report that null results carry more inferential weight than bio-
ists report (Hypothesis 1b). Such effects should be evident even
when reviewing the IP of generic RD descriptions, without
viewing any specific results.
If ideological biases are driving this debate, then it must be
the case that bioists and skeptics differ in their ideological
outlook. Based on the alleged biases, we can predict that com-
pared to skeptics, bioists should report a more conservative
orientation, particularly with regard to causes of social strati-
fication and efforts to reverse it (Hypothesis 2). That is to say,
compared to skeptics, bioists will report that social outcomes
are based more on inherent differences as opposed to social
forces and less should be done to reduce inequalities.
Ideological confirmation biases would be suggested if the
inferential value assigned to the results were associated with
the degree to which those studies supported a particular ideol-
ogy, holding all else constant. For instance, results suggesting
that a socially undesirable trait was associated with a biolo-
gical antecedent that occurred more frequently in a disadvan-
taged population should evoke a positive ideological
confirmation bias in bioists and a disconfirmation bias in
skeptics. Thus, bioists should rate studies suggesting such a
scenario as having a higher inferential value than those that
report such an association but do not mention relative
frequencies in advantaged versus disadvantaged groups
(Hypothesis 3a), as well as studies that mention that such
antecedents occur in higher frequencies in advantaged groups
(Hypothesis 4a). The opposite effects should occur among
skeptics (Hypotheses 3b and 4b, respectively).
RD and Methods
A convenience sample was recruited to take part in online
Qualtrics surveys through Amazon’s MTurk and through
social media sites. Only those who had received a master’s
degree or higher in the biological sciences, social sciences, or
relevant humanities were included. Two separate surveys
were conducted, an RD survey and IP (IP) survey. In both,
those entering through MTurk were offered US$0.10 to
answer a series of eligibility questions. If they were deemed
eligible, they were allowed to continue and received a
US$1.00 bonus for their participation. Social media
announcements were made via Facebook pages of relevant
groups (Biological Anthropology Society, Evolutionary
Anthropology Society, and Evolutionary Psychology).
E-mails were also sent out to eligible acquaintances with the
instructions to forward the e-mail to anyone they thought
might be eligible. This was done in part because of the lower
number of biologically oriented researchers on MTurk. Social
media recruits were offered to take part in a lottery in which 1
in 50 would win a US$50 Amazon gift certificate. The RD
survey was conducted in June 2015, and the IP survey was
conducted from August 2015 to September 2016.
Inclusion Criteria
Only participants who reported a master’s or PhD in a relevant
field were included. These fields included the social sciences,
biological sciences, gender studies, and philosophy. A US$1.00
bonus trends toward the high side of MTurk payouts and there-
fore might lead to higher participation rate of “professional”
MTurkers—those who seek the most lucrative “hits” and mini-
mize the time they spend engaged per hit (Berinsky, Margolis,
& Sances, 2014; Wu, Corney, & Grant, 2014). Furthermore,
recruitment materials advertised the fact that only master’s or
PhD holders were eligible so as to minimize the number of
ineligible participants needlessly beginning the survey. There-
fore, in addition to the degree and discipline requirements, a
number of additional inclusion criteria were used to ensure that
participants were completing the survey appropriately.
Participants were asked two timed questions (30 s) regard-
ing knowledge specific to academia to ensure honesty concern-
ing reported educational level. One question focused on the
final research product for a master’s degree (a thesis) and the
other on the meaning of “curriculum vitae.” In a previous sur-
vey of 256 MTurk participants who were not told of any edu-
cation requirements, 22 of the 24 (92%) master’s and PhD
holders answered both of these questions correctly, while only
92 of the 232 (40%) participants of other educational levels
Winking 5

answered correctly. Only those who answered both questions Mock Results
correctly were included in the surveys used here.
The original study design included tests of both positive and
Furthermore, MTurk participants were excluded if they fin-
null results. However, the mock null results included psycho-
ished the survey in under 6 min for the RD survey and under 9
logical traits that participants found too different to be mean-
min for the IP survey. This was based on the fastest times (6.7
ingfully comparable and are not used here. Similarly, one
min for RD and 9.3 min for IP) for completion among the social
condition employing mock positive results included a psycho-
media sample, which was expected to not include
logical trait that was deemed to be significantly more complex
“professional” survey takers. An attention check question was
than the others and was excluded from the study. A full descrip-
included; however, 17% of participants who answered incor-
tion of these can be found in the Online Supplemental Material.
rectly did not significantly differ in the amount of time spent
The remaining three conditions are described below.
taking the survey, and it was therefore not used as an exclu-
The creation of the mock results was based on a number of
sionary factor. Finally, Reddit’s “HitsWorthTurkingFor” page
criteria. Psychological traits were considered based on (1) degree
was regularly searched to ensure that inclusion criteria were not
of social preference and (2) familiarity to participants while not
being discussed. The IP survey was discussed on this site 4
being the subject of a large, well-known bodies of literature
times, resulting in drastic increases in the frequency of partici-
focusing on their genetic and biological associates (e.g., general
pants. Participants who completed the survey directly after
intelligence, schizophrenia). Based on the results from the RD
such postings were excluded.
survey, mock results were created using three RDs that were
deemed to have high IP. The designs were presented pseudoran-
Surveys domly (16 sequences). As described above and in the Online Sup-
plementary Material, data from only one RD are reported on here.
The RD and IP surveys (Supplementary Figures S1 and S2)
The mock results suggesting a biological influence were
begin with four identical sections. These include questions
based on a genetic allele design, which was rated as having the
covering participants’ prior beliefs concerning the role of bio-
third highest IP out of eight. Participants were presented with a
logy in influencing human behavior and psychology, political
plot showing a substantial association (Cohen’s d ¼ .86)
beliefs, epistemological approaches, and statistical knowledge.
between a particular allele and appetitive aggression. In the
Specifically, participants were asked where they place them-
first condition, the allele was described as being found in
selves on the “general spectrum that exists in academia con-
higher frequency in populations with African ancestry. In the
cerning the role of biological factors in determining
second condition, this language was absent, and in the third
psychological and behavioral patterns in humans.” Provided
condition, it was described as being found in higher frequency
with a 4-point scale, participants were forced to choose which
in populations with European ancestry. Appetitive aggression
side of the spectrum they placed themselves. For all other
was described as “pro-active and predatory aggres-
questions with ordinal responses, participants chose from a
sion . . . associated with things such as likelihood of committing
7-point scale for scales that had a meaningful center (e.g.,
violence and positive attitudes towards warfare.” By including
extremely liberal–extremely conservative); otherwise, a 5-point
references to both “violence” and “warfare,” such a trait can be
scale was used.
reasonably aligned with negative stereotypes or perceptions
In the RD survey, participants were then presented with
associated with African American culture (Devine, 1989;
eight RDs and asked to rate the inferential value of significant
Dixon, 2008) and European history (Wolfe, 2006).
results regarding the role of biology in influencing psycholo-
After participants rated the IP of each set of results, they
gical and behavioral traits and the influential value of null
were asked how socially preferred they believed the relevant
results regarding the lack of a role of biology. Participants used
trait to be on a scale ranging from 3 to 3. They were also
a sliding scale ranging from 0 to 100.
asked how psychologically complex they believed it to be
In the IP survey, participants were instead presented with
using a 5-point scale.
three mock data plots that were designed to appear, as if they
were scanned from a legitimate academic journal or confer-
ence poster (due to reasons described in the section below, Analysis
data from only one of these proved useful). Participants then Simple comparisons of bioist and skeptic measures are analyzed
read a brief synopsis of the findings and conclusions of the using Wilcoxon rank tests for ordinal data and t tests for
study and were asked to rate the inferential value of the study continuous ratings. Multivariate analysis of variance (MANOVA)
regarding the role of biology in influencing the outcome of the tests are employed for tests involving multiple dependent ratings.
trait in question. Participants used a sliding scale from 0 to All analyses were conducted in SAS Version 9.3.
100 as in the RD survey. Participants were also asked to rate
the psychological complexity of each trait and how socially
preferred they believed the trait to be. At the end of the sur-
vey, participants were asked whether they were suspicious
Results
that the results were forged and whether they believed this Of the 327 participants who began the RD survey, 60 were
influenced their responses. excluded for not having the appropriate educational level,
6 Evolutionary Psychology

Table 2. Descriptive Statistics.

Research Design Inferential Power

Factor
Bioist Skeptic Bioist Skeptic

n 32 39 140 225
Male (%) 15 (47) 20 (51) 91 (65) 149 (66)
Female 17 (53) 19 (49) 49 (35) 76 (34)
Mean age 35.8 34.8 39.0 39.5
MTurk 7 (22) 23 (59) 120 (86) 215 (96)
Social media 25 (78) 16 (41) 20 (14) 10 (4)
Master’s 15 (47) 20 (51) 106 (76) 194 (86)
PhD 17 (53) 19 (49) 34 (24) 31 (14)
Anthropology 21 (66) 14 (36) 37 (14) 17 (8)
Biological sciences 4 (13) 8 (21) 40 (29) 22 (10)
Economics 0 3 (8) 21 (15) 42 (19)
Gender studies 0 0 1 (1) 3 (1)
Philosophy 0 2 (5) 2 (1) 5 (2)
Political science 0 4 (10) 7 (5) 19 (8)
Psychology 5 (16) 3 (8) 16 (11) 43 (19)
Sociology 0 2 (5) 2 (1) 6 (3) Figure 1. Orientation.
Social science, other 2 (6) 3 (8) 31 (22) 68 (30)
Graduate student 6 (19) 7 (18) 11 (8) 19 (8)
Faculty 15 (47) 9 (23) 13 (9) 22 (10)
Research 4 (13) 2 (5) 19 (14) 28 (12)
Academia, other 2 (6) 5 (13) 12 (9) 9 (4)
Nonacademia/research 5 (16) 16 (41) 85 (61) 147 (65)

124 were excluded for working in a field that was not included,
36 were excluded for not answering the academia questions
correctly, 15 were excluded for taking the survey too quickly,
and 21 were excluded for not finishing the survey, leaving 71
included. Of the 2,631 participants who began the IP survey,
248 were excluded for education, 848 for their field, 235 for the
academia questions, 231 for taking it too quickly, 610 for hav-
ing started the survey after a Reddit post, and 94 for not com-
pleting the survey, leaving 365 included. Table 2 presents the
descriptive results for the two data sets. The RD survey
included only one condition and thus required a much smaller
sample size. Most participants could thus be recruited from
social media. The sample also has a higher proportion who
have doctorates, are anthropologists, and are faculty.
When asked to place themselves on the broad spectrum in
academia regarding their opinion on the role of biology in
influencing the development of behavioral and psychological Figure 2. Research designs.
traits, participants appear to have segregated themselves cor-
rectly. Across the four options, there is a linear increase in the Bioists did, however, report a stronger quantitative/empirical
reported degree to which participants believe biology com- orientation as opposed to a qualitative/interpretive orientation
pared to experience to be important in the development of compared to skeptics (on a 5-point scale, bioists x ¼ 2.74,
both general traits and in sex differences (Figure 1, from IP skeptics x ¼ 2.30, Wilcoxon Z ¼ 3.66, p < .001).
survey). Because of the small sample sizes at the tails, the four
groups were collapsed into two groups for both surveys: bio-
ists and skeptics. IP of RDs
Exploring only the IP survey, there was no difference in the Figure 2 presents the reported IP of eight different RDs in
reported familiarity with statistical tests across four areas (gen- descending order of pooled average for both significant (a) and
eral statistical knowledge, t tests, correlations, and p values). null (b) results. The eight RDs are described in Table 3. There
Winking
Table 3. Research Designs.
Research Design Description
Cross-cultural A trait or pattern exists universally among
universality cultures . . .
Twin studies Identical twins reared together are . . . compared
to identical twins reared apart . . . [and] are
also compared to adopted siblings reared
together.
Early developmental A trait appears very early in the development of
onset infants . . .
Genetic variants A particular genetic variant is found to be
associated with a trait . . .
Sex hormones within Fetuses of a particular genetic sex are exposed to
the uterus levels of sex hormones within the uterus that
are closer to the exposure levels typical of the
other genetic sex . . .
Adaptive design [A trait that] increases an individual’s ability to
survive and/or reproduce.
Hormone levels A trait is found to be associated with individuals’
circulating levels of a particular hormone.
Animal comparisons A trait is found to have close analogs in other
animal species.
was good internal agreement among ratings within each parti-
cipant, with a Cronbach’s a of .823 for positive results and .827
for null results. For seven of the eight RDs, bioists’ ratings were
higher than those of skeptics for positive results. The effect was
not significant, however, using either a sum of standardized
ratings (bioists x ¼ 0.093, skeptics x ¼ 0.077, t ¼ 1.06,
p ¼ .292) or an MANOVA of standardized ratings (F ¼
0.91, p ¼ .517). Similarly, for seven of the eight RDs, skeptics’
ratings were higher than those of bioists for null results. The
effect attained one-tailed significance as a test of the summed
standardized ratings (bioists x ¼ 0.194, skeptics x ¼ 0.113,
t ¼ 1.85, p ¼ .069) but did not as an MANOVA of standar-
dized ratings (F ¼ 1.59, p ¼ .151). Therefore, while both
effects are in the predicted direction, Hypothesis 1a was not
supported and there is only tentative support for Hypothesis 1b.
Political Orientation
The four questions referencing political orientation exhibited
fair agreement (a ¼ .79, n ¼ 365). These included the impor-
tance of reducing inequality, whether social position was deter-
mined by inherent traits versus social forces, and where
participants placed themselves on the social and the economic
political spectrum. Bioists and skeptics did not differ substan-
tially in an index consisting of the sum of these responses (bio-
ists, n ¼ 139, x ¼ 19.36; skeptics, n ¼ 225, x ¼ 19.64; t ¼ 0.53,
p ¼ .596). Nor do the two groups differ along any individual
response (Figure 3). Thus, Hypothesis 2 is not supported.
Confirmation Bias
In all three conditions, participants were shown the same
figure. Bioists, however, did not rate the inferential value of
7
Figure 3. Political orientations.
the result higher when accompanied with the information that
the allele was found in higher frequencies in populations with
African ancestry (Table 4). The difference actually approached
significance in the opposite direction than what was predicted
(n ¼ 70, t ¼ 1.77, p ¼ .081). Similarly, bioists did not rate the
inferential value of the result differently when the allele was
reported to occur in higher frequencies in populations of
European ancestry (n ¼ 72, t ¼ 0.49, p ¼ .310). Thus,
Hypotheses 3a and 3b are not supported.
Skeptics did not rate the results in the third condition as
having a higher IP when there was no mention of frequencies
(n ¼ 111, t ¼ 0.45, p ¼ .653), nor did they do so when it was
reported that the allele occurred in higher frequencies in popu-
lations with European ancestry (n ¼ 120, t ¼ 1.57, p ¼ .119).
Thus, Hypotheses 4a and 4b were not supported.
It should be noted that the ratings were quite variable with an
average coefficient of variation of 51%. As reference, the coef-
ficient of variance for height in U.S. men is 8.5% and for weight,
it is 38% (McDowell, Fryar, Ogden, & Flegal, 2008). Sample
sizes ranged from 33 to 72 for single groups, and consequently,
effect sizes that could be detected with 80% likelihood tended to
be of moderate size (ranging from 0.43 to 0.61). Many of the
reported relationships, however, were often in the opposite direc-
tion of those predicted, making Type II errors less likely.
Discussion
The only hypothesis to receive even marginal support was
Hypothesis 1b that bioists and skeptics differ in their inter-
pretation of null results of studies exploring biological influ-
ences on human psychological and behavioral traits.
Similarly, for seven of the eight study designs, bioists rated
the IP of positive results as being more inferentially powerful,
but the effect was not significant and Hypothesis 1a was not
supported. In the IP survey, bioists also rated the specific
(positive) results as having a higher inferential value overall,
8 Evolutionary Psychology
Table 4. Ratings of Inferential Values.
Condition n Mean
Bioists
Appetitive aggression higher in nonprivileged 37 53.84
No mention of frequency 35 71.09
Appetitive aggression higher in privileged 35 57.31
Skeptics
Appetitive aggression higher in nonprivileged 72 57.60
No mention of frequency 66 60.36
Appetitive aggression higher in privileged 48 49.29
after controlling for condition, adding some weight to
Hypothesis 1a (albeit post hoc weight). While such effects
are indicative of a true disagreement in the inferential value
of such results, it is possible that they result from ontological
confirmation biases, as well—that is to say, confirmation
biases that exist due to preexisting beliefs, irrespective of
ideology (e.g., that flossing prevents gingivitis).
Similar to findings reported in related studies (Lyle & Smith,
2012; Tybur et al., 2007), bioists and skeptics were not found to
differ in their political views, failing to support Hypothesis 2.
The remaining hypotheses (Hypotheses 3a, 3b, 4a, and 4b) were
not supported either. Participants’ perceptions of inferential
value did not vary by the degree to which results supported a
particular ideology, suggesting that ideological confirmation
bias is not affecting participant perceptions of inferential value.
There are a number of potential shortcomings that could be
addressed in future research. The high variance in reported
inferential values and the modest sample sizes lead to relatively
low power in these tests, and thus, a null effect cannot be
declared with great confidence. For most tests, only modest
effect sizes could be detected with an 80% probability. Recruit-
ing a sufficiently large sample of academics is not easy, and
while all participants in the present study had earned an
advanced degree in a relevant field, the majority were not
working in academia. Those who pursue a career path in aca-
demia might follow the literature more closely and have stron-
ger opinions on the role of biology and on equality, such that
ideological confirmation biases become more significant. In
the IP sample, those working in research or teaching reported
being more aware of the research relevant to the debate (5-point
scale, academics: x ¼ 3.14, n ¼ 112, other: x ¼ 2.38, n ¼ 253,
Wilcoxon Z ¼ 6.02, p < .001) and were more likely to choose
one of the “extreme” options on the 4-point scale (1 or 4)
concerning biological versus experiential orientation (aca-
demics: extreme ¼ 22.32%, other: extreme ¼ 12.65%, w2 ¼
5.51, p ¼ .019). A similar RD relying solely on engaged aca-
demics might therefore produce different results. For instance,
e-mail mailing lists from relevant academic societies could be
sampled. These lists tend to number in the thousands, allowing
for the creation of a list large enough to overcome the low
response rates associated with cold requests.
As described in the Online Supplementary Material, the
survey included assessments of mock results aimed at explor-
ing interpretations of null effects as well. However, participants
SD Direction p Value (Different From Previous)
32.45
26.01 Opposite .081
27.26 Opposite .310
27.58
28.87 Predicted .653
29.51 Opposite .119
did not rate the complexity or the social desirability of the
associated psychological traits in ways that allowed for mean-
ingful comparisons. Future studies could rely on a pilot study
which would include the ratings of a battery psychological
traits so that comparable traits could be paired. It would be
necessary to take into account the many factors, described
earlier, that would make a trait more or less appropriate for a
such a study—psychological complexity, social desirability,
presence of existing literature on the topic, believability, and
so on. Fortunately, the tests that remained meaningful, those
focusing on appetitive aggression, were those that provided the
cleanest tests of ideological bias. In these, the only differences
between treatments were small amounts of text that greatly
altered the degree to which the results conformed to the alleged
ideological biases.
The presentation of the mock results was perhaps more
successful. Twenty-nine percent of participants were com-
pletely unaware that the results were fake, and an additional
26% were only a little suspicious. Despite this, only 15% of
participants believed that their responses were affected more
than a little by these suspicions. Fewer than 1% believed their
suspicions affected their responses “very much.”
Ultimately, this study did not find any evidence of confir-
mation bias, despite the common allegations that have been
lodged by both sides for some time. Regardless of one’s posi-
tion, it is important to remind scholars that if they believe a
group of intelligent and informed academics could be so
unknowingly blinded by ideology that they wholeheartedly
subscribe to an unquestionably erroneous interpretation of an
entire body of research, then they must acknowledge they
themselves are equally as capable of being so misguided. And
while it is likely that biases do to some degree impact research-
ers’ interpretation of individual results and collective litera-
tures, it is also very likely that the overarching debate is also
driven by true, lucid, well-informed disagreements about the
nature by which complex traits develop.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to
the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for
the research, authorship, and/or publication of this article: The open
Winking
access publishing fees for this article have been covered by the Texas
A&M University Open Access to Knowledge Fund (OAK Fund),
supported by the University Libraries and the Office of the Vice
President for Research.
Supplemental Material
Supplementary material for this article is available online.
References
Allen, E., Beckwith, B., Beckwith, J., Chorover, S., Culver, D., Dun-
can, M, . . . Schreier, H. (1975). Against “Sociobiology” (pp.
43–44). The New York Review of Books.
Andrews, P. W., Gangestad, S. W., & Matthews, D. (2002). Adapta-
tionism, exaptationism, and evolutionary behavioral science.
Behavioral and Brain Sciences, 25, 534–553. Retrieved from
<Go to ISI>://WOS:000183687600056
Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014). Separating
the shirkers from the workers? Making sure respondents pay atten-
tion on self-administered surveys. American Journal of Political
Science, 58, 739–753.
Buller, D. (2006). Adapting minds: Evolutionary psychology and the
persistent quest for human nature. Cambridge: MIT Press.
Buss, D. M. (1989). Sex differences in human mate preferences:
Evolutionary hypotheses tested in 37 cultures. Behavioral and
Brain Sciences, 12, 1–14.
Campos, J. J., Bertenthal, B. I., & Kermoian, R. (1992). Early expe-
rience and emotional development: The emergence of wariness of
heights. Psychological Science, 3, 61–64.
Chapais, B. (2008). Primeval kinship: How pair-bonding gave birth to
human society. Cambridge, MA: Harvard University Press.
Coyne, J. (2012). Is evolutionary psychology worthless?. Why Evolu-
tion is True. Retrieved from https://whyevolutionistrue.wordpress.
com/2012/12/10/is-evolutionary-psychology-worthless/.
Devine, P. G. (1989). Stereotypes and prejudice: Their automatic
and controlled components. Journal of Personality and Social
Psychology, 56, 5.
Dixon, T. L. (2008). Crime news and racialized beliefs: Understanding
the relationship between local news viewing and perceptions of
African Americans and crime. Journal of Communication, 58,
106–125.
Eagly, A. H., & Wood, W. (1999). The origins of sex differences in
human behavior—Evolved dispositions versus social roles.
American Psychologist, 54, 408–423.
El-Guindi, F., & Read, D. (2012). Westermarck hypothesis reconsid-
ered. Current Anthropology, 53, 134–135. doi:10.1086/663576
Fuentes, A. (2012). Race, monogamy, and other lies they told you:
Busting myths about human nature. Berkeley: University of Cali-
fornia Press.
Gettler, L. T., McDade, T. W., Feranil, A. B., & Kuzawa, C. W.
(2011). Longitudinal evidence that fatherhood decreases testoster-
one in human males. Proceedings of the National Academy of
Sciences, 108, 16194–16199.
Gould, S. J., & Lewontin, R. C. (1979). The spandrels of San Marco
and the Panglossian paradigm: A critique of the adaptationist pro-
gramme. Proceedings of the Royal Society of London B: Biological
Sciences, 205, 581–598.
9
Hagen, E. (2005). Controversial issues in Evolutionary Psychology.
The Handbook of Evolutionary Psychology (pp. 145–176). Hobo-
ken, NJ: John Wiley & Sons.
Hamer, D. H., Hu, S., Magnuson, V. L., Hu, N., & Pattatucci, A. L.
(1993). A linkage between DNA markers on the X chromosome
and male sexual orientation. Science, 261, 321–327.
Henrich, J., Boyd, R., Bowles, S., Camerer, C., Fehr, E., & Gintis, H.
(Eds.). (2004). Foundations of human sociality: Economic experi-
ments and ethnographic evidence from fifteen small-scale societ-
ies. New York, NY: Oxford University Press.
Horowitz, M., & Yaworsky, W. (2014). Whither the blank slate? A
report on the reception of evolutionary biological ideas among
sociological theorists. Sociological Spectrum, 34, 489–509.
Jonas, E., Schulz-Hardt, S., Frey, D., & Thelen, N. (2001). Confirma-
tion bias in sequential information search after preliminary deci-
sions: An expansion of dissonance theoretical research on selective
exposure to information. Journal of Personality and Social Psy-
chology, 80, 557.
Kushnick, G., & Fessler, D. M. T. (2011). Karo Batak cousin mar-
riage, cosocialization, and the Westermarck hypothesis. Current
Anthropology, 52, 443–448. doi:10.1086/659337
Langlois, J. H., Roggman, L. A., Casey, R. J., Ritter, J. M., Rieser-
Danner, L. A., & Jenkins, V. Y. (1987). Infant preferences for
attractive faces: Rudiments of a stereotype? Developmental Psy-
chology, 23, 363.
Leavitt, G. C. (2007). The incest taboo? A reconsideration of Wester-
marck. Anthropological Theory, 7, 393–419. doi:10.1177/
1463499607083427
Lebow, R. N. (2013). You can’t keep a bad idea down: Evolutionary
biology and international relations. International Politics Reviews,
1, 2–10.
Lehner, P. E., Adelman, L., Cheikes, B. A., & Brown, M. J. (2008). Con-
firmation bias in complex analyses. IEEE Transactions on Systems,
Man, and Cybernetics Part A: Systems and Humans, 38, 584–592.
Lyle, H. F., & Smith, E. A. (2012). How conservative are evolutionary
anthropologists? Human Nature, 23, 306–322.
Marks, J. (2012). The biological myth of human evolution. Contem-
porary Social Science, 7, 139–165.
McDowell, M. A., Fryar, C. D., Ogden, C. L., & Flegal, K. M.
(2008). Anthropometric reference data for children and adults:
United States, 2003–2006. National Health Statistics Reports, 10.
McKinnon, S., & Silverman, S. (2005a). Introduction. In Complex-
ities: Beyond nature and nurture (pp. 1–20). Chicago, IL: Univer-
sity of Chicago Press.
McKinnon, S., & Silverman, S. (Eds.). (2005b). Complexities: Beyond
nature & nurture. Chicago, IL: The University of Chicago Press.
Mehta, P. H., Jones, A. C., & Josephs, R. A. (2008). The social endo-
crinology of dominance: Basal testosterone predicts cortisol
changes and behavior following victory and defeat. Journal of
Personality and Social Psychology, 94, 1078.
Munafò, M. R., Yalcin, B., Willis-Owen, S. A., & Flint, J. (2008).
Association of the dopamine D4 receptor (DRD4) gene and
approach-related personality traits: Meta-analysis and new data.
Biological Psychiatry, 63, 197–206.
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenom-
enon in many guises. Review of General Psychology, 2, 175–220.
10
Oaten, M., Stevenson, R. J., & Case, T. I. (2009). Disgust as a disease-
avoidance mechanism. Psychological Bulletin, 135, 303.
Pavelka, M. M. (2002). Resistance to the cross-species perspective in
anthropology. Cambridge Studies in Biological and Evolutionary
Anthropology, 25–44.
Pinker, S. (2002). The blank slate. New York, NY: Viking.
Plavcan, J. M. (2012). Sexual size dimorphism, canine dimorphism,
and male–male competition in primates. Human Nature, 23,
45–67.
Polderman, T. J., Benyamin, B., De Leeuw, C. A., Sullivan, P. F., Van
Bochoven, A., Visscher, P. M., & Posthuma, D. (2015). Meta-
analysis of the heritability of human traits based on fifty years of
twin studies. Nature Genetics, 47, 702–709.
Proctor, D., Williamson, R. A., de Waal, F. B., & Brosnan, S. F.
(2013). Chimpanzees play the ultimatum game. Proceedings of the
National Academy of Sciences, 110, 2070–2075.
Rose, S., & Rose, H. (2001). Alas, poor Darwin: Arguments against
evolutionary psychology. London, England: Cape.
Sanderson, S., & Ellis, E. (1992). Theoretical and political perspec-
tives of American sociologists in the 1990s. The American Sociol-
ogist, 23, 26–42.
Sayer, A. (1997). Essentialism, social constructionism, and beyond.
The Sociological Review, 45, 453–487.
Segerstrale, U. (2000). Defenders of the truth: The battle for science in
the sociobiology debate and beyond. New York, NY: Oxford Uni-
versity Press.
Snyder, M., & Swann, W. B. (1978). Hypothesis-testing processes in
social interaction. Journal of Personality and Social Psychology,
36, 1202.
Evolutionary Psychology
Stanovich, K. E., West, R. F., & Toplak, M. E. (2013). Myside bias,
rational thinking, and intelligence. Current Directions in Psycho-
logical Science, 22, 259–264.
Sussman, R. W. (2013). Why the Legend of the Killer Ape Never
Dies. War, Peace, and Human Nature: The Convergence of Evolu-
tionary and Cultural Views (pp. 97).
Taber, C. S., & Lodge, M. (2006). Motivated skepticism in the eva-
luation of political beliefs. American Journal of Political Science,
50, 755–769.
Thompson, R. A., & Newton, E. K. (2013). Baby altruists? Examining
the complexity of prosocial motivation in young children. Infancy,
18, 120–133.
Tybur, J. M., Miller, G. F., & Gangestad, S. W. (2007). Testing the
controversy—An empirical examination of adaptationists’ atti-
tudes toward politics and science. Human Nature—An Interdis-
ciplinary Biosocial Perspective, 18, 313–328. doi:10.1007/
s12110-007-9024-y
Winking, J., Gurven, M., Kaplan, H., & Stieglitz, J. (2009). The goals
of direct paternal care among a South Amerindian population.
American Journal of Physical Anthropology, 139, 295–304.
Wolfe, P. (2006). Settler colonialism and the elimination of the native.
Journal of Genocide Research, 8, 387–409.
Wrangham, R. W., & Glowacki, L. (2012). Intergroup aggression in
chimpanzees and war in nomadic hunter-gatherers. Human Nature,
23, 5–29.
Wu, H., Corney, J., & Grant, M. (2014). Relationship between quality
and payment in crowdsourced design. Proceedings of the 2014
IEEE 18th International Conference on Computer Supported
Cooperative Work in Design (CSCWD).