JOURNAL OF WOMEN’S HEALTH

Volume 28, Number 1, 2019 Original Articles

ª Mary Ann Liebert, Inc.
DOI: 10.1089/jwh.2018.7396

Implementation of the NIH Sex-Inclusion Policy:

Attitudes and Opinions of Study Section Members

Nicole C. Woitowich, PhD1 and Teresa K. Woodruff, PhD1,2

Abstract

Objective: On January 25th, 2016, the National Institutes of Health (NIH) implemented a policy requiring
investigators to consider sex as a biological variable (SABV) within their grant submissions. We surveyed NIH
Downloaded by 193.56.67.114 from www.liebertpub.com at 02/06/19. For personal use only.

study section members in 2016 and 2017 to determine their attitudes toward the policy and their perceptions
regarding its implementation.
Materials and Methods: Members of standing study sections and special emphasis panels who met in May,
June, or July of 2016 and 2017, and had a publicly accessible e-mail address, were invited to participate in the
survey (n = 4376 and n = 4710, respectively). The survey assessed participant demographics, knowledge and
awareness of the SABV policy, and opinions regarding its utility and implementation.
Results: A combined total of 1161 study section members participated in the survey for a response rate of
10.2% in 2016 and 15.1% in 2017. Respondents thought it was important for NIH-funded research to consider
SABV (63% vs. 68%, p = 0.141) and that it will improve rigor and reproducibility (54% vs. 58%, p = 0.208). In
terms of implementation, respondents indicated that the percentage of grants, which have successfully ad-
dressed and incorporated the policy, has significantly increased over time ( p < 0.0001 for all endpoints).
However, open-ended comments revealed concern for federal research funding, the overuse of experimental
animals, and uncertainty surrounding grant scoring, as it relates to the SABV policy.
Conclusions: In this study, we show improving attitudes toward the sex-inclusive policy at NIH and that a
statistically significant number of grants are addressing sex as a biological variable appropriately in their
submissions. These data suggest the policy is becoming more well accepted, and it is thus anticipated that the
reproducibility of scientific reports will increase over time and new discoveries using sex as a biological
variable are on the horizon.

Keywords: NIH, sex inclusion, research policy

Introduction Following years of advocacy,2,6 the NIH issued a notice in

S ex as a biological variable has been systematically ex-
cluded from fundamental research and clinical studies1–3
until two landmark injunctions by the National Institutes of
Health (NIH). The first of these policy changes was mandated
by Congress in 1993 through the NIH Revitalization Act,
which required the inclusion of females in NIH-funded
clinical research.4 After several decades of developing best
practices to improve the inclusion and retention of women in
clinical research, male and female participants are ap-
proaching parity.5 However, no such policies governed the
inclusion of males and females in basic science or preclinical
research.
2016 requiring NIH grantees to ‘‘consider sex as a biological
variable (SABV)’’ (NIH NOT-OD-15-102).7 Unlike the
federal mandate for inclusion, the more recent policy asks
only for justification and/or description of sex as part of the
ordinary scientific report. The anticipation is that when in-
vestigators ‘‘consider’’ sex as a biological variable, they may
be more inclined to ‘‘study’’ sex in the context of signaling
pathways and physiological research, and in disease settings.
While commentaries both for and against the policy ex-
ist,8–11 data surrounding its implementation are lacking.12 In
this study, we examined the attitudes of NIH study section
members as a large demographic of individuals charged with
the review of the policy as part of grant evaluation, and who

1
Women’s Health Research Institute, 2Department of Obstetrics and Gynecology, Feinberg School of Medicine, Northwestern University,
Chicago, Illinois.

9
 10 WOITOWICH AND WOODRUFF

are themselves grantees, thus providing a proxy for investi-
gators across all disciplines of science. A survey was ad-
ministered after the initial round of grant reviews in which
applicants were required to consider SABV and 1 year later,
thus providing the framework to assess change in compre-
hension and compliance. In aggregate, the data indicate an
increasing approval of the policy along and improved com-
pliance by investigators.
Materials and Methods
Survey
Questionnaires were administered through an online survey
platform (Qualtrics) in 2016 and 2017. Survey questions were
designed to determine study section members’ opinions and
attitudes toward the implementation and evaluation of the NIH
policy, requiring the consideration of SABV in preclinical
research (Supplementary Data; Supplementary Data are
available online at www.liebertpub.com/jwh). Twenty-four
Downloaded by 193.56.67.114 from www.liebertpub.com at 02/06/19. For personal use only.
scoring. In addition, participants had the ability to include
open-ended comments at the completion of the survey.
Study participants and recruitment
Following procedural review of the online consent form and
survey questions to be asked of intended participants, the
Northwestern University institutional review board deemed
this study exempt from further review. A comprehensive list of
standing study section and special emphasis panel members
was compiled from meeting rosters published on the Center for
Scientific Review (CSR) website.13 Meeting rosters corre-
sponding to the initial review of proposals required to consider
sex as a biological variable (May–July 2016) and those 1 year
later (May–July 2017) were included. Mail-in reviewers were
excluded from this study.
The 2016 and 2017 review cohorts contained 4686 and
4841 individuals respectively. E-mail addresses of 4376
(93.3%) individuals from the 2016 cohort and 4710 (97.3%)

questions were included. Major themes of the survey included
participant demographics, experience as a study section
member, previous experience conducting sex-based research,
opinions on the overall policy as it benefits the research en-
terprise, and evaluation of the policy as it relates to grant
individuals from the 2017 cohort were obtained through a
comprehensive web-based search utilizing Google search
engines and institutional directories.
An e-mail outlining the purpose of the study, along with a
link to participate in the survey was sent to the 2016 review

Table 1. Comparison of Respondent Demographics Between 2016 and 2017

2016 (%) 2017 (%) p-value
Total 448 100 713 100
Sex 0.111
Male 228 50.9 397 55.7
Female 220 49.1 316 44.3
Ethnicity 0.970
White (non-Hispanic) 358 79.9 574 80.5
Asian or Pacific Islander 48 10.7 74 10.4
Other 42 9.4 65 9.1
Employment sector 0.750
Academia 432 96.4 690 96.8
Other 16 3.6 23 3.2
Study section member status 0.554
Permanent 258 57.6 398 55.8
Temporary or unsure 190 42.4 315 44.2
Study section chairperson 0.225
Yes 18 4.0 40 5.6
No 430 96.0 673 94.4
Study section participation 0.822
Less than 1 year 49 10.9 72 10.1
1–2 years 63 14.1 110 15.4
3–5 years 132 29.5 218 30.6
6–9 years 72 16.1 122 17.1
10 years or more 132 29.5 191 26.8
Study section attendance 0.488
In person 411 91.7 662 92.8
Virtually (video or conference call) 37 8.3 51 7.2
Research involves study of sex differences 0.672
Yes 156 34.8 257 36.0
No, unsure, or does not conduct research 292 65.2 456 64.0
Grants reviewed involve use of vertebrate 0.130
animals or human subjects
Yes 442 98.7 694 97.3
No 6 1.3 19 2.7
 IMPLEMENTATION OF THE NIH SEX-INCLUSION POLICY 11

cohort on September 12th, 2016, and the 2017 review cohort

on October 16th, 2017. Two weeks later, a reminder e-mail
was sent to all participants encouraging them to complete the
survey. A total of 1618 individuals received an invitation to
participate in both years of the study. The survey response
rates were 10.2% in 2016 and 15.1% in 2017.

Data analyses
Data analyses were primarily qualitative, with a small quan-
titative component. Descriptive statistics were used where
appropriate. Nominal data were described as n (%). Chi-
squared tests were used to compare categorical data as well as
to assess associations between participant sex and experience
conducting sex-based research with attitudes toward the
SABV policy (GraphPad Prism, version 7.0). p-values <0.05
were considered significant.
Open-ended survey comments were analyzed using a
thematic and semiquantitative approach. Comments were
coded using deductive methods by the authors. Both authors
Downloaded by 193.56.67.114 from www.liebertpub.com at 02/06/19. For personal use only.

independently read the entire dataset and manually coded

participant comments into one or more of the following ca-
tegories: budget, policy, animal use, research area, review,
and survey/other. The two authors discussed the comments to
agreement and comments pertaining only to the survey/other
category were removed from further analyses.
Sentiment analysis software (Repustate, version 1.1.23) was
utilized to analyze the comments in a semiquantitative manner.
Repustate employs a combination of deep learning and support
vector machines to determine the polarity of a given piece of
text on a scale from -1.0 to 1.0. A score less than 0 is con-
sidered negative, 0 neutral, and greater than 1.0 positive. Key
features that play a part in the vector model include parts of
speech, grammatical structure, and occurrence of prior polarity
terms or phrases. Sentiment scores were determined for each
comment and the median score (quartile 1–quartile 3) reported
for each category theme. A Mann–Whitney U test was used to FIG. 1. Comparison of respondents’ perceived compre-
compare open-ended comment sentiment analysis scores be- hension and attitudes toward the SABV policy between
tween cohorts (GraphPad Prism version 7.0). 2016 and 2017. SABV, sex as a biological variable.

Data sharing plan
To ensure respondent anonymity, data were not analyzed
by study section or special emphasis panel. Deidentified data,
which exclude this information, can be provided by request
from the corresponding author.
Results
A total of 1161 study section members, representing over
175 standing study sections and continuing special emphasis
panels, participated in the 2016 and 2017 surveys (Table 1).
Briefly, the majority of respondents were white (80% vs.
81%) males (51% vs. 56%) employed in academia (96% vs.
97%), who served as permanent members of study section
(58% vs. 56%) with 3 years or more experience on study
section (76% vs. 75%).
Next, we queried respondents on their awareness, under-
standing, and attitudes toward the SABV policy (Fig. 1 and
Supplementary Tables S1–S3). The majority of respondents
became aware of the SABV policy through the NIH (60% vs.
57%) and not through their home institution (13% vs. 12%,
Supplementary Table S1). In 2016, the majority of respon-
dents felt confident in their understanding of the SABV
policy (83.5%), which increased significantly to 88.4% by
2017 ( p = 0.018, Fig. 1 and Supplementary Table S3).
In both years, over 70% of respondents indicated that the
NIH provided adequate educational resources regarding the
SABV policy (70.8% vs. 74.5%; p = 0.165), whereas the ma-
jority reported that they were unaware of or were not provided
any educational resources by their home institutions (75.2% vs.
75.6%; p = 0.886, Supplementary Table S2). In both years,
more than 90% of the respondents indicated that they discussed
the policy at their respective study section meetings (92.2% vs.
92.3%; p = 0.951, Supplementary Table S2).
To assess study section members’ attitudes toward the
SABV policy, we asked if it was important to consider SABV
in the experimental design of NIH-funded research, and if it
would improve rigor and reproducibility (Fig. 1). These two
questions highlight a direct application and goal of the policy,
respectively. A majority of respondents thought that it was
important to consider SABV in the experimental design
(63.4% vs. 67.6%; p = 0.141) and that considering SABV
would improve rigor and reproducibility (54.0% vs. 5%;
p = 0.208, Supplementary Table S3). In both years,
 12 WOITOWICH AND WOODRUFF

Table 2. Comparison of Respondent Attitudes Toward the Sex as a Biological Variable Policy
by Sex-Based Research Experience Between 2016 and 2017
2016 2017
Research Research
Research does not Research does not
involves involve involves involve
SSD SSD SSD SSD
(n = 156) (%) (n = 292) (%) p-value (n = 257) (%) (n = 456) (%) p-value
Importance of 0.0037 0.0001
considering SABV in
experimental design
Yes 113 72.4 171 58.6 197 76.7 285 62.5
No or unsure 43 27.6 121 41.4 60 23.3 171 37.5
Consideration of SABV 0.0002 0.0012
to improve rigor
and reproducibility
Yes 103 66.0 139 47.6 169 65.8 243 53.3
No or unsure 53 34.0 153 52.4 88 34.2 213 46.7
Downloaded by 193.56.67.114 from www.liebertpub.com at 02/06/19. For personal use only.

SABV, sex as a biological variable; SSD, study of sex differences.

respondents whose research portfolios involve the study of
sex differences were significantly more likely to think that the
consideration of SABV is important (73% vs. 59% in 2016;
p = 0.0037, and 77% vs. 63% in 2017; p = 0.0001) and would
improve rigor and reproducibility compared to their peers
(66% vs. 48% in 2016; p = 0.0002, and 66% vs. 53% in 2017;
p = 0.0012, Table 2).
Self-reported biological sex of the respondent did not in-
fluence these views in 2016, yet in 2017, females were sig-
nificantly more likely to think that the consideration of SABV
is important and would improve rigor and reproducibility
compared to males, respectively (73% vs. 63%; p = 0.005,
and 64% vs. 53%; p = 0.002, Table 3).
The NIH SABV policy only applies to grants that include
the use of human or vertebrate subjects.7 As such, respon-
dents who reviewed grants meeting these criteria (99% vs.
97%; p = 0.130, Table 1) were asked what proportion of
grants they reviewed accounted for the consideration of
SABV in the research strategy and adequately addressed the
incorporation of SABV into the experimental design, analy-
sis, and reporting, as instructed by NIH guidelines (Fig. 2 and
Supplementary Table S4). In both cases, there was a statis-
tically significant increase in the proportion of grants that
addressed the SABV policy between 2016 and 2017
( p < 0.0001 for each endpoint).
A majority of study section members indicated that the
consideration of SABV was consistently weighed into an
applicant’s approach score (55% vs. 61%; p = 0.0518). In
addition, there was a statistically significant increase in the
proportion of grants that provided consistent justification for
the use of a single sex between 2016 and 2017 (44% vs. 50%;
p = 0.0330).
Respondents had the option to leave open-ended com-
ments at the end of each survey (n = 130 in 2016 and n = 257
in 2017, Supplementary Table S5). There were no differences
in demographics between respondents who left comments in
2016 and 2017, with the exception that there were signifi-
cantly more study section chairpersons who commented in
2017 compared to 2016 (8.6% vs. 2.3%, respectively;
p = 0.018). Following initial review, comments were cate-
gorized into one or more of the following major themes:
budget, animal use, policy, research area, and grant review.
In the budget theme, comments highlighted the need for
increased federal funding for research citing that the SABV

Table 3. Comparison of Respondent Attitudes Toward the Sex as a Biological Variable Policy
by Sex Between 2016 and 2017
2016 2017
Male Female Male Female
(n = 228) (%) (n = 220) (%) p-value (n = 397) (%) (n = 316) (%) p-value
Importance of 0.3736 0.0051
considering SABV in
experimental design
Yes 140 61.4 144 65.5 251 63.2 231 73.1
No or unsure 88 38.6 76 34.5 146 36.8 85 26.9
Consideration of SABV 0.3278 0.0018
to improve rigor and
reproducibility
Yes 118 51.8 124 56.4 209 52.6 203 64.2
No or unsure 110 48.2 96 43.6 188 47.4 113 35.8
 IMPLEMENTATION OF THE NIH SEX-INCLUSION POLICY 13
Downloaded by 193.56.67.114 from www.liebertpub.com at 02/06/19. For personal use only.

FIG. 2. Comparison of respondents’ perception of SABV policy implementation between 2016 and 2017.

policy places, ‘‘[an] unreasonable demand given the limited
budgets allowed for NIH grants,’’ while comments pertaining
to animal use indicated that the SABV policy ‘‘requires in-
creasing the number of experimental animals.’’ Comments that
addressed the implementation of the policy itself were mixed,
as some lauded the policy, as ‘‘a simple and needed criterion,’’
while others were more critical citing it as, ‘‘[a] waste of time,
not at all getting to the root of reproducibility problems.’’
Finally, respondents provided comments directly related to
study section and the grant review process. Comments spanned
multiple topics such as reviewer education (‘‘Our review of-
ficer [redacted] was really good in explaining the new policy to
us. That is how I learned about how to incorporate it into my
proposals and my reviews of others proposals,’’), policy im-
plementation (‘‘The applicants have gotten the message and
now include sex as a biological variable in their proposals.’’),
and discussion of the policy itself (‘‘SABV was strongly em-
phasized in the premeeting discussions, and it was mentioned
for every discussed application.’’).
However, respondent comments revealed variability in the
way the policy impacts grant scores citing that, ‘‘It is difficult
to determine whether the individual reviewers modified their
score on Approach based on whether [sic] the criteria for sex
as a biological variable,’’ and ‘‘Reviewers have different
opinions on how much weight consideration of SABV should
have on the score of the application.,’’ While another re-
spondent shared that, ‘‘Consideration of SABV was not
usually a score-driving consideration.’’
In addition to our qualitative analysis, semiquantitative
in silico sentiment analysis was utilized to interpret the re-
spondents’ comments. In general, the overall comment sen-
timent analysis scores for both years were slightly negative
[median (quartile 1–quartile 3): -0.262 (-0.828–0) vs.
-0.181 (-0.726–0); p = 0.358, Table 4]. When analyzed by
theme, comments pertaining to animal use or budget/funding
issues received the lowest sentiment analysis scores, fol-
lowed by comments related to the SABV policy itself.
Because we identified biological sex and experience con-
ducting sex-based research as two potential factors that in-
fluence views toward the SABV policy, we analyzed
comments by these variables. While there were no significant
differences in 2016, sentiment analysis scores significantly

Table 4. Comparison of Respondents’ Comments Regarding the Sex as a Biological Variable Policy
by Sentiment Analysis Score Between 2016 and 2017
2016 2017
SA score median SA score median
N (quartile 1–quartile 3) N (quartile 1–quartile 3) p-value
Total 130 -0.262 (-0.828–0) 257 -0.181 (-0.726–0) 0.358
Comment themes
Animal use 14 -0.681 (-0.95–0.271) 30 -0.641 (-0.945–0.235) 0.915
Budget 32 -0.660 (-0.95–0.471) 72 -0.631 (-0.948–0.156) 0.268
Policy 63 -0.576 (-0.95–0) 133 -0.468 (-0.878–0) 0.337
Research area 41 0 (-0.802–0) 72 -0.001 (-0.559–0) 0.868
Review 58 0 (-0.414–0) 134 0 (-0.556–0) 0.188
 14 WOITOWICH AND WOODRUFF

Table 5. Examples of Respondents’ Comments in the Policy Theme by Sentiment Analysis Score
Comment
Positive I think it is an important issue and am glad that the NIH is stressing its importance.
sentiment Sex consideration and incorporation in grant applications greatly improve scientific rigor
(SA > 0) and reproducibility, hence should be promoted!
Sex as a biological variable is important beyond rigor and reproducibility. It informs the fundamental
validity of the premise and observation. It is high time this was treated seriously and I hope NIH will
continue to increase its emphasis on this point.
Neutral Sex is a reasonable biological variable to require in many instances; however, it is just one.
sentiment The requirement should be broadened to reasonable biological variables like sex, age, environment, etc.
(SA = 0) Why is sex being singled out as the key biological variable to be considered for review purposes?
The idea of focusing on sex as a biological variable is important and necessary. However, like many
regulations, the implementation of a single rule to all areas of research is difficult and may eventually
undermine the utility of the rule.
Negative For some areas of research, this is nonsense. It’s not applicable and requiring it stands in the way
sentiment of making progress.
(SA < 0) It seems simplistic and blunt to have a policy requiring that sex be included/considered biological
variable in all NIH studies.
This policy ignores the complexity of sex as a biological variable and the cost and time that would
Downloaded by 193.56.67.114 from www.liebertpub.com at 02/06/19. For personal use only.

be required to adequately address this in every grant. The policy introduces a variable that cannot
be adequately controlled for with typical budget and capacity constraints and weakens rigor
of the majority of applications.

differed for males and females (median = -0.2193 vs. 0;
p = 0.032), and those whose work involves the study sex dif-
ferences compared to those whose work does not (median = 0
vs. -0.2429; p = 0.007) in 2017 (Supplementary Table S6).
Table 5 provides representative examples of positive, negative,
and neutral comments in the policy category theme.
Discussion
Sex, as it relates to the biological determination of males
and females, XX or XY, has largely been left out of biological
studies not only in terms of inclusion but also in the analyses
and reporting of data.5,14,15 The NIH SABV policy was cre-
ated to address these inequities through the promotion of sex-
inclusive research studies, in an effort to enhance the rigor
and reproducibility of the biomedical research enterprise.16
As the policy is relatively new, it is difficult to determine
its direct impact on research output. Instead, we chose to
survey the attitudes and opinions of NIH study section
members who serve both as policy adjudicators and repre-
sentatives of the biomedical research community. The results
presented in this study suggest that awareness, perceived
comprehension, and implementation of the NIH SABV pol-
icy are improving over time, but that negative attitudes to-
ward the policy persist and additional training and/or
guidance may be warranted.
Our data indicate that the SABV policy is accepted and
understood by a majority, but not all, of study section members.
We identified potential biases that may influence attitudes to-
ward the SABV policy, such as field of study and sex. Re-
searchers who currently explore the study of sex differences
were significantly more likely to agree that it is important to
consider SABV and that it improves rigor and reproducibility.
This may indicate a potential knowledge gap for investigators
who are less familiar with the role of sex in health and disease.
Also, sex of the study section member appears to influence
attitudes toward the policy as female study section members
in the 2017 cohort were significantly more likely to view the
SABV policy as favorable compared to men. While women
are in the minority of study section membership (44% of our
respondents in 2017), they may see value of the policy at a
personal level, as sex-based research is directly implicated in
women’s health.6,12,15
The ability to leave open-ended comments provided study
section members with the opportunity to discuss issues not
directly assessed in the survey. Sentiment analysis, utilized in
an attempt to remove potential author bias, revealed negative
feeling toward the policy. However, open-ended responses
were entirely voluntary, and it may be that those with po-
larized feelings toward the policy were more likely to reply.
Nevertheless, these comments provide a contextual frame-
work in which the policy can be evaluated.
Budgetary and/or funding concerns were woven through-
out many of the comments, citing that increased funds were
needed to appropriately consider SABV. Also, many com-
ments voiced concerns for animal welfare, stating that the
SABV policy requires an increase in experimental animals.
This is likely derived from the assumption that to consider
SABV, sample sizes must be doubled and female animals at
varying stages of the estrous cycle should be utilized. While
estrous cycle staging may be required to conduct experiments
specifically designed to explore hormonal influences on a
given trait or condition, several studies reported that there is
no increase in intragroup variability between males and fe-
males, regardless of estrous cycle stage,17–19 thus allowing
for the inclusion of both male and female subjects, without an
increase in sample size (for a more detailed discussion of this
topic, consult a recent review by Beery20).
In terms of policy implementation, study section members
expressed differing viewpoints as to how the SABV policy
was to be evaluated and scored within the context of grant
review. This highlights an opportunity for increased reviewer
education, although many scientific review officers and sec-
tion chairs were praised for their efforts to educate their
 IMPLEMENTATION OF THE NIH SEX-INCLUSION POLICY
members on the policy. Finally, it came to our attention
during review of the comments that numerous individuals
misused the terms sex and gender, referring to gender in a
specific scientific context (i.e., cell or animal models) when
sex is the appropriate term.21 This could suggest that re-
searchers may benefit from additional training and/or edu-
cation on sex- and gender-based research practices.
In terms of how the policy has been accepted by grant
applicants, study section members found that the number of
grants that addressed SABV both in their research strategy
and specifically within the experimental design, analyses, and
reporting has significantly increased since the policy was
introduced. Likewise, there was a significant increase in the
number of grants, which provided consistent justification
of the use of a single sex. These data are encouraging and
demonstrate adherence to the policy. Future studies are
needed to examine whether this initial outcome translates
into sex-based or sex-inclusive research findings with direct
impacts on health and disease.
Downloaded by 193.56.67.114 from www.liebertpub.com at 02/06/19. For personal use only.
While this is the first assessment of study section mem-
bers’ attitudes toward the SABV policy, it is not without
limitations. First, there was a limited response rate for both
surveys (10% and 15% for 2016 and 2017, respectively).
Study section members may have felt uncomfortable par-
ticipating in a survey related directly to the NIH review
process. To ensure anonymity and the confidentiality of the
review process, we did not ask any questions related to spe-
cific applications or analyze data by study section member-
ship. This study was not funded by the NIH; however, this
information was not included in the consent materials, which
may have deterred study section members from participating.
In addition, this study is not a true follow-up as study
section membership differed between the two study cohorts.
This limitation was mitigated by the fact that the demo-
graphics and experience of respondents were not significantly
different between 2016 and 2017. It is also important to note
that this study assessed respondents’ perceived knowledge of
the SABV policy, and not through a direct assessment of
policy content. Thus, respondents who indicated that they felt
confident in their understanding of the SABV policy may not
actually have content knowledge surrounding the purpose
and scope of the policy. Likewise, respondents were tasked
with estimating the percentage of grants reviewed, which
incorporated the SABV policy several months after attending
study section. We acknowledge that these data, at face value,
serve only as estimate and do not reflect the actual number of
grants that adhere to the SABV policy. Due to confidentiality
of the grant review process, it would fall under the purview of
the NIH to conduct an exact analysis.
Finally, while sentiment analysis was utilized in an attempt
to remove reviewer bias of open-ended comments, implicit
bias may still persist into thematic coding of the comments
and the representation of such within this article. Blinded,
independent review was used to mitigate this risk.
In conclusion, inclusion of sex as a biological variable in
fundamental research funded by the NIH is a policy that is
accepted by a majority of study section members. Moreover,
study section members are knowledgeable about the policy
with information largely obtained from NIH. Most study
section members believe that the policy is improving rigor
and reproducibility and a statistically significant number of
grants are addressing sex as a biological variable appropri-
15
ately in their submissions. Yet, respondent comments re-
vealed that negative attitudes toward the policy do exist.
As women are only now approaching parity in clinical
research,5 25 years after the implementation of the NIH Re-
vitalization Act, we encourage the research community to
view policies that examine the influences of both sex and
gender on health and disease as scientific imperative. This
will help to ensure that the reproducibility of scientific reports
increase over time and that new discoveries using sex as a
biological variable are on the horizon.
Acknowledgments
We thank the individuals who participated in this survey.
We also thank Alexandra Rashedi and Megan Runge for their
assistance. This work was supported by the Thomas J. Wat-
kins Memorial Endowed Chair (to TKW) at Northwestern
University.
Author Disclosure Statement
No competing financial interest exists.
References
1. Geller SE, Adams MG, Carnes M. Adherence to federal
guidelines for reporting of sex and race/ethnicity in clinical
trials. J Womens Health (Larchmt) 2006;15:1123–1131.
2. Kim AM, Tingen CM, Woodruff TK. Sex bias in trials and
treatment must end. Nature 2010;465:688–689.
3. Klein SL, Schiebinger L, Stefanick ML, et al. Opinion: Sex
inclusion in basic research drives discovery. Proc Natl
Acad Sci U S A 2015;112:5257–5258.
4. National Institutes of Health. NIH Guidelines on the In-
clusion of Women and Minorities as Subjects in Clinical
Research. Fed Regist 1994;59:14508–14513.
5. Geller SE, Koch AR, Roesch P, Filut A, Hallgren E, Carnes
M. The More Things Change, the More They Stay the
Same: A Study to Evaluate Compliance With Inclusion and
Assessment of Women and Minorities in Randomized
Controlled Trials. Acad Med 2018;93:630–635.
6. Institute of Medicine Board on Population Health and
Public Health Policy. The National Academies Collection:
Reports funded by National Institutes of Health. Sex-
Specific Reporting of Scientific Research: A Workshop
Summary. Washington (DC): National Academies Press
(US). National Academy of Sciences; 2012.
7. National Institutes of Health. Consideration of Sex as a
Biological Variable in NIH-funded Research. 2015; Avail-
able at: https://grants.nih.gov/grants/guide/notice-files/NOT-
OD-15-102.html Accessed October 17, 2018.
8. Fields RD. NIH policy: Mandate goes too far. Nature 2014;
510:340.
9. McCullough LD, McCarthy MM, de Vries GJ. NIH policy:
Status quo is also costly. Nature 2014;510:340.
10. Shansky RM, Woolley CS. Considering sex as a biological
variable will be valuable for neuroscience research.
J Neurosci 2016;36:11817–11822.
11. Eliot L, Richardson SS. Sex in Context: Limitations of
animal studies for addressing human sex/gender neurobe-
havioral health disparities. J Neurosci 2016;36:11823–
11830.
12. Woodruff TK, Green S, Paller A, et al. Sex-based bio-
medical research policy needs an implementation plan.
Womens Health (Lond) 2015;11:449–452.
 16 WOITOWICH AND WOODRUFF

13. National Institutes of Health. Roster Index for Regular 20. Beery AK. Inclusion of females does not increase vari-
Standing Study Sections and Continuing SEPs. Available ability in rodent research studies. Curr Opin Behav Sci 2018;
at: https://public.csr.nih.gov/studysections/standing/pages/ 23:143–149.
default.aspx Accessed October 17, 2018. 21. Clayton JA. Studying both sexes: A guiding principle for
14. Beery AK, Zucker I. Sex bias in neuroscience and biomed- biomedicine. FASEB J 2016;30:519–524.
ical research. Neurosci Biobehav Rev 2011;35:565–572.
15. Mansukhani NA, Yoon DY, Teter KA, et al. Determining if
sex bias exists in human surgical clinical research. JAMA
Surg 2016;151:1022–1030. Address correspondence to:
16. Clayton JA, Collins FS. Policy: NIH to balance sex in cell Teresa K. Woodruff, PhD
and animal studies. Nature 2014;509:282–283. Women’s Health Research Institute
17. Mogil JS, Chanda ML. The case for the inclusion of female Feinberg School of Medicine
subjects in basic science studies of pain. Pain 2005;117:1–5. Northwestern University
18. Prendergast BJ, Onishi KG, Zucker I. Female mice liber- 303 East Superior Street
ated for inclusion in neuroscience and biomedical research. Robert H. Lurie Medical Research Center
Neurosci Biobehav Rev 2014;40:1–5. 10th Floor Suite 10-121
19. Becker JB, Prendergast BJ, Liang JW. Female rats are not Chicago, IL 60611
more variable than male rats: A meta-analysis of neuro-
science studies. Biol Sex Differ 2016;7:34. E-mail: tkw@northwestern.edu
Downloaded by 193.56.67.114 from www.liebertpub.com at 02/06/19. For personal use only.