Journal of Evidence-Based Social Work

ISSN: 2640-8066 (Print) 2640-8074 (Online) Journal homepage: https://www.tandfonline.com/loi/webs22

The Peril and Promise of Racial and Ethnic

Subgroup Analysis in Health Disparities Research

Jasney Cogua, Kai Yin Ho & W. Alex Mason

To cite this article: Jasney Cogua, Kai Yin Ho & W. Alex Mason (2019): The Peril and Promise of
Racial and Ethnic Subgroup Analysis in Health Disparities Research, Journal of Evidence-Based
Social Work, DOI: 10.1080/26408066.2019.1591317

To link to this article: https://doi.org/10.1080/26408066.2019.1591317

Published online: 05 Apr 2019.

Submit your article to this journal

Article views: 1

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=webs22
JOURNAL OF EVIDENCE-INFORMED SOCIAL WORK
https://doi.org/10.1080/26408066.2019.1591317

The Peril and Promise of Racial and Ethnic Subgroup Analysis

in Health Disparities Research
Jasney Cogua, Kai Yin Ho, and W. Alex Mason
Boys Town Child and Family Translational Research Center, Boys Town, NE, United States

ABSTRACT KEYWORDS
Objective: The purpose of this paper is to introduce a rubric for both Subgroup analysis; race and
ethnicity; health disparities
researchers and consumers of research to guide best practices in the
conduct of exploratory and confirmatory subgroup analyses studies.
Methods: Paper draws on a review of the literature on subgroup
analyses techniques and studies on racial and ethnic disparities in
behavioral, mental, and physical health outcomes.
Results: The paper highlights theoretical, methodological, and inter-
pretational challenges in the use of subgroup analyses and illustrates
them with specific examples.
Conclusion: The paper concludes with a series of specific recommen-
dations for the development of subgroup analyses specifically within
three practice areas: theoretical, methodological, and interpretational.
Such recommendations are captured in a rubric of best practices that
aim to strengthen the evidence-base derived from subgroup studies to
guide treatment and prevention interventions as well as the design and
evaluation of effective policy recommendations to ameliorate health
disparities.

Racial and ethnic minority members in the United States carry disproportionately higher
burden and increased risk for behavioural, mental, and physical health problems (National
Institutes of Health, 2016). These health disparities contribute to greater mortality,
morbidity, and other impairments among minority compared to non-minority members.
The United States Census Bureau estimates an increase of 40% in minority populations in
the United States during the first quarter of the century (Colby & Ortman, 2014; Humes,
Jones, & Ramirez, 2011). This change in population demographics has led to an apprecia-
tion among researchers of the need to understand the patterns (Borrell, Kiefe, Diez-Roux,
Williams, & Gordon-Larsen, 2013) and predictors (Mason, Mennis, Linker, Bares, &
Zaharakis, 2014; Reeb et al., 2015) of disparities in health-related outcomes. Greater
understanding of patterns and predictors related to health disparities will inform the
development of tailored prevention and treatment intervention efforts designed to ame-
liorate such differences in outcomes (Guerrero et al., 2013; Windsor, Jemal, & Alessi,
2015). To address these significant questions, researchers have increasingly conducted
subgroup analyses.
Subgroup analysis is used to test for group differences in means and rates, and also
includes techniques for examining moderation of the relationships between variables

CONTACT Jasney Cogua jasney.cogua@boystown.org Boys Town Child and Family Translational Research Center,
378 Bucher Drive, 68010, Boys Town, NE, United States
© 2019 Taylor & Francis
2 J. COGUA ET AL.

(e.g., a predictor and an outcome) by group status. This analysis method draws on
well-established statistical techniques, such as analysis of variance, product-term multi-
ple regression, and multiple-group structural equation modeling (Aguinis &
Gottfredson, 2010; Frazier, Tix, & Barron, 2004). However, as we discuss in subsequent
sections, researchers often rely on subgroup analysis practices with limitations that cast
doubt on the reproducibility and validity of findings (Supplee, Kelly, MacKinnon, &
Barofsky, 2013), particularly in tests of racial and ethnic group differences. Current
critiques of these subgroup analysis practices are spread throughout the literature in
different content areas. For instance, Rothwell (2005) as well as Wang, Lagakos, Ware,
Hunter, and Drazen (2007), highlights suspect subgroup analysis results found within
medical research; Farrell, Henry, and Bettencourt (2013) address subgroup analysis
results from universal school-based youth violence prevention trials; and Rothman
(2013) discusses how tests of subgroup effects, are filled with conceptual and analytic
challenges within intervention research (e.g., violence prevention). These are only few
of the multiple critiques that interested readers can explore in greater depth.
Here, we review and systematically summarize those critiques by focusing on specific
practice areas. Additionally, we provide a rubric of requirements for conducting sub-
group analysis at both the exploratory and confirmatory levels of research. The rubric
targets two audiences: (1) researchers aiming to implement best research practices and
(2) consumers of research looking to evaluate the quality of findings in subgroup
studies. The goal of this rubric is to guide decision making in the design of research
studies and the use of resultant findings to ensure a sound evidence base for effectively
informing policies and practices in ameliorating racial and ethnic group health dispa-
rities. The compilation of current challenges and the formulation of a rubric have not
been concisely summarized before and are much-needed tools in realizing the max-
imum potential of subgroup analyses.

Subgroup analysis rubric: best practices by area and type of research

Critiques of subgroup analyses can be organized into three practice areas: theoretical,
methodological, and interpretational. The challenges in these areas apply to exploratory as
well as confirmatory studies (Bloom & Michalopoulos, 2013). However, the particular
considerations and recommendations vary depending on the nature of the research and its
goals. Both exploratory and confirmatory subgroup studies play a role in the science of
understanding health disparities. Whereas exploratory subgroup analyses are best viewed
as the frontier of new inquiries, confirmatory studies provide empirical evidence suitable
for becoming the foundation of effective initiatives. As illustrated in Table 1, we propose
in our rubric a set of foundational requirements that must be considered in any type of
subgroup analysis (top row), and then address considerations unique to exploratory
(middle row) and confirmatory (bottom row) studies. Below, we highlight the challenges
faced in subgroup analyses within the three major practice areas and offer recommenda-
tions for best practices.
 JOURNAL OF EVIDENCE-INFORMED SOCIAL WORK 3

Table 1. Rubric of best practices for conducting subgroup analysis at both the exploratory and
confirmatory levels of research.
Practice Areas
Theory Methodology Interpretation
Foundational ● Contextualizes: ● Involves target group ● Reports findings with
Requirements members at all stages of transparency
○ Research questions research design & ● Clearly and fully
○ Study rationales implementation acknowledges study
○ Research settings ● Ensures that assessments limitations
● Defines & justifies are culturally and linguisti- ● Avoids over-
groups cally relevant generalization
● Implements mixed
methods
● Maximizes measurement
reliability

Exploratory Subgroup ● States exploratory ● Emphasizes avoiding Type ● Refrains from making
Analysis intent II errors policy & intervention
● Begins search for ● Comprehensively reports recommendations
confounds results from all analyses ● Explicitly calls for confir-
conducted matory studies and
further research

Confirmatory Subgroup ● Grounds study in the- ● Preregisters study, hypoth- ● Makes relevant policy &
Analysis ory and prior research eses, and tests intervention recommen-
● Offers clear testable ● Emphasizes avoiding Type dations, when findings
hypotheses and I errors allow
replications ● Uses confirmatory data
● Defines groups at suffi- analysis techniques
cient level of specificity ● Limits multiple testing or
● Tests confounds applies adjustments
● Implements probability
sampling techniques
● Has sufficient total & sub-
group samples sizes

Theoretical practice
Challenges
Subgroup analyses often lack adequate theoretical justification (Dressler, Oths, & Gravlee,
2005; Rivera, 2014). Though theory will be less well-developed in exploratory studies,
many such studies still fail to provide an adequate account of the relevant context
surrounding subgroup questions and research settings. This is especially important as
the likelihood of generating stigma or incorrectly confirming stereotypes (e.g., elevated
alcohol abuse among Native Americans) is greater when results from subgroup analyses
are inadequately situated within the larger context in which they are embedded (e.g.,
poverty, historical trauma). Atheoretical subgroup analyses are also more likely to over-
look considerations related to potential confounding. Even statistically significant group
differences from large-scale studies using reliable measures can be misleading if relevant
confounds have not been addressed. For example, as noted by the intersectionality theory
of racial discrimination (Lopez & Gadsden, 2016; Veenstra, 2011), race and ethnicity are
intertwined with socioeconomic status; therefore, it can be difficult to determine with
4 J. COGUA ET AL.

confidence whether any apparent group differences are due to racial and ethnic group
membership or identification, per se, or to socioeconomic factors.

Recommendations
(See Table 1, Theory column). As a foundational requirement, it is pertinent for research-
ers to contextualize the research being conducted by clearly specifying the research
questions, rationales, and setting of the study (e.g., describing the historical and current
conditions relevant to participants). These accounts allow consumers of research to have
a clearer understanding of potentially existing factors that may influence the study and
have an impact on the meaning and relevance of the findings. Furthermore, it is funda-
mental to carefully define and justify the groups being examined, which, in many studies,
are operationalized more generally than preferred (e.g., White versus Hispanic/Latino).
Meaningful subgroups within larger group categorizations often exist (Harachi, Catalano,
Kim, & Choi, 2001), and multiracial or multiethnic identifications are common (Choi,
Harachi, Gillmore, & Catalano, 2006) and should be acknowledged within subgroup
analyses.
Exploratory studies must be clearly labeled as such and must not be made to appear as
confirmatory in nature. Exploratory analyses understandably may lack full theoretical
specification, but ideally should examine subgroup differences within a carefully defined
context, as noted above, while beginning the search for potential confounds. For example,
tests of racial and ethnic subgroup differences in health-related outcomes might include
adjustments for socioeconomic status and other social determinants of health. Researchers
may need to specify subgroups more generally than preferred in exploratory research,
which is a limitation that can be somewhat mitigated through proper contextualization.
Confirmatory studies, by nature, should be based upon prior empirical research and/or
be theory-guided. With some notable exceptions (e.g., intersectionality theory of racial
discrimination), few theories currently exist to provide guidance on potential racial and
ethnic group differences in health disparities outcomes (Lopez & Gadsden, 2016; Rivera,
2014). Further theoretical advancements are needed before confirmatory analyses can
become the norm. Specifically, theories that address potential confounds and other
explanatory factors, as well as offer testable hypotheses about racial and ethnic group
differences, will contribute to research advancements. Replication studies examining
observed group differences and other findings from exploratory studies are also needed
(Valentine et al., 2011). Finally, confirmatory studies should define subgroups with
a greater and more meaningful level of specificity than exploratory studies, and should
have clear hypotheses about anticipated group differences.

Methodological practices
Challenges
Study designs and statistical analyses are often problematic in published subgroup analyses
(Rothman, 2013; Rothwell, 2005; Wang et al., 2007). Too frequently, studies are imple-
mented without adequate representation and involvement of group representatives. This
makes it difficult to properly contextualize the research as a whole. Moreover, published
 JOURNAL OF EVIDENCE-INFORMED SOCIAL WORK 5

subgroup studies commonly are based on small sample sizes. The recruitment of too few
participants can lead to underpowered subgroup analyses, as studies are typically powered
for main effects in the total sample. It has been shown that sample sizes in adequately
powered studies of main effects may need to increase as much as four-fold to achieve
adequate power for subgroup moderation tests (Supplee et al., 2013; Wang & Ware, 2013).
The relative sizes of the groups themselves also need to be considered, and not only for
statistical power considerations. The lack of subgroup member participation may exacer-
bate negative stigmas created by poorly defined or inadequately specified subgroups.
Likewise, the use of non-probability sampling, which is common in subgroup studies,
limits representativeness. These practices likely lead to the lack of reproducibility and poor
generalizability of findings commonly found in the social sciences (Ioannidis, 2005).
A significant concern in many subgroup analyses is the impact of measurement error
(Aguinis & Gottfredson, 2010), which is compounded with commonly used analytic
techniques, such as product-term regression. The extent to which either the moderator
(i.e., grouping variable) or the predictor is reliably measured requires careful consid-
eration, as bias in the estimated relationship between the product term and an outcome
can occur and reflect an underestimation of the interaction effect in the presence of
measurement error (Jaccard & Wan, 1995). Moreover, conducting a large number of
interaction tests in subgroup analyses is common practice and raises concerns about
multiple testing if there is a lack of correction for the family-wise error rate (Wang &
Ware, 2013), resulting in an increased probability of false positive subgroup differences.
Other challenges in the selection of a proper statistical model for testing hypotheses
about group differences, particularly related to racial and ethnic health disparities,
include the non-normal distribution of many health-related outcomes (e.g.,
a preponderance of zeros representing the absence of a condition that has a low base
rate in the population). Commonly used estimators, such as maximum likelihood, can
be inappropriate for these kinds of outcomes, leading to incorrect conclusions about
subgroup differences.

Recommendations
(See Table 1, Methodology column). To address these concerns at the foundational level,
we urge the active involvement of individuals representing the racial and ethnic groups of
interest throughout the research process, from the formulation of the research questions
to the analysis, interpretation, and reporting phases. Principles drawn from community-
based participatory research and participant-centered outcomes research are valuable here
(Israel, Schulz, Parker, & Becker, 2001; Witteman et al., 2018). The inclusion of diverse
community members will encourage the co-production of new knowledge and the devel-
opment of innovative and sustainable policy recommendations (Cacari-Stone, Wallerstein,
Garcia, & Minkler, 2014; Winterbauer, Bekemeier, VanRaemdonck, & Hoover, 2016).
Incorporating subgroup stakeholders helps ensure sufficient representation of the targeted
community and can facilitate the application of research findings into practice (Bogart &
Uyeda, 2009; Israel et al., 2001), a point to which we will return below. It is also important
to be attuned to the need for culturally and linguistically sensitive assessments in studies of
subgroups (i.e., instrument translation; Formea et al., 2014) and to have a thorough
6 J. COGUA ET AL.

understanding, ideally based on theory, of intra-group differences and disparities (Bogart

& Uyeda, 2009).
We also recommend the implementation of mixed methods techniques for all types of
subgroup studies. The use of both qualitative and quantitative research methods increases
the scale and depth of the understanding acquired (Creswell & Plano Clark, 2011; Wang &
Ware, 2013). For example, researchers can collect quantitative survey data and comple-
ment it with focus groups to explore how subgroup members describe or understand
a topic, and use the findings to develop and further refine quantitative data collection
instruments. Lastly, at the foundational level, we echo the call that researchers and
statisticians have put forth to maximize the reliability of measures and to consider the
use of latent variable analysis techniques, while implementing sound study designs that
balance internal and external validity to ensure results are accurate and generalizable
beyond the particular study (Shadish, Cook, & Campbell, 2002).
Smaller subgroup and overall sample sizes are common in exploratory studies, and
the resultant lower statistically powered analyses should be acknowledged openly. There
have been long-standing calls to adjust the alpha level depending on factors such as the
potential adverse consequences of making Type I versus Type II errors and the nature
of the study (e.g. small sample or exploratory; Salkind, 2016; Schumm, Pratt,
Hartenstein, Jenkins, & Johnson, 2013; Warner, 2008). Thus, we believe that concerns
over Type I errors or false positive findings may be relaxed in exploratory studies to
avoid overlooking emergent patterns of group differences. In these instances, research-
ers may choose to raise the alpha level for significance testing (e.g., p < .10) or conduct
multiple testing without adjustments. For example, Cohen (1992) called attention to
these issues in his influential primer on statistical power by acknowledging “circum-
stances in which a less rigorous standard for rejection is desired, as, for example, in
exploratory studies” (p. 156). These decisions must be clearly documented and their
limitations must be readily acknowledged. We recommend that all tests performed
within exploratory studies be comprehensively reported to plainly illustrate potential
patterns of group differences that can be targeted for further investigation in con-
firmatory studies.
As a best practice for confirmatory subgroup studies, we recommend pre-registration of
the research design and study’s hypotheses and analyses. Pre-registration helps prevent
post-hoc data snooping (e.g., p-hacking; Head, Holman, Lanfear, Kahn, & Jennions, 2015)
and improves the credibility and transparency of research by potentially reducing pub-
lication bias and selective reporting (Nosek, Ebersole, DeHaven, & Mellor, 2018; Van’t
Veer & Giner-Sorolla, 2016). By definition, confirmatory subgroup studies will favor the
use of confirmatory data analysis techniques, such as regression analysis and structural
equation modeling, over more exploratory techniques, such as latent class analysis.
Representative sampling procedures are also recommended as a best practice for con-
firmatory subgroup studies. Pragmatically, it can be difficult to obtain the required sample
size for certain target groups. In these cases, it may be useful to oversample less prevalent
racial and ethnic groups in larger-scale confirmatory studies of subgroup differences
(Aguinis & Gottfredson, 2010; Supplee et al., 2013) or to integrate multiple relevant
data sets to increase sample size. In contrast to exploratory subgroup analyses, confirma-
tory subgroup analyses should limit Type I errors and the potential for drawing false
positive conclusions. For example, multiple testing should be minimized and, if
 JOURNAL OF EVIDENCE-INFORMED SOCIAL WORK 7

implemented, adjustments should be made to control for the family-wise error rate or for
the false discovery rate (Benjamini & Hochberg, 1995).
To conclude this section, we note that addressing methodological concerns in subgroup
studies comes with challenges of its own. With fixed or decreasing budgets for federally-
funded health-related studies, researchers will need to balance considerations of sample
size against other design features, such as the breadth and depth of assessments. Standards
found in the Consolidated Standards of Reporting Trials (CONSORT; Schulz, Altman, &
Moher, 2010) and similar frameworks can provide valuable guidelines for researchers.
Most importantly, a new generation of more deliberately designed studies that are
optimized rather than retrofitted for tests of racial and ethnic group differences are
needed, for example, by posing questions about racial or ethnic group differences at the
study design phase and conducting appropriate sampling to adequately capture the groups
of interest (e.g., Haggerty, Skinner, MacKenzie, & Catalano, 2007).

Interpretation practices
Challenges
Findings from theoretically underdeveloped and poorly designed subgroup studies have
the potential to inform practices and policies that, instead of addressing racial and ethnic
disparities and promoting health equity, can reify rigid categorizations that perpetuate
negative stereotypes, oversimplify complex realities, and contribute to the stigmatization
of groups identified as having elevated needs. Given the finite resources available for
addressing health disparities, confidence in the validity of research results showing group
differences (or lack thereof) is required before making policy recommendations.
Misdirecting resources by unnecessarily singling out certain groups over others could
exacerbate rather than ameliorate health disparities, for example, if stigma due to the
reinforcement of negative stereotypes engenders hopelessness and leads individuals to
disengage from effective prevention and treatment interventions.

Recommendations
(See Table 1, Interpretation column). As best practices in theory and methods are
implemented according to our rubric, interpretation challenges will decrease. At the outset
of a study, in close consultation with group members, researchers should carefully con-
sider the potential positive or negative implications of hypothesized racial and ethnic
group differences tested in subgroup analyses. As a foundational research requirement,
transparency is essential for the correct interpretation of findings, for example, to make
clear the degree to which analyses are exploratory or confirmatory in nature (Ioannidis,
Stanley, & Doucouliagos, 2017). Additionally, researchers should clearly acknowledge the
limitations of their study (e.g., regarding sample size, power, sampling, etc.), to appro-
priately facilitate the interpretation of results and possible recommendations. Moreover, as
stated previously, both types of studies should include relevant information central for
situating subgroup analyses and their findings within the larger context of the phenomena
under investigation. We recommend taking steps to minimize the potential for generating
problematic stigmatization from singling out specific racial and ethnic groups. In
8 J. COGUA ET AL.

particular, researchers should avoid over-generalization in the interpretation of results

with respect to racial and ethnic groups.
Exploratory analyses typically will not be sufficiently developed to support strong policy
statements. Thus, researchers are encouraged to withhold such statements within the
context of exploratory studies and, instead, call for confirmatory replications.
Conversely, policy-oriented recommendations may derive from a confirmatory approach
in which research questions related to specific racial and ethnic groups are identified at the
outset of the study. Ideally, such interpretations and recommendations will be theory-
driven and provided with input from group members involved throughout the research
process.

Conclusion
The possibility of harm resulting from poorly conducted racial and ethnic group analyses
in research is significant. Under common research practices, there is considerable poten-
tial for non-reproducible findings that can lead to stigma and, if applied to policy and
practice, result in misguided intervention efforts and the misallocation of limited resources
for addressing behavioral, mental, and physical health disparities. By contrast, the promise
of subgroup analyses conducted well, reported transparently with careful consideration of
the implications for policy and practice, and in consultation with group members is
significant. Such work is time-consuming, costly, and difficult to conduct. However, the
stakes are too high to settle for anything less. Racial and ethnic minority health disparities
are deeply rooted in the broader socio-political and economic context within the United
States and elsewhere. Hope for redressing such disparities rests on developing
a foundation of sound, theory-guided knowledge about meaningful racial and ethnic
group differences in health-related outcomes and processes.
We have condensed current critiques of the use of subgroup analysis and offered
a practical rubric for best practices in subgroup studies to help guide both researchers
and consumers of research to maximize the knowledge gained. Some of our recommenda-
tions (e.g., minimize measurement error) cut across all areas of research, but we have
shown how general challenges to the reliability and validity of research results can have
a unique and disconcerting impact on racial and ethnic subgroup studies. Current
research practices in subgroup studies are plagued with long-standing problems of the
type described herein (e.g., atheoretical analyses, small sample sizes), and changes in those
practices are evolving too slowly to effectively guide rapid advancements in health
disparities interventions and policies. Our rubric provides a practical tool that holds
promise for accelerating improvements in the transparency and rigor of subgroup studies
to strengthen the knowledge base. Such knowledge can lead to the development of care-
fully designed intervention and policy efforts that responsibly use available resources for
best meeting the needs of at-risk groups.

References
Aguinis, H., & Gottfredson, R. K. (2010). Best-practice recommendations for estimating interaction
effects using moderated multiple regression. Journal of Organizational Behavior, 31, 776–786.
doi:10.1002/job.686
 JOURNAL OF EVIDENCE-INFORMED SOCIAL WORK 9

Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful
approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological), 57,
289–300. doi:10.1111/j.2517-6161.1995.tb02031.x
Bloom, H. S., & Michalopoulos, C. (2013). When is the story in the subgroups? Prevention Science,
14, 179–188. doi:10.1007/s11121-010-0198-x
Bogart, L. M., & Uyeda, K. (2009). Community-based participatory research: Partnering with
communities for effective and sustainable behavioral health interventions. Health Psychology,
28, 391–393. doi:10.1037/a0016387
Borrell, L. N., Kiefe, C. I., Diez-Roux, A. V., Williams, D. R., & Gordon-Larsen, P. (2013). Racial
discrimination, racial/ethnic segregation, and health behaviors in the CARDIA study. Ethnicity &
Health, 18, 227–243. doi:10.1080/13557858.2012.713092
Cacari-Stone, L., Wallerstein, N., Garcia, A. P., & Minkler, M. (2014). The promise of
community-based participatory research for health equity: A conceptual model for bridging
evidence with policy. American Journal of Public Health, 104, 1615–1623. doi:10.2105/
AJPH.2014.301961
Choi, Y., Harachi, T. W., Gillmore, M. R., & Catalano, R. F. (2006). Are multiracial adolescents at
greater risk? Comparisons of rates, patterns, and correlates of substance use and violence between
monoracial and multiracial adolescents. American Journal of Orthopsychiatry, 76, 86–97.
doi:10.1037/0002-9432.76.1.86
Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.
Colby, S. L., & Ortman, J. M. (2014). Projections of the size and composition of the U.S. population:
2-14 to 2060. (P25-1143). Washington, DC: U.S. Department of Commerce, Economics and
Statistics Administration, U.S. Census Bureau. Retrieved from https://www.census.gov/content/
dam/Census/library/publications/2015/demo/p25-1143.pdf
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (3rd
ed.). Thousand Oaks, CA: Sage.
Dressler, W. W., Oths, K. S., & Gravlee, C. C. (2005). Race and ethnicity in public health research:
Models to explain health disparities. Annual Review of Anthropology, 34, 231–252. doi:10.1146/
annurev.anthro.34.081804.120505
Farrell, A. D., Henry, D. B., & Bettencourt, A. (2013). Methodological challenges examining
subgroup differences: Examples from universal school-based youth violence prevention trials.
Prevention Science, 14, 121–133. doi:10.1007/s11121-011-0200-2
Formea, C. M., Mohamed, A. A., Hassan, A., Osman, A., Weis, J. A., Sia, I. G., & Wieland, M. L.
(2014). Lessons learned: Cultural and linguistic enhancement of surveys through
community-based participatory research. Progress in Community Health Partnerships: Research,
Education, and Action, 8, 331–336. doi:10.1353/cpr.2014.0037
Frazier, P. A., Tix, A. P., & Barron, K. E. (2004). Testing moderator and mediator effects in
counseling psychology research. Journal of Counseling Psychology, 51, 115–134. doi:10.1037/
0022-0167.51.1.115
Guerrero, E. G., Marsh, J. C., Duan, L., Oh, C., Perron, B., & Lee, B. (2013). Disparities in
completion of substance abuse treatment between and within racial and ethnic groups. Health
Services Research, 48, 1450–1467. doi:10.1111/1475-6773.12031
Haggerty, K. P., Skinner, M. L., MacKenzie, E. P., & Catalano, R. F. (2007). A randomized trial of
parents who care: Effects on key outcomes at 24-month follow-up. Prevention Science, 8,
249–260. doi:10.1007/s11121-007-0077-2
Harachi, T. W., Catalano, R. F., Kim, S., & Choi, Y. (2001). Etiology and prevention of substance
use among Asian American youth. Prevention Science, 2, 57–65. doi:10.1023/A:101003901
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015). The extent and
consequences of p-hacking in science. PLoS Biology, 13, 1–15. doi:10.1371/journal.pbio.1002106
Humes, K., Jones, N., & Ramirez, R. (2011). Overview of race and hispanic origin: 2010. (C2010BR-
02). Washington, DC: U.S. Department of Commerce, Economics and Statistics Administration,
U.S. Census Bureau. Retrieved from https://www.census.gov/prod/cen2010/briefs/c2010br-02.pdf
Ioannidis, J. P. (2005). Why most published research findings are false. PLoS Medicine, 2,
0696–0701. doi:10.1371/journal.pmed.0020124
10 J. COGUA ET AL.

Ioannidis, J. P., Stanley, T. D., & Doucouliagos, H. (2017). The power bias in economics research.
The Economic Journal, 127, F236–F265. doi:10.1111/ecoj.12461
Israel, B. A., Schulz, A. J., Parker, E. A., & Becker, A. B. (2001). Community-based participatory
research: Policy recommendations for promoting a partnership approach in health research.
Education for Health, 14, 182–197. doi:10.1080/13576280110051055
Jaccard, J., & Wan, C. K. (1995). Measurement error in the analysis of interaction effects between
continuous predictors using multiple regression: Multiple indicator and structural equation
approaches. Psychological Bulletin, 117, 348–357. doi:10.1037/0033-2909.117.2.348
Lopez, N., & Gadsden, V. (2016). Health inequalities, social determinants, and intersectionality.
National Academy of Medicine. Retrieved from https://nam.edu/health-inequities-social-
determinants-and-intersectionality/
Mason, M. J., Mennis, J., Linker, J., Bares, C., & Zaharakis, N. (2014). Peer attitudes effects on
adolescent substance use: The moderating role of race and gender. Prevention Science, 15, 56–64.
doi:10.1007/s11121-012-0353-7
National Institutes of Health. (2016). NIH health disparities strategic plan and budget fiscal year
2009-2013. Retrieved from https://www.nidcd.nih.gov/about/strategic-plan/health-disparities
/2009-2013
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration
revolution. Proceedings of the National Academy of Sciences, 115, 2600–2606. doi:10.1073/
pnas.1708274114
Reeb, B. T., Chan, S. Y. S., Conger, K. J., Martin, M. J., Hollis, N. D., Serido, J., & Russell, S. T.
(2015). Prospective effects of family cohesion on alcohol-related problems in adolescence:
Similarities and differences by race/ethnicity. Journal of Youth and Adolescence, 44, 1941–1953.
doi:10.1007/s10964-014-0250-4
Rivera, L. M. (2014). Ethnic-racial stigma and health disparities: From psychological theory and
evidence to public policy solutions. Journal of Social Issues, 70, 198–205. doi:10.1111/
josi.12055
Rothman, A. J. (2013). Exploring connections between moderators and mediators: Commentary on
subgroup analyses in intervention research. Prevention Science, 14, 189–192. doi:10.1007/s11121-
012-0333-y
Rothwell, P. M. (2005). Subgroup analysis in randomized controlled trials: Importance, indication
and interpretation. Lancet, 365, 176–186. doi:10.1016/S0140-6736(05)17709-5
Salkind, N. J. (2016). Statistics for people who (think they) hate statistics (6th ed.). Thousand Oaks,
CA: Sage.
Schulz, K. F., Altman, D. G., & Moher, D. (2010). CONSORT 2010 statement: Updated guidelines
for reporting parallel group randomised trials. BMC Medicine, 8, 698. doi:10.1186/1741-7015-
8-18
Schumm, W. R., Pratt, K. K., Hartenstein, J. L., Jenkins, B. A., & Johnson, G. A. (2013).
Determining statistical significance (alpha) and reporting statistical trends: Controversies, issues,
and facts. Comprehensive Psychology, 2, 1–6. doi:10.2466/03.CP.2.10
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs
for generalized causal inference (2nd ed.). Boston, MA: Houghton Mifflin Company.
Supplee, L. H., Kelly, B. C., MacKinnon, D. M., & Barofsky, M. Y. (2013). Introduction to the
special issue: Subgroup analysis in prevention and intervention research. Prevention Science, 14,
107–110. doi:10.1007/s11121-012-0335-9
Valentine, J. C., Biglan, A., Boruch, R. F., Castro, F. G., Collins, L. M., Flay, B. R., … Schinke, S. P.
(2011). Replication in prevention science. Prevention Science, 12, 103–117. doi:10.1007/s11121-
011-0217-6
Van’t Veer, A. E., & Giner-Sorolla, R. (2016). Pre-registration in social psychology—A discussion
and suggested template. Journal of Experimental Social Psychology, 67, 2–12. doi:10.1016/j.
jesp.2016.03.004
Veenstra, G. (2011). Race, gender, class, and sexual orientation: Intersecting axes of inequality and
self-rated health in Canada. International Journal for Equity in Health, 10, 3. doi:10.1186/1475-
9276-10-3
 JOURNAL OF EVIDENCE-INFORMED SOCIAL WORK 11

Wang, R., Lagakos, S. W., Ware, J. H., Hunter, D. J., & Drazen, J. M. (2007). Statistics in medicine—
Reporting of subgroup analyses in clinical trials. The New England Journal of Medicine, 357,
2189–2194. doi:10.1056/NEJMsr077003
Wang, R., & Ware, J. H. (2013). Detecting moderator effects using subgroup analyses. Prevention
Science, 14, 111–120. doi:10.1007/s11121-011-0221-x
Warner, R. M. (2008). Applied statistics: From bivariate through multivariate techniques. Los
Angeles, CA: Sage.
Windsor, L. C., Jemal, A., & Alessi, E. (2015). Cognitive behavioral therapy: A meta analysis of race
and substance use outcomes. Cultural Diversity and Ethnic Minority Psychology, 121, 300–313.
doi:10.1037/a0037929
Winterbauer, N. L., Bekemeier, B., VanRaemdonck, L., & Hoover, A. G. (2016). Applying
community-based participatory research partnership principles to public health practice-based
research networks. Sage Open, 6, 1–13. doi:10.1177/2158244016679211
Witteman, H. O., Dansokho, S. C., Colquhoun, H., Fagerlin, A., Giguere, A. M., Glouberman, S., …
Légaré, J. (2018). Twelve lessons learned for effective research partnerships between patients,
caregivers, clinicians, academic researchers, and other stakeholders. Journal of General Internal
Medicine, 33, 558–562. doi:10.1007/s11606-017-4269-6