JBR-08737; No of Pages 6

Journal of Business Research xxx (2015) xxx–xxx

Contents lists available at ScienceDirect

Journal of Business Research

Heresies and sacred cows in scholarly marketing publications☆

Barry J. Babin a,⁎, Mitch Griffin b,1, Joseph F. Hair Jr. c

a
Max P. Watson Jr., College of Business Research, Louisiana Tech University, Ruston, LA 71227, USA
b
Department of Marketing, Bradley University, Peoria, IL 61625, USA
c
Department of Marketing, Kennesaw State University, Kennesaw, GA 30144, USA

a r t i c l e i n f o a b s t r a c t

Article history: Merriam-Webster defines heresies as “dissent or deviation from a dominant theory, opinion, or practice.” This
Received 1 April 2015 Journal of Business Research special issue and the editorial examine heresies and sacred cows in marketing re-
Received in revised form 1 September 2015 search. Seven papers investigate different aspects of typical academic business journal presentations. Each man-
Accepted 1 September 2015
uscript critically analyzes generally accepted practices for the pursuit of publication in academic journals and
Available online xxxx
reveals ways these practices may do more harm than good, hindering the goal of presenting true growth of
Keywords:
knowledge through publication. The editorial provides an integrative schema for the manuscripts in the special
Heresy issue. Providing a series of broader topics to tie the papers together, this special issue illustrates how the findings
Knowledge of each study can help improve our pursuit of knowledge. In addition, the editorial discusses heresies and sacred
Journals cows not covered by manuscripts in the current issue. The editorial concludes with recommendations for both
Method authors and reviewers that may enhance the approach to research, methodologies employed, and reporting of
Marketing Research scholarly research.
© 2015 Elsevier Inc. All rights reserved.

Marketing academics, like those in other disciplines, conduct re-
search motivated by a desire to publish reports of their research in aca-
demic journals. Herein lays a potential dilemma. The demands of
maintaining the highest standards in research may conflict with the
norms of the publication process. When senior faculty train doctoral
students and mentor junior faculty, this dilemma evidences itself
when an emphasis on “playing the game” of publication takes precedent
over presenting a descriptive account of the research in a meaningful
way. Sometimes, this process plays out subtly, such as through sugges-
tions of building a paper's reference list with an eye toward flattering
particular members of a journal's ERB who might review the manu-
script. Other times, academics may withhold a preponderance of evi-
dence to emphasize primarily desirable results—in other words, those
results that are consistent with the author's enlightened predictions.
Taken to the extreme, study data and/or results may be, implicitly or
otherwise, a work of fiction to present what the author perceives to be
the received view (Enserink, 2012). The pressures and desires to be-
come an author in a noted journal are strong and, after all, researchers
☆ The authors thank O.C. Ferrell, Arch G. Woodside, and Marko Saarstadt for helpful
comments on this editorial. In addition, the authors thank Nina Krey and Christian
Bushardt for help in editing the manuscript.
⁎ Corresponding author. Tel.: +1 318 257 4012.
E-mail addresses: bbabin@latech.edu (B.J. Babin), mg@bradley.edu (M. Griffin),
Jfhair3@kennesaw.edu (J.F. Hair).
1
Tel.: +1 309 677 2287.
often have a lot of confidence in their theories (Stapel, 2012). Thus,
they may feel justified in writing a narrative instead of a report.
A cursory look at today's journals suggests a dogmatic presence in
the manner in which authors present research. For example, is there a
specific way that authors must present an empirical marketing research
article to survive the review process and eventually end up in a journal's
pages? Clearly, authors must write well and present discipline-relevant
topics. However, must papers stay generally within well-defined
boundaries and styles of presentation to appease reviewers and editors?
Must an article first present some deductively driven hypotheses based
on an already known and named theory, followed by an empirical test
that presents results corroborating those same hypotheses? Must arti-
cles generally follow current precedents and “generally accepted” pro-
cedures? Must the articles use fashionable theories, typically derived
outside the discipline, at the expense of discovering new theories from
within marketing? Finally, must the researchers employ trendy
methods and analytical tools even if they possess little understanding
of their actual relevance, precision, or appropriateness?
The articles in this special issue examine themes related to current
trends in marketing academic articles. In this essay, the special issue ed-
itors provide a brief overview of each paper and its contribution. In
doing so, they call attention to just a few of the common practices and
procedures applied in the academic marketing literature, with an eye
toward understanding those that may constitute “sacred cows” more
than vehicles for building a better and more meaningful literature. In a

http://dx.doi.org/10.1016/j.jbusres.2015.12.001
0148-2963/© 2015 Elsevier Inc. All rights reserved.

Please cite this article as: Babin, B.J., et al., Heresies and sacred cows in scholarly marketing publications, Journal of Business Research (2015),
http://dx.doi.org/10.1016/j.jbusres.2015.12.001
2 B.J. Babin et al. / Journal of Business Research xxx (2015) xxx–xxx
classic commentary on the marketing literature nearly 35 years ago,
Peter & Olson (1983, p. 116) argue not that marketing is science, but
“science is marketing.” Interestingly, when interviewed about the moti-
vation for fabricating study results as a basis for academic journal pub-
lications, Staple expresses the same notion:
It was a quest for aesthetics, for beauty—instead of the truth… it's
hard to know the truth… you need grants, you need money…
science is of course about discovering the truth, about digging to find
the truth… but it is also communication, persuasion, marketing.
(Stapel quoted in an interview by Bhattacharjee, 2013, p. MM44).
If a journal article reports truth objectively, the prose should avoid
falling into a narrative. However, when some passionate motivation
slips in and interferes with the objective presentation of study results,
the line between fiction and nonfiction may blur. Thus, when research
is presented in a manner other than that in which it is conducted—when
it is presented more with an eye toward passing review than simple de-
scription, when the meaning becomes obscured by technical complexity
intended to impress reviewers, when trivial (albeit perhaps statistically
significant) effects are presented as important, when data are portrayed
or treated with more precision than the research method allows, when
hypotheses are conceptualized post hoc based on study results, when
results that do not present the “desired” outcomes are not reported, ar-
ticles stray from the basic presentation of the truth. The narrative that
the authors might have written without any research remains
uninterrupted.
Hunt (2010, p. 306) presents a thorough discussion of various ways
that research can come to present fiction. He portrays a continuum be-
tween truth and TRUTH, with the former being an objective representa-
tion of reality and the latter being the T-hoc RUTH that MUST exist. Hunt
uses the table to illustrate how dogmatic philosophies result in a lack of
regard for reality. The mere existence of a continuum implies that the
difference between truth and TRUTH is not always so clear. Even if not
dogmatic in philosophy, does academic research slip into dogma in pre-
sentation? When papers are written with the idea in mind that they
MUST contain things (such as carefully deduced hypotheses, specific an-
alytical tools regardless of applicability, statistically significant results,
supported hypotheses, etc.) or MUST be written in certain ways (deduc-
tively testing in the context of justification), the literature risks
truth
Reporting results based on statistical significance
or ability to support hypotheses
Building conceptualizations after considering
study results
Employing irrelevant data
Plagiarism
Reporting tests that were not conducted
Data “tricking”–forcing a desired result through
slight manipulations
TRUTH
Fig. 1. Some issues affecting the truth of scholarly research.
Please cite this article as: Babin, B.J., et al., Heresies and sacred cows in scholarly marketing publications, Journal of Business Research (2015),
http://dx.doi.org/10.1016/j.jbusres.2015.12.001
becoming in part or perhaps fully TRUTH rather than truth. Fig. 1 depicts
this notion graphically.
Below are a series of assumptions about the publication standards
for papers that appear in marketing journals. The articles in this special
issue critically examine these widely accepted beliefs to see if in fact
they represent appropriate standards for scientific discovery or may
be sacred cows of the discipline.
1. Journal articles should be deductive
Researchers may not like to consider themselves as bumblers, but
Alba (2012) points to bumbling as the way most scientific break-
throughs occur. In fact, Alba (2012) reminds the marketing and con-
sumer research academy of the simple admonition that scientists
advance knowledge more often by discovering an unexpected regulari-
ty than by testing an expected regularity. Given that scientific progress
lies in discovery, it's somewhat ironic that the academic literature, and
the ubiquitous review process, often works to stifle discovery so much
so that even academics with a distinguished publishing career may
find it difficult to answer the “what did you discover” question
(Armstrong, 2003).
In the typical marketing article, the introduction section precedes a
section on “conceptual” or “theoretical” development, followed by a
section describing an empirical study. In some papers, the
author(s) may refer to their work as exploratory, simply as a way of
lessening the criticism of problems with generalizability or other short-
comings of the methodology. In the end, marketing authors sacrifice
discovery in favor of justification, although the scientific method clearly
requires attention to both (Hunt, 2010). The lack of theories originating
in marketing may be one price paid for the affinity of the academy to-
ward the hypo-deductive presentation of research. Authors perceive re-
viewers as much more comfortable with some famous theory based in
another discipline than with the derivations that would reflect a theory
born in marketing.
In this issue, Daugherty, Hoffman, and Kennedy (2015) point to the
dearth of reports of inductive studies in the marketing literature, in con-
trast to the “hard sciences” or applied fields like medicine. They illus-
trate an inductive orientation by applying a “reverse approach”
employing neurological measures to illustrate differences in brain
Employing analytical techniques that are not
understood, not called for, without
consideration of precision
Selecting methodologies and analytical
techniques aimed first at appealing to reviewers
Failing to call attention or even report study
nonresults
Improper measurement
Zealous data cleaning
Fabrication
 B.J. Babin et al. / Journal of Business Research xxx (2015) xxx–xxx
activity among consumers across different product types. The brain ac-
tivity may well aid in explaining differences in reactions to varying ad
appeals, including differences in memory across advertisements. The
authors call for greater acceptance of discovery oriented research and
the inductive processes as acceptable methods of discovery. Although
true exploratory research may sometimes be judged as lacking rigor, re-
lying purely on deduction dismisses needlessly all other methods of dis-
covery. Further, in an era of big data, practical marketing researchers
rely heavily on inductive processes for meaningful discovery. Perhaps
not as much of our research needs to follow the dogma of deductively
contrived and statistically supported hypotheses in order to render a
contribution to the discipline.
2. Journal articles should prioritize internal validity
Following the noteworthy debate between the merits of internal and
external validity appearing in the Journal of Consumer Research (for ex-
ample, see Calder, Phillips, & Tybout, 1981, 1983; Lynch, 1983), the
idea of choosing a sample with the purpose of testing an hypothesis rig-
orously with the aim toward falsification, as opposed to showing one
might be true in any circumstance, seems to have lost out. Seldom in
marketing and consumer research articles today do authors give any ex-
plicit mention of to whom the inferences made in their research apply?
Yet, inferences are ubiquitous. In fact, authors give data quality little at-
tention in most articles with the exception of references to standard ci-
tations to allay fears (perhaps “crocodile fears”) of nonresponse bias or
of common methods bias. As a consequence, authors report complicated
tests and draw countless statistical inferences based on samples of
which the reader knows very little, much less what population the sam-
ple represents. Thus, the risk of practical irrelevance looms large.
One can often find disclaimers in research that the authors do not in-
tend the results to generalize and thus any convenience sample should
suffice. One could ask at that point whether the word sample really ap-
plies. If no generalization arises, does a sample exist? In this issue, two
papers address issues related to data quality in the sense of meaningful-
ness based on deriving data from convenience samples. Espinosa and
Ortinau (2015) present data showing the high percentage of articles
employing student samples in the discipline's top journals, along with
the percentage of papers that acknowledge the lack of generalizability
as a result. A minority of published articles in their review note student
samples as a limitation. Only in the Journal of Business Research did more
than half of the papers with student samples note their use as a possible
limitation on the research findings. In addition, their research casts
doubt on the generalizability of student research to other populations,
even in common consumer contexts like restaurants. Clearly, practical
results require greater attention to the relevance of the population to
which samples generalize.
Amazon's mTurk provides a mechanism to crowd source respon-
dents and subjects by making often trivial payments per completed
hit. Smith, Roster, Golden, and Albaum (2015) take an in-depth look
at data quality issues from mTurk samples. In contrast to research
from some years ago, they demonstrate that the typical respondent to
an mTurk survey response request is Asian and likely resides outside
of the U.S.A. Additionally, in contrast to well-managed panels, one can
never be certain exactly who is responding to a request. Their research
demonstrates that mTurk respondents tend to be “speeders” relative to
panel respondents, meaning they work their way through requests at
very fast rates. In general, mTurk data, particularly those responses ob-
tained by non U.S.A. workers, is of relatively poor quality. Most astound-
ingly, mTurk participants average between 10 and 17 survey responses
per week, compared to 3.2 for panel respondents. Thus, the experience
effect looms large among mTurk respondents, making them a particu-
larly poor choice for subjects in marketing and consumer experiments,
as they likely are more prone to pick up on cues indicating how they
should respond in order to obtain the payment. While researchers may
call student subjects into question because of experience effect
Please cite this article as: Babin, B.J., et al., Heresies and sacred cows in scholarly marketing publications, Journal of Business Research (2015),
http://dx.doi.org/10.1016/j.jbusres.2015.12.001
3
concerns, students are not doing near this number of surveys. In fact,
few are likely to do 17 in an entire academic year, much less in a
week. The experience effect in crowd sourced data, particularly in mar-
keting and consumer experiments, should be a serious concern.
In addition, mTurk data requires the most “data cleaning” in general.
One of the greatest risk of mTurk type research participants in general,
beyond their nonnaiveté, resides in the fact that the same researcher
that does the cleaning is likely the one that tests hypotheses. Thus, the
subtle, or not so subtle, temptation is to clean the data in a manner
that shapes support for hypotheses (Chandler, Mueller, & Paolacci,
2014). In comparison, a professionally managed panel offers a huge ad-
vantage in terms of objectivity. High-quality panels often provide data
cleaning as a service. A panel associate removes respondents who are
flat-lining or otherwise failing attention filters from the sample prior
to making the data available to the researcher and therefore prior to
any data analysis related to hypothesis testing. The panel conducts
these services independent of researchers' aims. Unless the researcher
provides a panel employee with a clue, the panel employee who does
the cleaning is blind to the purpose of the research. Thus, a recommend-
ed practice of data cleaning is that the researcher involved in the study
should not perform the data cleaning. A panel often provides such ser-
vices. Further, any data involving the “cleaning” of large percentages
of respondents creates potentially more questions about nonresponse
bias than do late respondents in mail surveys.
3. Journal articles should follow current trends
Authors are particularly susceptible to following trends (or fads?) in
research approaches as reported in journal articles. Three current trends
in survey measurement include researchers' use of single-item mea-
sures, employing formative measures, and extended descriptions of
sometimes complicated assessments of common “variance” resulting
from the use of a single measurement approach. Three articles in the
special issue address these topics.
Can a single item measure a doubly concrete construct? Sarstedt,
Diamantopoulos, Salzberger, and Baumgartner (2015) address this
question. Their results suggest caution in using single-item measures.
Indeed, even with constructs that seem relatively simple such as atti-
tude toward the ad, techniques involving expert opinion and other
methods fail to converge on evidence pointing to which of multiple
items is the best for representing that concept. Furthermore, identifying
the best item through other methods requires measurement with mul-
tiple items, thus defeating the purpose of single-item measures with re-
spect to greater simplicity.
For the past few years, the marketing literature shows a trend toward
increasing numbers of formative measurement operationalizations. In-
deed, researchers sometimes criticize older literature for treating things
as reflective that should be formative (MacCallum & Browne, 1993;
Diamantopoulos & Siguaw, 2006). Wilcox, Howell, and Breivik (2008)
demonstrate the fallacy in thinking of concepts as formative or reflective
and illustrate how a scale might measure a concept either way depending
on context or instructions. In this issue, Chang, Franke, and Lee (2015) ex-
amine the potential for bias across reflective and formative measurement
specifications. Their simulation results demonstrate greater bias in pa-
rameter estimates from formative specifications and signal the greater
robustness of reflective specifications. Thus, with any ambiguity in the
nature of a scale, less bias will occur with reflective operationalization.
Measurement is and will remain critically important and measure-
ment problems will continue to be a source of rejection from publica-
tion. Although one can certainly conduct statistical analyses on and
with poor measures, the meaningfulness of any theory derived, theory
test, or hypothesis examination becomes illusory with deficient mea-
sures. Thus, a battery of measures organized into a meaningful structure
is every bit as much a “theory” as is a set of relationships between con-
structs. As such, just as much care and attention should go into deriving
and examining the measurement theory. Sajtos and Magyar (2015)
4 B.J. Babin et al. / Journal of Business Research xxx (2015) xxx–xxx
remind researchers that the auxiliary theory, being the one that brings
concepts to life for purposes of further analysis, requires the upmost
rigor. The habit of casually selecting scales without a rigorous examina-
tion of item content works against good scientific practice. The failure to
adapt a scale to a context and time works against a well-structured auxil-
iary theory. Dropping items for empirical reasons without due attention
to how the meaning of the concept changes with these deletions also rep-
resents poor practice and disregard for the auxiliary theory. A scale should
be eponymous in the sense that the concept measures should be explicitly
evident among the item battery. The auxiliary theory merits testing as a
whole, just as a structural theory merits testing as a whole.
All research contains problems and design questions with no perfect
answers. From a research standpoint, the researcher needs to possess
full awareness of the potential sources for error. It is important the re-
searcher understands the level of precision present in any measurement.
From a publishing standpoint, the author may work around potential
error problems and acknowledge the most obvious (like the use of stu-
dent samples). Likewise, while procedures may allow no more than two
significant digits, authors often report results to three, four, or more sig-
nificant digits (reporting whatever the software provides) sending a sig-
nal of false precision. The researcher often takes precedent from the
literature without careful consideration. A good example would be the
standard reference to Armstrong & Overton (1977) to allay criticism of
“nonresponse” bias through some check of “late” rather than “non” re-
spondents. The author aimed at publishing success uses precedent blindly
to cover inadequacies in the research. Researcher and author alike should
give issues related to basic data quality due respect and priority. This due
respect and priority results from simple presentation of basic descriptive
statistics.
The early principal components analysis and factor analysis literature
is replete with issues related to response style and halo effects (Hotelling,
1933; Crosby & Stephens, 1987; Oliver & Bearden, 1985). When such a
concern is present, statisticians recommend some form of ipsatization to
remove any potential nuisance or response-style effect (Cattell, 1966;
Johnston, 1973). Outside of personality research, authors seldom report
employing these simple steps today. In fact, basic facts about data go un-
mentioned, so much so that papers often do not report the basic descrip-
tive statistics for prescient variables. Thus, if some variable is in fact a near
constant, the reader may never know. The best solutions to such data
problems typically lie before collecting data rather than after.
In recent years, submissions to marketing journals routinely include
sections to allay the fear of one potential problem, the bias associated
with the fact that respondents recorded responses with only one type
of scale. Fuller, Dickerson, Atinc, Atinc, and Babin (2015) examine this
issue. Upon reviewing JBR articles mentioning procedures to assess
common methods variance, among articles that mention common
methods variance or bias, they find no reports of problematic bias due
to the use of common methods. In the current article, simulation data
suggests that substantial bias given the typical types of variables
employed in marketing studies would not exist until over half of the var-
iance in data is common across all variables (and thus potentially due to a
common method or perhaps some other common response style or halo).
In fact, the level of common method variance that would need to be pres-
ent to cause great concern for bias exceeds even the high estimates of
how much common methods variance may actually exist across studies.
Further, even a simple test of eigenvalues would be sufficient to provide
evidence that the data are free from bias due to common methods. The
so-called Harmon's one-factor test may be more powerful diagnostically
than previously thought. More conservatively, if the first eigenvalue ac-
counts for less than 40% of all the data variance, there appears to be little
concern of any amount of bias due to “common methods” that would dis-
tort meaning given the typical level of precision involved in survey mea-
sures. The research suggests that the amount of attention shown to CMV
in many journal articles may be disproportionate to the risk, particularly
when nothing substantively or practically changes as a consequence of
the tests.
Please cite this article as: Babin, B.J., et al., Heresies and sacred cows in scholarly marketing publications, Journal of Business Research (2015),
http://dx.doi.org/10.1016/j.jbusres.2015.12.001
4. Moving forward as researchers first
The editors of the special issue sought research that questioned some
of today's conventional publication trends and habits – the sacred cows.
The articles in the special issue hit on several of these topics. However, a
few others were not included in the issue. Noting some of the issues that
authors do not examine, this conclusion offers recommendations with
the intention of creating a more practically meaningful literature; a liter-
ature where authors write first as objective researchers rather than po-
tential journal authors desiring publication in prestigious journals to
the point that judgment is impaired. Passion should not create a rooting
interest. Passion should lie in conducting the research with the upmost
respect for representing and/or discovering the truth (not the TRUTH).
Importantly, these recommendations apply to reviewers as well as re-
searchers. All too often reviewers succumb to style over substance and
may wish to inflict the pain they feel as authors on others. Reviewers
may follow red herrings in the form of elegant statistical processes con-
ducted on irrelevant data. Thus, reviewers, as well as authors, need to be
mindful of what is real and meaningful.
Unquestionably, the desire and pressure to publish research and ob-
tain research funding has never been stronger. At times, researchers
stretch the boundaries of good ethics in writing about the results of
their research. Among the relatively small number of studies addressing
unethical publishing behavior, one reports that the number of articles
retracted for fraudulent reporting increased over 10 times since the
1970s (Fang, Steen, & Casadevail, 2012). A survey of Academy of Man-
agement Proceedings suggests that one in four of the papers displays
some elements of plagiarism (Karabag & Berggren, 2012). If the motiva-
tion to publish and receive financial support for research can lead to
outright fraud and plagiarism, then perhaps journal submission authors
also are tempted to design a paper to prioritize publication at the ex-
pense of faithfully reporting research results. Clearly, not all researchers
intentionally mislead. However, even subtle practices in reporting re-
search can perhaps make discovering the full meaning of research diffi-
cult. What might some of these practices be?
5. Lacking full disclosure
Hubbard and Armstrong (1992) report that the percentage of hy-
potheses supported in the top marketing journals is, astoundingly, well
over 90%! A cursory examination of recent articles shows that percent-
age may well have increased to about 95%. How can this be? The goal
of the researcher should be to conduct research with the aim of rejecting
hypotheses, yet journal articles find support for nearly all hypotheses.
Perhaps one reason is a belief that insignificant results will outweigh rig-
orous methodology, so researchers only test very safe hypotheses. So
safe, in fact, that they border on tautology. Another reason could be
that authors cherry-pick research results and only report inferences
that show support. Such cherry-picking could be born in the perhaps
not mistaken belief that reviewers reject papers based on an insufficient
number of statistically significant findings. Armstrong (2003) suggests
that reviewers in top consumer research journals expect hypotheses to
be supported at least 80% of the time, in contrast to high school students
who would expect about half of hypotheses to be supported.
Other potential reasons include the possibility of outright fraud. In a
comprehensive study examining the robustness of findings reported in
psychological science, the researchers find reason for pause. Even after
gaining the cooperation of many of the original authors, attempts to re-
produce findings from the studies generally failed to corroborate the re-
sults presented in journal articles. In the published journal articles, 97%
displayed statistically significant results (p b 0.05). In the attempt to val-
idate the results, only 36% displayed statistically significant results. Ad-
ditionally, the overall average effect size obtained in the replication
studies is less than half of that in the original studies (Open Science
Collaboration, 2015). Further complicating the matter is the file-
drawer effect, where studies examining theoretically hypothesized
 B.J. Babin et al. / Journal of Business Research xxx (2015) xxx–xxx
effects but not finding statistical significance often go unreported. The
end result is that the academic community misunderstands the nature
of relationships because the published literature likely overstates
those relationships and fails to take into account the studies not finding
statistical significance (Hubbard & Armstrong, 1992).
Many journals also display a trend toward articles reporting multiple
studies to address research questions. What happens to the studies con-
ducted along the way that provide unpleasing results to the authors? If
an article reports four studies suggesting some effect, but in the process
studies with results not supporting the researchers hypotheses go unre-
ported, does the article fully represent objective reality? Perhaps eight
studies exist but the literature only reports four. To the extent that un-
reported studies not supporting the authors' hypotheses exist, the sta-
tistical inferences reported in the journal article are misleading. The
literature may proceed toward TRUTH rather than truth.
6. Objective measurement
Objective measurement remains critically important in research
(Sajtos & Magyar, 2015). One can change the bathroom scale to show a
lower scale result, but that does not cause the weight of the person on
that scale to change. As such, formative indicators, from the perspective
of objective measurement, present this unpleasing implication that the
measure causes the phenomenon. Furthermore, they present greater po-
tential for bias (Chang et al., 2015) and, like single-item measures, defy
well-known and understood validation procedures (Sarstedt et al.,
2015; Wilcox et al., 2008). Above all the validation methods for any mea-
sure, face validity remains an imperative unequaled by other tests.
7. Meaningful samples and generalizable research
Researchers should apply procedures with an eye toward faithfully
depicting results, not in a manner to survive the review process. In
this process, the researcher investigates all types of potential research
error. Researchers should address basic questions like the legitimacy
of respondents. If one designs a questionnaire that inflates measure-
ment validity or enhances likelihood of supporting hypotheses, research
flaws exist from the start. If a researcher performs detailed tests of com-
mon methods bias on largely irrelevant data, the tests do not serve
much of a purpose. If the author intends results to have practical mean-
ing, but the research participants bring into question the ability to gen-
eralize results or present experience effects that may distort responses,
an article reporting those results is of reduced value.
Coinciding with sample concerns, generalizability also depends on
research conducted in realistic settings. Relatively new to the literature,
authors are encouraged to adopt the term “boundary conditions” to
specify exactly when some statistical effect occurs. However, the
boundary conditions often limit the practical meaning of the research
results to the point of near irrelevance (Yang & Lynn, 2014). In addition,
the narrow conditions employed in consumer experiments makes re-
producing results difficult. Despite these findings, authors sometimes
feel pressured to overstate the generalizability of their results in an ef-
fort to achieve acceptance for publication in top journals (Huber,
Payne, & Puto, 2014).
8. Conclusions
Considering the issues that this special issue raises, consider these
points as summary recommendations:
• The contribution of a journal article should not depend on the statisti-
cal significance of tests or support for hypotheses. If the research
methods are valid and the questions worth asking, then the support
or lack thereof of hypotheses does not determine the contribution.
Nonresults, particularly unexpected null findings, are essential to full
disclosure. Articles should report null findings in addition to
Please cite this article as: Babin, B.J., et al., Heresies and sacred cows in scholarly marketing publications, Journal of Business Research (2015),
http://dx.doi.org/10.1016/j.jbusres.2015.12.001
5
statistically significant results. In this way, more surprising results
emerge from the literature and perhaps create a climate where mar-
keting (business) scientists actually discover something (Armstrong,
2003).
• More journal articles should report studies in an inductive presenta-
tion style. In other words, if one truly starts off with an exploratory
study, aimed at discovery, why not present the research report in
that fashion rather than presenting in a style more suitable for justifi-
cation? Interesting data come from interesting subject matter. The
paper can present methods, data and then implications for practice
and theoretical development following the results. This style allows
for discovery and may closely match the way much research takes
place. Post-hoc deduction in which hypotheses are cherry-picked or
derived after examining the data should not present an avenue for
publication over an inductive presentation. One primary challenge
for research is making sense out of the surprisingly nonsensical
(Woodside, 2012). Good discussion of nonresults from concepts solid-
ly linked conceptually provide an avenue for such sense-making.
Under these circumstances, theory development becomes more pos-
sible and more prominent.
• A detailed presentation of basic data characteristics needs to appear
with every journal article. In particular, the report should include fre-
quencies for less than interval measures and means and variances for
metric variables at the variable level (not just the construct level).
Readers may take more from these data than from statistically mas-
saged results. Reports also should fully disclose reasons for excluding
participants from analyses.
• When multiple studies address a hypothesis or a closely related set of
hypotheses, researchers should report “failed” studies, those not pro-
viding consistent results or not supporting hypotheses. The studies
that do not support the hypotheses indeed qualify any statistical sig-
nificance supporting hypotheses. One way to deal with this is to con-
sider more of a meta-analysis type reporting for multiple studies.
Further, if a study contains flaws to the extent that it should not be in-
cluded, the authors should explain how it came to be so flawed. The
bottom-line is that nonresults are equally important to results in un-
derstanding the real world.
• Researchers should give proportionate attention to potential problems
in the research. For example, there is no need for reporting multiple
and sophisticated tests on issues that do not show high potential for
distorting results substantially given the characteristics of a study
(such as CMV). These tests often go beyond the level of precision pro-
vided by the data and need not take up a significant amount of space in
a journal article. Such extended discussion can be a distraction to the
reader. When space is a concern, such tests may appear in appendices.
Indeed, the remedy for problems that might cause issues like CMB lies
best in steps taken before collecting data, not after (Conway & Lance,
2010). In surveys, researchers must monitor for potential problems
like order bias that could artificially impact relationships among
items within a scale or between scales. Indeed, multiple articles in
this special section point to the critical importance of construct validity.
All aspects of construct validity are important including fit validity,
convergent validity, and discriminant validity. But, even rigorous CFA
examinations, which remain critically important, do not mean very
much when the items lack face validity. A scale assessing smiles should
not drop the only item in the scale mentioning a smile. The item con-
tent from two items on separate constructs should not match closely.
Sometimes, when authors fall back on using previously published or
handbook indexed scales, problems such as these arise. A previously
applied scale does not guarantee face validity particularly in the con-
text of a different study. Thus, researchers' hesitation to modify a
scale's content due to context represents a potential sacred cow. A
common sense appraisal of item content remains as important a step
in measurement validity as exists.
• Researchers should give more attention to practical meaningfulness
rather than statistical significance. In fact, when researchers conduct
6 B.J. Babin et al. / Journal of Business Research xxx (2015) xxx–xxx
tests with no intention for generalization, they need not report statisti-
cal significance. Effect sizes deserve greater attention relative to statis-
tical significance. Reviewers need to understand that research is not
perfect and they should not succumb to the charm of significant re-
sults, sophisticated but not well-understood or cogent tools, an inaccu-
rate presentation of precision, or an exhaustive amount of studies.
• Authors should remain researchers when writing. They should remain
objective and resist passion about their theory or hypothesis that cre-
ates a rooting interest in the results. What if articles reported results
as published blind, not just reviewed blind? In this way, ego and repu-
tation may not interfere with the accurate presentation of research re-
sults. Perhaps the literature would be more enlightening.
• Any mention of research integrity should acknowledge the disturbing
issue of plagiarism. Actions intended solely to manipulate the review
process should be discouraged. Authors should be able to clearly
state the contribution each had to a published piece of research. Col-
leagues with little direct role in a research project should not pressure
other colleagues for authorship without due cause. Plagiarism, includ-
ing self-plagiarism, represents a lack of integrity and causes disrepute
for the entire academic research community.
• More research on academic researcher honesty is encouraged. A search
of academic dishonesty in the academic literature reveals a plethora of
studies dealing with student cheating and plagiarism. Of course, aca-
demics were students once. Do only honest students become aca-
demics? Research needs to address the impact of pressures to
publish and to gain research funding in the form of substantial grants
on the tendency, whether intentional or not, to steer results toward
TRUTH rather than truth.
References
Alba, J.W. (2012). In defense of bumbling. Journal of Consumer Research, 38, 981–987.
Armstrong, J.S. (2003). Discovery and communication of important marketing findings:
Evidence and proposals. Journal of Business Research, 56, 69–84.
Armstrong, J. S., & Overton, T. S. (1977). Estimating nonresponse bias in mail surveys.
Journal of Marketing Research, 14, 396–402.
Bhattacharjee, Y. (2013). The mind of a con man. New York Times MM44.
Calder, B.J., Phillips, L.W., & Tybout, A.M. (1981). Designing research for application.
Journal of Consumer Research, 8, 197–207.
Calder, B.J., Phillips, L.W., & Tybout, A.M. (1983). Beyond external validity. Journal of
Consumer Research, 10, 112–114.
Cattell, R.B. (1966). The meaning and strategic use of factor analysis. In R.B. Cattell (Ed.),
Handbook of environmental psychology (pp. 174–243). Chicago: Rand McNally.
Chandler, J., Mueller, P., & Paolacci, G. (2014). Nonnaivete among Amazon mechanical
Turk workers: Consequences and solutions for behavioral researchers. Behavioral
Research, 46, 112–130.
Chang, W., Franke, G.R., & Lee, N. (2015). Comparing reflective and formative measures:
New insights from relevant simulations. Journal of Business Research (forthcoming).
Please cite this article as: Babin, B.J., et al., Heresies and sacred cows in scholarly marketing publications, Journal of Business Research (2015),
http://dx.doi.org/10.1016/j.jbusres.2015.12.001
Conway, J.M., & Lance, C.E. (2010). What reviewers should expect from authors regarding
common method bias in organizational research. Journal of Business & Psychology, 25,
3250344.
Crosby, L.A., & Stephens, N. (1987). Effects of relationship marketing on satisfaction, re-
tention, and prices in the life insurance industry. Journal of Marketing Research, 404-
411.
Daugherty, T., Hoffman, E., & Kennedy, K. (2015). Research in reverse: Ad testing using an
inductive consumer neuroscience approach. Journal of Business Research forthcoming.
Diamantopoulos, A., & Siguaw, J.A. (2006). Formative versus reflective indicators in orga-
nizational measure development: a comparison and empirical illustration. British
Journal of Management, 17(4), 263–282.
Enserink, M. (2012). Final report on Stapel also blames field as a whole. Science,
338(6112), 1270–1271.
Espinosa, J.A., & Ortinau, D.J. (2015). Debunking legendary beliefs about student samples
in marketing research. Journal of Business Research (forthcoming).
Fang, F.C., Steen, R.G., & Casadevail, A. (2012). Misconduct accounts for the majority of
retracted scientific publications. PNAS, 109, 17028–17033.
Fuller, C. M., Simmering, M. J., Atinc, G., Atinc, Y., & Babin, B. J. (2015). Common methods
variance detection in business research. Journal of Business Research (forthcoming).
Hotelling, H. (1933). Analysis of a complex of statistical variables into principal compo-
nents. Journal of Educational Psychology, 24(6), 417.
Hubbard, R., & Armstrong, S. (1992). Are null results becoming an endangered species in
marketing? Marketing Letters, 3, 127–136.
Huber, J., Payne, J.W., & Puto, C.P. (2014). Let's be honest about the attraction effect.
Journal of Marketing Research, 51, 520–525.
Hunt, S.D. (2010). Marketing theory: foundations, controversy, strategy, resource-
advantage theory. Armonk. NY: M.E. Sharpe.
Johnston, R.J. (1973). Possible extensions to the factorial ecology method: A note.
Environment and Planning, 5, 719–734.
Karabag, S.F., & Berggren, C. (2012). Retraction, dishonesty and plagiarism: Analysis of a
crucial issue for academic publishing, and the inadequate responses from leading
journals in economics and management disciplines. Journal of Applied Economics
and Business Research, 2, 172–183.
Lynch, L.G. (1983). The role of external validity in theoretical research. Journal of
Consumer Research, 10, 109–111.
MacCallum, R.C., & Browne, M.W. (1993). The use of causal indicators in covariance struc-
ture models: Some practical issues. Psychological Bulletin, 114, 533–541.
Oliver, R., & Bearden, W.O. (1985). Disconfirmation processes and consumer evaluations
in product usage. Journal of Business Research, 13, 235–246.
Open Science Collaboration. (2015). Estimating the reproducibility of psychological sci-
ence. Science, 349(6251), 943–952.
Peter, J. P., & Olson, J. C. (1983). Is science marketing? Journal of Marketing, 47, 111–125.
Sajtos, L., & Magyar, B. (2015). Auxilliary theories as translational mechanisms for mea-
surement model specification. Journal of Business Research (forthcoming).
Sarstedt, M., Diamantopoulos, A., Salzberger, T., & Baumgartner, P. (2015). Selecting single
items to measure doubly-concrete constructs: A cautionary tale. Journal of Business
Research forthcoming.
Smith, S., Roster, C.A., Golden, L.L., & Albaum, G.S. (2015). Respondent data quality in
managed consumer panels and mTurk samples. Journal of Business Research
(forthcoming).
Stapel, D. (2012). Ontsporing. Amsterdam: Prometheus.
Wilcox, J.B., Howell, R.D., & Breivik, E. (2008). Questions about formative measurement.
Journal of Business Research, 61, 1219–1228.
Woodside, A. (2012). Incompetency training: Theory, practice and remedies. Journal of
Business Research, 65, 279–293.
Yang, S., & Lynn, M. (2014). More evidence challenging the robustness and usefulness of
the attraction effect. Journal of Marketing Research, 51, 508–513.