Journal of Applied Psychology © 2017 American Psychological Association

2017, Vol. 102, No. 3, 291–304 0021-9010/17/$12.00 http://dx.doi.org/10.1037/apl0000081

Solving the Supreme Problem: 100 Years of Selection and Recruitment at

the Journal of Applied Psychology

Robert E. Ployhart Neal Schmitt

University of South Carolina Michigan State University

Nancy T. Tippins
CEB, Greenville, South Carolina
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

This article reviews 100 years of research on recruitment and selection published in the Journal of
This document is copyrighted by the American Psychological Association or one of its allied publishers.

Applied Psychology. Recruitment and selection research has been present in the Journal from the very
first issue, where Hall (1917) suggested that the challenge of recruitment and selection was the Supreme
Problem facing the field of applied psychology. As this article shows, the various topics related to
recruitment and selection have ebbed and flowed over the years in response to business, legal, and
societal changes, but this Supreme Problem has captivated the attention of scientist–practitioners for a
century. Our review starts by identifying the practical challenges and macro forces that shaped the
sciences of recruitment and selection and helped to define the research questions the field has addressed.
We then describe the evolution of recruitment and selection research and the ways the resulting scientific
advancements have contributed to staffing practices. We conclude with speculations on how recruitment
and selection research may proceed in the future. Supplemental material posted online provides addi-
tional depth by including a summary of practice challenges and scientific advancements that affected the
direction of selection and recruitment research and an outline of seminal articles published in the Journal
and corresponding time line. The 100-year anniversary of the Journal of Applied Psychology is very
much the celebration of recruitment and selection research, although predictions about the future suggest
there is still much exciting work to be done.

Keywords: selection, recruitment, staffing

Supplemental materials: http://dx.doi.org/10.1037/apl0000081.supp

The history of the Journal of Applied Psychology (JAP) is very
much the history of selection and recruitment. In fact, the first
article ever published in JAP devoted most of its discussion to
selection (Hall, Baird, & Geissler, 1917). The authors were pre-
occupied with the importance of selection and classification based
on what they called “mental equipment” (i.e., individual differ-
ences): “The problem is so far-reaching that one finds it difficult to
determine whether the burden of its significance attaches to its
psychological, its economic, or its social aspects” (Hall et al.,
1917, p. 6). In the second article in the same issue, Hall (1917, p.
12) made broad reference to selection and called it the Supreme
Problem: “. . . the supreme problem of diagnosing each individual,
and steering him toward his fittest place, which is really the
culminating problem of efficiency, because human capacities are
This article was published Online First January 26, 2017.
Robert E. Ployhart, Department of Management, University of South
Carolina; Neal Schmitt, Department of Psychology, Michigan State Uni-
versity; Nancy T. Tippins, CEB, Greenville, South Carolina.
We thank Michael C. Campion for his assistance in preparing this
manuscript.
Correspondence concerning this article should be addressed to Robert E.
Ployhart. Department of Management, University of South Carolina, Co-
lumbia, SC 29208. E-mail: ployhart@moore.sc.edu
after all the chief national resources.” And remarkably, the third
article (Terman et al., 1917) focused entirely on the use of psy-
chological assessments for the selection of police and firefighters.
Applied psychology grew from a desire to make the world a better
place, and selection was the field in which the movement gained
traction.1
Selection was and remains the Supreme Problem of applied
psychologists. Selection has historically been operationalized as a
predictive hypothesis linking individual knowledge, skills, abili-
ties, and other characteristics (KSAOs) to individual outcomes,
such as job performance and turnover (Guion, 1965). The predic-
tive hypothesis is simple: Hire those with the highest scores on
measures of the KSAOs critical for effective performance on the
job. Establishing the evidence needed to operationalize, test, and
apply this hypothesis is not simple. For example, selection is
dependent on the existence of a pool of potential candidates
(Taylor & Russell, 1939). Hence, the field of recruitment devel-
oped to focus on identifying people who are likely to have the
necessary KSAOs and attracting them, so that they will apply to
firms and those who possess the prerequisite KSAOs can be
1
It is interesting to observe that these themes—psychological, eco-
nomic, and social—remain (to varying degrees) consistently intermingled
over the 100 years of selection and recruitment research.

291
 292 PLOYHART, SCHMITT, AND TIPPINS
selected. Recruitment is the “attraction hypothesis” that is the
precursor to the “predictive hypothesis.” Recruitment and selection
are thus interrelated, but recruitment is the feeder into selection
(Schneider, 1987).
The purpose of this article is to celebrate a century of research
on selection and recruitment that has been published in JAP. This
is a selective review because it privileges research published
within JAP. However, given JAP’s leadership role in the field of
applied psychology, a review of selection and recruitment research
published in JAP is essentially a review of thought leadership
within the community of selection and recruitment scholars. The
review does not attempt to discuss every article related to selection
and recruitment that was published in the Journal over the last 100
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
years. Instead, the goal of this article is to provide a broad sum-
This document is copyrighted by the American Psychological Association or one of its allied publishers.
mary of how selection and recruitment research has evolved within
JAP since the Journal’s inception in 1917. We identify major
practice issues that contributed to what the Journal published and
the ways the Journal’s publications helped shape practice. We
highlight examples of noteworthy contributions; articles that de-
fined the field; and various trends, ebbs, and flows of scholarship
for the last 100 years. The way we identified these exemplary
contributions was based on our collective judgment. We did not
calculate interrater reliability and agreement indices because we
did not code the articles according to any precise criteria.
In the sections that follow, we first summarize the practice
trends that have occurred over time and then use these trends to
understand the evolution of recruitment and selection research.
Having such a long view to the past helps one make reasonable
speculations about the future, so we conclude with a brief discus-
sion of the major issues (both applied and research issues) we
expect the field to confront and address in the next several decades.
Summary of Practice Challenges Shaping Selection
and Recruitment
Articles published in the first issue of JAP noted that applied
psychology was finally in a place to become a legitimate science
and was no longer inferior to what these authors called “pure
science” (Geissler, 1917; Hall, 1917; Hall et al., 1917). For exam-
ple, Hall (1917, p. 12) questioned: “Must we not, therefore, infer
that such facts as these suggest that we read just the old differen-
tiation between pure and applied psychology, and realize that
research in the latter field may be just as scientific as in any other,
and that the immediate utility of our results is at least no longer a
brand of scientific inferiority?” What distinguished this new sci-
entific field of applied psychology was recognition that pressing
practical challenges were not directly being addressed by the more
“pure” or what we today call “basic” areas of psychology. Science
could benefit practice, and practice could benefit science. Indeed,
throughout the last 100 years, what has made applied psychology
so dynamic and vibrant has been its close connection to business
and industry. However, this linkage may not always be obvious,
and the strength of the relationship can be a source of some
concern for those interested in maintaining a healthy scientist–
practitioner model of organizational psychology research and prac-
tice.
Changes in business, economics, the legal and political envi-
ronment, and societal systems have shaped the trajectory of selec-
tion and recruitment research. In terms of business challenges,
business and industry in the first two thirds of the twentieth
century created specialized and relatively distinct jobs (e.g., fac-
tory employees in different positions involving specific work
activities). The task for recruitment and selection in many organi-
zations was one of identifying the KSAOs needed in each job and
then hiring people with those KSAOs. However, as work became
increasingly knowledge-based and even more specialized, the
number of qualified applicants in many fields decreased, and
competition for those candidates with advanced skills and training
contributed to job mobility and created a greater need to under-
stand how to source and attract top candidates (i.e., recruitment).
In addition, many jobs now encompass a wide range of work
activities and require a much greater degree of interdependence
with coworkers. Thus, some firms now hire based on a broad set
of KSAOs required to perform all of the work activities for a
number of jobs in a job family, the ability to acquire new knowl-
edge and skills related to the job family, and personal character-
istics related to effective interactions with others.
The enormous changes in politics, societal attitudes, and the
laws related to employment have fundamentally redefined the
nature of selection and recruitment and also created important
boundaries between what are considered lawful and unlawful
practices. Passage of the 1964 Civil Rights Act in the United States
made employment discrimination based on protected group status
unlawful and gave the Equal Employment Opportunity Commis-
sion (EEOC) authority to monitor such activity.2 The simultaneous
rise of employment litigation (with corresponding use of adverse
impact statistics to operationalize discrimination) and recognition
of moderate to large racioethnic and sex-based subgroup differ-
ences on many predictors led researchers to strive for ways to
balance the goals of validity and diversity. At the same time,
affirmative action requirements encouraged employers to change
their recruitment practices and seek out members of protected
classes. Finally, short of the legal context, nothing has affected
selection and recruitment as much as war. World War I and World
War II created a need for selection and classification research on
an unprecedented scale.
Supplemental Appendix A (available online in supplemental
materials) provides a summary of practice challenges and scien-
tific developments that shaped the nature of selection and recruit-
ment research. Supplemental Appendix B offers a detailed timeline
of research publications. The articles listed in supplemental Ap-
pendix B resulted from a reading of all article titles and abstracts
and, in many cases, portions of the articles. Our attempt was to
identify changes in the nature of problems explored in JAP articles
or new approaches to the study of problems. The reason for
including an article is provided in the “Comment” column of the
appendix. The articles referenced in that table are not necessarily
the best or even the most influential articles on the topics ad-
dressed; nearly always, subsequent research on the topic went well
beyond the article cited in providing understanding of the issue(s).
One of the authors did the initial work on this appendix; the other
two authors reviewed the appendix, added some articles, and
2
In 1978, Executive Order 12086 gave the Office of Federal Contract
Compliance Programs (OFCCP) the authority to monitor compliance with
Executive Order 11246, as amended; Section 503 of the Rehabilitation Act
of 1973, as amended; and the Vietnam Era Veterans’ Readjustment As-
sistance Act of 1974, as amended; of federal contractors.
 SOLVING THE SUPREME PROBLEM
deleted others when appropriate. The end result represents our
collective judgment about what represents a substantive change in
the topic or emphasis placed on an area. Supplemental Appendix
C concludes the article with speculations about future research
needs in the areas of recruitment and selection. We now turn to
examine how recruitment and selection research published in JAP
has reflected these changes in business, economics, the legal and
political environment, and societal systems.
Recruitment Research in JAP
Recruitment refers to a broad set of activities that connect
applicants to organizations and their jobs. Applicants may be
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
active (i.e., currently looking for or applying to new jobs) or
This document is copyrighted by the American Psychological Association or one of its allied publishers.
passive (i.e., open to new employment opportunities but not ac-
tively looking), and internal (looking for new job opportunities
within one’s existing firm) or external (looking for new job op-
portunities at a different firm). Recruitment is a process that occurs
over time and comprises different stages, starting with the gener-
ating and sourcing of active or passive candidates who may
become future applicants, to maintaining active applicants who
have applied to the firm, and ending with influencing job choice
decisions (e.g., Barber, 1998). The field of recruitment is broad
and overlaps with several other literatures, including selection,
classification, job choice, vocational choice, promotion and suc-
cession planning, turnover, and person– environment (PE) fit.
However, at its core, recruitment is about finding and putting the
right person in the right job at the right time and place to enable
firms to implement strategy and create competitive advantage.
Therefore, this review will focus on research published in JAP
primarily from the perspective of organizational recruitment, and
job choice and attractiveness perceptions held by applicants.
Early Recruitment Research in JAP
The first article in JAP that referred to recruitment appeared in
the fourth and last issue of the Journal’s first year (Anonymous,
1917). The article did not have an author and was simply titled, “A
note on the German recruiting system.” This article had more to do
with the selection and training of recruits into the German military
than it did with recruitment, and it was likely published because of
concerns about World War I in Europe and the United States’
impending entry into the war. Beyond that one article, there was
little published in JAP for almost 50 years that had much to do
with recruitment. This most likely occurred because World Wars I
and II created an abundance of skilled workers, and society placed
a great deal of value on strong loyalty to a single employer; hence
the Supreme Problem was primarily concerned with selection
challenges (see supplemental Appendix A for more details). How-
ever, there were a few notable exceptions. Kerr (1943) examined
electrical employees’ preference for music while at work. This
article did not directly examine applicants’ desires, but it at least
considered the implications for recruitment. This article is quite
possibly the first to question whether the work environment could
influence job choice. Bendig and Stillman (1958) continued this
line of work but examined a broader range of job characteristics
(e.g., opportunity to learn new skills, job security) and importantly,
considered the desirability of these characteristics from the per-
spective of future applicants (college students). This article is
293
noteworthy for two reasons. First, the focus on job and work
characteristics that affect applicants’ perceptions of job attractive-
ness and their job choice foreshadows research that would not
reappear until the 1980s. Second, it studied college students, which
at the time was considered quite novel: “Our interest is in isolating
such dimensions of job incentives among groups of undergraduate
college students as an unexplored research area related to occupa-
tional choice among students. . . .” This comment is interesting
given that an overreliance on college students as subjects would
become a concern of later recruitment research (e.g., Breaugh,
2013). Another noteworthy article was authored by Weitz (1956),
who conducted a field experiment on applicants to estimate the
effects of job expectations on turnover for life insurance agents.
This article appears to be the first that examined applicants’
“realistic job concept,” a precursor to the realistic job preview
(RJP).
In the 1970s, business and societal changes led to more special-
ized skills and increased employee mobility. Because of the chal-
lenges in finding and attracting capable candidates, recruitment
research began to be published more frequently and gain program-
matic momentum. There were three main areas of focus. The first
concentrated on the recruitment interview (i.e., interviews de-
signed primarily to attract applicants to a firm) and the factors that
influenced either applicant or interviewer responses to the inter-
view (e.g., Alderfer & McCord, 1970; Cohen & Bunker, 1975;
Schmitt & Coyle, 1976; Simas & McCarrey, 1979). The second
evaluated the effectiveness of different recruiting methods and
strategies (e.g., Decker & Cornelius, 1979; Rynes & Miller, 1983;
Sands, 1973). The third explored RJPs and applicant-focused
perceptions (Reilly, Tenopyr, & Sperling, 1979; Rynes & Lawler,
1983; Wanous, 1973).
Recruitment research in the 1980s largely continued the re-
search of the 1970s: recruitment interviews (Borman, Eaton,
Bryan, & Rosse, 1983; Rynes & Miller, 1983); sourcing strategies
(Swaroff, Barclay, & Bass, 1985); and realistic job previews
(Premack & Wanous, 1985). However, research began to expand
these recruitment strategies in new ways. For example, researchers
began to study discrimination in recruiter evaluations (Graves &
Powell, 1988) and examine different approaches to recruitment,
such as internships for college students (Taylor, 1988). Perhaps the
most significant new area of research was the inclusion of utility
analysis into recruitment (Boudreau & Rynes, 1985). While it had
been recognized that effective recruitment strategies could en-
hance the financial benefits of staffing systems (primarily by
influencing the selection ratio; Sands, 1973), the article by Boud-
reau and Rynes (1985) was the first to assess the utility of recruit-
ment.
The 1990s saw the Internet become a radical disruptive tech-
nology, and a strong economy created intense competition for
highly skilled talent. Recruitment research published during this
time reflected these trends and became increasingly specialized
and sophisticated. For example, Martin and Raju (1992) expanded
utility analysis research to include the more specific consequences
of various cut scores on recruitment costs. Barber and Roehling
(1993) used verbal protocol analysis methods from cognitive psy-
chology to understand applicant decision processes. Barber, Hol-
lenbeck, Tower, and Phillips (1994) manipulated different inter-
view formats (e.g., recruitment, selection) and tracked the
consequences on applicant information acquisition over time.
 294 PLOYHART, SCHMITT, AND TIPPINS
Current Recruitment Research in JAP
The research described in the previous section set the tone for
current recruitment research at JAP. Although this research con-
tinues to explore the effects of different recruiting strategies on
applicants’ behavior pre- and posthire, the dramatic changes in
business and society are pushing recruitment research to evolve in
new ways.
Recruitment and diversity. Many firms have diversity hiring
goals as part of their mission, so it is not surprising that research
is focusing on how recruitment can enhance workforce diversity.
Much of this work has attempted to identify the recruiting tactics
and media that increase the attraction of diverse candidates (Avery,
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
2003; Walker, Feild, Bernerth, & Becton, 2012; Walker, Feild,
This document is copyrighted by the American Psychological Association or one of its allied publishers.
Giles, Armenakis, & Bernerth, 2009). Other research has linked
recruitment to diversity and selection. For example, Newman and
Lyon (2009) found that targeting a diverse and qualified workforce
will reduce adverse impact in subsequent hiring decisions. To-
gether, these studies show that the effects of diversity in recruit-
ment messages are moderate to small and conditional on the
medium, but even small effects may be useful for increasing the
diversity of the applicant pool and reducing adverse impact. How-
ever, recruiting large numbers of minority candidates who are not
qualified or less qualified than majority candidates will only ex-
acerbate adverse impact. Because a diverse workforce is a goal for
most organizations, the need for more research on this topic is
apparent.
Recruitment practices and technology. One consistent pro-
gram of recruitment research leading into the 1990s focused on the
effectiveness of different recruitment practices. This research ma-
tured to the point where it became possible to use meta-analysis to
summarize the primary studies and identify the relative impact of
different practices on recruitment outcomes. For example, Chap-
man, Uggerslev, Carroll, Piasentin, and Jones (2005) conducted a
large-scale meta-analysis on the determinants of applicants’ per-
ceptions of organizational attractiveness and job choice. They
found that fit, hiring expectations, characteristics of the job and
firm, and recruiter behavior were the key determinants. They then
extended these findings to test different theoretically derived path
models.
Just as the study of some recruiting practices matured, technol-
ogy radically changed traditional recruiting practices and made
several of them obsolete. First the Internet, and now social media
and other sources of “Big Data,” have revolutionized the nature of
recruitment. Nearly all firms use the Internet for recruitment (e.g.,
career pages), and many use some form of social media (Mulvey,
2013), although it is arguable whether social media enhances
recruiting effectiveness. Research on recruitment practices is
therefore now primarily focused on Internet recruiting. Most of
this research manipulates different Internet/website features to
examine the effects on such outcomes as perceptions of organiza-
tional attraction and application decisions (e.g., Allen, Mahto, &
Otondo, 2007; Dineen, Ash, & Rowe, 2002; Dineen & Noe, 2009;
Dineen, Ling, Ash, & DelVecchio, 2007). It is interesting to
observe that most of this research has been published nearly a
decade after Internet recruiting became popular in industry, sug-
gesting about a 10-year lag between practice and research.
Recruitment and organizational image. A relatively new
area of recruitment research focuses on how applicants view the
organization in terms of its employment brand, reputation, or
image.3 Organizational image is important not only because it
affects applicant attraction and job choice, but also because it may
be used to differentiate strategically the firm from its competitors.
For example, research suggests that recruitment practices (partic-
ularly word-of-mouth) influence applicants’ brand equity percep-
tions, while awareness of the organization’s products can influence
the recruitment process and consequences of recruitment practices
(e.g., Collins, 2007; Collins & Stevens, 2002). Other studies have
focused on the roles of different channels used to communicate
information about the job or company on perceptions of organi-
zational image and applicant outcomes (Allen et al., 2007; Cable &
Yu, 2006). Applicants’ organizational image perceptions appear to
be highly stable across judges (Highhouse, Broadfoot, Yugo, &
Devendorf, 2009) and can therefore have important consequences
on recruitment outcomes. More research is needed to understand
the connections between marketing, recruitment, and organiza-
tional image.
Theories of recruitment. One of the most profound changes
beginning in the 1980s and carrying through to current times is a
movement toward more theoretically rigorous recruitment re-
search. Research conducted prior to (approximately) the 1980s had
more of a “practice-driven” focus, while subsequent research has
shifted to more of a “theory-driven” focus. Although there is not
currently a widely accepted “theory of recruitment,” research in
this area has benefitted from blending theory from related litera-
tures. One theoretical approach adapts theories from cognitive
psychology and social cognition that explain information process-
ing, attitude formation, and persuasion (Breaugh, 2013). For ex-
ample, social psychological theories have been employed to un-
derstand internships (Zhao & Liden, 2011), recruiting visits
(Slaughter, Cable, & Turban, 2014), and recruiting sources (Van
Hoye & Lievens, 2009). An alternative theoretical program fo-
cuses on PE-fit and has applied it to the understanding of websites
(Dineen et al., 2002; Dineen & Noe, 2009), stages of recruitment
(Swider, Zimmerman, & Barrick, (2014), and national culture
(Turban, Lau, Ngo, Chow, & Si, 2001). Other theoretical ap-
proaches include brand equity theory (Collins & Stevens, 2002)
and media richness theory (Cable & Yu, 2006).
While these various approaches are helpful in understanding
specific aspects of recruiting processes, they may come at the cost
of capturing the breadth of recruiting practices. One must question
whether such “micro-theories” of specific components within the
entire recruiting process inhibit a search for a holistic recruiting
framework that is more broadly useful to practitioners (cf.,
Breaugh, 2013). The comprehensive framework presented by
Chapman et al. (2005) offers an important start toward developing
a general model applicable for practice.
Research and Practice Issues in Recruitment
The business, economic, and societal forces of modern times are
radically different from those in prior decades. Globalization,
Internet and mobile technologies, demographic shifts, economic
uncertainty, and the speed at which change occurs create a number
of practice challenges in recruitment. Organizations now must find
3
Although there are distinctions among these constructs, these distinc-
tions are not critical for present purposes so we consider them jointly.
 SOLVING THE SUPREME PROBLEM
effective recruiting strategies that reach a diverse group of people
who live in many parts of the world, have different career expec-
tations, and have different levels and kinds of access to electronic
technology. However, the focus of current research may not al-
ways be in alignment with the immediate needs of practitioners. As
noted, there is an increasing emphasis on theoretical understanding
rather than descriptions of recruitment practices (that was more
typical in JAP prior to the 1980s). As a consequence, this research
is less likely to have immediate practical applications even when it
has interesting theoretical implications or potential future guidance
for practice. The practice of recruitment is also changing rapidly
because of technology and economic shifts affecting the location
of work and the mobility of the applicant pools for many occupa-
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
tions. Implementation of attracting and sourcing strategies is often
This document is copyrighted by the American Psychological Association or one of its allied publishers.
proceeding more by trial and error than by systematic, evidence-
based efforts. Yet because of the rapid changes in recruitment
practices, research is often focused on out-of-date practices. For
example, the use of want-ads, employment agencies, and company
employment offices to facilitate recruiting have been largely dis-
continued and replaced with job boards, recruiting websites on
company webpages, and applicant tracking systems that manage
the application and selection processes. Instead of recruiting ef-
forts being determined by the results of research, they are increas-
ingly driven by the appeal of the newest technologies. With these
challenges as background, we identify some of the major practice
issues in recruitment that would benefit from greater practitioner-
academic research collaborations.
First, in many situations, different sources of applicants need to
be identified. Even though the Great Recession has been over for
several years, employment rates remain quite variable across jobs,
industry sectors, and countries. While labor shortages exist for
some occupations in some places, the true magnitude of such talent
shortages is often unclear. Whether shortages are because of a lack
of qualified candidates, unrealistically high hiring expectations by
employers, or a combination of the two is not apparent (Cappelli,
2012). What is clear is that firms must increasingly focus on
identifying— or sometimes even creating—pools of the necessary
talent earlier in the recruitment process or in locations different
from their typical recruiting sources. For example, global recruit-
ment is becoming more important because candidates with the
necessary qualifications are not evenly represented in different
countries (Ryan & Delany, 2010). Developed countries tend to
have a larger percentage of older workers in the workforce than
developing countries, but developing countries have fewer people
with advanced skills. As a consequence, employers are forced to
recruit in places where there are more plentiful applicants. Alter-
natively, organizations may develop an applicant pool when a
qualified pool of candidates to source and attract does not exist
(e.g., organizations investing in high school and college pro-
grams). Organizations, particularly those in developed countries,
may also consider nontraditional applicants or atypical work
schedules for hard-to-fill jobs. For example, applicants beyond the
“normal” age of retirement may offer well-honed skills on a
part-time basis. In addition, organizations may need to turn to
passive candidates (i.e., those who are not actively engaged in a
job search) to find those who meet the requirements of the firm’s
jobs. Along with this effort to identify the passive candidates,
firms may also need to emphasize their competitive advantages to
attract such applicants. In summary, future collaborative research
295
should focus not only on effective recruiting strategies but also on
efforts to identify where those strategies will be most productive,
and for whom.
Second, organizations often fail to keep data that allow them to
assess the adequacy of their sourcing and attraction methods,
rendering evidence-based decisions difficult if not impossible.
Because of technology, multiple sources of job information exist,
and they continually evolve, making it is difficult to isolate the
sources that influenced a candidate. Therefore, future research will
need to be based on complete and accurate data about recruiting
activities as well as consider more broadly the sources of infor-
mation available to candidates and determine which are meaning-
ful to them.
Third, much of the recruiting research focuses on the effec-
tiveness of a particular platform or website rather than an
examination of the characteristics that promote its usefulness
for applicants or detract from it. In the future, researchers will
need to move from studying specific technological platforms
(e.g., Facebook) toward frameworks that capture the key fea-
tures of the platforms (McFarland & Ployhart, 2015).
Fourth, technology and organizations’ need for speed and flex-
ibility in staffing have created a very dynamic recruitment process
for many firms that is not reflected in the extant research on the
components of recruiting and is not as linear as it was in the past.
Research in the future that adopts designs that do not assume a set,
linear recruitment process is likely to produce a more realistic
model and provide more actionable conclusions for practitioners.
Finally, as the academic field has become more specialized, the
concerns of the practitioner world have become more integrated.
For example, recruitment and turnover are inextricably connected
in practice but not in most research (for an exception see Weller,
Holtom, Matiaske, & Mellewigt, 2009). The answers to questions
of whom to recruit and when to recruit are highly affected by
turnover. Recruitment research needs to consider recruitment and
turnover jointly; they are different sides of the same talent equa-
tion.
Selection Research in JAP
Selection involves identifying the critical KSAOs needed for
effective performance on critical aspects of the job, creating ac-
curate ways of measuring individuals on these KSAOs, and then
using the scores on these assessments to make hiring decisions
(Schmitt & Chan, 1998). The ultimate goal is to enhance not only
job performance, but also organizational performance and com-
petitive advantage. This is the essence of the Supreme Problem
introduced by Hall (1917), the “predictive hypothesis” noted by
Guion (1965), and the “performance prediction problem” dis-
cussed by Campbell (1990). Job analysis is used to identify the
critical tasks of a job and the critical KSAOs needed to perform
those tasks (Schmitt & Chan, 1998). Once measures of the neces-
sary KSAOs are identified or developed, appropriate forms of
validity evidence should be accumulated. Yet beyond predictive
validity, a selection system should also try to minimize or elimi-
nate adverse impact, be seen as acceptable to applicants, be con-
sistent with the firm’s employment brand, and be efficient and
cost-effective in delivering reasonable return on investment. Se-
lection research published at JAP has emphasized these topics to
varying degrees over the last 100 years.
 296 PLOYHART, SCHMITT, AND TIPPINS
Early Selection Research in JAP
As noted previously and in supplemental Appendix A, the
nature of the Supreme Problem in the early years of JAP was
largely one of addressing selection challenges created by World
War I and the abundance of skilled labor that resulted from that
conflict. As a result, most of the articles in the first couple of
decades (1917 to about 1940) of JAP dealt with the validation of
ability tests for selection into different occupations in various
contexts. By today’s standards, these were simplistic studies based
on sample sizes almost always fewer than 100, using only bivariate
correlations or charts to display the relationship between test
scores and performance data, which were usually supervisor rat-
ings. However, these studies were attempting to address important
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
practical problems. Increased sophistication of the predictor and
This document is copyrighted by the American Psychological Association or one of its allied publishers.
criterion measures used in these studies, and expansion of the
topics related to selection and validation covered, began to appear
almost immediately. For example, Thorndike (1920) noted the
presence of a constant error in ratings used as criteria in selection
research, and Remmers (1934) identified halo error in students’
appraisals of their teachers. The potential of personality tests was
addressed by Brandenburg (1925), and the value of the interview
as a research and teaching device was discussed by Hartmann
(1933). Developments in data analyses were also published in
these early decades. Kelley (1934) described the impact of range
restriction on reliability, a problem still being addressed in our
literature (e.g., Hunter, Schmidt, & Le, 2006). Multiple correlation
and regression were described in a book by Edgerton (1931),
which was reviewed in JAP, but we did not locate any articles
using these methods of data analysis until several years later (e.g.,
Hartkemeier & Brown, 1936; Moore, 1941). However, authors of
these early studies using multiple regression did not seem to
recognize the problem of shrinkage of the multiple correlation
when it was estimated using a large number of predictors in a
relatively small sample and the importance of cross-validation.
Soon additional modes of testing were reported, including bio-
data (Scott, 1938), situational tests (Karn, 1949), group exercises
(Bass & Norton, 1951), leadership measures (Fleishmann, 1953),
assessment centers (Bray & Campbell, 1968), and personality
inventories (Ghiselli & Barthol, 1953). The latter article represents
what is probably the first meta-analysis of selection test validities,
though modern corrections for the magnitude of observed relation-
ships were not employed.
The criterion domain also expanded during this period of time to
include measures of turnover (e.g., Kitson, 1933; Kerr, 1947),
accidents (Brown & Ghiselli, 1948), and team performance (Wiest,
Porter, & Ghiselli, 1961), as well as new measurement tools, such
as behaviorally anchored rating scales (Smith & Kendall, 1963).
Concerns with the criterion, in general, were also expressed (Gaier,
1952; Dunnette, 1963a). Toward the end of this time period, the
use of the multitrait multimethod approach to the study of con-
struct validity was introduced to JAP readers (Locke, Smith,
Kendall, Hulin, & Miller, 1964). Wernimont and Campbell (1968)
espoused a greater attention, and fidelity, to the criterion in con-
structing and selecting predictor measures.
Current Selection Research in JAP
Current concerns about the use of tests. The utility of
selection procedures, which occupied a number of JAP pages in
the 1980s and 1990s, was introduced by Taylor and Russell
(1939), and concerns about mediators and moderators (e.g.,
Ghiselli, 1963; Dunnette, 1963b) resulted in a much more
complicated model of the relationships between personal char-
acteristics and performance outcomes than was prevalent in the
first several decades of the Journal. The complexities of indi-
vidual difference-outcome relationships remain an important
concern of JAP authors (e.g., Aguinis, Beaty, Boik, & Pierce,
2005; Le et al., 2011).
Because of the Civil Rights movement and related legislation
and court cases in the 1960s, researchers reported more com-
parisons of the test performance of members of different de-
mographic groups (e.g., Bartlett & O’Leary, 1969; Kirkpatrick,
Ewen, Bartlett, & Kattell, 1968). The Civil Rights Act of 1964
produced an explosion of research exploring appropriate meth-
ods of comparing data from subgroups of examinees and eval-
uating the attributions about the relationships between predic-
tors and criteria by group. The courts continue to interpret the
nature and meaning of employment discrimination in selection,
and thus this stream of research continues into the 21st century.
Boehm’s (1972) article represents an early attempt to aggregate
studies reporting little evidence of differences in validity coef-
ficients though there were large differences in group averages
on some measures. These findings were further supported by
Schmidt, Berner, and Hunter (1973), who argued that findings
of differences in validity for majority and minority groups were
a function of the differences in sample sizes and the fact that
previous researchers had incorrectly considered tests of the
significance of the validity in each group separately rather than
conducting tests of differences in the validities across groups.
The accepted conclusion of this research, and a broadened
differential prediction formulation that considered both inter-
cept and slope differences in regression, was that there was no
evidence for demographic slope differences but that differences
in intercepts usually indicated some overprediction of the per-
formance of lower scoring (on the predictor) minority groups.
Prior to reaching this “consensus,” there was an active debate
regarding the appropriate definition of prediction bias that was
summarized by Hunter, Schmidt, and Rauschenberger (1977). The
definition of test bias (Cleary, 1968) that was finally accepted
depended on a bias-free criterion, which also received attention in
the literature (Hattrup, Rock, & Scalia, 1997; Kraiger & Ford,
1985). This accepted wisdom (i.e., that minority group perfor-
mance is slightly overpredicted by selection tests) has been chal-
lenged recently by Aguinis, Culpepper, and Pierce (2010). Mattern
and Patterson (2013), using a large database of educational data,
showed that there was a slight overprediction of Black and His-
panic student performance and an underprediction of female per-
formance. Their analyses employed corrections of validity data for
statistical artifacts, the absence of which was one of the Aguinis et
al. criticisms of previous differential prediction research. Berry,
Clark, and McClure (2011) have presented data that indicate some
group validity differences in the employment setting that dispel the
notion that differential validity of tests does not exist. However,
Berry and Zhao (2015) corrected for a biased intercept test and
used operational validities (corrected for artifacts) and found over-
prediction of African American performance, using meta-analytic
 SOLVING THE SUPREME PROBLEM
estimates of key relationships. This issue is likely to continue to be
an active area of research.
Current conceptual and measurement developments in the
criterion. Perhaps because of the articles cited above on the nature
of the criterion, and because of dramatic changes in the nature and
structure of work (see supplemental Appendix A for more detail), the
last several decades of the Journal have been marked by an
increased attention on the nature of the work performance con-
struct and the development and use of outcome measures other
than overall task performance. Rosenbaum (1976) described ef-
forts to predict employee theft that seem like a precursor of more
recent attention to counterproductive work behavior (Bennett &
Robinson, 2000). Parsing the nature of turnover into functional and
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
dysfunctional aspects as well as its voluntary or involuntary nature
This document is copyrighted by the American Psychological Association or one of its allied publishers.
(Dalton, Krackhardt, & Porter, 1981) allowed advances in our
understanding relative to early concerns with labor turnover. Look-
ing at the positive aspects of work behavior, Smith, Organ, and
Near (1983) introduced the notion that organizational citizenship
behavior was an important worker outcome that likely was not
included in traditional ratings of overall or task performance.
Pulakos, Arad, Donovan, and Plamondon (2000) made the case
that employee adaptability to changing work demands was an
important aspect of employee performance. Sackett, Zedeck, and
Fogli (1988) provided an important distinction between typical and
maximal performance. Finally, a leading influence in these at-
tempts to consider multiple criterion dimensions was the Project A
work directed by Campbell, McHenry, and Wise (1990).
Current developments in the nature and measurement of
predictors. The expansion of the predictor constructs considered
as well as the methods used to measure them has continued to be
a topic in many of the selection articles appearing in the Journal.
A relatively early delineation of what might be aspects of both the
predictor and criterion domains was the comprehensive taxonomy
of worker behavior provided by Fleishman (1967). New methods
of measuring individual differences included the accomplishment
record (Hough, 1984) and the situational judgment test (Motow-
idlo, Dunnette, & Carter, 1990). Ryan and Sackett (1989) brought
research attention to the use of individual assessments (recently
meta-analyzed by Morris, Daisley, Wheeler, & Boyer, 2015), and
Van Iddekinge, Putka, and Campbell (2011) may have resurrected
interest in the potential use of vocational interest inventories as
selection devices. Arthur, Bell, Villado, and Doverspike (2006)
provided a meta-analysis of person-fit measures.
Of all the various selection devices, however, personality mea-
sures have probably received the most attention in the last 25
years, particularly after Digman (1990) introduced the Big Five
and Barrick and Mount (1991) published their meta-analysis of the
validity of measures of these constructs. Instead of examining
self-reports of Big Five measures, Oh, Wang, and Mount (2011)
examined the meta-analytic validity of observer ratings of person-
ality. They found that observer ratings had higher validity (range
from .18 to .32 corrected) than did self-ratings (range from .05 to
.22). Moreover, the observer ratings displayed incremental validity
over self-ratings though the reverse was not true. Lievens and
DeSoete (2011) describe a number of innovative selection tech-
niques that show promise as measures of personality constructs
including a variety of serious games that may help organizations
improve their image, attractiveness, and, in some instances, the
validity of their selection measures. The search and evaluation of
297
new sources of information on personality measures will and
should continue.
Concerns about faking personality measures have appeared fre-
quently (e.g., Hogan, Barrett, & Hogan, 2007; Schmitt & Oswald,
2006; Zickar & Robie, 1999), as well as concerns about their use
in other cultures (e.g., Ghorpade, Hattrup, & Lackwitz, 1999).
There have also been articles that examine potential curvilinear
relationships between personality measures and performance out-
comes (Le et al., 2011) and the role of mediators in personality–
performance relationships (e.g., Barrick, Stewart, & Piotrowski,
2002). In addition, Drasgow and his colleagues (e.g., Tay, Dras-
gow, Rounds, & Williams, 2009) have presented evidence that
ideal point models might be better representations of responses to
personality items than dominance models. The ideal point response
models posit that some items cannot be endorsed by people who
are both more and less extreme in their response to an item.
Test reactions. Beginning perhaps with the publication of an
article by Keenan and Kerr (1952) on the fairness of tests as seen
by employment counselors, researchers and users have been con-
cerned with the perceptions of selection tests by examinees and
others. Gilliland’s presentation of selection system fairness from
an organizational justice perspective (Gilliland, 1993) and his test
of some justice predictions (Gilliland, 1994) have generated many
dozens of subsequent research articles both in this country and
beyond (Steiner & Gilliland, 1996) where these concerns are often
referred to as social validity (Von Rosenstiel & Woschee, 2011).
Most of this research has related these reactions to selection
procedures to examinee perceptions of the organization and its
people rather than to more objective outcomes such as acceptance
of jobs or subsequent performance. An exception is the recent
article by McCarthy et al. (2013), who found that reactions indi-
rectly influenced job performance through the applicants’ test
scores. These reactions did not impact the criterion-related validity
of the tests.
Concerns about establishing appropriate evidence of
validity. A primary concern of selection researchers is validity,
the nature of the evidence one uses to support the attributions
about employee performance made based on test scores. Early
selection research relied almost solely on criterion-related validity,
the correlation between test scores and scores on performance
outcomes of interest. This basic design of a validity study has been
elaborated and the efficacy of other approaches has frequently
been explored by JAP authors. The meaning of validity has been
discussed in the editions of the Standards for Educational and
Psychological Testing, most recently in 2014. The various man-
ners in which studies of criterion-related validity have been con-
ducted were described and evaluated by Sussman and Robertson
(1986). Binning and Barrett (1989) provided a model integrating
ideas about various types of validity evidence that has been fre-
quently adapted and discussed. This model will certainly need to
be expanded and modified as selection researchers begin to con-
sider multilevel and cross-level relationships. Questions about
what constitutes construct or content validity have been raised in
JAP (e.g., Murphy, Dzieweczynski, & Zhang, 2009). The use of
latent variable models and corrections for unreliability of measure-
ment represent a long term concern that individual difference
measures give us insight into an underlying construct, although
this research seems to have hit its zenith between 1985 and1995.
 298 PLOYHART, SCHMITT, AND TIPPINS
Research and Practice Issues in Selection
Currently, there are tremendous changes taking place in busi-
ness (e.g., globalization), society (e.g., demographic shifts, migra-
tion, values, and expectations regarding work), and technology
(e.g., mobile devices) (see Appendix A for more details). These
changes occur rapidly and often affect operational testing pro-
grams. Yet, research in selection frequently lags behind these
significant shifts, and practitioners have needs that are often unmet
by the research literature. First, we continue to search for and need
tests or item types that minimize subgroup differences while
maintaining high validity and cost-effectiveness. Research on this
topic seems to have waned in recent years, perhaps because none
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
of the “easy” approaches have been effective and because the
This document is copyrighted by the American Psychological Association or one of its allied publishers.
potential causes of subgroup differences occur outside of employ-
ment contexts (e.g., Cottrell, Newman, & Roisman, 2015). Emerg-
ing research on innovative methods of evaluating individual capa-
bilities that take advantage of modern technology may identify
new measures that have widespread practical utility and serve to
maintain validity while reducing subgroup differences. Second,
effective strategies for reducing cheating and faking and improv-
ing the reliability of various measures, particularly personality
tests and other self-rating item formats, remain at the forefront of
practitioners’ concerns. Third, it is perhaps more important than
ever to engage applicants in the testing process by providing
instruments that appear relevant to the target job(s) and reinforce
the organization’s brand. There needs to be a recognition that
selection processes have functions other than evaluating the appli-
cant’s KSAOs, such as conveying information about the job and
the type of organization to which one is applying and attracting the
applicant to the organization. Although some of this research has
been conducted within the area of applicant reactions and attrac-
tion (e.g., Bauer, Maertz, Dolen, & Campion, 1998; Chapman et
al., 2005), we are calling for research on a broader set of selection-
related outcomes and answers to questions such as: “How might
selection practices shape perceptions of the organization’s culture
(as opposed to attractiveness)? Do firms that use more “innova-
tive” selection practices attract or recruit higher-quality candi-
dates?” Fourth, new techniques for overcoming common problems
related to validating test scores in operational settings are needed.
Employers often need to establish the validity of their selection
procedures but lack the sample size to utilize traditional validation
designs and find no alternative validation strategies that provide
sufficient evidence to justify the use of a test. They also recognize
the deficiencies of criteria that are based on supervisory ratings
and their effect on validity; however, for many jobs there are no
practical alternatives. Finally, despite the fact that many firms now
hire within a global context, research on global and international
selection issues remains largely absent.
One obvious illustration of the gap between selection research
and practice concerns the nature of what is published. It has
become rare that primary validation studies are published in JAP,
as was typical for the first two thirds of the Journal’s history (the
often-cited reason is that such studies do not make a significant
theoretical contribution). Primary validation studies are still pub-
lished, but they tend to be published in journals with a more
narrow focus specifically on selection. We now seem to rely on
large scale meta-analytic research that is dependent on these pri-
mary studies to establish the existence, magnitude, and generaliz-
ability of predictor-criterion relationships. However, these meta-
analyses do already and will increasingly include large numbers of
unpublished studies. How we determine the quality of these pri-
mary studies and when to include them in our meta-analytic work
remain troubling issues (see the series of articles on integrity tests
headed by Van Iddekinge, Roth, Raymark, & Odle-Dusseau,
2012). In addition, the “file drawer” problem persists, and the
representativeness of the studies included in a meta-analysis usu-
ally remains unknown though methods of assessing its likely
impact on meta-analytic estimates are available (Rothstein, Sutton,
& Borenstein, 2005).
The concern about meta-analytic source data also hints at the
need to consider the role of Big Data that is transforming research
in most areas of science. In our field, Big Data will likely produce
the same opportunities and problems as have been true in other
areas of scientific endeavor (e.g., Reichmann, Jones, & Schild-
bauer, 2011). One could foresee large multiorganizational, multi-
country data banks that could be used to assess the validity and
utility of selection tests in a variety of contexts. One can also
envision Big Data enabling the use of nontraditional predictors
(e.g., information from social media, credit scores) and the accom-
panying rise of practical questions regarding the job relevancy of
the predictors and personal privacy. Many organizations will seek
to use Big Data for selection decisions, regardless of whether such
processes meet the rigor expected by applied psychologists and
required by the U.S. legal system. Therefore, it is vital that selec-
tion researchers consider the potential benefits and risks of such
approaches in their research.
Unproctored Internet testing, in which an examinee takes a test
in some convenient location without supervision, has become
increasingly popular in the last decade or so and has now become
the only method by which some organizations assess large portions
of their applicant pool and the primary method of delivery of
exams by test publishers and consulting firms (Tippins et al.,
2006). This mode of test delivery is convenient for both employer
and applicant, standardizes test administration and scoring proce-
dures, offers substantial reductions in test administration costs, and
provides results quickly. However, it produces significant con-
cerns about cheating, test security, and standardization of the
testing environment. Perhaps the most commonly recommended
approach to alleviate some of the concerns about these applications
is to use unproctored test scores as a screen and verify the scores
with a proctored exam administered to a smaller set of examinees
who met some standard on the unproctored test (e.g., Lievens &
Burke, 2011; Weiner & Morrison, 2009). More sophisticated ap-
proaches use a computer adaptive test as an initial screen and
follow that with a verification test that is also a computer adaptive
test administered in a proctored setting. The score on the initial test
can be used as a prior “ability” estimate in beginning the second
proctored exam, thus shortening the testing process considerably
and allowing for a quick determination of the possibility that a
person cheated in some way on the unproctored exam.
Finally, selection research grew out of an interest in individual
differences so it is natural that our primary focus has been on the
nature of relationships among variables measured at the level of
the individual. But businesses succeed or fail based on their ability
to create performance advantages relative to their competitors.
Even though such business-unit outcomes were expressed as rel-
evant and important in the early days of JAP (e.g., Hall et al., 1917,
 SOLVING THE SUPREME PROBLEM
drew a connection between “human capacities” and “national
resources”), and there was at one time an active literature on utility
analysis; it is only within the past decade that researchers have
directly examined the impact of selection across multiple levels
(i.e., individual and organizational levels). Multilevel issues and
applications were introduced to the selection community by Klein
and Kozlowski (2000), with an important implication for selection:
Validity observed at the individual level may not generalize to the
firm level. Indeed, studies of small groups and teams have found
that relationships between individual KSAOs and individual per-
formance frequently differ from a group’s KSAO composition and
group performance (e.g., Barrick, Stewart, Neubert, & Mount,
1998; Bell, 2007; LePine, Hollenbeck, Ilgen, & Hedlund, 1997;
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Neuman & Wright, 1999). Such differences are most profound
This document is copyrighted by the American Psychological Association or one of its allied publishers.
when trying to staff teams with distributed expertise (e.g., cross-
functional teams; Mathieu, Tannenbaum, Donsbach, & Alliger,
2014).
Most of the early work on multilevel selection was theoretical
(e.g., Ployhart, 2004, 2012; Ployhart & Moliterno, 2011). Recent
empirical research finds both similarities and differences from
individual level findings. Ployhart, Van Iddekinge, and MacKenzie
(2011) provide evidence that generic human capital in the form of
personality and cognitive ability leads to changes in unit-specific
human capital such as training and experience which in turn leads
to unit service performance and business outcomes. Van Iddekinge
et al. (2009) showed that the implementation of selection and
training programs was related to customer service which in turn
was related to restaurant profits. Kim and Ployhart (2014) found
that firms that were more selective in their hiring outperformed
competitors before, and especially after, the Great Recession.
Whether or not validity based on group-level criteria is an accept-
able replacement in practice for validity that is based on
individual-level data is unknown, but demonstrating the business-
level consequences of individual differences, selection, and re-
cruitment is helpful for showing that they offer strategic value and
thus elevates their importance to organizational leaders.
Summary
One hundred years of research in the areas of selection and
recruitment have answered some questions, left some unanswered,
and raised many new questions. The “answers” that exist are rarely
definitive and apply to all conditions; rather, the context in which
the research was conducted matters a great deal, with a few notable
exceptions (e.g., the power of cognitive ability measures to predict
job performance across many different occupations and levels). It
would be impossible to list all of the conclusions to be drawn from
100 years of research; however, there are several important in-
sights that can be summarized from the literature.
First, both selection and recruitment research have tended to
address real-world problems. For most of the 20th and 21st cen-
turies, organizations have had an obvious need to find capable
employees to maintain their competitiveness. The gap between
practitioners’ concerns and researchers’ interests is narrower in
recruiting and selection than many other fields within industrial
and organizational psychology; however, a gap still exists, partic-
ularly in the area of recruiting where researchers strive to create
and substantiate meaningful theories that often have questionable
usefulness for practice or to delve into the efficacy of recruiting
299
practices that are out-of-date by the time the research is published.
The extent of the gap may be narrower with respect to selection
research, where much effort is focused on refining predictors and
criteria, developing methods to assess the strength of their rela-
tionships, and examining the differences in those relationships for
members of protected groups. Most of this research is applicable to
the work of the practitioner. For selection research, the gap that
does exist often manifests itself in what is not addressed in the
research. For example, we have not reached professional consen-
sus on how best to assemble evidence of validity when the sample
size is very small.
Second, two topics in the selection and recruitment literature
appear to be overriding concerns: individual capability and diver-
sity. Most of the selection and recruiting research in JAP answers
one of three questions: How do I accurately determine who has the
KSAOs to perform a particular job? Where do I find them? How
do I identify people of diverse backgrounds? To date, the impetus
for much of the research on capability and diversity has originated
with the legal and social environment in the United States; how-
ever, in more recent years, the global business environment has
highlighted the need to identify a capable workforce with the
prerequisite KSAOs regardless of their race, creed, color, ethnic-
ity, sex, and so forth, as a means of maintaining a competitive
advantage.
Third, both selection and recruitment can be seen from two
points of view: that of the individual candidate and that of the
organization. The early research in JAP focused on how the
company could find people who possessed the right skill set. Later
research in the form of applicant reactions has explored candi-
dates’ responses to different recruiting and selection strategies.
Although research may address problems from either point of
view, most researchers and especially practitioners today recog-
nize the importance of both.
Fourth, technology is playing an increasingly important role in
recruitment and selection as well as other business processes. A
great deal of research from the last 20 –30 years looks at ways to
harness the power of technology to improve recruiting and selec-
tion processes in terms of speed and flexibility, and to solve the
problems it creates (e.g., unproctored Internet testing and the
problem of cheating).
The Supreme Problem, Version 2.0
One century ago, Hall (1917) referred to the challenge of re-
cruitment and selection as the “Supreme Problem.” As we look
toward the next century of research on recruitment and selection,
it seems inconceivable that this Problem will be any less Supreme.
Human talent is increasingly the engine and differentiator behind
organizational competitive advantage, which is leading to greater
recognition that selection and recruitment are strategically valu-
able. The enormous changes in politics, societal attitudes, and the
legal system have fundamentally defined the nature of selection,
but also created important boundaries that firms still struggle with
today. So long as there are organizations and jobs, there will be a
need to find people who best fit them.
Yet, in that same first issue of JAP one century ago, Hall et al.
(1917, p. 6) also noted: “The problem is so far-reaching that one
finds it difficult to determine whether the burden of its significance
attaches to its psychological, its economic, or its social aspects.”
 300 PLOYHART, SCHMITT, AND TIPPINS
We envision the science and practice of recruitment and selection
to change in ways to meet the psychological, economic, and social
challenges of the future although the precise nature of the changes
is difficult to anticipate.
In terms of psychological challenges, the needs of applicants
will become more important as the availability of labor in
occupational areas that are important to the hiring organization
diminishes. The technological revolution will continue to affect
recruiting and selection methodologies in ways that engage appli-
cants and meet the needs of highly qualified candidates, even as
concepts such as validity and reliability remain central require-
ments. In addition, those who work in organizations, including
those who work in recruitment or selection, have their own needs
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
and expectations of the processes, especially as they relate to
This document is copyrighted by the American Psychological Association or one of its allied publishers.
technology. Although the speed of recruitment and selection pro-
cesses, the accuracy of the selection tools, and the effectiveness of
the recruiting process have been important considerations of prac-
titioners for some time, they are likely to receive increased em-
phasis in light of the demands of business in the 21st century and
the opportunities technology-enhanced tools can provide.
Economically, organizations will likely operate under
conditions that the U.S. military calls VUCA: volatile, uncertain,
complex, and ambiguous. To remain competitive in a VUCA
environment, firms will increasingly rely on their human capital
resources and their ability to reconfigure and redeploy those re-
sources for different strategic purposes. Using teams as the basic
organizing structure for work is one way to create such flexibility.
However, we still have much to learn about how to best recruit and
select for effective team functioning and performance. Pursuing
this work in a systematic fashion is difficult because the team task
and temporal processes have a strong influence on team KSAO
composition-team performance outcomes. Moving from the team
to the organizational level introduces even more contextual influ-
ences, such as economic and industry effects. Yet within such
contextual influences, recruitment and selection are central to the
successful deployment of human capital resources, and will per-
haps finally be recognized as the strategically vital capabilities that
they are. Drawing the theoretical and empirical connections be-
tween recruitment and selection to team and organizational per-
formance is just starting to occur, and our hope is that it will
continue.
Socially, changing attitudes, values, and employment regula-
tions, both within and across countries, will almost certainly en-
sure recruitment and selection practices will continue to evolve
over time. By extension, research must adapt to changing social
expectations and comply with the relevant laws and regulations
related to recruitment and selection. It is difficult to predict how
these laws and regulations may change as the demographic com-
position of many countries becomes increasingly diverse and the
political environments shift, but it is likely that changes in who is
protected and how they are protected will occur. If the future is
anything like the past, myriad social changes that affect recruit-
ment and selection will also occur. For example, job mobility may
well continue to increase as younger generations place higher
values on learning and leave an employer in the absence of
opportunities to learn. Alternatively, workers are discovering the
wide reach of the Internet and the unintended consequences of
postings. We might also hypothesize that the value workers place
on work life balance will continue to grow and substantially
change what aspects of a job are attractive to them.
The Supreme Problem has remained constant for 100 years, but
our definitions, frameworks, and methods for understanding the
problem have developed considerably over that time. Enormous
progress has been made, and personnel selection is one of the
successes of applied psychology. We suspect the Problem will
reign Supreme for the next 100 years; yet how we address it will
evolve in ways driven by changing psychological, economic, and
societal forces. Supplemental Appendix C provides our specula-
tion into the future of recruitment and selection research, organized
according to the psychological, economic, and social aspects pro-
posed by Hall et al. (1917). So long as academics and practitioners
continue to collaborate and the field maintains a scientist–
practitioner model, the methods for effective recruiting and selec-
tion will continue to develop. Readers will need to wait for the
2117 issue of JAP to determine the accuracy of these speculations.
References
Aguinis, H., Beaty, J. C., Boik, R. J., & Pierce, C. A. (2005). Effect size
and power in assessing moderating effects of categorical variables using
multiple regression: A 30-year review. Journal of Applied Psychology,
90, 94 –107. http://dx.doi.org/10.1037/0021-9010.90.1.94
Aguinis, H., Culpepper, S. A., & Pierce, C. A. (2010). Revival of test bias
research in preemployment testing. Journal of Applied Psychology, 95,
648 – 680. http://dx.doi.org/10.1037/a0018714
Alderfer, C. P., & McCord, C. G. (1970). Personal and situational factors
in the recruitment interview. Journal of Applied Psychology, 54, 377–
385. http://dx.doi.org/10.1037/h0029690
Allen, D. G., Mahto, R. V., & Otondo, R. F. (2007). Web-based recruit-
ment: Effects of information, organizational brand, and attitudes toward
a Web site on applicant attraction. Journal of Applied Psychology, 92,
1696 –1708. http://dx.doi.org/10.1037/0021-9010.92.6.1696
American Educational Research Association, American Psychological As-
sociation, & National Council on Measurement in Education. (2014).
Standards for educational and psychological testing. Washington, DC:
Author.
Anonymous. (1917). A note on the German recruiting system. Journal of
Applied Psychology, 1, 382–383. http://dx.doi.org/10.1037/h0074264
Arthur, W., Jr., Bell, S. T., Villado, A. J., & Doverspike, D. (2006). The
use of person-organization fit in employment decision making: An
assessment of its criterion-related validity. Journal of Applied Psychol-
ogy, 91, 786 – 801. http://dx.doi.org/10.1037/0021-9010.91.4.786
Avery, D. R. (2003). Reactions to diversity in recruitment advertising: Are
differences black and white? Journal of Applied Psychology, 88, 672–
679. http://dx.doi.org/10.1037/0021-9010.88.4.672
Barber, A. E. (1998). Recruiting employees: Individual and organizational
perspectives. Thousand Oaks, CA: Sage.
Barber, A. E., Hollenbeck, J. R., Tower, S. L., & Phillips, J. M. (1994). The
effects of interview focus on recruitment effectiveness: A field experi-
ment. Journal of Applied Psychology, 79, 886 – 896. http://dx.doi.org/10
.1037/0021-9010.79.6.886
Barber, A. E., & Roehling, M. V. (1993). Job postings and the decision to
interview: A verbal protocol analysis. Journal of Applied Psychology,
78, 845– 856. http://dx.doi.org/10.1037/0021-9010.78.5.845
Barrick, M. R., & Mount, M. K. (1991). The Big Five personality dimen-
sions in job performance: A meta-analysis. Personnel Psychology, 44,
1–26. http://dx.doi.org/10.1111/j.1744-6570.1991.tb00688.x
Barrick, M. R., Stewart, G. L., Neubert, M. J., & Mount, M. K. (1998).
Relating member ability and personality to work-team processes and
team effectiveness. Journal of Applied Psychology, 83, 377–391. http://
dx.doi.org/10.1037/0021-9010.83.3.377
 SOLVING THE SUPREME PROBLEM
Barrick, M. R., Stewart, G. L., & Piotrowski, M. (2002). Personality and
job performance: Test of the mediating effects of motivation among
sales representatives. Journal of Applied Psychology, 87, 43–51. http://
dx.doi.org/10.1037/0021-9010.87.1.43
Bartlett, C. J., & O’Leary, B. S. (1969). A differential prediction model to
moderate the effects of heterogeneous groups in personnel selection and
classification. Personnel Psychology, 22, 1–17. http://dx.doi.org/10
.1111/j.1744-6570.1969.tb02051.x
Bass, B. M., & Norton, F. M. (1951). Group size and leaderless discus-
sions. Journal of Applied Psychology, 35, 397– 400. http://dx.doi.org/10
.1037/h0055478
Bauer, T. N., Maertz, C. P., Dolen, M. R., & Campion, M. A. (1998). A
longitudinal assessment of applicant reactions to an employment test.
Journal of Applied Psychology, 83, 892–903. http://dx.doi.org/10.1037/
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
0021-9010.83.6.892
This document is copyrighted by the American Psychological Association or one of its allied publishers.
Bell, S. T. (2007). Deep-level composition variables as predictors of team
performance: A meta-analysis. Journal of Applied Psychology, 92, 595–
615. http://dx.doi.org/10.1037/0021-9010.92.3.595
Bendig, A. W., & Stillman, E. L. (1958). Dimensions of job incentives
among college students. Journal of Applied Psychology, 42, 367–371.
http://dx.doi.org/10.1037/h0041127
Bennett, R. J., & Robinson, S. L. (2000). Development of a measure of
workplace deviance. Journal of Applied Psychology, 85, 349 –360.
http://dx.doi.org/10.1037/0021-9010.85.3.349
Berry, C. M., Clark, M. A., & McClure, T. K. (2011). Racial/ethnic
differences in the criterion-related validity of cognitive ability tests: A
qualitative and quantitative review. Journal of Applied Psychology, 96,
881–906. http://dx.doi.org/10.1037/a0023222
Berry, C. M., & Zhao, P. (2015). Addressing criticisms of existing predic-
tive bias research: Cognitive ability test scores still overpredict African
Americans’ job performance. Journal of Applied Psychology, 100, 162–
179. http://dx.doi.org/10.1037/a0037615
Binning, J. F., & Barrett, G. V. (1989). Validity of personnel decisions: A
conceptual analysis of the inferential and evidential bases. Journal of
Applied Psychology, 74, 478 – 494. http://dx.doi.org/10.1037/0021-9010
.74.3.478
Boehm, V. R. (1972). Negro-white differences in validity of employment
and training selection procedures: Summary of research evidence. Jour-
nal of Applied Psychology, 56, 33–39. http://dx.doi.org/10.1037/
h0032130
Borman, W. C., Eaton, N. K., Bryan, J. D., & Rosse, R. L. (1983). Validity
of army recruiter behavioral assessment: Does the assessor make a
difference? Journal of Applied Psychology, 68, 415– 419. http://dx.doi
.org/10.1037/0021-9010.68.3.415
Boudreau, J. W., & Rynes, S. L. (1985). Role of recruitment in staffing
utility analysis. Journal of Applied Psychology, 70, 354 –366. http://dx
.doi.org/10.1037/0021-9010.70.2.354
Brandenburg, G. C. (1925). Personality and vocational achievement. Part
II. Journal of Applied Psychology, 9, 281–292. http://dx.doi.org/10
.1037/h0070621
Bray, D. W., & Campbell, R. J. (1968). Selection of salesmen by means of
an assessment center. Journal of Applied Psychology, 52, 36 – 41. http://
dx.doi.org/10.1037/h0025353
Breaugh, J. A. (2013). Employee recruitment. Annual Review of Psychol-
ogy, 64, 389 – 416. http://dx.doi.org/10.1146/annurev-psych-113011-
143757
Brown, C. W., & Ghiselli, E. E. (1948). Accident proneness among street
car motormen and motor coach operators. Journal of Applied Psychol-
ogy, 32, 20 –23. http://dx.doi.org/10.1037/h0055798
Cable, D. M., & Yu, K. Y. T. (2006). Managing job seekers’ organizational
image beliefs: The role of media richness and media credibility. Journal
of Applied Psychology, 91, 828 – 840. http://dx.doi.org/10.1037/0021-
9010.91.4.828
Campbell, J. P. (1990). Modeling the performance prediction problem in
301
industrial and organizational psychology. In M. D. Dunnette & L. M.
Hough (Eds.), Handbook of industrial and organizational psychology
(2nd ed., Vol. 1, pp. 687–732). Palo Alto, CA: Consulting Psychologists
Press.
Campbell, J. P., McHenry, J. J., & Wise, L. L. (1990). Modeling job
performance in a population of jobs. Personnel Psychology, 43, 313–
333. http://dx.doi.org/10.1111/j.1744-6570.1990.tb01561.x
Cappelli, P. (2012). Why good people can’t get jobs: The skills gap and
what companies can do about it. Philadelphia, PA: Wharton Digital
Press.
Chapman, D. S., Uggerslev, K. L., Carroll, S. A., Piasentin, K. A., & Jones,
D. A. (2005). Applicant attraction to organizations and job choice: A
meta-analytic review of the correlates of recruiting outcomes. Journal of
Applied Psychology, 90, 928 –944. http://dx.doi.org/10.1037/0021-9010
.90.5.928
Cleary, T. A. (1968). Test bias: Prediction of grades of Negro and white
students in integrated colleges. Journal of Educational Measurement, 5,
115–124. http://dx.doi.org/10.1111/j.1745-3984.1968.tb00613.x
Cohen, S. L., & Bunker, K. A. (1975). Subtle effects of sex role stereotypes
on recruiters’ hiring decisions. Journal of Applied Psychology, 60,
566 –572. http://dx.doi.org/10.1037/0021-9010.60.5.566
Collins, C. J. (2007). The interactive effects of recruitment practices and
product awareness on job seekers’ employer knowledge and application
behaviors. Journal of Applied Psychology, 92, 180 –190. http://dx.doi
.org/10.1037/0021-9010.92.1.180
Collins, C. J., & Stevens, C. K. (2002). The relationship between early
recruitment-related activities and the application decisions of new labor-
market entrants: A brand equity approach to recruitment. Journal of
Applied Psychology, 87, 1121–1133. http://dx.doi.org/10.1037/0021-
9010.87.6.1121
Cottrell, J. M., Newman, D. A., & Roisman, G. I. (2015). Explaining the
black-white gap in cognitive test scores: Toward a theory of adverse
impact. Journal of Applied Psychology, 100, 1713–1736. http://dx.doi
.org/10.1037/apl0000020
Dalton, D. R., Krackhardt, D. M., & Porter, L. W. (1981). Functional
turnover: An empirical assessment. Journal of Applied Psychology, 66,
716 –721. http://dx.doi.org/10.1037/0021-9010.66.6.716
Decker, P. J., & Cornelius, E. T. (1979). A note on recruiting sources and
job survival rates. Journal of Applied Psychology, 64, 463– 464. http://
dx.doi.org/10.1037/0021-9010.64.4.463
Digman, J. M. (1990). Personality structure: Emergence of the five factor
model. Annual Review of Psychology, 41, 417– 440. http://dx.doi.org/10
.1146/annurev.ps.41.020190.002221
Dineen, B. R., Ash, S. R., & Noe, R. A. (2002). A Web of applicant
attraction: Person-organization fit in the context of Web-based recruit-
ment. Journal of Applied Psychology, 87, 723–734. http://dx.doi.org/10
.1037/0021-9010.87.4.723
Dineen, B. R., Ling, J., Ash, S. R., & DelVecchio, D. (2007). Aesthetic
properties and message customization: Navigating the dark side of web
recruitment. Journal of Applied Psychology, 92, 356 –372. http://dx.doi
.org/10.1037/0021-9010.92.2.356
Dineen, B. R., & Noe, R. A. (2009). Effects of customization on applica-
tion decisions and applicant pool characteristics in a web-based recruit-
ment context. Journal of Applied Psychology, 94, 224 –234. http://dx.doi
.org/10.1037/a0012832
Dunnette, M. D. (1963a). A note on the criterion. Journal of Applied
Psychology, 47, 251–254. http://dx.doi.org/10.1037/h0040836
Dunnette, M. D. (1963b). A modified model for test validation and selec-
tion research. Journal of Applied Psychology, 47, 317–323. http://dx.doi
.org/10.1037/h0047635
Edgerton, H. A. (1931). Review of methods of correlation analysis. Journal
of Applied Psychology, 15, 103–104. http://dx.doi.org/10.1037/
h0064178
 302 PLOYHART, SCHMITT, AND TIPPINS
Fleishman, E. A. (1967). Development of a behavior taxonomy for de-
scribing human tasks: A correlational-experimental approach. Journal of
Applied Psychology, 51, 1–10. http://dx.doi.org/10.1037/h0024073
Fleishmann, E. A. (1953). The description of supervisory behavior. Journal
of Applied Psychology, 37, 1– 6. http://dx.doi.org/10.1037/h0056314
Gaier, E. L. (1952). The criterion problem in the prediction of medical
school success. Journal of Applied Psychology, 36, 316 –322. http://dx
.doi.org/10.1037/h0063585
Geissler, L. R. (1917). What is applied psychology? Journal of Applied
Psychology, 1, 46 – 60. http://dx.doi.org/10.1037/h0071582
Ghiselli, E. E. (1963). Moderating effects and differential reliability and
validity. Journal of Applied Psychology, 47, 81– 86. http://dx.doi.org/10
.1037/h0047177
Ghiselli, E. E., & Barthol, R. P. (1953). The validity of personality
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
inventories in the selection of employees. Journal of Applied Psychol-
This document is copyrighted by the American Psychological Association or one of its allied publishers.
ogy, 37, 18 –20. http://dx.doi.org/10.1037/h0059438
Ghorpade, J., Hattrup, K., & Lackwitz, J. R. (1999). The use of personality
measures in cross-cultural research: A test of three personality scales.
Journal of Applied Psychology, 84, 670 – 679. http://dx.doi.org/10.1037/
0021-9010.84.5.670
Gilliland, S. W. (1993). The perceived fairness of selection systems: An
organizational justice perspective. The Academy of Management Review,
18, 694 –734.
Gilliland, S. W. (1994). Effects of procedural and distributive justice on
reactions to a selection system. Journal of Applied Psychology, 79,
691–701. http://dx.doi.org/10.1037/0021-9010.79.5.691
Graves, L. M., & Powell, G. N. (1988). An investigation of sex discrim-
ination in recruiters’ evaluations of actual applicants. Journal of Applied
Psychology, 73, 20 –29. http://dx.doi.org/10.1037/0021-9010.73.1.20
Guion, R. M. (1965). Personnel testing. New York, NY: McGraw-Hill.
Hall, G. H. (1917). Practical relations between psychology and the war.
Journal of Applied Psychology, 1, 9 –16. http://dx.doi.org/10.1037/
h0070238
Hall, G. S., Baird, J. W., & Geissler, L. R. (1917). Foreword. Journal of
Applied Psychology, 1, 5–7. http://dx.doi.org/10.1037/h0065981
Hartkemeier, H. P., & Brown, L. M. (1936). Multiple correlation and the
multiple factor method. Journal of Applied Psychology, 20, 396 – 415.
http://dx.doi.org/10.1037/h0062475
Hartmann, G. W. (1933). The interview as a research and teaching device.
Journal of Applied Psychology, 17, 205–211. http://dx.doi.org/10.1037/
h0069851
Hattrup, K., Rock, J., & Scalia, C. (1997). The effects of varying concep-
tualizations of job performance on adverse impact, minority hiring, and
predicted performance. Journal of Applied Psychology, 82, 656 – 664.
http://dx.doi.org/10.1037/0021-9010.82.5.656
Highhouse, S., Broadfoot, A., Yugo, J. E., & Devendorf, S. A. (2009).
Examining corporate reputation judgments with generalizability theory.
Journal of Applied Psychology, 94, 782–789. http://dx.doi.org/10.1037/
a0013934
Hogan, J., Barrett, P., & Hogan, R. (2007). Personality measurement,
faking, and employment selection. Journal of Applied Psychology, 92,
1270 –1285. http://dx.doi.org/10.1037/0021-9010.92.5.1270
Hough, L. M. (1984). Development and evaluation of the “accomplishment
record” method of selecting and promoting professionals. Journal of
Applied Psychology, 69, 135–146. http://dx.doi.org/10.1037/0021-9010
.69.1.135
Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct and
indirect range restriction for meta-analysis methods and findings. Jour-
nal of Applied Psychology, 91, 594 – 612. http://dx.doi.org/10.1037/
0021-9010.91.3.594
Hunter, J. E., Schmidt, F. L., & Rauschenberger, J. M. (1977). Fairness of
psychological tests: Implications of four definitions for selection utility
and minority hiring. Journal of Applied Psychology, 62, 245–260. http://
dx.doi.org/10.1037/0021-9010.62.3.245
Karn, H. W. (1949). Performance on the File-Remmers test, how super-
vise? Before and after a course in psychology. Journal of Applied
Psychology, 33, 534 –539. http://dx.doi.org/10.1037/h0056171
Keenan, V., & Kerr, W. A. (1952). Unfair employment practices as viewed
by private employment counselors. Journal of Applied Psychology, 36,
361–364. http://dx.doi.org/10.1037/h0061201
Kelley, T. L. (1934). Note upon Dickey’s formula for the effect of range
upon the reliability coefficient. Journal of Applied Psychology, 18,
855– 859. http://dx.doi.org/10.1037/h0075793
Kerr, W. A. (1943). Where they like to work: Work place preference of 228
electrical workers in terms of music. Journal of Applied Psychology, 27,
438 – 442. http://dx.doi.org/10.1037/h0061828
Kerr, W. A. (1947). Labor turnover and its correlates. Journal of Applied
Psychology, 31, 366 –371. http://dx.doi.org/10.1037/h0062050
Kim, Y., & Ployhart, R. E. (2014). The effects of staffing and training on
firm productivity and profit growth before, during, and after the Great
Recession. Journal of Applied Psychology, 99, 361–389. http://dx.doi
.org/10.1037/a0035408
Kirkpatrick, J. J., Ewen, R. B., Bartlett, C. J., & Kattell, R. A. (1968).
Testing and fair employment. New York, NY: New York University.
Kitson, H. D. (1933). A further study of the turnover of psychologists.
Journal of Applied Psychology, 17, 542–549. http://dx.doi.org/10.1037/
h0074213
Klein, K. J., & Kozlowski, S. W. J. (2000). (Eds.) Multilevel theory,
research, and methods in organizations: Foundations, extensions, and
new directions. San Francisco, CA: Jossey-Bass.
Kraiger, K., & Ford, J. K. (1985). A meta-analysis of ratee race effects in
performance ratings. Journal of Applied Psychology, 70, 56 – 65. http://
dx.doi.org/10.1037/0021-9010.70.1.56
Le, H., Oh, I. S., Robbins, S. B., Ilies, R., Holland, E., & Westrick, P.
(2011). Too much of a good thing: Curvilinear relationships between
personality traits and job performance. Journal of Applied Psychology,
96, 113–133. http://dx.doi.org/10.1037/a0021016
LePine, J. A., Hollenbeck, J. R., Ilgen, D. R., & Hedlund, J. (1997). Effects
of individual differences on the performance of hierarchical decision-
making teams: Much more than g. Journal of Applied Psychology, 82,
803– 811. http://dx.doi.org/10.1037/0021-9010.82.5.803
Lievens, F., & Burke, E. (2011). Dealing with the threats inherent in
unproctored Internet testing of cognitive ability: Results from a large-
scale operational test program. Journal of Occupational and Organiza-
tional Psychology, 84, 817– 824. http://dx.doi.org/10.1348/
096317910X522672
Lievens, F., & DeSoete, B. (2011). Instruments for personnel selection in
the 21st century: Research and practice. Gedrag en Organisatie, 24,
18 – 42.
Locke, E. A., Smith, P. C., Kendall, L. M., Hulin, C. L., & Miller, A. M.
(1964). Convergent and discriminant validity for areas and methods of
rating job satisfaction. Journal of Applied Psychology, 48, 313–319.
http://dx.doi.org/10.1037/h0043202
Martin, S. L., & Raju, N. S. (1992). Determining cutoff scores that
optimize utility: A recognition of recruiting costs. Journal of Applied
Psychology, 77, 15–23. http://dx.doi.org/10.1037/0021-9010.77.1.15
Mathieu, J. E., Tannenbaum, S. I., Donsbach, J. S., & Alliger, G. M.
(2014). A review and integration of team composition models: Moving
toward a dynamic and temporal framework. Journal of Management, 40,
130 –160. http://dx.doi.org/10.1177/0149206313503014
Mattern, K. D., & Patterson, B. F. (2013). Test of slope and intercept bias
in college admissions: A response to Aguinis, Culpepper, and Pierce
(2010). Journal of Applied Psychology, 98, 134 –147. http://dx.doi.org/
10.1037/a0030610
McCarthy, J. M., Van Iddekinge, C. H., Lievens, F., Kung, M. C., Sinar,
E. F., & Campion, M. A. (2013). Do candidate reactions relate to job
performance or affect criterion-related validity? A multistudy investiga-
tion of relations among reactions, selection test scores, and job perfor-
 SOLVING THE SUPREME PROBLEM
mance. Journal of Applied Psychology, 98, 701–719. http://dx.doi.org/
10.1037/a0034089
McFarland, L. A., & Ployhart, R. E. (2015). Social media: A contextual
framework to guide research and practice. Journal of Applied Psychol-
ogy, 100, 1653–1677. http://dx.doi.org/10.1037/a0039244
Moore, B. V. (1941). Analysis of results of tests administered to men in
engineering defense training courses. Journal of Applied Psychology, 25,
619 – 635. http://dx.doi.org/10.1037/h0060853
Morris, S. B., Daisley, R. L., Wheeler, M., & Boyer, P. (2015). A meta-
analysis of the relationship between individual assessments and job
performance. Journal of Applied Psychology, 100, 5–20. http://dx.doi
.org/10.1037/a0036938
Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative
selection procedure: The low-fidelity simulation. Journal of Applied
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
Psychology, 75, 640 – 647. http://dx.doi.org/10.1037/0021-9010.75.6
This document is copyrighted by the American Psychological Association or one of its allied publishers.
.640
Mulvey, T. (2013). SHRM survey findings: Social networking websites and
recruiting/selection. Retrieved from http://www.shrm.org/research/
surveyfindings/articles/pages/shrm-social-networking-websites-
recruiting-job-candidates.aspx
Murphy, K. R., Dzieweczynski, J. L., & Zhang, Y. (2009). Positive
manifold limits the relevance of content-matching strategies for validat-
ing selection test batteries. Journal of Applied Psychology, 94, 1018 –
1031. http://dx.doi.org/10.1037/a0014075
Neuman, G. A., & Wright, J. (1999). Team effectiveness: Beyond skills
and cognitive ability. Journal of Applied Psychology, 84, 376 –389.
http://dx.doi.org/10.1037/0021-9010.84.3.376
Newman, D. A., & Lyon, J. S. (2009). Recruitment efforts to reduce
adverse impact: Targeted recruiting for personality, cognitive ability,
and diversity. Journal of Applied Psychology, 94, 298 –317. http://dx.doi
.org/10.1037/a0013472
Oh, I. S., Wang, G., & Mount, M. K. (2011). Validity of observer ratings
of the five-factor model of personality traits: A meta-analysis. Journal of
Applied Psychology, 96, 762–773. http://dx.doi.org/10.1037/a0021832
Ployhart, R. E. (2004). Organizational staffing: A multilevel review, syn-
thesis, and model. Research in Personnel and Human Resources Man-
agement, 23, 121–176. http://dx.doi.org/10.1016/S0742-7301(04)
23003-1
Ployhart, R. E. (2012). Multilevel selection and the paradox of sustained
competitive advantage. In N. Schmitt (Ed.), The Oxford handbook of
assessment and personnel selection (pp. 667– 685). New York, NY:
Oxford University Press. http://dx.doi.org/10.1093/oxfordhb/
9780199732579.013.0029
Ployhart, R. E., & Moliterno, T. P. (2011). Emergence of the human capital
resource: A multilevel model. The Academy of Management Review, 3,
127–150. http://dx.doi.org/10.5465/amr.2009.0318
Ployhart, R. E., Van Iddekinge, C. H., & MacKenzie, W. I., Jr. (2011).
Acquiring and developing human capital in service contexts: The inter-
connectedness of human capital resources. Academy of Management
Journal, 54, 353–368. http://dx.doi.org/10.5465/AMJ.2011.60263097
Premack, S. L., & Wanous, J. P. (1985). A meta-analysis of realistic job
preview experiments. Journal of Applied Psychology, 70, 706 –719.
http://dx.doi.org/10.1037/0021-9010.70.4.706
Pulakos, E. D., Arad, S., Donovan, M. A., & Plamondon, K. E. (2000).
Adaptability in the workplace: Development of a taxonomy of adaptive
performance. Journal of Applied Psychology, 85, 612– 624. http://dx.doi
.org/10.1037/0021-9010.85.4.612
Reichmann, O. J., Jones, M. B., & Schildbauer, M. P. (2011). Challenges
and opportunities of open data in ecology. Science, 331, 03–705.
Reilly, R. R., Tenopry, M. L., & Sperling, S. M. (1979). Effects of job
previews on job acceptance and survival of telephone operator candi-
dates. Journal of Applied Psychology, 64, 218 –220. http://dx.doi.org/10
.1037/0021-9010.64.2.218
303
Remmers, H. H. (1934). Reliability and halo effect of high school and
college students’ judgments of their teachers. Journal of Applied Psy-
chology, 18, 619 – 630. http://dx.doi.org/10.1037/h0074783
Rosenbaum, R. W. (1976). Predictability of employee theft using weighted
application blanks. Journal of Applied Psychology, 61, 94 –98. http://dx
.doi.org/10.1037/0021-9010.61.1.94
Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2005). Publica-
tion bias in meta-analysis in meta-analysis: Prevention, assessment, and
adjustment. Chichester, United Kingdom: Wiley. http://dx.doi.org/10
.1002/0470870168
Ryan, A. M., & Delany, T. (2010). Attracting people to organizations. In
J. L. Farr & N. Tippins (Eds.), Handbook of employee selection (pp.
127–150). New York, NY: Routledge.
Ryan, A. M., & Sackett, P. R. (1989). Exploratory study of individual
assessment practices: Interrater reliability and judgments of assessor
effectiveness. Journal of Applied Psychology, 74, 568 –579. http://dx.doi
.org/10.1037/0021-9010.74.4.568
Rynes, S. L., & Lawler, J. (1983). A policy-capturing investigation of the
role of expectancies in decisions to pursue job alternatives. Journal of
Applied Psychology, 68, 620 – 631. http://dx.doi.org/10.1037/0021-9010
.68.4.620
Rynes, S. L., & Miller, H. E. (1983). Recruiter and job influences on
candidates for employment. Journal of Applied Psychology, 68, 147–
154. http://dx.doi.org/10.1037/0021-9010.68.1.147
Sackett, P. R., Zedeck, S., & Fogli, L. (1988). Relations between measures
of typical and maximum performance. Journal of Applied Psychology,
73, 482– 486. http://dx.doi.org/10.1037/0021-9010.73.3.482
Sands, W. A. (1973). A method for evaluating alternative recruiting—
Selection strategies: The CAPER model. Journal of Applied Psychology,
57, 222–227. http://dx.doi.org/10.1037/h0034712
Schmidt, F. L., Berner, J. G., & Hunter, J. E. (1973). Racial differences in
validity of employment tests: Reality or illusion. Journal of Applied
Psychology, 58, 5–9. http://dx.doi.org/10.1037/h0035408
Schmitt, N., & Chan, D. (1998). Personnel selection: A theoretical ap-
proach. Thousand Oaks, CA: Sage.
Schmitt, N., & Coyle, B. W. (1976). Applicant decisions in the employ-
ment interview. Journal of Applied Psychology, 61, 184 –192. http://dx
.doi.org/10.1037/0021-9010.61.2.184
Schmitt, N., & Oswald, F. L. (2006). The impact of corrections for faking
on the validity of noncognitive measures in selection settings. Journal of
Applied Psychology, 91, 613– 621. http://dx.doi.org/10.1037/0021-9010
.91.3.613
Schneider, B. (1987). The people make the place. Personnel Psychology,
40, 437– 453. http://dx.doi.org/10.1111/j.1744-6570.1987.tb00609.x
Scott, C. M. (1938). Background and personal data as factors in the
prediction of scholastic success in college. Journal of Applied Psychol-
ogy, 22, 42– 49. http://dx.doi.org/10.1037/h0059916
Simas, K., & McCarrey, M. (1979). Impact of recruiter authoritarianism
and applicant sex on evaluation and selection decisions in a recruitment
interview analogue study. Journal of Applied Psychology, 64, 483– 491.
http://dx.doi.org/10.1037/0021-9010.64.5.483
Slaughter, J. E., Cable, D. M., & Turban, D. B. (2014). Changing job
seekers’ image perceptions during recruitment visits: The moderating
role of belief confidence. Journal of Applied Psychology, 99, 1146 –
1158. http://dx.doi.org/10.1037/a0037482
Smith, C. A., Organ, D. W., & Near, J. P. (1983). Organizational citizen-
ship behavior: Its nature and antecedents. Journal of Applied Psychol-
ogy, 68, 653– 663. http://dx.doi.org/10.1037/0021-9010.68.4.653
Smith, P. C., & Kendall, L. M. (1963). Retranslation of expectations: An
approach to the construction of unambiguous anchors for rating scales.
Journal of Applied Psychology, 47, 149 –155. http://dx.doi.org/10.1037/
h0047060
Steiner, D. D., & Gilliland, S. W. (1996). Fairness reactions to personnel
selection techniques in France and the United States. Journal of Applied
 304 PLOYHART, SCHMITT, AND TIPPINS
Psychology, 81, 134 –141. http://dx.doi.org/10.1037/0021-9010.81.2
.134
Sussman, M., & Robertson, D. U. (1986). The validity of validity: An
analysis of validation study designs. Journal of Applied Psychology, 71,
461– 468. http://dx.doi.org/10.1037/0021-9010.71.3.461
Swaroff, P. G., Barclay, L. A., & Bass, A. R. (1985). Recruiting sources:
Another look. Journal of Applied Psychology, 70, 720 –728. http://dx
.doi.org/10.1037/0021-9010.70.4.720
Swider, B. W., Zimmerman, R. D., & Barrick. (2015). Searching for the
right fit: Development of applicant person-organization fit perceptions
during the recruitment process. Journal of Applied Psychology, 100,
880 – 893.
Tay, L., Drasgow, F., Rounds, J., & Williams, B. A. (2009). Fitting
measurement models to vocational interest data: Are dominance models
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.
ideal? Journal of Applied Psychology, 94, 1287–1304. http://dx.doi.org/
10.1037/a0015899
This document is copyrighted by the American Psychological Association or one of its allied publishers.
Taylor, H. C., & Russell, J. T. (1939). The relationship of validity coeffi-
cients to the practical effectiveness of tests in selection: Discussion and
tables. Journal of Applied Psychology, 23, 565–578. http://dx.doi.org/
10.1037/h0057079
Taylor, M. S. (1988). Effects of college internships on individual partici-
pants. Journal of Applied Psychology, 73, 393– 401. http://dx.doi.org/10
.1037/0021-9010.73.3.393
Terman, L. M., Otis, A. S., Dickson, V., Hubbard, O. S., Norton, J. K.,
Howard, L., . . . Cassingham, C. C. (1917). A trial of mental and
pedagogical tests in a civil service examination for policemen and
firemen. Journal of Applied Psychology, 1, 17–29.
Thorndike, E. L. (1920). A constant error in psychological ratings. Journal
of Applied Psychology, 4, 25–29. http://dx.doi.org/10.1037/h0071663
Tippins, N. T., Beaty, J., Drasgow, F. D., Gibson, W. M., Pearlman, K.,
Segall, D., & Shepherd, W. J. (2006). Unproctored, internet testing in
employment settings. Personnel Psychology, 59, 189 –225. http://dx.doi
.org/10.1111/j.1744-6570.2006.00909.x
Turban, D. B., Lau, C.-M., Ngo, H.-Y., Chow, I. H. S., & Si, S. X. (2001).
Organizational attractiveness of firms in the People’s Republic of China:
A person-organization fit perspective. Journal of Applied Psychology,
86, 194 –206. http://dx.doi.org/10.1037/0021-9010.86.2.194
Van Hoye, G., & Lievens, F. (2009). Tapping the grapevine: A closer look
at word-of-mouth as a recruitment source. Journal of Applied Psychol-
ogy, 94, 341–352. http://dx.doi.org/10.1037/a0014066
Van Iddekinge, C. H., Ferris, G. R., Perrewé, P. L., Blass, F. R., Heetderks,
T. D., & Perryman, A. A. (2009). Effects of selection and training on
unit-level performance over time: A latent growth modeling approach.
Journal of Applied Psychology, 94, 829 – 843. http://dx.doi.org/10.1037/
a0014453
Van Iddekinge, C. H., Putka, D. J., & Campbell, J. P. (2011). Reconsid-
ering vocational interests for personnel selection: The validity of an
interest-based selection test in relation to job knowledge, job perfor-
mance, and continuance intentions. Journal of Applied Psychology, 96,
13–33. http://dx.doi.org/10.1037/a0021193
Van Iddekinge, C. H., Roth, P. L., Raymark, P. H., & Odle-Dusseau, H. N.
(2012). The criterion-related validity of integrity tests: An updated
meta-analysis. Journal of Applied Psychology, 97, 499 –530. http://dx
.doi.org/10.1037/a0021196
Von Rosenstiel, L., & Woschee, R. (2011). Dei soziale validitat am
Biespiel der systematischen Personalbeurteilung [Social validity using
the example of systematic personnel assessment]. In P. Gelleri & C.
Winter (Eds.), Potenziale der personal-ssychologie (pp. 297–314). Cam-
bridge, MA: Hofgrebe.
Walker, H. J., Feild, H. S., Bernerth, J. B., & Becton, J. B. (2012).
Diversity cues on recruitment websites: Investigating the effects on job
seekers’ information processing. Journal of Applied Psychology, 97,
214 –224. http://dx.doi.org/10.1037/a0025847
Walker, H. J., Feild, H. S., Giles, W. F., Armenakis, A. A., & Bernerth,
J. B. (2009). Displaying employee testimonials on recruitment web sites:
Effects of communication media, employee race, and job seeker race on
organizational attraction and information credibility. Journal of Applied
Psychology, 94, 1354 –1364. http://dx.doi.org/10.1037/a0014964
Wanous, J. P. (1973). Effects of a realistic job preview on job acceptance,
job attitudes, and job survival. Journal of Applied Psychology, 58,
327–332. http://dx.doi.org/10.1037/h0036305
Weiner, J. A., & Morrison, J. D., Jr. (2009). Unproctored online testing:
Environmental conditions and validity. Industrial and Organizational
Psychology: Perspectives on Science and Practice, 2, 27–30. http://dx
.doi.org/10.1111/j.1754-9434.2008.01102.x
Weitz, J. (1956). Job expectancy and survival. Journal of Applied Psychol-
ogy, 40, 245–247. http://dx.doi.org/10.1037/h0048082
Weller, I., Holtom, B. C., Matiaske, W., & Mellewigt, T. (2009). Level and
time effects of recruitment sources on early voluntary turnover. Journal
of Applied Psychology, 94, 1146 –1162. http://dx.doi.org/10.1037/
a0015924
Wernimont, P. F., & Campbell, J. P. (1968). Signs, samples, and criteria.
Journal of Applied Psychology, 52, 372–376. http://dx.doi.org/10.1037/
h0026244
Wiest, W. M., Porter, L. W., & Ghiselli, E. E. (1961). Relationship
between individual proficiency and team performance and efficiency.
Journal of Applied Psychology, 45, 435– 440. http://dx.doi.org/10.1037/
h0046225
Zhao, H., & Liden, R. C. (2011). Internship: A recruitment and selection
perspective. Journal of Applied Psychology, 96, 221–229. http://dx.doi
.org/10.1037/a0021295
Zickar, M. J., & Robie, C. (1999). Modeling faking good on personality
items: An item-level analysis. Journal of Applied Psychology, 84, 551–
563. http://dx.doi.org/10.1037/0021-9010.84.4.551
Received May 12, 2015
Revision received October 8, 2015
Accepted December 8, 2015 䡲