Building Better Measures of Role Ambiguity and Role Con Ict: The Validation of New Role Stressor Scales

See discussions, stats, and author profiles for this publication at: https://www.researchgate.
net/publication/313476371
Building better measures of role ambiguity and role conﬂict: The validation of
new role stressor scales
Article in Work & Stress · January 2017

DOI: 10.1080/02678373.2017.1292563
CITATIONS READS
86 8,856
10 authors, including:
Nathan Bowling Caitlin Blackmore

University of Central Florida Wright State University
90 PUBLICATIONS 8,233 CITATIONS 7 PUBLICATIONS 295 CITATIONS
SEE PROFILE SEE PROFILE
Caleb Bragg Kellie Kennedy

Central Connecticut State University NASA
8 PUBLICATIONS 545 CITATIONS 23 PUBLICATIONS 311 CITATIONS
SEE PROFILE SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Safe Autonomous Systems Operations View project
Your attention please! Toward a better understanding of research participant carelessness View project
All content following this page was uploaded by Qiang Wang on 07 February 2018.
The user has requested enhancement of the downloaded file.

Work & Stress
An International Journal of Work, Health & Organisations
ISSN: 0267-8373 (Print) 1464-5335 (Online) Journal homepage: http://www.tandfonline.com/loi/twst20
Building better measures of role ambiguity and

role conflict: The validation of new role stressor
scales
Nathan A. Bowling, Steven Khazon, Gene M. Alarcon, Caitlin E. Blackmore,

Caleb B. Bragg, Michael R. Hoepf, Alex Barelka, Kellie Kennedy, Qiang Wang
& Haiyan Li
To cite this article: Nathan A. Bowling, Steven Khazon, Gene M. Alarcon, Caitlin E. Blackmore,
Caleb B. Bragg, Michael R. Hoepf, Alex Barelka, Kellie Kennedy, Qiang Wang & Haiyan Li (2017)
Building better measures of role ambiguity and role conflict: The validation of new role stressor
scales, Work & Stress, 31:1, 1-23, DOI: 10.1080/02678373.2017.1292563
To link to this article: http://dx.doi.org/10.1080/02678373.2017.1292563
Published online: 27 Mar 2017. Submit your article to this journal
Article views: 356 View related articles
View Crossmark data Citing articles: 1 View citing articles
Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=twst20
Download by: [130.108.121.216] Date: 07 November 2017, At: 05:48

WORK & STRESS, 2017
VOL. 31, NO. 1, 1–23
http://dx.doi.org/10.1080/02678373.2017.1292563
Building better measures of role ambiguity and role conflict:

The validation of new role stressor scales
Nathan A. Bowlinga, Steven Khazonb, Gene M. Alarconc, Caitlin E. Blackmorea,
Caleb B. Braggd, Michael R. Hoepfe, Alex Barelkaf, Kellie Kennedyg, Qiang Wangh
and Haiyan Lii
a
Department of Psychology, Wright State University, Dayton, OH, USA; bAmazon, Seattle, WA, USA; cAir Force
Research Laboratory, Dayton, OH, USA; dDepartment of Psychology, Central Connecticut State University,
New Britain, CT, USA; eNational Transportation Safety Board, Washington, DC, USA; fDepartment of
Management and Quantitative Methods, Illinois State University, Normal, IL, USA; gDepartment of
Psychology, Old Dominion University, Norfolk, VA, USA; hSchool of Business, East China University of Science
Downloaded by [130.108.121.216] at 05:48 07 November 2017
and Technology, Shanghai, People’s Republic of China; iInnovation & Entrepreneurship Education &
Incubation Center, Xiamen University, Xiamen, People’s Republic of China
ABSTRACT ARTICLE HISTORY

Occupational stress researchers have given considerable attention Received 10 December 2014
to role ambiguity and role conflict as predictors of employee Accepted 24 October 2016
health, job attitudes and behaviour. However, the validity of the
KEYWORDS
Rizzo, House, and Lirtzman’s (1970) scales – the most popular role Role ambiguity; role conflict;
stressor measures – has been a source of disagreement among role stressors; occupational
researchers. In response to the disputed validity of the Rizzo et al. stress; scale development
scales, we developed new measures of role ambiguity and role
conflict and conducted five studies to examine their psychometric
qualities (Study 1 N = 101 U.S. workers; Study 2 N = 118 workers
primarily employed in the U.S.; Study 3 N = 135 employed U.S.
MBA students; Study 4 N = 973 members of the U.S. Air Force
(USAF); Study 5 N = 234 workers primarily employed in the U.S.).
Across these five studies, we found that the new role stressor
scales have desirable psychometric qualities: they displayed high
levels of substantive validity, high levels of internal consistency
and test–retest reliability, they produced an interpretable factor
structure, and we found evidence of their construct validity. We
therefore recommend that these new scales be used in future
research on role stress.
Introduction
Occupational stress research largely focuses on the relationship between work stressors –
aspects of the work environment that have the potential to harm one’s well-being – and
psychological and physical symptoms, job attitudes and employee behaviour (see Beehr,
1995; Cooper, Dewe, & O’Driscoll, 2001). Considerable attention has been given to two
work stressors: role ambiguity and role conflict. These two variables are collectively
referred to as “role stressors.” The popularity of role stressors among organisational
researchers is reflected in the fact that three meta-analyses have been published
CONTACT Nathan A. Bowling nathan.bowling@wright.edu

© 2017 Informa UK Limited, trading as Taylor & Francis Group
2 N. A. BOWLING ET AL.
summarising the role stressor literature (Fisher & Gitelson, 1983; Jackson & Schuler, 1985;
Ortqvist & Wincent, 2006). The role stressor literature has grown considerably during the
ensuing years. Indeed, a 10 October 2016 PsycINFO search yielded 1316 references
appearing since 2006 that mention either “role ambiguity” or “role conflict.” Role stressors
thus continue to attract scholarly attention.
The validity of Rizzo et al.’s (1970) role ambiguity and role conflict scales – the most
widely used role stressor measures – has been a subject of scholarly debate. Whereas
some research supports the validity of the Rizzo et al. scales (e.g. Gonzalez-Roma &
Lloret, 1998; House, Schuler, & Levanoni, 1983; Kelloway & Barling, 1990; Murphy &
Gable, 1988; Netemeyer, Johnston, & Burton, 1990; Rosenkrantz, Luthans, & Hennessey,
1983; Smith, Tisak, & Schmieder, 1993), other research raises doubts about their validity
(e.g. Gilboa, Shirom, Fried, & Cooper, 2008; Harris, 1991; King & King, 1990; McGee, Fer-
guson, & Seers, 1989; Tracy & Johnson, 1981, 1983). In response to the disputed validity of
the Rizzo et al. scales, we developed new role ambiguity and role conflict scales that were
designed to address the criticisms levied against the Rizzo et al. scales. We review criti-
cisms of the Rizzo et al. scales in the following section.
Criticisms of the Rizzo et al. (1970) role stressor scales

Jackson and Schuler (1985) observed that at the time of their meta-analysis approximately
85% of studies used the Rizzo et al. (1970) scales to assess role stressors. More recently,
Gilboa et al. (2008) reported that most studies included in their meta-analysis of stres-
sor–performance relationships used the Rizzo et al. scales to assess role ambiguity and
role conflict. Although there are other role stressor measures (e.g. Breaugh & Colihan,
1994; Sawyer, 1992; Van Veldhoven & Meijman, 1994; Wännström, Peterson, Asberg,
Nygren, & Gustavsson, 2009), the Rizzo et al. measures continue to dominate the
literature.
Despite their popularity, the Rizzo et al. scales have been the target of criticism (see
Gilboa et al., 2008; Harris, 1991; King & King, 1990; McGee et al., 1989; Tracy &
Johnson, 1981, 1983). Much of this criticism has been centred around the fact that the
Rizzo et al. scales confound the construct purportedly being assessed with direction of
the item wording: all of the role ambiguity items are reverse-scored (e.g. “I know what
my responsibilities are”) and all the role conflict items are positively scored (e.g. “I
receive incompatible requests from two or more people”). This confound renders factor
analyses using the Rizzo et al. scales ambiguous (Kelloway & Barling, 1990; McGee
et al., 1989; Tracy & Johnson, 1981), and it threatens the construct validity of the Rizzo
et al. scales (King & King, 1990).
A second limitation of the Rizzo et al. scales is that many of their items have question-
able content validity (King & King, 1990). The Rizzo et al. role ambiguity item “I know
that I have divided my time properly,” for instance, may be contaminated with time man-
agement content and the Rizzo et al. role conflict items “I receive an assignment without
the manpower to complete it” and “I receive an assignment without adequate resources
and materials to execute it” may actually measure role overload.
Now that we have summarised criticisms of the Rizzo et al. role stressor scales, we will
discuss our efforts to develop new measures of role ambiguity and role conflict. As we
describe below, these new scales were designed to overcome the potential limitations of
WORK & STRESS 3
the Rizzo et al. measures. Specifically, the new scales avoid confounding the construct pur-
portedly being assessed with the direction of item wording and they were designed to
include items that have strong conceptual links with their corresponding role stressor.
In the following sections, we discuss the process we used to create and validate the new
role stressor scales. We first describe the item generation and initial item culling process.
We then discuss the item analysis we used to create the new scales (Study 1) and we discuss
studies examining the new scales’ substantive validity (Study 2), test–retest reliability
(Study 3), factor structure (Study 4) and construct validity (Study 5).
Item generation and initial item culling

Procedure
Item generation. An I/O psychology professor and four I/O psychology Ph.D. students
used shared definitions of role ambiguity (The extent to which one is confronted with
unclear work situations) and role conflict (The extent to which one experiences incompa-
tible work demands) as a basis to generate items for each construct. The item authors
were encouraged to write both positively scored and reverse-scored items. This process
generated an initial pool of 65 role stressor items (a list of these items is available from
first author).
Culling of items from the initial item pool. Seven subject matter experts (SMEs) – six
with Ph.D.’s in I/O psychology who had each published research on occupational stress,
and one I/O psychology Ph.D. student who was employed by a government agency that
sponsors occupational stress research – independently rated the face validity of each of
the 65 items from the initial pool. The SMEs were provided with the same definitions
of role ambiguity and role conflict that were used by the item authors (see definitions
above) and asked to judge on a 5-point scale (1 = No face validity; 5 = Strong face validity)
the extent to which each of the 65 items was a face valid measure of role ambiguity and a
face valid measure of role conflict. The SMEs were also asked to provide feedback on any
problematic items. We presented the items to the SMEs in random order.
The SME ratings helped us identify items that were face valid measures of their respective
role stressor, but not face valid measures of the other role stressor. An ideal role ambiguity
item, for instance, received high role ambiguity ratings, but low role conflict ratings. Using
these criteria, we retained 12 role ambiguity items and 12 role conflict items for inclusion in
the Study 1 item analysis. These 24 items received an average SME rating of 4.72 on their
respective role stressor and an average SME rating of 1.26 on the other role stressor.
Note that several of the items from the initial item pool that were not retained for the
item analysis were actually rated favourably by the SMEs. We chose to exclude several face
valid items from the item analysis for two reasons: (a) we wished to keep the item analysis
questionnaire brief and (b) in an effort to minimise item redundancy, the item analysis
questionnaire excluded some face valid items because they were highly similar to
another item included in the item analysis questionnaire.
Given that the new role stressor items were written by a professor and four doctoral
students, it is possible that the item content was inadvertently biased toward an academic
context. To address this concern, the remaining five studies each used participants from a
variety of occupations and employment settings.
Study 1: item analysis of new role stressor items

Item analysis, which is used to examine item discrimination and difficulty of individual
items, plays an important role in the development of new scales (Spector, 1991). In
Study 1, we conducted item analysis using 24 face valid items retained from the initial
item culling (a list of these 24 items is available from the first author).
Study 1: procedure
We collected the Study 1 data from 101 U.S. workers employed in a variety of jobs (sample
job titles include “accountant,” “correctional officer,” “dental hygienist,” “social worker,”
“teacher,” and “welder”). Each participant was an acquaintance of one of the study’s
authors and was recruited via a social media website. The response rate was 56%.
The average participant was 34 years old, was employed 7 years in his or her current
job, and worked 35 hours per week. Approximately 63% of participants were female. Par-
ticipants responded to the 24 items retained from the item culling. Each item was on a 7-
point scale from 1 (strongly disagree) to 7 (strongly agree).
Study 1: results and discussion

We retained items for the preliminary versions of the new role ambiguity and new role
conflict scales on the basis of three criteria: (a) we considered the item discrimination
of each item, (b) we eliminated items that were worded too similarly to each other,
and (c) we retained an equal number of positively scored and reverse-scored items
for each scale. We imposed the latter criterion because we wished to avoid confound-
ing the direction of item wording with the construct being assessed. Using these
criteria, we created a six-item role ambiguity scale and a six-item role conflict scale
(see Table 1).
Table 1. Item analysis of new role stressor scales (Study 1).

Item ITC
Six-item role ambiguity scale
I am not sure what is expected of me at work .68
The requirements of my job aren’t always clear .72
I often don’t know what is expected of me at work .75
I know everything that I am expected to do at work with certainty. (R) .78
My job duties are clearly defined. (R) .88
I know what I am required to do for every aspect of my job. (R) .79
Average across six-role ambiguity items .76
Six-item role conflict scale
In my job, I often feel like different people are “pulling me in different directions.” .70
I have to deal with competing demands at work .69
My superiors often tell me to do two different things that can’t both be done .71
The tasks I am assigned at work rarely come into conflict with each other. (R) .73
The things I am told to do at work do not conflict with each other. (R) .69
In my job, I’m seldom placed in a situation where one job duty conflicts with other job duties. (R) .72
Average across six role conflict items .70
Note: Ns ranged from 98 to 101. (R) indicates reverse-scored item. ITC is item-total correlation. The scale instructions were
as follows: “The following questions ask about your experiences at your current job. Please answer each question by
selecting the response option that most closely matches your opinion.” Each item was on a 7-point scale from 1 (strongly
disagree) to 7 (strongly agree).
WORK & STRESS 5
Each of the 12 retained items was judged to have high face validity (SME ratings ≥ 4.29)
in the initial item culling and each item yielded high item-total correlations (≥.68) in
Study 1 (see Table 1). As shown in Table 2, Study 1 found high internal-consistency
reliabilities for both the new role ambiguity scale (α = .91) and the new role conflict
scale (α = .89) and we observed a significant positive relationship between the new role
ambiguity scale and the new role conflict scale (r = .47, p < .01). These findings were gen-
erally replicated across our subsequent studies (see Table 2).
In sum, we used Study 1 to create the new role ambiguity and role conflict scales. We
conducted Studies 2 through 5 to further examine the reliability and validity of these new
measures. Note that Table 1 recreates the final versions of the role stressor scales.
Study 2: the substantive validity of the role stressor scales

King and King (1990) criticised several of the Rizzo et al. items for having questionable
content validity; we thus conducted Study 2 to examine the substantive validity of the
new role stressor items and the Rizzo et al. role stressor items. The substantive validity
of an item is the degree to which that item “ … is judged to be reflective of, or theoretically
linked to, some construct of interest” (Anderson & Gerbing, 1991, p. 732). Substantive val-
idity and content validity thus differ only in their level of analysis: substantive validity is a
characteristic of an individual item; content validity is a characteristic of a set of items.
Examining substantive validity is critical because substantive validity is a prerequisite to
construct validity (Anderson & Gerbing, 1991, p. 732).
Substantive validity is assessed by asking judges – people drawn from the population
who are typical of those who the items in question are intended to be administered to
(e.g. working adults) – to read non-technical definitions of multiple psychological con-
structs. The judges then independently assign each item from the scale(s) being validated
Table 2. Reliabilities and correlational analyses for role ambiguity and role conflict for both the new
scales and the Rizzo et al. (1970) scales (Studies 1, 3, 4 and 5).
M SD 1 2 3 4
Study 1
1. New role ambiguity scale 2.92 1.38 (.91)
2. New role conflict scale 3.69 1.47 .47** (.89)
Study 3
1. New role ambiguity scale (T1) 3.56 1.21 (.86)
2. New role conflict scale (T1) 4.17 1.22 .36a** (.83)
3. New role ambiguity scale (T2) 3.50 1.19 .65b** .37b** (.87)
4. New role conflict scale (T2) 4.17 1.14 .42b** .64b** .52c** (.80)
Study 4
Study 5
3. Rizzo et al. role ambiguity scale 2.94 1.17 .80** .47** (.88)
4. Rizzo et al. role conflict scale 3.50 1.34 .68** .68** .55** (.88)
Note: Cronbach’s αs appear on the diagonals within parentheses. For Study 1, N = 97. For Study 3, aN = 135; bN = 72; cN =
89. Differences in Ns are due to different response rates between Time 1 and Time 2. The Study 3 test–retest reliability
coefficients appear in bold print. For Study 4, N = 972 to 973. For Study 5, N = 234. The Study 5 correlations relevant to
convergent validity appear in bold print.
*p < .05.
**p < .01.
to one of the construct categories. A given item has high substantive validity to the extent
that different judges (a) consistently assign it to the construct category that the item’s
author intended it to assess and (b) do not consistently assign it to any one of the remain-
ing construct categories.
Study 2: procedure
Judges (N = 118 workers, most of whom were employed in the U.S.), recruited from Mech-
anical Turk (MTurk; https://requester.mturk.com/; see Landers & Behrend, 2015), evalu-
ated the substantive validity of both the new role stressor items and the Rizzo et al. role
stressor items. The average judge was 35 years old, was employed in his or her current
job for over 5 years, and worked roughly 40 hours per week. Fifty-six per cent of judges
were male. Example job titles of the judges include “bank teller,” “machinist,” “proofrea-
der,” and “teacher.”
Following the method described by Anderson and Gerbing (1991), we first provided the
judges with definitions of both role ambiguity and role conflict. It was critical that we use
non-technical definitions, given that the judges were not organisational researchers. We
provided judges the following definition of role ambiguity, which we referred to it as
“job uncertainty”:
In some jobs, workers are routinely placed in situations that lack clarity. They may, for
instance, be given assignments that are unclear or they may not be given information
needed to do their job. Other workers may be given unclear information about their own
authority or responsibilities. Still, other workers may have to follow organizational policies
or guidelines that are unclear. Each of these is an example of “job uncertainty.”
We provided judges with the following definition of role conflict, which we referred to as
“job conflict”:
In some jobs, workers are routinely placed in situations that make them feel conflicted. Some
workers, for instance, may feel conflicted because their job requires them to do things that are
opposed to their personal standards or values. Other workers may feel conflicted because
their supervisor gives them an order that is inconsistent with what others have told them
or is inconsistent with company policy. Each of these is an example of “job conflict.”
We then presented the judges with the new role stressor items and the Rizzo et al. role
stressor items. These items were presented in random order. We asked the judges to indi-
cate whether each item either best matched the definition of (a) role ambiguity, (b) role
conflict or (c) role overload (the inclusion of role overload is a vestige of our initial
goal of developing a new measure of role overload; we abandoned that goal after the
Study 2 data were collected). Judges were allowed to assign each item to only one role
stressor construct.
We used the data from this sorting task to compute two statistics described by Ander-
son and Gerbing (1991): (a) the proportion of substantive agreement (PSA; the proportion
of judges who assign a given item to its intended construct) and (b) substantive-validity
coefficient (CSV; the extent to which judges assigned a given item to its intended construct
more than to any other unintended construct). There are no strict criteria for what con-
stitutes acceptable PSA and CSV values. Because judges were asked to sort each item into
one of three construct categories, chance alone would result in PSA and CSV values of
WORK & STRESS 7
.33. It is important that any criterion values be higher than these chance values; we thus
used criterion values of .50 for both PSA and CSV (see Yao, Wu, & Yang, 2008).

Role ambiguity. The top half in Table 3 reports results for the role ambiguity items. As
shown in the table, the mean PSA score of the new role ambiguity items (PSA = .84) was
higher than the mean PSA score of the Rizzo et al. role ambiguity items (PSA = .68; d =
0.96). Similarly, the mean CSV score of the new role ambiguity items (CSV = .74) was
higher than the mean CSV score of the Rizzo et al. role ambiguity items (CSV = .46; d =
1.01). Note that all six of the new role ambiguity items and five of the six Rizzo et al.
role ambiguity items yielded PSA and CSV values >.50.
Role conflict. The bottom half in Table 3 reports results for the role conflict items. As
shown in the table, the mean PSA score of the new role conflict items (PSA = .74) was higher
than the mean PSA score of the Rizzo et al. role conflict items (PSA = .52; d = 1.04) and the
mean CSV score of the new role conflict items (CSV = .57) was higher than the mean CSV
Table 3. Substantive validity of the new role stressor scales and the Rizzo et al. (1970) role stressor
scales (Study 2).
Item PSA CSV
New role ambiguity items
I am not sure what is expected of me at work .87 .77
The requirements of my job aren’t always clear .91 .86
I often don’t know what is expected of me at work .87 .78
I know everything that I am expected to do at work with certainty. (R) .81 .68
My job duties are clearly defined. (R) .77 .66
I know what I am required to do for every aspect of my job. (R) .79 .66
Average across six new role ambiguity items .84 .74
Rizzo et al. (1970) role ambiguity items
I feel certain about how much authority I have. (R) .81 .66
Clear, planned goals and objectives for my job. (R) .74 .55
I know that I have divided my time properly. (R) .23 −.31
I know what my responsibilities are. (R) .76 .61
I know exactly what is expected of me. (R) .77 .62
Explanation is clear of what has to be done. (R) .77 .62
Average across six Rizzo et al. (1970) role ambiguity items .68 .46
New role conflict items
In my job, I often feel like different people are “pulling me in different directions.” .77 .61
I have to deal with competing demands at work .61 .33
My superiors often tell me to do two different things that can’t both be done .73 .60
The tasks I am assigned at work rarely come into conflict with each other. (R) .77 .61
The things I am told to do at work do not conflict with each other. (R) .80 .66
In my job, I’m seldom placed in a situation where one job duty conflicts with other job duties. (R) .77 .60
Average across six new role conflict items .74 .57
Rizzo et al. (1970) role conflict items
I have to do things that should be done differently .69 .46
I receive an assignment without the manpower to complete it .06 −.65
I work with two or more groups who operate quite differently .75 .56
I have to buck a rule or policy in order to carry out an assignment .79 .66
I receive incompatible requests from two or more people .68 .44
I do things that are apt to be accepted by one person and not accepted by others .72 .48
I receive an assignment without adequate resources and materials to execute it .20 −.24
I work on unnecessary things .29 −.06
Average across eight Rizzo et al. (1970) role conflict items .52 .21
Note: N = 118. (R) indicates reverse-scored item. PSA is the proportion of substantive agreement. CSV is the substantive-val-
idity coefficient. Values that surpass the criterion value of .50 appear in bold print.
score of the Rizzo et al. role conflict items (CSV = .21; d = 1.06). Five of the six new role
conflict items yielded PSA and CSV values >.50. The one potentially objectionable item
– “I have to deal with competing demands at work” – produces an acceptable PSA
(.61), but a low CSV (.33). The low CSV value is largely attributable to the fact that
27% of judges assigned this item to the “role overload” construct category. This is
not surprising, however, because role overload is often regarded as a specific form
of role conflict (Kahn, Wolfe, Quinn, Snoek, & Rosenthal, 1964; King & King,
1990). Note that six of the eight Rizzo et al. role conflict items were judged to have
low substantive validity: three of these items produced PSA values <.50; six of these
items produced CSV values <.50.
Having supported the substantive validity of the new role stressor scales, the remaining
studies examined the new scales’ test–retest reliability, their factor structure, and their con-
struct validity.
Study 3: the test–retest reliability of new role stressor scales

Study 3: procedure
The Study 3 participants were employed MBA students who attended a U.S. university.
We collected data at two time points (T1 N = 135; T2 N = 89) to examine the test–retest
reliability of the role stressor scales. Because the lag between T1 and T2 was only 4
weeks, we assumed that participants as a whole experience little change in their actual
levels of role stressors. Furthermore, the data were collected in January and February
(2013), which minimises the possibility that seasonal changes in actual role stressors
(e.g. fluctuations caused by holidays or tax season) occurred during data collection. It is
thus reasonable to examine the test–retest reliability of the new role stressor scales
using these data.
In order to match the T1 and T2 questionnaires – while at the same time maintaining
participant anonymity – we asked participants to answer four biographical questions (e.g.
“What was your high school mascot?” and “What was your first job?”). All of the MBA
students approached at T1 responded to the questionnaire and 72 participants provided
T2 data that could be matched to their T1 data.
Approximately 51% of Study 3 participants were male and the average participant had
worked for 3 years in his or her current job. Sample job titles included “accountant,”
“sales manager,” and “tax analyst.” Unfortunately, an omission in the study question-
naire prevented us from calculating participants’ mean age and number of hours
worked per week. However, based on our previous experience with this MBA student
population and after a discussion with the chair of the management department, we esti-
mate that the average participant was approximately 30 years old and worked roughly
35–40 hours per week.
Study 3: measures
The T1 and T2 questionnaires included the new role ambiguity (T1 α = .86; T2 α = .87)
and role conflict (T1 α = .83; T2 α = .80) scales. Each item used a 7-point scale from 1
(strongly disagree) to 7 (strongly agree).
WORK & STRESS 9

As shown in Table 2, the 4-week test–retest reliabilities were .65 (p < .01) for role ambi-
guity and .64 (p < .01) for role conflict. These estimates are similar to the test–retest
reliabilities found for other work-related measures, such as job characteristics (4-week
test–retest reliabilities ranged from .57 to .73 in Head, Molleston, Sorensen, & Gargano,
1986).
The above correlational analyses speak to the relative stabilities of the role stressor
scores. We conducted paired sample t-tests to examine the role stressor scores’ absolute
stabilities. These latter analyses indeed support the scores’ absolute stability: the mean
T1 role ambiguity score (M = 3.51) was not significantly different from the mean T2
role ambiguity score (M = 3.47, t = .33, n.s.) and the mean T1 role conflict score (M =
4.30) was not significantly different from the mean T2 role conflict score (M = 4.16, t =
1.11, n.s.). In short, both the new role ambiguity scale and the new role conflict scale
yielded an acceptable level of test–retest reliability.
Study 4: examination of the factor structure of new role stressor scales

In Study 4, we examined the factor structure of the new role stressor scales using confir-
matory factor analysis (CFA). Consistent with prior findings (see King & King, 1990), we
predicted that the new role ambiguity and role conflict items would load onto two distinct
– yet related – factors.
Study 4: procedure
We collected data from 973 members of the USAF. Participants were enrolled in a USAF
training programme and completed the study questionnaire as part of their classes. Both
officers (68% of sample) and enlisted personnel (32% of sample) were represented in the
dataset. Approximately 82% of participants were male. The average participant was 28
years old and had worked 1.47 years with his or her current supervisor. Approximately
41% of participants reported receiving at least some graduate education. The participants
represented several occupations, including aerospace maintenance, developmental engin-
eering and explosives safety. The response rate was 100%.
Study 4: measures
Participants completed the new measures of role ambiguity (α = .87) and role conflict
(α = .77). Each item was on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree).

We used Mplus (Muthén & Muthén, 2007) to test two CFA models: (a) a one-factor base-
line model in which the six new role ambiguity items and the six new role conflict items
loaded onto a single latent factor and (b) a two-factor hypothesised model in which the six
new role ambiguity items loaded onto one latent factor and the six new role conflict items
loaded onto a second latent factor. The one-factor model yielded poor fit (χ 2(54) =
1203.58, p < .01; CFI = .735; RMSEA = .148; SRMR = .117), whereas the two-factor model
yielded acceptable fit (χ 2(53) = 443.25, p < .01; CFI = .910; RMSEA = .087; SRMR = .059).
Furthermore, the two-factor model produced significantly better fit than did the one-
factor model (Δχ 2(−1) = −760.33, p < .01). The factor loadings of the 12 role stressor
items, which were taken from the analyses of the two-factor model, are each statistically
significant (p < .01) and are ≥.54 (see Table 4). Also of note, the latent role ambiguity vari-
able was positively correlated with the latent role conflict variable (r = .51; p < .01).
Study 5: examining the construct validity of the new role stressor scales
In Study 5, we examined the new role stressor scales’ and the Rizzo et al. role stressor
scales’ relationships with several theoretically derived predictors, correlates and criterion
variables. We specifically designed this study to address King and King’s (1990) sugges-
tions that construct validity studies should (a) examine the convergent and discriminant
validity of role stressor scales and (b) test a priori hypotheses involving external variables
expected to differentially related to role ambiguity and role conflict.
A nomological network for role stressors

To guide the construct validation effort of Study 5, we developed a nomological network
(see Cronbach & Meehl, 1955) describing the role stressors’ hypothesised relationships
with several external variables. As we discuss in the following subsections, this network
includes four types of external variables: (a) those expected to be related to both role ambi-
guity and role conflict, (b) those expected to be primarily related to role ambiguity, (c)
those expected to be primarily related to role conflict and (d) those expected to be unre-
lated to both role ambiguity and role conflict. We included these latter variables as a test of
discriminant validity.
Variables expected to be related to both role stressors. Conservation of resources
(COR) theory (Hobfoll, 1989) provides the basis for the inclusion of employee well-
being indicators in the nomological network. Specifically, COR theory suggests that the
Table 4. Role stressor item factor loadings (standardised solution; Study 4).
Latent factor
Role Role
Item ambiguity conflict
I am not sure what is expected of me at work .64
The requirements of my job aren’t always clear .54
I often don’t know what is expected of me at work .57
I know everything that I am expected to do at work with certainty. (R) .59
My job duties are clearly defined. (R) .70
I know what I am required to do for every aspect of my job. (R) .57
In my job, I often feel like different people are “pulling me in different directions.” .72
I have to deal with competing demands at work .74
My superiors often tell me to do two different things that can’t both be done .77
The tasks I am assigned at work rarely come into conflict with each other. (R) .72
The things I am told to do at work do not conflict with each other. (R) .79
In my job, I’m seldom placed in a situation where one job duty conflicts with other job .70
duties. (R)
Note: N = 973. (R) indicates reverse-scored item. All factor loadings are statistical significant at p < .01.
WORK & STRESS 11
presence of role stressors depletes one’s existing personal resources (e.g. physical stamina,
self-efficacy, time) and inhibits the acquisition of new resources. The net loss of resources,
in turn, is expected to undermine one’s well-being. In Study 5, we measured three well-
being indices: (a) global job satisfaction, (b) psychological symptoms and (c) physical
symptoms. We predict that both role stressor scales will be negatively associated with
global job satisfaction and positively associated with psychological and physical
symptoms.
Variables that reflect a worker’s positive evaluation of their supervisor (e.g. satisfaction
with supervision) or organisation (e.g. perceived organisational support; Eisenberger,
Huntington, Hutchison, & Sowa, 1986) are expected to be negatively associated with
both role stressor scales. This is because workers may generally believe that it is the
duty of their supervisor and organisation to provide a favourable working environment.
When the work environment is unfavourable – as in the case where one is exposed to
excessive levels of role stressors – workers are likely to believe that their supervisor and
organisation are not delivering on their obligations, and thus workers are likely to evaluate
the supervisor and organisation negatively. As a result, we predict that both role stressor
scales will be negatively related to subordinates’ satisfaction with supervision and per-
ceived organisational support.
Finally, we predict that both role stressor scales will be positively related to employee
withdrawal behaviour. Withdrawal behaviour, which involves failing to attend work
when scheduled (e.g. being absent, being late or leaving work early), is a specific form
of counterproductive work behaviour (CWB; Gruys & Sackett, 2003). Role stressors are
expected to be positively associated with CWBs – and by extension with withdrawal beha-
viours – via the mediation effects of negative emotional states (Fox, Spector, & Miles,
2001).
Variables expected to be primarily related to role ambiguity. Because the new role
ambiguity scale is expected to converge with existing measures of role ambiguity, we
included the Rizzo et al. role ambiguity scale in Study 5. Despite its potential shortcom-
ings, the Rizzo et al. scale has some level of validity (see Gonzalez-Roma & Lloret,
1998; House et al., 1983; Netemeyer et al., 1990; Smith et al., 1993), and thus we expect
that it will be positively related to the new role ambiguity scale.
Because role ambiguity is inherently related to a lack of information (Kahn et al., 1964;
King & King, 1990; Rizzo et al., 1970), we expect that any variable reflecting the quality of
communication within one’s workplace will be negatively related to role ambiguity. Thus,
we predict that satisfaction with workplace communication (Spector, 1985), feedback from
people and from job tasks (Hackman & Oldham, 1980), and the clarity of one’s job
description will be negatively related to role ambiguity.
Variables expected to be primarily related to role conflict. We predict that the new role
conflict scale will converge with existing measures of role conflict; thus, we included the
Rizzo et al. role conflict scale in Study 5. As we reviewed above, the Rizzo et al. scale
has several potential shortcomings that may undermine its validity. Despite its limitations,
however, there is evidence that the Rizzo et al. role conflict scale has at least some level of
validity (see Gonzalez-Roma & Lloret, 1998; House et al., 1983; Netemeyer et al., 1990;
Smith et al., 1993). As a result, we expected that it will be positively related to the new
role conflict scale. We also predict that role overload – the extent to which one has too
much work or has work that is too difficult (Cooper et al., 2001) – will be positively
related to the new role conflict scale. This latter prediction is based on the notion that role
overload is a form of role conflict (Kahn et al., 1964; King & King, 1990).
Role conflict occurs when workers face multiple work demands that are incompatible
with each other (Kahn et al., 1964; King & King, 1990; Rizzo et al., 1970). The presence of
incompatible job demands is particularly common among workers who occupy boundary-
spanning positions (Friedman & Podolny, 1992; Miles, 1976; Miles & Perreault, 1976).
This occurs because such workers are routinely subjected to role expectations originating
from multiple groups of people. These groups often impose differing and incompatible
demands on the target employee. The nomological network thus predicts that the
extent to which one’s job requires boundary spanning will be positively related to the
new role conflict scale.
Variables expected to be related to neither role stressor. In order to examine the dis-
criminant validity of the new role stressor scales, the nomological network included open-
ness to experience and self-monitoring (see Campbell & Fiske, 1959; Spector, 1991). Both
variables are conceptually unrelated to the two role stressors; thus, we predict that they will
yield non-significant relationships with the new role ambiguity and new role conflict
scales.
Study 5: procedure
We used MTurk (https://requester.mturk.com/; see Landers & Behrend, 2015) to recruit
employed adult (N = 234) for the Study 5 sample. Approximately 56% of the Study 5 par-
ticipants were female. The average participant was 34 years old, was employed for over 4.5
years in his or her current job, and worked approximately 40 hours per week. Sample job
titles included “cashier,” “chemist,” “paralegal,” “police officer” and “teacher.”
Study 5: measures
Participants completed self-report measures of the study variables. Unless otherwise indi-
cated, each measure used a 7-point scale from 1 (strongly disagree) to 7 (strongly agree).
New role stressor scales. The Study 5 questionnaire included the new role ambiguity
scale (α = .87) and the new role conflict scale (α = .82).
Rizzo et al. role stressor scales. We administered the six-item Rizzo et al. (1970) role
ambiguity scale (α = .88; an example item is “I know what my responsibilities are”
[reverse scored]) and the eight-item Rizzo et al. (1970) role conflict scale (α = .88; an
example item is “I receive incompatible requests from two or more people”).
Global job satisfaction. We used the average of three items from the Michigan Organ-
izational Assessment Questionnaire (MOAQ) job satisfaction scale (Cammann, Fichman,
Jenkins, & Klesh, 1979) to assess global job satisfaction. Meta-analytic evidence provides
support for the reliability and validity of this measure (Bowling & Hammond, 2008). An
example item is, “All in all I am satisfied with my current job.” The MOAQ job satisfaction
scale yielded an α of .91.
Psychological symptoms. We assessed psychological symptoms with the average of 12
items from the General Health Questionnaire (GHQ; Goldberg, 1978). Each item was on a
4-point scale from 1 (not at all) to 4 (very often). An example item is “Did you feel con-
stantly under strain?” The GHQ yielded an α of .90.
WORK & STRESS 13
Physical symptoms. We assessed physical symptoms with the average of 12 items from
Spector and Jex’s (1998) Physical Symptoms Inventory (PSI). The items asked participants
how often they had experienced a particular physical symptom during the last month.
Each item was on a 5-point scale from 1 (not at all) to 5 (every day). An example item
is “An upset stomach or nausea.” The PSI yielded an α of .88.
Satisfaction with supervision. We used the average of five items from Beehr et al.
(2006) to assess satisfaction with supervision. An example item is “All in all, I am very sat-
isfied with my supervisor.” This scale yielded an α of .92.
Perceived organisational support. We assessed perceived organisational support with
the short version (average of eight items) of the Eisenberger et al. (1986) Perceived Organ-
izational Support Scale (POSS). An example item is “The organization really cares about
my well-being.” The POSS yielded an α of .93.
Withdrawal behaviour. We assessed withdrawal behaviour using the poor attendance
subscale of the Gruys and Sackett (2003) CWB scale (average of five items). An example
item is “Been absent from work without a legitimate excuse.” This scale yielded an α of .86.
Satisfaction with communication. We assessed satisfaction with communication with
the average of four items from Spector’s (1985) Job Satisfaction Survey (JSS). An example
item is “Communications seem good within this organization.” The JSS subscale yielded
an α of .84.
Feedback from people. We used the average of three items from the Job Diagnostic
Survey (JDS; Hackman & Oldham, 1980) to assess feedback from people. Two of these
items were on a 7-point scale from 1 (strongly disagree) to 7 (strongly agree); the remaining
item was on a 7-point scale from 1 (not at all) to 7 (completely). An example item is
“Supervisors often let me know how well they think I am performing the job.” This
Task feedback. We assessed task feedback with the average of three items from the JDS
(Hackman & Oldham, 1980). Two of these items were on a 7-point scale from 1 (strongly
disagree) to 7 (strongly agree); the remaining item was on a 7-point scale from 1 (not at all)
to 7 (completely). An example item is “Just doing the work required by the job provides
many chances for me to figure out how well I am doing.” This scale yielded an α of .69.
Clarity of job description. We assessed clarity of job description with a single item
written specifically for the current study: “My job description is clearly written.”
Role overload. We assessed role overload with the average of three items from Beehr,
Walsh, and Taber (1976). An example item is “The performance standards on my job are
too high.” The role overload scale yielded an α of .77.
Boundary spanning. We used items written specifically for the current study to assess
four types of boundary spanning: (a) boundary-spanning involving multiple supervisors
(α = .88), (b) boundary-spanning involving multiple coworkers (α = .84), (c) boundary-
spanning involving other departments within one’s own organisation (α = .86) and (d)
boundary-spanning involving organisational outsiders (α = .90). Each of the four scales
was computed using the average of two items. The items from each scale used wording
that paralleled the wording of the other three scales. The supervisor items, for instance,
were “Several different supervisors provide me with instructions on how to do my job”
and “My job requires me to take orders from several different supervisors”; the coworker
items were “In my job, I often receive instructions from several different coworkers from
within my own department,” and “In my job, several different coworkers from within my
own department provide me with requests.”
Openness to experience. We assessed openness to experience with the average of 10
items from the International Personality Item Pool (IPIP, n.d.; Goldberg et al., 2006).
An example item is “I enjoy hearing new ideas.” This scale yielded an α of .83.
Self-monitoring. We assessed self-monitoring with the average of 10 items from the
IPIP (n.d.; Goldberg et al., 2006). An example item is “I use flattery to get ahead.” This

Correlational analyses
As described in the following subsections, we used correlational analyses to examine the
convergence between the new role stressor scales and the Rizzo et al. role stressor scales.
We also used correlational analysis to examine the new role stressor scales’ relationships
with the external variables included in the nomological network.
Convergence between new role stressor scales and Rizzo et al. scales. We report
convergent validity evidence from Study 5 in Table 2. As predicted, the new role ambi-
guity scale was positively correlated with the Rizzo et al. role ambiguity scale (r = .80, p
< .01) and the new role conflict scale was positively correlated with the Rizzo et al. role
conflict scale (r = .68, p < .01). These findings thus provide evidence of convergent
validity.
External variables expected to relate to both role stressors. The external variables that
we hypothesised would be related to both role stressors – global job satisfaction, psycho-
logical symptoms, physical symptoms, satisfaction with supervision, perceived organis-
ational support, and withdrawal behaviour – were each significantly (p < .05) correlated
with both the new role stressor scales and with the Rizzo et al. scales (see Table 5). All
of these correlations were in the direction predicted in the nomological network.
To summarise the magnitude of these relationships, we computed the means of the
absolute values of (a) the new role stressor scales’ correlations with these six external
variables and (b) the Rizzo et al. role stressor scales’ correlations with these six external
variables. As shown in Table 5, the mean correlation for the new role ambiguity scale
(mean |r| = .41) was similar to the mean correlation for the Rizzo et al. role ambiguity
scale (mean |r| = .38; z = 0.79, n.s.). The mean correlation for the new role conflict scale
(mean |r| = .28), however, was significantly smaller than the mean correlation for the
Rizzo et al. role conflict scale (mean |r| = .45; z = −3.51, p < .01).
External variables expected to relate to role ambiguity. The external variables
expected to be primarily related to role ambiguity – satisfaction with communication,
feedback from people, task feedback and clarity of job description – were each signifi-
cantly (p < .01) related to both the new role ambiguity scale and the Rizzo et al. role
ambiguity scale (see Table 5). These correlations were all in the direction predicted
by the nomological network. In general, the new role ambiguity scale (mean |r| = .57)
and the Rizzo et al. role ambiguity scale (mean |r| = .56) predicted these external vari-
ables equally well (z = 0.30, n.s.). The new role conflict scale (mean |r| = .35), however,
was more weakly related to these external variables than was the Rizzo et al. role conflict
scale (mean |r| = .50; z = −3.19, p < .01).
Table 5. Reliabilities, correlations and z-tests for the new role stressor scales and the Rizzo et al. (1970) role stressor scales (Study 5).
New role Rizzo et al. role z Comparing role New role Rizzo et al. role z Comparing role
ambiguity scale ambiguity scale ambiguity scales conflict scale conflict scale conflict scales
Hypothesised to be related to all role stressors

Global job satisfaction (α = .91) −.47** −.48** .28 −.26** −.42** 3.28**
Psychological symptoms (α = .90) .57** .51** 1.75 .41** .57** −3.59**
Physical symptoms (α = .88) .27** .21** 1.49 .23** .36** −2.60**
Satisfaction with supervision (α = .92) −.48** −.47** −.28 −.34** −.50** 3.42**
Perceived organisational support (α = .93) −.47** −.48** .28 −.34** −.55** 4.57**
Withdrawal behaviour (α = .86) .25** .16* 2.21* .13* .35** −4.33**
Mean |r| = .41 Mean |r| = .38 .79 Mean |r| = .28 Mean |r| = .45 −3.51**
Hypothesised to be related to role ambiguity
Satisfaction with communication (α = .84) −.71** −.61** −3.33** −.50** −.73** 5.91**
Feedback from people (α = .83) −.41** −.43** .53 −.17** −.38** 4.17**
Task feedback (α = .69) −.54** −.56** .58 −.32** −.44** 2.48*
Clarity of job description (α = NA) −.63** −.64** .32 −.42** −.48** 1.30
Mean |r| = .57 Mean |r| = .56 .30 Mean |r| = .35 Mean |r| = .50 −3.19**
Hypothesised to be related to role conflict
Role overload (α = .77) .51** .44** 1.94 .69** .73** −1.18
Boundary spanning – supervisors (α = .88) .32** .26** 1.51 .49** .49** .00
Boundary spanning – coworkers (α = .84) .29** .19** 2.49* .50** .51** −.23
Boundary spanning – other departments (α = .86) .38** .28** 2.56* .61** .57** .98
Boundary spanning – organisational outsiders .25** .19** 1.48 .42** .36** 1.26
(α = .90)
Mean |r| = .35 Mean |r| = .27 2.03* Mean |r| = .54 Mean |r| = .53 .23
Hypothesised to be unrelated to both role
stressors
Openness to experience (α = .83) −.05 .01 −1.44 .00 −.06 1.14
Self-monitoring (α = .88) .08 .09 −.24 .10 .24** −2.70**
WORK & STRESS

Mean |r| = .06 Mean |r| = .05 .24 Mean |r| = .05 Mean |r| = .15 −1.91
Note: N = 230 to 234. Cronbach’s αs appear within parentheses. NA = Not applicable (due to the use of a single-item scale). The correlations within the shaded cells are those predicted in the
nomological network. Mean |r| is the mean absolute value of correlations between the given role stressor scale and the given set of external variables.
*p < .05.
**p < .01.
15
External variables expected to relate to role conflict. The external variables expected to
be primarily related to role conflict – role overload and the four boundary-spanning vari-
ables – were each significantly (p < .01) related to both the new role conflict scale and to
the Rizzo et al. role conflict scale (see Table 5). All of these correlations were in the
direction predicted by the nomological network. In general, the new role conflict
scale (mean |r| = .54) and the Rizzo et al. role conflict scale (mean |r| = .53) predicted
these external variables equally well (z = 0.23, n.s.). The new role ambiguity scale (mean
|r| = .35), however, was more strongly related to these external variables than was the
Rizzo et al. role ambiguity scale (mean |r| = .27; z = 2.03, p < .05).
External variables expected to be related to neither role stressor. As expected, open-
ness to experience and self-monitoring were generally unrelated to the new role ambiguity
scale (mean |r| = .06), the Rizzo et al. role ambiguity scale (mean |r| = .05), the new role
conflict scale (mean |r| = .05), and the Rizzo et al. role conflict scale (mean |r| = .15).
There was, however, one exception to these null results: The Rizzo et al. role conflict
scale was significantly related to self-monitoring (r = .24, p < .01).
Do role ambiguity and role conflict scales yield distinct patterns of relationships
with external variables?
Although researchers generally regard role ambiguity and role conflict as distinct con-
structs, measures of the two variables often yield similar patterns of relationships with
external variables (King & King, 1990). This may suggest that role stressor scales have gen-
erally failed to adequately capture differences between distinct constructs; thus, we exam-
ined the similarity between role ambiguity’s and role conflict’s patterns of relationship
with external variables. We conducted these analyses separately for the new role stressor
scales and for the Rizzo et al. role stressor scales.
First, we computed the 17 effect sizes (r 2) reflecting the new role ambiguity scale’s
relationships with each of the external variables examined in Study 5. We repeated this
process for the new role conflict scale, thus creating two vectors of 17 effect sizes. Computing
the correlation between these two vectors, we found that the new role ambiguity scale’s
relationship with a given external variable was not significantly related to the new role con-
flict scale’s relationship with the same external variable (r = .35, n.s.). This suggests that the
two new role stressor scales yielded generally distinct patterns of relationships with the exter-
nal variables. Parallel analyses using the Rizzo et al. role stressor scales yielded a noticeably
stronger correlation (r = .54, p < .05), suggesting that the Rizzo et al. role stressor scales may
less effectively distinguish between role ambiguity and role conflict.
Incremental validity of the new role stressor scales. We conducted hierarchical regression
analyses to examine whether the new role stressor scales predicted unique variance in the Study
5 external variables controlling for the effects of the Rizzo et al. role stressor scales. In Step 1 of
each analysis we regressed the criterion variable onto one of the Rizzo et al. role stressor scales;
in Step 2 we added the corresponding new role stressor scale as a predictor. Rather than using
every external variable as a criterion variable, these analyses used only the external variables
hypothesised to be related to the given role stressor (see the shaded cells in Table 5).
Regression analyses for role ambiguity. We observed several instances in which the
new role ambiguity scale yielded incremental validity controlling for the effects of the
Rizzo et al. role ambiguity scale (see the top half in Table 6). Specifically, we found that
WORK & STRESS 17
Table 6. Hierarchical regression analyses examining the incremental validity of the new role ambiguity
and role conflict scales (Study 5).
Criterion variable Ordered predictors β R 2Δ R 2 Total
Global job satisfaction 1. Rizzo et al. role ambiguity scale −.28** .23**
2. New role ambiguity scale −.25** .02** .25**
Psychological symptoms 1. Rizzo et al. role ambiguity scale .15 .26***
2. New role ambiguity scale .45** .07** .33**
Physical symptoms 1. Rizzo et al. role ambiguity scale .00 .04**
Satisfaction with supervision 1. Rizzo et al. role ambiguity scale −.24** .22**
Perceived organisational support 1. Rizzo et al. role ambiguity scale −.30** .23**
2. New role ambiguity scale −.22* .01* .25**
Withdrawal behaviour 1. Rizzo et al. role ambiguity scale −.10 .02*

Satisfaction with communication 1. Rizzo et al. role ambiguity scale −.12 .38**
Feedback from people 1. Rizzo et al. role ambiguity scale −.27** .18**
2. New role ambiguity scale −.19 .01 .20**
Task feedback 1. Rizzo et al. role ambiguity scale −.36** .32**
Clarity of job description 1. Rizzo et al. role ambiguity scale −.38** .42**
Global job satisfaction 1. Rizzo et al. role conflict scale −.46** .18**
2. New role conflict scale .04 .00 .18**
Psychological symptoms 1. Rizzo et al. role conflict scale .53** .32**
Physical symptoms 1. Rizzo et al. role conflict scale .39** .13**
2. New role conflict scale −.03 .00 .13**
Satisfaction with supervision 1. Rizzo et al. role conflict scale −.49** .25**
2. New role conflict scale −.01 .00 .25**
Perceived organisational support 1. Rizzo et al. role conflict scale −.58** .30**
Withdrawal behaviour 1. Rizzo et al. role conflict scale .49** .12**
2. New role conflict scale −.20* .02* .14**
Role overload 1. Rizzo et al. role conflict scale .47** .53**
2. New role conflict scale .37** .07** .60**
Boundary spanning – supervisors 1. Rizzo et al. role conflict scale .29** .24**
Boundary spanning – coworkers 1. Rizzo et al. role conflict scale .32** .26**
Boundary spanning – other departments 1. Rizzo et al. role conflict scale .28** .32**
Boundary spanning – organisational outsiders 1. Rizzo et al. role conflict scale .14 .13**
Note: Ns ranged from 230 to 234. All βs are from the last (second) step of the regression equation.
*p < .05.
**p < .01.
the new role ambiguity scale predicted incremental variance in 9 of 10 criterion variables.
It is of note, however, that the Step 2 βs for the Rizzo et al. role ambiguity scale were sig-
nificant in 6 of the 10 regression analyses, indicating that the Rizzo et al. scale predicted
unique variance in several criteria controlling for the new role ambiguity scale.
Regression analyses for role conflict. We also found that the new role conflict scale pre-
dicted incremental variance in the manner implied by the nomological network for 5 of 11
criterion variables (see the bottom half in Table 6). Note that the negative unique relationship
between the new role conflict scale and withdrawal behaviour is inconsistent with the positive
relationship predicted in the nomological network; we thus interpret the analyses for withdra-
wal behaviour as being unsupportive of the incremental validity of the new role conflict scale.
The results for role conflict produced a noteworthy pattern: the new role conflict scale
predicted incremental validity in each of the five variables hypothesised to be specifically
related to role conflict, but it failed to predict incremental variance in any of the six vari-
ables hypothesised to be related to both role stressors. Also of note, the Step 2 βs for the
Rizzo et al. role conflict scale were significant in 10 of the 11 regression analyses, indicating
that the Rizzo et al. scale predicted unique variance in several criteria controlling for the
new role conflict scale.
In summary, Study 5 provided evidence of the construct validity of both the new role
stressor scales and the Rizzo et al. role stressor scales. First, the new role stressor scales
were each strongly correlated with the corresponding Rizzo et al. scale. Convergence
was particularly strong for role ambiguity. Second, both the new role stressor scales and
the Rizzo et al. role stressor scales were related to external variables in the directions pre-
dicted by the nomological network. The patterns of relationships with external variables,
however, suggests that the new scales may more effectively distinguish between role ambi-
guity and role conflict than do the Rizzo et al. scales. Finally, the new role stressor scales
and the Rizzo et al. scales each predicted unique variance in several variables from the
nomological network. In the General Discussion, we consider what the Study 5 findings
mean for the relative validity of the new role stressor scales and the Rizzo et al. scales.
General discussion
Role ambiguity and role conflict are among the most commonly studied work stressors. As
we noted in the Introduction, however, researchers have expressed conflicting views about
the validity of the most popular role stressor measures – the Rizzo et al. scales. In the
current article we described the development and validation of new role stressors scales
that we created to address the criticisms levied against the Rizzo et al. scales.
Potential advantages of the new role stressor scales

When developing the new role stressor scales, we gave explicit attention to avoiding pro-
blems inherent to the Rizzo et al. scales. First, the new scales avoid confounding the direc-
tion of item wording with the construct being assessed. This was achieved by creating new
role ambiguity and role conflict scales that each included an equal number of positively
scored and reverse-scored items. Second, the new role stressor items were generally
judged to have higher levels of substantive validity than were the Rizzo et al. role stressor
items (see Study 2).
WORK & STRESS 19
Additional evidence speaks to the favourable psychometric properties of the new role
stressor scales. The new scales yielded high levels of internal-consistency reliability (see
Studies 1, 3, 4 and 5), they had reasonable levels of test–retest reliability (see Study 3),
they produced an interpretable factor structure (see Study 4), and they yielded a pattern
of relationships with external variables that was consistent with the pattern hypothesised
in the nomological network (Study 5). We also found that the new role stressor scales may
more effectively distinguish between role ambiguity and role conflict than do the Rizzo
et al. scales (Study 5). Given these findings, there is a strong basis to conclude that the
new stressor scales are valid measures of their intended constructs; we thus recommend
their further use.
Are the Rizzo et al. (1970) scales valid measures of role stressors?
In the Introduction we noted that although most studies have used the Rizzo et al. scales to
assess role stressors, the validity of those scales has been the subject of scholarly debate.
That debate has largely centred on the scales’ factor structure (e.g. Harris, 1991; Kelloway
& Barling, 1990; McGee et al., 1989). In the current research, however, we examined the
Rizzo et al. scales’ substantive validity (Study 2) and construct validity (Study 5), thus con-
tributing to a better understanding of the quality of the Rizzo et al. scales. The following
subsections address the validity of the Rizzo et al. scales in light of the current research
findings.
Validity of the Rizzo et al. role ambiguity scale. With the exception of one item, the
Rizzo et al. role ambiguity items were each judged to have acceptable levels of substantive
validity (see Table 3). Furthermore, the Rizzo et al. role ambiguity scale yielded a high level
of internal-consistency reliability and it was strongly correlated with the new role ambi-
guity scale (see Table 2). It also yielded a pattern of relationships with external variables
that was consistent with the pattern hypothesised in the nomological network (see
Table 5). It is of note that the Rizzo et al. role ambiguity scale yielded a weaker mean
relationship with variables hypothesised to be related to role conflict than did the new
role ambiguity scale. This latter finding may suggest that the Rizzo et al. role ambiguity
scale is less contaminated with role conflict content than is the new role ambiguity
scale. Thus, despite being the target of criticism, the Rizzo et al. role ambiguity scale
appears to be an effective measure and researchers should not dismiss results obtained
using that scale.
Validity of the Rizzo et al. role conflict scale. Despite the fact that the Rizzo et al. role
conflict scale was correlated with the new role conflict scale (see Table 2) and with external
variables from the nomological network (see Table 5), Studies 2 and 5 raise concerns about
the Rizzo et al. role conflict scale’s validity. Study 2 found that six of the eight Rizzo et al.
role conflict items displayed inadequate levels of substantive validity (see Table 3). These
six items, in other words, appear to assess content that is not part of the role conflict con-
struct. As Anderson and Gerbing (1991) noted “ … measures that do not have adequate
substantive validity cannot have adequate construct validity” (p. 732); thus, the absence of
substantive validity is in itself grounds to question the acceptability of the Rizzo et al. role
conflict scale.
Study 5 also found that the Rizzo et al. role conflict scale yielded surprisingly strong
relationships with external variables that we hypothesised would be related to role
ambiguity. Furthermore, the Rizzo et al. role conflict scale was significantly related to one
variable we hypothesised would be unrelated to role conflict – self-monitoring. Like the
substantive validity findings from Study 2, these findings may suggest that the Rizzo
et al. role conflict scale is contaminated with extraneous content.
Also of note, Study 5 found that the Rizzo et al. role conflict scale yielded stronger corre-
lations with external variables we expected to relate to both role stressors than did the new
role conflict scale. On one hand, this finding could be interpreted as evidence of the Rizzo
et al. role conflict scale’s superior criterion-related related validity; however, in light of the
Rizzo et al. role conflict scale’s surprisingly strong relationships with variables hypothesised
to be related to role ambiguity and its significant relationship with self-monitoring, we ques-
tion this interpretation. That is, the Rizzo et al. role conflict scale appears to relate indiscri-
minately to a variety of external variables – including those that it should be either weakly
related or unrelated to. Thus, given the results of Studies 2 and 5, researchers should be cau-
tious when interpreting findings obtained using the Rizzo et al. role conflict scale.
Limitations
We note three limitations of the current research. First, with the exception of Study 3, each
study used cross-sectional data. Because this prevented us from testing causal relation-
ships, we encourage future longitudinal studies using the new role stressor scales.
Second, all five studies exclusively used self-report measures, which could have made
our results vulnerable to the effects of common-method variance (CMV). Note,
however, that we observed several non-significant relationships (e.g. correlations for open-
ness to experience and self-monitoring found in Study 5), providing direct evidence that
CMV did not necessarily inflate our observed relationships. Furthermore, self-reports are
perhaps the best means of assessing many of the variables measured in the current
research. This is particularly true for the measurement of internal psychological states,
such as perceptions of work stressors, job satisfaction and psychological symptoms.
Finally, we did not evaluate the extent to which the new role stressor scales capture the
full content domains of their respective constructs. It is thus possible that the new
scales do not assess every sub-facet of role ambiguity and role conflict. The new role con-
flict scale, for instance, does not appear to directly capture the extent to which respondents
are required to perform work tasks that violate their personal standards or values – a situ-
ation that can be construed as a form of role conflict (see King & King, 1990).
Conclusion
Other researchers have argued that improved role ambiguity and role conflict scales are
needed (e.g. Kelloway & Barling, 1990; King & King, 1990; McGee et al., 1989). As a
result, we conducted the current research in an effort to develop and validate new role
stressor scales. Across a series of five studies, we generally found support for the reliability
and validity of the new scales; we thus recommend their use in future role stress research.
Acknowledgements
The authors wish to thank Terry A. Beehr, Misty M. Bennett, Kevin J. Eschleman, Sharon Glazer,
Steve M. Jex, Jesse S. Michel, and Jeannie Nigam for serving as subject matter experts in the initial
WORK & STRESS 21
item culling. We would like to thank Todd Dewett, Melissa L. Gruys, and Natasha Munshi for assist-
ing with the Study 3 data collection and Brad Schlessman for assisting with Study 4 data collection.
Disclosure statement
No potential conflict of interest was reported by the authors.
References
Anderson, J. C., & Gerbing, D. W. (1991). Predicting the performance of measures in a confirma-
tory factor analysis with a pretest assessment of their substantive validities. Journal of Applied
Psychology, 76, 732–740.
Beehr, T. A. (1995). Psychological stress in the workplace. New York, NY: Routledge.
Beehr, T. A., Glaser, K. M., Beehr, M. J., Beehr, D. E., Wallwey, D. A., Erofeev, D., & Canali
Kristophor G. (2006). The nature of satisfaction with subordinates: Its predictors and importance
to supervisors. Journal of Applied Social Psychology, 36, 1523–1547.
Beehr, T. A., Walsh, J. T., & Taber, T. D. (1976). Relationship of stress to individually and organi-
zationally valued states: Higher order needs as a moderator. Journal of Applied Psychology, 61,
41–47.
Bowling, N. A., & Hammond, G. D. (2008). A meta-analytic examination of the construct validity of
the Michigan Organizational Assessment Questionnaire Job Satisfaction Subscale. Journal of
Vocational Behavior, 73, 63–77.
Breaugh, J. A., & Colihan, J. P. (1994). Measuring facets of job ambiguity: Construct validity evi-
dence. Journal of Applied Psychology, 79, 191–202.
Cammann, C., Fichman, M., Jenkins, D., & Klesh, J. (1979). The Michigan Organizational
Assessment Questionnaire. Unpublished manuscript, University of Michigan, Ann Arbor.
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-
multimethod matrix. Psychological Bulletin, 56, 81–105.
Cooper, C. L., Dewe, P. J., & O’Driscoll, M. P. (2001). Organizational stress: A Review and Critique
of Theory, Research, and Applications. Thousand Oaks, CA: Sage.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological
Bulletin, 52, 281–302.
Eisenberger, R., Huntington, R., Hutchison, S., & Sowa, D. (1986). Perceived organizational
support. Journal of Applied Psychology, 71, 500–507.
Fisher, C. D., & Gitelson, R. (1983). A meta-analysis of the correlates of role conflict and ambiguity.
Journal of Applied Psychology, 68, 320–333.
Fox, S., Spector, P. E., & Miles, D. (2001). Counterproductive work behavior (CWB) in response to
job stressors and organizational justice: Some mediator and moderator tests for autonomy and
emotions. Journal of Vocational Behavior, 59, 291–309.
Friedman, R. A., & Podolny, J. (1992). Differentiation of boundary spanning roles: Labor nego-
tiations and implications for role conflict. Administrative Science Quarterly, 37, 28–47.
Gilboa, S., Shirom, A., Fried, Y., & Cooper, C. (2008). A meta-analysis of work demand stressors
and job performance: Examining main and moderating effects. Personnel Psychology, 61, 227–
271.
Goldberg, D. P. (1978). Manual of the General Health Questionnaire. Windsor: NFER-Nelson.
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough,
H. C. (2006). The International Personality Item Pool and the future of public-domain person-
ality measures. Journal of Research in Personality, 40, 84–96.
Gonzalez-Roma, V., & Lloret, S. (1998). Construct validity of Rizzo et al.’s (1970) role conflict and
ambiguity scales: A multisample study. Applied Psychology: An International Review, 47, 535–
545.
Gruys, M. L., & Sackett, P. R. (2003). Investigating the dimensionality of counterproductive work
behavior. International Journal of Selection and Assessment, 11, 30–42.
Hackman, J. R., & Oldham, G. R. (1980). Work redesign. Reading, MA: Addison-Wesley.
Harris, M. M. (1991). Role conflict and role ambiguity as substance versus artifact: A confirmatory
factor analysis of House, Schuler, and Levanoni’s (1983) scales. Journal of Applied Psychology, 76,
122–126.
Head, T. C., Molleston, J. L., Sorensen, P. F., & Gargano, J. (1986). The impact of implementing a
quality circles intervention on employee task perceptions. Group and Organization Studies, 11,
360–373.
Hobfoll, S. E. (1989). Conservation of resources: A new attempt at conceptualizing stress. American
Psychologist, 44, 513–524.
House, R. J., Schuler, R. S., & Levanoni, E. (1983). Role conflict and ambiguity scales: Reality or
artifacts? Journal of Applied Psychology, 68, 334–337.
International Personality Item Pool: A scientific collaboratory for the development of advanced
measures of personality traits and other individual differences. (n.d.). Retrieved October 8,
2012, from http://ipip.ori.org/
Jackson, S. E., & Schuler, R. S. (1985). A meta-analysis and conceptual critique of research on role
ambiguity and role conflict in work settings. Organizational Behavior and Human Decision
Processes, 36, 16–78.
Kahn, R. L., Wolfe, D. M., Quinn, R. P., Snoek, J. D., & Rosenthal, R. (1964). Organizational stress:
Studies in role conflict and ambiguity. New York, NY: Wiley.
Kelloway, E. K., & Barling, J. (1990). Item content versus item wording: Disentangling role conflict
and role ambiguity. Journal of Applied Psychology, 75, 738–742.
King, L. A., & King, D. W. (1990). Role conflict and role ambiguity: A critical assessment of con-
struct validity. Psychological Bulletin, 107, 48–64.
Landers, R. N., & Behrend, T. S. (2015). An inconvenient truth: Arbitrary distinctions between
organizational, Mechanical Turk, and other convenience samples. Industrial and
Organizational Psychology, 8, 142–164.
McGee, G. W., Ferguson, C. E., & Seers, A. (1989). Role conflict and role ambiguity: Do the scales
measure these two constructs? Journal of Applied Psychology, 74, 815–818.
Miles, R. H. (1976). Role requirements as sources of occupational stress. Journal of Applied
Miles, R. H., & Perreault, W. D. (1976). Organizational role conflict: Its antecedents and conse-
quences. Organizational Behavior and Human Performance, 17, 19–44.
Murphy, C. A., & Gable, R. K. (1988). Validity and reliability of the original and abridged role con-
flict and ambiguity scales. Educational and Psychological Measurement, 48, 743–751.
Muthén, L. K., & Muthén, B. O. (2007). Mplus user’s guide (5th ed.). Los Angeles: Muthén &
Muthén.
Netemeyer, R. G., Johnston, M. W., & Burton, S. (1990). Analysis of role conflict and role ambiguity
in a structural equations framework. Journal of Applied Psychology, 75, 148–157.
Ortqvist, D., & Wincent, J. (2006). Prominent consequences of role stress: A meta-analytic review.
International Journal of Stress Management, 13, 399–422.
Rizzo, J. R., House, R. J., & Lirtzman, S. I. (1970). Role conflict and ambiguity in complex organ-
izations. Administrative Science Quarterly, 15, 150–163.
Rosenkrantz, S. A., Luthans, F., & Hennessey, H. W. (1983). Role conflict and ambiguity scales: An
evaluation of psychometric properties and the role of social desirability response bias.
Educational and Psychological Measurement, 43, 957–970.
Sawyer, J. E. (1992). Goal and process clarity: Specification of multiple constructs of role ambiguity
and a structural equation model of their antecedents and consequences. Journal of Applied
Smith, C. S., Tisak, J., & Schmieder, R. A. (1993). The measurement properties of the role conflict
and role ambiguity scales: A review and extension of the empirical research. Journal of
Organizational Behavior, 14, 37–48.
Spector, P. E. (1985). Measurement of human service staff satisfaction: Development of the job sat-
isfaction survey. American Journal of Community Psychology, 13, 693–713.
Spector, P. E. (1991). Summated rating scale construction: An introduction. Newbury Park, CA:
Sage.
WORK & STRESS 23
Spector, P. E., & Jex, S. M. (1998). Development of four self-report measures of job stressors and
strain: Interpersonal Conflict at Work Scale, Organizational Constraints Scale, Quantitative
Workload Inventory, and Physical Symptoms Inventory. Journal of Occupational Health
Tracy, L., & Johnson, T. W. (1981). What do the role conflict and role ambiguity scales measure?
Journal of Applied Psychology, 66, 464–469.
Tracy, L., & Johnson, T. W. (1983). Measurement of role stress: Dimensionality of scale items.
Social Behavior and Personality: An international Journal, 11, 1–7.
Van Veldhoven, M., & Meijman, T. (1994). Het meten van psychosociale arbeidsbelasting met een
vragenlijst: de vragenlijst beleving en beoordeling van de arbeid (VBBA) [Measuring psychosocial
workload with a survey: the questionnaire on the experience and evaluation of work (QEEW)].
Amsterdam: NIA.
Wännström, I., Peterson, U., Asberg, M., Nygren, A., & Gustavsson, J. P. (2009). Psychometric
properties of scales in the General Nordic Questionnaire for Psychological and Social Factors
at Work (QPS Nordic): Confirmatory factor analysis and prediction of certified long-term sick-
ness absence. Scandinavian Journal of Psychology, 50, 231–244.
Yao, G., Wu, C. H., & Yang, C. T. (2008). Examining the content validity of the WHOQOL-BREF
from respondents’ perspective by quantitative methods. Social Indicators Research, 85, 483–498.
View publication stats

Building Better Measures of Role Ambiguity and Role Con Ict: The Validation of New Role Stressor Scales

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Building Better Measures of Role Ambiguity and Role Con Ict: The Validation of New Role Stressor Scales

Uploaded by

Copyright:

Available Formats

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

Article in Work & Stress · January 2017

Nathan Bowling Caitlin Blackmore

SEE PROFILE SEE PROFILE

Caleb Bragg Kellie Kennedy

SEE PROFILE SEE PROFILE

Safe Autonomous Systems Operations View project

The user has requested enhancement of the downloaded file.

ISSN: 0267-8373 (Print) 1464-5335 (Online) Journal homepage: http://www.tandfonline.com/loi/twst20

Building better measures of role ambiguity and

Nathan A. Bowling, Steven Khazon, Gene M. Alarcon, Caitlin E. Blackmore,

To link to this article: http://dx.doi.org/10.1080/02678373.2017.1292563

Published online: 27 Mar 2017. Submit your article to this journal

Article views: 356 View related articles

View Crossmark data Citing articles: 1 View citing articles

Full Terms & Conditions of access and use can be found at

Download by: [130.108.121.216] Date: 07 November 2017, At: 05:48

Building better measures of role ambiguity and role conflict:

ABSTRACT ARTICLE HISTORY

CONTACT Nathan A. Bowling nathan.bowling@wright.edu

Criticisms of the Rizzo et al. (1970) role stressor scales

Item generation and initial item culling

Study 1: item analysis of new role stressor items

Study 1: results and discussion

Table 1. Item analysis of new role stressor scales (Study 1).

Study 2: the substantive validity of the role stressor scales

Study 2: results and discussion

Study 3: the test–retest reliability of new role stressor scales

Study 3: results and discussion

yielded an acceptable level of test–retest reliability.

Study 4: examination of the factor structure of new role stressor scales

Study 4: results and discussion

A nomological network for role stressors

Study 5: results and discussion

Hypothesised to be related to all role stressors

WORK & STRESS

scale was significantly related to self-monitoring (r = .24, p < .01).

2. New role ambiguity scale .34** .04** .06**

Potential advantages of the new role stressor scales

View publication stats

You might also like

2. New role ambiguity scale .34 .04 .06**