REACTIONS OF LEADERS TO

360-DECREE FEEDBACK FROM

SUBORDINATES
AND PEERS

Carolyn L. Facteau*
Center for Business and fconornic Development,
Auburn University

Jeffre

D. Facteau

Au l urn University

Lynn C. Schoel
Ohio Casualty Group

Joyce E. A. Russell
The University

of Tennessee

Mark L. Poteet
City of Clear-water

The current study examined factors related to leaders reactions to 360-degree feedback. The total sample
consisted of 220 supervising managers from a large public utility. We collected data in three waves which
involved pretest, performance rating, and posttest surveys (49 leaders had valid data from all three waves
of measurement). We hypothesized that overall ratings, organizational support, and perceived rater ability
would be positively related to four reaction criteria (acceptance and perceived usefulness of subordinate
feedback and peer feedback). Results showed that overall ratings were related to acceptance of peer and
subordinate feedback, but were less consistently related to perceptions of feedback usefulness. For perceived usefulness of subordinate feedback, organizational support accounted for unique variance beyond
overall ratings, and perceived rater ability was marginally significant. None of the predictions for perceived usefulness of peer feedback were significant. The authors discuss limitations, implications, and
suggestions for future research.

* Direct all correspondence

to: Carolyn L. Facteau, Center for Business
South Court Street, Suite 110, Montgomery, AL 36104.
Leadership Quarterly, 9(4), 427-448.
Copyright 0 1998 by JAI Press Inc.
All rights of reproduction in any form reserved.
ISSN: 1048-9843

and Economic

Development,

600

428

LEADERSHIP

QUARTERLY

Vol. 9 No. 4 1998

Traditionally, performance appraisal in organizations has consisted primarily of supervisor evaluations of subordinate performance. More recently, however, a number of organizations have begun to recognize the potential benefits of gathering performance feedback
from sources in addition to the supervisor such as peers and subordinates. These benefits
may include: better performance information (Mohrman, Resnick-West, & Lawler, 1989)
more reliable ratings than those from a single supervisor (c.f. Wohlers & London, 1989)
support for high involvement styles of management (Budman & Rice, 1994; McGarvey &
Smith, 1993), and improved leader (i.e., ratee) performance after receiving feedback
(Atwater, Roush, & Fischthal, 1995; Daw & Gage, 1967; Smither, London, Vasilopoulos,
Reilly, Millsap, & Salvemini, 1995).
Perhaps because of these potential benefits, with increasing frequency, organizations are
implementing systems where alternative raters, in addition to supervisors, are able to provide feedback on a target managers or leaders performance (London & Beatty, 1993).
Commonly called 360-degree feedback, these systems may include feedback from peers.
subordinates, self, supervisors, higher level managers, or customers, and often contain
feedback from many of these sources. As evidence of the increasing popularity of 360degree feedback systems, the number of off-the-shelf 360 instruments being sold has
nearly quadrupled in the past decade (VanVelsor & Wall, 1992). Further, in 1992 alone,
companies spent an estimated 152 million dollars on 360-degree feedback system development (Romano, 1994).
While 360-degree feedback systems are growing in popularity, research on their effectiveness is relatively limited. This has led researchers to conclude that, it is clear that this
is an area in which practice is well ahead of theory and empirical research (London &
Smither, 1995, p. 809). Therefore, the current study attempted to further empirical understanding of 360-degree feedback systems by exploring factors which might be related to
leaders (i.e., managers supervising a specific organizational unit) reactions to 360-degree
feedback.
Much of the existing research on 360-degree feedback has explored psychometric properties of ratings from various sources. The results of psychometric studies provide evidence of moderate convergent, discriminant, and predictive validity of ratings from various
sources (c.f. Harris and Schaubroeck, 1988; London & Wohlers, 1991; McEvoy & Beatty,
1989; Tsui & Ohlott, 1988; Wohlers & London, 1989). These studies also have found that
a number of factors such as dimension rated, sex of rater, and size of work group may
impact agreement between ratings from different sources. Further, a proposed model of
self appraisals suggests that agreement between self and other ratings may be influenced by
individual differences such as achievement status and locus of control (Atwater & Yammarino, 1996). Finally, Antonioni (1994) found that characteristics of the rating system,
that is, whether subordinates have to sign the rating form, also may have an impact on the
level of ratings made by subordinates.
In addition to psychometric studies, some research has investigated various outcomes of
360-degree feedback systems such as changes in leaders performance and leaders affective reactions. Research exploring outcomes of 360-degree feedback on leader performance has begun to show support for the positive relationship
between 360-degree
feedback and performance improvement
(Atwater et al., 1995; Daw and Gage, 1967;
Hegarty 1974; Smither et al., 1995). However, Hazucha, Hezlett, and Schneider (1993)

Reactions to 360 Feedback

429

found that leaders performance improvement following feedback was dependent on a

number of factors such as obtaining input on development plans from coworkers.
A second line of research concerning outcomes of 360-degree feedback has explored
leaders affective reactions to the feedback or feedback system. Much of this research has
focused on characteristics of the feedback system or process which may be related to leaders affective reactions. For example, McEvoy (1990) found that leaders reacted more
favorably to evaluations from subordinates if those evaluations were used for developmental and not administrative purposes. Further, research has demonstrated that leaders may
prefer that raters sign their rating forms, while raters (in this case, subordinates) prefer to
remain anonymous (Antonioni, 1994). Bernardin, Dahmus, and Redmon (1993) found that
leaders had a more favorable opinion of a subordinate appraisal system when they received
both supervisor and subordinate feedback rather than only subordinate feedback. Other
research found that leaders have more positive reactions to subordinate appraisal systems
when they receive both individual and normative (e.g., group level) feedback rather than
normative feedback alone (Smither, Wohlers, & London, 1995). Barclay and Harland
( 1995) found that rater competence influenced perceptions of fairness of a peer rating system. Finally, Albright and Levy (1995) found that source credibility and message had an
interactive effect on reactions to feedback from multiple raters. Specifically, this study
found that while feedback from credible raters was generally viewed more positively, the
influence of source credibility was diminished in certain rating discrepancy conditions
(i.e., when a less credible source gave feedback that was more positive than self ratings and
when a more credible source gave feedback which was more negative than self ratings).
However, while 360 evaluations are becoming increasingly
popular and research has
begun to investigate leaders reactions to 360-degree feedback, the research exploring factors associated with leaders affective reactions to 360-degree feedback has focused primarily on the effects of various feedback system components. Therefore, it is important to
further explore factors outside of the design components of the 360-degree feedback system which may also contribute to leaders affective reactions to 360-degree feedback such
as perceptions of rater ability and feedback climate. Understanding these factors can help
identify variables which, if attended to, may ultimately make 360-degree feedback processes more effective.
Thus, there is clearly a need for additional research about 360-degree feedback systems
in organizations that will contribute to the successful development and implementation
of
1993; London & Beatty, 1993; London &
360-degree feedback systems (Dunnette,
Wohlers, 1991). Further, it is important to investigate multiple criteria for determining
whether these systems are effective, including the factors associated with leaders reactions
to the feedback, which may ultimately influence overall system effectiveness (London &
Beatty, 1993). Accordingly, this study investigated leaders reactions to 360-degree feedback from subordinates and peers (i.e., feedback acceptance and perceived feedback usefulness), and determined whether the feedback message (i.e., overall ratings from peers and
subordinates) and leaders perceptions of the feedback environment were related to these
feedback reactions.
Understanding leaders reactions to 360-degree feedback is important because feedback
research and theory suggest that how leaders react to feedback is a critical determinant of
whether or not they will take action to improve their performance (Bannister, 1986; Ilgen,
Fisher, & Taylor, 1979). The relationship between reactions to feedback and leaders use

430

LEADERSHIP

QUARTERLY

Vol. 9 No. 4 1998

of the feedback for development may be particularly salient in a 360-degree feedback process because, in the majority of organizations, these systems are used solely for development (Romano, 1994) and it is therefore likely that specific results may only be shared
with ratees themselves. Thus, because 360 feedback is most often used for development,
and access to individual results may be limited to ratees themselves, organizations may
have limited ability to monitor or enforce the use of this feedback for development. Specifically, if 360-degree feedback results are not tied to administrative decisions such as pay
and promotions, leaders who receive this feedback may perceive limited accountability for
using the results. Given that reactions to feedback may be precursors to using the feedback,
understanding
leaders reactions may enable organizations to design systems which ultimately encourage these leaders to take positive actions to improve. Further, understanding
reactions may allow organizations to encourage feedback usage without having to set up
formal organizational
mechanisms such as extrinsic rewards or punishments to monitor
and ensure leaders use their feedback. Finally, a review of the state of performance
appraisal research concluded that the study of reactions to appraisals as well as how context and other variables influence these reactions is an area that has been neglected and
deserves increased research attention (Bretz, Milkovich, & Read, 1992).
leaders

Reactions to Feedback

There were two dependent variables of interest in the current study. The first was leaders perceptions that 360-degree feedback from peers and subordinates is useful for their
own individual development. For example, leaders who perceive that feedback is useful
see it as valuable for developing their leadership skills and providing them with information about their strengths and weaknesses that they would not have had otherwise. The second dependent variable of interest was leaders acceptance of feedback from peers and
subordinates. Acceptance of feedback is defined as the belief that feedback is an accurate
representation of job performance (Ilgen et al., 1979). In essence it reflects the extent to
which leaders agree with the feedback they receive from peers and subordinates. Leaders
perceptions about the usefulness of feedback for their development and their perceptions
that the feedback they receive is accurate are believed to be important precursors to their
using this feedback to improve performance. Ultimately, a 360 system will have little value
if leaders do not use the feedback they receive as a basis for developing their leadership
skills.
In the current study, feedback acceptance and usefulness were treated separately because
we believed that these two dependent measures were likely to represent related, but distinct
constructs. In other words, we believed that in many cases, managers perceptions that
feedback is accurate would not necessarily be the same as their perceptions that feedback
is useful for individual development. For example, it is possible that some managers may
not view subordinate ratings as an accurate representation of their true performance, but
may believe that those ratings do reflect subordinates actual perceptions
of their pegormance. In this scenario, while the manager does not view the ratings as accurate (i.e., he/
she would rate feedback accuracy as low), he/she could recognize that the ratings are still
valuable for his/her development as a leader because the information would give the manager the opportunity to change inaccurate perceptions in the future by improving communication, managing his/her impression, etc. To the extent that perceptions of feedback

Reactions to 360 Feedback

431

acceptance and usefulness are unique, it is possible that the strength of the relationships
between the predictors of interest and each of these criteria will vary, and therefore, we felt
it was important to use two separate dependent measures.
Factors Influencing

Leaders Reactions to 360-Degree

Feedback

To understand leaders reactions to 360-degree feedback, we explored the relationship of

three independent variables to the two dependent variables of interest. To date, little theoretical work has been published on the 360-degree feedback process. Further, because past
research in reactions to 360-degree feedback has been limited primarily to system components, there was little guidance in the 360-degree feedback literature for selecting specific
predictors of leaders reactions to the feedback. Therefore, we selected two reaction measures and used existing theory and research in other areas to select potential predictors and
to provide rationale for the hypothesized relationships. The chosen predictors and specific
hypotheses are discussed below.
Perceptions

of the Feedback Environment

We predicted that two variables relevant to leaders perceptions of the feedback environment would be related to leaders reactions to feedback. The first feedback environment
variable, perceived organizational support, is the extent to which leaders perceive they are
valued and cared about by the organization. Leaders with higher perceptions of organizational support are more likely to believe the organization shows concern for its employees
well being and considers its employees best interests when making decisions or taking
action. Previous studies have shown perceived organizational
support to be related to
important outcomes such as job performance (Eisenberger, Fasolo, & David-LaMastro,
1990; Orpen, 1994), job attendance (Eisenberger et al., 1990), and organizational citizenship behaviors (Shore & Wayne, 1993). In addition to behavioral outcomes, research has
demonstrated a relationship between perceived organizational support and job-related attitudes such as satisfaction with and involvement in the organization (Shore & Tetrick,
1991; Eisenberger, Fasolop, & Davis-Lamastro,
1990). Thus, there is evidence to suggest
that perceptions of organizational support can be related to other attitudinal variables.
In the current study, we proposed that perceived organizational support would be related
to leaders acceptance of feedback and their belief that 360-degree feedback is useful. This
is based on the premise that leaders who believe the organization shows concern for them
and their development (high perceived organizational
support) will be more likely to
believe the organization has positive intentions and motivations toward them and will
therefore be more likely to trust the intentions of an organizationally
sponsored 360-degree
feedback system. Thus, leaders who have higher perceptions of organizational support may
be more likely to view the feedback as accurate because they are more likely to trust the
intentions and therefore credibility of the process. In addition, if leaders perceive that the
organization is concerned about their development and is taking into account their best
interests (high perceived organizational support), they may be more likely to see the 360degree feedback as useful because they believe the organization will provide them with
resources to support their attempts to use the feedback for development. Finally, the fact
that perceived organizational support for employees has been shown to be related to job
performance suggests leaders who believe the organization supports them may be more

432

LEADERSHIP

QUARTERLY

Vol. 9 No. 4

1998

motivated to use the feedback, may see more opportunities for using the feedback, and
therefore may be more likely to believe the feedback will be useful for their personal development.
Hypothesis la. Leaders perceptions of organizational support will be positively
related to their acceptance of feedback from peers and subordinates.
Hypothesis lb. Leaders perceptions of organizational
support will be positively
related to their beliefs that feedback from peers and subordinates is useful.
The second feedback environment variable was perceived rater ability, or leaders perceptions that peers and subordinates have the requisite skills and abilities necessary to
accurately rate their performance. Previous research has provided evidence that the credibility of the source can have a strong influence on peoples reactions to feedback (c.f. Bannister, 1986; Halperin, Snyder, Shenkel, & Houston, 1976). In fact, Ilgen et al. (1979)
contend that source credibility, which is composed of both rater ability and motivation to
rate accurately, is a key determinant in an individuals acceptance of feedback. Further,
Bannister (1986) found that subjects who received feedback from a more credible source
saw the feedback as more accurate and insightful and were more satisfied with the feedback.
With respect to 360-degree feedback in particular, Albright and Levy (1995) showed in
a laboratory study that subjects rated feedback more favorably when it came from a high
credibility source. However, the dependent measure used in Albright and Levys (1995)
study was a four item measure which asked about subjects perceptions of feedback accuracy, their satisfaction with the feedback, their acceptance of the feedback, and their perceptions of the usefulness of feedback for future performance.
Further, this was a
laboratory study which examined only peer and expert rater sources. Similarly, Barclay
and Harland (1995) found that subjects were more satisfied with a rating process and evaluated the process as more fair when ratings were made by an evaluator with more expertise, or competence. However, their study did not involve an actual peer rating system, but
rather asked subjects to react to hypothetical rating scenarios. Further, the dependent measures used in their study were perceived fairness and satisfaction with the hypothetical rating process rather than acceptance and perceived usefulness of actual subordinate and peer
feedback. Thus, it remains to be seen how the credibility of peer and subordinate raters in
a 360-degree feedback system may be related to specific facets of feedback reactions such
as perceptions of feedback accuracy and usefulness in an organizational setting.
Based on previous research establishing source credibility as an important predictor of
feedback reactions, we predicted that leaders who believe that a rating source has the ability to rate their performance accurately would be more likely to accept the feedback and
find the feedback useful. Specifically, we believed that to the extent leaders felt that peers
and subordinates had the requisite skills and abilities necessary to rate performance accurately, they would be more likely to perceive that the feedback from these sources was both
accurate and useful for their development.
Hypothesis 2a. Leaders perceptions of rater ability will be positively
their acceptance of feedback from peers and subordinates.

related to

Reactions

to 360 Feedback

433

Hypothesis 2h. Leaders perceptions of rater ability will be positively

their beliefs that feedback from peers and subordinates is useful.
Overall

related to

Rating

In the current study, overall rating was operationalized as the average of each sources
(subordinates and peers) ratings across items on the 360 rating instrument. A great deal of
feedback research has demonstrated that the sign of feedback (i.e., favorability) is one of
the most important determinants of reactions to feedback (c.f. Ilgen et al., 1979). In fact,
research on traditional performance appraisal systems has shown that the favorability of
the rating received strongly influences a host of reactions that ratees have to the rating process, including satisfaction, perceptions of fairness and accuracy, and acceptance (Dipboye
& depontbriand,
1981; Pearce & Porter, 1986). Podsakof and Farh (1989) concluded that
people who receive positive feedback are more satisfied and see the feedback as more credible. Further, Cook (1968) found that subjects level of aspiration increased following feedback indicating success. Halperin, Snyder, Shenkel, and Houston (1976) demonstrated that
subjects were more likely to believe that positive personality feedback was accurate.
Previous research has operationalized
sign of feedback as an average overall performance rating (c.f. Hazucha et al., 1993). Given that research suggests people tend to have
more positive reactions when feedback is positive, it is likely that the lower a leaders ratings in a 360-degree feedback system, the more negatively he/she will respond. Further, in
an organizational
performance
appraisal context, previous research has shown that
employees tend to interpret even satisfactory feedback as negative (Pearce & Porter, 1986).
This finding suggests that leaders may be sensitive to, and react negatively to, ratings that
are less than exceptional. Therefore, ratings should be taken into consideration in understanding leaders reactions to 360-degree feedback.
Hypothesis 3a. Overall ratings will be positively
feedback from peers and subordinates.

related to leaders acceptance

of

Hypothesis 3b. Overall ratings will be positively related to leaders beliefs that
feedback from peers and subordinates is useful.
Given the amount of research establishing the relationship between sign of feedback and
feedback reactions, it was also important to determine whether additional predictors
accounted for variance in feedback reactions, beyond overall ratings. Therefore, in addition
to the above hypotheses, this study assessed whether organizational support and rater ability accounted for variance in the dependent variables over and above those that were
accounted for by overall ratings.
In summary, the current study explored how the 360-degree feedback message and environment variables are related to leaders reactions to feedback from peers and subordinates. The study contributes to existing research by exploring predictors of reactions to
feedback that have not received significant research attention in this context. Further, the
study attempts to identify factors, in addition to overall ratings, which account for unique
variance in leaders reactions to 360-degree feedback. Finally, the study explores reactions
to specific rating sources of 360-degree feedback information, rather than to 360-degree
feedback in general.

434

LEADERSHIP

QUARTERLY

Vol.

9 No. 4 1998

This research is important because leaders reactions can have a significant impact on
their eventual response to 360 feedback. Because 360-degree feedback systems often are
used for development, organizations may not be able to enforce leaders use of the feedback. Therefore, it is critical to understand what factors may increase the likelihood that
leaders will accept feedback and view it as useful. If leaders accept and value the information, they should then be more likely to use the information for development, thus achieving a primary goal of 360-degree feedback systems.

METHOD
Participants

Potential study participants were 220 supervising managers at a large southeastern utility. The participants represented a variety of organizational
functions (e.g., engineering,
administrative, corporate, power generation, etc.) and levels (e.g., senior vice president to
first line supervisor). The sample was 70% male and 82% Caucasian. All participants had
at least a bachelors degree and their average age was 45. Further, these leaders had an
average of 15 subordinate raters, and all of them nominated from four to eight peer raters.
A total of 49 leaders had complete data from all phases of data collection. This reduced
sample (i.e., the sample of leaders with valid data on all variables) was 80% male and 84%
Caucasian with an average age 44. Further, as in the total sample, these leaders had an average of 15 potential subordinate raters and all of these leaders nominated between four and
eight peer raters. A comparison of the reduced sample of 49 leaders (e.g., those without
missing values on variables of interest in this study) to the sample of all potential participants who were eliminated from the reduced sample (e.g., those with one or more missing
values) revealed no significant differences on any of these demographic variables. Therefore, the reduced sample appears characteristic of the entire sample in terms of race, sex,
age, and number of subordinate and peer raters.
Procedure

Data for the current study were collected in three waves of surveys administered during
a pilot program for implementation
of a 360-degree feedback system. Consistent with
many 360-degree feedback processes, the stated purpose of this 360 system was for leadership development. That is, there were no formal individual rewards or outcomes (e.g.,
promotions) associated with the results of the 360-degree feedback evaluation. Further, in
addition to completing self ratings, all leaders participating in the 360-degree feedback system received ratings from their peers and subordinates and provided ratings to their peers
and/or supervisors. Subordinates rated only one leader.
The first survey, referred to here as the pretest, was administered to all participating leaders (ratees) prior to their participation in the 360-degree feedback rating process. The pretest survey asked leaders for their specific attitudes and expectations about aspects of the
360-degree feedback system as well as their general attitudes about their jobs and the organization. The questionnaire
was distributed either by inter-office mail or when leaders
arrived at their 360-degree feedback training session. Participants completed the pretest
prior to the orientation training. The orientation training provided leaders and their peer

Reactions to 360 Feedback

435

and subordinate raters with an overview of important aspects of the 360 system such as
anonymity of ratings, dimensions rated, and process administration. All participants were
asked to include the last four digits of their social security numbers on the pretest to enable
us to match their responses to subsequent survey responses. However, no names were associated with these numbers, and participants were assured that their individual responses
would remain confidential. Pretest survey completion was not mandatory, and the response
rate for the pretest survey was 60%.
The second wave of data collection occurred after orientation training and involved gathering the 360-degree feedback performance ratings from peers and subordinates. Specifically, a leaders subordinates and peers completed performance ratings on a 27-item survey
that covered six performance dimensions. These performance dimensions were identified
by the organization as competencies that were critical for successful leadership in the organization. All raters evaluated leaders on items that measured the following performance
dimensions: teamwork, diversity, continuous improvement,
innovation, leadership, and
credibility/integrity.
In addition, subordinates (but not peers) rated another five items that
assessed their leaders performance on coaching and developing others. Rating surveys
were preceded with a numerical identifier unique to each leader (i.e., ratee) so that subordinate and peer ratings could be associated with the target leader. However, individual raters who completed a survey were anonymous and raters did not sign rating forms. All
completed rating forms were sent directly to an external vendor for data processing.
Employees were assured that their individual responses could not be identified, and that
leaders would only receive feedback if at least three raters responded to the survey. This
practice is common among organizations that have implemented 360 rating systems (London & Smither, 1995). The response rate for the performance rating data (both subordinate
and peer surveys) was 78%.
Approximately three to eight weeks after ratings were complete, leaders received feedback reports in a workshop session where a trained facilitator reviewed the contents of the
report, how to interpret the results, and how to use the feedback for improvement. In some
cases, leaders who could not attend a workshop were given a one-on-one session with a
trained facilitator to explain their report. The information covered in the one-on-one session was identical to that covered in the workshop.
The third wave of data collection occurred immediately following the distribution and
discussion of feedback reports. A posttest survey was sent to each leader on the day after
he/she received a 360 feedback report. This survey asked leaders about their affective reactions to the feedback they received from each of the different rating sources. Surveys were
assigned a random numerical identifier to allow us to match pretest, performance rating,
and posttest survey data. Subjects were ensured of their confidentiality and that their individual responses would not be shared with anyone. After receiving feedback reports, a
number of events occurred in the organization (e.g., restructuring, downsizing, etc.) which
resulted in 96 leaders withdrawing from the study immediately after receiving feedback. A
total of 124 posttest surveys were distributed to all leaders who remained in the study at
this point and 94 were returned, yielding a 76% response rate. The remaining leaders in the
study were not directly effected by the organizational changes which impacted those who
withdrew, because only those areas in the organization that remained relatively stable continued to participate in the study. Therefore, there is no reason to believe that the restructuring and downsizing directly influenced the sample that remained, or that any portion of

LEADERSHIP

436

QUARTERLY

Vol.

9 No.

4 1998

this remaining sample was effected in any unique way. Posttest surveys were sent to all
leaders whose organizations remained in the study, regardless of whether these leaders had
returned pretest surveys. Ultimately, as indicated previously, a total of 49 leaders (22% of
the original sample) had complete data from all waves of data collection and the demographic characteristics of this sample did not differ significantly from those of the original
sample.
Measures

To test the hypotheses, measures of five constructs were gathered. The pretest survey
measured leaders opinions about organizational support for employees and rater ability.
Overall subordinate and overalf peer ratings were computed from responses obtained from
these sources on the 360 rating survey. The posttest survey assessed leaders reactions to
feedback from each rating source (i.e., peers and subordinates). Specifically, the survey
measured acceptance offeedback and tdsefulrress qffeedback. With the exception of over&l
ratings, all items were measured with a 5-point Likert-type scale (I = Strongly Disagree
and 5 = Strongly Agree). Items on the 360 rating instrument were measured on a 5point
Likert-type scale (1 = Not at all Effective and 5 = Extremely Effective). Descriptions
of each of these measures are provided below.
Organizational

Support

This four-item scale was adapted from Eisenberger, Huntington, Hutchinson, and Sowa
(1986) and measured the extent to which leaders perceived that the organization demonstrates concern for employees. Leaders with greater perceptions of organizational support
are more likely to believe that the organization cares about employees and has their best
interests in mind when taking action. The internal consistency reliability estimate for this
scale in the current study was .87. A sample item is, This organization is willing to extend
itself in order to help me perform to the best of my ability.
Rater AbXfy

This is defined as the extent to which leaders believe that raters from a particular source
have the ability to rate their performance. While perceptions of rater ability may be influenced by a host of factors (e.g., amount of rater training, perceived opportunity to observe
performance), the measure used in this study was a direct assessment of leaders perceptions that their peers and subordinates had the requisite skills to provide accurate ratings.
In the current study, this was measured using a two-item scale for both peers (a = .85) and
subordinates (a = .86). A sample item is, My peers have the skills and abilities necessary
to rate my performance accurately.
Overall

Ratings

For each leader in the sample, an overall peer and an overall subordinate rating were
computed. For the overall subordinate rating, this first involved computing the mean of the
32 items included on the subordinate rating form for each subordinate rater. An overall
subordinate rating was then computed for each leader by averaging these scores across all
of the subordinate raters who evaluated them. Similarly, to compute the overall peer rating,
the mean of the 27 items contained on the peer rating form was computed for each peer

Reactions

to 360 Feedback

437

rater. The average of these scores across all of the peers who evaluated a particular leader
was then used to represent a leaders overall peer rating.
Although this practice for computing overall ratings is consistent with other research in
the area of 360-degree feedback (c.f. Hazucha et al., 1993), several analyses were conducted to first determine the appropriateness of aggregating the items on the rating instruments into composite variables, and the appropriateness of aggregating across raters within
each of the two rating sources to derive overall subordinate and overall peer ratings for
each leader.
To address the former issue, separate exploratory principal-components
analyses of the
32 items contained on the subordinate rating form and the 27 items contained on the peer
rating form were conducted. The rating data used in these analyses came from all the subordinate and peer raters who provided evaluations in the second wave of data collection
and who had complete data on all of the rating items they were asked to complete (N = 9 19
and N = 423 for subordinate and peer ratings, respectively). In both analyses, two factors
were extracted. For subordinate ratings, the first and second factors accounted for 67% and
4% of the variation in ratings, respectively. For peer ratings, the first and second factors
accounted for 53% and 9% of the variance, respectively. Thus, in both analyses, the second
factor accounted for relatively little additional variance beyond the first factor. Furthermore, in both analyses, all of the items had substantial positive loadings on the first unrotated factor, supporting the idea that a strong general factor existed in the data (Nunnally,
1978). Taken together, these results support using the average of the items contained on the
subordinate and peer rating forms as indices of raters overall evaluations of the leaders
they rated. Further bolstering this conclusion is the fact that the internal consistency reliability estimates for these indices were extremely high for both subordinates (a = .98) and
peers (a = .97).
The appropriateness of averaging individuals overall ratings within each rating source
(i.e., subordinates and peers) for each leader requires that substantial within-source agreement in these ratings exists. James, Demaree, and Wolfs (1984) index of within-group
interrater agreement (r,& is appropriate for examining the suitability of aggregating individual-level data to the group-level of analysis (George & James, 1990). As a result, for
each leader, two rwg values were computed. One indexed the level of agreement in overall
ratings among their subordinate raters while the other indexed the level of agreement in
overall ratings among their peer raters. Both of these values tended to be quite high. For
subordinate rating agreement, the average rwg value was .95, with 80% of the values
exceeding .96 (based on the 19 1 leaders who received enough subordinate ratings for these
values to be calculated). For peer rating agreement, these values were .98 and .97, respectively (based on the 1.52 leaders who received enough peer ratings for these values to be
computed). Given these high levels of agreement, it seemed justified to compute an average overall subordinate rating and an average overall peer rating for each leader.
Acceptance

of Feedback

This six-item scale was consistent with Ilgen et al.s (1979) conception of feedback
acceptance as the extent to which leaders believed that the feedback they received was an
accurate representation of their performance. Two items for this scale were adapted from
Northcraft and Earleys (1989) measure of feedback credibility while the remaining items
were developed for this study. Sample items include, I agree with the feedback I received

.56
.40
.73
.88
.73

3.85
3.56

.82

.85

.82

.87

SD

3.64
3.65
2.63

3.50

3.46

3.62

3.62

Mean

.31*
.OS

m***
.26*
.23

.17

.14

.6S***

.35**
.27*

.38**
.I2
.36**

.54***

.I7

-.05
.I6

.I1
&$***
-.04

.67***

--

.08
.20

.05
.34**
.08

.24*
-.Ol

.43***
.27*

.I0
.13

.14

.20
.25*

Means, Standard Deviations, and Intercorrelations Among Study Variables

.42**

Responses to all items comprising each scale were made on a five-point Liken-type rating scale (ranging from I=Strongly Disagree to 5=Strongly
Agree) for all variables except overall ratings (where the scale ranged from i=TJot at ai1 Effective to S=Extremely Effective). Correlations reported
here were computed using pairwise deletion of missing data and, as a result, the sample sizes ranged from 55 to 151.
*p < .05,two-tailed probability
p < .Ol, two-talled probability
*I*p < ,001,two-tailed probability.

Acceptance of
Subordinate Feedback
Usefulness of
Subordinate Feedback
Acceptance of
Peer Feedback
Usefulness of
Peer Feedback
Overall
Subordinate Rating
Overall Peer Rating
Organizational Support
Subordinate
Rater Ability
Peer Rater Ability

Nom:

9.

6.
7.
8.

5.

4.

.3

2.

1.

Variable

Table 1.

439

Reactions to 360 Feedback

from this source in the 360-degree feedback system and The feedback from this source
in the 360-degree feedback system was accurate. Leaders were asked to rate each of the
six items with respect to subordinate feedback first, and then with respect to peer feedback.
Coefficient alpha was .97 for both versions of the scale.
Usefulness of Feedback

Usefulness of feedback was a four-item scale which measured the extent to which leaders believed that the feedback from a particular rating source (i.e., subordinates, peers) provided them with information that would be valuable for their development. The items on
this scale were developed for use in this study. Sample items include, The feedback from
this source provided me with valuable information that I would not have had otherwise
and The feedback from this source gave me a clear understanding
of my strengths and
areas for development. As with the acceptance of feedback scale, leaders first responded
to these items with respect to subordinate feedback, and then with regard to peer feedback.
Coefficient alpha for both versions of the scale was .90.
A principal components analysis with varimax rotation of the items comprising the
acceptance and usefulness of feedback scales was conducted. When these items were completed by leaders (N = 74) with respect to subordinate feedback, two factors were extracted
which accounted for 83% of the variance. All of the items on the acceptance of feedback
scale had substantial loadings (i.e., greater than .80) on the first factor, while all of the
items on the usefulness of feedback scale had substantial loadings on the second factor. No
item on either factor had a cross-loading on the other factor which exceeded .41. When the
same items were completed by leaders (N = 79) with respect to peer feedback, nearly identical results were obtained in terms of the number of factors extracted, variance accounted
for, and the pattern and magnitude of factor loadings. Thus, the results of these two analyses support treating the acceptance and usefulness of feedback scales as measures of distinct constructs.

RESULTS
Table 1 presents the means, standard deviations, and intercorrelations
for all of the variables of interest in the study. As can be seen in the table, leaders had moderately favorable
reactions, in terms of acceptance and usefulness, to both the subordinate and peer feedback
they received from the 360-degree feedback system. On average, leaders received moderately favorable ratings from both their subordinates and their peers (MS = 3.64 and 3.65,
respectively). They also tended to agree that both subordinates and peers had the ability to
evaluate their performance, although their confidence in subordinate raters (M = 3.85) was
slightly higher than their confidence in peer raters (M = 3.56).
The correlations presented in Table 1 indicate that the acceptance and usefulness of feedback dependent measures were strongly correlated. The correlation between acceptance
and usefulness of subordinate feedback was .65 (p < .OOl) and the corresponding correlation for peer feedback was .67 (p < .OOl). Thus, the reaction to subordinate feedback measures shared 42% of their variation while the reaction to peer feedback measures shared
45% of their variation. While there was a large degree of overlap between the acceptance
and usefulness measures, these results demonstrate that these measures were not completely redundant. This, in addition to the results of the principal components analyses dis-

440

LEADERSHIP

QUARTERLY

Vol. 9 No. 4

1998

cussed earlier, supported treating acceptance and usefulness as separate dependent

measures in the analyses that follow.
A noteworthy trend in the correlations that appear in Table 1 is that reactions to subordinate feedback were significantly correlated with a greater number of variables than were
reactions to peer feedback. Furthermore, while acceptance of subordinate and acceptance
peer feedback were not significantly related (r = .14, ns), the correlation between usefulness of subordinate feedback and usefulness of peer feedback was ~54. This indicates that
leaders tended to perceive the usefulness of feedback from both of these sources similarly.
The hypotheses of interest in the study were tested using hierarchical multiple regression
analysis (Cohen & Cohen, 1983). A separate analysis was conducted for each of the four
dependent variables: acceptance of subordinate feedback, acceptance of peer feedback,
usefulness of subordinate feedback, and usefulness of peer feedback. For each dependent
variable, overall ratings were entered into the regression equation on the first step. We
should note that the standardized regression coefficients between overall ratings and the
dependent variables obtained in the first step of these analyses differ somewhat from the
corresponding zero-order correlations reported in Table 1. This is because the correlations
were computed using pairwise deletion of missing values, and hence are based upon somewhat larger samples than the regression analyses, which were conducted using listwise
deletion of missing data. Where the correlational results differ from the regression results
at this step, we rely upon the regression results to draw conclusions about the relationship
between overall ratings and a dependent variable because they served as the primary analyses for testing the hypotheses that were of interest in the study.
The second step of the hierarchical regression analyses involved entering the remaining
variables which were hypothesized to predict feedback reactions into the equation. This
hierarchical strategy, as opposed to one where all the predictors were entered into the
regression equation simultaneously,
was chosen because previous research has clearly
demonstrated that the favorableness of feedback is a strong predictor of how individuals
respond to feedback (e.g., Dipboye & de Pontbriand, 1981; Ilgen et al., 1979; Pearce &
Porter, 1986; Podsakoff & Farh, 1989). Because of this, it seemed important in the current
study to assess the contribution of the independent variables of interest (organizational sup-

Table 2. Results of Hierarchical Regression Analyses

Acceptance of Feedback Dependent Measures
Acceptance
Subordinate
step

Vuriah1e.s

Rating
Organizational Support
Rater Ability
AR2
R2
AF
F
Overall

ivorc 7:

Sample sizes for analyses

respectively.
***p

< .OOl

of

Acceptm-e

Feedback

.69**

Step 2

.64***

Peer
step

.56***

.I5
.03
.52

.3l

of acceptance
of subordinate

l8.07***

Step 2

.56***
.03
.Ol
.32

56

I .72
49.43***

of

Feedbark

-.I2

.09

.4x

for

24.19***

8.30***

feedback and acceptance of peer feedback were 55 and 56,

Reactions

441

to 360 Feedback

port, rater ability) to the dependent variables beyond what could be accounted for by the
overall level of ratings received. The increment in R2 at step two in these analyses allowed
for a direct assessment of the amount of variance in a dependent variable accounted for by
the independent variables, controlling for the effects of overall ratings.
Table 2 presents the results of the hierarchical regression analyses for acceptance of subordinate feedback and acceptance of peer feedback. It was predicted that organizational
support and rater ability would be significantly related to subordinate feedback acceptance,
controlling for the effects of overall subordinate ratings. The results reported in Table 2 do
not support this prediction. In the first step of the analysis, overall subordinate ratings
accounted for 48% of the variance in the dependent measure (F(t,,,) = 49.43, p < .OOl).
The standardized regression coefficient (b = .69, t = 7.03, p < ,001) indicated that, consistent with hypothesis 3a, acceptance of subordinate feedback increased with increases in the
favorableness of overall subordinate ratings. When organizational support and subordinate
rater ability were added in the second step of this analysis, the increment in R2 was small
(.03) and nonsignificant (AF(2,51) = 1.72, ns). Thus, organizational support and rater ability
did not predict acceptance of subordinate feedback beyond the prediction afforded by overall subordinate ratings.
Similar results were obtained for acceptance of peer feedback. As can be seen in Table
2, overall peer ratings accounted for a large and significant percentage of variance (3 1%)
in acceptance of peer feedback in the first step of the analysis (F(t,54) = 24.19, p < .OOl).
Consistent with hypothesis 3a, overall peer ratings were positively related to acceptance of
peer feedback (b = .56, t = 4.92, p < .OOl). However, in the second step of the analysis,
organizational support and peer rater ability did not account for significant additional variance in acceptance beyond the variance accounted for by overall peer ratings (AR2 ~01,
AFc2,52)= S6, ns). Taken together, the results for acceptance of subordinate feedback and
acceptance of peer feedback do not support hypotheses la and 2a, but they do support
hypothesis 3a.

Table 3. Results of Hierarchical Regression Analyses for

Usefulness of Feedback Dependent Measures
Usefulness
of

usejiibms oj
PeerFeedback

Subordinate Feedback
Variables

Step I

Step 2

Overall Rating
Organizational Support
Rater Ability
AR2
R2
AF
F

.29*

.I8
.33*
.24t
.I9
.27
6.59**
6.41***

Notes:

Sample

sires

respectively.
tp-C.06

*p < .os
**u < .Ol

for analyses

.09
4.9s*
of usefulness

of subordinate

feedback

Step I

Step 2

.22

.18
.05
.14
.02
.07
.62

.05
2.84
and usefulness

of peer

1.35
feedback

were

55 and 56,

442

LEADERSHIP

QUARTERLY

Vol. 9 No. 4 1998

In Table 3, the results of the hierarchical regression analyses for usefulness of subordinate and peer feedback are presented. It was predicted that organizational support and subordinate rater ability would predict leaders perceptions of the usefulness of subordinate
feedback. In step one of the analysis, overall subordinate ratings accounted for a significant
percentage (9%) of variance in usefulness of subordinate feedback (F(t,,,) = 4.98, p < .05).
As with acceptance of subordinate feedback, overall ratings were significantly and positively related to usefulness of subordinate feedback in this step of the analysis (b = .29, t =
2.23, p < .05), as was predicted in hypothesis 3b. The addition of organizational
support
and subordinate rater ability in the second step resulted in a significant increment in
explained variance (AR* = .19, AF (2,51) = 6.59, p < .Ol). Organizational support (b = .33, t
= 2.65, p < .05) was significantly related to leaders perceptions of the usefulness of subordinate feedback, while subordinate rater ability (b = .24, t = 1.96, p < .06) was marginally
related to these perceptions. Increases in these variables were associated with increases in
perceptions of feedback usefulness.
For leaders perceptions of the usefulness of peer feedback, overall peer ratings did not
account for significant variation in the first step of the analysis (R* = .05, Fc1,54j = 2.84, rzs),
despite the significant zero-order correlation (r = .34, p < .Ol) between the two variables.
Again, this discrepancy reflects a difference in the samples for the two sets of analyses (N
= 56 and N = 87 for the regression and correlational analyses, respectively) which resulted
from different methods of dealing with missing data (i.e., listwise vs. pairwise deletion in
the regression and correlational analyses, respectively). The addition of organizational support and peer rater ability in the second step did not result in a significant increment in the
variance accounted for in usefulness perceptions (AR* = .02, AF(,,5,) = .62, ns). Thus, none
of the variables of interest in this study predicted leaders perceptions of the usefulness of
peer feedback. Taken together with the results for leaders perceptions of the usefulness of
subordinate feedback, the results for organizational
support provide partial support for
hypothesis lb. However, hypothesis 2b was not supported in that rater ability was not significantly related to leaders perceptions of the usefulness of either subordinate or peer
feedback. Finally, because overall ratings predicted usefulness of subordinate but not peer
feedback, hypothesis 3b received only partial support.
DISCUSSION
In summary, overall ratings were significantly related to leaders acceptance of feedback
from subordinates and peers, and to leaders perceptions of the usefulness of subordinate
feedback. In general, overall ratings explained more variance in acceptance of feedback
than perceived usefulness of feedback. Further, for perceived usefulness of subordinate
feedback, after controlling for the effects of overall rating, perceived organizational
support accounted for unique variance and perceived rater ability was a marginally significant
predictor.
Consistent with previous research in the area of performance appraisal (c.f. Dipboye &
depontbriand,
1981; Pearce & Porter, 1986) the current study demonstrated that overall
ratings are an important variable in understanding
leaders acceptance of feedback from
peers and subordinates. In the current, study mean overall ratings were relatively high (e.g.,
mean ratings in the current study were 3.64 and 3.65 for subordinate and peer ratings,
respectively). However, similar to findings by Pearce and Porter (1986) it appears that as

Reactions to 360 Feedback

443

overall ratings, or feedback, become less positive (e.g., decreases from a perfect 5 rating), leaders will be less likely to accept the feedback as accurate. Given that various rating
sources are likely to provide unique perspectives about performance (c.f. Wohlers and
London, 1989), it is likely that managers will be receiving less than perfect ratings in a 360
system. The current findings suggest that as the level of these ratings decreases, leaders
may try to discount the information from peers and subordinates rather than accept it as
accurate, which could negate some of the potential developmental benefits of 360-degree
feedback.
To help leaders react positively to peer and subordinate feedback when ratings are less
than perfect, organizations may need to invest a significant amount of time and effort with
leaders prior to receiving feedback to help prepare them for the eventuality that the information may differ from past feedback, and to build knowledge that will enhance the possibility that they will react positively despite what they may perceive to be negative
information. Further, it also may be wise for organizations to provide external accountability mechanisms to ensure that leaders reactions to this feedback do not prevent them from
taking action to improve. For example, organizations may want to establish formal mechanisms for writing development plans based on 360-degree feedback and schedule periodic
follow-up sessions between leaders and their immediate supervisors to ensure leaders are
acting on these development plans. Finally, it may be important to monitor leaders reactions to feedback over time to determine if leaders initial reactions change over time. It is
possible that when leaders have had time to think about and discuss their feedback with
others they may become more accepting of it.
While overall ratings were a key determinant of leaders acceptance of feedback, ratings
were not as critical in predicting the extent to which leaders perceived the feedback as useful for their development. Therefore, it is possible that although leaders may be less likely
to perceive lower overall ratings as accurate, they still may believe that this feedback is
useful for their development. Given this evidence that ratings may be less critical in predicting perceptions of feedback usefulness, future research could explore additional factors
which may be associated with this feedback reaction such as the amount of feedback leaders are already receiving from other sources (e.g., the task itself, the formal performance
appraisal process), tenure in a leadership role (e.g., Do leaders who are newer to their leadership role see more value in the information?), self confidence of the leader (e.g., Do leaders who have more self confidence find the feedback more useful, while leaders with less
self confidence see it as threatening?), and feedback seeking tendencies (e.g., Do leaders
who are high feedback seekers see this information as more valuable?).
A reviewer pointed out that, in addition to the level of ratings received from subordinates
and peers, leaders reactions to their feedback may be influenced by the extent to which the
feedback is congruent with their own evaluations of their performance. In a study which
utilized a larger sample of managers from the same organization studied here, Facteau,
Facteau, McGonigle,
and Fredholm (1997) examined the impact of the congruence
between leaders self-ratings and the ratings they received from their subordinates and
peers on their reactions to the feedback that they received from these sources. Using the
hierarchical procedures recommended by Edwards (1994) for testing congruence hypotheses, they found no evidence that self-other rating congruence influenced the extent to
which leaders accepted or perceived as useful the feedback that they received from their
subordinates and peers. Thus, self-other rating congruence does not appear to have influ-

444

LEADERSHIP

QUARTERLY

Vol. 9 No. 4 1998

enced the way leaders reacted to the feedback that they received from the 360-degree rating system. Of course, additional research should examine the impact of self-other rating
congruence on leaders reactions to feedback to determine if Facteau et al.s findings generalize to other organizations.
Organizational
support was positively related to perceived usefulness of subordinate
feedback, but was not significantly related to any other criterion measures. Thus, this study
provides evidence to suggest that leaders who believed the organization would support
their job-related efforts may have been more likely to believe that their subordinate feedback would be useful for development. It is probable that leaders who saw the organization
as supportive also believed that the resources would be available for their improvement,
and that they would receive the support or encouragement
necessary to improve. These
findings also suggest that organizational support may be more important in understanding
reactions to subordinate than peer feedback. It is possible that leaders saw the organization
as playing a larger role in enabling them to improve their supervisory capabilities such as
their relationships with subordinates, but a lesser role in their performance capabilities with
peers. Further, previous authors have suggested that it is common for leaders to have significant concerns about receiving feedback from subordinates in particular (e.g., fear subordinates who are pushed harder will rate lower; concern over subordinates knowledge of
performance; resistance to changes in traditional power hierarchies; cf. Bernardin, 1986;
Bemardin, Dahmus, & Redmon, 1993; Dombusch & Scott, 1975). Managers beliefs that
the organization supports employees may have helped to alleviate some of their concerns
about subordinate feedback and helped them to see this feedback as useful. It is possible
that leaders do not have similar concerns about receiving peer feedback, and therefore, the
influence of organizational support was not important in understanding
perceived usefulness of peer feedback.
Finally, perceived rater ability was only marginally positively related to perceived usefulness of subordinate @ < .06), and was not related to usefulness of peer feedback. The
fact that the relationship between perceived rater ability and perceived usefulness of feedback was marginally significant for subordinate, but not peer feedback, may once again be
a function of the unique nature of subordinate feedback in organizations. While inconclusive in the current study, it is possible that perceptions of rater ability helped leaders to
overcome their initial fears about the usefulness of subordinate ratings in the organization
such that leaders who saw their subordinates as having the skills to rate their performance
accurately also were able to see this feedback as useful. In the case of peer feedback, ability
may not have played a central role in perceptions of feedback usefulness because leaders
did not have the same fears or concerns about peer feedback at the outset. However, future
research should try to replicate this finding with a larger sample to determine whether the
marginal significance found in the current study was a function of sample size.
If future studies could demonstrate more conclusively
the relationship between perceived rater ability and usefulness of subordinate feedback, these findings would suggest
important implications for practice in organizations. Specifically, to ensure the buy-in and
participation of leaders in a 360-degree feedback system, organizations should ensure that
subordinates are thoroughly trained in the rating process and that leaders perceive that
training as enhancing subordinates rating skills rather than just informing subordinates
about the rating process. Further, organizations
may want to consider giving managers
input into the selection of those subordinate raters who they feel have the best opportunity

445

Reactions to 360 Feedback

to observe and accurately rate their performance, rather than allowing all subordinates the
opportunity to rate their manager. If given the opportunity to select a sample of subordinate
raters, organizations should find ways to enhance the possibility that leaders will choose
appropriate subordinate raters.
Overall, the differential relationships between the independent and dependent variables
suggests that feedback source may be important in understanding factors which are related
to responses to feedback. More specifically, the findings of the current study suggest that
leaders may have varying reactions to ratings from peers and subordinates, and different
factors may be associated with these reactions. Further, these different reactions are not
attributable simply to the differences in overall ratings provided to the leader from these
sources. Thus, in studying reactions to 360-degree feedback it may be important to differentiate leaders reactions by source of feedback. One practical implication of this finding
is that organizations that are trying to design interventions to encourage positive leader
reactions to 360-degree feedback may have to tailor the intervention for each source of
feedback rather than 360-degree feedback in general.
Limitations

and Conclusions

There are several limitations to the current study that should be discussed. First, because
a number of the constructs of interest were perceptual in nature, it was necessary to measure them using survey methodology. Therefore, one limitation is that all data were collected with a common survey method. Further, the two criterion variables were measured
on a single survey. Therefore, it is possible that common method variance artificially
inflated some of the relationships. However, the predictors and criteria were collected in
three different waves, which should help to control for any response biases, and the rating
data were taken from two different sources. Additionally, the correlations in the correlation
matrix ranged from -.05 to .67, suggesting that relationships were not due solely to common method variance.
A second limitation of the current study is the relatively small sample size. Because the
study required leaders to complete two surveys in addition to their participation in the 360degree feedback process, some attrition occurred. As a result, our sample probably did not
provide adequate statistical power for detecting weaker effects. Therefore, future research
should attempt to replicate our findings using larger samples to determine if the nonsignificant findings reported here are a function of low statistical power, as well as to assess the
stability of our results.
A third limitation of the current study is the high correlation between the two criterion
variables of interest, feedback acceptance, and perceived usefulness (.65 for subordinates
and .67 for peers). It could be argued that these variables are not uniquely important to
explore. However, factor analytic results supported the use of separate scales for these two
variables. Further, the fact that overall ratings accounted for far more variance in acceptance than perceived usefulness of feedback suggests that it is important to explore these
reactions to feedback separately. Future research should explore additional reactions to
feedback, such as intentions to use feedback.
A final limitation of the current study is the fact that data are not presented linking the
criteria to additional outcome measures such as leaders behavioral responses to the feedback. The focus of the current study was to explore only affective reactions to feedback.

446

LEADERSHIP

QUARTERLY

Vol.

9 No.

4 1998

Theory suggests that these reactions should ultimately be related to behavioral outcomes
(Ilgen, et al., 1979). However, the mechanisms through which this relationship occurs may
be complex and were beyond the focus of this study (e.g., behavioral intentions, desire to
respond, and external constraints may all be involved; Ilgen, et al., 1979). Therefore, future
research should explore how these affective reactions influence other 360-degree feedback
outcomes such as developmental
activities pursued, behavior change, and performance
improvement.
In conclusion, this study provides evidence that leaders reactions to 360-degree feedback may be determined by multiple factors. Further, the variables which are related to
reactions to peer and subordinate feedback may differ. In addition, the importance of a particular variable in leaders reactions may differ depending on the type of reaction measure
studied. Therefore, organizations who want to implement successful 360-degree feedback
systems will need to consider a host of variables that may contribute to a variety leaders
reactions.
Future research should continue to explore multiple criteria for 360-degree feedback system success, such as factors associated with leaders behavioral responses to the feedback
(e.g., discussion of feedback with raters, developmental
goal setting, and goal pursuit).
Additionally, research should investigate the relationship of the variables of interest in the
current study to other 360-degree feedback outcomes. By increasing our understanding of
how and why leaders respond to 360-degree feedback, we will be able to better design and
implement systems which ultimately achieve their intended objectives.

REFERENCES
Albright, M. D., & Levy, P. E. (1995). The effects of source credibility and performance rating discrepancy on reactions to multiple raters. Journal of Applied Social Psychology, 25, 577-600.
Antonioni, D. (1994). The effects of feedback accountability on upward appraisal ratings. Personnel
Psychology,

47, 349-356.

Atwater, L. E., & Yammarino, F. J. (1996, in press). Self-other rating agreement: A review and
model. In G. R. Ferris & K. M. Rowland (Eds.), Research in personnel and human resource
management. Greenwich, CT: JAI Press.
Atwater, L. E., Roush, P., & Fischthal, A. (1995). The influence of upward feedback on self- and follower ratings of leadership. Personnel Psychology, 48(l), 35-60.
Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 37(2), 122147.

Bannister, B. D. (1986). Performance outcome feedback and attributional feedback: Interactive

effects on recipient responses. Journal of Applied Psychology, 71(2), 203-210.
Barclay, J. H., & Harland, L. K. (1995). Peer performance appraisals: The impact of rater competence, rater location, and rating correctability on fairness perceptions. Group and Organization Management,

Bernardin,

20(l),

3940.

H. J. (1986). Subordinate

Human Resource

Management,

appraisal: A valuable source of information

about managers.

2.5(3), 421439.

Bernardin, H. J., Dahmus, S. A., & Redmon, G. (1993). Attitudes of first-line supervisors toward
subordinate appraisals. Human Resource Management, 32, 3 15-324.
Bretz, R. D., Milkovich, G. T., & Read, W. (1992). The current state of performance appraisal
research and practice: Concerns, directions, and implications. Journal ofManagement, 18(2),
321-352.

Budman, M., & Rice, B. (1994). The rating game. Across the Board, 31(2), 34-38.

447

Reactions to 360 Feedback

Cohen, J., & Cohen, P. (1983). Applied regression/correlation

analysis for the behavioral sciences.
Hillsdale, New Jersey: Lawrence Erlbaum.
Cook, D. M. (1968). The impact on managers of frequency of feedback. Academy of Management,
1 I, 263-277.

Daw, R. W., & Gage, N. L. (1967). Effect of feedback from teachers to principals. Journal of Educational Psychology, 58(3), 18 l-l 88.
Dipboye, R. L., & depontbriand,
R. (1981). Correlates of employee reactions to performance
appraisals and appraisal systems. Journal of Applied Psychology, 66(2), 248-25 1.
Dombusch, S. M., & Scott, W. R. (1975). Evaluntion and the exercise of authority. San Francisco:
Jossey-Bass.
Dunnette, M. D. (1993). My hammer or your hammer? Human Resource Munagement, 32(2&3),
373-384.

Edwards, J. R. (1994). The study of congruence in organizational behavior research: Critique and
proposed alternative. Organizational Behavior and Human Decision Processes, 58, 5 I-100.
Eisenberger, R., Fasolop, P., & Davis-Lamastro, V. (1990). Perceived organizational support and
employee diligence, commitment, and innovation. Journal of Applied Psychology, 7.5, 5 l-59.
Eisenberger, R., Huntington, R., Hutchinson, S., & Sowa, D. (1986). Perceived organizational support. Journal of Applied Psychology, 71(3), 50&507.
Facteau, J. D., Facteau, C. L., McGonigle, T. P., & Fredholm, R. L. (1997). Characteristics offeedback and managers reactions in multisource appraisal systems. Paper presented at the twelfth
annual conference of the Society for Industrial and Organziational Psychology, St. Louis, MO.
George, J. M., & James, L. R. (1990). Personality, affect, and behavior in groups revisited: Comment
on aggregation, levels of analysis, and a recent application of within and between analysis,
Journal

of Applied Psychology,

78,798-804.

Halperin, K., Snyder, C. R., Shenkel, R. J., & Houston, B. K. (1976). Effects of source status and
message favorability on acceptance of personality feedback. Journal of Applied Psychology,
61(l),

142-147.

Harris, M. M., & Schaubroeck, J. (1988). A meta-analysis of self-supervisor, self-peer, and peersupervisor ratings. Personnel Psychology, 41,43-62.
Hazucha, J. F., Hezlett, S. A., & Schneider, R. J. (1993). The impact of 360-degree feedback on management skills development. Human Resource Management, 32 (2&3), 325-35 1.
Hegarty, H. H. (1974). Using subordinate ratings to elicit behavioral changes in supervisors. Journal
of Applied Psychology,

59, 764766.

Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behavior
in organizations. Journal of Applied Psychology, 64(4), 349-37 1.
James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliability with
and without response bias. Journal of Applied Psychology, 69, 85-98.
Kotter, J. P. (1996). Leading change. Boston, MA: Harvard Business School Press.
London, M., & Beatty, R. W. (1993). 360-degree feedback as competitive advantage. Human
Resource

Management,

32(2&3),

353-372.

London, M., & Smither, J. (1995). Can multi-source feedback change perceptions of goal accomplishment, self-evaluations.
and performance-related
outcomes. Personnel Psychology, 48,
803-839.

London, M., & Wohlers, A. J. (1991). Agreement between subordinate and self-ratings in upward
feedback. Personnel Psychology, 44, 375-390.
Maurer. R. (1996). Beyond the wall of resistance: Unconventional strategies that build supportfor
change. Austin, TX: Bard Books.
McEvoy, G. M., & Beatty. R. W. (1989). Assessment centers and subordinate appraisals of managers: A seven year examination of predictive validity. Personnel Psychology, 42, 37-52.

448

LEADERSHIP

QUARTERLY

Vol. 9 No. 4 1998

McEvoy, G. M. (1990). Public sector managers reactions to appraisals by subordinates.

sonnel Management.

Public Per-

19(2), 201-2 12.

McGarvey, R., & Smith, S. (1993). When workers rate the boss. Training, 30(3), 3 l-34.
Mohrman, A. M., Jr., Resnick-West, S. M., & Lawler, E. E. III. (1989). Designing pe$ormance
appraisal systems: Aligning appraisals and organizational realities. San Francisco: JosseyBass.
Northcraft, G. B., & Early, P. C. (1989). Technology, credibility and feedback use. Organizational
Behavior

and Human Decision

Processes,

44, 83-96.

Nunnally, J. C. (1978). Psychometric theory. New York: McGraw Hill.

Orpen. C. (1994). The effects of exchange ideology on the relationship between perceived organizational support and job performance. Journul ojSocia1 Psychology, 134(3). 407408.
Pearce, J. L., & Porter, L. W. (1986). Employee responses to formal performance appraisal feedback.
Journal

ofApplied Psychology,

71, 2 I l-2 18.

Podsakof, P. M., & Farh, J. L. (1989). Effects of feedback sign and credibility on goal setting and task
performance. Organizational Behavior and Human Decision Processes. 44, 45-67.
Romano, C. (March, 1994). Conquering the fear of feedback. HR Focus, 71(3), 9-19.
Shore. L., Tetrick, L. (1991). A construct validity study of the survey of perceived organizational
support. Journal ofApplied Psychology, 76, 637-643.
Shore, L. M., & Wayne, S. J. (1993). Commitment and employee behavior: Comparison of affective
commitment and continuance commitment with perceived organizational support. Journal of
Applied Psychology,

78(5), 774-780.

Smither, J. W., London, M., Vasilopoulos, N. L., Reilly, R. R., Millsap, R. E., & Salvemini, N.
(1995). An examination of the effects of an upward feedback program over time. Per.sonnel
Psychology,

48, l-34.

Smither, J. W., Wohlers. A. J., & London, M. (1995). A field study of reactions to normative versus
individualized upward feedback. Group and Organization Management, 20(l), 61-89.
Tsui, A. S.. & Ohlott, P. (1988). Multiple assessment of managerial effectiveness: Interrater agreement and consensus in effectiveness models. Personnel Psychology, 41, 779-803.
VanVelsor. E., & Wall, S. J. (1992, March). How to choose a feedback instrument. Training, 47-52.
Wohlers, A. J., & London, M. ( 1989). Ratings of managerial characteristics: Evaluation difficulty,
co-worker agreement, and self-awareness. Per.yonne/ Psychology, 42, 235-261.