360° feedback: a

shift in agenda
new directions for a different
leadership world

“The demand for greater ‘bang for buck’ - more

performance for less time and money - is
proving a powerful discipline in rethinking
and repositioning 360º feedback in an
increasingly uncertain leadership world.”
Overview
360º feedback is now a fairly well established practice within
most medium to large scale organisations, deployed as a stand-
alone application for personal development or, increasingly,
integrated within performance management and talent planning
processes.

As a “methodology” 360º feedback emerged from military

selection practices during World War 2 when selectors found that
the inclusion of peer evaluations improved predictions of future
effectiveness. Esso in the US experimented with the approach in
the 1950s, and through the ‘60s and ’70s, 360º feedback evolved
more widely within the US corporate sector, before migrating
internationally.

With over three decades of significant activity and research, what

do we know about 360° feedback and its impact?

In this review of the research base, and our own evaluation of

organisational experience of what is and isn’t working, we
highlight:  the challenges organisations face in implementing 360º
feedback when there is increasing pressure to optimise
productivity and effectiveness

 why 360º feedback processes and practices need to be

revisited to improve the “bang for buck” ratio and achieve
. higher levels of performance for less money and time

 practical recommendations to sustain the long-term impact of

feedback systems in a changing world

© AM Azure Consulting Ltd 2010 2

The growing popularity of 360º feedback
Disillusionment with conventional performance appraisal, and the
reliance on the line manager as the arbiter of judgements of
effectiveness, contribution and impact underpinned much of the
interest in “multi-rater” feedback. This trend in turn has been
reinforced by the shift in work patterns (e.g. higher levels of
flexible working) and working relationships (e.g. more project
based activity), dynamics that made the idea of the all-seeing
and all-knowing line manager even less defensible.

Organisations looked at alternatives to get a better insight into

current performance as well as indicators of potential to progress.

Assessment centres, announced in the ‘70s as “the next big

thing” have failed to deliver on their initial promise of significantly
better predictive validity. Whilst retaining reasonable levels of
measurement power, assessment centres are becoming an
increasingly expensive methodology. Worryingly, the latest
review of assessment centre validity reports a decline in reported
validity over the last 40 years and observes “the reasons for the
ostensible decline in validity are not clear.”¹

And the evidence of 360º feedback’s power to predict future
performance compares well with assessment centre validity².
360º feedback as a practice also possesses a work relevance
and credibility with participants that assessment centres can lack.
Faced with pressures for greater productivity, 360º feedback
looked a more cost effective option.
.
The time-consuming bureaucracy of the paper shuffle involved in 360º
feedback programmes however was a barrier to take up. The arrival of
the internet and on line technology in the ‘90s eliminated the need for
extensive administrative processing, making the practice accessible to
even small sized firms.
.

¹ “Validity of assessment centres for personnel selection”, Thornton et al, Human Resource
Management Review, 2009
² “Assessment centres and subordinate appraisals of managers: a 7 year examination of
predictive validity”, McEvoy et al, Personnel Psychology, 1989

© AM Azure Consulting Ltd 2010 3

A summary of the research: mixed findings
360º feedback is popular, not only with talent management
professionals, but most significantly, with participants themselves.
Typically participant satisfaction levels are at the 70-80% mark¹.

But does the evidence base indicate a methodology that provides

accurate measures of current performance and future
effectiveness? And does 360 feedback operate as a key dynamic
in facilitating the kind of change that drives improvements in
management and leadership performance?

Initial research reported encouraging signs in identifying positive

relationships between 360º feedback and objective measures of
production, customer satisfaction, sales and profitability.

But in one meta analysis of 26 longitudinal studies, the impact of

360 feedback in improving managerial performance was found to
be modest. In another meta-analysis of over 600 studies looking “lt is time for
at the impact of feedback on performance, a third of the studies
showed a performance decline. researchers and
practitioners to ask:
360 feedback - like any other practice - incorporates the good,
bad and ugly. We shouldn’t expect positive outcomes if the “under what
methodology is badly designed, poorly positioned and conditions and with
implemented, with minimal supporting infrastructure or follow
through.
whom is multi-source
feedback likely to be
What factors make the difference?
beneficial?” rather
than asking “does
multi-source feedback
work?” Smither et al

¹ “The impact of 360º feedback on leadership development”, Rehbine, 2006

© AM Azure Consulting Ltd 2010 4

Specific research findings
We know that some individuals benefit from 360 feedback. And
others don’t. Indeed, some individuals show a performance
decline after 360° feedback. We also know that some practices
and processes build momentum for change and improvement,
and others make a minimal impact, or even may be counter-
productive.

What, in the words of Chip and Dan Heath in Switch, are the
“bright spots”, the examples of positive practice that are making
an impact?

We know selection of feedbackers is key. Feedback outputs

are only as accurate and insightful as the quality of the feedback
inputs and a maturity that is “able and willing” to provide
meaningful feedback. It also depends on a broad range of
different working relationships. Individuals in roles where
contact is limited or restricted to a small number of colleagues do
not benefit from feedback activity.

Accuracy is lowest for feedbackers who have known the “target”

individual for 3-5 years. The most accurate feedback comes from
colleagues who have known the individual for 1-3 years. Less
than one year is also problematic. It looks like we need
reasonable exposure to provide insightful evaluations; too much
exposure however, and the feedback falls back on halo effect
stereotypes.

Feedback that is accessible and can be recalled weeks after

the initial debrief drives improvement. Complex and confusing
feedback outputs don’t trigger change.

Feedback that is specific and grounded in the practicalities of

leadership life works better than more generalised statements of
behavioural competency.

© AM Azure Consulting Ltd 2010 5

Specific research findings
Individual differences in personality make a difference. The
dimension of conscientiousness in the “Big Five” of personality is
associated with the willingness to participate in development activities,
as is Openness to Experience. Neuroticism is a contra indicator of
responsiveness and change. For highly anxious individuals, 360º
feedback seems to make things worse. Negative feedback triggers a
vicious circle of performance decline.

Over-estimators, those individuals with inflated self ratings compared

to others, tend to have “diminished organisational and personal
outcomes”. Under-estimators, critical of their own effectiveness
relative to the evaluations of their colleagues tend to be perfectionist,
more likely to focus on their weaknesses rather than build on
established strengths.

Individuals who are generally positive towards change are more

likely to act on the feedback, as are individuals who set goals based
on the feedback.

Improvements may be more evident after 2 - 4 years. We shouldn’t

therefore expect instant change and improvement following the
introduction of 360° feedback.

© AM Azure Consulting Ltd 2010 6

Where does this leave us?
Given the variety of 360º feedback practices - objectives, design,
content, response formats, report outputs, and processes for
debrief, follow through and evaluation - it is not surprising that
there is mixed pattern of research findings.

We are moving into a different business world, one in which there

will be greater pressure for cost efficiency, productivity and
outcomes. This is partly about process logistics and exploiting
technological innovation to speed up the user experience and
improve administrative efficiency. It is also about the
management of expectations - before and after the exercise - to
position the exercise and create momentum to translate feedback
into feedforward.

Importantly it is also about a shrewd insight into the psychology of

personal change to optimise the chances of the kind of practical
development planning and implementation that results in
sustained improvement. Here are our suggestions:

360° feedback design and practice needs to reflect these

changing realities to ensure greater “bang for buck”, and achieve
better outputs from smaller inputs.

© AM Azure Consulting Ltd 2010 7

Four big challenges
Speed and the need for shorter completion times
There is of course a trade-off between credible and
comprehensive coverage based on robust and defensible
measurement and the time required for feedback data capture.
But there is a sense that the balance has tipped to unnecessary
length and complexity. The challenge is to retain the power of
360º feedback at greater efficiency and speed.

Differentiation to avoid the “blob” factor which fails to discern

differences in performance, contribution, impact and outcome.
The combination of “central tendency” and positive skew isn’t
new to performance evaluation of course. The specific challenge
for 360 feedback is to rethink the questions we ask and how we
ask them to provide a spectrum of feedback outputs, identifying
the exceptional performer as well as highlighting those struggling
to make an impact.

Turning feedback into feedforward to drive the development

that makes for gains in individual and overall performance. “It is the feedback process, not the
Is 360º feedback an exercise in curiousity, emotional
reassurance or introduced to catalyse the change that produces
measurement process, that
greater effectiveness? The challenge for 360° feedback is to generates the real pay-offs.”
move it from the analysis of personal reports to set an agenda for
J Jones & W Bearley
development that results in significant improvements.

Integration with other performance and talent management

processes. There has been a move away from 360º feedback as
a stand alone application for personal development to
incorporate the exercise within the information flows and outputs
of wider talent management activity. The challenge here is to
ensure that this shift is well understood and positioned to provide
important gains in organisational intelligence and decision
making.

© AM Azure Consulting Ltd 2010 8

Speed and efficiency of process
Five individuals in your organisation undertake 360º feedback
once a year. Assuming fifteen feedbackers for each participant,
and the feedback takes each person 20 minutes, the overall
exercise requires 25 hours, or around 3.5 person days.

As the Americans might say: “Do the maths.” Multiply up the

numbers. If 100 individuals participate in the feedback
programme, the total organisational time will be 71 person days.

But if we manage to reduce feedback completion times to 10

minutes, we save over 35 person days. Or if we decide to move
to a more complex leadership framework, designing a feedback
questionnaire that takes 30 minutes to complete, our organisation
has to factor in 107 person days for the exercise.

In a busy world of competing pressures on organisational time,

360 feedback activity has to improve its speed and efficiency.

© AM Azure Consulting Ltd 2010 9

Speed and efficiency of process
 know what 360 feedback can and can’t do. 360 feedback is
powerful in identifying the nuts and bolts of leadership, the visible
activities of communication, delegation, coaching and so on. It isn’t
well positioned to address those more cerebral aspects of
leadership such as strategic thinking. Focus feedback content on
the activities which 360° feedback is genuinely good at addressing
rather than attempt to evaluate every aspect of leadership.

 deploy different feedback versions for different applications,

using short versions for high volume activity within a narrow space
in your organisational calendar (e.g. performance reviews), and
extended versions for those activities (e.g. executive coaching)
requiring a detailed and comprehensive insight.

 know who knows what. Line managers have their perspective and
can evaluate some areas of leadership activity and impact
meaningfully, but not others. Peers look sideways and have
exposure to a different experience. And team members have
access to many leadership behaviours, but not all. The trick is to
target questionnaire sub-sets for the different feedback groups.
Apart from the virtue of reducing overall completion times, this tactic
avoids the problem of high levels of “cannot says” which undermine
the credibility of the feedback.

 conduct regular reviews to keep fine tuning the process and

content. Technological innovation opens up new options for greater
process productivity. Keep tracking trends to upgrade your system
to exploit these gains.

© AM Azure Consulting Ltd 2010 10

Powerful questions to ask
Factor analysis is a powerful statistical technique to simplify life.
It identifies the underlying patterns that make most sense of
complexity, highlighting in the case of 360º feedback the
fundamental factors that explain most of the variation in
responses.
A 360º feedback process might attempt to assess 15 different
competency dimensions in a 60 item questionnaire; factor
analysis crunches the numbers ruthlessly to check if we are in
fact able to profile 15 distinct leadership themes. (The typical
answer is no.)
In our latest factor analysis of 360º feedback data, we
interrogated our database - over a decade of data and more than
150,000 responses - to identify the “big hitting” questions, those
questions that provide most insight into overall leadership
effectiveness and impact.
If we can only ask 20 questions in 360° feedback, which
questions would be most informative? Or, if time constraints
meant we could only ask 15, which would we choose? What
would be the most powerful questions if we were only allowed 2
minutes completion time?
If you’re looking to shorten and streamline your 360° feedback
processes, with maximum coverage of overall leadership
effectiveness for minimal time, it’s a good starting point to
incorporate the following into your feedback system:
© AM Azure Consulting Ltd 2010
“A question such as “Am I the best lover you’ve
ever had?” is dangerous because it has only one
answer that can make us happy. But a question
“What do you like best about my love-making?”
is brilliant because it has only one answer that
can make us truly miserable.”
Daniel Gilbert
Ten Powerful questions
How you would evaluate the individual’s effectiveness in:
1. making others feel special and valued through
enthusiasm and interest in their work
2. putting themselves on the line to say what others are
thinking but not saying out loud
3. utilising their personal authority and influence to persuade
difficult colleagues around to their point of view
4. bringing a real sense of excitement into the work place to
energise their work colleagues
5. making the optimum use of everyone’s time through
effective scheduling and coordination
6. generating creative proposals to rethink how we need to
position ourselves in the future market-place
7. committing quality time to encourage each individual in
the team to feel an important part of our overall efforts
8. tackling any issues of under-performance in the work
group that others may have been ignoring
9. reviewing and agreeing personal objectives and priorities
with each team member on a regular basis
10. stimulating a frank and open discussion about the major
changes that need to be made
11
Differentiation to avoid the “blob factor”
The art and science of 360 feedback continues to debate the pros
and cons of different response formats and rating scales. The
challenge of course is to encourage frank feedback that provides
a discerning insight into the spectrum of effectiveness.

 provide questionnaire statements within the feedback content

that make it “easy” for feedbackers to provide candid
feedback. The questionnaire statement , for example, “she
tells the truth” on a 1-5 frequency scale doesn’t work. But the
statement “she says what she means, and means what she
says” does well as one insight into integrity

 monitor response patterns to identify and remove

questionnaire statements that fail to differentiate. Every
360° feedback process should provide a balance of challenge
and support, and different organisations reflect their
aspirations of excellence differently in the design of 360º
feedback systems. It is also true that outputs that don’t reflect
the continuum of leadership contribution and impact are
pointless. Check the balance of feedback content is set at the
level of expectation your organisation requires.
 use a variety of response tasks. As well as conventional
rating scales (1-5 etc) and open ended text, it is worth
incorporating the “yes-no” format, presenting statements of
leadership activity against the format:
 is this is an exceptional strength?
 is this a potential constraint to effectiveness?
 is this a risk factor for the individual?
The value of 360º feedback systems is improved by technical
improvements in questionnaire design. But we shouldn’t lose
sight of the reality that 360 feedback “works”, (i.e. drives
improvement), for some organisations, but not others. 360º
feedback works when there is a culture of aspiration based on a
demanding set of expectations of excellence, what it looks like
and how it is possible to keep improving. And 360° feedback
programmes don't work in a culture of mediocrity, when the
prevailing climate is that “this is probably as good it will get.”
If you’re introducing 360º feedback for the first time, it’s worth
checking what changes need to be implemented to create a
response climate of maturity, honesty and challenge around
excellence rather than assuming 360º feedback alone will
transform your organisational culture.

¹ “The impact of 360º feedback on leadership development”, Rehbine, 2006

© AM Azure Consulting Ltd 2010 12

Developing to drive improvement
In an ideal world, 360º feedback outputs would be supported by a
one to one debrief with an exceptionally wise external coach,
followed up with a review with a progressive and insightful line
manager, and supported by an organisational infrastructure of
targeted training and development and access to a
comprehensive on line resource.

The new world won’t be ideal. Budgets for facilitated report back
via external coaches are being cut back. Line managers will be
busy, juggling competing priorities and preoccupied with their
personal concerns. In this new world we need to be imaginative
in how we deploy resources to optimise the impact of 360º
feedback.

We do know what drives development and it doesn’t demand a

huge budget. But it does require the discipline of accountability
and the diligence of follow through.

Well designed and implemented 360º feedback systems might fill

the “feedback void”, the information that current and emerging
leaders need as a reality check on what’s helping and hindering
their effectiveness. But we also need to avoid the “feedforward
stall” to accelerate the speed from review of feedback results to “We found that participants who
implement the practical actions that underpin improvement. do not follow up make no progress.
The people who improve most have
a support group or network that
holds them accountable.”
Marshall Goldsmith

© AM Azure Consulting Ltd 2010 13

Developing to drive improvement
 simplify the output. This is a genuinely tough nut to crack.
Some participants only want a summary of the key findings.
Others want a detailed breakdown of the full analysis. Some
prefer text based output, others want a visual display of every
pattern of the feedback. However complex reports based on
every permutation of number crunching don’t help. Find a
way to summarise the data to indicate the key themes, and
move from analysis to action.

 don’t assume an initial debrief is enough. As Kenneth

Nowack points out many 360º feedback processes seem to
assume that the truth will make us free, and that “self directed
insight will result in motivated behavioural change”. This is
“diagnose then adios”. It might for a handful of highly
motivated individuals; it doesn’t for most. Personal change
and improvement requires the discipline of follow up, and the
application of deliberate practice.
 put in place a mix of facilitated debrief and on line support.
 hold individuals and their line managers accountable. Face to face review and coaching is extremely useful, but
The days in which participants are allowed to file their report expensive. On line leadership resources such as
into a folder called “Action Later Maybe” are over. Instead, www.theleadershipdirectory.com, are highly cost-effective,
those participating in a 360º feedback exercise should agree but lack the personal touch. The combination of the two
as a minimum to schedule a review session with their line strategies is powerful, optimising expensive coaching time to
manager to discuss the headline findings and the implications focus face to face meetings on the key issues and using on
for priorities and development. line resource for preparation, “homework” and follow on
guidance.
 set an expectation that participants will share the results
with their direct reports. No doubt a difficult exercise for
some individuals who may be surprised or shocked by the
feedback from their direct reports. But if participants don’t
conduct this debrief with their team, apart from any damage
to personal credibility, they are unlikely to change and
develop.

© AM Azure Consulting Ltd 2010 14

Integration with other talent management practices
360º feedback is widely used as a key activity to support
personal and professional development, typically, in combination
with coaching or as part of a structured leadership development
programme. Increasingly organisations are looking at ways to
integrate the feedback process and outcomes within wider
performance and talent management programmes.

This requires a rethink in both feedback content and process

management.

A shift in feedback content. Most 360º feedback systems are

designed around a framework to evaluate the inputs of
behavioural competency, but don’t address the bottom line of
outputs: what contribution is the individual making, and what is
their business impact.
The inputs of different leadership capabilities and skills are
helpful in opening up a development agenda to identify any gaps
that may be constraining effectiveness, current strengths that can
be deployed more productively, and any risks of over-playing
existing styles. But this format of feedback doesn’t provide a
direct answer to the all-important questions of:
 is this individual adding significant value to the organisation -
right now?
 is this an individual who may be ready to make a bigger
contribution in future?
Introduce into your feedback process the bottom line questions
that matter. And of course the bottom line is different for different
work colleagues who have different expectations of outcome.
 line managers want business results and gains in
productivity and innovation
 for peers, the bottom line is responsive support and a
willingness to take the initiative to resolve any problems of
task coordination
 team members’ bottom line is a combination of clear
direction, meaningful priorities and the willingness to support
their career development
 stakeholders appreciate the proactivity and “extra mile” that
creates superior levels of service

Repositioning 360º feedback to identify the outcomes that matter

to others rather than evaluate only the inputs of competence is
illuminating, differentiating those individuals who understand and
manage the “art of leadership juggling” and make a consistent
and sustained contribution from those who are failing to meet the
expectations of their work colleagues.

© AM Azure Consulting Ltd 2010 15

Integration with other talent management practices
A shift in feedback content is important in moving towards a more
integrated performance and talent management game plan. It
also requires attention to process management.

If 360° feedback reports are accessible only to the individual,

clearly organisations can learn little about the activity, either
about specific leaders and relative levels of effectiveness and
contribution, or about the pattern of trends that might indicate
potential organisational opportunities and risks.

The trend now is towards using 360° feedback data to inform

talent decision making:
 producing “pen portrait” summaries that identify key
themes in line management, peer and team member
evaluations. These summaries, integrated with other career
and performance data, feed into succession reviews and
shape priorities and action planning.

 smarter database management to integrate 360° feedback 
summaries into talent planning software. The days of “files in
folders in filing cabinets” should be over, but in reality
organisations still struggle to integrate their knowledge of
their people. The import of 360° feedback information
enriches your talent database, providing important
intelligence about the breadth and depth of professional and
leadership capability.
data mining to identify “over rated” talent (e.g. high line
management evaluations but indifferent feedback from peers
and direct reports) and neglected talent (low line
management evaluations but positive evaluations from other
feedback groups) who rarely surface during succession
discussions. This is using 360° feedback outcomes to
challenge the established “top down” perceptions of the
“usual suspects” and rethink the pool of current and emerging
talent.

© AM Azure Consulting Ltd 2010 16

Conclusions
360º feedback, from its origins over 50 years ago, has been
dominated by a specific paradigm, the mind-set of the
psychometricians. Measurement rigour has been emphasised
over the realities of personal change or the pragmatics of
organisational life.

At best, this tradition has ensured that feedback programmes

have been accompanied by systematic evaluation and the
consolidation of an evidence base. At worst, this approach has
resulted in over-engineered instruments that are time-consuming
and in their complexity, actually hold back the process of
feedforward and development planning.

360° feedback has many virtues. It is viewed positively by

participants as relevant and credible. It reflects the reality of
shifting working dynamics. And well positioned and implemented
it does achieve performance gains.

The challenge now is to optimise these gains through the design

of shorter and simpler tools with big hitting power, the building of
an organisational culture and infrastructure of accountability and
discipline, repositioning processes to connect to the broader
talent management agenda, and exploiting the benefits of
technology to provide supporting resource.

And, to reflect these priorities, it may be time for a change of

name, and move from feedback to Feedforward.

© AM Azure Consulting Ltd 2010 17

About us
AM Azure Consulting Ltd works with a broad portfolio of
clients in the design and implementation of on line services
in recruitment and selection; management assessment,
development and career management; on line leadership
tool kits, 360° feedback, performance management; and
talent and succession management.
If you are interested in our approach to leadership
assessment and development, our assessment tools and
our on line resource at www.theleadershipdirectory.com call
us:
„ 44 (0) 1608 654007 or email
„ admin@amazureconsulting.com
© AM Azure Consulting Ltd 2010
„ we’re professionals but we’re not pompous. We are at
the edge of the latest research and thinking in the field of
people management, but we’re not precious about the “one
thing”. We have some good ideas to help your organisation
perform even better, but we know that you have some better
ones, but want support in making them work. We don’t
impose the “solution”.
„ we design but we also implement. Our content, design
and technology can build cost effective solutions quickly.
Our consultancy experience of “real world” implementation
will move things forward from initial concept to results
rapidly.
„ we start things to build momentum but we also follow
through. Results come from the discipline of “making it
stick”, of evaluation, learning and continual improvement.
And we maintain ongoing relationships with our clients to
keep achieving positive outcomes.
18