MEASURING RESULTS

AND BEHAVIORS

Herman Aguinis, University of Colorado

Measuring Results: Key questions

• Where should each individual focus efforts?

Key Broad areas of a job for which employee is
Accountabilities responsible for producing results

• What are the expected objectives?

Expected Statements of important and measurable
Objectives outcomes

• How do we know how well the results were achieved?

Performance Yardstick used to evaluate how well employees have
Standards achieved objectives

Herman Aguinis, University of Colorado

at
How to Determine Accountabilities?
Collect information about job with the help of Job
Description.()
 J o b description provides information on the task
performed. Task can be grouped into clusters of Task
based on their relatedness.
Determine the relative degree of importance of task or
cluster of tasks
• % of employee’s time spent performing task
• Impact on unit’s mission if performed inadequately
• Consequences of error

Herman Aguinis, University of Colorado

Determining Objectives
Purpose:

• to identify Limited number of Highly important results

that When achieved will have a dramatic impact on
overall organization success

•After objective are set employee should receive the feedback on their
progress towards attaining the objective.
•Rewards should be allocated to those employees who have reached their
objective Herman Aguinis, University of Colorado
at
Characteristics of Good Objectives

• Specific and Clear • Bound by Time

• Challenging • Achievable
• Agreed Upon • Fully
• Significant Communicated
• Prioritized • Flexible
• Limited in
Number
Herman Aguinis, University of Colorado
Determining Performance Standards
Standards refer to aspects of performance objectives, such
as:
• Quality
– How well the objective is achieved
• Quantity
– How much, how many, how often, at what cost
• Time
– Due dates, schedule, cycle times, how quickly

Herman Aguinis, University of Colorado

Standards must include:
• A verb
• A due date
• Some type of indicator
– Quality and/or
– Quantity

Verb Indicator
Reduce overtime from 150 hours/months to 50 hours/month by
December 1,2014 at a cost not to exceed 1,20,000
Due date

Herman Aguinis, University of Colorado

at
Characteristics of Good Performance Standards

• Related to Position
• Concrete, Specific, Measurable
• Practical to Measure
• Meaningful
• Realistic and Achievable
• Reviewed Regularly

Herman Aguinis, University of Colorado

Accountabilities, Objectives,
and Standards

Job Title:

Graduate Associate,
Sourcing &
Procurement

Herman Aguinis, University of Colorado

Accountabilities:
 Provide analytical support for sourcing projects impacting
business units. Successful analytical support will ensure that
project decisions are made on well-researched reasons.
Poorly made project decisions may delay client deliverables
or damage the Disney brand. (20% of
time)

 Benchmark current pricing models and develop new

approaches to pricing/buying various products and services
that yield creative and business advantages.
This is extremely important to the continued sustainability of
Disney, and will be critical to capture new portions of the
market share. Incorrectly identifying pricing models would
be crushing to the Disney business and would directly
Herman Aguinis,
at
impact
University of Colorado
Cont.
 Assist in the development of spend profiles, key
stakeholder lists, savings opportunities where existing
contracts are leveraged, savings opportunities in
commodity areas that have not been sourced, and sourcing
and procurement strategy. Will play a small role in these
extremely important activities.
Successfully developing spend profiles, identifying key
stakeholders, identifying savings opportunities, and
developing a successful business strategy are keys to the
financial stability of Disney. (50% of time)

Herman Aguinis, University of Colorado

at
Objectives: :
• Develop spend profiles, key stakeholder lists, and a sourcing and
procurement strategy. These tasks need to be completed by July 1, 2020.
Identifying savings opportunities where existing contracts are leveraged
and identifying savings opportunities in commodity areas that have not
been sourced are ongoing tasks. Report weekly on
savings identified.

• Benchmark current pricing models and develop new approaches to

pricing/buying various products and services that yield creative and
business advantages. Benchmarking project due by August 1, 2020.
Developing new approaches to pricing is an ongoing project.
Turn in bi-weekly reports on new findings to your manager by COB every
other Friday.

• Provide analytical support for sourcing projects impacting business units.

Ongoing project where information must be delivered inside project
Herman Aguinis, University of Colorado
at
Performance Standards::
• Identified savings opportunities where existing contracts were
leveraged and identified commodity areas that have not been sourced
on a weekly basis for total savings of 5% over previous quarter
spending.
• Benchmarked current pricing models by August 1, 2008 in 80
hours or less of time billed.
• Submitted new approaches to pricing/buying various products and
services on a weekly basis resulting in at least two new pricing
approaches being adapted by the company by September 1, 2008.
• Provided ongoing analytical support for sourcing projects as rated
by monthly project evaluations conducted by the relevant project
managers and communicate0d to your manager on a monthly basis.

Herman Aguinis, University of Colorado

at
Measuring Behaviors: Overview

• Identify competencies
• Identify indicators
• Choose measurement system

Herman Aguinis, University of Colorado

Identify Competencies

Measurable clusters of KSAs

– Knowledge's
– Skills
– Abilities
That are critical in determining how results
will be achieved

Herman Aguinis, University of Colorado

 Types of
Competencies

Differentiating Threshold
Distinguish between Needed to perform
superior and job at minimum
average Adequate
performance standard
Herman Aguinis, University of Colorado
 We don’t measure competency directly but we
measure Indicators
A competency can have several indicators.

competency

Indicator 1 Indicator 2 Indicator 3 Indicator 4

Herman Aguinis, University of Colorado

Necessary Components for
Describing Competencies

• Definition
• Description of specific behaviors
– When competency demonstrated
– When competency not demonstrated
• Suggestions for developing the competency

Herman Aguinis, University of Colorado

Measurement System to evaluate competencies

Comparative
Compares employees
system
with each other

Choose Measurement
System

Absolute
system Compares employees
with pre-specified
performance
standard
Herman Aguinis, University of Colorado
 Types of Comparative
Systems

Simple rank Alternation Paired Forced

order rank order comparisons distribution

Advantages Disadvantages
• Easy to explain • Rankings may not be specific
• Straightforward enough for
• Better control for biases and errors – Useful feedback
– Protection from legal
found in absolute systems
challenge
– Leniency
• No information on relative distance
– Severity between employees
– Central tendency • Specific issues with forced
distribution method
Herman Aguinis, University of Colorado
 Types of Absolute
Systems

Behavior Critical Graphic

Essays
checklists incidents rating scales

Herman Aguinis, University of Colorado

Essays

• Advantage:
– Potential to provide detailed feedback
• Disadvantages:
– Unstructured and may lack detail
– Depends on supervisor writing skill
– Lack of quantitative information; difficult
to use in personnel decisions

Herman Aguinis, University of Colorado

Behavior checklists

• Advantage:
– Easy to use and understand
• Disadvantage:
– Scale points used are often arbitrary
– Difficult to get detailed and useful
feedback

Herman Aguinis, University of Colorado

Critical incidents

Two kinds of measurement

– Report of specific
employee behavior
• Allows focus on specific behavior
• Very time-consuming
– Examples of behavior illustrative
of core competencies
• Easier to use
• Describes behavior desired
Herman Aguinis, University of Colorado
Graphic rating scales
• Clear meaning for each response category
• Consistent interpretation by outside readers
• Supervisor and employee should have same understanding of
rating

Herman Aguinis, University of Colorado

Graphic rating scales:
BARS improvement

• Behaviorally Anchored Rating Scales (BARS)

– Use critical incidents as anchors
– Involves multiple groups of employees
in development
• Identify important job elements
• Describe critical incidents at various levels of
performance
• Check for inter-rater reliability

Herman Aguinis, University of Colorado

Herman Aguinis, University of Colorado
Measuring Performance

• Several types of methods

• Differ in terms of:
– Practicality (time and
effort)
– Usefulness (quantifiable)

Herman Aguinis, University of Colorado

 Make a group of 5 students

Creating BARS-Based Graphic Rating

Scales for Evaluating Business
Student Performance in Team
Projects

Herman Aguinis, University of Colorado

In many universities, students are required to
conduct Team Projects.

Job Description for students-:

• Work with team member to deliver project outcome on
time and according to specification .
• Complete all individual assignment to highest
quality ,completing necessary background
research , making analysis and preparing final
documents.
• Foster a good working environment
Herman Aguinis, University of Colorado
Please do the following:

1. Generate a list of competencies for the

position described.
2. Identify the list of critical behavioral indicators
for each competency.
3. Generate critical incident (High ,average and
poor performance for each behavioral indicator.
4. Create Graphic rating scales using BARS to
measure each competency.
Herman Aguinis, University of Colorado
Competencies:
• Teamwork: Ability to work with a team to finish all project deliverables on
time and according to instructions. Assists others as necessary and
constantly seeks to further relationships with team members. Failure
to develop teamwork will result in reduced team effectiveness due to lack of
communication and cooperation between team members.

• Business knowledge: Aware of current business trends and applies relevant

business research skills to projects at hand. Always learning
new skills, staying current on business news, and talking with business
professionals. Failure to develop business knowledge will result in a
decrease in the quality of work produced by the team.

• Communication: Ability to articulate verbally or through written

communication ideas on project deliverables. Use of correct grammar,
professional presentation, and clear and concise messages characterize good
communication. Failure to communicate well will result in the
decreased performance of the team, because team members will no longer
Herman Aguinis, University of Colorado
understand ideas put forth. at
Behavioral Indicators
• Teamwork
– Friendly environment exists
– Team members’ input is accepted
– Offers to help other team members when necessary
• Business knowledge
– Student offers input when group is discussing business
problems
– Actively researches solutions presented
– Reads business news sources on a daily basis to stay up-
to- date on business trends
• Communication
– Clearly articulates points in meetings
– Uses correct grammar and word usage in
written communications
– Professionally presents ideas and findings in
project
presentations Herman Aguinis, University of Colorado
Critical Incidents:
Average
Indicat High Performan Poor
or
Student offers input Performance ce Performance
when group is Continuously provides Offers input on areas of
discussing business relevant input into group expertise in group Does not offer input
problems discussions discussions into group discussions
Researches practical and Does not research Actively researches theoretical
implications of Researches practical implications of business solutions presented decision implications of
decision decisions
Reads business
news sources on a Comments on a daily basis Comments on a weekly Does not
comment
daily on to stay onthe status
basis of ongoing basis on the status of the status
up-to-date of ongoing on business developments of ongoing business business
business trends developments
interest developments of interestDoes of interest
not get point
across in meetings,
which results in
Presents ideas in a clear and arguments with other
concise manner and team members or mis-
Clearly articulates ensures that team Presents ideas in a clear education of team
nis, University of Colorado
points in meetings members understand point and concise mannerHerman members
Indicato High Average Poor
rUses Performance
correct Written communications Performance Performance
grammar and word are clear, free of spelling Poor spelling
usage in written and grammatical Clear use of language in grammatical usage a
communications mistakes written reports written reports n
Professionally Prepares presentation so Minimally prepares for d
presents ideas and ideas are communicated presentations, so in
findings in project well and in a professional communication is rushed Does not prepare for
presentations manner or drags on presentations
Takes an interest in team
Friendly members outside of Talks to team members to
environment exists project complete tasks Withdrawn from team

Solicits, but does not Does not solicit input

Team members’ Supportive of other team support team members’ from other team
input is accepted members’ ideas ideas members

Volunteers for extra

assignments so
Offers to help other team members can finishthat
Offers to help team
team members projects on time and to members if deadline is Does not offer to help
Herman nis, University of Colorado
when necessary the highest standards approaching team members
Scales

Herman Aguinis, University of Colorado

Teamwork:
Ability to work in a team to finish all project deliverables on time and according to
instructions. Assists others as necessary and constantly seeks to further
relationships with team members. Failure to develop teamwork will result in
reduced team effectiveness due to lack of communication and cooperation between
team members.

5 Exceptional : Takes an interest in team members outside of

3 project.
Competent : Talks to team members to complete
1 tasks.
Major Improvement Needed : Withdrawn from
team.
5 Exceptional : Supportive of other team members’
3 ideas.
Competent : Solicits, but may not support team members’
1 ideas. Improvement Needed : Does not solicit input from other team
Major
members.
Exceptional : Volunteers for extra assignments so that team members can finish projects
5 on
3 time and to :the
Competent highest
Offers standards.
to help team members if deadline is
1 approaching.
Major Improvement Needed : Does not offer to helpHerman teamAguinis, University of Colorado
Business Knowledge:
Aware of current business trends and applies relevant business research skills to
projects at hand. Always learning new skills, staying current on business news,
and talking with business professionals. Failure to develop business
knowledge will result in a decrease in the quality of work produced by the team.
5 Exceptional : Continuously provides relevant input into group discussions.
3 Competent : Offers input on areas of expertise in group discussions.
1 Major Improvement Needed : Does not offer input into group discussions.

5 Exceptional : Researches practical and theoretical implications of decisions.

3 Competent : Researches practical implications of decisions.
1 Major Improvement Needed : Does not research implications of business decisions.

Exceptional : Comments on a daily basis on the status of

5 ongoing business developments of interest.
Competent : Comments on a weekly basis on the status of
3 ongoing business developments of interest.
Major Improvement Needed : Does not comment on the status of ongoing business
Herman Aguinis, University of Colorado
1 developments of interest.
Communication:
Ability to articulate verbally or through written communication ideas on project
deliverables. Use of correct grammar, professional presentation, and clear and
concise messages characterize good communication. Failure to communicate
well will result in the decreased performance of the team, because team members
will not understand ideas expressed.

5 Exceptional : Presents ideas in a clear and concise manner and ensures that team members understand point.

3 Competent : Presents ideas in a clear and concise manner.

Major Improvement Needed : Does not get point across in meetings, resulting in arguments
1 with other team members or miseducation of team members.

5 Exceptional : Written communication is clear, free of spelling and grammatical mistake

3 Competent : Clear use of language in written reports.
1 Major Improvement Needed : Poor spelling and grammatical usage in written
reports.
Exceptional : Prepares presentation so ideas are communicated well and in a
5 professional manner.
3 Competent : Minimally prepares for presentations, so Herman Aguinis, University of Colorado
at communication is rushed or
Herman Aguinis, University of Colorado