Behind BARS: Evaluating Employees with Behaviorally Anchored Rating Scales The BARS (behaviorally anchored rating scales)

method of evaluating employees carries typical job appraisals one step further: Instead of relying on behaviors that can be appraised in any position in a company, the BARS method bases evaluations on specific behaviors required for each individual position in an individual company. Development of BARS evaluations requires an in-depth understanding of each positions key tasks, along with an understanding of the full range of behaviors displayed by individuals in carrying out such tasks. You rate these behaviors for each employee; then you anchor each behavior to points on a rating scale, which indicates whether the behavior is exceptional, excellent, fully competent, or unsatisfactory. The result is a rating scale for each task. For example, in a hypothetical position of human resources coordinator, one of the job holders responsibilities is to complete status change notices, which update the personnel system regarding changes in employee pay, position, title, supervisor, and personal data. The BARS method for this specific task in this specific job could read as follows: 5 Exceptional performance: Accurately completes and submits all status change notices within an hour of request. 4 Excellent performance: Verifies all status change notice information with requesting manager before submitting. 3 Fully competent performance: Completes status change notice forms by the end of the workday. 2 Marginal performance: Argues when asked to complete a status change notice. 1 Unsatisfactory performance: Says status change notice forms have been submitted when they havent. Pros and cons of the BARS method The BARS approach offers several key advantages:

Its behaviorally based. The BARS system is totally focused on employee performance. Ideally, it removes all uncertainty regarding the meaning of each numerical rating. Its easy to use. The clear behavioral indicators make the process easier for the manager to carry out and the employee to accept. Its equitable. With its heavy emphasis on behavior, the evaluation process comes across as fair. Its fully individualized. From the standpoint of consistency within a company, BARS is designed and applied individually and uniquely for every position. Its action-oriented. With an understanding of the specific performance expectations and standards of excellence, employees can much more easily take steps to improve their performance, and theyre more likely to do so as a result.

Like any method, BARS isnt perfect. Here are some of the drawbacks to the BARS approach:

The process of creating and implementing BARS is time-consuming, difficult, and expensive. Each BARS form must be created from scratch for every position in the company.

Sometimes the listed behaviors still dont include certain actions required of the employee, so managers can have difficulty assigning a rating. Its high maintenance. Jobs change over time, which means that BARS requires a high degree of monitoring and maintenance. Its demanding of managers. In order to successfully conduct BARS evaluations, managers need detailed information regarding the actions of their employees. Gathering such data can be quite time-consuming, and many managers end up letting this slide.

Behaviorally anchored rating scales Behaviorally anchored rating scales (BARS) are scales used to rate performance. BARS are normally presented vertically with scale points ranging from five to nine. It is an appraisal method that aims to combine the benefits of narratives, critical incidents, and quantified ratings by anchoring a quantified scale with specific narrative examples of good, moderate, and poor performance.[1] Background BARS were developed in response to dissatisfaction with the subjectivity involved in using traditional rating scales such as the graphic rating scale.[2] A review of BARS concluded that the strength of this rating format may lie primarily in the performance dimensions which are gathered rather than the distinction between behavioral and numerical scale anchors.[3] [edit] Benefits of BARS BARS are rating scales that add behavioral scale anchors to traditional rating scales (e.g., graphic rating scales). In comparison to other rating scales, BARS are intended to facilitate more accurate ratings of the target person's behavior or performance. However, whereas the BARS is often regarded as a superior performance appraisal method, BARS may still suffer from unreliability, leniency bias and lack of discriminant validity between performance dimensions.[3][4] [edit] Developing BARS BARS can be developed using data collected through the critical incident technique,[5] or through the use of comprehensive data about the tasks performed by a job incumbent, such as might be collected through a task analysis. In order to construct BARS, several basic steps, outlined below, are followed. 1. Examples of effective and ineffective behavior related to job are collected from people with knowledge of job using the critical incident technique. Alternatively, data may be collected through the careful examination of data from a recent task analysis. 2. These data are then converted into performance dimensions. To convert these data into performance dimensions, examples of behavior (such as critical incidents) are sorted

into homogeneous groups using the Q-sort technique. Definitions for each group of behaviors are then written to define each grouping of behaviors as a performance dimension 3. A group of subject matter experts (SMEs) are asked to re-translate the behavioral examples back into their respective performance dimensions. At this stage the behaviors for which there is not a high level of agreement (often 5075%) are discarded while the behaviors which were re-translated back into their respective performance dimensions with a high level of SME agreement are retained. The re-translation process helps to ensure that behaviors are readily identifiable with their respective performance dimensions. 4. The retained behaviors are then scaled by having SMEs rate the effectiveness of each behavior. These ratings are usually done on a 5- to 9-point Likert-type scale. 5. Behaviors with a low standard deviation (for examples, less than 1.50) are retained while behaviors with a higher standard deviation are discarded. This step helps to ensure SME agreement about the rating of each behavior. 6. Finally, behaviors for each performance dimensions, all meeting re-translation and criteria, will be used as scale anchors.

Pay for Performance INTRODUCTION A major goal of any compensation program is to motivate employees to perform their best. In American industry, this goal gained importance when companies realized they were in danger of losing product markets to foreign competitors. In order to make American products and services better and more competitive, many programs were launched to elicit employee cooperation and increased effort on the job. This course discusses the resurgence of merit pay, which is now commonly called pay-forperformance. The term pay-for-performance is NOT intended to imply that performance is the only criterion for pay determination. Instead, it means that at least one component of movement within the pay range is relative performance. We'll begin by describing which organizations are suited to pay-for-performance plans. You'll also learn how to set up a pay-for-performance program, including how to establish:

an open rate pay range system a guide chart a performance appraisal system

You will learn how to choose raters and effectively administer performance appraisal systems. Finally, you'll see how software can be used to integrate labor market data into the pay-forperformance salary decision.

DESIRABILITY OF PAY-FOR-PERFORMANCE

The idea of relating pay directly to performance is highly attractive to most managers so much so that almost all organizations claim that they have pay-for-performance in the form of a merit pay system. A survey by Hewitt Associates, LLC, found: Nearly 8 in 10 companies use some sort of performance pay system, up from 5 in 10 companies in 1990.1 Both management and employees agree that tying pay to performance is desirable, and today the practice is spreading to more employee groups. Whereas managers have always worked under merit pay systems, the emphasis for other employee groups has usually been equity. Today, however, pay-for-performance is extending to such nontraditional groups as teachers. Still, there is a great deal of evidence that pay-for-performance is NOT:

easy to implement always desirable as prevalent as surveys would indicate

Drawbacks Despite its obvious appeal, not all aspects of pay-for-performance are desirable. Inequity. A focus on performance often conflicts with the compensation goal of equity: in a pay-for-performance system, employees in the same work group doing the same work may be earning greatly different pay rates. Employees can resent this, especially if the program is not well designed and communicated, or if employees do not

perceive performance as a proper criterion by which to set pay. Competition. A pay-for performance program implicitly or explicitly puts people in competition with each other. Yet cooperation is what is really needed for the work of the organizational unit to be accomplished. Where everyone has to work together, differential pay can have a divisive effect that may produce lower and not higher performance for the group as a whole. This may explain why first-line supervisors are often not as enthusiastic about pay-for-performance as higher-level managers. Effort. Pay-for-performance takes managerial time and effort, and must be designed and administered carefully. Failure to put forth the managerial and staff effort required can lead to a program that:

does NOT actually tie pay to performance makes employees distrust management

Distrust. Pay-for-performance most often relies on the judgments of managers about the level of performance of employees. Unless employees trust the judgment of the managers and perceive that it is in fact employee performance that is being rewarded, there is a good possibility that employees will reject the plan. The problem is that trust cannot be entirely created by the compensation program. Although a good program can enhance the feeling of trust, trust between employees and management must already be present. GETTING STARTED A pay-for-performance program will only succeed if it and the organization are compatible. To examine the feasibility of installing pay-for-performance in your organization, it is useful to review the 3 components of expectancy theory:

valence performance-reward connection

performance-effort connection

Valence Valence is a subjective value.

How much do I really value the available outcomes?

Since our subject is pay, we can ask: Does the employee want (value) the reward the

organization is offering?
We can be confident that the answer is yes but not the same size 'yes' for all people. People differ in how valuable money is to them compared with other things on and off the job. A pay-for-performance program is going to work best where pay is highly valent to employees.2 This valence cannot be assumed, but must be determined by research. Example: a researcher was called into a company where a group of women seemed unable to meet production standards, despite the attractiveness of the incentives provided. The researcher discovered that this was a group of women who believed they should not make more money than their husbands and felt guilty about not being at home when their children got out of school. The researcher suggested to management that the women be allowed to go home as soon as they had met their standard for the day. The suggestion was accepted and the productivity immediately.3 of the group improved

These workers were not completely motivated by money. E.E. Lawler suggests that programs such as pay-for-performance be installed only in units where the employees clearly have a high need for money.4

Performance-Reward Connection For pay-for-performance to work, there must be a connection between pay and performance. This is easy to say, but difficult to achieve. Organizations are complex social systems whose members are subject to many influences on their performance at any one time. To isolate a simple pay-performance connection is not possible. A number of problems increase the complexity of the connection:

multiple goals employee perception performance appraisal employee acceptance

Multiple goals. Compensation programs try to achieve a number of things at the same time and these different goals are often in conflict with one another. Employee perception. Even if the program does connect pay with performance, employees must perceive the connection. Secrecy in pay often leads employees to inaccurately guess at the connection.5 In addition, connecting pay and performance is not always comfortable to employees, who may therefore try to assume the connection does not exist. Performance appraisal. Pay-for-performance is only as good as the company's underlying performance appraisal system that defines "good performance." A perception that the performance appraisal system is biased or does not appraise actual performance destroys the connection for the employee. (We'll discuss reliable performance appraisal methods later in this course.) Employee acceptance. Management and employees may not agree on the performance ratings. This creates a serious complication. Management often views employee performance as a normal distribution bell curve. In this normal distribution, the bulk of employees have average performance and fall in the middle of the curve.

Normal Distribution Yet oftentimes, employees believe their performance is much higher. Example: A survey of pay-for-performance in British public schools showed that 67% of teachers thought they would be rewarded with performance pay.6

The disconnect between management and employee expectations can lead to hurt feelings, or worse mistrust. Skewed Distribution The Performance-Effort Connection Employees MUST perceive that their efforts lead to their performance ratings. A pay-for-performance program assumes that performance varies among employees and that this difference is observable. But in many jobs, variation is impossible or is so little that it is

unrealistic to try to measure it for pay purposes. Even if there are differences, measuring or attributing the differences to employee effort may be difficult. For instance, the efforts of an individual in a group project may not be able to be divorced from the efforts of the other members of the group. Employees may not feel that they control the important measures of performance. Teachers, for example, realize that for them the important measure is student learning, but teachers feel only minimal control over that variable. Pay-for-Performance Won't Solve All Problems Pay-for-performance is not a solution for all motivation and performance problems in organizations. Pay-for-performance can be extremely effective where the requirements of expectancy theory are met. However, in many circumstances installing a pay-for-performance program is likely to lead to frustration.

PROGRAM DEVELOPMENT A pay-for-performance program is a particular method for determining the movement of employees within a pay range. The goal of the program is to match employee performance level with position in the pay range over time. This concept is shown below. The Concept of Pay for Performance

Upward movement. Movement upward in the rate range occurs only if the employee's wage rate is lower in the rate range than his/her performance is on the performance scale. Pay-forperformance allows the organization to rapidly move high performers upward in the rate range by giving large increases to these employees. Downward movement. Pay-for-performance also allows movement downward in the rate range, by freezing the wage rate at the current level. In pay-for-performance systems, employees will not receive wage cuts due to poor performance. The worst thing that can happen to their pay is that it will remain the same. Requirements of a Pay-for-Performance Program A pay-for-performance program requires the use of:

an open rate range a guide chart for pay increases a good performance appraisal system

We'll now review each of these requirements in-depth. Open Rate Range A pay-for-performance program relies on an open rate range, which defines only the minimum, the maximum, and the midpoint. The rate range needs to be broad enough so that it is possible to give large pay increases to good performers. Movement within the pay grade is determined strictly by the performance of the employee, and the position of the employee within the range is maintained only by good performance over time. Having reached a particular point in the range, the employee may slip back in relative terms the next time the pay structure is adjusted upward, if his/her performance is not as good as in the present period. (For more information on rate ranges, see Chapter 16 of ERI's Internet Based Benefit and

Compensation Administration textbook.)

Performance appraisal rating

It must be possible to rank each employee by performance. This distribution is assumed to be divisible into segments such as quartiles, and each individual can be identified as being within a particular segment. This system does not allow for everyone being rated high or low; it assumes that there is an even spread of performance a normal distribution. If this distribution does not appear in the ratings, the distribution must be developed by spreading out the ratings along a continuum. Where to start The starting point for determining a pay increase is the position of each employee in the rate range after a pay structure adjustment has been made. This is illustrated in the figure below for 3 employees: A, B and C. Structural Pay Adjustment in Pay-for-Performance

Before the structure adjustment:

A was between the first and second quartiles B was just above the midpoint C was at the top of the pay grade

After the adjustment:

A is at the bottom of the pay grade B is in the second quartile C is between the third and fourth quartiles

These new positions are the starting points for determining the pay increases for the next period.

Pay-for-Performance Guide Chart Rate range and performance rating are combined in a guide chart, as shown in the example below. Pay-for-Performance Guide Chart Performance Rating Outstanding (1) Exceeds Position Second Quartile 10% 8% Third Quartile 8% 6% no increase no increase no increase Fourth Quartile 6% no increase no increase no increase no increase

First Quartile 12% 10%

Requirements (2) Meets Position

Requirements (3) Meets

8%

6% no increase no increase

Minimum special consideration no increase

Requirements (4) Does not meet

requirements

If you know an employee's performance ranking and which quartile they are in (rate range), then you can determine where the employee falls on the guide chart. In doing so, you have determined the appropriate salary increase for the employee. Example: Eileen and Sam are both market research analysts at the same company in Philadelphia, Pennsylvania. Based on the following information and using the Pay-forPerformance Guide Chart, we need to determine the salary increase for both Eileen and Sam. Current Salary:

Eileen: $48,000 Sam: $43,000 Performance Rankings: Eileen: Exceeds Position Requirements Sam: Meets Position Requirements Market Research Analyst Rate Range:

Salary Range: $35,000 - $55,000 (Midpoint = $45,000)

From the rate chart above, we can determine the quartiles for both Eileen and Sam. Eileen: 3rd Quartile Sam: 2nd Quartile Now using the Pay-for-Performance Guide Chart, we plug in the performance ranking and the current salary quartile for both Eileen and Sam in order to determine the percentage salary increase for both individuals. Eileen: 6% Sam: 6%

Time-frame More movement for good performers and less for poor performers can be allowed by altering the time period between adjustments, such as giving increases to good performers every 6 months, while granting lower performers increases every 18 months. Such alterations allow the top percentage increases to not appear so large, and the bottom percentage increases to appear larger than they are in reality. More refined charts This sample guide chart that we've demonstrated is a simple one: it varies only the amount of the pay increase with performance and place in the rate range. Rather than having a set percentage increase, as illustrated here, each of the boxes could have a range, say 11% to 14%. Then finer adjustments could be made for those close to the boundaries of the boxes.

PERFORMANCE EVALUATION METHODS Evaluating performance is a necessary organizational process that takes place naturally in the act of managing. For certain purposes, the evaluation process must be systematized.

Performance appraisals should be specific, and the results should be recorded in such a way that employees can be compared with one another. Many Uses for Performance Evaluation Performance evaluation is a part of the majority of personnel activities in an organization. As such, performance evaluation has a number of functions. As expected, these functions are not necessarily congruent with one another, so there is a great deal of trouble in developing performance evaluation systems that are equally useful for all purposes. Differential performance Many managers will tell you that differential performance is not just an ideal; it is a fact. Some people are capable of producing 2 to 3 times what others are; the best workers can produce as much as 5 or 6 times what the worst performers can. The reward system of organizations could create much higher levels of performance, and therefore productivity in employees, if it were clear to the employees that they would be rewarded for the increased productivity. But it should be kept in mind that not all jobs permit differences in performance and not all organizations require or desire them. Assembly-line jobs. Assembly-line jobs are often designed so that variation in performance is impossible or irrelevant to the desired outcomes. Variation in performance where tight coordination of activities is necessary creates trouble, not increased productivity. Sales, engineering, managerial jobs. Jobs such as sales, engineering, and managerial jobs have a great deal of latitude in their effects on outcomes. Middle performers. Middle performers represent an intermediate position between the two extremes (discussed above) and may be extremely common in organizations. Most raters can identify the few employees who are doing an outstanding job and the few doing a poor job. But most performance appraisal systems ask that performance distinctions be made among all employees. Making distinctions in the middle of the performance scale is difficult, because oftentimes the differences are so small as to not warrant differentiation. How to define good performance

Many assume that good performance means higher output. While higher output is certainly an important definition of good performance, it is not the only one. How the job is done may also be important. In addition, an organization may wish to reward a series of behaviors, as well as the productivity of the employee. Incentive pay plans. Incentive pay plans focus strictly on the outcomes of work, such as sales volume. Incentive pay plans allow the organization to pay directly for those outcomes. This type of payment system is the topic of DLC Course 75: Creating an Incentive Pay Plan. Pay-for-performance plans. If the organization wishes to focus on more than just outcomes or finds it difficult to measure the outcomes, then it will need a pay-for-performance system, as described in this course. Incentive pay pay based on measurable output Pay for performance pay based on performance

appraisal

PERFORMANCE EVALUATION METHODS Evaluating performance is a necessary organizational process that takes place naturally in the act of managing. For certain purposes, the evaluation process must be systematized. Performance appraisals should be specific, and the results should be recorded in such a way that employees can be compared with one another. Many Uses for Performance Evaluation Performance evaluation is a part of the majority of personnel activities in an organization. As such, performance evaluation has a number of functions. As expected, these functions are not necessarily congruent with one another, so there is a great deal of trouble in developing performance evaluation systems that are equally useful for all purposes. Differential performance

Many managers will tell you that differential performance is not just an ideal; it is a fact. Some people are capable of producing 2 to 3 times what others are; the best workers can produce as much as 5 or 6 times what the worst performers can. The reward system of organizations could create much higher levels of performance, and therefore productivity in employees, if it were clear to the employees that they would be rewarded for the increased productivity. But it should be kept in mind that not all jobs permit differences in performance and not all organizations require or desire them. Assembly-line jobs. Assembly-line jobs are often designed so that variation in performance is impossible or irrelevant to the desired outcomes. Variation in performance where tight coordination of activities is necessary creates trouble, not increased productivity. Sales, engineering, managerial jobs. Jobs such as sales, engineering, and managerial jobs have a great deal of latitude in their effects on outcomes. Middle performers. Middle performers represent an intermediate position between the two extremes (discussed above) and may be extremely common in organizations. Most raters can identify the few employees who are doing an outstanding job and the few doing a poor job. But most performance appraisal systems ask that performance distinctions be made among all employees. Making distinctions in the middle of the performance scale is difficult, because oftentimes the differences are so small as to not warrant differentiation. How to define good performance Many assume that good performance means higher output. While higher output is certainly an important definition of good performance, it is not the only one. How the job is done may also be important. In addition, an organization may wish to reward a series of behaviors, as well as the productivity of the employee. Incentive pay plans. Incentive pay plans focus strictly on the outcomes of work, such as sales volume. Incentive pay plans allow the organization to pay directly for those outcomes. This type of payment system is the topic of DLC Course 75: Creating an Incentive Pay Plan.

Pay-for-performance plans. If the organization wishes to focus on more than just outcomes or finds it difficult to measure the outcomes, then it will need a pay-for-performance system, as described in this course. Incentive pay pay based on measurable output Pay for performance pay based on performance

appraisal

Performance Appraisal For most jobs in most organizations, employee performance is appraised rather than measured. Performance appraisal is a formal method of evaluating employees. This method assumes that employee performance can be observed and assessed, even when it cannot be objectively measured. Typically, the performance that is evaluated is:

outcomes of work activity and behavior involved in work

In order to appreciate the complexity of performance appraisal, we'll discuss several factors:

Which employees are appraised? Appraisal methods Performance standards Rating scales Employee comparison systems

Which employees are appraised? Most organizations have some form of performance appraisal. But some employee groups are more likely than others to be covered. Employee Groups Subject to Performance Appraisal

Likely

Unlikely white collar jobs clerical managerial professional government workers

blue collar jobs union jobs

Performance appraisal is used less among blue-collar workers due to the use of:

job-rate pay plans rate ranges where movement is based upon seniority incentive plans

Union jobs are often not subject either, as unions typically do not like the use of performance appraisal. Appraisal methods There are a large number of performance appraisal methods. However, they can be divided into two basic types. Types of appraisal methods Comparison with:

a standard

OR

another person

Comparison with a standard. This approach requires a well-developed performance standard and allows direct comparisons on this standard throughout the organization.

Comparison with another person. This approach does not require a strong performance standard and, under certain conditions, can provide more reliable results. Performance standards If you are going to use a set of criteria or standards, you need to follow the following steps. Step 1. Review the job description A well-written job description tells you the important performance standards for the job. The job description should state the tasks required by the job and the purpose of those tasks. (To learn how to write reliable job descriptions, see DLC Course 33:

Conducting Job Analysis.)

Step 2. Define acceptable performance Define how well the task must be performed to represent acceptable performance. Depending upon the type of appraisal or measurement system used, this may be done through:

employee-supervisor conferences analysis of records committee work work measurement

The more objective the standard, the easier the ratings task. Rating Scales The rating scale is the most common form of rating against a standard. Oftentimes, rating scales are used in conjunction with some other method, usually management by objectives (MBO). A rating scale defines a number of factors or criteria. The rater appraises the degree for each of these factors that best describes the employee's performance. Ordinarily, the factors and

degrees are defined so as to permit point values to be assigned to each degree statement. Then a total score can be calculated for the employee. Rating scales may be described as rulers against which employees are compared. A ruler is developed for each factor to be rated. Then each ruler is divided into "inch marks" or degrees. But the analogy should not be carried too far. A ruler is a ratio scale, since it contains a zero point. A performance appraisal scale, if well-designed, is an interval scale a scale whose units (inches, in our example) are equivalent.7 Rating scales typically provide a line for each factor, along which the degrees are arrayed in either increasing or decreasing order. Since rating scales are the most common performance appraisal method used, they have the advantage of familiarity. Rating Scale Variations The rating scale approach has a number of variations. We'll examine the following 4 approaches in greater detail: 1. Graphic Rating Scale 2. Behaviorally Anchored Rating Scale (BARS) 3. Behavioral Observational Scale (BOS) 4. Behavioral Discrimination Scale (BDS) 1. Graphic Rating Scale The most common performance rating scale is the Graphic Rating Scale. Below is an example:

Factors. The factors or criteria used in the Graphic Rating Scale are usually those that are organizationally important in determining performance. A single Graphic Rating Scale is typically used for a variety of jobs within the organization, so the criteria/factors are organizational instead of job-related. At best, the factors/criteria are outcomes (quantity of work) or behaviors (adaptation to change). At worst they are personal characteristics (good personality). The degree statements can also range from descriptions (such as those in the example above) to simply a scale from "Most" to "Least" with no explanation of what these terms mean.8 Advantages. The Graphic Rating Scale has the advantage of being applicable to a large part of the employees of an organization. When well designed, the Graphic Rating Scale provides a clear definition of the criteria the organization considers to constitute "good" performance. This definition enables managers to discuss the relative performance of an employee against a known standard.

2. Behaviorally Anchored Rating Scale (BARS)

The Behaviorally Anchored Rating Scale (BARS) is job-specific, or at least occupationally specific. BARS was developed to eliminate the worst inherent features of graphic rating scales. The factors and the degree statements are arrived at through a complex system in which a group of experts who know the content of the job sort out behavior statements.9 The format itself is not different from that of a graphic rating scale, except that the BARS dimensions and steps have been carefully arrived at for each job. Below is an example of a behaviorally anchored rating scale.

Source: Beatty and Schneier, Personnel Administration, 1981; Addison Wesley Publishing Co. Inc., Reading, MA; p. 129, Form 8. Reprinted with permission. Other Scales Research into BARS led to 2 other types of rating scales:

behavioral observational scale (BOS) behavioral discrimination scale (BDS)

These scales are different from BARS, but they are developed the same way. 3. Behavioral observational scale (BOS) The Behavioral Observational Scale (BOS) states a behavior and asks the rater to indicate where an employee's performance falls on a scale. Below is an example of a BOS.

From: Performance Appraisal and Review Systems, by Carroll and Schneier. Copyright 1982 by Scott, Foresman and Company. Reprinted by permission.

4. Behavioral Discrimination Scale (BDS) The Behavioral Discrimination Scale (BDS) is a more complex scale, for which the rater must judge 3 areas: 1. opportunity to exhibit the behavior 2. satisfactoriness of exhibiting the behavior 3. level of performance of the behavior9

Rating Scale Disadvantages We've covered several variations of rating scales and discussed how each of the variations differs from one another. Now let's examine some common disadvantages of rating scales:

commonality performance factors total score weighting errors number of scales

Commonality Rating scales require managers and employees to all agree about the meaning of factors and their degrees. If there is no such agreement, the 'standard' is an illusion and becomes a disadvantage. Performance factors The advantage of commonality can also be seen as a disadvantage if in fact there is a great deal of difference among the performance factors required in different jobs. Different jobs may really require that different factors be used. This is a basic argument for the use of BARS. Total score

The summation of a number of factors always assumes that a deficiency in one can be made up for by strength in others. Where this is not true, the summated scores are not useful. Weighting Weighting is a problem in rating scales. If factors overlap in the behavioral domain that they measure, then some dimensions of performance may be inadvertently over weighted. Since the actual weighting of the scales may be quite different from the weights specified when the system was developed, weights should be applied after the ratings have been completed and checked statistically. Errors The most common criticisms of rating scales, particularly graphic rating scales, are the set of constant errors that occur in rating. These errors include:

degree central tendency halo effect proximity

Degree. Degree errors occur when everyone is rated too leniently or too severely. Central tendency. In central tendency errors, the rater overuses the middle of the scale making it hard to distinguish among employees. Halo effect. Raters tend to have a global impression of an employee, and this impression tends to color how the rater rates all factors. Different levels of performance on different factors are not recorded. Proximity. Proximity can occur when raters assume that because 2 factors are next to each other on the ratings form, they are likely to correlate. Logical errors occur when the rater assumes that two factors are similar and should therefore be rated the same.10 Number of scales

A criticism from BARS advocates is that using a single rating scale over a large number of jobs is not useful, since the behavioral dimensions of jobs differ greatly. This is a dilemma. While the criticism is valid, the solution that of having many different scales has its own difficulties. It is expensive and time-consuming to develop a series of rating scales. Employing many scales also makes it more difficult to directly compare employees in different jobs. End Result Despite all of these criticisms of rating scales, the Human Resource Department must be able to use rating scales in a pay-for-performance program, since this is the most common performance appraisal program used by organizations. Where there are common factors and a summated score, the task is easy, if not always accurate. All employees can be placed upon a single ranking, making the performance axis complete. If constant errors are prevalent in the rating process, the responses may need to be statistically spread out on the scale. The best results can be obtained if:

unambiguous descriptions of factors and degrees are developed evaluations are NOT shared with the ratee raters have been trained

Reducing Errors Several methods are available to reduce errors in rating scales, including:

factor choice behavioral checklists critical incident method

Factor choice A focus on behavior and/or results improves the actual observation ability of the raters; they can focus on what they have observed. Systems that include the raters in their development have a better chance of success. Behavioral checklists

Raters need to record, NOT evaluate. You can encourage this by having raters use behavioral checklists. Behavioral checklists are a set of statements about the behaviors that an employee might engage in on the job. The rater's task is to indicate whether the employee does or does not engage in the behavior. This puts the rater in the position of recording behavior more than evaluating it, and should lead to more reliable reporting. A behavioral checklist can be made more complex by requiring the rater to indicate how much the employee engages in the behavior. Forced choice Forced choice is a special form of a behavioral checklist. In the forced choice method the rater is presented with a set of behaviors and is requested to choose the most descriptive and least descriptive behaviors of the employee. In a set of 4 items, 2 appear favorable and 2 unfavorable. However, only 1 of the favorable items adds to the total score and only 1 of the negative ones detracts from it. The value of the items is determined statistically by an item analysis of successful and unsuccessful employees. The scores are not known to the rater, who is in essence rating blind. This last feature is intended to reduce the constant errors of rating scales. Origin. The armed forces developed the forced choice method where the problem of leniency had led to everyone being rated excellent. Advantages. The forced choice method reduces error and has the advantage of having the rater record rather than evaluate. Disadvantages. The secrecy feature of forced choice leaves raters questioning how they in fact rated the employee. This is uncomfortable and leads to resistance to and subversion of the method. As a method of providing feedback to the employee, forced choice is not useful, since neither rater nor ratee knows what the important behaviors are. In fact, the armed forces have abandoned its use.

In the end, any behavioral checklist that evaluates employee behaviors so as to provide an overall score would be useful in establishing the relative performance of employees. Critical incident method The critical incident method is another method developed to overcome the constant errors of rating scales. The critical incident method involves determining those behaviors that are critical to success or failure on the job. When the rater observes these behaviors in the employee, the rater records them, along with the date, and places the data in the employee's performance record. An informal version of this method is often used in other performance appraisal formats, where the rater is asked to indicate what the person's job is and how well the person is doing it. In some cases this format is just one part of the system, but in others it constitutes all of it. The newer methods of performance appraisal, such as BARS and BOS, use critical incidents as their criteria. Advantages. The amount of observable information that is available for feedback and judgments is the primary advantage of the critical incident approach and the accompanying performance record. Equally important, the critical incident method gives the manager a means of observing and encouraging employees. Disadvantages. The usefulness of the critical incident method in a pay-for-performance program, however, may be minimal. Typically, the critical incident method does not offer any way to summate the rating of an employee, so the ranking of employees would be a qualitative exercise. The critical incident method is costly to develop, install, and operate, since this method requires managers to keep a record on each employee. The critical incident method's record keeping requirements can foster negative feelings among employees that big brother is watching them. Management by Objectives

Management by Objectives (MBO) is an alternative method to behavioral checklists and the critical incident methods, which focus on behavior. Also known as appraisal by objectives, MBO involves comparing the employee against a standard of expected results. MBO requires 3 things: 1. a set of clearly defined goals 2. participation of both manager and employee in setting the goals 3. feedback to the employee as to how well he or she is progressing toward the goals11 Theoretically, MBO should be an effective method of appraising and managing employees. The principles of MBO virtually coincide with E.A. Locke's goal theory of motivation.12 From a practical standpoint, it is job outcomes that are important to an organization, for it is these outcomes that the organization probably wishes to pay for. So why doesn't MBO always work? There are a number of practical problems that keep MBO from working. These problems include:

explaining goals trust levels between manager and employee feedback to the employee the economy may be too dynamic to set objectives and have them mean anything in a month, much less six months

Explaining goals. In applying MBO, it is difficult to make goals clear and explicit. Not all goals that are important can be neatly defined (such as qualitative goals). Another problem with explaining goals is that focusing on particular goals may lead employees to ignore other parts of the job. In dynamic environments, vagueness in goals often gives employees the maneuverability needed to get the job done. Realistically, it may be futile to attempt to set goals in a dynamic environment.

Trust. MBO requires participation of manager and employee in goal setting requiring a level of trust that is hard to achieve in a situation of uneven power. The employee can perceive joint goal setting as manipulative if the relationship with his/her manager is not good. If we assume that goal theory is operating, a natural tension exists. According to goal theory, more productivity is a result of higher goals being set and accepted by the employee alone. Time. Goal setting is a difficult task to handle within the supervisor-subordinate relationship. Participation takes a great deal of both parties' time. One or both may feel the time can be better spent. Feedback. The information required to provide feedback to the employee may not be developed in the organization or may be impossible because of the nature of the task. MBO also assumes that the outcomes of work are the only important variables to consider in defining good performance. Yet often, how the work is done is as important as what is accomplished. The 'how' variable is hard to factor into MBO programs. MBO conclusions MBO is not extremely useful as a performance appraisal method for a pay-for-performance program. There is no way other than qualitative judgments to decide who is doing better or worse, other than accomplishing or not accomplishing goals. This leads to a nominal measurement, but more is needed for a pay-for-performance program to operate. MBO is much better suited to bonus or incentive systems, the topic of DLC Course 75: Creating

an Incentive Pay Plan.

Employee Comparison Systems Now that we have discussed how to conduct performance appraisal based upon standards, we'll look at another approach, in which employees are compared against each other to come up with a performance rating.

Employee comparison systems compare employees directly with each other and not against any standard of performance. This gives an organization a relative positioning of all rated employees. Although it is possible to rate employees against each other on a number of factors, employeecomparison systems typically rely on a global evaluation of employees. We'll now take a look at 4 different employee comparison systems: 1. Rank-order rating 2. Alternation ranking 3. Paired-comparison ranking 4. Forced distribution system 1. Rank-order rating Rank-order rating is the simplest form of employee comparison. In rank-order rating the rater is required to rank all the employees from best to worst. Ranking is typically facilitated by providing raters with a pack of cards, one for each employee. The rater then numbers the cards in sequence. Rank ordering works best where all employees occupy similar jobs. Large engineering organizations are likely to use this kind of ranking for all their engineers. Advantages. Rank order rating has the advantage of simplicity. It also is not unrealistic, since, as discussed, most uses of rating scales have a global impression that influences their ratings anyway. Disadvantages. It is difficult to keep the performance of many employees in mind at one time. Ranking systems often involve a number of raters ranking their employees and then amalgamating their lists into a master list. This can be done by having all raters meet together with their manager acting as arbitrator. 2. Alternation ranking Alternation ranking requires the rater to indicate the best and the worst performers. Then the rater records the next best and the next worst, and so on until all employees have been rated.

Advantages. Alternation ranking avoids central tendency. Disadvantages. In situations where there is a small gap between individual performance levels, the altering between "best" and "worst" creates an illusion of a wide disparity between performance levels when none may exist. 3. Paired-comparison ranking In paired-comparison ranking, one employee (Employee A) is paired with another employee to achieve a ranking based on predetermined criteria. After the results of this pairing are recorded, the same employee (Employee A) is paired with another employee and is ranked again. This continues until Employee A has been paired with all other employees. Then, the next employee (Employee B) is compared to all employees in the same manner. This continues until all employees are compared to all other employees. Advantages. Paired-comparison ranking is more accurate, especially when multiple raters are used. Disadvantages. This ranking method is cumbersome to use when appraising large numbers of employees. Paired-comparison ranking is more time consuming than most appraisal techniques. Forced Distribution System A major variation of employee-comparison systems is the forced distribution system. Here the rater distributes all employees among finite performance categories, so a prescribed percentage of employees are in each category. An example of a typical distribution:

Forced distributions systems can be set up in various ways. The following is a sample of some companies and their method for distribution:13

General Electric: Top performers 10%; Middle 80%; Bottom 10% Microsoft: 1-5 rankings, with most employees between 2.5 4.5 Hewlett-Packard: 1-5 rankings (5 is highest ranking) with 15% ranked 5; 5% ranked 1; and various percentages for 2, 3, and 4 rankings

Ford Motor Company: rankings A-C (no ranking lower than a C) with 10% A's, 80% B's, and 5% C's

Forced distribution may be done globally or for a number of factors. This system assures that employees form a normal distribution in terms of their performance. Advantages. The forced distribution system is simplistic and accurate in that it is easier to make relative judgments than to make a comparison against a standard. Forced distribution takes advantage of, instead of fights, raters' tendency to make a global judgment. In pay-for-performance systems, forced distribution works well. Forced distribution provides a relative positioning of employees that can be compared with their relative position in the pay range. Disadvantages. Like most employee-comparison systems, forced distribution falls short in certain areas:

lack of information employee comparability raters' knowledge large companies supervisory expectations lawsuits

Lack of information. Other than the ranking itself, there is a definite lack of information to provide a basis for discussion with the employee. Where a global ranking is used, there is no agreement as to what the appropriate criteria are. The forced distribution system provides a

standard, but it is the group average. This offers little help to the manager, who needs to discuss performance with an employee. Employee comparability. A disadvantage shared by all employee-comparison systems is that of employee comparability. This has two aspects: 1. Are the jobs sufficiently similar? 2. Are employees rated on the same criteria? It is likely that one employee rates high for one reason and another rates low for an entirely different reason. Raters' knowledge. Raters do not always have sufficient knowledge of the people being rated. Normally the immediate supervisor has this knowledge, but oftentimes in large ranking systems, supervisors two and three levels removed have to do the rating. Large companies. The size of the organization poses a problem. The larger the number of employees to be ranked, the harder it is to do so. However, the larger the number in the group, the more logical it is that there is a normal distribution. Supervisor expectations. If a manager knows that some employees must be rated below average, he/she may start thinking of those employees that way. This leads to a self-fulfilling prophecy: the manager now treats the employees as if they cannot do well, and the employees respond by not doing well.14 Lawsuits. Employees of Microsoft, Ford, and Conoco have filed lawsuits that allege the forcedranking systems used by their companies were biased against certain groups. In each case, the disaffected employees represented a different group of workers: U.S. citizens at Conoco, blacks and women at Microsoft, and older workers at Ford.15 ADMINISTRATION OF PERFORMANCE APPRAISAL The Human Resource Department typically designs the performance appraisal system and oversees its operation. This is not an easy task. Managers do not like conducting performance appraisal and will avoid it unless required to do so. To the extent that managers see the process

as belonging to Human Resources and offering little help in managing their employees, managers not only avoid it, but also resent having to take the time to perform the appraisals. Besides the design of the performance system itself, let's take a look at the major administrative questions:

When should the appraisal be done? Who should perform the appraisal? How can the performance appraisal process be improved?

When Should The Appraisal Be Done? Part of the administration of performance appraisal is deciding when the appraisals should be done. This involves determining the timing of the appraisals, whether to tie performance appraisals to pay increases, and how frequently performance appraisals should be done. Timing Ideally, performance feedback should occur as the job is being performed. When adequate feedback occurs, a great deal of the emphasis on the performance interview is unnecessary. There is also the question of when formal ratings need to be done in order to be coordinated with pay increases. The Great Debate: Tying Performance Appraisal to Pay Increases The two should take place as close in time as possible. This way the performance-reward connection is clear in the For minds of the employees. The greater the time lag, the less likely an employee will see that what he or she did was related to the pay increase. Close timing makes it difficult to create a meaningful change Against in the employee's behavior because he/she may be defensive if there is negative feedback. Pay-for-Performance: A current performance appraisal MUST be

available for all employees when the time comes to allocate increases under a pay-for-performance program. Frequency Ratings can be improved by increasing their frequency. Since current events outweigh past events, the more often ratings are given, the less important events will be lost in the shuffle. Usually 6 months is the longest that should elapse between ratings. Keep in mind that some jobs have a long time span, so a 6-month limit may not make much sense.

Raters A rater is an individual responsible for assessing performance. Typically, the rater is a first-line supervisor or other designated management official. But raters do not have to be supervisors, or even management officials. Ideally, raters should be the people who best know the employee's performance and are in a position to evaluate performance. Possible raters are:

Supervisors Coworkers Subordinates Customers

Supervisors Almost all formal appraisals in organizations are made by the employee's supervisor. However, in some instances this is not the person who best knows what the employee is doing, or how well. In addition, performance appraisal puts the supervisor's employees in a competitive position while the supervisor is trying to obtain cooperation and coordination.

At best, performance appraisal is an uncomfortable thing for supervisors. Supervisors often feel like they are "playing God." And because employees know that pay is a direct outcome of this evaluation, they will put as much pressure as possible on supervisors to receive a positive rating. In the end, the best argument for having the supervisor do the rating is that the organization desires for the manager to be seen as having the power of reward and punishment. Coworkers Using peers as raters provides a great deal of reliability, validity, and freedom from bias in ratings. Example: Johnsonville Foods, a Wisconsin-based sausage

manufacturer, installed a successful employee-designed peer-review process to determine performance. This process allowed the supervisors to feel "more like facilitators and less like judges."16 In this peer-review program, at the beginning of each month, each employee writes a contract that includes how that employee will achieve his/her goals for the next month and 6-month goals as well. Employees then post the contracts on the company's electronic bulletin board system so other employees can comment on them. Employers also send their contracts to 3 customers so that the customers can provide feedback as well.

At the end of each month, company teams meet to review the contracts and the comments received. Bonuses are then distributed to each team, based upon company performance. But it is up to each team to decide how much each individual receives of the bonus. If an individual has met their contract, then they may get a full bonus (which equals between 10-25% of base pay). If the employees do not meet the contract's goals, they don't receive any bonus, but this is rare. To find out more about this program, please click here. Peer-review programs are often resisted because supervisors do not like giving up power. Employees resist peer-review programs because they are concerned that peers will have their own interests at heart, especially if all employees perceive the situation as a zero-sum game.

Choosing Multiple Raters Research evidence tends to support the multi-rater approach. Averaging the ratings of several people is more reliable than basing pay adjustments on the rating of one person.

The problem often is to determine who the second or third rater should be. It is important to identify what behaviors a prospective rater knows about. Subordinates Subordinates of the employee may serve as raters. Each subordinate has unique information that might be useful in a complete evaluation, but by itself might be incomplete or biased. Customers Customers are often an untapped resource in performance rating even though customers may have the best perspective on the behaviors of employees. Example: Sears replaced a commission-based system for its automobile repair stores in California with a bonus plan, based on customer service scores. Under the old system, mechanics were paid based on the amount they charged customers. This led to widespread evidence of consumer fraud when employees recommended unneeded repairs. The new system gives incentives based on customer service scores, which has led to rapid improvements in customer service.17 Top supervisors Higher-level supervisors may be useful raters, assuming they have a chance to observe the employee's behavior. Like the other sources, the higher-level supervisor offers the advantage of a different perspective. A supervisor one or two levels removed is less likely to be influenced by immediate events and is more likely to look at the employee's customary behavior or beliefs. 360-Degree Feedback The process of using multiple raters all of whom interact with the person being rated has come to be known as 360-degree feedback.

The 360-Degree Feedback method assesses employee performance from 5 perspectives:

supervisor coworker self customer subordinate

The 360-degree method is most useful for managers and those that deal with the public. The different perspectives provide more information about the ratee. Organizations that use this method believe that it has improved the performance of their managers. Clearly this method requires a great deal of effort to implement and administer. 360-degree programs are more commonly used for employee development. Some companies use 360degree feedback as the sole determining factor in deciding pay increases, while others use 360-degree feedback in conjunction with other appraisals (such as the appraisal completed by the employee's supervisor) to set pay increases. Rater Training Rater training is a powerful device for improving performance ratings. Raters must understand the meaning of the factors and the definitions of scale positions. Raters should also practice rating in order to improve reliability. There are 2 types of training programs:

Rater-Error Training (RET) Rater-Accuracy Training (RAT)

Rater-error training Rater-error training (RET) is directed at reducing the response errors discussed earlier, such as the halo effect. Studies show that RET does reduce error and bias, but does not improve accuracy. Rater-accuracy training

Rater-accuracy training (RAT) is aimed at developing observational skills that improve the accuracy of ratings. RAT is better than RET at improving accuracy but one study found that it was not superior to a control group given no training.18 This study seemed to indicate that with training, raters can reduce errors but not necessarily improve their overall accuracy. Despite these mixed results, the weight of evidence is that rater training improves performance ratings. Given the importance of good ratings in a pay-forperformance program, rater training seems absolutely necessary.

Market Data Ordinarily a pay-for-performance program uses market data simply to determine the amount by which the entire wage structure will be adjusted each year. In this exercise, however, we will show you how to use labor market data from ERI's Salary Assessor (SA) software to directly determine the appropriate salary for a group of employees that reflects:

current market rates experience level of each employee performance level of each employee

Background Jim Stillwell is the Manager of the Chemical Extrusions Development Department of the Rocket Manufacturing Co. located in Bakersfield, California. The company has annual revenue of $500 million. Mr. Stillwell supervises a number of Chemists. Each year the wage rates are reviewed for all employees after the performance review cycle has been completed. Forced distribution The performance review system at Rocket Manufacturing Co. is a forced distribution system for the technical/professional staff. Each professional is evaluated and placed in a distribution. At the end of the forced distribution each employee is assigned a percentile ranking. The company is aggressive in paying market wages for its employees. In addition, the company values and wishes to reward the experience and performance of the employees. The table

below shows the experience, performance level and current total compensation (base salary plus incentive) for two of Mr. Stillwell's employees. Years Experience 12 of Performance Percentile 90th Current Compensation $85,462 Total

Name Shirley Cantwell Mark Tipton

25th

$61,680

This interactive exercise will walk you through the process of updating the salaries for these two employees. Click on the Load Step-by-Step button to begin. Now it's your turn to solve a real-life problem! Using ERI's Salary Assessor (SA) software, analyze the following problem statement. Note: If you have a copy of ERI's Salary Assessor software loaded on your machine OR if you have ERI's Platform Library/DLC Wizard DVD (which houses the SA software), you can simply open this application now. If you don't have the DLC Wizard installed, you may download this FREE software from the Instructions page of this course. (Click on Instructions in the left-hand toolbar.) After downloading and opening the Salary Assessor software, you can return to this page by clicking on Access Any Page in the left-hand toolbar. Then click on the link to the Your Turn page. Problem Statement You are planning salary increases for your organization. Find the correct pay level for the employee below. He is a Chemist at the Manufacturing Department of Rocket Manufacturing Co., in Bakersfield, California. Rocket Manufacturing has revenues of $500 million a year.

This company has an eSIC code of 3080, and you'll want to use today's date as your salary planning date. Years Experience 7 of Performance Percentile 65th Current Salary $70,110 Base

Name Leroy Gerhart

SUMMARY The purpose of having rate ranges is so that organizations can reward employees for factors other than the value of the job. Performance is a primary factor for which organizations wish to reward employees. This course presented a method for clearly rewarding performance within the confines of the typical wage structure, through the use of a pay-for-performance program. Take Caution Such programs need to be approached cautiously. Pay-for-performance programs are not always possible nor desirable. The advantages and the costs of a pay-for-performance program should be carefully considered before installing such a program. Operation Pay-for-performance programs operate by relating all wage increases to employee performance. The goal is to correlate the position of the employee in the rate range with their relative position on a performance scale. Result Pay-for-performance results:

employees whose wage rate is at the low end of the range and whose performance is high receive large increases

employees whose performance matches their location in the rate range receive average increases those whose performance falls below their location in the rate range receive no increase

These results differ greatly from the usual distribution of wage increases given by an organization. Performance Appraisal Pay-for-performance programs require the organization to have a good performance evaluation system. Where possible, performance should be measured. However, this requires that there be some clear outcome from the job that is the result only of the labors of an individual. This is often not likely in an organizational setting. Performance appraisal is the most common form of performance evaluation. Methods of appraisal involve comparing the employee's performance to some standard:

some form of rating scale standard of performance goals and objectives other employees' performance

No one method can be said to be superior, and all methods have problems that put their use into question. Constant attention to performance appraisal results is necessary if employees are to view the pay-for-performance program as a fair and reasonable way to allocate increases. Rater training and multiple raters should be used, if possible; and employees should have a say in the design and administration of performance ratings. This will increase acceptance of your organization's pay-for-performance plan.