HR 1 – Recruitment and Selection (Semi-Final)

Topic 1: REFERENCES AND RATINGS

The Eye of the Beholder

I. Intended Learning Outcomes

At the end of the lesson, the students should be able to:

1. Know the difference between an interview and references and ratings
2. Discuss ratings and differences
3. Explain the importance of reliability and validity
4. Be familiar with the reasons of poor validity
5. Know the objectives of doing a performance appraisal.
6. Identify the factors or matrix in performance evaluation and the different
performance appraisal methods.

II. Discussion/Lecture/Suggested Readings

INTRODUCTION

Interviews allow the applicant to speak for him - or herself, on the principle that
the best way of finding out about someone is to ask them.

References and ratings work on a different principle - that the best way of finding
out about someone is to ask someone who knows him or her well (former
employers, schoolteachers, colleagues or fellow trainees). They have seen the
candidate all day every day, perhaps for years, and can report how he or she
usually behaves, and what he or she is like on 'off days'.

REFERENCES

 The Price-Waterhouse-Cranfield survey found references widely used throughout

Westen Europe.
 Most American employers take up references on new employees
 References may be structured (questions, checklists, ratings) or unstructured
(Tell me what you think of John Smith in your own words)
 Unstructured are still widely used on both sides of the Atlantic.
 Moser and Rhyssen note that references can be used for two different purposes -
either to check facts, or to get an opinion on the quality of the candidate's work.
RELIABILITY

 American research suggests that references are unreliable.

 Referees agree among themselves very poorly about applicants for (US) civil
service jobs.
 References given by supervisors bore no relation to references given by
acquaintances, while references given by supervisors and co-workers (who both
see the applicant at work) agreed only very moderately. Agreement between
raters is always fairly low.
 Murphy and Cleveland make the point that different people see different sides of
the target person, so would not all be expected to say the same thing about him
or her. And if they did all say the same thing, what would be the point of asking
more than one person?
 Some very early British research approached the problem from a different angle,
and found references more useful. In the 1940s, the Civil Service Selection
Board (CSSB) collected references from school, college, armed services and
former employers. The CSSB used five or six references, not two or three, which
may increase reliability.

VALIDITY

Morsel and Goheen reported several investigations of the Employment

Recommendation Questionnaire (ERQ), a structured reference request form that was
developed by the US Civil Service. The ERQ covers the following:
 occupational ability-skill, carefulness, industry, efficiency
 character and reputation
 whether the applicant is specially qualified in any particular branch of trade
in which he or she seeks employment
 whether you would employ him or her in a position of the kind he or she
seeks
 whether the applicant has even been discharged from any employment to
your knowledge, and if so, why.

REASONS FOR POOR VALIDITY

Leniency - numerous researches report that most references are positive. Early
research by Mosel and Goheen found that ratings were highly skewed, with
'outstanding' or 'good' opinions greatly outnumbering 'satisfactory' or 'poor' opinions.
Referees are usually nominated by the candidate, who will obviously choose someone
who is likely to give them a good reference.

Leniency - These days many employers fear that an unfavourable reference may result
in a libel suit. However, if referees are reluctant to say anything negative, references
must remain a poor source of information. Manager's can observe employees'
shortcomings but have no incentive to communicate them to others, and many reasons
(fear of creating ill feeling, not wanting to admit that they have poor employees because
this reflects on their management, etc.) Murphy's argument implies that references
could be an excellent source of information, if only referees could be persuaded to part
with it.

Idiosyncrasy - Baxter et.al. (1981) searched medical school files to find 20 cases
where same two referees had written references for the same two applicants. If
references are useful, what referee A says about applicant X ought to resemble what
referee B says about applicant X. Analysis of the qualities listed in the letters revealed a
different and much less encouraging pattern. What referee A said about Applicant X did
not resemble what referee B said about applicant X, but did resemble what referee A
said about applicant Y. Each referee had his or her own idiosyncratic way of describig
people, which came through no matter who he or she was describing.

Idiosyncrasy - The free-form reference appears to tell you more about its author than it
does about its subject. Aamodt, Nagy and Thompson (1998) confirm this, showing that
a referee who describes one candidate in terms of conscientiousness will tend to
describe another in terms of conscientiousness as well. Differences in reference writing
may reflect personality (of the author, not the candidate). Judge and Higgins showed
that happier people write more favourable references.

Free-form references - It is much more difficult to assess the validity of the free-form
reference, as it is complex and unquantified. There is obviously much more to reference
letters than global favourability.

Forced-choice format - Carroll and Nash (1972) used a forced-choice reference rating
form. Items in each pair are equated for social desirability, but only one statement
predicts job success:
 has many worthwhile ideas/completes all assignments
 always works fast/requires little supervision

Scores predicted performance ratings 4 months after hire quite well in university clerical
workers.

Key-word counting - Peres and Garcia (1962) factor-analysed data from 625 reference
letters for engineering applicants, and found five factors that distinguished good from
poor candidates. Key-word counting can only be done with free-form references, and
has difficulties with the documented idiosyncrasy of reference writers. As Aamodt et al
(1998) showed, some referees tend to describe people in a baseline for each referee.
However, scanning software would make key-word counting techniques much more
feasible.

Relative percentile method - McCarthy and Goffin (2001) have described the relative
percentile method (RPM), and report that it gives substantially better validity than
conventional ratings in a Canadian military sample. The technique may work by allowing
people to be lenient, but also to differentiate at the top end of the scale, giving someone
whom they consider to be really responsible.
WHAT IS THE PURPOSE OF THE REFERENCE?

 We have been assuming that employers use references to evaluate potential

employees, although research suggests that this is unlikely to be very successful.
 The reference gives the employer control over the employee, even after he or
she has left. The employer can refuse a reference or give a bad one, and so
block the employee's chance of getting another job. To be certain of obtaining a
good reference, the employee must avoid doing anything that might offend the
employer.
 The reference system is used by employers to control existing staff, not to
assess new staff.

RATINGS

 American personnel practice uses ratings a lot - far more than UK personnel
work. In personnel selection, ratings can be used as the predictor, but are more
often used as the criterion. Ratings that are used as the predictor in selection are
usually made by external referees, or by the candidate's peers. Criterion ratings
were traditionally made by a supervisor or manager, but increasingly ratings by
co-workers and subordinates are being used as well (so-called '360-degree
feedback').
 Ratings are used for regular performance appraisals in American industry, so
they are big business. Performance appraisals often determine promotion, salary
or even survival, so rating systems come under keen scrutiny.

Objectives of Performance Appraisal

 It provides information upon which promotion, transfer, demotion, layoff,

discharge, and salary decisions can be made. It could justify reward decisions
including merit increases, promotions, and other forms of rewards.
 It provides an opportunity for the supervisor and his/her subordinates to review
and identify their strengths and weaknesses or work-related behavior. This is turn
allows both of them to develop a plan for correcting any deficiencies that the
appraisal might have unearthed or reinforce the things the subordinate does
right. This feedback clarifies for employees the job expectations held by their
supervisor.
 It forms the basis in identifying the training needs of employees as well as
evaluating the success of training, thus, development initiatives are not based on
opinions but rather on results.
 It helps in the firm’s career planning process because it provides a good
opportunity to review the person’s career plans in light of his/her opportunity for
career progression.
 It allows easy monitoring and supervision.
  It helps evaluate the individuals share relative to the teams contribution in
achieving the organizations goal.
 It provides information to evaluate effectiveness of selection and placement
decisions.

Factors or Matrix that can Help Measure Employee Performance

1. Quantity – the number of units produced, processed or sold is a good objective

indicator of performance.
2. Quality - the percentage of work output that must be redone or is rejected is one
indicator of quality. In a sales environment, the percentage or inquiries converted
to sales is an indicator of salesmanship quality.
3. Timeliness – how fast is work performed; it might be the number of units
produced per hour.
4. Cost-Effectiveness – the cost of work performed should be used as a measure
of performance only if the employee has some degree of control over costs.
5. Absenteeism/Tardiness – an employee is obviously not performing when he or
she is not at work.
6. Creativity – supervisors and employees should keep track of creative work
examples and attempt to quantify them.
7. Adherence to Policy – this may seem to be the opposite of creativity, but it is
merely a boundary on creativity. Deviations from indicate an employee whose
performance goals are not well aligned with those of the company.
8. Gossiping and Other Personal Habits – they may not seem performance
related to the employee, but some personal habits, like gossiping or rumor
mongering, can disrupt job performance and interfere with the performance of
others.
9. Personal Appearance/Grooming - most people know how to dress for work,
but in many organizations, there is at least one employee who needs to be told.

Who Should Evaluate Performance?

1. Manager/Supervisor Appraisal – hierarchical arrangement of formal authority in

most organizations gives the supervisor or the manager legitimate authority to
evaluate subordinates.
2. Self-Appraisal - the employee appraises his or her own performance, in many
cases comparing the self-appraisal to managements review.
3. Subordinate Appraisal – provide unique information because subordinates
know better than anyone else whether leadership is good or bad.
4. Peer appraisal – research on peer evaluation has found to be predictive success
and yielded good reliability and validity. This method is based on the assumption
that co-workers are most familiar with an employee’s performance.
5. Customer/Supplier - customer, vendor, or suppliers can be potential
evaluators. This kind of appraisal would be more relevant for service-oriented
companies such as banks, where the inputs provided by external
 6. Team Appraisal - similar to peer appraisal in that members of a team, who may
hold different positions, are asked to appraise each other’s work and work styles.
7. Assessment Center – The employee is appraised by professional assessors
who may evaluate simulated or actual activities.
8. 360-Degree or Full-Circle Appraisal – the employee’s performance appraised
by everyone with whom he or she interacts, including managers’ peers,
customers, and members of other departments. This is the most comprehensive
and expensive way to organizational and employee needs.

Sources of Data in Appraisal

 Production Data – evaluate the degree of dependable task accomplishment by

measuring quantity and quality of performance.

 Personnel Data - type of information found in an individual’s personnel files.

 Judgment of Others - many of the spontaneous and innovative behaviors that

are important to organizational effectiveness can only be assessed by the
judgments of others and ought to be obtained in every evaluation

Performance Appraisal Methods

MULTIPLE PERSON EVALUATION METHODS

Ranking Method - ranking the employee from the most efficient to the least capable on
each trait or quality to be used in judging the employees’ performance or just simply
ranking the employee from best to worst.

Paired Comparison Method – consists of asking an evaluator to consider only two

names at one time and to decide who is better.

Forced Distribution - is a method of performance appraisal that ranks employees

through forced distribution.

Individual Evaluation Methods

1. Critical Incident Method – is a method in which the manager writes down

positive and negative performance behavior of employees throughout the
performance period.
2. Checklist and Weighted Checklist – is a set of objectives or descriptive
statements. This method describes a performance appraisal method where the
rater, familiar with the jobs being evaluated, prepare a large list of descriptive
statements about effective and ineffective behavior on jobs.
3. Graphic Rating Scales - this is the oldest and most widely used for
performance appraisal.
4. Behaviorally Anchored Rating Scales - this method is used to describe a
performance rating that is focused on specific behaviors or sets as indicators of
 effective or ineffective performance. It is a combination of the rating scale and
critical incident techniques of employee performance evaluation.
5. Management by Objective (MBO) – is a process in which managers and their
subordinates set objectives for the employee to achieve within a specific rating
period.
6. Multi-rater Assessment or the 360-Degree Performance Feedback - is a
system or process in which employees receive confidential, anonymous
feedback from people who work around them, their immediate superior, peers,
customers, or suppliers.

III. Rev Up!

ESSAY

1. Discuss the difference between interviews and references and ratings as basis in
hiring employees.
-The difference between interviews and references and ratings as basis in
hiring employees is in interview we can see if how confident the person in
answering the question and the potential of the person to work in a job,
reference is not always accurate because when you can get an information
about the employee which is not even true there is a tendency that the
person will be disqualified, and the ratings can be used by the hiring
employee as a predictor that can be a criterion.
2. What is the importance of conducting reliability and validity?
-The importance of conducting reliability and validity is we will know if the
background of the person is accurate because in terms of reliability it will
be the consistent information about the person and it the information can
be trusted while the validity will let us know if the information of the person
is real.
3. Discuss the reasons for poor validity.
-The reason for poor validity is they give an unfair treatment because some
people give a false information to a person for two reason: whether they
hate the person or the person is their close friend.
4. What is the purpose of reference?
 -The purpose of reference is we can identify the person’s background as a
candidate to the work and we can also check the facts of the person.
5. Discuss the objectives of performance appraisal.
-The objectives of performance appraisal is we can identify the strength
and weaknesses of the employees if they are right in the job or if they
deserve a promotion because they show a good performance in work.
6. Among the (9) factors or matrix that can help measure employee performance,
choose 3 that you think are the most important and discuss the reasons why.
-Timeliness – because it shows how an employee determined to do his job
properly, hardworking and can finish his/her work in a faster way.
- Absenteeism/Tardiness – because we can see if the person is very
serious in his job if he/she will not be always absent or late.
- Gossiping and Other Personal Habits – because it can give a negative
impact to the other employee that can lead to disrupt job performance.
7. Among the individual or group of persons that will evaluate employees, choose
(2) on which you think will give an objective result and why?
- Customer/Supplier- because they are the one who complaints if the
service of the employee is not good.
- Peer appraisal- because they are the one who knows the personality of
the employee because they are familiar with the employee.
8. Among the methods of performance appraisal, choose one from Multiple Person
and one from Individual on which you think will result to an accurate result. Why?
-In Multiple Person I choose Ranking Method because we can really identify
the best and and worst employee in the company that can really give an
accurate result.
-In Individual I choose Critical Incident Method because we can really see
what are the good and bad things the employee have than in the company.

IV. References

R1. Cook, Mark and Blackwell, Wiley (2016) Personnel Selection – Adding Value
Through Peope – A Changing Pictue: 6th Editiion. 2 Penn Plaza, New York
R.2 Denisi, Angelo S. and Griffin, R.W. (2016) Human Resources, Cengage
Learning,20 Channel St., Boston, USA
R3 Gatewood, Robert D. and et,al (2016) Human Resource Selection. 8 th Edition.
Cengage Learning, 20 Channel St., Boston, USA
R4 Nikolaou, Ioanonis and Oastrom, J.K. (2015) Employee Recruitmtment, Selection
and Assessment-Contemporary Issues for Theory and Practice. Psychology
Press, 711 Third Avenue, New York
R.5 Noe, Raymond A. and et al. (2018) Fundamentals of Human Resource
Management. South-Western Cengage Learning
R7 Rue, Leslie W and et.al (2016) Human Resource Management, 11 th Edition. Mc
Graw Hill Education, 2 Penn Plaza, New York
R8 Snell, Scott A. and et,al (2013) Managing Human Resources, 17 th Edition.
Cengage Learning, 20 Channel Center St., Boston, USA