Psychological Assessment

Seniors’ In-Service Training

Worksheet #8

Name: Bula, Tabac, Trocio

Date: August 7, 2020

KEY TERM EXERCISE:

Key Term Definition
Utility
usefulness or practical value of testing to improve
efficiency

Psychometric Soundness
reliability and validity of a test

Cost
disadvantages, losses, or expenses in both
economic and noneconomic terms

Benefits
profits, gains, or advantages; can be viewed in
both economic and non economic terms

Utility Analysis
a family of techniques that entail a cost–benefit
analysis designed to yield information relevant to
a decision about the usefulness and/or practical
value of a tool of assessment

Cut Score
(usually numerical) reference point derived as a
result of a judgment and used to divide a set of
data into two or more classifications

A reference point in a distribution of test scores

Relative Cut Score used to divide a set of data into two or more
classifications that is set based on norm-related
considerations rather than on the relationship of a
test scores to a criterion

A type of cut score set with reference to the

Norm-referenced Cut Score
Fixed Cut Score
performance of a group (or some target segment
of a group)
A reference point in a distribution of test scores
used to divide a set of data into two or more
classifications typically set with reference to a
 judgment concerning a minimum level of
proficiency required to be included in a particular
classification
The use of two or more cut scores with reference
Multiple Cut Scores to one predictor for the purpose of categorizing
testtakers

A cut score is in place for each predictor used.

Multiple Hurdle Each cut score is designed to ensure that each
applicant possesses some minimum level of a
specific attribute or skill
Key Term Definition
Compensatory Model of Selection
an assumption is made that high scores on one
attribute can, in fact, “balance out” or compensate
for low scores on another attribute

Angoff Method setting fixed cut scores can be applied to

personnel selection tasks as well as to questions
regarding the presence or absence of a particular
trait, attribute, or ability

Known-Groups Method
collection of data on the predictor of interest from
groups known to possess, and not to possess, a
trait, attribute, or ability of interest

Item-Mapping Method
technique that has found application in setting cut
scores for licensing examinations;
arrangement of items in a histogram, with each
column in the histogram containing items deemed
to be of equivalent value

Bookmark Method
IRT-based method of setting cut scores that is
more typically used in academic applications;
training of experts with regard to the minimal
knowledge, skills, and/or abilities that testtakers
should possess in order to “pass.”

Method of Predictive Yield

technique for setting cut scores which took into
account the number of positions to be filled,
projections regarding the likelihood of offer
 acceptance, and the distribution of applicant
scores

Discriminant Analysis Shed light on the relationship between identified

variables and two naturally occurring groups

PSYCHOMETRIC SOUNDNESS

The following are different coefficients that determine psychometric soundness.

Describe each one in the table below.

Coefficient Description

Index of Reliability how consistently a test measures what it

measures

Index of Validity whether a test measures what it purports

to measure

Index of Utility the practical value of the information

derived from scores on the test;
use in a particular situation helps us to
make better decision and is cost-effective

TESTING YOUR KNOWLEDGE

Indicate whether the following statements are true or false by putting (T) or (F) right next
to the sentence.

1. Test scores are said to have utility if their use in a particular situation helps us make better
decisions. T
2. Reliability sets a ceiling on validity. T
3. The higher the criterion-related validity of test scores for making a particular decision, the lower
the utility of the test is likely to be. F
4. A valid test is a useful test. F
5. The endpoint of utility testing is typically an educated decision about which of many possible
courses of action is optimal. T
6. Multiple-Hurdle Selection Methods assume that an individual must possess a certain minimum
amount of knowledge, skill, or ability for each attribute measured by a predictor to be successful
in a desired position. T

ECONOMIC VS. NON-ECONOMIC COSTS IN TESTING

Fill out the table below with examples of economic and non-economic costs of testing.
 Economic Costs Non-Economic Costs

purchase test discrimination can happen

supply a blank test of protocols harm or injury to to airline passenger and crew as
a result of incompetent pilots

computerized test processing, scoring, and consequences and potential costs of failing to
interpretation from the test publisher or some diagnose the injuries
independent service

payment to professional personnel and staff insignificant results can be acquired if the test is
associated with test administration, scoring, and not culturally appropriate
interpretation

facility rental, mortgage, and/or other charges “loss of confidence”

related to the usage of the test facility

ECONOMIC VS. NONECONOMIC BENEFITS IN TESTING

Fill out the table below with examples of economic and non-economic benefits of testing.

Economic Benefits Non-Economic Benefits

greater overall company profits good work/learning environment

financial returns in dollars and cents in terms of a good university, it will produce
happy, successful graduates

reduction in the number of accidents so there will high morale

be less cost on insurance

decrease in the time needed to train workers so good reputation

the budget allotted will be given to other
departments that need it

reduction in worker turnover so less cost in reduced load on personnel loads

testing, interviews, recruitment
 COMPARING GENERAL APPROACHES TO UTILITY TESTING

Fill out the table below to compare the two general approaches to utility analysis.

Approach Description

Expectancy Data provide an indication of the likelihood that a

testtaker will score within some interval of scores
on a criterion measure—an interval that may be
categorized as “passing,” “acceptable,” or “failing.”
;
can provide vital information to decision makers;

for example, the higher a worker’s score is on this

new test, the greater the probability that the
worker will be judged successful;

the test is working as it should

Taylor-Russell Tables provide an estimate of the extent to which

inclusion of a particular test in the selection
system will improve selection;

provide an estimate of the percentage of

employees hired by the use of a particular test
who will be successful at their jobs, given different
combinations of three variables: the test’s validity,
the selection ratio used, and the base rate

Naylor-Shine Tables obtaining the difference between the means of the

selected and unselected groups to derive an index
of what the test (or some other tool of
assessment) is adding to already established
procedures;

determining the increase in average score on

some criterion measure
 PRACTICAL CONSIDERATIONS IN TESTING

Explain the following practical considerations of testing:

Consideration Why it matters in testing

The Pool of Job Applicants based on the assumption that there will be a
ready supply of viable applicants from which to
choose and fill positions;

in some jobs, there might be a surplus in talents

but some jobs require unique skills or demand
such great sacrifice that few people would reply;

may also vary with economic climate like high

unemployment periods produce many potentials;

may tend to overestimate the utility tool as such

assuming that people will readily accept the job
offered to them;

may have to adjusted downward to create a

realistic view

The Complexity of the Job Whether or not the same utility models apply to
jobs of varied complexity, and whether or not the
same utility analysis methods are equally
applicable, remain matters of debate;

the more complex the job, the more people differ

on how well or poorly they do that job

The Cut Score in Use reference is frequently made to different types of

cut scores;

like how relative cut scores depends on the

performance of the group as a whole or how fixed
cut score as that there is a specific minimum level
of proficiency required