iMOVE DIGITAL LEARNING LECTURE NOTES | PSYCHOLOGICAL TESTS: AN INTRODUCTION

PSYCHOLOGICAL TESTS: AN INTRODUCTION

Test
 refers to a device or procedure designed to measure a variable

Psychological Test
 refers to a device or procedure designed to measure variables related to psychology like
intelligence, personality, aptitude, interests, attitudes, and values
 involves analysis of a sample of behavior and mental processes
 psychological tests differ with respect to test content, format, administration procedures, scoring,
interpretation procedures, and technical quality.

Test Content
 content (subject matter) of the test will vary with the focus of the particular test. But even two
psychological tests purporting to measure the same thing—for example, personality —may differ
widely in item content

Test Format
 pertains to the form, plan, structure, arrangement, and layout of test items as well as to related
considerations such as time limits.
 Format is also used to refer to the form in which a test is administered: computerized, pencil-and-
paper, or some other form

Administration
 Refers to the manner in which the test is given to the test takers or examinees.
 Some tests are designed for administration on a one-to-one basis, which may require an active and
knowledgeable test administrator.
 Some test administration may involve demonstration of various kinds of tasks on the part of the test
taker as well as trained observation of a test taker’s performance.
 Some tests are designed for administration to groups, which not even require the test administrator
to be present while the test takers independently do what the test requires

Scoring
 A score is a code or summary statement, usually but not necessarily numerical in nature, that
reflects an evaluation of performance on a test, task, interview, or some other sample of behavior.
o A cut or cutoff score is a reference point, usually numerical, derived by judgment and used
to divide a set of data into two or more classifications. Some actions or decisions are taken
or made on the basis of these classifications like who gets admitted or not in a degree
program offerings of universities, or who gets hired or promoted among applicants in a
particular job position.

Copyright 2021 iMove Digital Learning 1

All Rights Reserved
www.imove.ph
iMOVE DIGITAL LEARNING LECTURE NOTES | PSYCHOLOGICAL TESTS: AN INTRODUCTION

 Scoring is the process of assigning such evaluative codes or statements to performance on tests,
tasks, interviews, or other behavior samples.
realibility (Consistency) validy (accuracy) of the test
Technical Quality or Psychometric Soundness
 refers to how consistently and how accurately a psychological test measures what it purports to
measure OR the reliability and validity of a test
 A test is said to be psychometrically sound for a particular purpose if reliability and validity
coefficients are acceptably high.
 An index of reliability can tell us something about how consistently a test measures what it measures
while an index of validity can tell us something about whether a test measures what it purports to
measure. People often refer to a particular test as “valid” if scores on the test have been shown to
be good indicators of how the person will score on the criterion.
*** Reliability and Validity will be further discussed in the succeeding topic on Test Construction

Test Utility or Usefulness

 refers to the usefulness or practical value that a test or assessment technique has for a particular
purpose
 In psychological testing, utility refers to the usefulness or practical value of testing to improve
efficiency
 In psychological assessment, utility is also used to refer to the usefulness or practical value of a
training program or intervention (especially clinical intervention)
 an index of utility can tell us something about the practical value of the information derived from
scores on the test. Test scores are said to have utility if their use in a particular situation helps us to
make better decisions—better, that is, in the sense of being more cost-effective.

A. Factors that affect utility

1. Psychometric Soundness
 Generally speaking, the higher the criterion-related validity of test scores for making a
particular decision, the higher the utility of the test is likely to be. However, there are
exceptions to this general rule. This is so because many factors may enter into an
estimate of a test’s utility, and there are great variations in the ways in which the utility
of a test is determined.
 it is not the case that “a valid test is a useful test.” even though a test may be
psychometrically sound, it may have little utility—particularly if the targeted testtakers
demonstrate a tendency to “bend, fold, spindle, mutilate, destroy, tamper with,” or
otherwise fail to scrupulously follow the test’s directions.

Copyright 2021 iMove Digital Learning 2

All Rights Reserved
www.imove.ph
iMOVE DIGITAL LEARNING LECTURE NOTES | PSYCHOLOGICAL TESTS: AN INTRODUCTION

2. Costs (economic, financial or budget-related factors)

 one of the most basic elements in any utility analysis is the financial cost of the selection
device (or training program or clinical intervention) under study. However, the meaning
of “costs” as applied to test utility can extend far beyond money. It includes both the
economic and noneconomic sense.
o Economic costs include the allocation of funds to purchase (1) a particular test, (2) a
supply of blank test protocols, (3) computerized test processing, scoring, and
interpretation from the test publisher or some independent service, (4) payment to
professional personnel and staff associated with test administration, scoring, and
interpretation, (5) facility rental, mortgage, and/or other charges related to the
usage of the test facility, and (6) insurance, legal, accounting, licensing, and other
routine costs of doing business
o Non-economic costs include “loss of confidence” to some services when the
assessment or testing of personnel is omitted in the selection process.
3. Benefits
 Benefit refers to profits, gains, or advantages accrued from testing.
 The cost of administering tests can be well worth it if the result is certain noneconomic
benefits
o In industrial settings, a partial list of such noneconomic benefits—many carrying with
them economic benefits as well—would include
 increase in quality of workers’ performance
 increase in quantity of workers’ performance
 decreases in time to train workers
 reduction in number of accidents
 reduction in worker turnover
o A good work environment and a good learning environment in universities result from
a well-administered admissions program that consistently selects qualified
students who will keep up with the work and “fi t in” to the environment of the
university
o Use of a diagnostic test used to make decisions about involuntary hospitalization of
psychiatric patients is a benefit to society at large, particularly from individuals who
have intent on inflicting harm or injury to others and themselves. On one hand, errors
in clinical determination made in cases of involuntary hospitalization may cause
people who are not threats to themselves or others to be denied their freedom.
Hence, the stakes involving the utility of tests in clinical diagnosis can indeed be
quite high
 In summary, psychometrically sound test of practical value is worth paying for, even
when the financial cost is high, if the potential benefits of its use are also high or if the
potential costs of not using it are high.

Copyright 2021 iMove Digital Learning 3

All Rights Reserved
www.imove.ph
iMOVE DIGITAL LEARNING LECTURE NOTES | PSYCHOLOGICAL TESTS: AN INTRODUCTION

B. Utility Analysis
 utility analysis may be broadly defined as a family of techniques that entail a cost–benefit
analysis designed to yield information relevant to a decision about the usefulness and/or
practical value of a tool of assessment.
 The endpoint of a utility analysis is typically an educated decision about which of many
possible courses of action is optimal
 If undertaken to evaluate a test, the utility analysis will help make decisions regarding
whether:
o one test is preferable to another test for use for a specific purpose
o one tool of assessment (such as a test) is preferable to another tool of assessment
(such as behavioral observation) for a specific purpose
o the addition of one or more tests (or other tools of assessment) to one or more tests
(or other tools of assessment) that are already in use is preferable for a specific
purpose
o no testing or assessment is preferable to any testing or assessment
 If undertaken for the purpose of evaluating a training program or intervention, the utility
analysis will help make decisions regarding whether:
o one training program is preferable to another training program
o one method of intervention is preferable to another method of intervention
o the addition or subtraction of elements to an existing training program improves the
overall training program by making it more effective and efficient
o the addition or subtraction of elements to an existing method of intervention
improves the overall intervention by making it more effective and efficient
o no training program is preferable to a given training program
o no intervention is preferable to a given intervention

 How Is Utility Analysis Conducted?

o Expectancy data (Tables)
 provides an indication of the likelihood that a testtaker will score within some
interval of scores on a criterion measure-an interval that may be categorized
as “passing,” “acceptable,” or “failing
o The Brogden-Cronbach-Gleser formula
 formula for the monetary amount of a utility gain resulting from the use of a
particular selection instrument under specified conditions
 Utility gain refers to an estimate of the benefit (monetary or otherwise) of
using a particular test or selection method.
 Hence, BCG formula measures the productivity gains or the estimated
increase in work output

Copyright 2021 iMove Digital Learning 4

All Rights Reserved
www.imove.ph
iMOVE DIGITAL LEARNING LECTURE NOTES | PSYCHOLOGICAL TESTS: AN INTRODUCTION

 Considerations in Utility Analysis

o Pool of applicants
 There are certain jobs, that require such unique skills or demand such great
sacrifice that there are relatively few people who would even apply, let alone
be selected.
 Also, it may be that in periods of high unemployment there are significantly
more people in the pool of possible job applicants than in periods of high
employment
 How many people would actually accept the employment position offered to
them even if they were found to be a qualified candidate. Many of the top
performers on the test are people who, because of their superior and
desirable abilities, are also being offered positions by one or more other
potential employers. Consequently, the top performers on the test are
probably the least likely of all of the job applicants to actually be hired
o Complexity of Job
 the more complex the job, the more people differ on how well or poorly they
do that job
o Cut scores – the reference point derived as a result of a judgment and used to divide
a set of data into classifications
 Relative or norm-referenced cut score – the reference point in the
distribution that classifies the set of data based on norm-related
considerations
 Fixed or Absolute cut score - reference point in the distribution that
classifies the set of data based on judgments concerning a minimum level of
proficiency required to be included
 Multiple cut score – two or more cut scores with reference to one predictor
for categorizing testtakers
 Multiple Hurdles - the achievement of a particular cut score on one test is
necessary in order to advance to the next stage of evaluation

 Setting Cutoff or Cut Scores

o Angoff Method
 Presence or absence of a particular trait, attribute, or ability
 Provides an estimate on how testtakers with the least minimal competence
should answer the items correctly
 The judgments of the experts are averaged to yield the cut scores for the test
o Known Groups Method
 Entails collection of data on the predictor of interest from groups known to
possess a trait, attribute, or ability
 Involves a problem of determination on where to set cutoff score due to the
composition of groups

Copyright 2021 iMove Digital Learning 5

All Rights Reserved
www.imove.ph
iMOVE DIGITAL LEARNING LECTURE NOTES | PSYCHOLOGICAL TESTS: AN INTRODUCTION

o IRT-based or “bookmark” Method

 Based on testtakers’ performance across all items on the test.
 Some of the total number of items must be scored correct, and each item is
associated with a particular level of difficulty
 In order to pass, the testtaker must answer items that are deemed to be
above some minimum level as determined by experts
 This method is used in licensing examination
o Discriminant analysis or Discriminant function analysis
 a family of statistical techniques typically used to shed light on the
relationship between certain variables such as scores on a battery of tests,
and two or more naturally occurring groups such as persons judged to be
successful on the job and persons judged not to be successful on the job

Hits and Misses in Decision-making Related to Utility Analysis

Term General Definition Implications

The predictor test has successfully predicted
performance on the criterion; it has successfully
Hit A correct classification predicted on-the-job outcome.
A qualified job applicant is hired; an unqualified job
applicant is not hired.
The predictor test has not predicted performance on
An incorrect classification; a the criterion; it has failed to predict the on-the-job
Miss
mistake outcome. A qualified job applicant is not hired; an
unqualified job applicant is hired.
The proportion of people that an
The proportion of qualified test takers with a passing
assessment tool accurately
score who were actually hired. Also, the proportion of
Hit Rate identifies as possessing or
unqualified test takers with a failing score who were not
exhibiting a particular trait, ability,
hired.
behavior, or attribute
The proportion of people that an
The proportion of test takers whom the test
assessment tool inaccurately
Miss inaccurately predicted to be qualified. Also, the
identifies as possessing or
Rate proportion of test takers whom the test inaccurately
exhibiting a particular trait, ability,
predicted to be unqualified.
behavior, or attribute
A specific type of miss whereby an
An applicant who is hired is not qualified.
assessment tool falsely indicates
False (The test indicates that the new hire will perform
that the testtaker possesses or
Positive successfully on the job but, in fact, the new hire does
exhibits a particular trait, ability,
not.)
behavior, or attribute
False A specific type of miss whereby an The test says to not hire but the applicant would have
Negative assessment tool falsely indicates been rated as qualified. (The test indicates that the new

Copyright 2021 iMove Digital Learning 6

All Rights Reserved
www.imove.ph
iMOVE DIGITAL LEARNING LECTURE NOTES | PSYCHOLOGICAL TESTS: AN INTRODUCTION

Term General Definition Implications

that the testtaker does not hire will not perform successfully on the job but, in fact,
possess or exhibit a particular the new hire would have performed successfully)
trait, ability, behavior, or attribute

Reference Sources About Published Tests and Assessment Tools

 Test catalogues
o publishers’ catalogues distributed by the publisher of the test usually contain only a brief
description of the test and seldom contain the kind of detailed technical information that a
prospective user might require. Moreover, the catalogue’s objective is to sell the test. For
this reason, highly critical reviews of a test are seldom, if ever, found in a publisher’s test
catalogue.
 Test manuals
o Detailed information concerning the development of a particular test and technical
information relating to it, which is usually available from the test publisher.
o However, for security purposes, the test publisher will typically require documentation of
professional training before filling an order for a test manual.
 Reference volumes
o The Buros Institute of Mental Measurements provides “one-stop shopping” for a great deal
of test-related information. The initial version of what would evolve into the Mental
Measurements Yearbook was compiled by Oscar Buros in 1933.
 Journal articles
o Articles in current journals may contain reviews of the test, updated or independent studies
of its psychometric soundness, or examples of how the instrument was used in either
research or an applied context.
o In addition to articles relevant to specific tests, journals are a rich source of information on
important trends in testing and assessment
 Online databases
o One of the most widely used bibliographic databases for test-related publications is that
maintained by the Educational Resources Information Center (ERIC). Funded by the U.S.
Department of Education and operated out of the University of Maryland, the ERIC Web site
at www.eric.ed.gov contains a wealth of resources and news about tests, testing, and
assessment.
o American Psychological Association (APA) maintains a number of databases useful in
locating psychology-related information in journal articles, book chapters, and doctoral
dissertations.
 PsycINFO is a database of abstracts dating back to 1887.
 ClinPSYC is a database derived from PsycINFO that focuses on abstracts of a clinical
nature.
 PsycSCAN: Psychopharmacology contains abstracts of articles concerning
psychopharmacology.
 PsycARTICLES is a database of full-length articles dating back to 1988.
Copyright 2021 iMove Digital Learning 7
All Rights Reserved
www.imove.ph
iMOVE DIGITAL LEARNING LECTURE NOTES | PSYCHOLOGICAL TESTS: AN INTRODUCTION

 Health and Psychosocial Instruments (HAPI) contains a listing of measures created

or modified for specific research studies but not commercially available; it is
available at many college libraries through BRS Information Technologies and also
on CD-ROM (updated twice a year).
 PsycLAW is a free database, available to everyone, that contains discussions of
selected topics involving psychology and law.

Information
Pros
Source
Test Contains general description of test,
catalogues including what it is designed to do and
who it is designed to be used with.
Readily available to most anyone who
requests a catalogue.
Test Manual Usually the most detailed source
available for information regarding the
standardization sample and test
administration instructions. May also
contain useful information
regarding the theory the test is based
on, if that is the case. Typically
contains at least some information
regarding psychometric soundness of
the test.
Reference Much like a Consumer Reports for
Volumes tests, contains descriptions and
critical reviews of a test written by
third parties who presumably have
nothing to gain or lose by praising or
criticizing the instrument, its
standardization sample, and its
psychometric soundness.
Journal Up-to-date source of reviews and
Articles studies of psychometric soundness.
Can provide practical examples of how
an instrument is used in research or
applied contexts.
Cons
Primarily designed to sell the test to
test users and seldom contains any
critical reviews. Information not
detailed enough for basing a decision
to use the test.
Details regarding the test’s
psychometric soundness are usually
self-serving and written on the basis
of studies conducted by the test
author and/or test publisher.
A test manual itself may be difficult to
obtain
by students, as its distribution may be
restricted to qualified professionals.
Few disadvantages if reviewer is
genuinely trying to be objective and is
knowledgeable, but as with any
review, can provide a misleading
picture if this is not the case. Also, for
very detailed accounts of the
standardization sample and related
matters, it is best to consult the test
manual itself.
As with reference volumes, reviews
are valuable to the extent they are
informed and, to the extent that is
possible, unbiased. Reader should
research as many articles as possible
when attempting to learn how the
instrument is actually used; any one
article alone may provide an atypical
picture.

Copyright 2021 iMove Digital Learning 8

All Rights Reserved
www.imove.ph
iMOVE DIGITAL LEARNING LECTURE NOTES | PSYCHOLOGICAL TESTS: AN INTRODUCTION
Information
Pros
Source
Online Widely known and respected online
Databases databases such as the ERIC database
are virtual “gold mines” of useful
information containing varying
amounts of detail. Although some
legitimate psychological tests may be
available for self-administration and
scoring online, the vast majority are
not.
Copyright 2021 iMove Digital Learning
All Rights Reserved
www.imove.ph
Cons
Consumer beware! Some sites
masquerading as databases for
psychological tests are designed more
to entertain or to sell something than
to inform. These sites frequently offer
tests you can take online.
As you learn more about tests, you will
probably
become more critical of the value of
these self-administered and self-
scored “psychological tests.”
9