Psychological Assessment

Seniors’ In-Service Training

Worksheet #7

Name: Bula, Tabac, Trocio

Date: Aug. 7, 2020

KEY TERM EXERCISE:

Key Term Definition
Validity
the extent of how a test measures what it intends
to measure

Inference
a result made through logical deduction

Validation
collecting and assessing the gathered evidence
about validity

Validation Studies
finding out to how well the test measures what it is
trying to measure in order to establish the validity
of the test through research

Content Validity description of a judgement of how a test samples

Criterion-related validity
Construct validity
Trinitarian view
ecological validity
Face Validity
behavior being measured based on a
representative sample
Measure of validity obtained by evaluating the
relationship of scores obtained on the test to
scores on other tests or measures
Measurement of validity arrived at by executing a
comprehensive analysis of (1) how scores on the
test related to other tests scores and measures
and (2) how scores on a test can be understood
within some theoretical framework for
understanding the construct
Construct as the umbrella validity while the others
fall under it; all three types of validity evidence
contribute to a unified picture of a test’s validity
Refers to a judgment regarding how well a test
measures what it purports to measure at the time
and place the variable is being measured
Relates more to what a test appears to measure
to the person being tested than what the test
actually measures
 Content Validity description of a judgement of how a test samples
behavior being measured based on a
representative sample
Test Blueprint
a comprehensive plan containing the overview of
a test such as the number of the items the test
may contain and its contents

Key Term Definition

Criterion-related validity an assessment of how a score on a test can be
used to make inference of an individual’s standing
on a criterion

Concurrent Validity a form of index of how criterion measures

obtained at the same time period are related to
one another

Predictive Validity
the capability of a test score to predict behavior
based on a criterion mesure
criterion The standard against which a test or a test score
is evaluated.
Criterion contamination Term applied to a criterion measure that has been
based, at least in part, on predictor measures
base rate Extent to which a particular trait, behavior,
characteristic, or attribute exists in the population
hit rate The proportion of people a test accurately
identifies as processing or exhibiting a particular
trait, behavior, characteristic, or attribute
miss rate The proportion of people the test fails to identify
as having, or not having, a particular characteristic
or attribute
False Positive an error in measurement wherein the test believes
that the testtaker possess a trait, ability, or
behavior when in fact they do not possess the
said trait, ability, or behavior at all

False Negative an error in measurement wherein the test believes

that the testtaker does not possess a trait, ability,
or behavior when in fact they do possess the said
trait, ability, or behavior

Validity Coefficient a measure of the relationship of test scores based

on a criterion measure

Incremental Validity presents the influence of additional predictors

besides the predictors being used already
 Construct Validity Measurement of validity arrived at by executing a
comprehensive analysis of (1) how scores on the
test related to other tests scores and measures
and (2) how scores on a test can be understood
within some theoretical framework for
understanding the construct
construct Informed, scientific idea developed or
hypothesized to describe or explain behavior
homogeneity How uniform a test is in measuring a single
concept
method of contrasted groups Demonstrating that scores on the test vary in a
predictable way as a function of membership in
some group

Key Term Definition

Convergent evidence evidence from other tools that measure the same
construct being measured and provide the same
evaluation in regards to the test being used

Discriminant Evidence data that shows that there is only a weak link
between test scores

Multitrait-Multimethod Matrix
a method that is used to assess construct validity
through simultaneous examination of both
convergent and discriminant evidence through
table of correlations of traits and methods
Factor Analysis group of mathematical procedures that are
designed to identify which variables people may
possibly differ from

Exploratory Factor Analysis a group of mathematical procedures that decide

which factors are need to be retained

confirmatory factor analysis Test of the degree to which a hypothetical model

(which includes factors) fits the actual data
Factor loading Conveys information about the extent to which the
factor determines the test score or scores
bias Factor inherent in a test that systematically
prevents accurate, impartial measurement
rating Numerical or verbal judgment or both that places
a person or an attribute along a continuum
identified by a scale of numerical or word
descriptors
rating scale A scale of numerical or word descriptors
 Rating Error a judgement that stems from a unintentional
misuse of a rating scale

Leniency/Generosity Error occurs when a rater posses the tendency to be

generous in giving a rating

Severity Error an error wherein rater is too critical of giving their

rating

Central Tendency Error

rating of an individual that neither falls on the
positive or negative side of the continuum and
ratings end up in the middle instead

Rankings
the ordinal sequencing of people, items, or
concepts based on a assigned value

Restriction of Range Rating Errors phenomenon wherein sampling procedure has

restricted the variance in a correlational analysis
which results the correlation coefficient to be
lower

Halo effect Describes the fact that, for some raters, some
rates can do no wrong
fairness The extent to which a test is used in an impartial,
just, and equitable way
 CATEGORIES OF VALIDITY

Type of Validity Description

Content Validity a measure of validity that is based on the

assessment of how the test covers all the
subjects or topics needed to be measured

Criterion-Related Validity obtained through assessment of the

relationship of test score with scores from
other measures

Construct Validity a holistic analysis of how scores of a test is

related to other scores of other tests and how
it can be understood from a viewpoint of using
a specific theoretical framework based on the
construct being measured

TESTING YOUR KNOWLEDGE

Indicate whether a statement is True by putting (T) and False by putting (F) next to the
statement.

1. There are some tests that are universally valid for all time, all uses, and with all types of testtaker
populations. (F)
2. Ecological Momentary Assessment (EMA) has the limitations of retrospective self report (T)
3. A personality test where respondents are asked to report what they see in inkblots may be
perceived as a test with low face validity. (F)
4. For an employment test to be content-valid, its content must be a representative sample of the
job-related skills required for employment. (T)
5. If test scores are obtained at about the same time as the criterion measures are obtained,
measures of the relationship between the test scores and the criterion do not provide evidence of
concurrent validity. (T)
 CHARACTERISTICS OF A CRITERION

Discuss the three major characteristics of criterion.

1. A criterion should be relevant to the matter at hand. The criterion should be verified by a
set of data that is representative of the population and the topic at hand or what it is trying
to measure An example would be of taking a test that would enable for the test taker to
know if they have the same skill set wit professional programmers in which those
successful programmers have taken the test already and are the norms of the test.

2. A criterion should be valid for what purpose it may be used. This means that the criterion
should be reviewed by those who are deemed worthy enough if the criterion is valid
enough and actually measures the targeted construct. In proving the criterion is valid
evidence must be also provided.

3. A criterion should not be contaminated. The basic explanation of this characteristic is that
the criterion should not bear nor be manipulated

THE RESPONSIBILITY OF THE TEST USER IN USING VALIDATION DATA

What questions must a test user ask to determine the validity of a test? List at least 3 of
them below.

1. What were the characteristics of the sample used in the validation study?

2. Will this test consistently yield information similarly to the other widely accepted
tests?

3. How matched are the characteristics to the people for whom an administration of
the test is contemplated?
 EVIDENCE OF CONSTRUCT VALIDITY

Explain how each of the following is used to establish construct validity. Fill in the table
below.

Evidence How it is Used to Establish Validity

Homogeneity This refers to how uniform a test is in
measuring a single concept. The Pearson
r could be used to correlate average
subtest scores with the average total test
score. The more the subtests are not
correlated, the lower the validity. On the
other hand, the higher the correlation, the
stronger the validity.
Changes with age There are constructs that changes in a
span of time. If a test score purports to be
a measure of a construct that could be
expected to change over time, then the
test score, too, should show the same
progressive changes with age to be
considered a valid measure of the
construct.
Pretest-Posttest Changes Changes of the score due to a result of
experience between the post or pre-test
can show big evidence of higher
construct validity. In a lot of instances,
lower to no changes in score indicates
low construct validity.
Distinct Groups If the test can be considered valid in
measuring a specific construct it is
presumed that test scores are different
from groups who are expected to yield
different scores. For instance, Anorexia
scale, it is expected that people who
doesn’t have one will score lower
 compared to individuals who are clinically
diagnosed with Anorexia.

TEST BIAS

Highlight 5 major points that relate to Test Bias and explain each one.

1. Rating error is an error judgment that is caused by intentional or unintentional usage of a

rating scale. For instance, leniency or also known as generosity error is caused by the
rater’s tendency to be lenient in scoring, marking, and/or grading.

2. Severity Error is a type of error in which the ratings are either overly positive or negative.
A lot of times, this is because something is consistently or consensually viewed as good or
bad by a lot of people.

3. Central Tendency Error is when the rater neither displays extremely good or bad in rating
something. The raters tend to stay neutral and in the middle of the continuum.

4. Halo effect is when raters think that rates can not make any mistake. It is also where a
rater gives someone a score that is higher than what he or she originally deserves. This
is because of the rater’s failure to see other distinct behavior of an individual.

5. To avoid test bias, test users may opt to choose ranking. This is where instead of using
an absolute scale, the rater measure individuals against one another. By using this, the
ranker will select first, second, third choices, and so forth.