Dr. K. A.

Korb, UniJos 1/28/2

Introduction
• Tests are used to:
a) Make important life decisions about individuals
Test Development: b) Conduct research
• For both purposes, a good test is vital to ensure life
Scale Construction decisions are helpful and research conclusions are
truthful
• A poorly designed test leads to test scores that are
Dr. K. A. Korb inaccurate, which will lead to wrong decisions
UniJos • Developing a good test is VERY important.
• Developing a good test takes a lot of time, thought,
and preparation

Process of Developing a Test Test Conceptualization

• Test Conceptualization: Idea for a test • Step 1: Describe purpose and rationale for test
– What the test will measure and for what purpose will
• Test Construction: Drafting test items the test will be used?
• Test Tryout • Step 2: Describe the target population for the test
• Item Analysis • Step 3: Clearly define the key variable of interest
• Test Revision • Step 4: Create item specifications
• Step 5: Choose item format
• Step 6: Specify administration and scoring
procedures
Dr. K. A. Korb, UniJos 1/28/2

Test Conceptualization: Test Conceptualization:

Purpose and Rationale Target Population
• What is the test designed to measure? • Relevant characteristics to consider:
• What is the purpose for the test?
– Age
• Who will use the test?
– Educational Status
• Who will take the test?
• How will the test be administered? – Language (which language and level of language
ability)
• What is the ideal format for the test given the test
taker characteristics and time/financial considerations? – Literacy Level
• Is there any potential harm that can result from test – Disabilities
administration?
• How will test scores be meaningful?

Test Conceptualization: Test Conceptualization:

Define Variable Item Specifications
• Develop by referencing: • List of major content areas to be included in
– Theory the test
– Empirical Literature – Also can include number of items
– Cultural Definitions • For clinical tests, the DSM-V (or other
– DSM-V or other diagnostic criteria guidelines for diagnosis) can provide
guidelines for item specifications
• Otherwise, develop item specifications from
theory or definitions of variable
Dr. K. A. Korb, UniJos 1/28/2

Separation Anxiety Disorder (SAD) Test Conceptualization

Diagnostic Criteria (DSM-IV-TR)
• Evidenced by three or more of the following:
– Recurrent excessive distress when separation from home or major
attachment figures occurs or is anticipated
– Persistent and excessive worry about losing or possible harm to major
attachment figures
– Persistent and excessive worry that an unpleasant event will lead to
separation from a major attachment figure (e.g., kidnapping)
– Persistent reluctance or refusal to go to school or elsewhere because
of fear of separation
– Persistently and excessively fearful or reluctant to be alone without
major attachment figures
– Persistent reluctance or refusal to go to sleep without being near a
major attachment figure or to sleep away from home.
– Repeated nightmares with the theme of separation
– Repeated complaints of physical symptoms (e.g., headaches, nausea)
when separation from major attachment figures occurs or is
anticipated
Types of Instruments
• Self-Report: Participants report their own demographic
characteristics, attitudes, beliefs, knowledge, feelings,
and behavior
• Can take the form of Questionnaire or Interview
• Performance Assessment: Directly assess performance
on a contrived task
• Observation Schedule: Researchers observe participants’
behavior under specified conditions
• Checklist: Identify the frequency or presence of
behaviors or characteristics
• Examination/Test: Test participants’ knowledge of a
topic

Self-Report Questionnaire
• Self-Report: Participants are asked to report • Advantages:
their own attitudes, beliefs, knowledge, – Easy to administer and score
feelings, and behavior – Can be administered to larger numbers of test-
– Self-report can be either a questionnaire or takers
interview • Disadvantages:
– Attitudes, beliefs, values, interests, knowledge, – The data is typically not as in-depth as an
feelings, and some types of behavior are typically interview
best assessed by self-report for most target
– Requires a high level of literacy
populations
Dr. K. A. Korb, UniJos 1/28/2

Interview Types of Items for Self-Report

• Advantages:
– Gathers rich, in-depth information
– Can ask test-taker to expand on unclear responses
– Can be used with any test-taker (e.g., cannot read)
• Disadvantages:
– Takes time
– Typically subject to issues of inter-rater reliability
• Open-ended: Test-takers give their own response
• Advantages:
– More in-depth information
• Disadvantages:
– Test-takers need to have good language/literacy skills
– Test users have to categorize and code responses
(scoring is subjective)
– Subject to inter-rater reliability issues
– Test-takers’ responses may not be understandable
(Questionnaires)
– Test-takers often skip these items on questionnaires

Types of Items for Self-Report Sample Items

• Close-ended: Test-takers select their response from • Open-Ended: What causes you anxiety?
given categories on the test
• Advantages: • Close-Ended: What causes you anxiety?
– Easy and objective scoring a) Public speaking
– Response is clear to the test user
• Disadvantages: b) Large crowds
– Test developer must anticipate most common responses c) Examinations
– Test developer must develop response categories that the
target population understands d) Driving
– Test developer must develop response categories that are
exhaustive (all participants have a place to appropriately
respond)
Dr. K. A. Korb, UniJos 1/28/2

1 2 3 4 5
Test Item Formats
Strongly 2 3 4 Strongly
Disagree Agree
• Rating Scale: Grouping of words, statements,
or symbols on which judgments concerning Not at all
true
2 3 4 Very true

the strength of a particular trait, attitude, or

Not at all 2 3 4 Very
emotion are indicated by the test-taker. important important

Not at all 2 3 4 Extremely

Strongly Strongly
Disagree Disagree Neutral Agree Agree
1 2 3 4 5 6
1 2 3 4 5

0 1 2 3 4
Test Item Formats
Never Rarely Sometimes Most of Always
the time
• Paired Comparisons: Test-takers are
Never Yearly Monthly Weekly Daily
presented with pairs of stimuli that they are
asked to compare and select one.
Rarely Monthly Weekly 1-2 times Every • Useful to place test-takers into categories
a week Day
• Avoids the problem of test-takers agreeing (or
1-29 30 minutes 1-2 hours 3 hours
0
minutes minutes – 1 hour or more
disagreeing) to every category
Dr. K. A. Korb, UniJos 1/28/2

Paired Comparisons Example: 5 Love Test Format:

Languages Questionnaire
1. A) I like to receive notes of affirmation
B) I like to be hugged
2. A) I like to spend one-to-one time with a person
who is special to me.
B) I feel loved when someone gives practical help
to me.
3. A) I like it when people give me gifts.
B) I like leisurely visits with friends and loved
ones.
Checklists
• Checklists identify the frequency or presence
of behaviors or characteristics
– Adjective checklist: Tick list of adjectives that
apply to yourself (or who is being rated)
• Parents’ perceptions of children
• Clients’ perceptions of therapist
• Employees’ perceptions of Managers
– Psychological Processing Checklist–Revised (PPC–
R)(learning difficulties)
– PTSD Checklist (PCL-5)

Test Format:
Adjective Checklist
Checklists
1. Absentminded 8. Adaptable 15.Conservative • Hare Psychopathy Checklist uses a semi-
structured interview to identify the presence or
2. Adventurous 9. Assured 16.Confused absence of psychopathological symptoms such
3. Articulate 10.Careful 17.Cautious as:
– Grandiosity
4. Acquiescent 11.Confident 18.Conscientious – Pathological lying
5. Artistic 12.Cynical 19.Daring – Cunning and manipulating
– Lack of remorse
6. Active 13.Careless 20.Deceitful – Impulsiveness
7. Assertive 14.Changeable 21.Deep – Failure to accept responsibility for own actions
– Lack of realistic long-term goals
Dr. K. A. Korb, UniJos 1/28/2

Test Format: Test Format:

Checklists Performance Assessment
Indicate if you have personally done in any behavior within the • Directly assess performance on tasks
past three years that you have been in school.
1. Copy your continuous assessment answer from another
• Professional performance
student. YES NO – Practicals in Medical Boards
2. Read another student’s answers on their exam script during – PSY 804
the exam. YES NO • Neuropsychological performance (e.g., memory,
3. Ask another student for an answer during the exam. YES NO attention)
4. Write answers on an exam script beforehand to carry it into – Doors and People (Memory)
the hall. YES NO
– Wechsler Memory Scale
5. Trade your script with a student in the exam so they write
your answer. YES NO – Iowa Gambling Task (real-life decision-making)
6. Write answers or notes on your body, clothing, or personal – Stroop Task (reaction time)
belongings. YES NO
7. Use your handset to get answers. YES NO

Test Format:
Observation
• Measures behaviors only
• Examples
– Parent-Child Interaction Assessment-II (PCIA)
– Parent-Child Early Relational Assessment
– Dyadic Parent-Child Interaction Coding System-II
Dr. K. A. Korb, UniJos
Test Format:
Observation
• Consider:
– Where and when will the behaviors be observed?
– What possible situation will the test-taker will be
exposed to?
– How long will test-takers be observed?
– How will behaviors be recorded?
• Duration: Length of time that behaviors occurs
• Frequency Count
• Interval: Set a stopwatch for a specific interval of time,
then record what behavior the test-taker is engaged in
Administration and Scoring Guidelines
• Standardized assessment requires very clear
administration guidelines so all participants
complete the assessment under similar
conditions
– Critical for performance and observation
assessments
• Scoring guidelines should provide specific
score interpretations based on empirical
research
1/28/2
Item Construction
Developing Good Items
• Use clear words so all participants understand
• Be precise and specific in the wording of each
item so different test-takers do not interpret the
item differently
• Ensure the item is not biased or leading toward
one response.
• Each item should have only one point (avoid
double-barreled items)
• Ensure that each test-taker has a place to
respond
Test Try-Out
• Try items out on people similar in critical
respects to the people for whom the test was
designed
• Sample size: No fewer than 5 participants
(preferably at least 10 participants) for every 1
item on the test
• Conduct under conditions as identical as
possible to conditions under which the final
test will be administered