psychology assessment

Distinguishing between testing

and assesment
History of Testing and Assessment
Distinguishing Between Testing and Assessment

Tests are a subset of assessment yielding scores based on collective data.

Example: Finding the sum of correct items on a multiple-choice exam

Assessment procedures can be:

Formal: Rigor in test development (e.g., statistics, good, valid, and reliable)

Informal: Rigor has not been demonstrated in the test development

psychology assessment Distinguishing between testing and assesment 1

 Historical Context: Ancient

2200 B.C.E - Chinese used essay examinations for civil service employees.

Plato (428-327 B.C.E) noted:

Greeks assessed intellectual and physical ability of men when screening for state
service.

Historical Context: Precursors to Modern-Day

Esquirol (1830s)

Used language ability to identify intelligence

Retardation on continuum: "idiocy" to low grade

Forerunner of verbal intelligence testing

Seguin (1800s)

Worked with individuals with intellectual deficits to increase motor control and
sensory discrimination

Developed "Form Board"

Forerunner of performance intelligence testing

Understanding uniqueness of the humans:

Darwin (mid 1800s)

Theory of evolution: Scientific method

Set tone for others who followed

Galton (mid to late 1800s; Darwin's cousin)

English biologist studied relationship of sensory motor activities to intelligence

Instrumental in others developing correlation coefficient

Individual differences

Wundt (1879)

Founded 1st psychological laboratory

psychology assessment Distinguishing between testing and assesment 2

 Looked at sensitivity to visual, auditory, and other sensory stimuli and reaction
time

James Cattell (late 1800s; Am. Psychologist)

Phrased term "mental test"

Used statistical concepts to understand differences ( e.g. chi suared, t-test, etc)

G.S. Hall (late 1800s)

experimental lab at john hopkims

mentor to many

first president of american psychological assiciation

Emergence of Ability Tests: Testing in Cognitive Domain

Three areas

Individual Intelligence Testing

Neuropsychological Assessment

Group Tests of Ability

Individual Intelligence Testing

Binet (1890s)

Hired by French Ministry of Public Education to integrate "sub-normal" children

into the schools

Developed first modern-day intelligence test

Terman (At Stanford)

Revised Binet scale - Stanford-Binet

I.Q. = MA/CA

Neuropsychological Assessment

Head injuries: 5,000-year-old Egyptian medical documents

W W I: Increased interest in brain injuries

20th century: Development of braining imaging techniques

psychology assessment Distinguishing between testing and assesment 3

 Today: Suspected changes in brain function yields neuropsychological assessment

Group Testing

WW I: Army Alpha and Army Beta

Developed by Yerkes, Terman, and others

Take Army Alpha

Used by Eugenics Movement

theory of "racial improvement" and "planned breeding," which gained popularity

during the early 20th century. Eugenicists worldwide believed that they could
perfect human beings and eliminate so-called social ills through genetics and
heredity.

After WWII

SATs: Developed by James Bryant Conant to equalize educational opportunities for

all

Vocational counseling led to special aptitude tests and multiple aptitude tests

Objective Personality Assessment

Kraeplin (18 92): Word association test to study schizophrenia

Woodworth's Personal Data Sheet (WWI)

Crude test to access for neuroses and pathology

Forerunner to MMPI

Projective Testing

Galton (1879) and projective testing

"Experiments such as these allow an unexpected amount of illumination to

enter into the deepest recess of the character, which are opened and bared by
them like the anatomy of an animal under the scalpel of a dissector in broad
daylight."

IM Cattell: Examined associations made by "healthy people" to a standard list of

words,

psychology assessment Distinguishing between testing and assesment 4

 eugenics - (best characteristics through selective breeding), theories of
intelligence

Emergence of Informal Assessment Procedures

20th Century

Situational Tests: "Contrived naturalistic situations"

Clinical Interview

DSM-5 (DSM-I developed in 1952)

Observation

Rating Scales

Classification Techniques

Records and Personal Documents

Environmental Assessment

Performance-based Assessment

graph of informal assessment

Questions to Consider When Assessing People

psychology assessment Distinguishing between testing and assesment 5

 How valid is the information gained from assessment instruments? How should that
information be applied?

How do assessment instruments invade an individual's privacy? Does the

government have, at times, the right to insist an individual be assessed?

Can the use of assessment instruments, in some cases, lead to labeling? What are
the implications for the individuals who are labeled?

Are assessment procedures used to foster equality for ALL people or do they tend
to create a society based on class?

test are a subset of assessment yielding scores based on collective data.

example: finding the sum of correct items on a multiple choice exam

Emergence of informal assessment procedures

20th century

Computers: statistical procedures has increased the quality of tests

today, can view tests in the ff

Diagnosis in the Assessment Process

The Importance of Diagnosis

Some reasons for its current importance:

Increased emphasis on testing in the schools for children with disabilities

Diagnosis now viewed as aspect of holistic assessment process

Laws like RA 10029, MH Act make it necessary for professionals to know diagnosis
to help clients understand their diagnosis and to assist them with their needs at
work

Some reasons for its current importance:

To communicate with other clinicians

To help clients understand their diagnosis and their prognosis

To help in treatment planning

DSM: A Brief History

psychology assessment Distinguishing between testing and assesment 6

 Diagnosis: Derived from the Greek words dia (apart) and gnosis (to perceive or to
know)

Mid 1800s: U.S. Census Bureau counted incidence of "idiocy" and "insanity"

Medical 203 became basis for DSM-I in1952 by American Psychiatric Association

11 categories

185 diagnoses

DSM-Ill released in 1980 to be compatible with ICD manuals

265 diagnoses

Multiaxial

D S M-IV: 19 94

D S M-IV-TR: 2000

365 diagnoses

Five Axes

Axis I: Clinical disorders

Axis II: Personality disorders and mental retardation

Axis III: General medical conditions

Axis IV: Psychosocial and environmental factors

Axis V: Global assessment of functioning

DSM-5 (2013)

Replaces Roman numerals with numbers

Like computer software, can follow with new editions/versions (5.1, 5.2, etc.)

Online component available for supplemental materials

www.psychiatry.org/dsm5

Numbers aligned with I C D-9 and I C D-10

DSM-5: Final Thoughts

One piece of total assessment process

psychology assessment Distinguishing between testing and assesment 7

 Often should be included with:

A clinical interview

Assessment procedures

The more pieces of information, the better we can gain a clear picture of our client
and develop effective treatment plans

The Assessment Report Process: Interviewing the Client and Writing the Report

Purpose of the Assessment Report

The assessment report is the "deliverable" or "end product" of the assessment

process.

Reports have been used to:

Respond to the referral questions being asked

Provide insight to clients for therapy

Assist in the case-conceptualization process

Develop treatment options in counseling

Suggest educational services for students with special needs

Offer direction when providing vocational rehabilitation services

Offer insight and treatment options for individuals who have incurred a cognitive
impairment

Assist the courts in making difficult decisions

Provide evidence for placement into schools, jobs

Challenge decisions made by institutions and

Gathering Information for the Report

How you gather information for the report is as important as writing the report

In gathering information, take into account the breadth and depth of your
assessment procedures.

Breadth: Based on purpose of the report; cast a wide enough net

psychology assessment Distinguishing between testing and assesment 8

 Depth: Assuring that one is using techniques that reflect the intensity of the
issues being examined

The Interview as a Test

Similarities Between an Interview and a Test

Method for gathering data Used to make predictions

Evaluated in terms of reliability

Evaluated in terms of validity

Group or individual

Structured or unstructured

Principles of Effective Interviewing-The Proper Attitudes

Interpersonal influence is related to interpersonal attraction.

Important interviewing attitudes

Warmth

Genuineness

Acceptance

Openness

Honesty

Fairness

A patient's perception of the interviewer's feelings is an important factor in his or her

evaluation

Responses to Avoid

Stress interviews

Interviewers generally want their interviewee to feel relaxed and comfortable

Judgmental or evaluative statements can create anxiety and inhibit the process

Probing statements can push boundaries

May be necessary in some circumstances

psychology assessment Distinguishing between testing and assesment 9

 Use with caution and only when appropriate

Hostile statements are to be avoided unless they have a specific purpose

Reassurance can be appropriate, but only if it is genuine

Effective probing statements

Effective Responses

Open-ended questions

Cannot be answered specifically

Those that require more than a simple "yes" or "no" response

Give interviewee latitude to decide what is important to share

Provides more information about interviewee

- Closed-ended questions

Can bring the interview to a halt

Does not serve to keep interaction flowing

close and open ended questions

psychology assessment Distinguishing between testing and assesment 10

 Responses to Keep the Interaction Flowing

Interviewer should remain quiet while responses are being given and give small
prompts to keep answer flowing

Cursory responses may not be adequate, so more answer-relevant probes may be

needed

Verbatim playback

Paraphrasing and restatements

Summarizing and clarifying

Empathetic or understanding responses

psychology assessment Distinguishing between testing and assesment 11

 Measuring Understanding

Five-point scoring system of empathetic responses

Level one responses

Unrelated to precipitating statement

Level two responses

Superficial awareness of statement's meaning

Level three responses

Interchangeable with interviewee's statement

Level four responses

Accurate empathy and goes beyond statement given

Level five responses

Goes significantly beyond statement given

Active listening

Mental Status Examination

Used to diagnose psychosis, brain damage, and other major mental-health

problems

Evaluates neurological or emotional problems

psychology assessment Distinguishing between testing and assesment 12

 Several areas are evaluated examples include:

Appearance

Attitudes

General behavior

Emotions

Attentional skills

Sensory factors

Developing Interviewing Skills

Whether interviewing skills can be learned is an area of debate

Steps for learning these skills:

1. Learn the research and theory on interviewing skills

2. Supervised practice

3. Conscious effort to apply practices, including ongoing evaluation

Sources of Error in the Interview Interview Validity

The Halo effect

General standoutishness

Misunderstanding cultural differences

Bias in selection processes

This can be reduced using a structured interview

Interview validity may be questionable, but it provides a wealth of unique data

Consider it as tentative until verified by other sources of information

Interview Reliability

Interviewer reliability coefficients are quite variable

Unstructured interviews have the lowest reliability, though they may lead to fairer
outcomes than other assessment tools

psychology assessment Distinguishing between testing and assesment 13

 Interviews vary in their standardization--they can focus on different areas of
importance

Structured interviews provide higher reliability estimates

Don't provide as much or as varied information as unstructured or

semistructured interviews

Structured, Unstructured, and Semi-Structured Interviews

Interview accomplishes the following:

Sets tone

Allows client to become desensitized to intimate and personal info

Allows examiner to assess nonverbal signals to determine what might be important

to focus upon

Allows examiner to learn firsthand the problem areas of the client and place them in
perspective

Gives client and examiner opportunity to see whether they can work together

Benefits of Structured Interview

Offers broad enough areas of content to cover topics a practitioner may otherwise
miss or forget to ask (assures breadth of coverage)

Increases reliability of results by ensuring that all prescribed items will be covered

Ensures examiner will cover all items (they are listed in detail and full coverage
expected)

Ensures items will not be missed due to interviewer or interviewee embarrassment

Drawbacks of Structured Interview

Examiner may miss information because items are predetermined and examiner
does not feel as free to go off on a tangent or a "hunch"

Clients may experience the interview as dehumanizing

Clients, particularly minorities, may misinterpret or be unfamiliar with certain items

Follow-up by the examiner to alleviate any confusion on the part of the examine is
less likely as compared to other kinds of interviewing

psychology assessment Distinguishing between testing and assesment 14

 Does not always allow for depth because interviewer is more concerned with
gathering info than detail about a particular sensitive area

Benefits of Unstructured Interview

Creates atmosphere more conducive to building rapport

Allows client to feel as if he or she is directing the interview, thus allowing client to
discuss items he or she deems important

Offers potential for greater depth of information because the clinician can focus
upon a potentially sensitive area and possibly uncover underlying issues the client
might otherwise

Drawbacks of Unstructured Interview

Because it does not allow for breadth of coverage, the interviewer might miss
information because he or she is "caught up" in the client's story instead of following
a prescribed set of questions

Interviewer may end up spending more time on some items than he or she might
like

Semi-structured interview: Has advantages (and drawbacks) of both the structured

and unstructured

Choosing an Appropriate Assessment Instrument

Assessors administer, adapt, score, interpret, or use assessment techniques,

interviews, tests, or instruments in a manner and for purposes that are appropriate
in light of the research on or evidence of the usefulness and proper application of
the techniques. (A P A, Section 9.02.a)

Writing the Report

After you have conducted a thorough assessment of your client, you will be ready to
write your report.

Clients will generally have access to their records due to laws passed over the
years

Family Educational Rights and Privacy Act (FERPA)

Freedom of Information Act

psychology assessment Distinguishing between testing and assesment 15

 Health Insurance Portability and Accountability
Act (HIPAA)

Writing The Report: Points to Consider

Fifteen Suggestions for Writing Reports

1. Omit passive verbs.

2. Don't try to dazzle the reader of your report with your brilliance.

3. Be nonjudgmental.

4. Reduce the use of jargon.

5. When possible. describe behaviors that are representative of client issues.

6. Do not use a patronizing tone.

7. Only label when it is necessary and valuable to do so for the client's well-being.

8. Increase the use of subheadings.

9. Reduce the use of and define acronyms.

10. Write the report so a non-mental-health professional can understand it (e.g.. a

teacher)

11. Minimize the number of difficult words.

12. Try to use shorter rather than longer words.

13. Make sure paragraphs are conise and flow well.

14. Don't be afraid to take a stand if you feel strongly that the information warrants it
(e.g.. the information leads you to believe a client is in danger of harming self).

15. Point out strengths and weaknesses of your Client.

Writing the Report: Format

Although formats can vary, often they will include:

Demographic information

Basic information about the client including name, address, phone number,
e-mail, date of birth, age, sex, ethnicity, date of interview, interviewer.

Presenting problem or reason for the report

psychology assessment Distinguishing between testing and assesment 16

 Person who referred the client is generally noted

Explanation as to why the individual has come for counseling and/or why
the examiner has been asked to do the assessment

Family background

Significant medical/counseling history

Delineate significant medical history, especially any that affected client's

psychological state

Note history of any counseling

Substance use and abuse

Report use and abuse of any legal or illegal substances that may be
addictive or potentially harmful

Although formats can vary, often they will include:

Vocational and educational history

Other pertinent background information

Mental status

Assessment or test results

Diagnosis

Summary and conclusions

Recommendations

The Mental Status Exam

Often a one or two paragraph statement which is an assessment of client's

appearance and behavior, emotional state, thought components, and cognitive
functioning

Appearance and behavior: Client's observable appearance and behaviors during

the interview, including: dress, hygiene, body posture, tics, significant nonverbals
(eye contact, wringing of hands, swaying), and manner of speech (e.g., stuttering,
tone).

Emotional state:

psychology assessment Distinguishing between testing and assesment 17

 Affect and mood are examined

Affect: Current feeling state

Happy, sad, joyful, angry, depressed, etc.

Mood: Long-term, underlying emotional well-being of client

Client may seem anxious and sad during session (affect) and report that his
or her mood has been depressed.

Thought Components:

Often addresses whether client has delusions, distortions of body image,

hallucinations, obsessions, suicidal or homicidal ideation, and so forth.

Often include circumstantiality, coherence, flight of ideas, logical thinking, intact as

opposed to loose associations, organization, and tangentiality

Cognition:

Whether the client is oriented to time, place, and

person

An assessment of client's short- and long-term memory

An evaluation of the client's knowledge base and intellectual functioning

A statement about the client's level of insight and ability to make judgments

Assessment Results:

Begin with a list of assessment procedures used

Offer the results of the assessment procedures

Try not to give out raw scores

Offer converted or standardized scores that reader will understand (e.g.,

percentiles, D I Qs, etc.)

Remember: Client, parents, or non-professional may read results.

State results in understandable, unbiased language

Be concise, yet cover all items relevant to presenting concerns, or items that clearly
stand out as a result of the assessment

psychology assessment Distinguishing between testing and assesment 18

 Present results obiectively.

Interpretations should be kept to a minimum, if used at all

Summary and Conclusions

Examiner's chance to pull together all of the information that has been gathered

Often only section of the report that is read by others

Must be accurate and not leave out any main points

Should be succinct, and relevant

Do not add any new information!

Although inferences can be made, they must be logical, sound, defendable, and
based on facts that are in report.

Recommend that a paragraph or two address strengths of the individual

Recommendations

Last section of report

Should be based on all the information gathered

Should make logical sense to the reader

Some prefer writing this section in paragraph form, others prefer listing each
recommendation

The signature of the examiner generally follows this last section

Communicating Results

Often one of the most important parts of assessment process

Surprisingly, little research related to communication of results

Clients who receive test interpretation experience greater gains than those who do
not (Goodyear, 1990)

Tentative interpretations more helpful than absolute (Jones &

Gelso, 1988)

Clients prefer individual interpretation (Goodyear, 1990)

Test Worthiness

psychology assessment Distinguishing between testing and assesment 19

 Cornerstones to test worthiness:

Validity

Reliability

Cross-cultural fairness

But first, we must learn one statistical concept:

Correlation coefficient

Correlation Coefficient

Correlation: Statistical expression of the relationship between two sets of scores (or
variables)

Positive correlation: Increase in one variable accompanied by increase in other

"Direct" relationship

Negative correlation: Increase in one variable accompanied by decrease in other

"Inverse" relationship

What is the relationship between:

Gasoline prices and grocery prices?

Grocery prices and good weather?

Stress and depression?

Depression and job productivity?

Partying and grades?

Study time and grades?

Correlation coefficient (r)

A number between -1 and +1 that indicates direction and strength of the relationship

As "" approaches +1, strength increases in a direct and positive way

As "" approaches -1, strength increases in an inverse and negative way

As "r" approaches 0, the relationship is weak or non existent (at zero)

Correlation Scatterplots

psychology assessment Distinguishing between testing and assesment 20

 Plotting two sets of scores from the previous examples on a graph

Place person A's S A T score on the x-axis, and his/her G P A on the y-axis

Continue this for person B,C, D etc.

This process forms a scatterplot

- left side scatterplot = there is a correlation (positive); right side scatterplot = there
is no correlation (negative).

Coefficient of Determination (Shared Variance)

psychology assessment Distinguishing between testing and assesment 21

 Correlation between depression and anxiety = .85

shared variance= .72

what factors might unederlie both depression and anxiety?

Validity

Concerns what instrument measures and how well it does so

Not something instrument "has" or "does not have"

Informs counselor when it is appropriate to use instrument and what can be inferred
from results

Reliability is a prerequisite for validity

The degree to which all accumulated evidence supports the intended interpretation
of test scores for the intended purpose

It is a unitary concept; however, there are 3 general types of validity evidence

Content validity

Criterion-related validity

Construct validity

Evidence Based on Test Content

psychology assessment Distinguishing between testing and assesment 22

 Degree to which the evidence indicates that items, questions, or tasks adequately
represent intended behavior domain

Central focus is typically on how the instrument's content was determined

Content-related validation evidence should not be confused with face validity

Content Validity

Is the content valid for the kind of test it is?

Developers must show evidence that the domain was systematically analyzed and
concepts are covered in correct proportion

Four-step process:

Step 1 - Survey the domain

Step 2 - Content of test matches the domain

Step 3 - Specific test items match the content

Step 4 - Analyze relative importance of each objective (weight)

psychology assessment Distinguishing between testing and assesment 23