Psychological Assessment 1

Issues and Trends in Psychological Testing

IV. Issues and Trends in Psychological Testing

Overview
For this module, we begin by asking, "What are the issues that need to be examined in the field of
psychological testing? If a certain person scores below average on a test of intelligence, can we really
conclude that he/she does possess a low level of IQ?" While there are many concerns that can affect our
conclusion, in this module, we will focus mainly on– common response bias set and test faking.
Furthermore, we will tackle the current trends in test scoring, mainly to discuss the best practice in
assessment and testing. Last but not least, an examination of the problems encountered in cross-cultural
testing will be explored.

Basic Issues in Psychological Testing

The following are basic issues in psychological testing as they affect the manner/way individual
answers a test, affecting the reliability and validity of the tests. Hence, careful consideration of these issues
needs to be addressed, especially in the test construction phase.

1. Response Set Biases

As cited by Domino & Domino in 2006, Cronbach (1946) defined response sets as any inclination of a
person to consistently give different responses to test items than he or she would have given if the same
content was presented in a different form.

He then listed a number of such response sets that include the following:
• The tendency to gamble

As described by Cronbach (1946) in his article, he explained this concept as the tendency to answer an
item when uncertain by omitting responses, by being cautious thru answering with a neutral response
alternative rather than an extreme alternative (as in Likert scales choosing “unsure” rather than strongly
agree or strongly disagree). Such guessing can increase the reliability of a test because it results in greater
individual differences (some guess and some don’t) but tends to lessen validity.

• Definition of judgment categories.

Domino & Domino (2006) described this in their book since most tests oblige the subject to answer using
given response categories, such as the Likert response scale. But different persons give different meanings
to the response options,

As an example, the meaning assigned to such response option as “frequently”: What does it mean? Does
it mean daily, weekly or monthly period?
 Psychological Assessment 2
Issues and Trends in Psychological Testing

• Inclusiveness.

Domino & Domino (2006) described this in their book that when the person can make answer as many
responses as he or she likes, some individuals make more responses than others.

This happens not only on essay exams, where one person's answer may be substantially longer but also
on tests such as the Thematic Apperception Test, where one person may give an elaborate story about the
picture.

• Bias or acquiescence.

Domino & Domino (2006) explained this in their book as the tendency to endorse “true” or “yes” to
dichotomous response items. Such response tendencies affect an answer only when the individual is to
some degree unclear about the item content. Consequently, acquiescence tends to make false items more
valid and true items less valid.

• Speed vs. Accuracy Responding.

In testing, speed of response (answering) is also an important element, as the individual can answer
carefully, sacrificing speed, or can answer rapidly, sacrificing accuracy.

• Response sets on essay tests.

As cited by Domino & Domino in 2006, Cronbach (1946) argued that there were individual differences in
these response sets (e.g., some people are more willing to guess if uncertain than others are), and these
individual differences were reliable. He emphasized that response sets have the highest impact on a test
score when the items are unclear or unstructured .

As a matter of fact, Cronbach (cited by Domino & Domino, 2006) suggested the use of multiple choice
items as a way to reduce response sets.

The following are the common responses sets (also known as response biases that have been extensively
studied:
1. social desirability bias (or faking good)
2. faking bad
3. acquiescence (or yes saying)
4. opposition (or no saying)
5. positional set (refers to selecting one response position on multiple choice tests significantly more
often, regardless of the item content.)
6. random responding (carelessness and/or inconsistency)
 Psychological Assessment 3
Issues and Trends in Psychological Testing

Seeing these response sets as potential threats to the reliability and validity of self-report measures, thus, a
number of efforts have been made to handle such biases.

Domino & Domino (2006) suggested the following ways:

1. Incorporating a response bias scale in the self-report measure such as the Negative Impression
Management Scale of Personality Assessment Inventory.

2. Compare (typically correlate) the test score of self-report measure with a measure of such bias. For
example, if we were to administer a malingering measure, we might also include a response bias scale as
part of our administration.

3. Analyze how prone a scale is to faking, usually by having test takers complete the test under a different
way (e.g., standard direction vs. fake good), and see to what degree the answers differ.

Why does response sets is an issue to Psychological testing?

It is important to carefully consider the role of response styles because not only can they confound test
results but also represent legitimate dimensions that may reflect personality traits. A person who is more
likely to risk guessing on a multiple-choice exam may in real life be more of a gambler, willing to take
risks, perhaps more impulsive, and so on.

Cognitive styles
In reality, individuals constantly differ from each other in so many ways, such as the manner we perceive,
think, remember, solve problems, or perform other intellectual operations. These are what we call cognitive
styles, and a significant number of them have been identified.

As an illustration, in decision-making, some individuals choose to take risks while others choose to be
cautious.

Understandably, these styles can affect test performance, yet at the same time, they possibly reflect the
person's behavior. For example, an impulsive test taker may score poorly on a multiple-choice test by not
carefully answering the test – but it is possible that they may behave like that in work situations.

Faking

“How do we know the test taker is not manipulating his/her answers in a test?”

A literature search will yield numerous studies about the types of faking and different sets of terms.
In this module, we will focus on the term malingering.
 Psychological Assessment 4
Issues and Trends in Psychological Testing

As cited by Domino & Domino in 2006, APA (1987) defined malingering as intentional faking to obtain
some external incentive such as monetary compensation, avoiding punishment for criminal behavior, or
avoiding military duty.

Hence, Faking can be defined as the deliberate, systematic distortion of the responses given to test items
because the respondent wishes to create a particular impression. This impression may be made up of two
components: an emphasis on socially desirable characteristics and/or a denial of negative characteristics.

Usually, it is not impossible that some test taker will fake their answers either to fake good or fake bad.

Faking good is often composed of two independent concepts, typically labeled as “self-deceptive
enhancement” and “impression management.”
Usually, a test taker “fake good” by answering test items that portray personal honesty and virtue.

Faking good is often manifested by a failure to acknowledge commonly held weaknesses and endorsement
of a constellation of unusual virtues.

On the other hand, faking bad happens when a test taker over endorses certain symptoms. Another method
is to endorse specific symptoms that represent the respondent's concept of what mental illness is all about.

Which brings us to this important question, "Why does a person fake good or fake bad in answering tests?"

On personality tests, faking bad may reflect a desire to appear poorly adjusted, perhaps mentally ill. Or it
may mean a cry for help, a negativistic stance, or a manipulative attempt to gain a particular goal (for
example, ending up in a drug rehabilitation center rather than prison).

Random responding.

In testing, random responses should also be considered. Random responding can also be attributed to an
honest error, such as placing answers incorrectly on an answer sheet, lack of understanding as with a person
of borderline intelligence, or willful behavior, such as someone who is passive aggressive and does not
wish to answer a personality inventory.

Trends in Test Scoring

In the early 1960s, the first computer-assisted psychological testing program was developed and used at the
Mayo Clinic in Minnesota. There was an urgent need for a rapid and efficient screening procedure to
determine the nature and extent of psychiatric problems and symptoms since the clinic had a large number
of patients but a small psychology staff.
 Psychological Assessment 5
Issues and Trends in Psychological Testing

Thus, they developed a solution to this problem by the administration of the MMPI using IBM cards, which
could be read by a computer as answer sheets.

A program was written that scored 14 MMPI scales, changed the raw scores to standard scores, and
printed a series of descriptive statements depending on the patient’s scores.
Since then, a number of other computer-based test interpretation systems have been developed,
each one more sophisticated than the earlier one, using data based on configural patterns, special scales,
and indices, taking into account whether the client is a psychiatric patient or a college student, and other
aspects.

Testing in the Twenty-first Century

As the old adage says, "Change is the only permanent thing in the world; with the new advances in computer
technology, the field of psychological assessment has no choice but to ride the waves of the changing times.

A more complex statistical method of test construction as the growing use of computer technology is
adopted.
Likewise, there has been an increasing number of assessment materials of all kinds, many showing a
significant improvement over existing ones or improving previous versions.

One of the current trends today involves computerized testing. Most computers have played a growing
role in item evaluation and construction, as well as in administration, scoring, and interpretation of tests,
and in providing reports. Due to ease of usage, the use of software to input responses directly has made
scoring easier and more accurate, and the overall administration is less time-consuming.

The role of Computers in Testing

The major roles that computers play in psychological testing, as enumerated by Domino & Domino (2006),
are as follows:
(1) It can quickly and efficiently score tests and transform the raw scores into various forms, such as
percentiles, T scores, etc.;
(2) It can present and administer the test directly;
(3) It can be programmed to generate interpretations of the test results.
(4) It can store large amounts of information such as normative data or “banks” of items; and
(5) It can control other pieces of equipment such as optical scanners, video disc presentations, and so on.

Computerized Testing
 Psychological Assessment 6
Issues and Trends in Psychological Testing

One of the major trends in the field of assessment is that it is now possible for the process of testing to be
automated, from initial instructions, presentation of test items, scoring, and transformation of raw scores,
to the final narrative interpretation, and it will probably be automated in the future.

At present, there are a number of promising areas of interfacing are being discovered. This includes the
assessment of reaction time another the use of test-item formats not commonly adapted in traditional paper-
and-pencil tests, such as interactive graphics, movement, and speech – all possible on a computer screen.

COMPUTER-BASED TEST INTERPRETATIONS (CBTI)

Another advantage of computerized testing is that the test results can also be interpreted by the computer.
The computer can be programmed to produce such an interpretation based on the test results. Such CBTIs
are commercially available on a significant number of tests, especially personality inventories.

Types of Computer-based Test Interpretations

There are three kinds of CBTIs Interpretation which are the following:

1. Descriptive reports.

This kind of report interprets the test based only on a single scale without reference to the other scales, and
the comments made are directly reflective of the empirical data and are usually fairly brief.
For example, a high score on an anxiety scale might be interpreted as "Mr. Jones reports that he is
very anxious.” These reports, though limited, can be useful when a multivariate instrument (such as the
PAI) has many scales or when there are many protocols to be processed.

2. Screening reports.
This kind of report interprets the test based on scale relationships rather than the individual scales taken one
at a time. Thus a descriptive statement tied to a particularly high score on a depression scale might appear
only if a second scale is also high; if the second scale score is low, then a different statement might be
printed out.

3. Consultative reports.
This kind of report is the most complex and looks very much like the report a clinician might write on a
client who has been assessed. The intent of these reports is to provide a detailed analysis of the test data,
using professional language, and typically written for professional colleagues.

Problems in Cross-cultural Testing

There are many problems associated with cross-cultural research, particularly with specific tests in different
cultures.
 Psychological Assessment 7
Issues and Trends in Psychological Testing

For instance, translations from one language to another may result in instruments that are not really
equivalent, which will hugely affect the test takers' scores.

Moreover, psychological constructs such as depression, resilience, or intelligence may not have an exact
equivalent across languages and/or cultures.

There may be critical differences between cultures in terms of test sophistication and test-taking behavior.

As cited by Domino & Domino (2016), a test may well have content validity in one country but not in
another.

These authors made use of Holtzman (1968) when he identified these potential problems into three
categories:

(1) cross-national differences,

(2) cross-language differences, and
(3) subcultural differences (such as ethnic origin and degree of urbanization).

Translation of tests from one language to another

Most of the major commercially published available tests, such as the Wechsler Scales, Stanford-Binet
Scales, the PAI, and the MMPI, are available in various languages.

However, translation of one test to another language is not as simple as looking up the equivalent words in
a dictionary.

Domino & Domino (2009) stated that common objects in one culture might be peculiar in another; plain
words in one language may have strong emotional meanings in another.

Some phrases may be translated correctly in a literary

sense but not be colloquially correct; hence, maintaining the language equivalence of an instrument across
languages can be difficult. This might result in an inaccurate interpretation since the test fails to deliver
what it purports to measure in the first place.

Etic and emic approaches.

 Psychological Assessment 8
Issues and Trends in Psychological Testing

In studying cross-cultural approaches, the terms “etic” and “emic” will always appear in the cross-cultural
literature.

For ease of understanding, Etic studies compare the same variable across cultures.
As an example, suppose you are interested in developing a test about resilience and might administer a
resilience scale to nationals of different countries.

Meanwhile, Emic studies focus on only one culture and do not attempt to compare across cultures. We
might administer a suicidal tendency questionnaire to a sample of Japanese adults, for example, and
determine how scores on the questionnaire are related to everyday behaviors in that culture.

The emic approach adds to another problem in a cross-cultural setting. Using a depression scale developed
from a sample of Jewish adults in another population, for example, Filipino adults, will not yield an accurate
result about depression. Hence, we cannot generalize from a test that will not relate to a Filipino setting.

Test bias. Specifically, what are the problems that are perceived to be present in using tests with minorities?
As cited by Domino & Domino in 2006, C. R. Reynolds (1982) lists six kinds of test bias:
1. Inappropriate test content; test items are used that reflect primarily white middle-class experiences to
which minority children have had little or no exposure.
2. Inappropriate standardization samples; the samples are either all white, or ethnic minorities are
underrepresented.
3. Examiner and language bias; lower test scores for minority children reflect their intimidation by a
white examiner who speaks Standard English.
4. Inequitable social consequences; because of bias in tests, minority group members who are already at
a disadvantage in the marketplace are subject to further discrimination.
5. Tests measure different constructs in minority children than they do in majority children.
6. Tests have differential predictive validity; they may be valid for white middle-class children, but they
are not valid for minority children.

References and Supplementary Materials

Books and Journals
Cohen, R. J., & Swerdlik, M. E. (2018). Psychological testing and assessment: An introduction to
tests and measurement. New York, NY: McGraw-Hill Education.
 Psychological Assessment 9
Issues and Trends in Psychological Testing

Kaplan, R.M., & Sacuzzo, D.P. (2018). Psychological testing; principles, applications, and issues.
Belmont, CA: Wadsworth Cengage Learning

Apruebo, R.A. (2010). Psychological Testing Volume 1 (1st ed). Quezon City;
Central Book Supply

Groth-Marnatt, G., Wright, A.J. (2010). Handbook of Psychological Assessment (6th Edition).

Domino, G., & Domino, M. L. (Eds.). (2006). Psychological testing: An introduction. New York,
NY, US: Cambridge University Press.

Coaley, K (2010). An Introduction to Psychological Assessment. London: SAGE

Online Supplementary Reading Materials

Naglieri, Jack & Drasgow, Fritz & Schmit, Mark & Handler, Len & Prifitera, Aurelio & Margolis,
Amy & Velasquez, Roberto. (2004). Psychological Testing on the Internet: New Problems, Old Issues. The
American psychologist. 59. 150-62. 10.1037/0003-066X.59.3.150.

Kamphaus, R. W., Petoskey, M. D., & Rowe, E. W. (2000). Current trends in psychological testing
of children. Professional Psychology: Research and Practice, 31(2), 155-164.
http://dx.doi.org/10.1037/0735-7028.31.2.155

American Psychological Association. (2014). Intelligence and Achievement Testing: Is the Half-Full
Glass Getting Fuller? Retrieved from https://www.apa.org/research/action/intelligence-testing.aspx

American Psychological Association. (2019). Testing issue. Retrieved from

https://www.apa.org/topics/testing/index.aspx
Online Instructional Videos
1. Shelva Alderman. (2014, February142). Ethical and Professional Issues in Psychological Testing [Video
File]. Retrieved from https://www.youtube.com/watch?v=2Hv7Xh-AfBw

2.[CrashCourse ]. (2014, July 21). Controversy of Intelligence: Crash Course Psychology #23. [Video File].
Retrieved from https://www.youtube.com/watch?v=9xTz3QjcloI
3. [Audioversity ]. (2018, September 25). What is Cross-cultural psychology?, Explain Cross-cultural
psychology. [Video File]. Retrieved from https://www.youtube.com/watch?v=CTyLQJyWDUk