Validity of Psychiatric Disorders

Ateev Chandna 1st year JR

Mentor: Dr. Evan Thomas
Chairperson: Dr. Biju Vishwanath
 Validity v/s Reliability
• Validity : How far a test actually measures what it is supposed
to measure, meaning "the nature of reality".
• Reliability : the extent to which a person, diagnosed as having
a psychiatric disorder at one time-point, has, when re-
interviewed or over time, the same disorder.
• Thus, replication by other professionals or over time is the
hallmark of reliability.
• Low Reliability means Low Validity however the converse
doesn’t hold true.
 Concept of Syndrome,Disease & Disorder
• Syndrome is a condition characterized by a particular symptom profile whose etiology, clinical
significance or severity is variable.

• DISEASE is a condition with specific aetiopathogenesis with biomedical connotations.

• The term DISORDER, first introduced as a generic name for the unit of classification in DSM-I in
1952, and is a term midway between a disease or illness and a syndrome, in terms of
consistency, correlates and significance.

• The typical progression of knowledge begins with the identification of clinical manifestations,
i.e. syndrome, and the deviance from the "norm"; understanding of the pathology and etiology
comes much later. There is no fixed point or agreed threshold of description beyond which a
syndrome can be said to be a "disease".

• With our current knowledge, Alzheimer's dementia is a condition that can be called a disease
as its pathophysiology and tentative causes have been elucidated and proven. In contrast,
schizophrenia still does not qualify as a disease
 Concept of Validity
• Validity is a difficult term to define because its meaning differs based on the
context.
• When applied to measuring instruments, validity refers to how well the
instrument measures what it purports to measure, to a disease entity, such as
bacterial pneumonia, validity refers to the evidence that bacteria is the cause.
• The attribution of validity to scientific concepts and theories is in fact an
unending quest: what was regarded as valid knowledge in the past is quickly
superseded by new evidence, and this in the nature of scientific endeavor.
• Zachar proposed the term ‘comparative validity’, to summarize the progression
of scientific knowledge, which “emphasizes rationally justified criteria we use to
say that current theories/models are improvements on past theories/models”
• Aragona concluded that all systems of DSM share the same view of validity as a
“correspondence to external reality”, with the ultimate ideal of validation by
neurobiological data.
 Validity in the area of Mental Health
• The first reference to the concept of validity in the area of mental health was made
in psychology and psychometry by Cronbach and Meehl in 1955 in their article
entitled “Construct Validity in Psychological tests.”
• In this paper, the authors explain the importance of validity to define whether a
psychological test is a good measure of a variable that can not be directly measured
(latent variable).
• They introduced various types of validity:
1. content validity
2. criterion validity
3. construct validity
• These validation criteria are still those currently advocated by the American
Psychological Association (APA) for the validation of scales.
• The types of validity currently employed in the context of psychiatric diagnosis –
construct, content, concurrent and predictive – are borrowed off the shelf of
psychometric theory in psychology
 Construct Validity
• A construct refers to a concept or characteristic that can’t be directly observed, but
can be measured by observing other indicators that are associated with it.
• Construct validity refers to the extent to which a particular measure relates to other
measures consistent with theoretically derived hypotheses.
• Typically, researchers formulate a hypothesis (construct) that a variety of behaviors
will correlate with one another.
• The construct of diagnosis of schizophrenia relies on the young age onset, the
presence of psychosis, the absence of organic cause of psychosis, and positive
family history of schizophrenia.
• The construct of dementia relies on later onset of the illness, impairment of short-
and long-term memory, disturbances of higher cortical function (e.g., aphasia), and
psychological testing consistent with dementia.

• In 1970, Robins and Guze were the first to articulate the elements of construct
validity in psychiatry
 Robin and Guze’s Classification
• Robins and Guze were probably the first to propose formal criteria for
establishing the validity of psychiatric diagnoses, They listed five criteria:
1. clinical description (including symptom profiles, demographic
characteristics, and typical precipitants),
2. laboratory studies (including psychological tests, radiology and
postmortem findings),
3. delimitation from other disorders (by means of exclusion criteria),
4. follow-up studies (including evidence of diagnostic stability), and
5. family studies.
• They used these five criteria to show that “good prognosis schizophrenia
is not mild schizophrenia but a different illness,” a demonstration that
subsequently underpinned the distinction in DSM-III between
schizophrenia and schizophreniform disorder.
 Construct Validity
• Construct validity is woven into the theoretical fabric of social
sciences and psychiatry
• It is very important to note that construct validity is the
product of clinical experience, clinical research, laboratory,
epidemiological, and other research data
• Researchers and clinicians should utilize construct validity to
revisit and redefine content validity of psychiatric disorders
 Content Validity
• Content validity refers to the degree to which an empirical
measurement reflects a specific domain of content.
• In medicine and psychiatry, clinicians agree on important
features that make up a disease, a syndrome, or a disorder.
• Neurologists agree that Parkinson's disease has three main
features: slow movement (bradykinesia), increased tone, and
resting tremor.
• Psychiatrists agree that a patient with schizophrenia has
delusions, hallucinations, disorganization, and bizarre behavior.
• The items that represent the domain or disorder are derived
from the consensus of experts in the field.
 Advantages of content validity
• It facilitates communication among clinicians
• It provides an initial framework for further validation.
• The worldwide use of the DSM and ICD diagnostic criteria
reflects great progress with regard to content validity because
clinicians across the globe use the same nomenclature of
mental disorders and know the specific criteria of each
proposed disorder
 Criterion Validity
• Criterion validity is measuring something that is external to
the measuring instrument itself, called the criterion.
• Physicians agree that diabetes mellitus has four main
symptoms: polyuria, polyphagia, polydipsia, and unexplained
weight loss (content validity).
• In criterion validity, an external measure is used to validate
the diagnosis that is made by content validity. For example in
diabetes, findings would include an abnormal glucose
tolerance test or fasting blood sugar.
 Types of Criterion Validity
• Depending on the timing of the measurements, the criterion
validity can be
1. antecedent,
2. concurrent
3. predictive.
 Kendlers Criteria
• This schema was elaborated by Kendler (10), who
distinguished between antecedent validators (familial
aggregation, premorbid personality, and precipitating factors),
concurrent validators (including psychological tests), and
predictive validators (diagnostic consistency over time, rates
of relapse and recovery, and response to treatment). He then
used these expanded criteria to demonstrate that paranoia
(simple delusional disorder) is probably a distinct syndrome
rather than a mild form of schizophrenia or a subtype of
affective illness
 LEAD Standard by Spitzer
• LEAD (longitudinal evaluation) is done by expert clinicians who
utilize all the data available. The LEAD standard is an important
step toward obtaining the best estimate diagnosis by requiring
expert clinicians to utilize all the available data over time,
including information from family members, hospital records,
psychological evaluation, and laboratory results.
• Limitations- The requirement of LEAD to have expert clinicians
make independent assessments, discuss diagnostic
disagreement, and make a consensus diagnosis accounts for
the difficulty in implementing the LEAD standard and its
limited use
 Criticism of the Validity Criterion
• The weakness of the validity criteria of both Robins and Guze and Kendler
was that those criteria implicitly assumed that psychiatric disorders are
discrete entities and that the role of validity criteria is to determine
whether a disorder, such as “good-prognosis schizophrenia” or paranoia, is
a valid entity in its own right or a mild form or variant of some other
entity.
• The possibility that disorders might merge into one another with no
natural boundary in between—what Sneath called a “point of rarity,” but
what is better regarded as a zone of rarity—was simply not considered.
• The variation between extensive, handicapping symptoms or pathology
and an almost total absence of symptoms or pathology appears to be
continuous in each case with no demonstrable zone of rarity. As a result,
the boundary between normality and disorder has to be decided
arbitrarily on pragmatic grounds.
 Criticism of the Disorder Approach
• Traditionally, mental illnesses have been conceptualized as disorders that are diagnosed on the basis
of the number and type of symptoms, and the presence of distress or impairment.

1. Attempts to demonstrate natural boundaries between related syndromes (mania and schizophrenia)
or a common syndrome and normality (depression, anxiety, personality) by locating a "zone of
rarity" between them has failed.

2. If disorders are independent categories, the coexistence of other disorders within a specific disorder
should be just by chance. In fact, this is not the case. Psychiatric comorbidity seems to be the rule
rather than the exception. Across the lifespan, most individuals move in and out of comorbid
diagnostic categories (e.g., anxiety disorders and depression).

3. The separation between depression and anxiety has been criticized on many grounds. The most
common form of affective disorder is actually mixed anxiety-depression, comorbid anxiety and
depression showed a greater stability in a 15-year follow-up study than each disorder alone. Results
from family and twin studies suggest common etiologic factors underlying both conditions. Despite
claims of specificity, the same classes of drugs have been increasingly used to treat the whole range
of anxiety and depressive disorders.
 Criticism of the Disorder Approach
4. Research based on diagnostic categories can suffer from problems with heterogeneity because of the
varied ways people can qualify for a symptom-based disorder diagnosis. Two people, in some cases,
can be diagnosed with the same disorder despite having few symptoms in common. This makes it
difficult for researchers to pinpoint specific aspects of disorders because the neurobiological
mechanisms may differ greatly among patients who share little to no symptomatology.
5. Researchers seeking to reduce heterogeneity in their samples often limit participants to those with
“pure” diagnoses. In order to do this, they do not enroll individuals representing the larger spectrum
of functioning or those with related disorders; however, this type of variation can be important for
understanding the underlying contributors to mental health and illness.
6. Clinical criteria for defining a disorder, are somewhat arbitrary. Research indicates that there are
important similarities between those whose symptoms meet the criteria for a disorder versus those
who just miss the cutoff for diagnosis due to fewer and/or less severe symptoms. To understand the
full spectrum of mental health and illness, it is important to adopt dimensional conceptualizations.
Therefore, dividing research subjects into two groups based on symptom counts may obscure
important information about the ways in which psychopathology gradually emerges across
development, how risk factors operate, and how quantitative outcomes of prevention and treatment
trials can be implemented.
 Importance of Disorder approach
• This view of mental disorders – and the resulting diagnostic systems
– provides benefits such as reliability and ease of diagnosis across a
variety of contexts
• It influences estimates of psychiatric disorders in the community
• It has potential legal implications in criminal cases and in awarding
disability benefits.
• Lack of a clear conceptual definition can contribute to abuses of
psychiatric diagnoses as a means of controlling or stigmatizing
socially undesirable behavior (e.g., misuse of psychiatric diagnosis in
former USSR and China to jail dissidents of the system).
• Lack of a proper definition of psychiatric disorder reduces confidence
in our discipline among our colleagues and the general public.
 Recent Approaches to Validity
Andreasen’s approach
• proposed “a second structural program for validating psychiatric
diagnosis” by listing several additional validators—molecular
genetics and molecular biology, neurochemistry, neuroanatomy,
neurophysiology, and cognitive neuroscience—that are all
potentially capable of linking symptoms and diagnoses to their
neural substrates.
Kendell and Jablensky (2003):
• One of two conditions be met
1. Zones of rarity between one syndrome and others
2. Clear qualitative differences in biological underpinnings of one
syndrome and another.
 Recent Approaches to Validity
In the last 2 decades, several authors have defended 2 different
approaches to validity:

1. The realistic one - According to a realistic view, validity

depends on how the theoretical constructs correspond to the
external reality as it is. In this case, only the validity criteria–
based on the physical cause and neurobiological location of
the mental disorders are valid. The aim is to gradually
approach a single and irreversible solution (the search for a
specific neurobiological cause)
 Recent Approaches to Validity
• In modern times, new ways have been proposed to explore the complex
and multidimensional relationship between mental symptoms and
neurobiological activity.
• Among them, are new versions of the translational model, which seek new
strategies to find biological correlates of the psychiatric symptoms through
neuroimaging techniques.
• New research criteria, especially Research Domain Criteria (RdOC), have
attempted to approximate the validity of psychiatric classifications to what
actually happens in neural Circuits.
• Moreover, the complex role of hermeneutics in shaping a neurobiological
signal as mental symptom has been considered in models trying to
reconcile neurobiology and humanities
 Recent Approaches to Validity
2)The instrumental one -According to a more instrumental view,
validity corresponds to the adequacy of the constructs to the
reality as we see it and how we deal with it (not necessarily as it
is).
• This happens, for example, with the clinical criteria of validity
in psychiatry (clinical presentation, clinical evolution, distress,
disability, etc.)
 Recent Approaches to Validity
• Shaffner finds an integrative position (between realism and
instrumentalism), stating that until we find the final truth ,we
must have an intermediate position.
• Therefore it is essential that we continue to seek useful,
instrumental clinical criteria without forgetting to chase the
neurobiological bases of mental illness.
• Some authors such as Kincaid and Sullivan prefer to use the
concept of natural kind to designate mental illness within a
vision of realistic validity and practical kind within a vision of
instrumental validity
 Phases of Validity

• Phase I. The patient has specific complaint(s) addressed to the

clinician. The clinician needs to determine whether the
patient has real symptoms or the patient is feigning symptoms
for secondary gain. The clinician needs to measure the
symptoms, observe the patient's behavior, and make a
provisional diagnosis. Content validity plays an important role
in this phase
 Phases of Validity

• Phase II. 1) The clinician collects more data (e.g., from family,
old records) and orders laboratory, psychological, or imaging
studies as indicated. 2) The clinician formulates an entity with
suffix disease, syndrome, or disorder. 3) The clinician initiates
treatment to alleviate the suffering of the patient.
 Phases of Validity

• Phases I and II happen during the first visit of the patient

because the patient needs immediate treatment and cannot
wait for full validation.
• Moreover the clinician's goal is not quest of knowledge per
se, but the ability to use the available knowledge and skills to
prevent and diminish the suffering and disabilities of the
patient. Content and construct validity play important roles in
phases I and II.
 Phases of Validity

• Phase III. The clinician collects more evidence that may

confirm or refute his initial diagnosis. The course and the
progression of the illness and the response to treatment can
provide more valuable information to the clinician. The new
evidence collected in this phase can result in redefining or
changing the diagnosis. Construct validity continues to play a
major role in this phase. Validity is an ongoing process and
may continue beyond the termination of a particular doctor-
patient relationship.
 Proposals to improve validity skills of
clinicians
Aboraya and Compton proposed the acronym DR.SEE:
• Data and knowledge
• Reference guide.
• Rating scales and structured interviews
• Experience
• External Validators
 Data and knowledge

• Data and knowledge- Clinicians use their skills to build a

rapport with patient, use the appropriate proxy information
sources and observe the patient's behavior to get the most
valid data. The knowledge and education of mental health
clinicians is key to adequately make a diagnosis through this
clinical assessment.
 Reference guide
• Reference definition refers to the definition of psychiatric symptoms
and their levels of severity.
• First, clinicians need to define and agree among themselves on the
definitions of psychiatric symptoms.
• Second, when clinicians ask patients questions, they should convey
the meaning of the questions to their patients. Additionally, clinicians
should understand the expression of symptoms in different cultures.
• Psychiatric symptoms reported by the patient, assessed and
observed by the clinician are the main source of information the
clinician utilizes to diagnose and treat the patient.
• The measurement of psychiatric symptoms is still the main source to
assess whether the diagnostic criteria of the disorder are met.
 Rating Scales and Structured Interviews

• The use of standardized or semistandardized instruments helps the clinician in many ways.
• First, standardization forces the clinician to cover all the areas of psychopathology under question.
• Second, standardization provides similarities in the way questions are asked and minimizes
variability among clinicians. Standardization applies to the detailed structured or semistructured
interviews.
• Rating scales take less time and can help clinicians to obtain more accurate data. These also provide
benchmarks for comparing a particular patient with those who have been studied in treatment
trials.
• These rating scales can be especially useful when information needs to be obtained from others
who observe the patient's behavior (e.g., parents and teachers for the Connors, caretakers for the
IADL) or to quantify observable signs of illness and measure changes with treatment (e.g., level of
irritability for the YMRS or degree of depressed mood for the HAM-D).
• Although the use of structured interviews can provide a more accurate diagnosis in comparison
with routine clinical diagnosis, most clinicians do not use them for three main reasons. First,
structured interviews are time-consuming; a SCAN or SCID interview lasts from 1 to 2 hours.
Second, structured interviews are cumbersome, complicated, and interfere with establishing a
rapport with the patient. Third, many structured interviews require lengthy and extensive training
 Experience
• Mental health clinicians may be psychiatrists, clinical
psychologists, therapists, or others who have actual experience
and contact with patients with mental disorders
• Clinical experience with psychiatric abnormalities and the
development of skills to elicit them and ascertain their
significance have been viewed as the reference standard of
psychopathology assessment.
• Clinical experience is indispensable when it comes to diagnoses
of psychotic, bipolar, and personality disorders, especially with
regard to judging the significance of symptoms.
• The longer the experience of the clinician, the more likely the
diagnosis is accurate
 External Validators

• External validators are elements external to the disease definitions

• Structural magnetic resonance imaging (MRI), functional MRI (fMRI),
magnetic resonance spectroscopy (MRS), single proton emission computed
tomography (SPECT), and positron emission tomography (PET) are some of
these new brain-imaging techniques.
• Andreasen has used the term new external validators for these brain
imaging techniques and other new branches of neuroscience and has
emphasized their importance in understanding the relationship between
individual symptoms and the changes in structure and/or function of the
brain.
• one area where neuroimaging is actually emerging as a diagnostic external
validator is in the use of PET scans to detect early Alzheimer's disease.
• The new external validators hold promise of validating psychiatric diagnosis
and predicting treatment response in psychiatry in the near future.
 Conclusion
• To quote Jaspers Validity is an “an objective which one cannot reach
since it is unending, but all the same it indicates the path for fruitful
research and supplies a valid point of orientation for particular
empirical investigations”
• At present there is little evidence that most contemporary
psychiatric diagnoses are valid, because they are still defined by
syndromes that have not been demonstrated to have natural
boundaries.
• This does not mean, though, that most psychiatric diagnoses are not
useful concepts. In fact, many of them are invaluable. But, because
utility often varies with the context, statements about utility must
always be related to context, including who is using the diagnosis, in
what circumstances, and for what purposes.
 References
• Jablensky A. (2016). Psychiatric classifications: validity and utility. World
psychiatry : official journal of the World Psychiatric Association (WPA),
15(1), 26–31. https://doi.org/10.1002/wps.20284
• Aboraya A, France C, Young J, Curci K, Lepage J. The Validity of
Psychiatric Diagnosis Revisited: The Clinician's Guide to Improve the
Validity of Psychiatric Diagnosis. Psychiatry (Edgmont). 2005;2(9):48–55.
• telles correia, Diogo. (2017). The concept of validity throughout the
history of psychiatry. Journal of evaluation in clinical practice. 23.
10.1111/jep.12750.
• Kendell R, Jablensky A. Distinguishing between the validity and utility of
psychiatric diagnoses. Am J Psychiatry. 2003;160:4–12.
• https://www.nimh.nih.gov/research/research-funded-by-nimh/rdoc/
about-rdoc.shtml
THANK YOU