You are on page 1of 6

Clinical Neuropsychiatry (2012) 9, 3, 123-128



Julie Norgaard and Josef Parnas

Objective: The contemporary methodology in obtaining psychopathological information relies almost exclusively
on the use of structured questionnaires and interview schedules. These interviews yield high interrater reliability and
reduce cost. The assessments of anomalous self-experience and of mental status (disorders of experience and expression)
are often considered as soft, subjective and hence unreliable. In spite of the advantages of the structured interviews,
concerns have been raised about the epistemological coherence and the validity of the structured interviews.
To examine the interrater congruens between experienced clinicians with a semi structured, phenomenologically
oriented psychopathological interview assessing anomalous self-experience and mental status.
Method: Seventeen inpatients were interviewed by one of the raters, with both raters were present. The interview
comprised of a thorough psychosocial history and the EASEscale.
Results: The interater-congruens for the total EASE-scale showed nearly perfect agreement with kappa=0.94.
The interrater-reliablility for the MSE assessment showed substantial to complete agreement with kappa above 0.81,
except for four items: Withdrawn/shy (kappa=0.77), Suspicious, guarded and hostile (kappa=0.77), Raport insecure
and anxious (kappa=0.76), Restless (kappa=0.64).
Conclusions: High interrater-congruens can be achieved for the assessment of anomalous self-experience and
mental status in the context of conversational, phenomenological-oriented semi-structured interview.
Limitations: The major limitation of the study is a relatively small sample size, conditioned by the time-consuming
nature of the individual interviews. Second, we should have included a measurement of reliability in a less experienced-
recently EASE-introduced rater, since it is that kind of researchers that are typically enrolled for the empirical data

Key words: interrater agreement, semi structured, phenomenological, interview, examination, anomalous, self-experience,
EASE, mental, status.

Declaration of interest: Julie Norgaard was speakers bureau with Bristol - Myers Squibb

Julie Norgaard, MD1 and Josef Parnas, MD, Dr.MSc.1,2

University Department of Psychiatry, Mental Health Center Hvidovre
Mental Health Center Hvidovre, Copenhagen, Denmark
Center for Subjectivity Research, University of Copenhagen

Corresponding author
Dr. Julie Nordgaard
Mental Health Center Hvidovre, Broendbyoestervej 160, 2605 Broendby,
Denmark. Ph. no. +45 38 64 57 33

Introduction Initially, the structured interviews, conducted by

trained non-clinicians, were used in large scale
Contemporary methods of obtaining psycho- epidemiological surveys for inexpensive estimates of
pathological information consist almost exclusively in the prevalence of disorders such as phobias or
the use of structured questionnaires or checklists alcoholism (e.g. Epidemiological Catchment Area
(Faravelli 2004) and self-rating instruments. This studies) (APA 2000, Faravelli 2004). With time,
methodology was introduced as a component of the however, all diagnostic categories came to be studied
reliability-driven operational revolution. Robert in this way. Currently, the structured interviews rapidly
Spitzer, in his famous programmatic article, entitled find an inroad to clinical practice as well. Main-stream
Are clinicians still necessary? strongly advocated for psychiatry has become explicitly behaviourist, with a
the use of structured interviews as a straightforward and general subjectivity-phobic attitude (Stanghellini
unproblematic solution to psychiatrys nagging 2004).
reliability problem (Cooper et al. 1972, Spitzer 1983). There have been critical voices questioning the
Psychiatrists were both unreliable and expensive. epistemological foundations of the operationalist
2012 Giovanni Fioriti Editore s.r.l. 123
Julie Norgaard and Josef Parnas

assumptions behind the structured interviewing (Parnas clinical diagnosis allocated by the clinician in charge.
and Bovet 1995, Parnas et al. 2002, Parnas et al. 2008, Yet, we applied the following exclusion criteria: primary,
Sadler et al. 1994, Stanghellini 2004). Similar worries or clinically dominating alcohol/substance abuse,
have been expressed concerning the validity of a history of brain injury, mental retardation, organic brain
checklist approach to diagnosis and its dehumanizing disorder, and age above 46 years. Because of ethical
impact on clinical psychiatry (Andreasen 2007, Ghaemi concerns, we excluded involuntarily admitted and legal
2012, Parnas et al. 2008). To the best of our knowledge, patients.
the advocates of structured interviews and operational
criteria (with few exceptions, e.g., (Hempel 1965)
never presented a convincing epistemological defence The interviewers
or rationale for the structured-operational approach,
apart from persistent emphasis of the reliability issue The interviewers alternated in performing the
We have argued in theoretical and empirical interview, with the other psychiatrist having observer
publications that we consider phenomenology to be status during the session. Thus both raters were present
indispensable for psychiatry in its praxis of clinician- at the interview (JN & JP). This procedure improves
patient interaction, diagnosis, therapy, theoretical work, reliability by providing equal access to paraverbal
formation of pathogenetic hypotheses and empirical data information (Cermolacce et al. 2010). The observer was
collection. Psychiatric phenomenology is an approach allowed to ask a couple of clarifying questions at the
that targets a faithful description of experience, end of the session.
expression and existence (Parnas et al. 2002, Parnas et JN was, at the time of the study, an experienced
al. 2008) A cardinal aspect of the phenomenological clinician with more than 10 years of clinical experience
stance is a suspension of the background or tacit and with specific training in psychopathology and
assumption of belonging to a shared world, in which research experience in the diagnostic issues in the
some things are and others are not. Instead, this very domain of schizophrenia spectrum disorders (Jakobsen
assumption is considered as an aspect of experience that et al. 2005, Jakobsen et al. 2007). She had completed
can be made explicit. This allows exploring the sense the Examination of Anomalous Self-Experience (EASE)
of reality itself and the variations and alterations of its course and had performed several, supervised EASE-
structure. This specifically phenomenological approach interviews prior to the study. The second rater (JP), a
does indeed facilitate a heightened appreciation of the consultant psychiatrist, is the principal author of the
ways in which, and the extent to which, other peoples EASE scale (Parnas et al. 2005b), and has decades of
experience of the world can differ from ones own. We clinical and research experience in the domain of
suspect that all curious clinicians, even if implicitly and schizophrenia spectrum disorders.
unknowingly, adopt what can be called a weak variant
of the phenomenological stance.
The goal of this study (a part of a larger The interview-approach
psychopathological project) was to examine the
agreement between experienced, similarly educated The interview was semi structured. That implies
clinicians, in the process of identifying and typifying the following features in the context of this study. The
anomalous self-experience assessed with a semi structure of the interview is assured by the obligation
structured, phenomenologically-oriented psychopa- of the interviewer to be able to score all the items of
thological checklist (Examination of Anomalous the interview schedule, implying that all items must
Experience (EASE)). In addition, we examined the be explored. Yet, the concrete, practical conduct and
agreement of the ratings of subtle expressive features the sequence of the interview is dictated by the
characteristic of the schizophrenia spectrum disorders. dynamics and context of the encounter, i.e. the style is
Both of these dimensions of descriptive psycho- free and conversational. The questions are contextually
pathology (disorders of experience and expression) are adapted and follow the logic of the patients narrati-
often considered as soft, subjective, and hence ve, yet with a possibility to ask for more detail and
unreliable. further examples. The patient is encouraged to speak
freely, is rarely interrupted and is given time for
reflection and recollection. The interviewer steers the
Sample and Methods interview in the desired direction through questions
and comments. Typically, the interview starts with a
The sample detailed psychosocial history, which creates the
departure points for the EASE-specific probing. All
Seventeen patients were randomly drawn from the relevant responses from the patient are examined in
inpatient population of the Mental Health Center depth: the patient is prompted for more detailed
Hvidovre, University of Copenhagen, a center that descriptions and clarifications. In other words, a simple
provides psychiatric in-patient facilities to a population affirmative answer is never accepted without the
of 150.000 in a catchment area within the City of patients own examples and descriptions. The
Copenhagen. The patients participated on the condition interviewers task is to detect invariances across
of informed consent. The interviews took place on several related experiences or across several examples
prescheduled days in 2010-2011. of the same experience. The metaphorical responses
The sample consisted of eight men and nine call for (if possible) a distinction between their con-
women, with an average age of 28,2 years (range 18-45 crete or abstract subjective significance. Each
years). Patients were included independently of their interview session took 60-120 min.

124 Clinical Neuropsychiatry (2012) 9, 3

Agreement in a phenomenological inerview

The schedule modalities of intentionality (kappa 0.73) and Dimi-

nished presence (kappa 0.77).
The interview comprised a thorough psychosocial For MSE, the interrater-agreement was perfect to
history (including circumstances leading to the hospital complete, with kappa values above 0.81, except for 4
admission) and the EASE-scale was scored on the items where the agreement was substantial: Withdrawn/
lifetime basis. After the interview, the raters scored the shy (kappa .77), Suspicious, guarded and hostile
mental status examination sheet and the EASE coding (kappa 0.76), Raport insecure and anxious (kappa
sheet. The EASE is a descriptive-psychopathological 0.76) and Restless (kappa 0.64).
checklist consisting of 57 main items. The items of the Although the samples become very small if
EASE scale are described through the typical partitioned into subgroups, we did not observe major
statements, collected from clinical samples and existing changes in kappa levels as a function of the sex of the
textual sources examined during the period of EASE patient or of which one of the two raters that was
construction (Berze 1914, Gross et al. 1987, Parnas and interviewing.
Handest 2003). The EASE contain 5 thematic domains:
1) stream of consciousness (and cognition), 2) self-
presence, 3) bodily experience, 4) transitivism, and 5) Discussion
existential reorientation. The interviewer was free to
explore the items in the sequence that was felt was The results of this study are in line with previous
appropriate and adequate to the subjects own concerns reports on the reliability of psychometric measures of
and responses. subjective experience: the BSABS scale by our own
The list of items scored on the Mental Status group (Vollmer-Larsen et al. 2007) and the EASE scale
Examination (MSE) was specifically created for the by a Norwegian study (Moller et al. 2011) We have,
rating of subtle expressive features in schizophrenia however, chosen not to present our result as a standard
spectrum and used in the US-DK adoption-, high-risk reliability study. Given the fact of the raters (JN and
studies (Kety 1975, Kety 1968, Parnas 1993 ) and the JP) have been involved together in studying
Copenhagen Linkage Study (Matthysse et al. 2004). schizophrenia for several years, our results are not
directly applicable to a setting of a standard reliability-
Data analysis The EASE-kappa values in the present study are
therefore higher than those reported in the study by
The main EASE items and the Mental Status Mller et al. (Moller et al. 2011) In addition, this
Examination items were scored dichotomously as 0 difference may have been amplified by two other factors:
(absent or questionably present) or 1 (definitely present, 1) the joint presence of the raters during the live
all severity levels). The descriptive congruence was interview provides a better access to paraverbal and
assessed by calculating Cohens K (kappa), a measure interactive information that increases reliability. For
of agreement between two raters, corrected for example, such joint presence amplified reliability in the
agreement occurring by chance and taking frequency studies of bizarre delusions (Cermolacce et al. 2010).
into account. The typical interpretation of K values is 2) The observers possibility of making a couple
0.00 poor agreement; 0.00-0.20 light agreement; 0.21- of supplementary or clarifying questions at the end of
0.40 fair agreement; 0.41-0.60 moderate agreement; the session increases the clarity of potentially vague
0.61-0.80 substantial agreement; 0.81-0.99 nearly initial descriptions.
perfect agreement; 1.00 complete agreement (Software On the whole, our study demonstrates that it is
2001). possible to achieve high agreements in the assessment
A minimum cumulative frequency threshold of of single psychopathological items in a conversational,
10% between the two raters, i.e. estimated frequency phenomenologically oriented interview. In such a type
by rater 1 + rater 2 > 10% of the sample (at least 2 of interview, the primary source of information is the
subjects out of the 17 subjects), were considered because patients narrative with self-description of experience
the kappa values for very infrequent items are not and behaviour. The raters task is to identify, from the
meaningful. conversational flow, the chunks of information that are
psychopathologically relevant and eventually serve to
individuate those chunks as signifying as this or that
Results particular experience. This is a more demanding
situation than a rating of the symptom-presence on the
The seventeen patients had following primary basis of observing an interview with a question-answer
clinical diagnoses, allocated by the clinician in charge: structured sequence. Second, reliability studies often
major depressive disorder (n=3), adjustment disorder use ratings of video-clips, a procedure in which the very
(n=1), schizophrenia (n=6), schizotypal disorder (n=4) selection of the clips prefigures in advance what seems
and personality disorder not otherwise specified (n=3). be a relevant symptom.
The interrater agreements appear in tables 1 and This study also shows excellent congruence in the
2. The interrater kappa value was 0.94 for the total assessments of subtle disorders of expression, such
EASE. For the five domains of the EASE, the interrater as qualities of rapport, thinking, gesture, and ap-
agreements were also high (kappa values between 0.89 pearance, all highly pertinent to diagnosis of the
and 0.95). For the individual EASE-items the agreement schizophrenia spectrum disorders.
was perfect to complete with kappa values above 0.81 Notably, there was a perfect agreement on the
except for two items: Inability to discriminate suspicion of autism, i.e. a sense that the patient lives

Clinical Neuropsychiatry (2012) 9, 3 125

Julie Norgaard and Josef Parnas

Table 1. Interrater congruens for the EASE items

% agree-
Rater 1 Rater 2 ment on
n Frequ- n Frequ- indivi-
scored ency in scored ency in Cohens dual
present % present % Kappa level
Item no. And designation
1.1 Thought interference 4 24% 4 24% 1 100%
1.2 Loss of thought ipseity 3 18% 3 18% 1 100%
1.3 Thought pressure 10 59% 10 59% 1 100%
1.4 Thought block 4 24% 4 24% 1 100%
1.5 Silent thought echo 0 0% 0 0% n.c 100%
1.6 Ruminations - obessions 13 76% 13 76% 1 100%
1.7 Perceptualization of inner speech or thought 9 53% 8 47% 0,89 94%
1.8 Spatialization of experience 3 18% 4 24% 0,87 94%
1.9 Ambivalence 7 41% 7 41% 1 100%
1.10 Inability to discriminate modalities of intentionality 5 29% 3 18% 0,73 88%
1.11 Disturbance of thought initiative or intentionality 5 29% 4 24% 0,89 94%
1.12 Attentional disturbances 3 18% 3 18% 1 100%
1.13 Disorder of short-term memory 6 35% 6 35% 0,9 100%
1.14 Disturbance of time experience 2 12% 3 18% 0,83 94%
1.15 Discontinous awareness of own actions 2 12% 2 12% 0,87 100%
1.16 Disconrdance between expression and expressed 4 24% 4 24% 1 100%
1.17 Disturbance expressive language function 4 24% 4 24% 1 100%
Domain 1 0,94 98%
2.1 Diminished sense of basic self 6 35% 7 41% 0,9 94%
2.2 Distorted first-person perspective 2 12% 2 12% 1 100%
2.3 Other states of depersonalization (self-alienation) 1 6% 2 12% 1 94%
2.4 Diminished presence 5 29% 4 24% 0,77 94%
2.5 Derealization 5 29% 5 29% 1 100%
2.6 Hyperreflectivity; increased reflectivity 10 59% 10 59% 1 100%
2.7 I-split 2 12% 2 12% 1 100%
2.8 Dissociative depersonalization 1 6% 1 6% n.c 100%
2.9 Identity confusion 2 12% 1 6% 0,9 94%
2.10 sense of change in relation to chronological age 1 6% 1 6% n.c 100%
2.11 Sense of change in relation to age 4 24% 3 18% 0,89 94%
2.12 Loss of common sense/natural evidence/perplexity 4 24% 4 24% 1 100%
2.13 Anxiety 10 59% 10 59% 1 100%
2.14 Ontological anxiety 0 0% 1 6% n.c 94%
2.15 Diminished transparency of conciousness 1 6% 1 6% n.c 100%
2.16 Diminished initiative 8 47% 8 47% 1 100%
17 Hypohedonia 10 59% 10 59% 1 100%
2.18 Diminised vitality 4 24% 5 29% 0,89 94%
Domain 2 0,95 98%
3.1 Morphological change 0 0% 0 0% n.c 100%
3.2 Mirror-related phenomena 6 35% 6 35% 1 100%
3.3 Somatic depersonalization 2 12% 2 12% 1 100%
3.4 Psychophysical misfit 1 6% 1 6% n.c 100%
3.5 Body disintegration 0 0% 0 0% n.c 100%
3.6 Spatialization 2 12% 1 6% 0,83 94%
3.7 Cenesthetic experiences 7 41% 6 35% 0,9 94%
3.8 Motor disturbances 2 12% 2 12% 1 100%
3.9 Mimetic experinces 0 0% 0 0% n.c 100%
Domain 3 0,95 99%
4.1 Confusion with the other 0 0% 0 0% n.c 100%
4.2 Confusion with ones own specular image 0 0% 0 0% n.c 100%
4.3 Threatening bodily contact 2 12% 3 18% 0,83 94%
4.4 Passivity mood 2 12% 2 12% 1 100%
4.5 Mimetic experiences 2 12% 2 12% 0,83 100%
Domain 4 0,89 99%
5.1 Primary self-reference phenomena 6 35% 6 35% 1 100%
5.2 Feeling of centrality 3 18% 3 18% 1 100%
5.3 Feeling as if subjects exp. Field is only extant reality 2 12% 2 12% 1 100%
5.4 Feelings of extraordinary creative power or insights 2 12% 2 12% 1 100%
5.5 Feelings that the experienced world is not real 3 18% 2 12% 0,83 94%
5.6 Magical ideas linked to the subjects way of experiencing 3 18% 2 12% 0,83 94%
5.7 Existential or intellectual change 1 6% 1 6% n.c 100%
5.8 Solipsistic grandiosity 2 12% 2 12% 1 100%
Domain 5 0,95 99%

Total EASE scale 0,94

All kappas p<0.001

126 Clinical Neuropsychiatry (2012) 9, 3

Agreement in a phenomenological inerview

Table 2. Interrater congruens for the Mental Status Examnination

Rater 1 Rater 2 ment on
n scores Frequ- n scores Frequ- Cohens individu-
present ency present ency kappa al level
Language and thougths
Speech difficult to understand 5 29% 6 35% 0,87 94%
Quiet, mumling voice 4 24% 3 18% 0,82 94%
Derailment or semantic disturbances 4 24% 5 29% 0,85 94%
Vagueness 9 53% 8 47% 0,88 94%
Poverty of speech 2 12% 2 12% 1 100%
Autism suspected 9 53% 9 53% 1 100%
One-way emotional contact 12 71% 13 76% 0,85 94%
Withdrawn/shy 2 12% 3 18% 0,77 94%
Suspecious, guardedt and hostile 7 41% 7 41% 0,76 100%
Rapport disturbed by psychotic symptoms
or thought disorders 8 47% 7 41% 0,88 94%
Rapport insecure and anxious 9 53% 9 53% 0,76 100%
Rapport silly and inadequate 4 24% 5 29% 0,85 94%
Indifferent rapport 2 12% 2 12% 1 100%
Evasive eye contact 4 24% 5 29% 0,85 94%
Frozen 0 0% 2 12% 88%
Lack of gesture 6 35% 6 35% 1 100%
Parakinesia 3 18% 3 18% 1 100%
Lack of variation in facial expression 5 29% 5 29% 1 100%
Limp handshake 2 12% 2 12% 1 100%
Voice unmodulated 6 35% 5 29% 0,87 94%
Appearance and behaviour
Tense and anxious 11 65% 11 65% 1 100%
Cooperating badly 5 29% 5 29% 1 100%
Hostile 1 6% 2 12% 1 94%
Restless 1 6% 2 12% 0,64 94%
behaviour/clothes/appearance 4 24% 4 24% 1 100%
Do not relax during the interview 9 53% 8 47% 0,88 94%
Increasinly tense during the interview 2 12% 2 12% 1 100%

All kappas p<0.001

in a private universe of fundamentally altered psychiatric inquiry (Stanghellini 2004). The psychiatric
subjectivity (or frame of reference), which only partially interview is a second person approach, guided by
overlaps the shared, intersubjective, world. In a phenomenological distinctions, a process through which
previous, poly-diagnostic study from our group, a we extract, represent and individuate from the flow of
judgment on the suspicion of autism, based on a the patients subjective life, certain repeatable
review of the case-summaries, achieved interrater kappa constellations of experience, certain meaningful wholes.
of 0.6 (Parnas et al. 2005a). The higher agreement A psychiatric symptom or sign only emerges as an
obtained in the current study probably reflects the individuated entity (as this or that symptom) in a context
methodological issues addressed above, but also the fact of other, simultaneous, preceding and succeeding,
that EASE inquiry provides a unique access into the experiences.
patients modes of experience, and alterations in these Our study shows that high agreement between
modes of experience define the concept of autism clinicians can be achieved for the assessment of anomalous
(Arnfred et al. 2008, Parnas et al. 2002). self-experience and mental status in a phenomenologically
It is beyond the scope of this paper to discuss its oriented semi-structured interview. This is an important
wider epistemological implications. Suffice to say that finding because the subjective dimension of descriptive
the current drive towards the structured interviews, must psychopathology is often considered unreliable, and
presuppose that the symptoms and signs have a nature therefore unfit for empirical study.
and a mode of being akin to that of physical, thing-like
entities, which unproblematically announce their ready- Acknowledgements: The study was supported by the
made presence in response to a predesigned stimulus- Danish National Research Foundation and by a PhD-
question. Such epistemology is inadequate for the grant from the University of Copenhagen (Dr Nordgaard)

Clinical Neuropsychiatry (2012) 9, 3 127

Julie Norgaard and Josef Parnas

References families of adopted schizophrenics. The Transmission of

Schizophrenia. Pergamon Press ltd.
Andreasen NC (2007). DSM and the death of phenomenology Matthysse S, Holzman PS, Gusella JF, et al. (2004). Linkage of
in america: an example of unintended consequences. eye movement dysfunction to chromosome 6p in
Schizophr Bull 33, 1, 108-12. schizophrenia: additional evidence. Am J Med Genet B
Arnfred SM, Hansen LK, Parnas J, et al. (2008). Regularity Neuropsychiatr Genet 128B, 1, 30-6.
increases middle latency evoked and late induced beta brain Moller P, Haug E, Raballo A, et al. (2011). Examination of
response following proprioceptive stimulation. Brain Res anomalous self-experience in first-episode psychosis:
1218, 114-31. interrater reliability. Psychopathology 44, 6, 386-90.
American Psychiatric Association (2000). Handbook of Parnas J, and Bovet P (1995). Research in psychopathology:
Psychiatric Measures. American Psychiatric Association. epistemologic issues. Compr Psychiatry 36, 3, 167-81.
Berze J (1914). Die primre Insuffizienz der psychischen Parnas J, Bovet P, and Zahavi D (2002). Schizophrenic autism:
Aktivitt. Franz Deuticke. clinical phenomenology and pathogenetic implications.
Cermolacce M, Sass L, and Parnas J (2010). What is bizarre in World Psychiatry 1, 3, 131-6.
bizarre delusions? A critical review. Schizophr Bull 36, 4, Parnas J, and Handest P (2003). Phenomenology of anomalous
667-79. self-experience in early schizophrenia. Compr Psychiatry
Cooper JE, Kendell RE, Gurland BJ, Sharpe L, Copeland J 44, 2, 121-34.
(1972). Psychiatric Diagnosis in New York and London, Parnas J, Licht DM, and Bovet P (2005a) (eds). Personality
Vol. 20. Oxford university Press. Disorders. Cluster A Personality Disorders: A review, Vol.
Faravelli C (2004). Assessment of psychopathology. Psychother 8. John Wiley & Sons.
Psychosom 73, 3, 139-41. Parnas J, Moller P, Kircher T, et al. (2005b). EASE: Examination
Ghaemi SN (2012). Taking disease seriously: beyond pragmatic of Anomalous Self-Experience. Psychopathology 38, 5,
nosology. In K Kendler and J Parnas (eds) Philosophical 236-58.
Issues in Psychiatry II. Nosology. Oxford University Press. Parnas J, Sass LA, and Zahavi D (2008). Recent developments
Gross G, Huber G, Klosterktter J, et al. (1987). Bonner Skala in philosophy of psychopathology. Curr Opin Psychiatry
Fr die Beurteilung von Basissymptomen. 21, 6, 578-84.
Hempel CG (1965). Explanation and Other Essays in the Parnas J, Jacobsen B, Schulsinger H, Schulsinger F, Mednick
Philosophy of Science. Free Press. SA (1993). Lifetime DSM-III-R diagnostic outcomes in
Jakobsen KD, Frederiksen JN, Hansen T, et al. (2005). Reliability the offspring of schizophrenic mothers. Results from the
of clinical ICD-10 schizophrenia diagnoses. Nord J Copenhagen High-Risk Study. Arch Gen Psychiatry 50, 7.
Psychiatry 59, 3, 209-12. Sadler J Z, Wiggins OP, and Schwartz MA (1994) (eds).
Jakobsen KD, Hansen T, and Werge T (2007). Diagnostic stability Philosophical Perspectives on Psychiatric Diagnostic
among chronic patients with functional psychoses: an Classification. The John Hopkins University Press.
epidemiological and clinical study. BMC Psychiatry 7, Software (2001). Stata 7 Manual. Stata Corporation.
41. Spitzer RL (1983). Psychiatric Diagnosis: Are Clinicians Still
Kety SSRD, Wender PH, Schulsinger F, Jacobsen B (1975). Necessary? Comprehensive Psychiatry 24, 5.
Mental illness in the biological and adoptive families of Stanghellini G (2004). The Puzzel of the Psychiatric Interview.
adopted individuals who have become schizophrenic: A Journal of Phenomenological Psychology 35, 2.
preliminary report based on psychiatric interviews. Genetic Vollmer-Larsen A, Handest P, and Parnas J (2007). Reliability
Research in Psychiatry John Hopkins University Press. of measuring anomalous experience: the Bonn Scale for
Kety SSRD, Wender PH, Schulsinger F (1968). The types and the Assessment of Basic Symptoms. Psychopathology 40,
prevalence of mental illness in the biological and adoptive 5, 345-8.

128 Clinical Neuropsychiatry (2012) 9, 3