219
THE AI/RHEUM KNOWLEDGE-BASED COMPUTER
CONSULTANT SYSTEM IN RHEUMATOLOGY
Performance in the Diagnosis of 59 Connective Tissue
Disease Patients from Japan
JAMES F. PORTER, LAWRENCE C. KINGSLAND, 111, DONALD A. B. LINDBERG,
INDRAVADAN SHAH, JAMES M. BENGE, SUSAN E. HAZELWOOD, DONALD R. KAY,
MITSUO HOMMA, MASASHI AKIZUKI, MAKOTO TAKANO, and GORDON C. SHARP
AI/RHEUM is a knowledge-based computer con-
sultant system for the diagnosis of rheumatic diseases.
Its diagnostic accuracy was evaluated using information
that was supplied by Japanese rheumatologists on 59
patients with connective tissue diseases. The diagnoses
of the AURHEUM model were in full or partial agree-
ment with those of the Japanese rheumatologists in 54 of
59 cases (92%). Preliminary evaluation of the criteria
used by the model to diagnose mixed connective tissue
disease showed a sensitivity of 90% and a specificity of
96%.
AURHEUM is a computer consultant system
that uses artificial intelligence techniques to model the
consultative behavior of the expert rheumatologist in
From the University of Missouri-Columbia Multipurpose
Arthritis Center, and the Department of Internal Medicine, Keio
University, Tokyo, Japan.
Supported by NIH-NIAMS Multipurpose Arthritis Center
grant 2P60, AR-20658.
James F. Porter, MD: University of Missouri-Columbia
Multipurpose Arthritis Center; Lawrence C. Kingsland, 111, PhD:
University of Missouri-Columbia Multipurpose Arthritis Center (cur-
rent address: National Library of Medicine, Bethesda, MD); Donald
A. B. Lindberg, MD: University of Missouri-Columbia Multipurpose
Arthritis Center (current address: National Library of Medicine,
Bethesda, MD); lndravadan Shah, MD: University of Missouri-
Columbia Multipurpose Arthritis Center; James M. Benge, MD:
University of Missouri-Columbia Multipurpose Arthritis Center;
Susan E. Hazelwood, BS: University of Missouri-Colurnbia Multi-
purpose Arthritis Center; Donald R. Kay, MD: University of Mis-
souri-Columbia Multipurpose Arthritis Center; Mitsuo Homma, MD:
Keio University; Masashi Akizuki, MD: Keio University; Makoto
Takano, MD: Keio University; Gordon C. Sharp, MD: University of
Missouri-Columbia Multipurpose Arthritis Center.
Address reprint requests to Gordon C. Sharp, MD, Division
of Immunology and Rheumatology, MA427 Medical School Addi-
tion, I Hospital Drive, Columbia, MO 65212.
Submitted for publication March 3. 1986; accepted in re-
vised form July 9. 1987.
Arthritis and Rheumatism, Vol. 31, No. 2 (February 1988)
the diagnosis of rheumatic diseases (1,2). The system’s
current knowledge base includes diagnostic criteria
tables for the 26 rheumatic diseases shown in Table 1.
The criteria tables, which represent the knowledge
that is the basis of the AI/RHEUM reasoning proc-
ess, were developed at the University of Missouri-
Columbia and were reviewed externally by American
rheumatology consultants (3). The tables are struc-
tured as combinations of major and minor decision
elements, required elements, and exclusions which
lead the system to diagnostic conclusions categorized
as definite, probable, or possible.
The AURHEUM system is a consultative tool
for use by physicians who do not have specialty training
in rheumatology. To address the problem of potentially
inaccurate input that might mislead the system, AI/
RHEUM makes available on line more than 180 defini-
tions of items from the patient data checklist. Many of
these definitions oEer information in 4 categories:
WHAT (is the observation), WHY (is it being re-
quested), HOW (is the observation performed), and
REFS (specific citations from the literature).
The physician user enters information into the
system in terms of positive, negative, unknown, and
numeric findings. The AI/RHEUM reasoning process
leads the system to conclusions that are categorized as
definite, probable, or possible and are presented in a
differential diagnosis. For each component of the
differential, the system presents a statement of the
findings for the patient that support the conclusion, a
statement of currently unknown findings which, if
known and positive, would tend to strengthen the
conclusion, and a statement of findings true for this
patient that are not normally expected with this dis-
220 PORTERETAL

Table 1. List of diseases known to the AURHEUM diagnostic
model
Chronic inflammatory arthritis
Rheumatoid arthritis
Juvenile rheumatoid arthritis, pauciarticular onset
Juvenile rheumatoid arthritis, polyarticular onset
Juvenile rheumatoid arthritis, systemic onset
Systemic rheumatic diseases
Systemic lupus erythematosus
Scleroderma (progressive systemic sclerosis)
Primary Raynaud’shndifferentiated connective tissue disease
Pol y m yositis/dermatomyositis
Mixed connective tissue disease
Sjogren’s syndrome
Vasculitides
Giant cellitemporal arteritis
Polymyalgia rheumatica
Polyarteritis nodosa
Spond y larthropat hies
Ankylosing spondylitis
Psoriatic arthritis
Reiter’s syndrome
Enteropathic arthritis
Osteoarthritis
Crystal-induced arthritides
Gout
Calcium pyrophosphate deposition disease
Microbial-associated arthropathies
Bacterial, nongonococcal arthritis
Gonococcal arthritis
Tuberculous arthritis
Rheumatic fever
Nonarticular rheumatism
Fibromy algia
Carpal tunnel syndrome
ease. The statement of findings that are present and
are not components of the disease is an attempt by the
program to flag for the user conditions which should
lead him or her to consider the presence of other
disease processes.
The system can understand, store, and reason
from 877 observations on each case, Its reasoning
process uses whatever information is given, and can
reach correct diagnostic conclusions from a handful of
observations, if those observations are important indi-
cators. If the information offered is not sufficient to
trigger any diagnostic conclusions, the system will
indicate so. The data-entry phase of the interaction
takes 6-8 minutes as the user, beginning with a
filled-out patient data checklist, responds to menu-
oriented question frames. The actual reasoning time in
which the system’s logic is executed varies from 3 to 6
seconds on a VAX-111780 supermini computer, and
from 9 to 16 seconds on an ISM PC AT microcom-
puter.
AIiRHEUM was developed using system-
building software called EXPERT, which was pro-
duced at Rutgers University (New Brunswick. NJ).
EXPERT is a general-purpose system for building
artificial intelligence models in various domains (4).
The AURHEUM system has been evaluated
using more than 500 clinical cases in several series. It
was 94% correct, agreeing with a consensus diagnosis
of rheumatologist clinicians in 360 of 384 cases, when
tested at the University of Missouri-Columbia on a
series of cases selected retrospectively because they
had discharge diagnoses that were in the system (5).
The system was 85% correct, agreeing with
rheumatologist clinicians in 63 of 74 cases, in an
unselected series that consisted of all patients admit-
ted to the rheumatology service at the University of
Missouri-Columbia during 2 60-day periods. In this
series, every patient who had a disease that was listed
in the AI/RHEUM knowledge base (63 of 74) received
a correct diagnosis. Of the 1 1 patients with diseases
not in the system, diagnosis was correctly refused on
5. This brought the number of cases on which the
system had made an appropriate statement to 68 of 74
(92%). It was wrong on 6 patients (6,7). It should be
noted that although the system’s knowledge base
includes information on only 26 rheumatic diseases, its
coverage in this unselected series of cases was 85%: 63
of the 74 patients had diseases already defined in the
knowledge base.
We report on a third series of patients and
address issues of how the consistency with which
trained rheumatologists make observations half a
world apart will affect the performance of knowledge-
based consultant systems. The AVRHEUM model was
challenged with data from 59 connective tissue disease
cases sent by colleagues at Keio University in Tokyo,
Japan, an institution not involved in the previous model
development or testing. This report also provides a
preliminary evaluation of the sensitivity and specificity
of the proposed criteria for mixed connective tissue
disease (MCTD) used by the model.
METHODS
The reasoning of the AI/RHEUM system is embod-
ied in a series of more than 1,000 production rules of the
form “IF (premise) . . . THEN (conclusion).” The produc-
tion rules are derived directly from the diagnostic criteria
tables specified for each disease in the system’s knowledge
base. Criteria tables are formulated in terms of major and
minor decision elements, required (must have) elements, and
exclusionary (must not have) elements. Any of these deci-
sion elements may be individual findings from a patient data
checklist (e.g., clinical signs, symptoms, laboratory test
DIAGNOSIS OF MCTD BY AURHEUM 22 1

results, radiologic observations, tissue biopsy results), or Table 2. Diagnostic criteria for mixed connective tissue disease*
may be intermediate hypotheses.
Maior criteria Minor criteria
Intermediate hypotheses are derived from combina-
tions of findings, and usually represent specific pathophysio- 1. Severe myositis 1 . Alopecia
logic states. There are 467 such intermediate hypotheses in 2. Pulmonary involvement, 2. Leukopenia (<4,000
the knowledge base. In its reasoning process, AIiRHEUM with I or more of: WBC/mm3)
DLco 170% normal 3. Anemia
begins with patient findings, determines the status of inter- Pulmonary hypertension (510 gm/dl, women,
mediate hypotheses such as “serious renal involvement,” Proliferative vascular 512.0 gm/dl, men)
and combines intermediate hypotheses and other findings to lesions on lung biopsy 4. Pleuritis
reach conclusions regarding disease. 3. Raynaud’s phenomenon, 5. Pericarditis
The criteria form of knowledge representation devel- OR esophageal 6 . Arthritis
oped for AI/RHEUM is new to medical artificial intelligence. hypomotility 7. Trigeminal neuropathy
In the system building process, it allows the physician 4. Swollen hands observed, 8. Malar rash
subject-matter expert to define definite, probable, and pos- OR sclerodactyly 9. Thrombocytopenia
sible categories of disease; major and minor decision 5. Highest observed (<100,000/mm3)
anti-ENA rl:10,000, 10. Mild myositis
elements let the expert define very specific clinical presen- AND anti-RNP+ , 1 1 . History of swollen hands
tations for the knowledge base when appropriate and let him AND anti-Sm -
or her be more general when presentations are less well ~~~~~ ~ ~

characterized. The criteria are readily understood by con-
sultants, who can begin useful critiquing almost immediately
upon reading them.
In some circumstances, a critical diagnostic feature
may be constituted by a pattern of events, rather than by a
simple event. Examples include the various patterns of
arthritis. AURHEUM asks the user physician to answer if
the patient’s pattern of arthritis is “onset with maximum
inflammation within 24 hours,” “chronic with acute exacer-
bations,” etc. Thus, multiple choices with respect to the
arthritis distribution, duration, onset, course, and response
to therapy are presented; the physician who knows the
patient selects the correct response. In this way, he or she
codes a conclusion that is relatively easy for a human being
and relatively difficult for a computer system. Other inves-
tigators are interested in constructing computer programs
that can draw such distinctions when presented with com-
puter records of multiple visits by a patient, over time. AI/
RHEUM circumvents this serious problem.
Though we examined the published literature for
other criteria as we added new diseases to the knowledge
base (8,9), in each case we found it necessary to derive our
own. One reason for this was that published criteria often
focus on patients with advanced disease in order to assure
consistency in retrospective or collaborative studies. To
make AURHEUM effective in a general clinical setting, we
needed criteria appropriate to early and intermediate stages
of disease, as well as to full-blown presentations. The
criteria we postulate for mixed connective tissue disease
(MCTD) are presented in Table 2.
In the test series reported here, 59 patients under
continuing treatment at Keio University were studied (Table
3). Medical records were reviewed by the 3 Japanese rheu-
matologists (MH, MA, MT). Patients were included if they
met the following 3 criteria: 1) serum was definitely positive
for antinuclear antibodies (ANA) by indirect immunofluoresc-
ence; 2) sufficient clinical and laboratory findings were pre-
sent to satisfy criteria for 1 of the following systemic rheu-
matic diseases: systemic lupus erythematosus (SLE) (lo),
progressive systemic sclerosis (PSS; scleroderma) (1 I),
polymyositis (PM) (12), rheumatoid arthritis RA (13), MCTD
(14), Sjogren’s syndrome (SS) (15), or overlap syndrome
(criteria fulfilled for 2 or more of these diseases); and 3) a
* DLco = carbon monoxide diffusing capacity; ENA = extractable
nuclear antigen; WBC = white blood cells. For a diagnosis to be
considered definite, the patient had to have 4 major criteria with
serologic requirements of anti-ENA 2 1:4,000, positive anti-RNP,
and negative anti-Sm. For a diagnosis to be considered probable, the
patient had to have present either 3 major criteria or 2 major criteria
(including 1 or more criteria from 1 , 2 , or 3) and 2 minor criteria and
serologic requirements of anti-ENA 2 1 :1,000 and positive anti-
RNP. For a diagnosis to be considered possible, the patient had to
have 3 major criteria and no serologic requirements, or 2 major
criteria, or I major criterion and 3 minor criteria with anti-ENA
~1:lOOand positive anti-RNP.
sufficiently extensive database was collected to definitely
support 1 or more diagnoses and to exclude other differential
diagnoses.
These selection requirements explain the relative
paucity of cases of RA and the absence of cases of polyar-
teritis and early undifferentiated connective tissue disease in
this study. The large number of complicated cases with
multiple diagnoses or overlap syndromes reflects the special
interest of the Keio University group, and posed a special
challenge for the AI/RHEUM system. The patient informa-
tion and the clinicians’ diagnoses were obtained from
Japanese rheumatologists who completed the AI/RHEUM
patient data checklist. The patient information was entered
into an AIIRHEUM model running on a DEC system-20
mainframe computer, and the AI/RHEUM diagnostic state-
ment was obtained.
The results of this testing were tabulated as follows:
For cases assigned a single diagnosis by the Japanese
clinicians, the AI/RHEUM program was considered correct
if it listed this diagnosis at the highest level in its differential.
AI/RHEUM was considered partially correct if it listed this
diagnosis in its differential, but listed another diagnosis at a
higher level (e.g., probable rather than possible). The model
was judged incorrect when it failed to include the diagnosis
assigned by the Japanese rheumatologists in its differential
diagnosis. The model was also judged incorrect when a
diagnosis the physicians had not mentioned was listed as
definite.
In the diagnosis of overlap syndromes, the situation
was slightly different. When the Japanese rheumatologists
222 PORTER ET AL

Table 3. Diagnoses by Japanese physicians and AI/RHEUM in 59 Japanese patients with connective tissue disease*
Pa- Pa- Pa-
tient Clinicians’ tient Clinicians’ tient Clinicians’ AImHEUM
no. diagnosis AIiRHEUM diagnosis no. diagnosis AIfliHEUM diagnosis no. diagnosis diagnosis
I t Overlap Def PSS, prob MCTD, 21 PSS, ss Def ss,poss PSS, 4of SLE Def SS, poss SLE
(SLE, PSS) poss SLE poss RA, poss SLE 41 PM Poss PM, poss
2 PSS, ss Def SS, poss PSS, 22$ PSS Def PSS, def SS, poss SLE
poss MCTD PM, poss SLE 42 SLE Def SLE, poss
3 t SLE Prob MCTD, poss 25 PSS Poss PSS, poss MCTD MCTD
SLE, poss PSS 24 MCTD, SS Def MCTD, def SS, 43 PM Prob PM, poss
4t Overlap Def MCTD, prob PSS, poss PSS, poss SLE SLE
(SLE, PSS, poss SLE 25t Overlap (PSS, Def PSS, prob SS, 44 SLE Prob SLE, poss
PM, MCTD) PM) poss PM, poss SLE, PAN
5 PM Def PM poss MCTD 45 SLE Poss SLE
6 SLE Prob SLE 26t MCTD, PSS, Def MCTD, prob PSS, 46t MCTD,SLE, Prob MCTD, poss
7 MCTD Prob MCTD, poss PM poss SLE PSS SLE
SLE, poss PSS 27t MCTD, PSS, Def MCTD, def SS, 47 SLE, SS Def SS, poss SLE
8 PM Def PM PM, SS poss PSS, poss SLE 48$ SLE Def SS, poss SLE
9 PSS Poss PSS, poss 28 PSS Def PSS, prob SS, 49 SLE Prob SLE
MCTD, poss SLE poss SLE 50 PSS Def PSS
10 PSS Def PSS, prob SLE, 29 Overlap (PSS, Def PSS, def SS, poss 51 SLE Prob SLE
poss ss RA, SS) MCTD, poss PM, 52t PSS Prob SS, poss PSS,
I1 PM, SS Def PM, poss SS poss RA, poss SLE poss PM, poss
12 PM Prob PM 30 MCTD Prob MCTD, prob MCTD
13 PSS Def PSS PSS, poss SLE 53 SLE Poss SLE
14 MCTD, PSS, Def SS, prob MCTD, 31f MCTD Def PSS, prob SLE, 54 RA, ss Def ss,poss RA,
ss prob PSS, poss SLE poss RA, poss PM poss PSS, poss
15 PSS Def PSS, prob SLE, 32 PSS, ss Def PSS, poss ss MCTD
prob RA, poss PM, 33 PSS Def PSS, poss SLE 55 SLE Prob SLE
poss PAN 34 SLE Prob SLE, poss SS 56t PSS Prob SLE, poss
16 PSS Poss PSS, poss 35 SLE Poss SLE, poss SS PSS
MCTD, poss SLE 36$ MCI-D Def MCTD, def SS, 57 PSS Def PSS, poss RA,
17 Overlap Def SLE, def PSS, prob PSS, poss SLE poss SLE, poss
(SLE, PSS) poss PM, poss PAN 37 SLE Poss SLE, poss RA MCTD
18 SLE Prob SLE, poss PAN 38 PM Poss PM, poss PSS 58 PSS Poss PSS
19 PSS Def PSS, poss ss 39 SLE Prob SLE 59 SLE Prob SLE
20 PSS Def PSS, poss SLE
* SLE = systemic lupus erythematosus; PSS = progressive systemic sclerosis; def = definite; prob = probable; MCTD = mixed connective
tissue disease; poss = possible; SS = Sjogren’s syndrome; RA = rheumatoid arthritis; PM = polymyositis; PAN = polyarteritis nodosa.
t Patients whose diagnoses by AURHEUM were partially correct. See Results for explanation.
f Patients whose diagnoses by AI/RHEUM were incorrect. See Results for explanation.

listed multiple connective tissue disease diagnoses for a single
patient, the computer model was judged as correct only when
it listed each of these diagnoses in its differential at a higher
level of probability than any diagnosis not listed by the
physicians.
For example, if a patient was diagnosed by the
Japanese clinicians as having an overlap syndrome with
features of SLE, RA, and PM, the computer model’s re-
sponse was scored correct if it listed a sequence of diagnoses
such as definite SLE, probable RA, possible PM, possible
SS. However, if AURHEUM listed definite SLE, probable
RA, probable SS, and possible PM, it would no longer be
considered fully correct because it had listed the diagnosis of
SS (not mentioned by the Japanese clinicians) at a level
higher than PM. In this situation, the model was considered
partially correct.
The computer model was also considered partially
correct when, for the 3- and 4-diagnosis overlap syndrome
cases, it listed in its differential diagnosis all but 1 of the
diagnoses assigned by the clinicians.
CASE REPORT
The patient was a 47-year-old man (patient 27,
Table 3) whose onset of symptoms occurred at age 38.
Over the subsequent 9 years, the patient’s clinical
findings included Raynaud’s phenomenon, tempera-
tures as high as 40°C, hypertension (blood pressure
>140/90 mm Hg basal), sclerodactyly , telangiectasias,
ecchymoses, keratoconjunctivitis sicca, xerostomia,
parotid enlargement, symmetric polyarticular arthritis,
mild proximal muscle weakness, lymphadenopathy,
pleuritis, evidence of pulmonary interstitial disease,
and hepatornegaly. Diffuse sclerosis, digital ulcers,
calcinosis cutis, alopecia, photosensitivity, rash, sub-
cutaneous nodules, pericarditis, splenomegaly, and
neurologic abnormalities were not present.
Laboratory findings were as follows: hemoglo-
DIAGNOSIS OF MCTD BY AI/RHEUM
bin 11.6 gm/dl, white blood cell count 4,900/mm3,
eosinophil count 45/mm3, platelet count 170,000/
mm3, erythrocyte sedimentation rate (Westergren) 82
mm/hour, normal urinalysis results, blood urea nitro-
gen 16.9 mg/dl, serum creatinine 0.91 mg/dl, creati-
nine clearance 96 ml/minute, serum uric acid 7.1
mg/dl (normal 5 5 mg/dl), serum gamma globulin 4.3
gm/dl, creatine kinase 42 (normal 512), serum glu-
tamic oxaloacetic transaminase 57 (normal 540). Car-
bon monoxide diffusing capacity (DLco) was 9.4% of
normal. Electrocardiogram showed a prolonged PR
interval. There was esophageal hypomotility. Elec-
tromygram showed myopathy, and muscle biopsy
showed mild myositis. Lip-biopsy showed SS, and
there was a positive result on the Schirmer’s test.
Serologic results included positive rheumatoid factor
(RF) at a titer of 1:640,C3 50 mg/dl (normal r70),
positive fluorescent ANA with a speckled pattern and
a titer of 1: 1,024, anti-extractable nuclear antigen
(ENA) antibody (hemagglutination method) positive at
a titer of 1 :20,480,anti-DNA antibody (hemagglutina-
tion method) positive at a titer of 1:14.By immunodif-
fusion, RNP, SS-A, and SS-B antibodies were all
positive and Sm, PM-I, and Scl-70 antibodies were all
negative.
This patient had 4 diagnoses given by the Jap-
anese clinicians: MCTD, PSS, PM, and SS. The
differential diagnosis given this case by AI/RHEUM
was definite MCTD, definite SS, possible PSS, and
possible SLE. The diagnosis of PM would also have
been triggered, but was excluded by the presence of
the anti-RNP antibody in conjunction with an ENA
titer > 1 :1,000.We scored AI/RHEUM partially cor-
rect, since the system indicated 3 of the 4 clinician’s
diagnoses.
The AURHEUM system explained its reason-
ing by stating that the diagnosis of definite MCTD was
concluded on the basis of the findings of Raynaud’s
phenomenon, sclerodactyly , pleuritis, esophageal hy-
pomotility, DLco <70% of normal, anemia (an inter-
mediate hypothesis), anti-RNP antibody present with
an ENA titer > I : 10,000and anti-Sm negative (another
intermediate hypothesis).
The AURHEUM diagnosis of definite SS was
supported by the findings of keratoconjunctivitis sicca,
xerostomia, parotid enlargement, a positive Schir-
mer’s test result, a lip biopsy specimen positive for SS,
positive RF, and positive fluorescent ANA at a titer
> 1 :40 (an intermediate hypothesis).
The system’s diagnosis of possible PSS was
triggered by the findings of sclerodactyly, pulmonary
223
interstitial disease or DLco <70% of normal (an
intermediate hypothesis about pulmonary involve-
ment; actually, both findings were present), telangiec-
tasias, and Raynaud’s phenomenon, esophageal hypo-
motility, or digital ulcers (another intermediate
hypothesis; the first 2 findings were present).
The final diagnosis in the AI/RHEUM differen-
tial was possible SLE. The system stated that this
diagnosis was supported by the findings of serositis (an
intermediate hypothesis), temperature >38”C (an in-
termediate hypothesis), hypergammaglobulinemia (an
intermediate hypothesis triggered at levels > 1.8
gm/dl), hypocomplementemia (an intermediate hy-
pothesis triggered by comparison with normal ranges),
the presence of Raynaud’s phenomenon, positive flu-
orescent ANA at a titer >1:40,and positive anti-DNA
antibody.
Other portions of the AI/RHEUM output state-
ment list, for each disease in the differential diagnosis,
the findings present which are not explained by the
disease, and the findings not yet known, which, if
known and positive, would strengthen the conclusion.
RESULTS
The cases presented by the Japanese clinicians
at Keio University were truly complex (Table 3). Most
frequent single diagnoses were of SLE (17 patients)
and PSS (16 patients). Seven patients had 2 diagnoses,
and 5 patients were labeled as having overlap syn-
drome, with features of 2-4 connective tissue diseases
specified. Four additional cases were not specified as
overlap syndrome, but carried 3 and 4 separate diag-
noses for an individual patient.
The AI/RHEUM computer consultant system
presented a differential diagnosis for each of the 59
cases in the series, listing from 1 to 6 connective tissue
diseases. Each diagnosis in the differential was cate-
gorized as definite, probable, or possible.
The model was in complete agreement with the
Japanese rheumatologists in 45 of 59 cases (76%), and
was judged partially correct in 9 additional cases.
These 9 partially correct cases included 5 in which
AURHEUM listed the correct diagnosis in its differ-
ential but listed another diagnosis at a higher confi-
dence level. Two of these 5 cases involved the diag-
nosis of SS. In the remaining 3 partially correct cases,
AURHEUM included in its differential all but 1 of the
3 or 4 diagnoses listed by the Japanese clinicians in
overlap syndromes. For these 3 cases, the missing
diagnosis was PM. In each case, the features of the
224
patient’s clinical disease could be adequately ex-
plained by the alternative diagnoses AI/RHEUM did
make. For example, in a patient with MCTD with an
inflammatory myositis, MCTD adequately explains
the myositis; therefore, PM would not be separately
diagnosed by AURHEUM. The computer model was
considered to have made a correct or partially correct
diagnosis in 54 of 59 cases (92%).
F~~~of the 5 judged incorrect occurred
when AI/RHEUM diagnosed definite SS, but the Jap-
anese clinicians did not mention this diagnosis. In each
of these cases, the findings of keratoconjunctivitis
sicca, a positive result on Schirmer’s test, and xero-
stomia listed on the patient data checklist would seem
to offer justification for the diagnosis. In the fifth
incorrect case AI/RHEUM missed a diagnosis of
MCTD, in part because although anti-ENA antibody
was present, the titration value was unknown.
This study also provided an opportunity for a
preliminary evaluation of the sensitivity and specificity
of the criteria used for MCTD (Table 4). The Japanese
clinicians diagnosed 10 patients as having MCTD,
including those with multiple diagnoses and overlap
syndromes. The computer model correctly identified 9
of these patients by listing the diagnosis of MCTD at
the highest level in its differential. For example,
MCTD might be listed at the probable level, with no
other diagnosis listed as definite. In the last of these
cases, the model incorrectly listed SS as definite and
MCTD as probable.
AI/RHEUM correctly diagnosed as not having
MCTD 47 of the 49 cases the Japanese rheumatologists
diagnosed as not having MCTD. In only 2 of these 49
cases was MCTD diagnosed at a probability higher
than the correct diagnosis. Both cases were listed as
probable MCTD by the model, whereas the correct
diagnosis of SLE was listed only at the possible level.
DISCUSSION
The opportunity to test this model with cases
from Keio University was appealing because the Jap-
anese patient data collection and diagnostic processes
would be totally independent of those which formed
the background of AURHEUM. At the same time, we
recognized that expert rheumatologists abroad would
probably make highly reliable patient observations.
Thus, the testing of the model would be fair and
independent, but not so severe as the ultimate (future)
test with nonrheumatologist physicians.
A special difficulty in testing the model in the
PORTERETAL
Table 4. Sensitivitykpecificity evaluation of the proposed criteria
for mixed connective tissue disease (MCTD) used in AIIRHEUM”
Diagnosis by AI/RHEUM
computer consultant program
Diagnosis by
Japanese clinicians MCTD Other than MCTD
MCTD 9 1
Other than MCTD 2 47
* These data result in 90% sensitivity and 96% specificity.
context of Japanese rheumatology practice was the
use of the diagnostic term overlap syndrome. The term
in Japan refers to the simultaneous occurrence of
findings associated with multiple connective tissue
diseases, usually rather strongly manifested. It should
be noted that AURHEUM has the ability to make
multiple diagnoses in the same patient when findings
are present that trigger multiple conclusions.
This capability, while obviously a traditional
part of the clinical approach, is not at all a common
part of most computer models of diagnosis. Computer
models that use the Bayesian approach typically rank-
order individual diagnoses (16). Models based upon
“branching tree” or flow chart logic are inherently
incapable of dealing explicitly with the overwhelming
combinations of the many individual findings that map
to multiple simultaneous diagnoses in the complex
clinical case. Even in the case of INTERNIST-1
(17,18), arguably the most extensive and substantial of
computer diagnostic models, the system would likely
not be well suited to dealing with overlap syndromes.
INTERNIST-1 is limited by the use of a formalism
which allocates each finding to a single diagnosis. This
effectively excludes the capability of dealing with
highly interrelated disease diagnoses such as we see in
rheumatic conditions like the overlap syndrome as it is
defined in Japan. AURHEUM specifically is capable of
dealing with an overlap syndrome as a combination of
individual diseases.
Thus, AI/RHEUM has proven successful in our
own academic setting (1,2,6,7) and in association with
specialists elsewhere. Problems in diagnosing these 59
connective tissue disease cases centered on the sys-
tem’s not listing diagnoses mentioned by the Japanese
clinicians or on the system’s identifying a diagnosis as
definite, when it was not listed in the clinician’s
differential.
In all 4 of the cases in which a diagnosis was not
listed by the model, the patient had an overlap syn-
drome; AI/RHEUM justifiably incorporated the pa-
DIAGNOSIS OF MCTD BY AI/RHEUM
tients’ clinical findings into its conclusion of another
appropriate connective tissue disease. An example of
this situation would be the incorporation of clinical
findings of P M into a diagnosis of MCTD.
In 4 of the 5 cases in which the computer model
identified as definite a diagnosis not listed by the
Japanese clinicians, the disease in dispute was SS.
Review of the patient data checklists for these cases
showed strong evidence for the presence of SS; it is
uncertain why the Japanese physicians did not diag-
nose SS in these cases. Possibly, their criteria for
diagnosis of definite SS (strongly positive staining with
rose bengal dye on biomicroscopy , histopathologic
confirmation of destructive lymphocytic infiltration, or
abnormal findings on sialography [ 151) were more
demanding than those of AI/RHEUM. In general,
however, there appears to be a high level of agreement
between the Japanese and American physicians in
their conceptualization of rheumatic disease condi-
tions, which may be because of their use of the same
(American Rheumatism Association) o r similar criteria
for systemic rheumatic diseases.
The AIiRHEUM knowledge base incorporates
criteria tables for each of the 26 diseases it is capable
of diagnosing. Preliminary results based on the 59
patients in this study suggest that the criteria for
MCTD shown in Table 2 are helpful in diagnosing this
syndrome. The model correctly identified 90% of the
patients diagnosed by the Japanese clinicians as hav-
ing MCTD. The model correctly did not diagnose
MCTD in 47 of the 49 patients the Japanese clinicians
diagnosed as not having MCTD. For both of the cases
in which it incorrectly diagnosed MCTD, AI/RHEUM
listed MCTD at the probable level and listed the
correct diagnosis of SLE at the possible level. It is
anticipated that testing of these preliminary criteria by
other groups may provide additional new insights
regarding the relationship between MCTD and other
connective tissue diseases.
Our plans for future investigations are influ-
enced by these encouraging findings. One limitation to
testing in a typical clinical environment is the fact that
the current AURHEUM system deals only with the
diagnostic process. Knowledge concerning the man-
agement of patients with rheumatic diseases will add
an important component in making AURHEUM a
useful tool for the primary care physician. This man-
agement phase is currently under development.
Finally, field testing by nonrheumatologist phy-
sicians will be required before AI/RHEUM can be
considered appropriate for general use. This validation
225
and evaluation process may take 2-3 years, after
which we anticipate the system’s release in a config-
uration for the IBM PC A T microcomputer. We have
criteria tables for 6 additional diseases in draft form,
which when tested and incorporated into the knowl-
edge base, will bring the system’s coverage to 32
diseases. Since a system such as AURHEUM can be
only as accurate and reliable as the observations and
laboratory test results on which it bases its diagnostic
conclusions and therapy recommendations, we are
now developing a n adjunct program called AI/
LEARN, which is intended to help improve the accur-
acy of clinical observations of rheumatic disease made
by physicians (19).
ACKNOWLEDGMENT
We thank Shem Potts for expert secretarial assistance.
REFERENCES
1. Lindberg DAB, Sharp GC, Kingsland LC Ill, Weiss
SM, Hayes SD, Ueno H, Hazelwood SE: Computer
Based Rheumatology Consultant, Proceedings of
MEDINFO 1980. Edited by DAB Lindberg, S Kaihara.
Amsterdam, North-Holland, 1980, pp 131 1-1315
2. Kingsland LC 111, Lindberg DAB: Research methods in
A1 model building: the history of a project, Proceedings
of the American Association of Medical Systems and
Informatics Congress 1983. Edited by DAB Lindberg, E
van Brunt, MA Jenkin. Bethesda, AAMSI, 1983, pp
76-80
3. Lindberg DAB, Kingsland LC 111, Waugh W, Benge JM,
Sharp GC: Criteria tables as a possibly general knowl-
edge representation, Proceedings of the American As-
sociation of Medical Systems and Informatics Congress
1984. Edited by DAB Lindberg, MF Collen. Bethesda,
AAMSI, 1 9 8 4 , : ~187-192
~
4. Weiss SM, Kulikowski CA: EXPERT: a system for
developing consultation models, Proceedings of the
Sixth International Joint Conference on Artificial Intel-
ligence. Tokyo, 1979, pp 942-947
5. Kingsland LC 111, Lindberg DAB, Sharp GC: AI/
RHEUM: a consultant system for rheumatology. J Med
Syst 7~221-227, 1983
6. Benge JM, Sharp GC, Kay DR, Capps RJ, Hazelwood
SE, Kingsland LC 111, Reese GR, Lindberg DAB:
Evaluation of an A1 consultant in rheumatology, Pro-
ceedings of the American Association of Medical Sys-
tems and Informatics Congress 1983. Edited by DAB
Lindberg, EE van Brunt, MA Jenkin. Bethesda.
AAMSI, 1983, pp 83-86
7. Kingsland LC 111, Sharp GC, Capps RJ, Benge JM, Kay
DR, Reese GR, Hazelwood SE, Lindberg DAB: Testing
of a criteria-based consultant system in rheumatology,
Proceedings of MEDINFO 1983. Edited by JH van
226
Bemmel, MJ Ball. 0 Wigertz. Amsterdam, North-Hol-
land, 1983, pp 514-517
8. Cohen AS, Reynolds WE, Franklin EC, Kulka JP,
Ropes MW, Shulman LE, Wallace SL: Preliminary
criteria for the classification of systemic lupus erythe-
matosus. Bull Rheum Dis 21:643-648. 1971
9. Rodnan GP, Schumacher HR, Zvaifler NJ, editors:
Primer on the Rheumatic Diseases. Eighth edition. At-
lanta, Arthritis Foundation, 1983, pp 207-208
10. Tan EM, Cohen AS, Fries JF, Masi AT, McShane DJ,
Rothfield NF, Schaller JG, Tala1 N , Winchester RJ: The
1982 revised criteria for the classification of systemic
lupus erythematosus. Arthritis Rheum 25: 1271-1277,
1982
I 1. Subcommittee for Scleroderma Criteria of the American
Rheumatism Association Diagnostic and Therapeutic
Criteria Committee: Preliminary criteria for the classifi-
cation of systemic sclerosis (scleroderma). Arthritis
Rheum 23581-590, I980
12. Bohan A, Peter JB, Bowman RS, Pearson CM: A
computer-assisted analysis of I53 patients with poly-
myositis and dermatomyositis. Medicine (Baltimore)
56:255-283, 1977
13. Ropes MW, Bennett GA, Cobb S, Jacox R , Jessar RA:
1958 revision of diagnostic criteria for rheumatoid arthri-
tis. Bull Rheum Dis 9:175-176. 1958
14. Kasukawa R , Tojo T, Miyawaki S, Yoshida H, Tani-
PORTERETAL
moto K, Nobunaga M, Suzuki T, Takasaki Y . Tamura
T: Preliminary diagnostic criteria for classification of
mixed connective tissue disease, Mixed Connective
Tissue Disease and Anti-Nuclear Antibodies. Edited by
R Kasukawa, GC Sharp. Amsterdam, Elsevier, 1987, pp
41-47
15. Diagnostic criteria of the Japanese Ministry of Health
and Welfare for Sjogren’s syndrome. Annual Report,
Japanese Intractable Diseases Research Foundation,
1978
16. Lusted LB: Some studies of medical diagnoses using
Bayes’ theorem and computers, Introduction to Medical
Decision Making. Springfield, IL, Charles C Thomas,
1968, pp 24-69
17. Miller RA, Pople HE, Myers JD: INTERNIST-I, an
experimental computer-based diagnostic consultant for
general internal medicine. N Engl J Med 307:468-476,
I982
18. Miller RA: INTERNIST-I/CADUCEUS: problems fac-
ing expert consultant programs. Methods Inf Med 23:9-
14, 1984
19. Kochtanek T, Kay DR, TenBrink T: AI/LEARN: an
education innovation in rheumatology, Proceedings of
the American Association of Medical Systems and In-
formatics Congress 1985. Edited by AH Levy, BT
Williams. Bethesda, AAMSI, 1985, pp 395-398