Professional Documents
Culture Documents
1093/rheumatology/keh605
Advance Access publication 29 March 2005
Objectives. Systemic lupus erythematosus (SLE), a chronic illness with an unpredictable and variable course, profoundly
affects the quality of life (QOL). General health questionnaires are used to assess QOL in SLE, but a disease-specific
instrument could offer enhanced responsiveness and content validity. We detail the steps we took to develop and validate a new
Systemic lupus erythematosus (SLE), a chronic autoimmune The QOL domains of importance to lupus patients include
illness that usually begins in early adulthood, affects multiple health, work and income, identity and independence, and social
organ systems and may be associated with considerable and family life, which can be measured with a variety of general
morbidity and mortality. In addition, SLE is incurable and health instruments [4–7]. Qualitative research has yielded as
runs a variable course over the patient’s remaining years. The many as 12 concepts: uncertainty/unpredictability of lupus,
response and complication of drug treatment are generally fatigue, pain/symptoms, social support, misunderstood by others,
unpredictable. These characteristics of the disease impact the fear, dependence/feelings of inadequacy/loss of self, limitations/
quality of life (QOL) in unique ways, thus measuring the restricted activities, personal self-management, medical treatment,
objective outcomes of morbidity and mortality only does not emotional stress and financial issues [8]. The unpredictability of
fully reflect the burden of disease borne by the sufferers [1]. The SLE in terms of the course of disease, extent of organ involve-
assessment of QOL has become an important facet of manage- ment and response to treatment is a significant component of
ment of chronic diseases such as this and is considered to the patient’s QOL that is not well represented by general health
be very relevant for clinical practice, interventional trials and instruments even though it may be indirectly assessed with a
outcome monitoring [2, 3]. helplessness index [9, 10].
General health instruments have been shown to be valid for There are recognized advantages to using a disease-specific QOL
measuring QOL in SLE patients. They possess construct validity instrument rather than a general health one [11]. Disease-specific
and responsiveness and they allow the comparison of QOL instruments respond to smaller changes in the QOL (improved
between different rheumatic diseases [4]. Criticisms levelled at responsiveness). One benefit of this is that smaller number of
their use include the argument that they are not sufficiently patients will be needed in clinical trials. Disease-specific instru-
comprehensive for SLE, especially those that cater to the physical ments also cover a wider range of QOL (superior face and
and neglect the mental dimension [5]. content validity) issues arising from that illness.
Department of Rheumatology, Allergy and Immunology, Tan Tock Seng Hospital, Singapore and 1Health Promotion Board, Singapore.
As a part of a wide-ranging SLE study in our institution, we representing better self-perceived QOL. The physical health
recently developed a SLE-specific QOL scale (SLEQOL) using domains are physical functioning, role physical, bodily pain and
an established methodology [12, 13]. In this paper, we describe general health. The mental health domains are vitality, social
the steps we took to develop and validate the SLEQOL in our functioning, role emotional and mental health. In this paper,
patient population. We also compared our work with an we did not further collapse the scores into the physical and mental
independent QOL instrument for SLE that has been published component summaries. SLICC/ACR-DI tracks irreversible organ
recently [14]. dysfunction across 12 systems. The dysfunction must be present
for 6 consecutive months in order to register as damage. The
organs systems of interest are ocular (0–2 points), neuropsychi-
Patients and method atric (0–6 points), renal (0–3 points), pulmonary (0–5 points),
cardiovascular (0–6 points), peripheral vascular (0–5 points),
Patients gastrointestinal (0–6 points), musculoskeletal (0–6 points), derma-
All the patients fulfilled the 1982 American Rheumatism tological (0–3 points), gonadal (0–1 point), diabetes (0–1 point)
Association (ARA) classification criteria for SLE or the 1997 and malignancy (0–2 points). The score ranges from 0 to 46,
American College of Rheumatology (ACR) revision [15, 16]. They higher scores signifying more damage.
were recruited from our out-patient Rheumatology Clinic and
from the hospital wards in a single institution. The study was
approved by the Hospital’s Ethics Committee. The patients had Statistical analysis
to be fluent in written and spoken English. Informed consent was Factor structure was studied by factor extraction with principal
obtained according to the Declaration of Helsinki.
Results
Data collection Item generation
Data were captured systematically during every study visit. In May 2002, we asked a group of rheumatologists and nurse
In addition to clinical data and SLEQOL, the SLE Disease clinicians to nominate items that they believe were relevant to SLE
Activity Index (SLEDAI) [17], Systemic Lupus Activity Measure patients. By 13 June 2002, 51 items had been identified (Table 1).
(SLAM) [18], Rheumatology Attitudes Index (RAI) [9, 10, 19], As a gauge of their comprehensiveness, we noted that these items
MOS 36-Item Short-Form Health Survey (SF-36) [20–22] and cover Wiginton’s 12 concepts very well [8]. A draft questionnaire
the Systemic Lupus International Collaborating Clinic/American consisting of these items was administered to 100 patients in
College of Rheumatology damage index (SLICC/ACR-DI) [23, 24] October 2002.
were recorded. Patients with disease duration shorter than 3 yr This group of 100 SLE patients comprises 11 males and
were interviewed every 3 months. 89 females. There were 84 Chinese, four Indians, 11 Malays
The SLEDAI is a 24-item instrument for assessing SLE activ- and one whose ethnicity was not specified. The mean age was
ity in nine organ systems. Clinical and laboratory data are 39.4 13.7 yr. Patient input was sought by encouraging these 100
required to complete the questionnaire. The score ranges from patients to suggest activities in their lives that had been affected
0 to 105 points, with higher values signifying disease activity. by their illness that had been omitted in the questionnaire. We
SLAM assesses SLE activity from data on nine organ systems also asked patients to score the importance and the frequency
and seven laboratory values. The RAI, known as the Arthritis of occurrence of every item using a seven-point Likert scale.
Helplessness Index before 1988, is a 15-item scale that is internally A seven-point scale was chosen over a five-point one because
consistent and correlates significantly with the independent indices this allowed a finer assessment of his or her QOL and provided a
of control, self-esteem, anxiety and depression [19]. The items better approximation to continuous data required for factor
resolve into two factors, one depicting the subjects’ belief in their analysis.
own ability to control their arthritis and the other, consisting
of five items, the concept of helplessness. Initially designed for
patients with rheumatoid arthritis, the RAI has been validated for
Item reduction and creation of SLEQOL
use in lupus patients [9, 10]. The SF-36 is a 36-item QOL
questionnaire. The items are summarized into eight domain scores, We analysed the input from the 100 patients to produce the
each derived from a summation and transformation of items final questionnaire. The patients did not add any items to the
scores. Each domain score ranges from 0 to 100, higher values list. The data on the importance of the items were subjected
Developing an SLE quality-of-life instrument 1269
TABLE 1. Rasch analysis of 51 initial items based on data provided by 100 SLE patients
The items are ranked by the degree of difficulty, with the easiest at the top of the table.
to Rasch model analysis and factor analysis, followed by expert (model underfit). Zstd values are t-tests of the observed data
review, for item elimination. against model-predicted values. Zstd values below zero indicate
In Rasch model analysis, the probability of the respondent that the fit is too predictable and values above zero indicate lack
successfully performing a certain task is a function of the ability of predictability. If the INFIT or OUTFIT values are between
of the person and the difficulty of the task. It places data (both 0.6 and 1.4, Zstd can usually be ignored [32].
person ability and item difficulty) on some reasonable hierarchy The most difficult item for our cohort of lupus patients is
on a single continuum of interest [31]. In this case, Rasch model concern about being a financial burden to her or her family,
analysis was used to locate the level of difficulty of each of the followed by worry and concern that medicines do not work. The
51 items on a linear logit scale. The item of mean difficulty items with which our patients experience the least difficulty are
is assigned the value of zero. Negative measures denote that the social activities, getting in and out of bed and walking outdoors
items are more difficult. The INFIT and OUTFIT values (reported (Table 1).
as mean squares) indicate the closeness of fit of the data to the The items resolved into 11 components on factor analysis. The
stochastic Rasch model. Values less than 1.0 indicate observations first two components account for almost half of the variance
are too predictable while those above 1.0 indicate unpredictability and all the items for about three-quarters (Table 2). Using the
1270 K. P. Leong et al.
TABLE 2. Factor analysis of the 51 initial items based on data provided by 100 SLE patients (values below 0.4 are suppressed)
Item/factor 1 2 3 4 5 6 7 8 9 10 11
Fatigue 0.40
Joint pain and swelling 0.67
Missed work or school 0.78
Interference with career or education 0.75
Difficult relationship 0.60
Difficult social activities 0.71
More alcohol or tobacco 0.73
Embarrassment 0.43 0.46 0.47
Self-consciousness 0.55
Feeling low 0.67
Lack of a good night’s sleep 0.46
Worry 0.78
Cannot go out under the sun 0.60
Wishing others did not know that I have SLE 0.74
Made fun of 0.41 0.45
Loss of income 0.47 0.45
Itchy skin 0.64
Sore, painful or stinging skin 0.69
The factors were extracted by principal components analysis and varimax-rotated with Kaiser normalization.
results from these two analyses, we shortened the questionnaire A new questionnaire with 40 questions was devised in
according to a few principles, in the following order of importance. November 2002 (Appendix 1). The Flesch–Kincaid Grade
First, we ensured that there was a good distribution of questions Reading Level was 6.0 and Flesch Reading Ease was 70.2. The
across the different factors and difficulty levels. Second, items questions were grouped into six convenient subsections that
that have similar difficulty level and factor resolution were candi- were guided by the outcome of factor analysis. Later, we shall
dates for elimination. Third, items with INFIT or OUTFIT values show that most of the subsections are internally correlated and
below 0.6 or above 1.4 were also considered for removal [32]. correspond to QOL domains of relevance in SLE.
Accordingly, the 11 items which were eliminated were: ‘lack
of a good night’s sleep’, ‘worry’, ‘slow in thinking’, fear of painful
procedures’, ‘inability to go for long holidays’, ‘difficulty with Preliminary validation of the SLEQOL
dressing’, ‘difficulty with getting in and out of bed’, ‘difficulty with We proceeded to study the performance of the SLEQOL on SLE
lifting a glass of water’, ‘difficulty with picking things from the patients. We made use of our ongoing prospective study on
floor’ and ‘difficulty with getting in and out of a car’. Wiginton’s SLE outcomes. Data from 275 patients were available at the time
12 concepts are still well represented by the remaining items. of analysis. In this group of patients, there were 249 females and
Developing an SLE quality-of-life instrument 1271
26 males, with a mean age of 40.1 13.4 yr. The mean age at suggests no or negligible correlation, r ranging from 0.20 to
diagnosis was 31.6 15.2 yr. There were 213 Chinese, 14 Indians, 0.29 suggests weak negative correlation, r ranging from 0.30
41 Malays and seven of other races. Prospective data at 3-month to 0.39 suggests moderate negative correlation, r ranging from
intervals were available for 95 patients and repeat data collection 0.40 to 0.69 suggests strong negative correlation and 0.70
in 2 weeks for 51 patients. or lower suggests very strong negative correlation. There was
The SLEQOL has 40 items scored from 1 to 7, with higher little correlation between SLEQOL and SLEDAI, and between
values corresponding to worse QOL. Therefore the minimum SLAM and SLICC/ACR-DI. Generally, the summary score
score was 40 and the maximum 280, with a range that spanned of the SLEQOL correlated better with the SF-36 than do
240 units. We took the summary score as the raw sum of these its individual subsections. As support of the construct validity of
responses. It is also possible to derive the summary score from SLEQOL, subsection 1 (questions on physical ability) correlated
the sum of the mean of each of the six subsections, but this somewhat with the Physical Functional domain of the SF-36 and
may place undue emphasis on those subsections with a greater subsection 6 (questions regarding self-esteem, embarrassment and
number of items. This study was not designed to determine if unpredictability) showed some correlation with the helplessness
and how of the items should be weighted. These issues have to be subscale of the RAI.
addressed in future analyses.
Rasch analysis of SLEQOL
Face and content validity Rasch model analysis of SLEQOL is shown in Table 4. The
These are subjective properties of QOL instruments. Face validity outcomes are generally the same though not identical to those
is satisfied when a group of informed individuals (health-care in the previous investigation. Rasch model analysis is designed
TABLE 3. The cross-sectional correlation between the SLEQOL (and its subsections) and SF-36, Rheumatology Attitudes Index (and its helplessness
index) the two SLE activity indices SLEDAI and SLAM, and the damage index SLICC/ACR-DI
SLEQOL
Subsection 1:
Summary Physical Subsection 2: Subsection 3: Subsection 4: Subsection 5: Subsection 6:
score functioning Activities Symptoms Treatment Mood Self-image
SF-36
Physical functioning 0.112 0.234 0.069 0.066 0.066 0.055 0.042
Role physical 0.140 0.139 0.106 0.129 0.087 0.067 0.044
Bodily pain 0.171 0.171 0.098 0.159 0.084 0.090 0.084
General health 0.127 0.090 0.058 0.112 0.063 0.102 0.080
Vitality 0.140 0.059 0.087 0.134 0.043 0.127 0.082
Social functioning 0.107 0.069 0.087 0.059 0.042 0.108 0.055
Role emotional 0.061 0.056 0.029 0.045 0.030 0.076 0.030
Mental health 0.125 0.034 0.071 0.096 0.045 0.171 0.082
Rheumatology Attitudes Index 0.081 0.042 0.016 0.068 0.041 0.053 0.110
Helplessness subscale 0.115 0.053 0.048 0.078 0.046 0.099 0.119
SLEDAI 0.022 0.003 0.018 0.025 0.013 0.010 0.011
SLAM 0.018 0.016 0.010 0.010 0.029 0.007 0.009
SLICC 0.054 0.081 0.022 0.019 0.047 0.000 0.091
1272 K. P. Leong et al.
TABLE 4. Rasch analysis of the 40-item SLEQOL based on data provided by 275 SLE patients
The items are ranked by the degree of difficulty, with the easiest at the top of the table.
TABLE 5. Number of patients whose responses were at floor and ceiling values of the subsections of the SLEQOL and SF-36
Number (%) who responded with the Number (%) who responded with the
floor value (good QOL) ceiling value (poor QOL)
SLEQOL
Subsection 1: Physical functioning 121 (44%) 0 (0%)
Subsection 2: Activities 49 (17.8%) 0 (0%)
Subsection 3: Symptoms 41 (14.9%) 0 (0%)
Subsection 4: Treatment 112 (40.7%) 1 (0.4%)
Subsection 5: Mood 107 (38.9%) 7 (2.6%)
Subsection 6: Self-image 46 (16.7%) 1 (0.4%)
SF-36
Physical functioning 2 (0.7%) 40 (14.6%)
Role physical 52 (18.9%) 143 (52.0%)
Bodily pain 1 (0.4%) 77 (28.0%)
General health 0 (0%) 1 (0.4%)
Vitality 0 (0%) 3 (1.1%)
Social functioning 3 (1.1%) 115 (41.8%)
Role emotional 62 (22.6%) 162 (58.9%)
Mental health 0 (0%) 7 (2.6%)
The SF-36 and SLEQOL are scored from opposite directions but we have made the scoring consistent in this table. For both instruments, the floor
values represent good perceived QOL and ceiling values poor QOL.
Developing an SLE quality-of-life instrument 1273
The finding that SLEQOL has a more significant floor effect and entire subsection 2); they are the social and occupational activities.
the SF-36 a ceiling effect suggests that they could compensate The second factor consisted of questions 16, 17, 19 and 28–31
for each other’s deficiency and they should be used together. (parts of subsections 3 and 5); they appear to be the mood-related
and self-image questions. The third factor was made up of
questions 1–6 (exclusively subsection 1), all regarding physical
Test–retest reliability functioning. The fourth factor contained questions 16–23 (exclu-
Fifty-one patients with stable disease were asked to repeat the sively subsection 3), concerning physical symptoms. The fifth
questionnaire after 2 weeks to determine the test–retest reliability. factor consisted of items 36–39 (part of subsection 6), concerned
The intraclass correlation coefficient was 0.83 for the summary unpredictability of the response to treatment. Factor 6 was
score. For subsections 1–6, the intraclass correlations were 0.59, made up of items 32–35 (another part of subsection 6), regarding
0.57, 0.72, 0.52, 0.60 and 0.80, respectively. self-esteem. Factor 7 consisted of items 24–27 (exclusively sub-
section 4), all unpleasant aspects of the treatment of lupus.
Factor 8, of very low eigenvalue, consisted of items 6, 14 and 15,
Internal consistency regarding the ability to walk 3 km, ability to go out under the
sun and making money.
For the entire SLEQOL, Cronbach’s alpha was 0.95. For
subsection 1 (questions 1–6), Cronbach’s alpha was 0.85, for
subsection 2 (questions 7–15) it was 0.90, for subsection 3 Responsiveness of SLEQOL and SF-36 to
(questions 16–23) it was 0.89, for subsection 4 (questions 24–27)
changes in QOL
it was 0.76, for subsection 5 (questions 28–31) it was 0.93 and for
TABLE 6. Factor analysis of the 40 items of the SLEQOL (values below 0.4 are suppressed)
No. Item/factor 1 2 3 4 5 6 7 8
1 Walking outdoors on level ground 0.812
2 Shopping 0.778
3 Turning taps on and off 0.663
4 Going to the market 0.828
5 Bathing and drying yourself 0.791
6 Walking 3 km 0.563 0.442
7 Work and school performance 0.836
8 Interference with career or education 0.851
9 Missed work or school 0.799
10 Difficult relationship 0.709
11 Sports 0.483
12 Sex 0.490
13 Difficult social activities 0.594
14 Cannot go out under the sun 0.659
15 Making less money 0.500
16 Poor memory 0.437 0.470
17 Loss of appetite 0.421 0.503
18 Fatigue 0.488
19 Poor concentration 0.481 0.493
20 Itchy skin 0.654
21 Sore mouth 0.781
22 Sore skin 0.780
23 Joint pain 0.498
24 Fear of needles 0.659
25 Dietary restrictions 0.621
26 Inconvenience of daily medication 0.686
27 Inconvenience of clinic visits 0.581
28 Self-consciousness 0.540
29 Feeling low 0.818
30 Depression 0.838
31 Anxiety 0.782
32 Wishing others did not know that I have SLE 0.644
33 Made fun of 0.572
34 Low self-esteem 0.631
35 Embarrassment 0.760
36 Concern about financial burden 0.662
37 Concern medicines don’t work 0.783
38 Concern about side-effects of medicines 0.785
39 Fear of receiving bad news from doctors 0.808
40 More alcohol or tobacco 0.449
Eigenvalues 15.11 3.02 2.23 1.92 1.71 1.38 1.27 1.10
% of variance 37.76 7.56 5.57 4.81 4.26 3.46 3.17 2.76
Cumulative % 37.76 45.32 50.89 55.69 59.96 63.42 66.59 69.34
The factors were extracted by principal components analysis and varimax-rotated with Kaiser normalization.
1274 K. P. Leong et al.
Mental
Functioning Physical Pain Health Vitality Functioning Emotional Health
0.01
0.01
0.01
0.12
0.12
0.12
0.01
0.01
6 ‘a great deal worse’, 5 ‘a good deal worse’, 4 ‘moderately
worse’, 3 ‘somewhat worse’, 2 ‘a little worse’, 1 ‘almost the
same, hardly any worse at all’, 0 ‘no change’, þ1 ‘almost the
same, hardly any better at all’, þ2 ‘a little better’, þ3 ‘somewhat
Role
0.13
0.02
0.14
0.02
0.15
0.02
0.06
0.06
better’, þ4 ‘moderately better’, þ5 ‘a good deal better’, þ6
‘a great deal better’, and þ7 ‘a very great deal better’. Patients
who rated the change as 1, 0 or þ1 were regarded as having
no difference in QOL, while those who rated it as þ2, þ3, 2 or
Social
0.21
0.02
0.20
0.01
0.18
0.01
0.06
0.06
3 were felt to have a small but important change. Scores of
þ4, þ5, 4 and 5 suggested moderate change and scores of
þ6, þ7, 6 and 7 suggested large change [25].
We used a variety of responsiveness statistics to analyse the
0.11
0.01
0.11
0.01
0.11
0.01
0.03
0.03
data. Liang’s relative efficacy (RE) is formed by squaring the
SF-36
ratio of the appropriate t statistics of each instrument to be
0.31
0.10
0.26
0.09
0.26
0.09
0.32
0.31
by the standard deviation of the change in scores [28]. Effect size
is calculated as mean change divided by the standard deviation
of the baseline scores [29]. Guyatt’s coefficient is equal to the
0.13
0.08
0.12
0.07
0.12
0.07
0.12
0.11
mean of change in scores divided by the between-subject variability
or the within-person change in stable subjects [30].
0.11
0.08
0.12
0.08
0.03
0.03
In all, 119 data pairs from 95 patients were available for
analysis. Other investigators have also used more than one pair
of QOL data from one patient when available [33]. There were
12 data sets in which a decreased global QOL was reported, 50
0.00
0.29
0.00
0.20
0.00
0.20
0.04
0.03
were only 12 in which the QOL worsened, we used the data from
the unchanged and improved patients to generate the responsive-
ness statistics (Table 7). All the four responsiveness statistics
show that the SLEQOL is more sensitive than any of the eight
0.23
0.16
0.19
0.17
0.22
0.17
0.24
6
–
domains of the SF-36. However, in the patients whose QOL
was unchanged, the SLEQOL scores varied to a greater degree
than those of the SF-36. Therefore, it is reasonable to state that
the SLEQOL was more sensitive but less specific to change than
0.37
0.07
0.33
0.07
0.35
0.07
0.54
5
–
the SF-36.
However, it is also clear from Table 7 that subsections 3, 4
and 5 of the SLEQOL were the best indicators of change of QOL,
superior to the summary score and all the domains of the SF-36.
0.23
0.05
0.25
0.04
0.22
0.04
0.20
They were both very sensitive and specific to the changes in
4
–
the QOL.
SLEQOL
0.59
0.00
0.48
0.00
0.57
0.00
1.00
3
–
Minimal clinically important difference
The MCID is a valued property of a QOL instrument because
0.51
0.27
0.39
0.19
0.43
0.19
0.73
0.29
0.12
0.31
0.12
0.17
TABLE 7. Responsiveness of the SLEQOL and SF-36
0.44
0.21
0.33
0.15
0.37
0.15
1.00
0.97
Rheumatology
we felt that most of the domains described in that paper reflect A new systemic lupus erythematosus-
our patients’ concerns. An analysis of the 51 items shows that they specific quality-of-life instrument,
11. Murawski MM, Miederhoff PA. On the generalizability of statistical Index for systemic lupus erythematosus. Arthritis Rheum
expressions of health related quality of life instrument responsiveness: 1996;39:363–9.
a data synthesis. Qual Life Res 1998;7:11–22. 25. Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining
12. Webb SM, Prieto L, Badia X et al. Acromegaly Quality of Life a minimal important change in a disease-specific quality of life
Questionnaire (ACROQOL) a new health-related quality of life questionnaire. J Clin Epidemiol 1994;47:81–7.
questionnaire for patients with acromegaly: development and 26. Wright JG, Young NL. A comparison of different indices of
psychometric properties. Clin Endocrinol 2002;57:251–8. responsiveness. J Clin Epidemiol 1997;50:239–46.
13. McKenna SP, Doward LC, Whalley D, Tennant A, Emery P, 27. Liang MH, Larson MG, Cullen KE, Schwarz JA. Comparative
Veale DJ. Development of the PsAQoL: a quality of life instrument measurement efficiency and sensitivity of five health status instru-
specific to psoriatic arthritis. Ann Rheum Dis 2004;63:162–9. ments for arthritis research. Arthritis Rheum 1985;28:542–7.
14. Grootscholten C, Ligtenberg G, Derksen RH et al. Health-related 28. Liang MH, Fossel AH, Larson MG. Comparisons of five health
quality of life in patients with systemic lupus erythematosus: develop- status instruments for orthopaedic evaluation. Med Care 1990;
ment and validation of a lupus specific symptom checklist. Qual Life 28:632–42.
Res 2003;12:635–44. 29. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting
15. Tan EM, Cohen AS, Fries JF et al. The 1982 revised criteria for changes in health status. Med Care 1989;27:S178–89.
the classification of systemic lupus erythematosus. Arthritis Rheum 30. Guyatt G, Walter S, Norman G. Measuring change over time:
1982;25:1271–7. assessing the usefulness of evaluative instruments. J Chronic Dis
16. Hochberg MC. Updating the American College of Rheumatology 1987;40:171–8.
revised criteria for the classification of systemic lupus erythematosus. 31. Bond TG, Fox CM. Applying the Rasch model. New Jersey:
Arthritis Rheum 1997;40:1725. Lawrence Erlbaum Associates, 2001.