Professional Documents
Culture Documents
William E. Pollard
Office of Communication
Centers for Disease Control and Prevention
1600 Clifton Rd., N.E., Mailstop D-42
Atlanta, Georgia 30333
KEY WORDS: Survey Methods, Mail Panel than a million individuals across the range of census
Surveys, Health Communication Data demographics for use in market research.
2720
Joint Statistical Meetings - Section on Health Policy Statistics (HPSS)
sample obtained through conventional random digit dial on age, race, sex, income, and family size (Korn and
procedures. They found that the two samples were very Graubard, 1999; Lohr, 2000). For a full description of
similar on consumer behavior, lifestyle, and health the survey see Maibach, et al. (1996).
status items. In addition, they found no differences in
positive outlook, altruism, and amount of free time. A number of the items on the HealthStyles
survey are the same as, or similar to, items on the
In an extensive, widely-discussed study of Behavioral Risk Factor Surveillance System (BRFSS)
civic involvement in the U.S., Putnam (2000) used survey. This is a national probability sample telephone
results from the General Social Survey and the DDB interview survey. It is a collaborative project of the
Needham Lifestyles survey. The General Social Survey CDC and U.S. states. The sample size for core items
is a personal interview survey conducted by the Na- administered in all interviews is 150,000 to 180,000
tional Opinion Research Center (NORC) using a respondents annually. It is an accepted and widely-
national probability sample. The Lifestyles survey is a used public health surveillance and planning survey.
national mail panel survey conducted annually with Annual statistics for selected items from the survey are
independent samples by Market Facts for DDB summarized on the CDC web site (www.cdc.gov) and
Needham, a large advertising agency. The Lifestyles are used here in the comparisons with HealthStyles
samples are drawn from the panel through stratified items. Most of the national prevalence percentages
random sampling. The two surveys contained a number reported in the tables on the web site are medians of
of similar opinion and lifestyle items. Putnam com- state values.
pared results of the surveys over a period of 25 years
and found very close agreement in (1) the level of There were nine items from the BRFSS on the
response, (2) trends over time, and (3) the pattern of web site that were comparable to items on the
demographic correlates. He writes that “for purposes of HealthStyles survey from 1995 to 2001. These include:
explaining this wide range of attitudes and behavior, the
two surveys are virtually indistinguishable” (p. 422). Health Conditions: arthritis, asthma, diabetes,
high blood pressure, and overweight or obese,
CURRENT STUDY where this weight category is defined in terms
of Body Mass Index (BMI) of 25.00 or
Objectives greater;
The objective of this study is to examine the Attitudes: health status self-rating;
reliability and validity of a health communication data
base in which the data are obtained from a national mail Behaviors: flu shot, pneumonia vaccination,
panel survey. Reliability is assessed in terms of stabil- smoke cigarettes.
ity of item results over time: results should be stable
from year to year in the absence of known trends or Note that not every item is measured every year on
events that would be expected to cause changes. either survey.
Validity is assessed in terms of agreement with other
measures of the items that are themselves considered to The HealthStyles survey is not a surveillance
be valid. survey, but is designed to provide data for understand-
ing audiences for communication and health promotion
Methods planning. Many of the items do not have counterparts
on other surveys that can be used to examine validity.
In this paper we examine data from the However to the extent that results for the above items
HealthStyles mail panel survey conducted for Porter can be shown to be valid in comparison with results
Novelli, a social marketing and public relations firm. from an accepted national probability sample survey, it
This survey has been conducted annually with inde- provides indirect support for the validity of the results
pendent samples since 1995 and contains items on of other items on the survey.
health attitudes, behaviors, conditions, and knowledge.
From 1995 to 2001, the survey has been administered Results
by Market Facts to respondents to the DDB Needham
Lifestyles panel survey, with oversampling for low The basic findings are shown in the bar charts
income and minority groups. The sample size is around in Figure 1. The 95% confidence interval for the
2,500 to 3,000 respondents per year. Results are HealthStyles values is approximately plus or minus two
poststratified and weighted to US census benchmarks percentage points around the reported value. In terms
2721
Joint Statistical Meetings - Section on Health Policy Statistics (HPSS)
In quantitatively summarizing the validity, for Maibach, E., Maxfield, A., Ladin, K., and Slater, M.
the nine items over the seven years, there were 34 (1996). Translating health psychology into
same-year data pairs where the two surveys could be effective health communication: The Ameri-
directly compared. For these 34 pairs of percentages, can HealthStyles Audience Segmentation
the average difference was 2.35 percentage points and Project. Journal of Health Psychology, 1,
the average of the absolute value of the differences was 261-277.
2.88 percentage points. A scatterplot of the
HealthStyles and BRFSS percentages is shown in Market Facts (1994). Mail Panels vs. General Sam-
Figure 2. For these 34 paired percentages, the correla- ples: How Similar and How Different? Re-
tion between the HealthStyles and BRFSS values was search on Research, Report No. 59. Arlington
r = .99, reflecting the close agreement. Heights, IL, Market Facts.
These findings, along with those from the The author would like to express his thanks to Dr.
previous research, also have some more general practi- Edward Maibach and Dr. Deanne Weber from Porter
cal and theoretical implications for survey research. As Novelli for their encouragement of this project, and to
mentioned earlier, panels have a number of significant Dr. Weber for help in understanding the HealthStyles
advantages for facilitating the conduct of surveys and methodology and the features of the data files.
reducing costs. It would be helpful to have some
general guidelines concerning the conditions under
which panel and non-panel surveys will yield compara-
ble results. Additional empirical and theoretical work
on this topic would be very useful to survey research-
ers.
2722
Joint Statistical Meetings - Section on Health Policy Statistics (HPSS)
Figure 1. Bar charts showing HealthStyles and BRFSS results from 1995 to 2001.
2723
Joint Statistical Meetings - Section on Health Policy Statistics (HPSS)
2724