You are on page 1of 27

P1: FQK/FPO P2: FLW/FFO P3: FCT

March 31, 2000 10:30 AR-100 CHAP-25

Annu. Rev. Public Health. 2000. 21:613–37


Copyright
c 2000 by Annual Reviews. All rights reserved

TELEMEDICINE: A New Health Care


Delivery System

?
Rashid L. Bashshur,1 Timothy G. Reardon,2 and
Gary W. Shannon3
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

1Telemedicine Program, University of Michigan Health System, Ann Arbor, Michigan


48109-0826; e-mail: bashshur@umich.edu
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

2Vector Research Incorporated, Ann Arbor, Michigan 48108;

e-mail: reardont@vrinet.com
3Department of Geography, University of Kentucky, Lexington, Kentucky 40506-0027;

e-mail: gwshan00@pop.uky.edu

Key Words telehealth, informatics, information technology, on-line health,


Internet, health telematics
■ Abstract The resurgence of telemedicine can be attributed to its potential for
addressing intransigent problems in health care, including limited accessibility, cost in-
flation, and uneven quality. After discussing definitions and the genesis of telemedicine,
this review focuses on conceptual issues and an assessment of past research. The scope
and methodological rigor necessary for sustained development and policy making
have been limited in this area of research, owing to the nature of extant telemedicine
projects and the lack of a comprehensive research strategy that specifies the objectives
of telemedicine research regarding accessibility, cost, and quality. Research strategies
and a framework for analysis are discussed. Without a commitment to the types of
research objectives, framework, and strategy presented here, the considerable promise
of telemedicine, as an innovative system of care, may not be fully realized.

INTRODUCTION AND DEFINITION


This review provides (a) a brief overview of the definition and genesis of telemedi-
cine, (b) a description of the enabling technology and its continuing development,
(c) a conceptual framework for analysis, and (d) an assessment of telemedicine
research.
Often, telemedicine is used as an umbrella term to refer to the remote deliv-
ery of health care and health information. Some have used the term telehealth
interchangeably with telemedicine despite obvious differences in their referents,
as is explained below. Both telemedicine and telehealth involve the electronic
transfer of medical and health information between distant sites and participants.

0163-7527/00/0510-0613$14.00 613
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

614 BASHSHUR ■ REARDON ■ SHANNON

In both instances, the distinguishing feature is reliance on telecommunications and


electronic information technology as a substitute for personal contact between the
participants or units involved in the process. Full-service telemedicine networks
or systems provide a broad range of clinical services in various specialty areas, as
well as continuing medical education and preventive health.
In most secondary and tertiary clinical-specialty areas, the term telemedicine
is applied generally and augmented by specific clinical appellations, for example,
teleoncology, teledermatology, or telepsychiatry. In addition, diagnostic medical

?
services such as radiology and pathology that use this technology to capture,
transmit, store, and retrieve information are also provided specific designations,
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

in this instance, teleradiology and telepathology, respectively.


The early definitions of the field focused on patient care as the core activity of
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

telemedicine. For example, Bird (13), who can justifiably be considered the pioneer
of telemedicine, provided the first formal and published definition of telemedicine
as “the practice of medicine without the usual physician-patient confrontation.
. . . via [an] interactive audio-video communications system.” A somewhat more
inclusive definition of telemedicine was proposed in 1971 by Willemain & Mark
(46) as “any system in which the doctor and the patient are at different loca-
tions.” A few years later, an operational definition was proposed by Bashshur (5).
This definition viewed telemedicine as a system of care composed of six essential
elements: (a) geographic separation between provider and recipient of informa-
tion, (b) use of information technology as a substitute for personal or face-to-face
interaction, (c) staffing to perform necessary functions (including physicians, as-
sistants, and technicians), (d ) an organizational structure suitable for system or
network development and implementation, (e) clinical protocols for treating and
triaging patients, and ( f ) normative standards of behavior in terms of physician
and administrator regard for quality of care, confidentiality, and the like.
Again, it should be noted that early definitions focused on medical care as the
only function and justification for telemedicine. However, as early as 1978, the
scope and, therefore, definition of telemedicine were expanded by Bennet and
associates (10) to include the concept of telehealth. In this expanded context, tele-
health included “systems [that] support the health care process by providing the
means for more effective and more efficient information exchange.” It was sug-
gested that telehealth should incorporate “a broad range of health-related activities,
including patient and provider education, and health services administration—as
well as patient care.” Nonetheless, in both telemedicine and telehealth, all appli-
cations share two common elements, namely the geographic separation between
two or more actors engaged in health care and the use of telecommunication and
related technology to enable, facilitate, and possibly enhance clinical care and the
gathering, storage, and dissemination of health-related information (7).
It is noteworthy that, in most European countries, the domain of telemedicine
is sometimes referred to as telematics, reflecting a heavy emphasis on information
technology in general and telemetry in particular. The World Health Organization
has fostered the inclusive concept of health telematics to refer to various “health
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 615

related activities, services, and systems carried out over a distance by means of
information and communications technology” (25).
The multiple and emerging definitions of telemedicine reflect the facts that the
current generation of telemedicine is evolving and that new uses for the technology
are being discovered and implemented. In the interest of enabling and facilitating
this discussion, it is probably most efficacious to consider telemedicine to be sub-
sumed under the more broadly defined and interpreted concept of “telehealth.” In
other words, the focus here is on telemedicine as defined by its role of electronic

?
communication in the provision of medical care in the clinical setting. Therefore,
discussions of the impact of information technology purely on health care man-
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

agement and health behavior/health education activities are beyond the scope of
this assessment of telemedicine and its impact on clinical care.
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

ORIGIN AND EVOLUTION

Typically, it is difficult to trace the development of major innovations to a sin-


gle origin, especially when such innovations are based on a complex technology.
The genesis of telemedicine is no exception. It represents the confluence of two
parallel developments in information technology. Insofar as this technology is con-
cerned, the first application was developed in the National Aeronautics and Space
Administration’s (NASA) manned space flight program and the development of
sophisticated technologies for biomedical telemetry, remote sensing, and commu-
nication in space. The second application stemmed from the telecommunications
industry in the private sector, which was subsequently expanded immensely with
the growth of computer technology.
In the first instance, during the early stages of the manned space program,
NASA’s scientists were concerned about the physiological ill effects of zero gravity
on astronauts. Vital functions, including heart rate, blood pressure, respiration rate,
and temperature, were monitored constantly during space flight. The high-intensity
surveillance was later reduced to periods of high stress. These early systems pro-
vided for the unobtrusive monitoring of astronauts’ vital functions during short-
duration space flight. Longer flight times and the plans for orbital stations led to
the development of telemedical capabilities for diagnosis and treatment of medical
emergencies, establishment of health maintenance systems, and biomedical exper-
imentation. One illustrative byproduct of this era was the portable cardiac monitor-
ing and resuscitation package now widely used by paramedical emergency teams.
Telemedicine technology is not solely a derivative of the space program, how-
ever. For example, in 1957 Jutras transmitted radiographs from the Hotel Dieu
Hospital in Montreal (41), and Wittson & Benschoter (47) used closed-circuit
television in 1959 to conduct group therapy sessions between the Nebraska Psy-
chiatric Institute in Omaha and the Norfolk State Hospital located some 112 miles
away. DeBakey (22) broadcast an open-heart surgery to Switzerland in 1965,
using Early Bird, the world’s first communications satellite. Nonetheless, it was
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

616 BASHSHUR ■ REARDON ■ SHANNON

Bird who, in 1967, established the first prototypical interactive telemedicine sys-
tem, which linked the medical station at Boston’s Logan International Airport and
the Massachusetts General Hospital. By microwave relay, this system provided
a complete range of primary-care and emergency services, staffed by attending
registered nurses at the medical station and remote consulting physicians at the
hospital. Although primitive by today’s technology, this system ushered in the
systematic use of telemedicine in a multiservice primary-care clinic. This was
followed by another prototype project on the Papago reservation (now Tohono

?
O’dham) in 1973 (33). NASA and the Indian Health Service joined to offer com-
prehensive primary-care services and radiology via telemedicine and a mobile
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

health clinic. NASA’s concept was developed several years earlier as part of a pro-
gram called Integrated Medical and Behavioral Laboratory Measurement Systems.
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

The telemedicine component of that program was technically referred to as Area


Health Service Unit, later renamed “STARPAHC” (Space Technology Applied to
Rural Papago Advanced Health Care) (33).
The first-generation telemedicine programs in the United States ended rather
abruptly. Government funding was terminated before any program had matured
to a sustainable status or reached a steady state of operation. Still in the devel-
opmental stage, the technology was expensive, cumbersome, and unreliable. All
projects were phased out within <5 years of their initiation, including the one
at Logan Airport medical station. In retrospect, these projects demonstrated the
technological feasibility of telemedicine, its ready acceptance by both providers
and clients who used it, the substitution of technology for travel, and the poten-
tial for greater coordination of medical and administrative functions within large
institutions. However, despite their historical importance in demonstrating the
technological and clinical feasibility of telemedicine, all of these applications fell
short of expectations, perhaps because the expectations were not reasonable.
The recent resurgence of telemedicine, starting in the early 1990s, has evolved
amid and because of the rapid expansion of information and telecommunication
technologies. To be sure, telemedicine is a product of the information age, just
as the assembly line was a product of the industrial age. Both innovations have
capabilities for producing dynamic and systemic changes with far-reaching effects
on productive processes, reallocation of resources, and market dynamics.
The resurgence has generated some of the same excitement and promise that
accompanied its introduction in the 1970s, no doubt because the nation is still
struggling with the same intransigent problems of limited access to care for certain
segments in the population, escalating cost, and uneven quality of care.
Federal policies promoting and supporting the development of the National
Information Infrastructure (NII) have produced additional impetus for establishing
a more robust telemedicine presence (34).
Currently, every state in the union has at least one telemedicine program. Some
of these, such as the Georgia Statewide Telemedicine Program (1), are statewide,
offer a comprehensive range of clinical services and continuing education, and
involve a large number of hospitals and clinics throughout their respective states.
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 617

A recent survey of rural applications of telemedicine (30) of all nonfederal rural


hospitals in the United States (n = 2472) revealed that nearly one of four such
hospitals (n = 558) had telemedicine. As of 1997, about two thirds of telemedicine
programs in rural hospitals had operated for ≤2 years (30). However, nearly two
thirds of these programs used telemedicine for teleradiology only. Teleradiology, in
one form or another, now accounts for >50% of all radiology practice in the United
States (20). The high volume in teleradiology may reflect the fact that this service
is currently reimbursable, and the American College of Radiology was the first

?
professional association to develop explicit standards for quality of images in tele-
radiology, referred to as “DICOM” (Digital Imaging and Communications in
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

Medicine), a vendor-independent standard for data formats and transfer, now in


its third version, DICOM3.
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

Curiously, forces outside the health care system and independent of research in
telemedicine seemingly have secured a thriving future for this field. Strong techno-
logical and commercial forces, including the entertainment industry, commerce,
education, interpersonal communications, and the military, are forging ahead into
the information age, modernizing their services and production functions with the
ever expanding capabilities of information technology. Nowhere is this more evi-
dent or compelling than in electronic commerce and entertainment, which threaten
to capture the initiative in the development of telecommunications-based medical
care, such as “e-health.” (This term is a derivative of the burgeoning “e-commerce”
that is expected to gain prominence in the health care vernacular.) Reason for
concern has been supported by informal and formal evaluations and even legal
challenges to services provided by medical care web sites, which provide on-line
medical consultations with minimal medical information, lacking continuity and
accountability.

CONCEPTUAL FRAMEWORK
Telemedicine is a complex innovation bundle in that it is a technical as well as an
organizational and social innovation. The communications and computing tech-
nologies being developed and used in the delivery of health care represent only
one dimension of the innovation. However, the capabilities of the equipment must
be such that the information transmitted is at least as complete as and equal in
quality to the information transmitted in the traditional setting. In some instances,
the information is enhanced. Indeed, the capabilities of the technology are ex-
panding rapidly, constantly presenting faster, more efficient, and richer forms of
information, often without a concomitant increase in cost. However, the mere fact
that electronic information can serve as an adequate substitute for physical pres-
ence may not be sufficient to overcome the inertia of comfort and familiarity of
the personal encounter, even in consultant care. Therefore, the process leading
to acceptance of telemedicine, namely, awareness, interest, evaluation, trial, and
adoption/rejection, as a routine part of the medical care process on the part of
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

618 BASHSHUR ■ REARDON ■ SHANNON

physicians and patients, is complex. At the same time, information technology has
become a critical component of clinical practice, clinical decision making, health
management, and medical and health education. “There is now so much depen-
dence on electronic information processing and exchange in health care that it is
difficult to imagine how the system would function without this technology” (8).
A second important dimension of telemedicine innovation pertains to social,
organizational, and human elements. Telemedicine alters the traditional physician-
patient relationship, in terms of complexity and essence. In the former, rather than

?
a one-on-one relationship with a physician, the patient now encounters (most fre-
quently) a local provider, with whom there is direct personal contact, and a remote
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

provider, with whom there may be only electronic or virtual contact. In addition,
in the majority of instances as currently configured, the telemedicine clinical visit
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

often involves technicians who assist in operating the telecommunications equip-


ment, computers and peripheral devices, as well as other auxiliary medical person-
nel. The essential nature of the (usually) specialist physician-patient encounter is
changed from one of human to one of electronic contact and information exchange.
Organizationally, telemedicine provides challenges to the traditional notions of
regionalized health care systems as well. In the past, there have been numerous
attempts at defining and developing regional medical- and health-planning pro-
grams (29). Among the persistent problems not successfully addressed in these
efforts was determination of the constituent parts, the organizational structure, the
operational system, and the boundaries of the planning regions. Moreover, regions
were mostly static, and the distribution of need within them was not homogeneous.
Telemedicine has the potential to create “virtual” regions that can attend to a hierar-
chical range of needs and transcend the traditional geographic limits of previous
attempts. Theoretically, specialty medical care can be delivered from point to point
as needed, regardless of location and duration. Therefore, these virtual regions
are, in fact, networks that may include a discontinuous set of points in an elec-
tronically connected virtual hierarchy of primary-, secondary-, and tertiary-care
providers.
The significance of telemedicine in regional delivery will vary by specialty,
patient characteristics, and even stage of care. At the same time, advances in
enabling technologies continue to modify the flow of information and people.
Hence, sharing expertise and information and collaborative relationships among
providers will likely occur more readily with further development of the Internet.
The Next-Generation Internet (NGI) already under development promises more
speed, more security, and greater capacity than the current Internet. Such dis-
tributed regional networks will provide advantages of ready access to different
types of information, thereby affecting patient care quality and cost as well as
access. In a review of the impact of technology on this type of regionalization,
Marsh (36) describes the Euromed Project’s attempt to define a regional infor-
mation infrastructure as part of an emerging web-based, telemedical-information
society, where a patient’s medical data can be accessed globally and transparently
from any storage and communication medium. Also, Chronaki et al (18) describe a
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 619

hierarchy of virtual workspaces in terms of lifetime, location, and access privileges


to fit the needs of different applications and users.
However, these virtual medical regions cannot stand alone in the delivery of
health care. Ultimately, the electronically delivered specialty expertise, at least
currently, must be applied locally and in person. Therefore, we must consider not
only the content and extent of the virtual medical care region, but also the interface
between the virtual region and the traditional distribution and regionalization of
health resources. Making the situation more complex is the fact that, although

?
virtual telemedicine regions can ignore or transcend the realities of traditional
geographic boundaries, political boundaries still impose formidable barriers in
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

terms of interstate licensure, legal liability, and other administrative regulations.


Access provided by 160.152.162.233 on 08/16/21. For personal use only.

THE ENABLING TECHNOLOGY OF TELEMEDICINE

The potential functions of telemedicine, as well as its processes and outcomes,


depend heavily on the specific technological infrastructure in place. This technol-
ogy has undergone considerable change and transformation during the past several
decades, and it continues to change. Hence, an understanding of the evolution of
current and emerging technologies will facilitate an understanding of the feasible
range of telemedicine activities and outcomes, now and in the future.
In broad terms, the history of telemedicine technology can be characterized as
consisting of three major eras. The first was the telecommunications era of the
1970s. Early telemedicine during this era provided a secondary solution to a very
limited range of in-person, medical care problems. It was dependent on broadcast
and television technologies in which audio and visual data were not integrated
with other clinical data or easily stored and accessed. Telecommunications was
merely a mode of transport for signals, in which the coordination of multiple
complex signals was costly, cumbersome, and not sufficiently reliable for many
clinical applications. While telemedicine in this era had the same purpose as today’s
telemedicine, the technological configurations and potential impacts then were
dramatically different from those of current systems.
The second telemedicine era was the digital era, which was ushered in during
the late 1980s by digitalization in telecommunications and grew during the 1990s.
The digital era facilitated the integration of telecommunications and computer ser-
vices and enabled the transmission of more information on limited bandwidths.
Essentially, telephone lines and switches were used in various combinations, rang-
ing from a fraction of a dedicated telephone line to a full “T1” line (which has a
capacity of 1.544 Mbit/s) and to combinations of T1 lines. (The most advanced
combination with a switching device is currently a T3 line, which has a capacity of
44.736 Mbit/s.) Often, dedicated telephone lines were used to connect several part-
ners, and the connection could be point to point (e.g. hospital to hospital), point to
multipoint (e.g. hospital to remote clinics), or multipoint to multipoint (as in a full
multiplexed network). The digital era was supported by Integrated Service Digital
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

620 BASHSHUR ■ REARDON ■ SHANNON

Network technology, which consists of digital telecommunications channels and


protocols that permit the integrated transmission of voice, video, and data at high
speed and provide connections to a universal network.
The digital era in telemedicine is now phasing into the era of a more complex
and ubiquitous communications network of networks, the Internet. Nonetheless,
the earliest designs of the digital era were still dependent on television technology
for imaging capabilities and were uniformly dependent on private networks with
vendor-defined standards. Although limited by funding, the usability of equipment

?
(a term that refers to the efficiency of the human-machine interface) and associated
costs appear to have restricted telemedicine activity to pilot projects or narrowly
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

defined special groups. Further integration of telecommunications and computers


has been coupled with standardized protocols and tools to facilitate widely acces-
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

sible connectivity. In the broadest sense, these have spawned the Internet and the
third era of telemedicine technology. Although the Internet originated in 1969, its
regular use in telemedicine is relatively recent. The Internet allows open access to a
global-communication environment. Within this open global network, a heteroge-
neous set of activities and technologies is bringing about a new environment for a
wide spectrum of personal and professional interactions. Moreover, economic the-
ory posits that, as information becomes more available more quickly and cheaply,
markets become broader and more efficient. In fact, many of the frustrations of
current telemedicine users can be traced to the inability of telemedicine proponents
to provide adequate, accurate, timely, and appropriate information. Indeed, poor
information and the high cost of Integrated Service Digital Network technology
have generated much of the current interest in the Internet.
Advances in computer technology have created very large data repositories,
substantially enhanced visualization, and the integration of multiple media oper-
ating from a single platform. Extensive image, audio, and text information can be
stored and retrieved at the points of origin and care. In addition, computer-aided
remote clinical manipulation and robotics are opening new vistas in telesurgery
considered until recently the domain of science fiction.
The Internet era is a radical departure from the past in very significant ways.
The technology is cheaper, more ubiquitous, and accessible to an ever increasing
user population. Its successful use in commerce, entertainment, and other sectors
of society has sustained a flurry of investment and new applications (31a). Health
care web sites, such as drkoop.com and medscape.com, are joining the ranks of
instant cyber successes. According to a recent Harris poll, 74% of web users look
for information about specific diseases or health problems (4). It is estimated that,
of the 70 million web users seeking health and health care information, many
were looking for information on specific diseases, including, in decreasing order
of magnitude, depression, allergies or sinus conditions, cancer, bipolar disorders,
arthritis or rheumatism, high blood pressure, migraine, anxiety disorders, heart
disease, and sleep disorders (4).
The spectrum of Internet services in health care ranges from posted informa-
tion about health issues and treatment modalities to the provision of professional
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 621

services, including diagnosis, treatment, and drug prescriptions. Most sites spe-
cialize in selected areas of information or services, but others, especially more
recent arrivals, provide a spectrum of options. Major categories of web health
services are the following: (a) information, including general health information
and information on treatment options; (b) professional services, including diag-
nostic services, treatment services, and pharmaceutical services; and (c) informal
information and referral, including chat groups and on-line support groups.
Over the last few years, the Internet has evolved to provide information in all

?
areas of patient concern, but recently a number of significant ventures appeared
offering on-line health care services. News stories on “cyberdoctors” providing
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

diagnostic and treatment services through the Internet are quite common (e.g.
see www.Cyberdoc.com, www.LatinMD.com, and www.Worldcare.com). In re-
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

sponse, specialty areas within medicine, including pediatrics, obstetrics and gyne-
cology, psychiatry, neurology, oncology, surgery, radiology, and ophthalmology,
as well as the American Medical Association and the American Telemedicine
Association, are beginning to discuss the current and future potential impact of
the Internet (2). Researchers and professional organizations are raising concerns
about the accuracy, suitability, and safety of on-line information and treatment. The
vague affiliation and sheer commercialism of some of these sites have increased
the risks for consumers seeking information. A recent study documented alarming
levels of misinformation on health web sites (31).
The problems and rapid growth of the Internet may bring increased attention to
all forms of telemedicine. Scrutiny is likely to become more intense as changes in
Internet protocols and speed make it a more viable environment for telemedicine.
Yet, despite concerns about the expected uncontrolled growth of the Internet, our
understanding of its impact on medical care is limited. Extrapolation from past
telemedicine research is dubious, especially because the preceding technologies in
telemedicine have had limited success, whereas Internet telemedicine seems most
promising in terms of attracting public attention, patient and provider interest, and
greater volume of use. Along with the potential gains to telemedicine, Internet
health care has other looming risks that are becoming apparent. Some of the issues
of Internet telemedicine are similar to those of traditional telemedicine. Projects
that simply use the Internet as a communication medium have to face issues of
quality assurance, security, and accountability. Programs open to all Internet users
present new challenges, such as “blind” provider shopping, ill-advised treatment
modalities, and consumer abuse. Unfettered and uncontrolled access to on-line
care may expose consumers or clients to misinformation about health and disease,
medical treatment, and sources of care. Commercial interests and consumers may
not wait for researchers to guide the orderly development of on-line health care.
Hence, the mere availability of information cannot be expected to improve the
quality of care when the quality of information is not established.
Nevertheless, as the delivery of medical care in the 20th century has been largely
determined by access to technology in medical institutions, medical care in the
21st century will likely be defined by networked virtual access to organizations that
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

622 BASHSHUR ■ REARDON ■ SHANNON

include health, social services, and informal care. Yet the questions of efficiency,
quality, and equitable access remain.
Given the enormous potential of information technology in health care, the
issue of equity is bound to surface. If the technology represents a new and valued
resource available only to the educated and those who can afford it, we should be
concerned about creating another underclass lacking the ability to use it.

?
OBJECTIVES OF TELEMEDICINE RESEARCH
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

To date, a preponderance of telemedicine research has focused on determining di-


rectly evident technological capabilities, acceptance, and cost. That most analyses
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

have converged on the direct effects of technological configurations may explain


the notable tendency to offer new, often simplistic renditions for defining the field.
For example, defining telemedicine simply as the use of telecommunications in
health care not only provides a loose description of the technology involved, but
may also detract from approaching telemedicine as a new system of care capable
of restructuring the medical care landscape. It has been argued that only when
telemedicine is viewed as an integrated network providing services in a single
specialty or multiple specialties will telemedicine’s unique capabilities and inte-
grative functions be realized (7). It is true that telemedicine functions are derived
from the substitution of information technology for face-to-face contact, but its in-
tegrative functions are derived from networking capability, information exchange,
and shared decision making. When optimally implemented, telemedicine embod-
ies functional potentials involving innovative organizational structures, unique
manpower mixes and technological configurations, and standardized normative
standards and protocols.
Despite this potential, telemedicine research has yet to provide the documen-
tary evidence necessary for program strategies and overall health policy. A few
researchers have responded with more detailed methodological frameworks for
analysis, especially economic frameworks. Still others suggest the need for an en-
tirely new paradigm for telemedicine development and research. The impediments
of obtaining guidance from small, suboptimal programs have elicited arguments
for abandoning the process of retrospective analysis for one of modeling and
simulation (16). Although each of these positions may be shown to have merit,
perhaps the most significant gains may be made through a clear understanding of
the functional role of telemedicine as it relates to the existing body of knowledge
in medical care research. This begins with a clear set of expectations or objectives
in telemedicine programs that is consistent with the basic objectives of health care
delivery. Moreover, telemedicine applications may have an array of functional
impacts on the flow of people and information. These include organizational and
patient management as well as clinical care. Still, most definitions require a clinical
programmatic focus, having organizational, management, and patient-processing
components as subsidiary activities.
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 623

TABLE 1 Areas of telemedicine functionality


Organizational-management
oriented areas Patient management areas Clinical-care areas

Financial—billing Patient registration and scheduling Diagnosis


Management decision support Admission Therapy
Personnel management Results reporting and records Monitoring

?
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

PROPOSED RESEARCH STRATEGY


Access provided by 160.152.162.233 on 08/16/21. For personal use only.

Recent experience has demonstrated that the practice of telemedicine may entail
significant changes in the medical care process and subsequent outcomes, as a re-
sult of changes in the manner in which health information is obtained, processed,
and exchanged. The nature and magnitude of these changes depend on a complex
set of related factors, including the nature of the clinical and/or educational appli-
cations being implemented, the technological configurations in use, the manpower
mixes in effect, and the organizational and community contexts in which such sys-
tems are installed. Hence, it is futile to assess the effects of telemedicine in general
terms without specifying the content and context of each application under consid-
eration. Moreover, beyond the assessment of technical sufficiency, forming sound
expectations about the impact of telemedicine on health care delivery requires
an understanding of the functional relationships between telemedicine technology
and the outcomes of cost, quality, and access. Finally, the determinants of these
functions must be malleable or at least observable for policy and organizational
decision makers. Understanding these relationships along with a well-conceived
research methodology provides the basis for developing efficient research strate-
gies and understanding.
Telemedicine research must define the constituencies and the role of telemedi-
cine in meeting user needs. Indeed, the clinical capabilities of telemedicine con-
stitute a precondition to an effective system, but, once capabilities are established,
there are several key questions. Is there a unique functional role of telemedicine
in health care delivery? How does telemedicine achieve this role? How does this
role affect health system performance in terms of organizational efficiency, patient
management, and health outcomes?
Ideally, these and related questions would be addressed in the context of a com-
prehensive research strategy with the following characteristics: (a) standardized
methodologies and metrics so that results augment the existing body of knowledge,
(b) operational definitions of basic functional processes that facilitate insights into
findings and provide a basis for research expectations, (c) a focus on generalizable
metrics such as cost shares, ratios, etc, as opposed to simple monetary values,
(d) guidelines for sensitivity impacts caused by changes in enabling technologies
and volume of use, (e) creation of a telemedicine technology-monitoring approach
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

624 BASHSHUR ■ REARDON ■ SHANNON

to identify trends and likely needs for analysis, and ( f ) establishment of a research
investment methodology with a return-on-investment approach to telemedicine
research.

ASSESSMENT OF TELEMEDICINE RESEARCH


Some of the effects and expectations of telemedicine are predictable and tend to

?
occur in the short term, whereas others are long term and more difficult to ascer-
tain. Nonetheless, much has been published in both the professional and popular
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

literature in this country and elsewhere concerning the potential of telemedicine.


A large portion of this literature deals with technological and clinical feasibility
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

issues as well as the effects of telemedicine on one or more objectives of health


care, namely increased access to care, cost control, and/or quality control. De-
spite the burgeoning professional literature that now exceeds 2600 journal articles
and reviews, many providers and policy makers are still uncertain as to the value,
suitable vision, or effective strategy for attaining the forecasted potential.
Ideally, the weight of the empirical research will affect determination of the
fate of telemedicine alternatives and the rate at which telemedicine becomes part
of mainstream health care. Although the links among research, policy, and oper-
ational programs may be neither direct nor predictable, inadequate research and
dissemination of findings will only increase the risk of ineffective or less bene-
ficial developments in the use of this technology. Telemedicine could become a
reasonable and balanced response to cost inflation and uneven quality and access,
as did the health maintenance organization a few decades ago. Or like health main-
tenance organizations and managed care, its promise could remain unfulfilled and,
in fact, become a source of discontent. Consequently, the task here is to identify
the critical objectives of telemedicine research, problems and impediments in cur-
rent research, telemedicine research strategies and methodologies, and the current
status of this research.
There is substantial health services research on determinants, metrics, and
methodologies for assessing programmatic effects on cost, access, and quality.
Each of these effects or outputs includes a distinct set of intermediate-impact
metrics, conceptual models, and methodological approaches that may be used for
analysis. Some of these have a distinct disciplinary focus, such as economics or so-
ciology, but often the researcher finds it productive to supplement these approaches
with comprehensive health services research conceptualizations.

ACCESSIBILITY
The geographic separation between provider and patient was the original impetus
for the development of telemedicine, and it continues to be a critical factor in its
evolution. Some proponents have suggested that telemedicine will eliminate the
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 625

problems related to the geography of medical care, and this attribute of telemedicine
is emphasized in both professional literature and mass media reports (32).
There is evidence that telemedicine enables a higher percentage of remote pa-
tients to be cared for while they remain in their local community (19). Therefore, it
is not surprising to read that a major contribution of telemedicine is the increased
access to medical care. However, this notion of access is limited in terms of defini-
tion and application. If the true impact of telemedicine on access is to be assessed
and evaluated, the concept of access must be more comprehensively considered

?
and applied.
Generally, access refers to an individual’s (or a group’s) ability to obtain needed
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

services (9). Although access certainly has geographic dimensions, it also has
financial, social, cultural, and psychological dimensions. Hence, a comprehensive
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

assessment of access to care must incorporate measures of all of these dimensions


(39). Perhaps because of the complexity of the task, most research on access to
medical care, including telemedicine, focuses on geographic separation. This type
of access is important and relatively easy to measure and assess objectively, and
actions can be taken to increase geographic accessibility. In this sense, access
becomes a function of the availability of health services. But even this notion is
more complex than it first appears. To a large degree, availability is a function
of the geographic distribution of health care resources. Thus, access also entails
the type, volume, and location or the supply of such resources compared with the
needs or demands of a given population. Access is also a function of the provider’s
willingness to serve patients.
Given the complexity of the concept, perhaps it is not too surprising that the
literature on the impact of telemedicine on accessibility is meager to date. What is
surprising, however, is that there have been relatively few objective and systematic
assessments of the impact of telemedicine on even the geographic component of
accessibility. It appears that the existence of “common knowledge” based on the
assumption of the positive impact of telemedicine on the geographic dimension
of accessibility to medical care for patients is sufficient. At a minimum, however,
assessments of telemedicine’s actual impact on geographic accessibility to medi-
cal care should be completed to provide input for comprehensive cost analyses of
telemedicine. Expansion of the assessments of the impact of telemedicine to the
other components of accessibility would contribute to a better understanding of
its potential role in the delivery of care. At the same time, more comprehensive
assessments of the role of geographic distance, financial costs, social class, culture,
and psychological factors related to the acceptance of telemedicine can contribute
to a more informed development and implementation of telemedicine.
Thus far in the discussion of accessibility, the impact of telemedicine on the
accessibility to medical care has been considered only from a patient perspec-
tive. It is perhaps equally important to consider the perspective of the provider.
This discussion is predicated on the current configuration of the vast majority of
telemedicine networks, that is, a limited set of remote telemedicine referring sites
(usually) linked to a single consulting station in a secondary-/tertiary-care health
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

626 BASHSHUR ■ REARDON ■ SHANNON

center. In the foreseeable future, telemedicine will become available at the desktop
computer for both providers and clients.
In the current configuration of telemedicine networks and systems, geographic
distance, financial costs, and social, cultural, and psychological factors affect
telemedicine’s impact on accessibility for all users, including clients/patients and
providers. In fact, to a large extent from the perspective of the remote provider, the
degree to which telemedicine is accepted and adopted depends on accessibility to
telemedicine. For example, there is considerable literature that suggests that, with

?
increasing (geographic) distance from a location, there is a decreasing likelihood
that an individual will have contact or interact with that location because of the
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

friction of space (43). Therefore, those remote providers at greater distance from
a telemedicine site will be disinclined to use it in their regular practice. For the
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

remote provider, geographic availability of telemedicine is a critical factor. With


increasing distance and associated travel time from the consulting site, there is a
corresponding increase in the potential loss of income due to time lost in travel.
Within the tertiary-care center, consulting specialists must also consider costs as-
sociated with “travel” to the telemedicine site provided for them.
Individual psychology and related attitudes also have an impact on providers’
accessibility to telemedicine. This psychology and these attitudes are related to
both the technological and social aspects of telemedicine innovation. In the former,
physicians must learn to use new technology, and, for them to adopt it, it must prove
to be at least as convenient and comfortable as the technology they have used previ-
ously. The technology also carries potentially increased threats or burdens in terms
of liability, licensure, and credentialing. Perhaps just as great an impediment to use
of telemedicine are the social aspects of the innovative technology. In place of the
traditional in-person examination and direct one-to-one contact, the patient is now
viewed (and the physician is viewed in turn) on a video screen. Electronically mon-
itored and transported telemetry and biometric measures replace direct observation
and examination. Lost to the consulting physician who uses telemedicine, at least
currently, are direct auscultation, olfaction, and palpation. Research is underway to
develop technology that can perform these functions electronically. Nonetheless,
this departure from the traditional practice of medicine would certainly affect the
providers’ access to and ability to use telemedicine.
Telecommunications and the potential of telemedicine have changed the char-
acter and scope of regional medical systems. Barriers imposed by the geographical
distribution of medical resources as well as the need to redistribute these resources
geographically can be transcended by telemedicine, thereby reducing the need for
physical access. However, greater access to sources of telemedicine is still needed.
Among >100 federally funded telemedicine networks, the major pattern for civil-
ian care is statewide or smaller regional networks with hierarchical hub-and-spoke
structures. These systems appear to have increased access to care to an additional
18 million persons in the United States, who are now “virtually” located within
40 miles of specialty services. These networks typically include more than three
hospitals. On the other hand, small networks of two to three hospitals tend to be
peer organizations at a great distance from each other (42).
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 627

COST
The rapid development and deployment of information technologies appear to be
redefining the cost and financial risk of telemedicine, along with several other vari-
ables. Overall, advances in digital medical applications and lower-cost information
technologies are improving the financial prospects for telemedicine. Yet, some of
the changes may also be redefining relationships in medicine and introducing new
important issues for telemedicine researchers. For example, the digitization and

?
loss of physical contact in the medical encounter is cause for concern among physi-
cians (48a). More conspicuous have been the concerns surrounding the Internet as
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

it has introduced more ubiquitous digital communication. By virtue of availability


and open, standardized access, the Internet is fostering a virtual community with
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

interactions in a medical care marketplace that is very different from traditional


medical care or classic telemedicine.
Although several factors, including its nature as a systemic innovation requiring
behavioral change, concern over reimbursement, and other regulatory issues, may
account for the slow adoption of telemedicine by mainstream medicine, the most
significant factor is telemedicine’s costs in relation to its benefits. Indeed, the
cost/benefit relationship is the most salient issue affecting both professional and
policy-maker reluctance to embrace telemedicine. If only the cost containment
potential of telemedicine is demonstrated to policy makers, the medical profession,
payers, and the general public, we will likely witness massive conversions to
telemedicine that would surpass the movement to health maintenance organizations
and managed care.
Decision making for telemedicine policy and planning requires cost research
that has the following essential attributes: (a) internal validity or scientific rigor,
(b) external validity or a reasonable basis to extrapolate from evidence to new
situations, and (c) conceptual boundaries on telemedicine cost impacts that are
consistent with health care resource allocation issues. These attributes provide a
solid basis for policy makers and planners to assess and apply research results,
as well as for researchers systematically and collectively to create an understand-
ing of telemedicine’s impact on resource allocation. Moreover, research designs
that embody these attributes must also contain a detailed evaluation methodology,
conceptualization of significant relationships between costs and benefits, appro-
priate measurement, and a clear definition of the parameters of the program under
investigation.
Of course, the value of cost studies in telemedicine is derived from the gains
in decision making that they support. Their findings are taken seriously when
they are based on sound methodology and theory. Much of the current cost lit-
erature in telemedicine, although providing useful information, contributes little
to decision-making confidence in cost findings. Cost findings are often ancillary
reports in studies of clinical and technological feasibility and effectiveness (27).
Often, cost findings are based on case studies, providing little conclusive insight but
perhaps questions for subsequent analyses. This type of cost study is too numerous
and idiosyncratic to consider in a summary assessment of telemedicine.
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

628 BASHSHUR ■ REARDON ■ SHANNON

COST ANALYSIS
Analysis of the costs of telemedicine has been constrained by the level of tech-
nology in use, certain methodological flaws, and various other strictures in imple-
mentation. Much of the data derives from short-lived or marginally supported pilot
projects, which are peripheral to the core institutional operations where they are im-
plemented. The fast pace of technological changes has made it difficult to estimate
true capital (fixed) costs or even operational (variable) costs. Moreover, analysts of-

?
ten report costs derived from dissociated operations based on an inadequate volume
of services. At the same time, cost outcomes have been bounded by the method-
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

ological and conceptual inclinations of researchers. Reports of telemedicine costs


often are derived from studies that focus on technological capabilities rather than
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

economics. These studies vary in the rigor of their research designs and usually
suffer from lack of an appropriate economic paradigm for cost assessment. In most
studies, the unit of analysis has been focused on the individual patient-provider
encounter as the core process that generates costs. The value of looking at the im-
pact of telemedicine on episodes of care is apparent in a study reported by Burgiss
and his associates (15), in which teledermatology cost savings were dependent on
changes in patterns of use. Still, partly because of a preoccupation with clinical
effectiveness, research is mostly absent on the potential cost impact beyond imme-
diate provider-patient interactions. Cost effects from changes in the organization
and content of a patient’s overall care, institutional transactions, and the medical
care marketplace have received little attention to date.
With few opportunities for experimentation in well-functioning programs, un-
derstanding telemedicine costs continues to be the most imposing and durable
challenge to the orderly development of telemedicine. Little useful cost informa-
tion from research is available to guide health policy and to mediate the role
of telemedicine in the health care system. These financial uncertainties make
providers, health care systems, and policy makers hesitant to pursue appropri-
ate telemedicine testing and research. The required economic research agenda
begins with a realistic assessment of existing knowledge regarding telemedicine
costs in the context of expected decision-maker informational needs. An efficient
strategy for cost research would identify what we need to know about telemedicine
costs, the use of such information, the specific methodologies to be used, and a
conceptualization of cost impact. Evidence on cost is also being provided by devel-
opments in medical care markets. Some applications that appear to offer promis-
ing prospects for cost studies, for example, teleradiology, are rapidly appearing in
market ventures. Telehome health care, for example, is moving forward based on
optimistic expectations of a sizeable market but without any research substantia-
tion on the character or size of this market. It appears that at least some of the fail-
ure of telemedicine research to establish policy-relevant evidence may be obviated
through consumer demand in the medical marketplace, but only private costs and
benefits of telemedicine for providers and consumers will become apparent with
commercial success. Medical care market issues associated with societal costs and
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 629

consumer demand inefficiencies will remain, and even gain significance. Unfor-
tunately, commercial and consumer forces may threaten an orderly telemedicine
development with a new technological imperative through the Internet. Moreover,
the issues of social equity and efficiencies in use of health care resources associated
with electronic media remain unresolved.
The emerging use of the Internet in health care suggests that, in contrast to earlier
generations of telemedicine, financial uncertainties and societal costs are likely to
increase with policy inaction, because the technology has assumed a life of its own

?
through consumer and commercial demands. As Internet technologies mature, new
relationships among consumers, providers, suppliers, and telemedicine technology
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

will loom larger in medical care and health policy. The most potent protection from
social cost and development uncertainties will be in establishing well-defined
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

and efficient research programs. Analyses of costs, especially the second- and
third-order system costs, may help guide telemedicine development as part of the
evolving technological transformation.
The basic economic analytical framework for cost analysis is to evaluate the
health-related outcome of telemedicine relative to the value of required resources—
defined as costs—as compared with the alternatives. A variety of analytic tools
are used for this assessment, most notably cost-effectiveness analysis (CEA) and
cost-benefit analysis (CBA). Typically, CEA/CBA defines the dimensions of the
resource allocation decisions. It is most easily applied when we have exogenous,
well-defined alternatives, such as drugs or specified clinical interventions. How-
ever, this methodology should be considered an extension of, rather than an alter-
native to, other methodologies for obtaining valid inference. Impact assessment
methods help gauge the valid effect of a program, intervention, or policy on the
extent of a desired change in achieving an objective, and CBA/CEA provides the
conceptual framework for measurements and interpretation. Although discussed
elsewhere in this volume, a few basic remarks about CBA/CEA are in order here.
CBA/CEA attempts to measure, in concrete, often monetary terms, the tradeoff
between resource use and benefits. Its proper application requires adherence to an
extensive set of concepts, metrics, and processes consistent with a decision-making
model. Partly, this includes defining the parameters of the telemedicine program
and its objectives to clarify for users whether the result will apply to specific real-
world situations. The methodology helps establish and communicate valid and
explicit economic criteria and results in decision making. Accordingly, its value is
greatly enhanced when researchers use standard methodologies of CBA/CEA in
the research (26).
Several recent articles in the telemedicine literature provide overviews of CBA/
CEA issues, terminology, and specific considerations in cost assessment (35, 38).
These include discussions of alternative perspectives in decision making (e.g. cost
and benefit impacts on society and the individual), the meaning of costs, such
as opportunity costs, and methods of measurement. Several authors have pointed
out general cost and benefit types and the need to incorporate secondary effects
and their ramifications, but defining specific detail for telemedicine requires both
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

630 BASHSHUR ■ REARDON ■ SHANNON

further research and conceptualization of the issues. For example, providers will
face new transaction and coordination costs. Clients may change patterns of use as
a result of a change in access costs and changes in the production of health care.
However, telemedicine research has been limited by its bounded conceptualization
as primarily having value in changing spatial/temporal functions. As Berki (12)
indicated earlier, “telemedicine . . . could well restructure the entire medical care
system,” but to do so would require a full understanding of its performance as a
production process and the processes for producing outputs as well as changes

?
in the outputs themselves. For instance, telemedicine might alter the traditional
relationships between specialists and primary-care physicians, perhaps changing
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

the role of the consultant specialist from one of direct contact with patients to one
of service to primary-care providers. Indeed, telemedicine may alter the provider
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

mix and intensity of care during an episode of illness.


Few of the reviewed cost studies provide sufficient cost conceptualization or
reach the usual level of rigor for empirical inference. However, they establish an
important foundation of approaches, measures, and insights for future studies.
The sample populations of these studies are small. Nonetheless, analytic rigor
is introduced by providing comparative nontelemedicine alternatives, developing
metrics, and using sensitivity analysis to examine results.
Consistent patterns of cost savings from telemedicine are provided by sev-
eral studies of telemedicine in correctional institutions (37, 48) and teleradiology
(11, 21, 44). Among the more notable findings in the telemedicine cost literature are
those of apparent substitutions in resource utilization, seemingly because of knowl-
edge transfer from the specialist to the primary-care provider (11, 48). Some cost
savings appear to result from introducing new information through telemedicine.
However, programs that are defined by their capability to collapse distance and
time currently provide the strongest results for telemedicine’s impact. When pop-
ulations are locationally confined, such as in prisons (37), ships (45), and homes,
or when time to treatment is critical, as in emergencies (44), telemedicine often
appears to be cost effective.
The conclusions of these studies generally need further corroboration, but they
may be summarized as follows. The cost savings of telemedicine compared with
traditional alternatives depend on transportation costs, volume, time sensitivity
of care, and the cost of the alternative. The transportation costs of alternatives
to telemedicine for prisoners provide a substantial offset to telemedicine tech-
nology costs and low volumes in correctional settings. The transportation effect
shows up as an important determinant of cost savings when lack of an emergency
service places patients at risk for high-priced alternatives (17) or when significant
patient dispersion and distance require physician travel as the alternative (24). In
teleradiology, travel costs for both the patient and the radiologist often enter into
the cost equation, and patient transportation expenses for radiology constitute a
significant determinant of savings when associated with emergency care (44). A
variety of studies of telemedicine specialty applications demonstrate the impor-
tance of transportation and emergency care in determining costs, including studies
of teleneurology, teledermatology, and teleoncology (15, 17, 24).
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 631

Despite reported findings, conclusions still need to be tempered by the lack of


rigorous methods of inference. Confidence in each of these studies varies with the
type of cost comparison used in the analysis. Most of the studies rely on historical
patient data, data from concurrent programs of care for comparison groups, or the
projected costs if telemedicine patients were to receive traditional care. Few studies
assess the threats to the validity of analytic conclusions that are associated with
the comparison group. Nonetheless, these conclusions are associated with factors
that might lead the analyst to infer a causal relationship between telemedicine and

?
costs, when the true cause is unobserved. For example, the normal expectation is
for self-selection of patients into alternative-care settings depending on severity
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

of illness or other preferences. Selection may be difficult to eliminate in a study,


but discussion of the possible biases it introduces, such as the effect of a sicker
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

population moving to the traditional program, would help in interpreting such


results. In addition, few if any telemedicine cost studies consider the statistical
properties of their observations. Thus, the cost savings thought to be created by
telemedicine might be a result of random variation in other variables that affect
costs. In effect, we really don’t know whether these studies show lower costs
caused by variations in patient behavior or by telemedicine.

QUALITY

As with access and cost, the concept of quality is multidimensional and complex,
and it has been narrowly defined in telemedicine research. Quality may be measured
in terms of structural, process, and outcome indicators (23). However, greater
attention is now placed on health outcomes as a basis for policy determination
and program support. Donabedian (23) identified two basic aspects of quality,
technical and interpersonal. The latter refers to caring, as well as client and provider
satisfaction with care, whereas the former refers to the process and outcome of care.
Much of the published research to date on the interpersonal quality of tele-
medicine services is based on the satisfaction of clients and providers and on the
process of care for technical quality. The scope and generality of this research have
been severely constrained by inadequate research designs, samples, measurement,
and analysis. Prospective randomized clinical trials are rare, if not nonexistent.
Utilization volumes have not permitted adequate statistical controls for effective
quasi-experimental designs. Only indirect measures of quality have been used, and
analysis of telemedicine performance has not been conducted under optimal con-
ditions for testing hypotheses. Ideally, information on telemedicine performance
would be gathered after these systems have achieved a steady state of operation,
although not necessarily at peak levels. Moreover, the effects of the system on
quality of care can be detected only after a reasonable level of maturation, which
represents the time needed for the effects to become manifest. This is especially
significant when attributing delayed health outcomes to telemedicine.
There is strong and consistent evidence of general satisfaction with telemedicine
among both providers and clients, regardless of whether satisfaction is measured
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

632 BASHSHUR ■ REARDON ■ SHANNON

in attitudinal or behavioral terms, and this evidence is consistent with typical


findings from most studies of satisfaction with health care (6, 14). There have been
no strong indications of feelings of discomfort, concerns about potential breach of
confidentiality, or the impersonality of the medium on the part of patients, as was
feared. One study reported that some oncology patients preferred not to repeat the
telemedicine visit (3).
Proponents of telemedicine have suggested that innovative information tech-
nology, remote sensing, and computers can be used effectively to extend the pro-

?
ductive capacity and distributive efficiency of available health care resources (8).
Yet, research on the structural effects of telemedicine has yet to take form.
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

The preponderance of research on quality of care in telemedicine has focused


on diagnostic accuracy, precision, specificity, and reliability of information gar-
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

nered by telemedicine vs in-person situations. In an exhaustive review of the


literature, Grigsby et al (27) concluded that it is not possible “to assess the util-
ity of telemedicine vis-a-vis conventional care” because of the lack of “large-
scale, cross-cutting evaluations.” That is to say, the studies of quality of care in
telemedicine, although numerous, are limited to categorical clinical interventions
or diagnostic procedures. In each of these studies, the accuracy of clinical or diag-
nostic information rendered by telemedicine was compared with that obtained in
person, as the gold standard. Findings from this research support two conclusions:
(a) telemedicine delivers adequate information for a majority of categorical clin-
ical and diagnostic procedures, and (b) specific instances of failure in diagnostic
accuracy were fully attributable to the specific technology that was used, such as
the lack of color in dermatological images (39).

SUMMARY AND CONCLUDING REMARKS


Telehealth generally and telemedicine specifically use increasingly sophisticated
electronic communications, advanced computing, and telematics technology for
the exchange of medical and health information. The concept of telemedicine em-
bodies major transformations occurring simultaneously in medical care and in in-
formation technology. Technological advances provide enhanced data acquisition,
manipulation, transmission, and storage. However, the application of these tech-
nologies is not occurring in a vacuum. In telemedicine particularly, they represent
a complex and radical change in the traditional practice of clinical diagnosis and
treatment. In diagnostic areas such as radiology and pathology, telemedicine is
making substantial inroads. National guidelines and standards have been estab-
lished, and reimbursement is available, owing in large part to the strictly techno-
logical aspect of the information transferred and, equally, the absence of the need
for human interaction. In the clinical setting, the situation is made more complex.
Telemedicine replaces the traditional face-to-face, personal encounter between the
patient and physician. Therefore, in the latter instance, telemedicine is not only a
technological innovation but a sociocultural one as well.
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 633

The preponderance of telemedicine research to date has focused on the tech-


nological aspects of the innovation, its quality, and cost. Yet, there is insufficient
evidence on which to base program development strategies as well as overall health
policy related to telemedicine. The research thus far has been retrospective, and
it may be necessary to incorporate prospective research designs and simulation
to overcome some of the limitations of and deficiencies in the current research
effort. In this regard, research pertaining to the potential of telemedicine and its
impact should be developed against a framework that uses the matrix of effects on

?
accessibility, cost, and quality for patients/clients, providers, and society. More-
over, the research framework must recognize the hierarchical nature of electronic
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

telemedicine networks, from the home, institution, and other remote sites through
secondary health facilities to the tertiary medical center as the point(s) of ultimate
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

consultation and referral. Relatively little research has been directed toward the
nature of electronic or “virtual” regionalization of medical care delivery and/or the
integration of virtual and traditional medical care regions.
The initial impetus for telemedicine was derived from recognition of the geo-
graphical imbalance of the distribution of health services and the lack of equitable
accessibility to medical care for certain segments of the population. In particular,
people living in medically underserved rural and inner-city areas as well as those
confined populations in long-term care institutions were targeted for the develop-
ment of telemedicine. The goal was to increase the accessibility to medical care
among these disadvantaged populations. The ability of telemedicine to transcend
geography and to alleviate problems associated with the maldistribution of medical
care is emphasized in both the professional literature and mass media. Evidence
has accumulated that telemedicine reduces the need for travel by patients, allowing
them to be treated in their local community. However, research on even this most
basic aspect of telemedicine is limited. Systematic and comprehensive research on
other dimensions of accessibility, including its social, cultural, and psychological
dimensions, is sorely lacking, and it is this research that is necessary if the impact
of telemedicine is to be understood and included in plans for its development,
implementation, and acceptance. Accessibility must be assessed from not only the
patient/client’s perspective but also the provider’s perspective.
The issue of cost, another factor in the accessibility equation, can be considered
the most salient issue pertaining to acceptance of telemedicine by medical health
care professionals, administrators, and policy makers. Thus far, telemedicine’s
potential to contain the costs of medical care from the providers’ perspective has
not been demonstrated adequately. It is interesting that, as telemedicine receives
increased attention as a means of cost containment and even reduction, greater
emphasis is being placed on its costs and financial benefits than on the issue of
accessibility.
The emphasis on costs and benefits in telemedicine is receiving increased
scrutiny owing to the ever rising costs of medical care, shifting patterns of care from
the hospital to dispersed care networks, and expected growth in medical care de-
mands by an aging population usually with limitations in mobility. Improvements
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

634 BASHSHUR ■ REARDON ■ SHANNON

in information and telecommunications technology suggest a potential for tele-


medicine in enhancing access to quality care. Still the lure of hypothetical gains
has most often faded when operational programs have demonstrated little pay-off.
Developmental missteps in project implementation have plagued many plans for
telemedicine, including slow adoption by mainstream medicine, which has been
resistant to behavioral change, and concerns about clinical outcomes. The most
significant deterrent to the acceptance of telemedicine may be usability of systems
and reimbursement policies that impede patient volume. Ironically, the outcome

?
has been high costs, limited data for cost analyses, and a hesitation by policy
makers and care providers to embrace telemedicine.
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

Much of the cost-benefit research in telemedicine is derived from low-volume,


immature, short-lived, and marginally or federally supported projects, many of
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

them experimental in nature. Therefore, there is a dearth of reliable information


on the costs and benefits of telemedicine that might serve as an adequate basis for
informed and, therefore, effective policy development. It is important to establish
a new economic research agenda in telemedicine based on sound economic theory
and methodology.
Although generally there is supportive evidence of acceptance of and satisfac-
tion with telemedicine by clients and providers, technical quality has been assessed
primarily in terms of concordance in diagnostic accuracy between telemedicine
and in-person care. Unfortunately, the generality of these findings is limited to the
specific diagnostic procedures and clinical interventions cited. Missing from this
research is any assessment of structural effects of telemedicine.
Thus far, efforts to evaluate the various accessibility, cost, and quality dimen-
sions of telemedicine continue with limited success. At the same time or perhaps as
a result, the dialogue on telemedicine policy, developmental strategies, and research
appears to be fragmented. Discussions of definitions persist as new applications of
electronic communications are discovered. Although the number of published pa-
pers on telemedicine increases, there remains a paucity of definitive accessibility,
cost-benefit, and quality information based on mature, operational, and fully func-
tional telemedicine applications. Analytic and research efforts have been hampered
by rapidly changing telemedicine technology, inadequate volumes of medical care
episodes for study, usability of systems, and incomplete and fragile organizational
and technological infrastructures. In terms of high-end telemedicine testbeds, fail-
ures have been attributed to the inability to use the most efficient technological
infrastructure already available. As a result, most clinical telemedicine programs
continue to provide more promise than pay-off.
It is not clear if and when these deficiencies in telemedicine research will
be remedied. In this review, we have attempted to identify some of the major
problems related to telemedicine research and to suggest a framework and strategy
for future research efforts. There is no guarantee that the framework proposed and
guidelines provided here will ensure that the necessary research will be initiated and
completed. However, it is almost certain that, without an established framework
for research and evaluation, such as that proposed here, it is very unlikely that
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 635

telemedicine will fulfill its considerable promise and potential in the areas of
improving accessibility, containing costs, and providing high quality care.
Telemedicine may still be medicine at a distance, but the prominence, expec-
tations, technology, and range of applications have changed it considerably. It
started as a telecommunications augmentation to medical care, and it evolved into
an integrative process of information technology and health care. It appears to rep-
resent the functional set of activities that are expected to redefine future medical
care institutions—patient-centric care with virtual health centers, hospitals, and

?
long-term care institutions. Whether this innovation will be the millennial land-
mark change in health care delivery, similar to the development of the modern
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

hospital a century ago, or a set of footnotes representing only technological alter-


natives for the near future depends on well-guided research, prudent policy, and
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

the development of enabling technologies.

Visit the Annual Reviews home page at www.AnnualReviews.org

LITERATURE CITED
1. Adams LN, Grigsby K. 1995. The Georgia 9. Bashshur RL, Homan D, Smith D. 1994.
state telemedicine program: initiation, de- Beyond the uninsured: problems in access
sign, and plans. Telemed. J. 1(3):227–35 to care. Med. Care. 32(5):409–19
2. Alessi N, Huang M, Quinlan P. 2000. The 10. Bennet AM, Rappaport WH, Skinner EL.
Internet as a source of information for dis- 1978. Telehealth Handbook. PHS Publ.
ease management: focus on psychiatry. Int. 79-3210, US Dep. Health Educ. Welf.,
J. Dis. Manage. In press Bethesda, MD
3. Allen A, Hayes J. 1995. Patient satisfaction 11. Bergmo, TS. 1999. Economic analyses of
with teleoncology: a pilot study. Telemed. J. telemedicine. In European Telemedicine
1(1):41–46 1988/1989, pp. 30–32. London: Kensing-
4. American Telemedicine Association. 1999. ton Publ.
Internet health and medical web sites. 12. Berki SE. 1975. Telemedicine: some eco-
Am. Telemed. Assoc. Backgr. Pap. August. nomic implications. In Telemedicine: Ex-
http://www.atmeda.org plorations in the Use of Telecommuni-
5. Bashshur RL. 1975. Telemedicine and med- cations in Healthcare. ed. RL Bashshur,
ical care. In Telemedicine: Explorations PA Armstrong, ZI Youssef, pp. 175–91.
in the Use of Telecommunications in Springfield, IL: Thomas. 356 pp.
Healthcare, ed. RL Bashshur, PA Arm- 13. Bird KT. 1971. Teleconsultation: a new
strong, ZI Youssef. Springfield, IL: Thomas. health information exchange system. Third
356 pp. Annu. Rep. Veterans Admin., Washington,
6. Bashshur RL. 1978. Public acceptance of DC
telemedicine in a rural community. Biosci. 14. Brick JE, Bashshur RL, Brick JF,
Commun. 4(1):17–38 D’Alessandri RM. 1997. Public knowl-
7. Bashshur RL. 1995. On the definition and edge, perception, and expressed choice
evaluation of telemedicine. Telemed. J. of telemedicine in rural West Virginia.
1(1):19–309 Telemed. J. 3(2):159–72
8. Bashshur RL. 1997. Critical issues in 15. Burgiss, SG, Julius CE, Watson HW,
telemedicine. Telemed. J. 3(2):113–26 Haynes BK, Buonocore E, et al. 1997.
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

636 BASHSHUR ■ REARDON ■ SHANNON

Telemedicine in dermatology care in rural 30. Hassol A, Irvin C, Gaumer G, Pushkin D,


patients. Telemed. J. 3(3):227–34 Mintzer C, et al. 1997. Rural applications
16. Cameron AE, Bashshur RL, Kalbritter K, of telemedicine. Telemed. J. 3(3):215–25
Johnson EM, Cameron JW. 1998. Simula- 31. Henson DE. Cancer and the Internet. Can-
tion methodology for estimating financial cer 86(3):373–90
effects of telemedicine in West Virginia. 31a. Joch A. 1999. All aboard. Healthcare In-
Telemed. J. 4(2):125–44 formatics 16(9):43–53
17. Chodroff PH. 1999. A three-year review 32. Johnson E, Cameron A, Debold V, Sch-

?
of telemedicine at the community level- off J. 1996. Telemedicine: an annotated
clinical and fiscal results. J. Telemed. Tele- bibliography, I. Telemed. J. 1(2):155–65
care. 5(Suppl. 1):1–28 33. Justice JW, Decker PG. 1979. Tele-
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

18. Chronaki C, Tsiknakis M, Orphanoudakis medicine in a rural health delivery system.


S. 1997. Virtual workspaces in I2Cnet. Int. Adv. Biomed. Eng. 7:101–71
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

J. Med. Inf. 47:115–19 34. Kantor M, Irving L. 1997. Telemedicine:


19. Chuang D. 1997. Client acceptance of Report to Congress. US Dep. Commer.,
telemedicine. PhD thesis. Univ. Mich., Ann Washington, DC
Arbor, MI. 138 pp. 35. Lobley D. 1997. The economics of
20. Dakins D. 1997. Market targets. Telemed. telemedicine. J. Telemed. Telecare. 3(3):
Telehealth Netw. 3:25–29 117–25
21. Davis MC. 1997. Teleradiology in ru- 36. Marsh A. 1998. The creation of a global
ral imaging centres. J. Telemed. Telecare. telemedical information society. Int. J.
3(3):146–53 Med. Informatics 49(2):173–93
22. DeBakey ME. 1995. Telemedicine has now 37. McCue MJ, Mazmanian PE, Hampton C,
come of age. Telemed. J. 1(1):3–4 Marks TK, Fisher EJ, et al. 1998. Cost-
23. Donabedian A. 1980. Explorations in minimization analysis: a follow-up study
Quality Assessment and Monitoring, Vol I: of a telemedicine program. Telemed. J.
The Definition of Quality and Approaches 4(4):323–28
to Its Assessment. Ann Arbor: Health Adm. 38. McIntosh E, Cairns J. 1997. A framework
Press. 163 pp. for the economic evaluation of telemedi-
24. Doolittle GC, Harmon A, Williams A, cine. J. Telemed. Telecare 3(3):132–39
Allen A, Boysen CD, et al. 1998. A cost 39. Penchansky R, Thomas J. 1981. The con-
analysis of a tele-oncology practice. J. cept of access: definition and relation to
Telemed. Telecare. 3(Suppl. 1):20–22 consumer satisfaction. Med. Care 19(2):
25. Ferrer-Roca O, Sosa-Iudicissa M, eds. 127–40
1998. Handbook of Telemedicine. Wash- 40. Deleted in press
ington DC: IOS Press. 295 pp. 41. Picot J. 1998. Telemedicine and telehealth
26. Gold MR, ed. 1996. Cost-Effectiveness in in Canada: forty years of change in the
Health and Medicine. New York: Oxford use of information and communication
Univ. Press. 425 pp. technologies in a publicly administered
27. Grigsby J, Kaehny M, Sandberg E, healthcare system. Telemed. J. 4(3):199–
Schlenker R, Shaughnessy P. 1995. Effects 205
and effectiveness of telemedicine. Health 42. Reardon T. 1997. Assessing the effective-
Care Financ. Rev. 17(1):115–31 ness of telemedicine: results from the fed-
28. Deleted in press eral telemedicine gateway. Presented at
29. Hassinger EW. 1982. Rural Health Organi- HII 97: The Emerging Health Information
zation: Social Networks and Regionaliza- Intrastructure Conf., April 1997, Wash-
tion. Ames, IA: Iowa State Univ. Press ington, DC
P1: FQK/FPO P2: FLW/FFO P3: FCT
March 31, 2000 10:30 AR-100 CHAP-25

TELEMEDICINE 637

43. Shannon G, Bashshur R, Lovett J. 1986. 46. Willemain TR, Mark RG. 1971. Models
Distance and the use of mental health ser- of healthcare systems. Biomed. Sci. In-
vices. Milbank Mem. Fund Q. II 64(2): strum. 8:9–17
302–30 47. Wittson CL, Benschoter R. 1972. Two-
44. Stoeger A, Strohmayr W, Giacomuzzi SM, way television: helping the medical center
Dessl A, Buchberger W, et al. 1997. A reach out. Am. J. Psychiatr. 129(5):624–
cost analysis of an emergency computer- 27
ized tomography teleradiology system. J. 48. Zincone LH, Doty E, Balch DC. 1997.

?
Telemed. Telecare. 3(1):34–39 Financial analysis of telemedicine in a
45. Stoloff PH, Garcia FE, Thomason JE, prison system. Telemed. J. 3(4)247–56
Shia DS. 1998. A cost-effectiveness analy- 48a. Zyger A. 1999. Are doctors losing touch
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

sis of shipboard telemedicine. Telemed. J. with hands-on medicine? N.Y. Times,


4(4):293–304 USA. July 13:F1
Access provided by 160.152.162.233 on 08/16/21. For personal use only.
Annual Review of Public Health
Volume 21, 2000

CONTENTS
PUBLIC HEALTH GENETICS: An Emerging Interdisciplinary Field for
the Post-Genomic Era, Gilbert S. Omenn 1
HOST-PATHOGEN INTERACTIONS IN EMERGING AND RE-
EMERGING INFECTIOUS DISEASES: A Genomic Perspective of
Tuberculosis, Malaria, Human Immunodeficiency Virus Infection,
Hepatitis B, and Cholera, Janet M. McNicholl, Marie V. Downer,
Venkatachalam Udhayakumar, Chester A. Alper, David L. Swerdlow
15
NUTRITION, GENETICS, AND RISKS OF CANCER, Cheryl L. Rock,
Johanna W. Lampe, Ruth E. Patterson 47
POPULATION SCREENING IN HEREDITARY
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org

HEMOCHROMATOSIS, Arno G. Motulsky, Ernest Beutler 65


THE INTERFACE OF GENETICS AND PUBLIC HEALTH: Research
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

and Educational Challenges, Melissa A. Austin, Patricia A. Peyser, Muin


J. Khoury 81
LOOKING BACK ON ""CAUSAL THINKING IN THE HEALTH
SCIENCES, J. S. Kaufman, C. Poole 101
CAUSAL EFFECTS IN CLINICAL AND EPIDEMIOLOGICAL
STUDIES VIA POTENTIAL OUTCOMES: Concepts and Analytical
Approaches, Roderick J. Little, Donald B. Rubin 121
BUILDING BRIDGES BETWEEN POPULATIONS AND SAMPLES
IN EPIDEMIOLOGICAL STUDIES, W. Kalsbeek, G. Heiss 147
MULTILEVEL ANALYSIS IN PUBLIC HEALTH RESEARCH, Ana V.
Diez-Roux 171
SHOULD WE USE A CASE-CROSSOVER DESIGN, M. Maclure, and
M. A. Mittleman 193
WATER RECLAMATION AND UNRESTRICTED NONPOTABLE
REUSE: A New Tool in Urban Water Management, Daniel A. Okun 223
EPIDEMIOLOGY AND PREVENTION OF INJURIES AMONG
ADOLESCENT WORKERS IN THE UNITED STATES, Carol W.
Runyan, Ronda C. Zakocs 247
THE EFFECTS OF CHANGING WEATHER ON PUBLIC HEALTH,
Jonathan A. Patz, David Engelberg, John Last 271
TOXICOLOGICAL BASES FOR THE SETTING OF HEALTH-
RELATED AIR POLLUTION STANDARDS, M. Lippmann, R. B.
Schlesinger 309
RELIGION AND HEALTH: Public Health Research and Practice, Linda
M. Chatters 335
A REVIEW OF COLLABORATIVE PARTNERSHIPS AS A
STRATEGY FOR IMPROVING COMMUNITY HEALTH, Stergios
Tsai Roussos, Stephen B. Fawcett 369
ORAL HEALTH IN THE UNITED STATES: The Post-Fluoride
Generation, P. Milgrom, S. Reisine 403
THE NEW PUBLIC HEALTH LITIGATION, W. E. Parmet, R. A.
Daynard 437
BABY AND THE BRAIN: Advances in Child Development, Janet A.
DiPietro 455
HEALTH PROMOTION IN THE CITY: A Review of Current Practice
and Future Prospects in the United States, N. Freudenberg 473
THE RISE AND DECLINE OF HOMICIDE- AND WHY, Alfred
Blumstein, Frederick P. Rivara, Richard Rosenfeld 505
NCOME INEQUALITY AND HEALTH: What Does the Literature Tell
Us?, Adam Wagstaff, Eddy van Doorslaer 543
EVALUATING THE STATE CHILDREN''S HEALTH INSURANCE
PROGRAM: Critical Considerations, Barbara Starfield 569
PREFERENCE-BASED MEASURES IN ECONOMIC EVALUATION
IN HEALTH CARE, Peter J. Neumann, Sue J. Goldie, Milton C.
Weinstein 587
TELEMEDICINE: A New Health Care Delivery System, Rashid L.
Bashshur, Timothy G. Reardon, Gary W. Shannon 613
THE CHANGING NATURE OF RURAL HEALTH CARE, Thomas C.
Ricketts 639
ASSESSMENT IN LONG-TERM CARE, R. L. Kane, R. A. Kane 659
Annu. Rev. Public Health 2000.21:613-637. Downloaded from www.annualreviews.org
Access provided by 160.152.162.233 on 08/16/21. For personal use only.

You might also like