OVERVIEW

CHAPTER 1

UsEs oF MULTTMEDTA
INFORMATION

fohn F. Koegel Buford

U niversity of M assachusetts Lowell

1.1 rNTRoDUcrroN

A complex weave of communications, electronics, and computer technolo-

gies is emerging to create a new multimedia fabric for the next decade. The
nature of this cloth is still evolving as an assortment of industries-telecom-
munications, consumer electronics, computers, cable and broadcast televi-
sion, and information providers---compete for the emerging market. The
potentially biggest near-term market will be the home and consumer, as
recent developments such as consumer-oriented computer products, inter-
active television, and video-on-demand suggest. However, in time many
expect the new technologies to be as pervasive as today's television and
telephone and the impact to reach to the sum of computers, telecommu-
nications, and electronics, touching all parts of society from industry to
government, education to recreation.
The widespread interest in multimedia is largely based on the perception
that computer control of digital multirnedia forrns provides a low-cost easy
accessibility to rnultimedia infonnation that so far has been lirnited to
specialists and production houses with million dollar equiprnent budgets.
If the computer-integrated version of rnultirnedia can be made as approach-
 U5E5 OF MULTIMEDIA INFORMATION

able as today's word processor, then digital multimedia technology be-

comes the Gutenberg press for visual forms of communication media.
At the same time, the convergence of rnultimedia technology with the
telecommunications industry is creating a scenario with similar impact. If
the features of the telephone and television are combined, the resulting
visually enriched communication makes applications such as home shop-
ping, distance learning, remote collaboration with specialists, and interac-
tive access to live and stored video sources around the world possible.
These two technology revolutions are underway and are synergistic. The
resulting development and realignment of industries, produc{;, and infra-
structure is creating many technical chalienges, which are compounded by
the scope and interdisciplinary aspects of multimedia information process-
ing. The design of new multinrcdia systerns to support these applications will
lead to changes in the architecture of today's computers, networks, operat-
ing systems, user interfaces, and public communications systems. Because
of the scope of these changes and the expected pervasiveness of the
technology, the full impact is yet to be determined. The remainder of this
chapter addresses this issue by examining some of the applications that
have been demonstrated so far.

1.2 wHAT rs MUTTTMEDIA?

"Multirnediu: Arr opplicatiorr requiring rnore tharr two trips to tlrc cor fut operote."
t rtcroctive Multi rned i a Ass ocioti or r
-l
The concepts behind what is emerging today date back to over four decades
to a series of visionary thinkers who foresaw the evolution of computers
towards richer personalized devices that would become an extension of the
individual. In 1945 Vannevar Bush, then the Director of the Office of
Scientific Research and Development in the U.S. government, suggested [1]
that one of the future devices available for individuals would be a memex,
"a device in which one stores all his books, records, and communications,
and which is mechanized so that it can be consulted with exceeding speed
and flexibility. It is an enlarged intimate supplement to his memory." The
memex would additionally be an associative device, so that related items
could be easily located.
'loday, the iinking of associated data for easy access is called hypertext, a
term coined by Ted Nelson l2l, or hypermedia, when any type of media
form can be linked. It is the simultaneous use of data in different media
forms (voice, video, text, animations, etc.) that is called multimedia. Digital
video and auc'lio media are the most demanding of the new rnedia that are
being added to the repertoire of computing and communications systems.
1.3 TARLY HYPERTEXT AND COLLABORATIVE RESEARCH

Because of their time-sampled nature, these types of media are frequently

referredto as continuous media (CM). The term multimedia computing
commonly refers to the use of multimedia data types in computer applica-
tions and systems, and multimediii communications denotes communica-
tions systems which support the real-time transmission of continuous
media.

I.3 EARLy HypERTExr AND coLLABoRATTvE REsEARcH

The idea of lihking related information gave impetus to Ted Nelson and
Doug Englebart, who separately conceived and developed the first comput-
erized versions of hypertext-style editing systems. In 1968 Englebart dem-
onstrated the NLS system developed at SRI [3,41, which had interactive
multiperson editing, branching to different files, text search facilities, and
outline processing. In the late 1960s, Ted Nelson and Andries van Dam
collaborated at Brown University to develop a hypertext editing system on
an IBM 360 [4,5]. A fourth-generation system developed at Brown, I'nter-
medid, has continued this research and includes animation and video tools.

1.4 uulrrMEDrA AND pERsoNAuzED coMpurrNc

ln 7967 , Nicholas Negroponte formed the Architecture Machine Group in
the $rchitecture Department at MIT. Although the initial goal was to use
computers for architectural design [6], a new focus developed: that .of
making computers easier to use.

1.4.1 Spatial Data Management System

In early 7976, the Architecture Machine Group proposed a research pro-
gram to the U.S. Defense Advanced Research Projects Agency (DARPA)
entitled "Augmentation of Human Resources in Comrnalrd and Control
through Multiple Media Man-Machine lnteraction." A basic precept of the
project would be the use of spatial cues to aid in task perforrnance and
memory. Information organization would be tied to locality, and the user
interface would include a large wall-size display and an octaphonic sound
system to provide spatial audio cues. The user would access a data item by
traveling to it in a virtual space as opposed to referencing it by narne. Objects
in the space could be text, graphics, videos, or active procedures.
The Spatial Data Management System (SDMS) was a major step toward
moving the computer interface away from the conventional video display
terminal to one which was close to human perceptual space. The SDMS
 USES OF MULTIMEDIA INFORMATION

Figure 'l .1 SOVS rledia room with it'rstrurnerrtecl Eatues chair,

large screen clisplal', ancl sicle touch screerl lllollitors (Reprocluced
courtesy of the MIT N,{eclia Laboratory, O MII' Mecjia i,aboratory
1 9 7ri)

media room contained an instrumented Eames chair, large proiection

screen, and side view video screens (Figure 1.1). While seated in the chair,
the user coulci use ioysticks, a touch screen, or stylus. These controls were
used to navigate through the information space viewecl on the large screen
(Figure 1"2), which acted as a window to the data space. one of the side
view monitors provided a continuotts top-level view of the information
landscape.
The first versiolt of the system organized the space hierarchically, with
lower levels reachable via ports. SDMS II used a single global space and
zooming could be used to inspect an obiect in greater detail. Wl-rile navi-
gating tire space, auclitory cues provided a sense of direction and distance
irottl in oblect. later, voice-based navigation and control were aclded to the
systern [7].

1.4.2 Movie Maps and Surrogate Travel

In the late 1970s, the theme of spatially organized data continued with the
work of researcher Andy Lippman [8] and doctoral stuclent Robert Mohl [91.
In what catne to be known as the Aspen proiect, filrn sl'tctts taken frotn a
moving veh icle traveling through the town of Aspe n, Colorado, were stored
 1.4 MULTIMEDIA AND PERSQNAL|ZED COMPUTINC

Flgure tr.2a Top-level vieu,' of DaiaL.and iri StllvlS (llcprodLrced

courtesy of the MIl- lv{eclia Laboratory, O L{lT Meclia I-airoratory
1978)

Flgure l .2b snr,ls rr.nitor rri.-,riace sh'w rrg tools avail-

''ri.rrs
able (Reproclucc(l courtrsv of tlte N{l'l NIecli,t lrbonrtorv-, O iUIT
Media l.aborator,v 1 97tt)
 USES OF MULTIMEDIA INFORMATION

Flgure 1.3a Surrogate travel ttsing movie maps san.rple irlterface

frorn the MIT Aspen proiect (Reproduced courtesy of the MIT Media
Laboratory, O MIT Media Laboratory 197tt)

on videodiscs. Tfiese were then accessed interactively to simulate driving

through the town. A touch screen or ioystick interface allowed the user to
control the speed and direction of travel (Figure 1.3).
Irnages of the facade of significant buildings in the town were stored on
videodisc as well. The driver could stop at any of these buildings and access
data associated with the building. For example, the town records were
accessed from the town hall.
Mohl's work added a simultaneous aerial view of the town. Different
techniques for zooming, which maintained constant position of the target,
were triecl. Aerial views could be annotated to include landrnarks of places
of interest or to identify where the user had traveled. Experiments with users
indicated that spatial context is a significant aid to memory; users of the
system upon a later visit to the town reported cornplete comfortability in
fincling their way. Further, the use of spaiial reference points as a contextual
aid for data navigation could apply to any domain with a spatial topogra-
phy, florn molecular chemistry to socio-econotnic patterns [91.

1.4.3 The Electronic Book

From 1980 to 1983, a demonstration system comoining features of print,
broadcast, and computer media was developed by Dave Backer tor his
L4 MULTIMEDIA AND PERSONALIZ'ED COMPUTINC

Flgure 1.3b Moving video view of Aspen streets with control

interface (top) ancl aerial view (bottorn) for top-level navigational
control (ReprocLrcecl courtesy of the MI'l Meclia l,aboratory, O MIT
Media Laboratory 1 97tt)

doctoraldissertation on a prototype of the eiectronic book [101. The project

used automobile repair as its therne, ancl is significar-rt troth for its book-like
user interface as well as the rrrtclerlyirtg authoring envirorrrrrent. Front the
users' perspective, a sopiristicatecl rnultirneclia book was liter;illv at their
 USEs OF MULTIMEDIA INFORMATION

Figure 1 .3c L.ntering the Aspett coLtrthotrse (tctp) attcl tlttlr cottrt
clerk's oifice (l)ottonr) ior access to towll recrlrcls (Reprtltltrcecl
courtes)' of the N4lT N4etlia Laboratory, O MIT Meciia Laboratory
197u)

fingertips becar,rse of the touchscre'etr interface (Figure 1.4). Pages turned

like ordinary books, but se-architrg and indexing had the added benefit of
the computer. The material coulcl be personalized by annotations and the
addition of nL.w rnaterial. 1'he unclerlyirtg authoring system was available
tt-r the user, ancl anticipatectr toclay's authoring tools by providing an
object-orientctl storage rnoclel atlcl a scripting language.
 .I.4
MULTIMEDIA AND PERSONALIZED COMPUTINC

Flgure 1.4 Screen frorn Dave Backer,s prototype electronic book

showing how to repair an autornobile transrnissior.r. Vicleo co.trols
are overlaid on the irnage irr the right-hand corner. As tlre vide<t
plays, the accornpanying text cl.ranges in step. (ReproclLrcecl cotrr_
tesy of the MIT Media Laboratory, O MIT Mectia Laboratory l97fi)

The prototype workstation integrated sound, data, irnages, and video

presentation. The first use of coordinated access to synchronizecl media
appears here. Video and accompanying text were synchronized in time. If
the user changed the position of either text or video during presentation,
both streams repositioned autolnatically.

7.4.4 Formation of the MIT Media Lab

"Tltc future of vidcodiscs is situutcl ot tltc irttcrscctiotr of'tltrcr'
ltrcviously di.stittc-t
itulustries: telcvisiotr, publishitrg, otrd cottrputcrs. As sut'lr, u trcw ttrctliuttt ntust etncrsc,
chorocterized by thc ortditory uil vistutl riclrtrcss of talavi.siort, tlta tt't-cs.sibility orrl
pcrsortol quolity ttf books, orrd tlre itftcractivity ottd cxprcssivc potctttiol of'cotttputcrs.,'
Ncaroponte, hftalligettt videodiscs otrd rlrcir Apitlicotiotrs, 19t)0.
-Niclrolos
"Sornc tcnnitrols of thc fifture will bc all-kttowirrg rootns witlutut tytlls. Otlrcrs will ba
llat, thirr, flexible tout'h-sctrsitive disploys. Attd otlrcrs will be wrist wotclrc.s atrd c'uff
litrks witlr the riglrt hottd tolkitrg to the lcft by sotcllitt,.,,
Netn4nttc
-Nicholos
As these research projects proceedecl, Negroponte was ileveloping a vision
of the future uses of rnultimeclia technology and an agencia to pursue it.
Together with Jerorne wiesner, president of MIT at the tirne, Negroponte
raised $40 rnillion for facilities for a new Media l-ab. More than 4o cirrporate
sponsors picked up tl're startup tab and the $4 million annual operating
10 USES OF MULTIMEDIA INFORMATION

Table 1.1 vtf Media Lab (circa 19tt5)

Group hrvestigator(s) Ittterasts

Electronic Publishing
Film and Video
Visual Langr.rage
Workshop
Electronic Music
Spatial In.raging and
Photography
Learning ancl
Episternology
Movies of the Futtrre
Advanced Televisior.r
Researcl'r
Speech Processing
Hurnan-Machine
Interface Grotrp
Walter Bender On-line, personalized
multimedia
newspapers/rnagazines
Ricky Leacock Convergence of film
production ar-rd
presentation with computer
technology
Muriel Cooper, New user interface and
Ron McNeil authoring techniques
Barry Vercoe Corn puter cornpositior.r,
synthesis, anci perfrlrntance
of rnusic
Steve Benton Technolgy for holograms
and holgraphic rnovies
Seynrour Papert DeconstrLrctible
cornputer-basetl learniltg
envirottrnents
Anrly l-ipprnan Digital vicleo as rneclia;
hierarchical cociing of video
Williarn Scl.rreiber Higlt-ciefin ition a r.rci cligital
telcvisiorr t odrrrg tet lrrriques
Chris Schrr.rarrdt Uses of atrdio ancl speech
recognition in the user
in terface; cont puter-Llased
telephor"ry
Dick Bolt Developrnent anci use of
new input devices suclt as
gloves ancl eye-trackers

expgnses. After more than five years of funcl raisirrg, the lab opened in l9B5
with ten research groups (Table 1.1).

1.5 MULTTMEDTA oN THE MAp

During the 1980s, other multirneclia research groups were beirrg forrned as
well. Two notable ones were:
I Olivetti Research t.ab
'l'he carnbridge, linglanc], research group
hati two multiyear rnulti-
rnedia proiects. Panclora, a joint effort witlt C-antbridg,e ijpiversity,
focusecl on cieveloltirtg a peripheral l;ox for creatirtg i rrrr-rltirleciia
.I
.6 A TOUR OF EMERCINC APPLICATIONS 1',|

workstation testbed. The second proiect developed a dynamic locator

system called Active Badge. By using sensors located throughout the
building and sensor badges carried by each individual, anyone could
be located.
The California research group's focus was desktop audio. Some of
the earliest work in combining telephony functions with worksta-
tions resulted. A server architecture called VOX was developed for
managing audio and is discussed in Chapter 15.
I Apple Computer Multimedia Lab
The Apple Multimedia Lab, directed by Kristina Hooper Woolsey,
carried out a large number of innovative multimedia proiects, par-
ticularly focusing on the area of education as discussed later in this
chapter.

A TOUR OF EMERGING APPTICATIONS

f!,
n- r
)]r;t1,1

As thetechnology has moved to the mainstream, applications in tnany areas

have appeared. Entertainment, home shopping, education, health care, and
engineering are a few examples.

1.6.1 Entertainment: Growing lnteractivity and

Group Participation
The use of interactive media for entertainment is no new phenomenon, yet
the scale and sophistication of games and movies will drarnatically advance
in three stages. First, the amount of programming available to audiences
will increase in a drive to video-otr-demand services. Second, the use of
interactive televisiorr facilities for video-on-demand will make it possible to
incorporate audience participation into existing programrning. Third, par-
ticipation will move from a highly controlled audience mode to open-ended
group collaboration and teaming, from many-to-one to many-to-many.
The size of the entertainment market will be a signif icant force in creating
the econornies of scale necessary for other applications of the techrrology
to emerge. Further, the evolution of multimedia computer hardware could
be determined by the success of multirnedia consumer products.

Video-on-Demand
The first stage is developing as an effort by the cable television industry, the
telephone industry, and broadcasters to provide greater flexibility in what
programmilrg is'shown when. The video-on-dernand model [11] takes
advantage of some forrn of two-way communication between the home
and the video source, which could be a low-ban(width back channel on a
CATV path or a phone line with sulficient banciwidth to carry a compressed
video signal. The home may be furnished wittr a low-cost box, which accepts
 l2 U5E5 OF MUTTIMEDIA INFORMATION

siSnals from a hand-held control and which transmits the appropriate status
to the video control center. The viewer uses the hand-held control to
navigate a selection menu and choose a program. shortly after the selection
is made, the program begins playing. During the course of the movie the
viewer is able to pause, reposition, and use other VCR-like controls.

Interactive Cinema
In what is billed as the first interactive cinema, a New york production
company hopes to usher audiences toward a day when the next step of the
hero or heroine is in the hands of movie watchers. The company, Control-
led Entropy Entertainment, uses low-cost technology, which allows the
audience to vote on the direction of the plot. Their first fiim, I'm your Man,
has three principal characters. About every 90 seconds during the drama
the audience is given a three-way branch in the plot. They register their
votes using a three-button pistol grip on the armrest of each seat (Figure
1.5), the choice of the majority being taken.
unpredictability of the film after repeated showings appears to be part of
the attraction. The production has 68 scene variations for a total of about
9O minutes of footage, though a given showing lasts only 20 minutes.
whether the audience feels more in control or more involved in the creative
process may depend on how frequently their votes coincided with the

Flgure 1.5 Scene frorn the interactive filn I'trt Your Mort, in whiclr
the auciience votes to control the direction of the plot (Reproclucecl
with perrnission of Controllecl Entropy Erttertainntertt)
1.6 A TOUR OF EMERCINC APPLICATIONS t3

majority. In the context of home viewing, this aspect might be more

evident.
While the videodisc technology used to deliver the film appears conven-
tional, the scripting of interactive movies is more challenging. The script
writer must ensure that every path connects in a coherent way to the
previous scenes.

Collaborative Computer-Supported Games

During the 1992 presidential campaign, the computer service Prodigy
initiated a collaborative on-line game called The Next President, which let
thousands of users participate in a simulated election campaign (Figure 1.6).
The game paralleled the real campaign, closely following the primary and
convention process, but used fictitious candidates who campaigned on
platforms determined by their supporters. Each week the game creators
would marshall the candidates through press conferences, elections, cam-
paign crises, and rumors, using the messages and votes of the game's players.
Unlike the typical computer game that sets player against the computer,
The Next President illustrates the evolution of entertainment towards
participation and collaboration.

Flgure 1.6 ttre Next President, an on-line game in which hundreds of participants
compete and collaborate to determine the iate of their favorite fictitious presidential
aspirant. (Reproduced with permission of Prodigy services company, @ 1992 prodigy
Services company; game designed and developed by crossover Technologies)
t4 USES OF MULTIMEDIA INFORMATION

1.6.2 Home ShoPPlng

Today, upwards of 60 million U.S. homes have access to cable television
channeliin which a variety of retail goods are marketed. Potential custom-
ers can make a purchase by dialing their telephone an-d using their credit
card. Two possible directions that this application might take are:
convergence of TV
n Interactive television: In the model based on theincrease
and the computer, greater interactivity anC an in the num-
ber of available chainels will permit home shopping to be conducted
to many retailers throughout the community'
n Video-telephony: In the model based on the convergence of the TV
and the teiephone, subscribers would dial the retailer of interest, but
would connect to a live video salesperson or a prerecorded video
showcase.
Home shopping is currently a $2 billion industry. It offers Sreater con-
venience bui iestricts the buyer to a limited seller-conducted test drive'
However, home shopping can work {or buyer prepatation even if the final
sale must be conducted in Person'
In another type of home shopping, touchscreen kiosks have entered the
business of rejidential real-estate sales. Developed by a company in Den-
mark, Home-VisionrM allows buyers to interactively browse a photo data-
base of homes, seeing different views and the rooms of the property (Figure
1.7). The interface is visually oriented and entirely screen-based. The buler
selects price rang€ and geographic area of interest. The system then presents
a collage of properties matching the specification. From there the buyer can
chooseanyhouse for a visual tour. The interface includes an accompanying
audio track, which can be enabled or disabled bv the viewer.

Hgun 1.7 User interface of Home-Vis'o6tM, z multimedia ki-

osf application that uses an IBM PS/z@ running OS/2@ and was
developed by the Danish company Multimedia Management
1;6 A TOUR OF EMERCING APPLICATIONS l5

Fgurc l.E Demonstration of a muttimedia inventory manage-

ment interface, designed by GainTechnology and created using the
GainMomentum authoring environment (Courtesy of Gainl'ech-
nology, Inc.)

The Home-Vision system reduces the number of homes that agents must
take a buyer to see by 50 percent. An added benefit is the transparent
tracking of buyers as they browse the database. These statistics can be used
by real-estate agencies to generate marketing reports to the seller and to
better understand the buyer.
A crucial advantage of multimedia technology is the shift to a visually-
oriented interface. The Home-Vision system is one illustration of this.
Replacing tables and text with pictures can make the information and the
interaction process more direct for users. For example, the mock-up phar-
macy inventory interface (Figure 1.8) developed by GainTechnology shows
a section of the store shelf. Store personnel and customers can point at the
items as if browsing through a store catalog.

1.6.3 Multlmedla Communlcatlons for Healthcare f::t r 1} f'':1 n

ue!#.*wr.*de

Environments in which imaging applications are prominent are good

candidates for the use of multimedia technology, since the technology
permits visual data to be more easily viewed, shared, and processed. certain
segments of the health-care industry are examples of this, and several
systems involving multimedia communications and computers have been
tested [12-16].
- h lBoston-area
four
recent multiyear Media Broadband Services (MBS) study involving
hospitals, the regional Beil operating company NyNEi.
16 USES OF MULTIMEDIA INFORMATION

provided broadband interconnections to field test the benefits of high-

bandwidth communications. NYNEX and the hospitals also collaborated
on developing applications software needed for medical specialists to
perform workstation-based video conferencing and image retrieval.
In their assessment of the case for multimedia communications in health-
care, NYNEX observed the following:

tr One hospital has a staff of 59 people to maintain and transport

hardcopy imaging between different hospital facilities. The MBS
facilities permit the hospital to use electronic transfer for these
images.
n Access to specialists is a frequent problem, leading to delays in
diagnosis or requiring the specialist to travel from hospital to hospi-
tal. MBS provides multimedia workstations for the home of the
specialists, from which they can review computer tomography (CT)
and magnetic resonance imaging (MRI) images and consult with
attending physicians.
f, One hospital's on-line medical record system is text-based; by adding
imaging and multimedia communications, a more complete record
is available that can be shared on-line by collaborating physicians.
While acknowledging the potential substantial costs of providing multi-
media communications, NYNEX expects benefits in three areas:
1. Reduced cost of delivering healthcare
Although a typical S0o-bed hospital creates over two million paper
record documents annually, less than 1 percent of health-care facili-
ties with more than 300 beds have systems for storing paper docu-
ments on optical disks, despite the potential savings. Additionally,
on-line storage of reports would permit remote access, saving the
costs of moving records between the storage facility and the hospital.
2. Increased revenue opportunities
Major hospitals can use high-bandwidth communications from
smaller hospitals to increase referrals for complex procedures and
attract wider usage of advanced facilities. The communication facili-
ties act to strengthen the relationship between the teaching hospital
and the client hospitals, which become more dependent on the
higher-qual itY service.
3. Improved patient care.
Multimedia communiCations can provide significantly greater access
to specialists, particularly valuable for complex treatments. Informa-
tion sharing and video conferencing are key components of multi-
media communications.
The use of imaging techniques (X-rays, CT, MRI, etc.) is growing ln
healthcare. The collection, maintenance, processing, and distribution of
1.6 A TOUR OF EMERCINC APPLICATIONS 17

Flgurc 1.9 vultimedia communications for

a radiology application (Reproduced with per-
mission of NYNEX Science and Technology)

these records can be significantly improved by using computer-based stor-

age and high-bandwidth communications. Once these records are inte-
giated withlhe on-line patient information and easily shared by both local
Ind remote physicians, a large number of benefits in reduced costs and
improved care result.

1.6i:4 Geographlc lnformatlon Systems

The management of facilities such as buildings, roads, power lines, and
railroad traiks is a problem that concerns Sovernment offices, utilities, and
many industries. Specially designed computer database manaSement sys-
tems called geogtaphlc information systems (GIS) are available to provide
on-line srpportloi these types of applications. Such systems typically offer
a storage model for spatially organized data so that queries based on area
or locaiion can be quickly answered. Frequently, the user interfaces of such
applications are visually oriented, showing maps or blueprints of the facility
and allowing fast zooming and panning.
With the addition of multimedia information, visual and audio data can
be associated with landmarks and other points of interest. The map acts as
the spatial context for organizing the multimedia information for easy
accesi. A recent survey of multimedia GIS [16] lists a number of existing
uses, including:

n The National Capital Planning Commission (NCPC) in Washington,

DC, uses on-line video clips and images of buildings and intersections
as an aid in park planning and maintenance. Significant time is saved
by having all the site information immediately available to the
d-ecision makers, who are able to quickly compare different areas and
alternatives.
l8 USES OF MULTIMEDIA INFORMATION

! Union Pacific Railroad used on-line video records in a pilot project

in which right-of-way ilnages for 160O miles of track were captured.
The on-line system permits us'e{s to interactively evaluate crossings
where accidents have occurred, andcan significantly reduce the need
for time-consuming field visits.
tr Jefferson County Property Valuation Administration in Kansas City,
Missouri links a videodisc record of over 2QO,00O properties to its
property database for use in assessments and hearings. The system
reduces the need for field trips when properties ate reassessed.
Georgia Power Company has developed a multimedia GIS for the Georgia
Resource center (GRC) presentation system. GRC is used as an information
resource for business executives who are considering locating o\ expanding
in the state (Figure 1.10). The use of multimedia GIS is beneficia'l both as a
promotional tool and for quickly navigating the various sites of ihterest.
The system contains information on 250 communities, s00 industrial
parks, and 550 available industrial facilities. The user of the system enters
the specifications for his/her project, such as:
Building and site parameters
Community demographics
Labor, wage, and tax criteria
Educational statistics and institutions
Transportation services

Flgure 1"lO fne Georgia Power Cornpany,s

mrrltirnedia GIS for the Georgia Resource Cen-
ter's business locator presentation system. (Re-
produced with perr.nission of Georgia power
Company)
 1.6 A TOUR OF EMERCINC APPLICATIONS 19

Display Disolav Disolav Table

CPU.1 CPU-/ cPti-3' DCPUs

Display Display Display Wall

CPU.4 CPU-5 cPu-6 DCPUs
1
6
M
lnterlece Communi- Async
calions
T Manager Manager
o NTSC
k Video
o Devices
n Network Utah Cables
5€rver

R
i
n
Relational
Database +t+t
s Server Cables Wall
/Vall &Tabl€ Proiectors
Display
Query
Soltware CPUs Cables Table
LaDtoD Projectors

Elgure l.ll Architecture of the multimedia presenta-

tion system developed by the Georgia Power Company.
(Reproduced with permission of Georgia Power Company)

Sites which meet the given requirements are highlighted on a 3-D map
of Georgia. Sites can be accessed in any order to viewlmages of buildingi
or community video clips illustrating quality of lif€-and business highlights.
Once the sites of interest are identified, arrangements for site visits are made.
The underlying system (Figure 1.11) uses a distributed set of computers
tp control six independent video proiectors for three wall screens and three
rhgnitors built into the navigation table. A table-mounted laptop computer
has custom software for user queries. eueries are sent to the relational
database that contains the indices for maps, device controls, and reports
stored on the network server. once the query is completed, the interface
manager is notified. The interface manager tells each display cpU to present
specific parts of the data on the corresponding screen. c-onirol of videodiscs
and other devices is handled by the communications manager, which has
32 serial interfaces for communicating with devices. The inteiface manager
handles user input, in particular allowing the user to move the pointer from
screen to screen in one continuous virtual coordinate system.

,.s-
-1.6.5 Education
:r/
*
The ability of multimedia materials to convey by picture, sound, animation,
or movie what is otherwise hard to express, to capture for reuse on any
2A USES OF MULTIMEDIA INFORMATION

occasion remote lands and singular events, and, with the use of the
computer, to provide this information in a form that can be engagingly
interactive and easily recast by any aspiring communicator is the explana-
tion for the popularity and particularly long history of multimedia technol-
ogy's role in education. For many years educational titles have been one of
the dorninant product areas for multimedia computing, surpassed only by
entertainment. A view of some of the innovative uses that have been
produced for education purposes can be obtained from a sampling of the
projects and products developed by Apple Computer's Multimedia Lab,
directed by Kristina Hooper Woolsey.
The Visual Alman4c is an ambitious early proiect [17] that created a large
audiovisual database (stored on videodisc), a set of HyperCard-based mul-
timedia tools, and a number of sample activities tor educators to use as
models. The audiovisual database is organized as twelve collections having
a variety of cultural, historical, and scientific themes; the total database
consists of over 70OO media objects, with accompanying keywords, cita-
tions, and descriptive material. Specific tools for searching, editing, and
composing, with the media obiects as the raw materials, are included with
the software.
The Visual Almanac has been a resource for many subsequent efforts,
both within and external to the Apple Multimedia Lab. One interesting
effort, Beyond the Desktop [18], provides students with picture cards of
objects in the visual database. Each card has a barcode, which, when
scanned by the student, will call up the corresponding media obiect on the
computer. The card becomes a tangible reminder, like a baseball card, that
can be carried around and referred to by the child. Everyday obiects like
pages of a book, pencils, and rocks can be barcoded and used in the same
way. This allows familiar obiects and experiences to be tightly associated
with the information in the computer.
The Classroom Multimedia Kiosk proiect [18] uses a kiosk equipped with
video and computer presentation and a barcode scanner and printer as a
centerpiece for class discussion. In a U.S. history scenario (based on a
Geography Television videodisc), students use the kiosk to view a video
segment on a historical topic and to make small prirrted History Cards. The
cards are barcoded and are associated with a specific video sequence selected
by the student. The student can later scan the card to replay the video
sequence. The card also contains an image taken from the video sequence.
Collections of history cards can be used by the students to collaboratively
create timelines, maps, and other compositions'
Irr 101 Activities [18], the computer becomes an advisor, susgestinS
interesting and creative recreational activities at home. Following My First
Activity Book by Angela Wilkes, the child first goes on a treasure hunt to find
as.many items as possible needed for the activity. After the collection step
is completed, the child selects the What Can I Do? button. Given the
availabie items, the computer then lists possible activities, their complexity,
and duration. For any selected activity, the computer provides a guided tour
 .I.6
A TOUR OF EMERCINC APPLICATIONS
21

I:O-t: l^.3. Survey of Setected

(1987-1992)
Apple Muttimedia Lab Rrolects/rrroOucf

Proiect Description
Visual Almanac Avideodisc collage of audiovisual materials
for
educators, organized in l2 collectio"r.
Ou., iOOO
media objects, 5000 from external sources.
GTV: A Geographical Combines still images and music in an
Perspective on MTV_stvle
presentation popular with middle school
American History ug" gioup.
Interactive NOVA: Supplements the NOVA movie about animal
Animal Pathfinders migration with additional documentation
and three
activities that involve the students in the
subject
matter.
Life Story Supplements BBC,s rnovie about the discovery
of
DNA with accessory materials including --l
docurnentary interviews, text transcriplions,
simulations, references, and navigational
tools.
Mystery of the Using raw footage from a TV documentary
Disappearing Ducks on
wetlands, provides an interactive rneans
ti explore
the ecological controversy. Includes a rnystery
game
about disappearing ducks developed Uy frigh'schoot
students and professional designiri.

with pictures of intermedia steps of the proiects

a.d video crips of compli_
cated tasks.
The Appre Multimedia Lab is noteworthy
not onry for the varied educa-
tional titles that it has producecl (Table 7 .2)" buturro't*
ir," rich set of ideas
that have .o*..iI?T iis quarters regarding the shape
and direction of the
technology [18,19]. During its six-year hist6ry,
ou"riooi.,uividuals partici-
pated in its proiects.

7.6.6.'"lvlultrmedra communrcatrons: An Enabring

Technotogy f* c;;;irent Englneering
,_/'-
_*,-/- and Manufacturing
The Institute for Defense. Anarysis
[20] has identified various practices,
known as concurrent engineering, wrrictr- orgu"iruiioi,
r,aue adopted in
order to accererate-the disign an"d manufact"ure
products. one.of the key precepts of these
of .on-'pte* systems and
practicei'ii' ,n" coordinated
activity of engineering, manufalturing, una
proiect through ail phases -unug..!nt activities of a
of its deverop"ment. Th" b;;;ili, of this approach
are reduced time_to_market and improvecl
product qualitv.
The concurrent engineering methooorogy
u...i.ir"r';r"ouct deverop_
ment by incorporating more parailerism u"t*"".
*r.,riul" .onu.nrionally
22 USES OF MULTIMEDIA INFORMATION

serial phases of a proiect. This parallelism, however, requires closer coordi-

nation between the different functions within the organization, with
correspondingly increased group communication requirements. The use of
multimedia communications tools is one of two uses of multimedia tech-
nology that play an enabling role for concurrent engineering. These tools
permit product groups to communicate in distributed environments, shar-
ing applications and data, using on-line group decision support tools, and
interacting in group conference calls.
p,ngineering and manufacturing groups rely increasingly on CAD and
CAM software to design, manufacture, and maintain their products. To-
gether with technical publishing software, these tools permit design speci-
fications and technical documentation to be created and accessed on-line.
The second enabling role of multimedia technology is to supplement these
existing tools with a richer information processing environment. For exam-
ple, audiovisual product information, voice annotations of design docu-
mentation, and on-line interactive help manuals can be supported by the
availability of on-line multimedia information, The richer information
content indirectly enables the concurrent engineering process by providing
a more effective and direct form for expressing an idea.
- The practice of concurrent engineering can be enhanced by use of
computer and communication technologies. In particular, multimedia
technologies permit work groups to collaborate in a richer information
environment. Researchers at the Center for Productivity Enhancement
(CPE) at the University of Massachusetts Lowell, funded by NYNEX, have
developed a demonstration system for the use of multimedia technology
for concurrent engineering. The project extends an existing factory-of-the-
future prototype robotic workcell with a distributed multirnedia worksta-
tion environment.
The CPE Factory-of-the-Future system (Figure 1.12) was designed in the
late 1980s to demonstrate a number of advanced manufacturing concepts,
including:
Integrated design and manufacturing software environments
Flexible robotic workcell techniques, including coordination between
robots during subsystem errors
Automatic generation of manufacturing instructions from the engineer-
ing design
Intelligent application of design rules during the design process
The distributed multimedia workstation enviro,nment, developed in
199o, combines a cross-campus video network, existing applications soft-
ware, and shared applications to allow multimedia documents to be shared
during group video conferences (Figure 1.13). Using the Factory-of-the-Fu-
ture as a backdrop, group discussions on common engineering/manufac-
turing issues such as design reviews, rerrrote diagnosis of erl uiltrncnt failu res,
and change orders were enacted ancl various rnultirncciia r.lesigrr rnaterials
 1.6 A TOUR OF EMERCINC APPLICATIONS
23

Flgure 1.12 ffre University of Massachusetts Lowell Fac_

tory-of-the-Future as used for demonstration of multimedia
technology in concurrent engineering and rnanufacturing

were provided in on-line manuals. These experiences demonstrated tlie

two
roles of multimedia technology in concurrent engineering:

n F.ngineering and manufacturing as a visually rich domain is a natural

beneficiary of multimedia information processing
r Group communications, when conducted via the computer worksta-
tion, allows the participants to refer to and share any on-line rnate-
rials that might be appropriate to the discussion

Flgure l.l3 ViOeo conferencing tool used as part of the

mul ti rneclia c<lrnrn urr ications envi ron rnent i n the Un iversi tv
cf Massachusetts Lowell concurrent engineerirrg demonstra'_
tion
24 USES OF MULTIMEDIA INFORMATION

1.6.7 The lmpact of Ublqultous Multlmedla Servlces

There is an impressive range of applications of rnultimedia technology,
some of which may not have been conceived of yet. Independently, each
one offers testimony to the potential benefits of multimedia systems.
However, a greater impact will result when the parts are knit into the whole,
when multimedia computing and communications become as widely avaii-
able as today's telephone and television. Then multimedia documents,
presentations, mail, games, and other applications will be the common
fl€nominator by which people and organizations communicate, work, and
play together. This prospect is one that will be the most socially transform-
rngnot onty becauie of enriched modes of expression, Sreater information
aciess, and more accessible tools, but because of the potential for greater
involvernent of the individual on the creation side of the media content.

1.7 MULTTMEDTA sYsrEMS: THE cHALLENGES

The challenges ahead to developing ubiquitous multimedia systems include

solving a gamut of technical problems that are discussed in the remainder
of the book:
Temporal and intermedia synchronization within the operating system,
network architecture, and presentation system
Higher performance networking of time-based media
On-line storage, a'ccess, and interchange of multimedia content
New user interface Paradigms
New tools for authoring and using multimedia information
widely adopted standards, many of which are underway, will be a
significant ass-et to the development of the market. The cooperation of
in-formation providers, system integrators, communications and computer
companies, and applicaiion developers are likewise essential to the direc-
tion of the technology and the pace of its growth'

1.8 AcKNowLEDGMENTS
'the following individuals in providing infor-
I appreciate the assistance of
*itio.t used in the preparation of this chapter:
Dave Backer (Course TechnologY)
Diane Blackman (Entropy Entertainment)
Valerie Eames (MlT Media