You are on page 1of 12

Real-Time Applications

KEY POINTS OF THE CHAPTER are l. Real-timesystems comPlex. 2. The first real-timeapplicationmay have been Project Whirlwind. are 3. Real-timedatabases specialapplications. 4. Real-timeimaging is an importantbut challengingapplicationarea. may not be suitablefor "real" applications. 5. Real-timelanguages of to It is appropriate closethe text with a discussion severalinterestingspecialty and somereal-timeapplications. areasfor real-time systemsdesigners

Although tremendous problems are involved in building real-time multiprocessing systems (all the problems of uniprocessingsystems plus intercommunication,resourcesharirg, load balancing,etc.), most real-time systems multiprocessingsystems.This is true in are usually distributed heterogeneous most military applications and in industrial applications where standard by connected local areanetworks(LANs) suchas RISC workstations components (WANs) are the implementations of choice. Specialand wide area networks such as those discussedin Chapter 72, are increasingly purpose architectures, found to be noncompetitive over the life of a system when compared to that use off-the-shelfhardware. approaches


Chap. 14 I


realthat the construction large-scale, of It is increasingly being recognized The fact is that is time systems not just about real-timesystemsand scheduling. most large, modern,practical computer applicationsembody many propertiesthat most combatsystems arecurrentlybeing studiedlargelyin isolation.For example, are real-time and distributed, seekto be reliable, and are fault-tolerant.Although it is important to study these issuesindependently,collectively these properties activity in the area Globally,thereis increasing have a complexinterrelationship. this of "complex systems"as a new, multidisciplinaryfield of study.In a sense, designer book is an attemptto discusssomeof the areasthat the complexsystems (no longerthe "real-time" systems Throughout designer) will needto understand. this text it has been argued that a holistic approach is the only way to build complex, distributed,real-timeapplications.

of In 1947MIT, in conjunctionwith the U.S. Navy, begandevelopment Project Whirlwind (see Figure 14.1). This projept was the first high-speedelectronic digital computer able to operatein "real-time" and to be put to use as a practical device. The goal of the project was to develop an airplane trainer/analyzerthat would simulate aerodynamicforces acting on the pilot's controls. The pilot's reactionswouid have to take effect as promptly in the simulator as they would in a real airplane. Thus, the equipment had to operate within these "real response who soughtto develop times" [32]. Whirlwind was begun by Jay W. Forrester,

Figure 14.1 Photograph of the U.S. Navy's Whirlwind computer. (Courtesy of Jay Forrester and the MIT Museum.)



Real-Time Imase Databases


an airplane trainer/analyzerthat would simulate aerodynamicforces acting on the pilot's controls.The pilot's reactions would have to take effect as promptly in the simulator as they would in a real airplane. Thus, the equipment had to operate within these "real responsetimes." We might also argue that Whirlwind was "complex" because addition to speed, in reliability was also of greatimportance, and it was during this project that the military conceptof "command and control" was introduced|32). Whirlwind's features were made possible through the use of ferrite cores, which were faster than the cathode-ray tube memories of previous systems. Ferrite coresof the samekind were also used for computer memoriesin the 1950s in the United Statessuch as the 1953 UNIVAC 1003 model, the 1954IBM 704, and others[75]. Another distinctivefeatureof Whirlwind was the inclusion of an "algebraic compiler," an early pseudoprogramminglanguageresulting from the work of Laning andZierler [751. During the project's final stages,the Whirlwind team became assimilated into the Air Force's Semiautomatic Ground Environment (SAGE) air defense system project [132]. As a result, the basic ideas of Project Whirlwind were incorporated into SAGE. Thus, while it is widely held that SAGE was the first real-time application, Project Whirlwind seemsto deservethis credit.

An important application area for real-time systemsis in databases. real-time A database systemis a transactionprocessingsystemin which transactionsand data have explicit timing constraints (deadlines). Real-time databasesare used in a variety of applications, including process control, radar systems, computerintegrated manufacturing, traffic control, virtual reality, and multimedia. Designing real-time databases involves 1. Analyzing the system requirementsto identify data 2. Determining the hardware capabilities 3. Identifying the different data criticalities and integrity constraints Ordinary database management systems provide concurrency control techniquesthat seek to preserve data consistency (e.g., read and write locking) (e.g.,two-phaselocking). Real-timedatabases and logical consistency must also impose concurrency control that preserves the schedule and data temporal consistencyrequirements.As with most real-time systems,minimizing average response is less important than guaranteeingthe timing constraints of critical systems,serializableexecution of concurrent transactions.In the caseof database transactionsmay also be of less interest. A rcal-time databasesystem must operate within the context of the bsr operating system and use existing services. Hence, construction of a real-time


Chap. 14 I


is operating systems,and most database nearly impossiblewith non-real-time. on real-timedatabases constructed top of specialreal-timeoperatingsystems. are systemshave yet to emerge.Most Consequently, commercialreal-timedatabase use proprietary algorithms and are of the available "hard" real-time databases deadlines are suitablechiefly for softer real-timeapplicationswhere transaction "negotiable." the for As an exampleof a real-time database, architecture the U.S. space station"Freedom" is shown in Figure l4.2.The primary role of this systemis to provide integrateddataprocessing and communicationservices both the core for function and payloads. The systemneedsto be robust in that it must support30 yearsof continually changingdata processing and networking needs.

Figure 14,2 Data managementsystem of spacefreedom.

The system consists of a Fiber Distributed Data Irfterface (FDDI) ring. Sel eral standard data processorsare attached to the core network, along with specializedhardware such as communication and tracking hardware, defined as units (ORU). Thesedevicesare then interfacesto other devices orbital replaceable (MDM) or via the Mil-Std-15538 bus.architecture via multiplexer/demultiplexer units. The standarddata processoris actually a multiprocessingsystemconnected via Multi Bus II and containsa network interfaceunit (NIU) that runs the network operating systemand a bus interface unit (BIU) that interfaceswith the MDMs or (EDP) data processor the ORU hardwarethrough the 15538'bus.The embedded This runs a version of real-time UNIX (LynxOS), which is POSlX-compatlble. real-time system is mission critical, and simulations complex, multiprocessing queries in a period have shown that it must handle approximately 5000 database

Sec. 14.,4I

Real-TimeImase Processins


of 6 seconds. Variousschedulingdisciplines,including rate-monotonic being are investigated optimizeperformance. detaileddescnptionof the s1'stem be to A can found in [28].

14.4 REAL.TIME IMAGE PROCESSING Real-timeimage processing differs from "ordinary" image processing that the in logical validity of the systemrequiresnot only correctbut also timely outputs.It is often said that image processing real-timeis no more than a minor variation in on imageprocessing without regardto time; fastermachinerywill makeany speed problems eventually go away. But real-time imaging is not just about speedy hardware.It involves at least three fundamentaltradeoffs:performanceversus image resolution,performanceversus storageand input/outputbandwidth, and number of tasks versus synchronization. theseproblems,only the first and Of possibly the second might be solved by faster machines. Furthermore,the problems of expressing image algorithms (especially for multiprocessing architectures), finding appropriateprogramming languages,of testing and of reliability, and of practicalsoftwareengineering techniqueare not readily solved simply with fasterhardware. Because its nature,thereare both supportsfor and obstacles real-time of to imageproeessing. the positiveside,many imagingapplications well-suited On are for parallelization and hence faster, parallel architectures.Furthermore, many imaging applicationscan be constructed without using languageconstructsthat destroydeterminism. Finally, specialreal-timeimaging architectures available are or can theoreticallybe constructed. On the down side, many imaging applicationsare time critical and are computationally intensive or data intensive. And there are no standard programminglanguages availablefor real-time image processing. Finally, realtime processing science itself is still struggling to produce usable results, especiallyfor parallel processing machines. Real-time image processingcovers a multidisciplinary range of research areas,including (but not limited to) image compression, target acquisitionand tracking, remote contrel and sensing,image enhancement and filtering, networking for real-time imaging, advancedcomputer architectures, computer vision. optical measurement and inspection,and simulation.These researchar€asare critical in such applicationsas robotics, virtual reality, multimedia, industrial inspection,high-definition television,advancedsimulators,computer-integrated manufacturing, and intelligent vehicles. In summary,real-time systemsare unique in that semanticvaliditl'entails not only functionalcorrectness also deadlinesatisfaction. generalized but A realprocessing time image systemis shown in Figure 14.3.It functionsby taking an analogvideo input from a Qamera sensingdevice and digitizing it. The image or is then enhancedor filtered to remove noise. If obiect or motion detectionis


Chap. 14 I


Figure 14.3 A generalizedreal-time image processingsystem.

required,thatprocessis applied.If appropriate,an object recognition algorithm or someson of otherdecisiorilogic is appliedto the modified image.In most cases, image display is also required. For many applications,these steps must be executedin real-time-for,example, at the 30 frames per secondrate neededfor continuous motion PercePtion.

VirtualReality 14.4.1


To investigateone real-time imaging application briefly, consider virtual reality. Virtual reality (VR) systemsare complex computer simulations involving visual, audio, tactile, and other feedbackqo entice a person'sperceptualmechanismsinto believing they are actually in an \rtificial world. Although virtual reality has obvious applications in combat simulation and training, its most promising applicationsare civilian, including exerciseand recreation,physical rehabilitation and therapy,occupationaltraining, and psychological diagnosistraining. In these types of systems, high performance and deadline guaranteesare essential to ensurethat the illusion of reality is maintained' Virtual reality systemscome in at least three paradigms: I Telepresence-where a human operator can remotely control robits or other devices as if the operatorwere physically present.For example, an individual can don a special suit that would pefmit the repair of an underwaterstructurevia a robot whose movernentsreflected those of the operator. The operator "thinks" that they are underwater and are working on the real sffucture. I Immersion:-where a simulator is implementedthat can actually fool the sensesof the human subject into believing that the simulated scenariois

14.4 I

Real-TimeImage Processing


real. Fansof televisionsciencefiction programs,suchas "Star Trek: The Next Generation,"are familiar with this type of virtual reality. r Augmentedreality-where a displayof live informationis supplemented with simulatedinformation.A simulatedX-ray view of a patient'sbody on superimposed a monitor display of the live patient for diagnostic purposes such an examPle. is for architecture eachparadigmis shownin Figure I4.4.In case(a), A generalized display, hand a telepresence, human operatorinteractswith a helmet-mounted thenrelays The to connected the VR processor. processor controls,and foot levers remote ship. The ship is attachedto a the control information via satellite to a The human with the human'smovements. robot that movesin accordance tethered also receives feedback information from the robot in the form of visual images and possibly force fqedbackon the hand or foot levers. In case (b), immersion, a specially designed room provides a human participant with simulated images (criminals and bystanders)through a helmetmounted display. Motion sensorsin the ceiling and light reflectors on the floor help to track the participant's movement in order to aid in generatingthe correct perspectivefor the 3-D images in the room. reality, a diver is equippedwith a goggleFinally, in case (c), augmented mounted display that augments the real-world view of the submarine with computer-generatedoverlays that give the diver information about the inner workings of the ship for the purpose of repair. In each case, extensive run-time support is needed for high-resolution force and displays,digital cameras,motion sensors, graphics,helmet-mounted tactile feedbackperipherals,and other devices. complex,multitasking,graphicssimulations represent VR systems Because problems,they are perhapsthe most challenging that involve control and tracking of all real-time systems; using today's technology, perfect results are not obtainable. Virtual reality also involves concurrent programs and processors, shared peripherals, and the important notion that synchronization is at least as important as timeliness. For example, in virtual-reality-type flight simulators, even a slight skew in the synchronizationof a pilot's commandsand the resultant display update(e.g.,a turn is made) can causenausea' 14.4.2 Multimedia Advances in multimedia computers have significantly affected research in multimedia computing. Multimedia systems can be collectively viewed as a combination of voice, images, animation, full-motion video and audio. The prim4ry area. of research in multimedia computing deals with multimedia hardwareand software systems,as well as tools developmentfor computer-aided leaming (e.g., encyclopedias,interactive games). In particular, much research focuses on distributed multimedia systems. Distributed multimedia systems


Chap. 14 I


(b) Figure 14.4 Virtual reality paradigms: (a) Telepresence, immersion, (c) augmentedrealr-u.


Real-TimeImage Processing


require synchronization,large storage, indexing, and retrieval of data. For example,as with VR, in multimedia applications,it is clear that audio speech output must be in synchronywith the image of a personspeaking. Multimedia systems are complex systems that need to deal with high processing power, high-speednetworks,large storagedevices,video and audio compressiontechniques.real-time operating systems, software development, storage and retrievalof data,paralleltechniques, architectures distributed and for environments.Large-scalemultimedia storagewill be an integral part of the emergingdistributed multimedia cofilputing infrastructure.Work has beendone to provide high bus bandrvidth afrd a high-performance multimedia real-time operating system that supports real-time scheduling. In the software tools development, research has beencenteredarounda new object-oriented language paradigm. major concernin multimediasystemis audioand video compression. A An importdnt consideration in constructing deterministic, predictable, and synchronizedmultimedia systemsis whether the compressiontechniquesare deterministic. A generalized multimediaarchitecture given in Figure l4.5.In this design, is compressgdand uncompressed audio and video (and control inputs such as timing) are input into the system.A local database may be accessed the for (such as in vector quantizationtechniquqs)or to purpose of decompression provide local information to supplement the system(e.g.,backgroundsOgnery).

Control inputs

Multimedia processrng sysrem

Figure 14,5 A generalizedmultimedia architecture.


Chap. 14 I


the data in a variety of ways (for example, The processingsystemmanipulates considerthe foreign film systemshown in Chapter5), but ultimately the system outputsvideo and audio.

14.5 REAL.TIME UNIX An important real-time "application" is not really an application at all but an operating system.However, becauseof its commercial importance,the UNIX operatingsystemand its real-time relativesare of great interestto applications developers. Real-time UNIX solutions seek to guarantee response times thrt -:r in considerations <ineof the following areas:kemel features,memor) rrrj,i-lrment, real-time I/O, and user tools. amenable real-timebecause to The standard UNIX kernel is not considered the are and because kernel services it doesnot provide for preemptivescheduling not re-entrant.Real-time UNIX kernels seek to guaranteeresponsetimes by priority scheduling. providing a re-entrant and interruptiblekernel and preemptive are and synchronization also necessary. Good interprocess communication by Real-timeUNIX alsoprovidesreal-timememory management providing data integrity for all data structures.Some real-time kernels use over 3000 (Is to semaphores protect as many data structures. there a potentialfor deadlock (and of here?)Also availableare preallocation files to preventdisk fragmentation its insidiousproblems)and processlocking in memory to reduceswap times. Many real-time UNIX versions also provide user tools in the form of programmercontrol of systemresources(not afforded by ordinary UNIX) and An for rate-monotonic schedulingtools basedon spreadsheets. architecture one real-time UNIX is shown in Figure 14.6. For an excellent discussionof this implementation real-timeUNIX, see[48]. of

14.6 BUILDING REAL.TIME APPLICATIONS WITH REAL.TIME PROGRAMMING LANGUAGES real-time Over the last 20 yearp,and especiallyin the last 10, severalspecialized It languages have been developed. is probablytrue, however,that most real-time code is written in C, Ada, FORTRAN, and assembly language, with C++ and Ada95 gaining rapidly. Nevertheless, certain applicationsare built using specialty languages, and lheir suitability for the task merits some discussion. As we mentioned briefly in Chapter 3, real-time programming languages seek to provide for schedulability analysis in one of three ways:

To eliminate nondeterminancy To augment the languageto provide explicit timing and control

Applicationswith Real-TimeProgrammingLanguages Sec. 14 4 B Building Real-l-irne


user level Kernel level

Kernel level hardware level

tbr Figure 14.6 One architecture real-time UNIX see [48].

jointly with the operatingsystem(e"g.,POSX r To constructthe language compliant languages) that have been deveiopedover the last 20 yearsto handle Someof the languages include real-timeapplications r Ada95: A language that seeks to addressthe real-time problem b1 This uas augmentation(in this case of the standardAda lan-sua-eei in discussed Chapter3. that ts occam-2:A programminglanguagedesignedfor use on translruters uses augmentationto provide some explicit timin,s and control. This use has seencontmercial in the UK. language


Chap. 14 I


PEARL: The Processand ExperimentAutomation Realtime Language Pearl developedin the early 1970sby a group of German researchers. has fairly wide application in uses the augmentation strategy and Germany,especiallyin industrial control settings. language Real-timeEuclid:An experimental that enjoysthe distinctionof to beingone of the only languages be cornpletelysuitedfor schedulability restriction.Unfortunately, the analysis. This is achievedthroughlanguage languagehas not found its way into mainsfrainapplication. Real-timeC: Actually a genericname for a variety of C macroextension packages. typically provide timing and control These macroextensions constructs that.arenot found in standardC. r Real-time C++: A generic name for several object class libraries C++ to specificallydevelopedfor C++. Theselibrariesaugmentstandard provide an increased level of timing and control. r Java: A languageused frequently in World Wide Web pages.It was systemslike consumerelectronicsand originally intendedfor embedded is still gaining acceptancein that domain. Java combines a virtual machine (an interpreter)with a C++-like languageand an associated compiler that convertsthe sourcecode into interpreterdirectivescalled byte codes.The advantage this approachis that the code is machine of (i.e., it is targeted the virtual machine)-hence its use in for independent world of the Web. In someways, Java is a greatrealthe heterogeneous time language.It allows for multithreading (with POSIX compliance when running on UNIX platforms), provides explicit synchronization but rnechanisms such as spinlocks,and is object-oriented, without such drawbacksas garbagecollectton. environments with Thereare,of course,many otherreal-timelanguages/operating nameslike MACH, EIFFEL, MARUTI and ESTEREL that are widely referenced in the literature. Many of these languagesare used for highly specialized of or only. For a thoroughdiscussion real-timelanguages, applications in research see[156].

l. Researchthe real-time programming languagesdiscussedin this chapter and write a report f'eatures discussed theselanguages with respectto the language comparingand contrasting in Chapter3. 2. From the research literature, identify and discuss at least three real-time languages not in mcntioned this book. in above,constructa softwaremodel for the systemdescribed 3. Using any of the languages are availablefor fiee on the Intemet. Section 13.5.Many compilersfor theselanguages