You are on page 1of 17

The Real-Time Application Interface

Abstract Different mechanisms are


implemented to ensure that
this sharing is done
The Real-Time Application efficiently. Mainly, sharing is
Interface provides Hard enforced using aging
real-time capabilities in a schemes and execution
Linux environment. It precedence of important
provides FIFO, semaphore, system code. This design
mailbox, message and RPC philosophy is very well
communication primitives suited to the workstation
and includes a Posix and server environments
compliant subsystem. where it finds its roots. For
Hooking on to Linux is systems where real-time
provided via a Real-Time deterministic response
Hardware Abstraction times are critical, this
Layer, hence diminishing approach is ill adapted and,
kernel intrusion. in essence, does not fit the
requirements.
Built on top of RTAI, the
LXRT module provides for In designing such systems,
seamless integration of real-time operating systems
normal Linux processes and (RTOSes) may be useful and
RTAI-bound real-time tasks. are in fact often used to
It enables these two types of guaranty such things as
tasks to communicate interrupt response times,
symmetrically while context switching and
enabling normal Linux priority inheritance.
processes to enter the real- The problem with these
time execution domain OSes however is that they
using a single system call very often support a much
and provides memory more restricted set of
protection and trap hardware platforms and
handling for hard real- time devices compared to their
tasks. RTAI is supported by general purpose
various other projects counterparts. Not to speak
including the Linux Trace of the cost of quality RTOSes
Toolkit and RTNet. and their restrictive
distribution licensing.

It is in this context that a


Introduction hybrid form of operating
systems was born where a
GPOS is provided with the
General purpose operating control of most hardware
systems (GPOSes) are resources but is itself
designed to offer most subject to the control of a
applications a “fair” share of deterministic hardreal-time
the system resources. OS which enforces strict
scheduling policies within With the release of Linux
the context of the GPOS. One 2.2.xx in early 1999 it
such hybrid, the DIAPM became possible to
Real-Time Application implement the RTHAL
Interface, originated from concept since the hardware
the work done by Paolo management interface was
Montegazza in using MS- properly layered. This
DOS as the basis of a real- enabled minimal
time operating system for modifications to the kernel
the purposes of his research while maintaining the
in the ’80s. At the time, the dynamic loading of the real-
real-time OS ran as a time executive as a common
TSR1program. kernel module. By March of
the same year the first
With the advent of release of RTAI was made.
mainstream 32-bit OSes it Since then, RTAI has seen
became desirable to have many increments and now
such capabilities provided includes many widely used
on such systems. As the communication methods, is
Linux kernel was becoming supported by a wide-variety
more and more popular and of projects and runs on both
its sources were available, the i386 and the PowerPC
an attempt was made to architectures. It is also
port the previously widely used to implement
available functionality to it. real-time systems and is
Yet, at the time, the Linux supported by a variety of
2.0.xx kernel wasn’t deemed vendors.
mature enough to support
the RTHAL (Real Time This paper will focus on
Hardware Abstraction presenting the different
Layer) necessary for the capabilities of RTAI, how
undertaking. But this was they work, how they can be
soon overcome as the used and how they interact.
RTLinux project effectively Section 2 will discuss the
diverted control from Linux intrinsics of RTAI’s
for its realtime needs. functionality. Section 3 will
Hence, a DIAPM-RTL discuss the different
implementation was born schedulers provided with
which used the NMT patch RTAI. Section 4 will present
as the basis of its the different
functionality. communication facilities

Linux

RTHAL
B

A RTAI Core
1 Terminate and stay resident. B

Hardware

Figure 1: Control flow from and to the hardware interrupt facilities.


and other services interrupts that will need to
implemented by RTAI. be diverted from Linux
Section 5 will discuss the without hindering its
LXRT symmetrical normal behavior. In
communication facility. addition to diverting the
Last, section 6 will look at interrupts from Linux it will
the various extensions being also be important to ensure
made to RTAI and the that Linux is not in a
various future directions of position to control the
development. occurrence of these
interrupts. Hence, the RTOS
will have to be provided
with means to divert these
two control mechanisms:
the flow from the interrupts
Intrinsics to Linux and the control
Linux has of the interrupts.
In RTAI, this is done using
the RTHAL which is the only
Real-time systems are addition made to the Linux
designated as such because kernel in the form of a 100
of their deterministic line (approximately) patch.
response times to outside
events. Yet, we know that Figure 1 presents the
GPOSes are not capable of RTHAL. In the case where
providing for such RTAI is not loaded, case A,
requirements. The Linux then Linux is directly
kernel, however efficient it interfacing with the
may be, is no exception. It is, hardware. When RTAI is
although, possible to loaded, case B, control of the
provide a RTOS running interrupts goes through
alongside Linux to provide RTAI’s core in either
for real-time needs. To direction. To achieve this,
accomplish this hybrid the RTHAL is made up of
configuration it is necessary function pointers which will
to provide for Linux not to point to Linux’s native
divert critical hardware functions and tables initially
events from the RTOS as and which will be diverted
these are the events that to RTAI’s internal functions
will dictate the response and tables when RTAI is
time of the system. loaded. 2.2 Managing the
diverted resources
Taking control from
Linux
Once in control, RTAI will
seamlessly provide
interrupt control to and
As critical hardware events
from Linux and, in addition,
mostly take the form of
will provide a number of
outside interrupts it is the
abstractions and facilities to
occurrence of these
real-time tasks. These implements replacements to
include interrupt allocation the cli()/sti() couple. These
and timer control. Also, replacements are actually
RTAI provides a way RTAI functions which will
set flags within internal
Interrupt Occurrence
maintained structures to
keep record of whether
Linux would like to be
informed of incoming
RTAI Dispatcher
interrupts (sti) or whether
he’s ignoring them (cli).
Since, in normal
circumstances,
RT Int Handler all pending
interrupts would be
signaled upon the
occurrence of a hardware
Linux Dispatchersti, RTAI implements such a
mechanism by registering
all pending Linux interrupts
SRQ Dispatcher and signaling them at the
appropriate time.

Linux Intr RetFigure 2 illustrates the path


an interrupt may follow
from the moment it occurs
to the moment where
control is returned to
Interrupt Return
program execution.

When a global interrupt


occurs (meaning through
Return to Program Execution
the 8259 PIC), the first
function encountered is
Figure 2: Path followed dispatch global irq(). If a
upon the occurrence of an real-time handler is
interrupt in an RTAI/Linux registered for this interrupt,
system. it will be called here. Once
done, or if there are no
handlers, a verification will
be made as to whether
to call on the non- Linux is expecting this
deterministic Linux facilities interrupt or not and
using System Requests whether he has “disabled
(SRQs). This facility can also interrupts” (remember that
be used to dynamically such requests are caught by
extend the services RTAI). If interrupts are
provided to user-space enabled, the Linux interrupt
programs. dispatcher will be called
upon. Otherwise, RTAI
To provide virtual interrupt immediately exits the
control to Linux, RTAI
interrupt context and programmed in various
returns to normal program ways.
execution. When the Linux
interrupt dispatcher is On that architecture, the
summoned, it may be timer may be programmed
followed by the system to emit interrupts at fixed
request dispatcher which time-intervals, in periodic
will call upon all SRQs mode, or may be
which may have been reprogrammed at each
activated by an interrupt interrupt, in one-shot mode.
handler. It is using this Periodic would seem ideal
facility that RTAI modules as it costs approximately 3
may have access to the microseconds to reprogram
standard non-deterministic the timer, but it may be ill-
kernel facilities. Once SRQs adapted to certain
are called, if any, RTAI situations where intervals
proceeds on jumping to the may vary. Hence the one-
normal ret from intr code to shot mode which has also
finish the normal return the advantage of
sequence. reprogramming intervals
using the CPU frequency
and not the timer frequency
which, on the PC
architecture, is considerably
Schedulers
lower.

On the PowerPC
The RTAI schedulers use the
architecture, the existing
facilities and abstractions
timing facility is embodied
provided by the core RTAI
by the decrementer and
module described in the
provides a mechanism akin
previous section to provide
the one-shot mode of the
high-level programming
8254 found on the PC. In
abstractions to the
actuality, the processor’s
programmer such as tasks
internal counter is used to
and various communication
generate an interrupt at a
mechanisms. There are
given time and needs to be
three schedulers that come
reprogrammed upon firing.
with RTAI: a uniprocessor
Periodic mode is therefore
(UP) scheduler, an SMP
attained on this architecture
scheduler and a Multi-
by reloading a constant
Uniprocessor (MUP)
value.
scheduler. All these
schedulers can be used
The following subsections
either in one-shot mode or
provide details about each
in periodic mode. These
scheduler available. Keep in
modes relate to the way the
mind that regardless of the
timer is programmed and
differences, the facilities
take their roots from the PC
provided by RTAI remain
architecture where the
the same and need not be
8254 timer may be
used differently from one SMP scheduler
scheduler to the next.

The SMP scheduler differs


Uniprocessor scheduler from the UP scheduler in
that it can schedule tasks to
more than one CPU. This
The uniprocessor scheduler involves different degrees of
enacts a scheduling additional services such as
algorithm to select a task to the capability to set CPU
be run on a single CPU. As affinity for a task or to
such, its operation is very assign an IRQ to be dealt
much straight forward: with on a specific CPU. Also,
whichever process has the contrary to its UP
highest priority gets the counterpart on the PC
CPU. In effect, it is a multi- architecture, the SMP
list priority based scheduler scheduler is not limited to
with support for priority the usage of the 8254 as the
inheritance. In this scheme, single source of timer
Linux is a real-time task as interrupts since the APIC
any other but remains at the possesses its own timer. The
lowest priority level. SMP scheduler remains,
though, a prioritydriven
As for the implementation of scheduler.
the scheduler proper, it is
split between two different Multi-
yet complimentary Uniprocessor
functions: rt schedule() and
rt timer handler(). The scheduler
former is invoked by the
different facilities to enforce
a scheduling change to As the name suggests, the
reflect a modification in the multi-uniprocessor
state of a process. The later scheduler views a
is exclusively targeted at multiprocessor machine
dealing with the timer (SMP) as being a collection
interrupt. At first it may of many uniprocessors. This
seem that these means that each CPU can
functionalities should be have its timer programmed
grouped together to form a differently. Hence, one could
single scheduling function have a CPU running in
but given the slightly periodic mode while
different approaches and another running in one-shot
uses, a choice was made to mode. This is very useful for
keep them separate. some types of applications
but involves setting CPU
affinity at task creation,
although it may be migrated
later using the proper
facilities.
Communication • rt mbx init() Initializes
facilities and other a mailbox with a given
services size.

• rt mbx delete() Deletes


One of the main features of
RTAI is the wide array of the resources used by a
communication facilities mailbox.
and other services made
available to the • rt mbx send() Sends a
programmer throughout the
different schedulers while sized message to a
providing identical given mailbox.
interfaces. Some of these
services are part of the
scheduler modules as they • rt mbx receive()
are simple enough in Receives a sized
implementation while
providing basic message from a
communication facilities.
mailbox.
Others, more complex or
less common, are
implemented within their Timed and conditional
own separate modules. The versions of the send/receive
following subsections primitives also exist. These
discuss each of the services can be used to provide the
provided by RTAI. programmer with greater
control over the way his
requests are dealt with by
the system.
Mailboxes

Mailboxes provide for a way


Messages and RPCs
to exchange data between
multiple tasks using a Unlike mailboxes, messages
pointer to a mailbox and RPCs are taskbased.
structure as the reference One sends a message to or
point. Typically, this will be receives a message from a
used to send n bytes from a task. There is, therefore, no
given buffer to a specific need to instantiate or
mailbox. On the receiving initialize any structures or
end, the reader can read m identifiers proper to
bytes from the mailbox into messages or RPCs other
its own buffer. The than ensuring that the
following is a sample of the recipient or source of
services provided by the messages is actually a live
mailbox facility: task. This is somewhat
explicit as all the calls to this
facility require the passing
of a pointer to the differently by the
designated task. Hence, if programmer.
we are waiting for a
message from task X, we
need to pass the message Semaphores
API the pointer to task X’s
structure. It is still possible
to receive a message from Semaphores are a basic
any task by passing a NULL synchronization mechanism
pointer to the message API that enables multiple tasks
when asking for such a to coordinate their work in
reception. Also, the a coherent way. In RTAI,
messages being transferred semaphores are identified
are not variable sized using their structures.
message, but are fixed size Hence, tasks wanting to
unsigned integer values. synchronize their work
The following is a sample of using a given semaphore
the services provided by the will need a pointer to that
message and RPC interface: semaphore’s structure. In all
other respects, RTAI
semaphores behave the
• rt send() Sends a same way as conventional
message to the given semaphores do. Here is a
task. sample of the semaphore
API:
• rt receive() Receives a
message from a given • rt sem init() Initializes
task. a semaphore to a given
• rt rpc() Sends a value.
message to a given task
• rt semdelete() Deletes a
and expects a reply. given semaphore.

• rt isrpc() Determines
• rt semsignal()
whether the given task Signals a
semaphore.
is waiting for a
response to an RPC. • rt semwait() Waits on a
• rt return() Replies to an semaphore.

RPC from a given task.


The wait call has timed and
Some of these services have conditional variants which
timed and conditional may be useful in some
variants. It is the case of the situations.
send, receive and RPC calls
which may need to be used
FIFOs • rtf create() Creates fifo
with a given size and
Contrary to the three ID.
previous facilities, the FIFO
facility is a separate module • rtf destroy() Destroys a
that is optionnaly loaded if fifo.
useful. As its name implies,
this facility provides tasks • rtf reset() Empties the
with a way to put data content of a fifo.
within a buffer which will
then be read on a first-in- • rtf put() Puts data in a
first-out basis. The main fifo.
usage of the FIFO module is
the sharing of data from/to
user space to/from real- • rtf get() Gets data from
time tasks. When used from the fifo.
within a kernel module, the
FIFO API identifies a FIFO
using its ID. In user space, • rtf create handler()
this ID corresponds to an
Associates a handler to
entry in the /dev directory.
Fifo 1 is visible as /dev/rtf1 deal with the addition
from user space, for of data to the fifo in an
example. Using this, a real- asynchronous way.
time task can collect data in
real-time while making this
data available to a normal In addition to these services,
Linux process that is not semaphore primitives have
bound by any real-time been added to provide for
constraint. Data acquisition the synchronization of the
is one of the uses of this access to the fifos. Also, it is
facility. From user-space, a now possible to name the
task can communicate fifos being created in order
through this facility by using to increase the flexibility of
the conventional open, read, the facility. Using this,
write, close and other identification of a
services of the Unix file API. correspondent depends on
To the user application, the knowing the name of the fifo
FIFO is just another file in he uses.
the system.

The following is a sample of


the API available to Shared memory
modules:

Another way of sharing data


between execution domains
is through shared memory.
To this end, RTAI provides a
shared memory facility. invited to take a look at the
Basically, this module extensive documentation
provides for the allocation available on RTAI’s web-
and freeing of memory site.
regions. Identification of
these memory regions is Memory management
done using a name scheme
which will ensure that
further allocation of the A memory management
same name will only result facility may be seen as the
in the mapping of the wrong type of service to
designated region to the provide in a deterministic
process’s memory map hard-real-time system, but
while providing the caller it has some very practical
with a pointer to said uses. Least of which,
region. As with the FIFO support for higher-level
facility this is an optional languages like C++ which
module that, once loaded, is require the existence of the
usable by both user-space new and delete operators.
applications and realtime The algorithm used by the
tasks. Communication of memory management unit
user space requests to the has been designed to
shared memory module is provide for real-time
provided by the use of the memory allocation. This
SRQ mechanism described works by initially reserving
above. a chunk of memory from the
kernel using the
conventional means.
Thereafter, chunks of
Posix
memory are provided upon
request to the callers of rt
malloc() using a
As with other fields of deterministic algorithm.
computer science, the Freeing of the request
realtime field possesses its memory is done using the rt
share of standards. One free() call. The new and
such standard is the Posix delete mechanisms are
standard for real-time. based on those basic
Actually, there are multiple memory management
Posix standards for real- primitives.
time. The RTAI Posix
module implements the
1003.1c pthreads standard Watchdog
and a part of the 1003.1b
standard, the message
queues. It is not the In an effort to further insure
intention of this paper to that RTAI is a safe
discuss the APIs provided programming environment,
by the Posix standard, but a watchdog facility has been
the interested reader is implemented. This facility
can be used to insure that
no one task will freeze the
system because of its User Space

misbehaviors. Using the Kernel Space


watchdog facility, it is
possible to ensure that
infinite loops and task
scheduling overruns2 do not Figure 3: Possible LXRT
handicap the system’s symmetrical
ability to continue operating communication.
by enforcing a configurable
reaction to such
occurrences. It is therefore have shown that
possible to suspend communication through
offending tasks or even kill LXRT is very fast.
them.

User space services

LXRT symmetrical
interface Communication from user
space to kernel space is
possible through a software
Of all the services and interrupt handled by LXRT.
abstractions provided by Using this software
the different RTAI modules, interrupt, user space tasks
LXRT remains the most can call on exported RTAI
flexible and the most services in very much the
complex of them all. By same way they call on
providing the programmer exported Linux system calls.
with a symmetrical Among the services
programming interface, exported to userspace using
LXRT integrates the best of LXRT, we find all the
both worlds in the hybrid services previously only
GPOS/RTOS combination. In available to loadable
effect, it provides user modules such as mailboxes,
applications with means to messages, RPCs and
communicate transparently semaphores. In actuality,
with real-time tasks and the LXRT API makes it
vice-versa. Figure 3 shows possible to render usage of
the possible communication these services to be
interactions between tasks completely transparent to
belonging to different the context. In other words,
execution domains. Note one can use the same
that tests functions and semantics in
either user space or kernel
space with the same effect.
The only difference being
2 When a task is rescheduled
the usage of a main()
before it had the time to complete
its intended job. function instead of an init
module() and cleanup GPOS/RTOS configuration
module() interfaces. This tasks could either belong to
makes it possible to one domain or the other and
effectively test real-time would be programmed
applications in user space differently depending on the
prior to inserting them as domain they belonged to.
kernel modules. However, The initial user space
note that user space services provided by LXRT
applications using LXRT to blurred this divide. The
access RTAI services are not addition of a routine
hard-real-time tasks, they enabling normal GPOS tasks
are only soft-realtime tasks. to become RTOS tasks takes
Although, as we will see in this further by providing a
the next section, they can means for normal Linux
become hard-real-time processes to become hard-
tasks using LXRT. In any real-time bound tasks
case, prior to using any through the use of the rt
other LXRT services, the make hard real time() call.
user space applications Contrary to loadable
need to instantiate a real- modules, such real-time
time shadow task which will tasks run in their own
be used to maintain isolated memory space and,
coherent data structures hence, provide for memory
within RTAI while dealing protection of real-time
with intertask tasks.
communication and
scheduling. Process stealing is done in
two steps. The first part of
With that said, LXRT is not the transition is done as
limited to the predefined set part of the call made by the
of exported services and Linux process and consists
may be extended quite of the following sequence:
easily by providing an
alternative function table
which includes the initial 1. Disable global
table while adding to it the interrupts.
extra entries required. Such
2. Set Linux process state
an extension is used by the
to TASK LXRT OWNED.
RT COM module to provide
real-time com port 3. Raise the priority of the
communication to tasks idle task (this is
through LXRT. necessary for the
second part of the
transition).
Stealing tasks from
4. Enqueue the function
Linux dealing with the
secondstep as part of
the normal Linux
It was originally thought
that with the hybrid
IMMEDIATE tasks to return to soft-real-time,
queue. as a normal Linux process,
using the inverse of the
5. Mark the IMMEDIATE above steps.
bottom half to run.

6. Call the Linux


scheduler. Exception handling

7. Enable global
interrupts. Given the memory
protection possible with the
8. Reset idle task to its process stealing method
original priority. and the growing need to full
(and fool) proof real-time
programming, it becomes
After this first step, the
useful to implement
Linux process is in a state of
exception handling to
limbo and will remain in
enforce protection policies
this state until the second
and provide for other
part of the stealing process
capabilities such as
is carried out. This second
debugging. For this purpose,
part will come to run within
LXRT now handles
the standard bottom-half
processor exceptions. As
framework in Linux and
RTAI is the first to receive
consists of the following:
processor exceptions, it
provides for identifying the
1. Disable global current execution context
interrupts and passes exceptions onto
the Linux exceptions
2. Set the real-time task’s handlers whenever
state as READY necessary.
3. Run the LXRT
scheduler
QNX-like services
4. Enable global
interrupts
As some RTOSes have been
widely used and adopted for
The LXRT scheduler will different uses, it is desirable
take the necessary steps to to being able to use the
insure that the task runs in same functionalities on
a consistent memory open realtime kernels. Such
configuration and will is the case with the the
interact with the other RTAI synchronous IPC services
modules to provide for provided in LXRT akin
scheduling of the task as if it similar QNX services. This
were yet another RTAI task. service enables tasks to
communicate together
Just as it was possible to synchronously using name
transition into hard- schemes to locate
realtime space, it is possible
recipients. In addition to • rt Proxy attach()
synchronous
Attaches a proxy to a
communication, this facility
also adds raw proxies given task.
functionality. Proxies are
real-time tasks which can • rt Proxy detach()
send a predefined message
Detaches the proxy of a
to a waiting task and hence
trigger a certain behavior. given task.
Proxies may be used within
interrupt handlers to signal • rt Trigger() Triggers
a certain event to a waiting the action of a proxy.
task; provided that the
trigger is the last action
taken by the handler.
Unix Server
The following is a sample of
the services provided by As real-time tasks do not
this facility: have access to standard
Linux services many ways
• rt Name attach()
have been provided to
Attaches a name to the circumvent this limitation,
the unix server capability
current task. from LXRT is one of them.
• rt Name locate() By starting a unix server
prior to entering hard-real-
Locates a task time mode, a Linux
processor can have access
identifying itself with
to some of the most
the given name. commonly used Linux
services. Starting a unix
• rt Name detach() server is done through the
rt start unix server() call. In
Detaches a name from
effect, starting a unix server
a given task. forks the current process to
execute an agent who will
• rt Send() Send a be in charge of the non-
message to a task and deterministic
communication with Linux.
wait for an answer. Exchanges between the
agent and the real-time
• rt Receive() Receives a
tasks are done via a shared
message from a given memory region to minimize
overhead.
task.
The following services are
• rt Reply() Reply to a
provided by the unix server:
received message.

• rt scanf()
• rt printf() • aio open()

• rt open() • aio close()

• rt close()
• aio return()
• rt write()
• aio cancel()
• rt read()
• aio fsync()
• rt select()

To deal with I/O requests,


• rt lseek()
threads are started and
• rt sync()
handled by LXRT to carry

• rt ioctl() out the requested service.

5.7 User library


Note that calls to Linux
services via the unix server (liblxrt)
remain non-deterministic
and the caller will have to
wait for Linux to complete In an effort to provide
servicing the request before developers with a way to
continuing its operations. develop their applications
without having to run a
modified kernel or RTAI, a
Asynchronous I/O user library LXRT has been
provided. This library
provides for standard
The asynchronous I/O interfacing with LXRT for
recent addition to LXRT development and enables
provides programmers with development to be carried
an asynchronous I/O based out in parallel.
on the glibc sources but
adapted to provide the same Future directions
functionality within the
LXRT framework. This
provides programmers with As RTAI is constantly
the following services: evolving, there are different
future directions which will
be investigated while
• aio read() continuing to improve the
currently available facilities.
The following is a non-
• aio write()
exhaustive list of things to • Integration of Linux
come: Trace Toolkit hooks.

• Using RTAI services on


• More ports of RTAI RTLinux.
(including ports of
LXRT)to other
• Standalone RTAI.
architectures. As it
stands now, prime
• Port uClibc to RTAI
targets are the MIPS kernel space.
and the ARM
processors, but others • Fix uClibc to work with
are also being LXRT seamlessly.
considered. As for
LXRT, it is currently
functional on i386 only • Better testing suite.
and there is a desire to
have it running on the • Standard real-time
PPC too. development
environment.
• Extensive framework
for C++ • Multiple interrupt
programmingfor RTAI. priorities.
This is not limited to
having C++
• Latency verification of
code paths.
code run with RTAI, but
having a real
framework that would • More advanced
be usable both from a memory management.
loadable module
standpoint and from a
user application Many other enhancements
standpoint. are possible and the RTAI
development team is open
• Real-time RAM to any suggestions and
filesystem. contributions.

• Flash-based filesystem. Acknowledgements

• POSIX I/O layer to RTAI is the collective work


support filesystem. of a team of developers
which is built on a tradition
• Integration of RTNet of openness and
and socket layer. cooperation. The rapid
development of RTAI under
Paolo Montegazza’s lead
and its mainstream
adoption are a testament to
this effort. Hence, a special
thanks to all the RTAI
developers (listed in no
particular order, except for
Paolo):

Paolo Montegazza
Stuart Hughes
Lorenzo Dozio
Trevor Wolven
Giuseppe Renoldi
Tomasz Motylewski
Pierre Cloutier
Steve Papacharalambous
David Schleef
Ian Soanes
Emanuele Bianchi
Brendan Knox
Erwin Rol
Karim Yaghmour

A great deal of thanks goes


out to all the RTAI users and
supporters who have made
this effort all the while more
enjoyable.

References

http://www.aero.polimi.it/
projects/rtai/

You might also like