Computer Networks 55 (2011) 31543168

Contents lists available at ScienceDirect

Computer Networks
journal homepage: www.elsevier.com/locate/comnet

A methodology for testing intersection related Vehicle-2-X applications

Sebastian Rglinger
Institute for Applied Research, Ingolstadt University of Applied Sciences, Esplanade 10, 85049 Ingolstadt, Germany

a r t i c l e

i n f o

Article history:
Available online 21 June 2011
Keywords:
Test methodology
Vehicle-2-X communication
Environment stubbing
Intersection related applications
Proof of concept

a b s t r a c t
Intelligent Transportation Systems based on Vehicle-2-X communication have been a
major research topic for several years. Meanwhile, deployment strategies for the used
Vehicle-2-X systems need to be developed not only to coordinate the market introduction
but also to manage the manufacturers development projects to achieve market-ready
products. Such development processes include among others testing periods as major
tasks. Thus, the deployment of market-ready Vehicle-2-X system demands for test methodologies designed for that purpose.
Among the rst introduced Vehicle-2-X applications, intersection related ones will play a
major role because of their information ows simplicity. So, the near future calls for test
methodologies particularly for those applications because testing is important for the
development of market-ready systems. For that reason, a test methodology is presented
which treats systems implementing intersection related Vehicle-2-X applications as systems under test. The stimulation of the system under test is based on input streams and
the evaluation is based on output streams. In addition, results of a proof of concept using
the Audi travolution system are enclosed.
2011 Elsevier B.V. All rights reserved.

1. Introduction
Novel Intelligent Transportation Systems (ITS) based on
Vehicle-2-X (V2X) communication comprises vehicles-tovehicle and vehicle-to-infrastructure communication in order to accomplish the next level of cooperative advanced
driver assistance systems (ADAS). According to [13] the intended applications can be categorized into: safety, transport efciency, and infotainment/other applications. The
applications themselves gain high complexity levels because V2X communication will be realized by different
communication technologies and has to cope with a highly
dynamic and non-deterministic environment and trafc
situations. According to [2,4], at least in Europe, the communication network will consist of Ad-Hoc communication
based on Dedicated Short Range Communication (DSRC) (e.g.
WiFi using IEEE 802.11 p [5] or ETSI ES 202 663 [6]) and cellular mobile communication (e.g. 3G and beyond).

E-mail address: sebastian.roeglinger@haw-ingolstadt.de

URL: http://www.haw-ingolstadt.de
1389-1286/$ - see front matter 2011 Elsevier B.V. All rights reserved.
doi:10.1016/j.comnet.2011.06.014

According to [7], most of the intended V2X applications

require a high market penetration rate in order to achieve
proper functionality. In contrast, intersection related V2X
applications do only require one equipped intersection
plus one equipped vehicle within DSRC communication
range or cellular mobile communication facilities to provide an additional value to drivers of equipped vehicles.
So, intersection related V2X applications will most properly be among the rst available V2X communication
applications. Indeed, current development activities and
eld trials (see [8]) already deal with such types of
applications.
V2X communication systems require a high system
quality already during the market introduction phase. This
is relevant for an early user acceptance and mandatory for
safety applications. Consequently, effective and efcient
test methodologies for the rst available applications
and thus for intersection related applications are required
right now. So, serious deployment strategies for V2X communication systems should consider that a successful introduction of novel systems demand for sophisticated test

S. Rglinger / Computer Networks 55 (2011) 31543168

phases which may be time-consuming. To make testing of

such a novel type of applications manageable, test methodologies targeted to intersection related V2X applications are
introduced.
However, testing in the automotive domain is strongly
assisted by simulations. But, especially in the V2X communication domain, there are currently no validated simulation models (e.g. radio propagation models) available.
This is a consequence of the lack of yet introduced V2X
communication applications which are required to validate
simulation results in true-to-life circumstances. Therefore,
at present, it is not possible to test the functionality of V2X
communication applications prior to expensive prototype
test phases. However, those tests are time-consuming
and expensive because the prototypes have to be built up
rst. To overcome this problem, the rst introduced applications require test methodologies which can yield to reliable test results with a limited number of real test drives.
However, real test drives could not be omitted. So, the goal
is to provide more prot from the essential real test drives.
Therewith, deployment projects could be supported to
realize them in budget and in time.
To achieve a well suited test methodology, the introduced test methodology targets on high-level functional
testing and high-level non-functional testing of the vehicles side of intersection related V2X systems. However,
the test levels component testing, integration testing and
acceptance testing are not in focus of this test methodology. But, the system test level is addressed by exploiting
real-world test drive traces.
The remainder of this paper is structured as follows. In
Section 2 V2X communication systems are briey introduced and describe intersection related V2X applications
more in-depth in Section 3. Thereupon, test needs of intersection related applications including requirements on a
test system are presented in Section 4. Furthermore, the paper introduces a stream based test approach in Section 5
and shows its feasibility by a proof of concept in Section 6.
2. Overview of V2X application architecture
This section provides a brief overview of the V2X hardware and software architecture, and message types. Since
research and development in this area is not nalized yet,
minor or even major naming and architectural changes
may happen before the market introduction. Especially for
the reason, that different naming and architectural denitions are in use which have not been consolidated up to now.
2.1. Hardware architecture
As described in [911] the European V2X communication architecture will most probably consist of four main
ITS communication components. Those components are
ITS Vehicle Stations (IVSs), ITS Roadside Stations (IRSs), ITS
Central Stations (ICSs), and ITS Personal Stations (IPSs). All
these components inherit their internal architecture from
a generic ITS station.
All those stations will be equipped with various communication channels to be able to cooperate with a variety

3155

of partners. For cooperation with the local environment,

DSRC based Vehicular Ad hoc NETworks (VANETs) will be
used. The especially adapted WiFi standard IEEE 802.11 p
[5] which is an amendment of the IEEE Std. 802.11 [12] will
play a very important role in that case [2,9]. A short introduction about the draft standard IEEE 802.11 p can be
found in [13]. By now, ETSI has also started its work on
standardizing the MAC layer of DSRC based VANETs in ETSI
ES 202 663 [6]. Moreover, to prevent DSRC from suffering
from excessive interference, resulting from other systems
or services between 5875 MHz and 5905 MHz, a dedicated
frequency band has been allocated by the European Commission [14,15].
To be able to exchange information in situations with
no local cooperation partners (e.g. shortly after market
introduction, by night, or rural locations as reported in
[16]) cellular mobile communication could be used as an
alternative communication channel. This channel will provide access to the Internet and therefore to ICSs.
2.2. Software architecture
According to [9,10], the ITS software architecture will
most probably consist of different layers. The horizontal
layers are: ITS Applications, ITS Facilities, ITS Network &
Transport and ITS Access Technologies. These layers are supported by the vertical layers ITS Management and ITS
Security.
2.3. DSRC message types for ITS applications
Details about the DSRC message types are not of major
interest in this paper, so, only some of the most relevant
literature for further reading is linked. There are several
basic DSRC message types like:
 Cooperative Awareness Message (CAM),
 Decentralized Environment Notication Message
(DENM),
 Intersection Topologies, and
 Signal Phase and Timing (SPAT) of trafc lights.
The CAMs and DENMs have been already specied by
ETSI in: ETSI TS 102 637-2 [17] for CAM, and ETSI TS 102
637-3 [18] for DENM. Beyond that, the messages content
denitions are strongly connected with the SAE J2735 Dedicated Short Range Communications (DSRC) Message Set Dictionary [19]. However, the message formats are not
nalized yet. Since this paper targets on intersection related V2X applications, only intersection topology messages, and SPAT messages are relevant for this paper.

3. Description of intersection related applications

This section gives a more detailed description of the
information ow between those parts of the V2X facilities
which are strongly concerned with intersection related
V2X applications. Moreover, typical intersection related
V2X applications are introduced.

3156

S. Rglinger / Computer Networks 55 (2011) 31543168

3.1. Information ow

ICS
Intersection related applications comprise a special
type of V2X applications which are perfectly convenient
for the introduction phase of V2X communication systems.
That is the case because of the simplicity of the information ow. If DSRC is used, the information ow consists
of a unidirectional broadcast of single-hop transmissions
from the intersections IRSs to vehicles IVSs (see Fig. 1).
If cellular mobile communication is used, IVSs build up cellular mobile communication connections and download
the required information about near intersections from
an ICS (see Fig. 2). However, this does not change the information ow within vehicles substantially.
The relevant information to realize intersection related
V2X applications comprise the intersection topology per
intersection (for example see Fig. 12) and the signal state
timing per each lane of each intersection (for example see
Fig. 11) [1820]. Both are represented by messages which
are introduced in Section 2.3. The intersection topologies
are modeled by an intersection ID, vectorized driving lanes,
driving lane numbers, suitable connection points, stop
lines, and more. The signal state timing data are modeled
by an intersection ID, lane numbers, minimum time to
change, maximum time to change, likely time to change,
and more. Both message types are broadcasted periodically
by intersections IRSs or could be downloaded from ICSs.
Although the vehicles Electronic Control Units (ECUs)
provide relevant information to realize intersection related
V2X applications which is currently already exchanged by
vehicle internal bus systems like the CAN-Bus. Typical relevant vehicle internal information are among others: vehicle speed, single wheel speed, engine speed, yaw rate,
direction indicator status, or fog light status. So, the access
of those data could easily be realized by vehicle manufactures and, thus, be used by V2X implementations.
Finally, an IVS collects all those information from its
vehicle and its environment and uses it to realize the applications functionalities. In addition, the applications could
use additional facilities of vehicles (e.g. HMI or signaling
tones) to interact with the drivers. Examples of those
applications are described in the following Section 3.2.
3.2. Examples of applications
According to [3,9,21], typical intersection related applications are among other:

IVS

ECU

IRS

IRS

1..*

<<flow>>
backbone

ICS

<<flow>> 0..*
cellular mobile

0..*

IVS

<<flow>> 1..*
vehicle intern

ECU

Fig. 2. Information ow of cellular mobile based intersection related V2X

applications.

Green-light Optimal Speed Advisory (GLOSA) Drivers

receive a recommendation in order to hit the next trafc lights in green phase and to avoid waste acceleration. [9]
Red-light Violation Warning (RVW) Warns drivers
when they are going to violate a red trafc light signal.
[9]
Typically, the warnings occur in a double-staged
approach. E.g., the rst stage realizes a visual or audible
signaling; whereas, the second stage realizes a short
jerky braking maneuver if the driver shows no reaction
on the rst signaling.
Trafc Light Assistant (TLA) If the vehicle has to stop
because of the trafc light signaling, the vehicle shows
the remaining waiting time of the corresponding signaling group to the driver [21].
4. Test needs
Testing software intensive systems is a complex and
permanent task throughout the software life cycle and
should therefore be done very efciently and effectively.
Moreover, the test process quality has signicant inuence
on the resulting products quality. Since no V2X applications have already been brought into market so far, no sufcient experiences in testing of those novel systems are
available. Therefore, the test needs of intersection related
V2X applications are analyzed from the literatures point
of view. Finally, based on these analysis requirements on
the test system are derived.
4.1. Test levels and test types

IVS

ECU

IRS

IRS

0..*

<<flow>>
DSRC

IVS

<<flow>>
vehicle intern

1..*

ECU

Fig. 1. information ow of DSRC based intersection related V2X

applications.

According to [22], test levels ordinary comprise component testing (unit testing), integration testing, system testing, and acceptance testing; whereas, the transitions are
smooth (see Fig. 3). As component testing and low-level
integration testing targets low-level functional testing
and low-level non-functional testing close to software
implementations, test methodologies for these parts of test
levels are already available (see [2225]). So, testing
activities closely related to software implementations of
V2X applications should be organized analog to common

S. Rglinger / Computer Networks 55 (2011) 31543168

software tests. To avoid contradictions, the test levels and

test types are dened as follows:
component testing Testing of individual hardware
or software components or
groups of related components.
[26];
integration testing Testing performed to expose
defects in the interfaces and in
the interactions between integrated components or systems.
See also component integration
testing, system integration testing. [27];
system testing The process of testing an integrated system to verify that it
meets specied requirements.
[28];
acceptance testing Formal testing with respect to
user needs, requirements, and
business processes conducted
to determine whether or not a
system satises the acceptance
criteria and to enable the user,
customers or other authorized
entity to determine whether or
not to accept the system. [27]
after [26];
functional testing Testing based on an analysis of
the specication of the functionality of a component or system.
[27];
non-functional testing Testing the attributes of a component or system that do not
relate to functionality, e.g. reliability, efciency, usability,
maintainability and portability.
[27].
In contrast, testing activities for high-level integration
testing and system testing, namely, high-level functional
testing and non-functional testing could not be handled
adequately with available test technologies and methodologies. Except for the hardware equipment required to
emulate the V2X communication partners and vehicles

acceptance testing

check system behavior

against consumers
requirements
high-level

system testing

integration testing

functional and
non-functional testing

component testing
low-level
Fig. 3. Test levels with test types.

3157

ECUs, a test system has to be able to handle diverse

requirements in order to address the specic needs of
V2X applications. Section 4.3 establishes a proposal for
those requirements.
Additionally, acceptance testing is another wide area of
testing software intensive systems; especially, for cooperative V2X applications. For intersection related V2X applications, acceptance testing has to target among others: the
driver acceptance, inuences on a road networks trafc
throughput, or the trafc ow smoothness. So, this area
of testing should be considered separately.
This also applies for testing safety, security, and privacy
aspects of ITS systems in general and, thus, also for intersection related V2X applications. Since ITS systems include
various communication interfaces, there might be much
potential for security violation attacks. Moreover, vehicle
take part in road trafc and, thus, security violation attacks
might induce serious road accidents. Therefore, also for
safety, security, and privacy, separate test methodologies
should be explored and applied.
In summary, testing high-level functional and nonfunctional aspects of intersection related V2X applications
opens a differentiated and unsolved area. So, this paper focuses on these two aspects of testing.

4.2. Testing of system characteristics

As described in Section 2.2, the software architecture of
V2X systems will most probably consist of a stack architecture. Here, multiple applications and underlaying run-time
platform components (e.g. middleware or the operating
system) will run on one system simultaneously. So, the
workload of those systems varies over time and thus the
execution time of algorithms and the response-time to
interrupts is non-deterministic. The relevant effects which
lead to non-deterministic systems (e.g., scheduling algorithms, concurrency) are described in [29]. Moreover, the
whole system might be distributed over multiple ECUs
which are connected over communication networks with
non-deterministic timing properties (e.g. CAN-Bus). So,
the timing characteristic of intersection related V2X systems should be targeted as non-deterministic. For that reason, functional and non-functional testing which targets
the timing of the systems should be repeated several times
and evaluation statistically.
Moreover, different environmental and vehicle internal
information sources and information ows of IVSs might
suffer from communication channel disturbances, measurement and intermediate calculation inaccuracies, or
other negative effects. Since it is impossible to resolve
the effects of input signal variations to the applications
outputs, statistic test methodologies have to be applied.
By using randomized algorithms as described in [30] for
test input signal modications and statistical evaluations
of resulting applications reactions and outputs, the former
problem could be addressed.
Finally, for testing activities targeted to non-functional
aspects of the system, at least performance testing, load
testing, stress testing, and reliability testing should therefore be taken into account [22].

3158

S. Rglinger / Computer Networks 55 (2011) 31543168

4.3. Requirements on the test systems

In this subsection, a proposal for requirements on test
strategies and test architectures for intersection related
V2X systems are presented. These requirements are derived from the intersection related V2Xs architecture (Section 2) and the test needs (Section 4).
Requirement 1 Emulation of the specied behavior of
cooperation partners who are connected via DSRC.
Requirement 2 Emulation of inuences of wireless
communication on the DSRC communication. Suitable inuences are among
others radio interferences, package collisions, communication channel congestion, fading, or shading [31,32]. Those
inuences show according to [33,34]
effects on package loss rates and statistical distributions of package transmission delays and might be caused by
communication of other vehicles or by
environmental objects like buildings or
trees.
Requirement 3 Emulation of the specied behavior of
cooperation partners who are connected
via cellular mobile communication.
Requirement 4 Emulation of inuences of cellular
mobile networks on the communication
between ICSs and IVSs. Suitable inuences are among other: network coverage, and available local eld strength.
Those inuences show effects on: probability of connections loss, round trip
time, channel throughput, and others.
Requirement 5 Emulation of the specied behaviors of
other ECUs which are also located
inside the vehicle.
Requirement 6 Emulation of the vehicles self localization facilities. In most cases, GPS and
odometer based techniques are used.
Requirement 7 Test runs have to be reproducible
exactly which addresses synchronization problems. Those problems may
appear by reapplying test runs on distributed systems.
Requirement 8 It must be possible to modify the test
input data in order to adjust their quality (e.g., add noise or reject messages).
Requirement 9 It must be possible to arbitrary combine
various test input data to stress different test cases.
Requirement 10 It must be possible to build test verdicts
based on statistic evaluation of test
runs.
To avoid contradictions, the term emulation is dened
by technique of one machine obtaining the same results
as another [35] whereas simulation is dened by the
technique of representing the real world by a computer
program; a simulation should imitate the internal pro-

cesses and not merely the results of the thing being simulated [35].
Summing up, test strategies and test architectures for
intersection related V2X applications have to fulll all or
at least a subset of the above established requirements.
In case that just a subset is fullled, not all test needs could
be satised. E.g., the test approach, presented in Section 5,
uses a two-layered strategy to use real-world traces for
test input stimulation, whereas, the rst layer realizes
not all requirements.

5. Approach
In this section, an approach on how to test intersection
related V2X applications is presented. The approach
emphasizes the points identied in Section 4:
 test levels: high-level integration testing and system
testing,
 test types: high-level functional and non-functional
testing,
 system characteristics: test input modication and statistical test output evaluation.
Moreover, the approachs design fulll the requirements established in Section 4.3 by adapting a channel
and stream concept.
5.1. Identication of system under test
For testing functional and non-functional aspects of distributed systems, like intersection related V2X applications, all involved components have to be considered.
However, as described in Section 3, most of the complexity
of intersection related V2X applications is located in IVSs.
Hence, IRSs are also completely new products; those types
of stations should also be tested. However, the complexity
of the IRSs application implementations does not reach
the difculty of the IVSs application implementations. Furthermore, ECUs are already well tested and not modied
because they are only information sources. So, they do
not need extra testing in terms of testing intersection
related V2X applications. For all those reasons, most
testing activities have to take IVSs into account. Therefore,
IVSs are treated as System Under Test (SUT) in the test
methodology.
5.2. Test strategy
The test strategy comprises the following two points:
stimulations of SUTs to stress test cases, and evaluation
of test results. The realization of these is shown for intersection related V2X applications in the following subsections. Since both points are closely related to the channel
and stream concept, the next subsection describes how
this concept could be applied for intersection related V2X
applications. The following subsection treats stimulation
and evaluation techniques, and the last subsection handles
denitions of assessment strategies.

3159

S. Rglinger / Computer Networks 55 (2011) 31543168

5.2.1. Concept of channels and streams

The inputs and outputs of V2X applications could be
modeled by a channel and stream concept introduced by
Broy in [36]. Channels are also called ports and encompass
input and output interfaces of systems. Streams describe a
concrete ports input/output communication history and,
so, could be assigned to a channel. According to [36],
streams could be classied in one of the following four
types: non-timed streams and discrete streams with discrete and continuous time, and nally dense streams with
continuous time. Since timing is very important for V2X
applications, only timed streams are taken into account
for modeling V2X applications communication ows.
For testing intersection related V2X applications, those
streams are grouped as follows:
(1) event based continuous-time streams over a discrete
non-ordered set of messages,
(2) event based continuous-time streams over a continuous or discrete totally ordered set of messages,
(3) sampling based continuous-time streams over a discrete non-ordered set of messages, and
(4) sampling based continuous-time streams over a
continuous or discrete totally ordered set of
messages.
The term event based intends that messages appear sporadic; whereas, the term sampling based intends that messages appear periodic with a xed sampling time. The term
continuous-time means that for any message, there is a
time stamp which indicates the point in time where the
message appeared; whereas, time is represented by reals
R. The term discrete non-ordered set of messages means that
the content of a streams message is in a discrete set,
whereas, there is no relation to order its elements (e.g.
{,5, n, a}, or the Boolean set B). Discrete sets are typically
either nite or countably innite [37]. In contrast, the term
continuous or discrete totally ordered set of messages means
that the content of a streams message is in a continuous
set (e.g. in R) or in a discrete set (e.g. in N). In both cases,
there must be a relation which fullls the properties of totally ordered sets according to [38]. These types of message
sets are combined as one group to be able to handle quantized (e.g. represented by integers) and non-quantized (e.g.
represented by doubles) signal representations with the
same evaluation techniques.
In this work, another classication of streams as in [36]
is used, to achieve a better applicability of streams for the
evaluation of test result of intersection related V2X applications. As it could be seen in the above classication of
streams, the four combinations of event versus sampling
based streams and, orthogonally, streams of messages in
a discrete non-ordered set versus continuous or discrete totally ordered set are used. How this assists the evaluation
of streams is described in the next Section 5.2.2.
Additionally, it should be mentioned that there could be
sampling based streams which are only partially sampled.
This means that there are intervals in time where samples
appear and, other way round, where no samples appear.
In this paper, streams s 2 S for V2X applications are notated by sequences of messages p 2 P as follows:

P : R  U

with p 2 P

and p ht; ui

and

S : PN

with s 2 S and s hht 1 ; u1 i; . . . ; ht N ; uN ii;

with
p2P
s2S
t2R
u2U
N 2 N [ f0g

message
stream
time stamp
message content or value, and
number of messages (N = jSj)

5.2.2. Input stream modication and output stream

evaluation of test cases
There are many ways to modify a test cases input
streams (requirement 8 of Section 4.3) and to evaluate the
resulting outputs (requirement 10). Those ways are covered
by paths through the morphologic box depicted in Fig. 4. The
morphologic box needs choices for the following dimensions: test type, modication of input streams, gather output
variation, evaluation of output stream, and output stream
property. Hence, the modications (Mcon) are applied on test
cases (TC) conventional input streams (Icon) to determine
the resulting conventional and system-internal output
streams (Ocon and Osi), following notation is introduced:

TCI # Mcon;1      Mcon;a  Isi;1      Isi;b  Ind

! Ocon;1      Ocon;c  Osi;1      Osi;d ;

with
a; b; c; d 2 N
Mcon,1, . . . ,
Mcon,a
Isi,1,. . .,Isi,b:
Ocon,1, . . . ,
Ocon,c
Osi,1, . . . ,
Osi,d
Ind 2 N

indices
sets of modications of conventional
input streams Icon,n
sets of system-internal input
parameters
conventional output streams
system-internal output streams, and
index set to distinguish different test
runs
non-functional
testing

test type

functional testing

modification of
input streams

static dynamic
both
(non-static)
not
on
outside
targeted inside
bound- bound- boundaries
aries
aries

gather output
variation
evaluation of
output stream
output stream
property

no

rawstream
review

yes

metrics and assessment

correctness
of values

correctness
of timing

Fig. 4. Morphologic box: test evaluation.

3160

S. Rglinger / Computer Networks 55 (2011) 31543168

Fig. 5 depicts how input streams and their modications could be applied to SUTs. The additional timing modications T cause positive or negative delays of the
modied input streams to create new test cases by using
the same input streams (requirement 9 of Section 4.3).
Thus, a set of Ts could be seen as a conguration of different test cases using the same input streams. However, this
construct is not in scope of this section but will be picked
up in Section 5.3.
Additionally, it could be seen in Fig. 5 and (3) that there
is a distinction between conventional and system-internal
channels. On the one hand, conventional channels are also
part of the nal product. So, a is set to the nal products
number of input channels and c is set to the number of recorded nal products output channels. On the other hand,
system-internal channels are not part of the nal product
but may be necessary to adjudicate a test verdict.
To construct test cases, the morphologic box whose rst
column treats the test cases test type is used. For functional
testing, only conventional input streams and sometimes
the ow of internal communication interfaces is relevant.
So, b is set to 0 and d is set to the number of additionally,
probed internal communication channels. In contrast, for
non-functional testing, also ows of system-internal output
streams like:





CPU load,
memory usage,
network I/O, or
event count

are important. Additionally, to have inuence on systeminternal parameters (e.g. additional CPU load) is relevant
to target the test goals: performance testing, load testing,
stress testing, and reliability testing as already postulated
in Section 4.2. So, b is set to the number of modied system-internal parameters (e.g. additional CPU load for stress
testing) and d is set to the number of relevant, recorded system-internal output streams (e.g. resulting CPU load).
At second, it has to be decided if inuences of modications of input streams should be taken into account. If inuences of input stream modications are not targeted, all sets
Mcon,i = {} with i = 1, . . . , a are empty. Otherwise, the sets
Mcon,i contain potential modications of the corresponding
input streams Icon,i (see Fig. 5). Since inuences of those input stream modications to the systems outputs could not

conventional
Icon,1

IVS

value

timing

Mcon,1

Tcon,1

Mcon,a

Tcon,a

system internal

Isi,1

Icon,a

ITS Stack

Ocon,1

Ocon,c
Osi,1

be resolved analytically, randomized algorithms for determining the modications may be applied [30].
Moreover, there are these two types of input streams
regarding their variation over time:
 input streams which are static within one test run (e.g.
intersection topologies), and
 input streams which are not static within one test run
(e.g. time to green/red of trafc light signal timings).
This effects the modication of input streams as stated
so far. For the static type of input streams, a single modication of the input stream (e.g. shift) is chosen per test run.
For the non-static type of input streams, modications of
input streams (e.g. noise, glitches, drift, message loss, or
additional messages) have to be dened as mapping over
a non-modied input stream I = PN, p 2 P, and time t 2 R
such that f : R  P ! P with p0 = f(t, p).
Before modications of input streams are determined, it
has to be chosen, if the modications are inside, on, or out of
the specied tolerances. In the rst case, all modications
are inside their ranges of tolerances. In the second case,
at least one modication is chosen exactly on its tolerance
boundaries. In the last case, at least one modication is out
of its specied tolerances.
Next, V2X applications have to deal with many nondeterministic effects like communication channel disturbances, ego-positioning inaccuracies (e.g. limited GPS
accuracy), measurement inaccuracies, or non-deterministic run-time platforms. Those effects might cause variations of output streams even when reapplying the same
test case. For gathering those output variations, it is suggested to reapply a test cases input streams several times
in the exactly same fashion. The index set Ind provides the
possibility to distinguish the different test runs.
Additionally, the evaluation of output streams could
either be done by humans reviewing the raw output
streams or by using metrics and assessment strategies. Tables 1 and 2 list not exhaustive lists of metrics and assessment strategies for the evaluation of the correctness of the
streams values and the evaluation of the correctness of the
streams timing for single and multiple test runs. Since
streams combine the properties used in the tables, the
evaluation of streams requires a combination of the tables
metrics and assessment strategies.
5.2.3. Denition of assessment strategies
By using the previous denitions of messages (1),
streams (2), and test cases (3), it is possible to dene
assessment strategies formally. To show how this approach works, the approach is demonstrated with the following two examples:
(1) evaluation of distribution of points in time where
messages appear, and
(2) evaluation of inuences of input data tolerances.

System
(OS, HW, ...)

Isi,b
Fig. 5. Input and output of IVSs.

Osi,d

For both examples, the RVW application is used to

demonstrate the denition of assessment strategies. In
addition, these examples are picked up in Section 6 to
show their feasibility by a proof of concept.

3161

S. Rglinger / Computer Networks 55 (2011) 31543168

Table 1
Metrics and assessment strategies for timing of streams.
Metrics

Assessment strategies
Single run

Multiple runs

Stream property: event based appearance

Appeared/not appeared

Time stamp of
appearance

 Ratio

 Descriptive statistics

Stream property: sampling based (periodic appearance)

Jitter of sampling time
 Descriptive statistics for uctuation of
sampling interval

 Descriptive statistics for single-run results (e.g. distribution of mean

values of single runs)

Lack of samples

 Ratio

 Descriptive statistics for single-run results (e.g. distribution of ratio

of single runs)

Start/stop time stamp of

sampling

 Start and stop time stamp

 Duration of sampling

 Descriptive statistics for start/stop time stamps

 Descriptive statistics for durations

Table 2
Metrics and assessment strategies for messages of streams.
Metrics

Assessment strategies
Single run

Stream property: messages in a discrete non-ordered set

Messages value in subset of correct values
Jumping messages values

Multiple runs

 Test result

 Ratio of in set and not in set of correct messages

 Set of appeared messages

 Count of appearances per message (e.g. stored
in a set like: U  NN )

Stream property: messages in a continuous or discrete totally ordered set

Raw values
 Descriptive statistics

 Descriptive statistics

Derivatives of values

 Descriptive statistics

 Descriptive statistics

Properties of signals modulated onto the samples (e.g. ramps,

or frequencies) (see [39])

 Signal properties (e.g. max.

slope)
 Local signal properties
(within window)

 Descriptive statistics for signal properties

5.2.3.1. First example. Scenario: A vehicle approaches an

intersection with constant speed, whereas, the trafc light
will signalize red light for the vehicles driving lane at the
point in time when the vehicle will cross the stop line. The
RVW application (see Section 3.2) is active and should generate output events because a red light violation will occur.
The events content should indicate a red light violation.
Moreover, the IVS CPU could be stressed with various
additional CPU load conditions and, thus, the response
time of the RVW application might suffer.
Description: For the rst example, the following path
through the morphologic box as depicted in Fig. 4 is used:
test type: non-functional testing ? modication of input
streams: not targeted ? gather output variation: yes ? evaluation of output stream: metrics and assessment ? output
stream property: correctness of timing. Thus, the variables
are dened as follows: Mcon,1 =    = Mcon,a = {}, b = 1, c = 1,
and d = 0. So, the test case could be notated as a subset of
the mapping:

TCI # Isi;1  Ind ! Ocon;1 ;

whereas, the set Isi,1 contains various additional CPU load

conditions. Furthermore, it is assumed that the RVW applications output stream Ocon,1 is of the type: event based continuous-time stream over the set {GONG, BRAKE}. So, multiple
test runs are needed and descriptive statistics (see Table 1)
have to be applied for the output streams evaluation.

Evaluation: In order to not exceed the scope of the

example, only denitions of normalized histograms are given exemplarily. Furthermore, it is expected that the RVW
application generates two messages in this example,
whereas, the rst one has to have the value GONG and
the second one has to have the value BRAKE to follow
the double-staged warning approach. So, the histogram
Hl,x with the k histogram bars Hl;x
k is dened for the load
condition l and the points in time where the x-th message
with x 2 {1, 2} appeared as follows:

Hl;x
k :

1 n hl;ii
hl;ii
hl;ii
l
 tc 2 TCI jN con;1 2 ^ ucon;1;1
jTClI j

GONG ^ uhl;ii
con;1;2

o

BRAKE ^ w  k 6 thl;ii
con;1;x < w  k 1 ;

with

D
D
EE
tchl;ii hl; ii ! ohl;ii
2 TClI ;
con;1

ftchl;ii 2 TCI jl xg;

TClx
I
ohl;ii
con;1

D


E
hl;ii
thl;ii
;
u
;
.
.
.
;
t hl;ii
con;1;1
con;1;1

hl;ii

con;1;N con;1

; uhl;ii

hl;ii

con;1;Ncon;1


:
8

3162

S. Rglinger / Computer Networks 55 (2011) 31543168

The symbols in (5)(8) are dened as follows:

Hl,x
Hl;x
k
hl;ii

N con;1
l 2 Isi,1
i 2 Ind
k 2 N [ f0g
TClI
tchl,ii
hl;ii

ocon;1
w 2 R>0

histogram for load condition l and the xth appeared message

height of k-th histogram bar of Hl,x
number of messages in conventional
output stream 1 for load condition l and
the i-th test run
additional CPU load
index of test run
index of histogram bar
set of test cases for input streams I and
load condition l
test cases of i-th test run for load
condition l
conventional output stream 1 for load
condition l and the i-th test run, and
width of histogram bar in seconds

The histogram bars Hl;x

k as dened in (5) have all the
same width. This is not mandatory for all histograms and
thus the histogram bars might also be dened with varying
widths. Please note that Figs. 13 and 14 in Section 6.2.1
show histograms exemplary.
5.2.3.2. Second example. Scenario: A vehicle approaches an
intersection with constant speed, whereas, the trafc light
will signalize red light for the vehicles driving lane at the
point in time when the vehicle will cross the stop line. The
RVW application (see Section 3.2) is active and should generate output events because a red light violation will occur.
The events contents should indicate a red light violation.
Moreover, impacts of intersection topology center point
shifts and, thus, shifts of the whole intersection topology,
on the points in time where the RVW implementation detects the red light violation should be analyzed.
Description: For the second example, the following path
through the morphologic box is used as depicted in Fig. 4:
test type: functional testing ? modication of input streams:
static + inside
boundaries ? gather
output
variation:
no ? evaluation of output stream: metrics and assessment ? output stream property: correctness of values. Thus,
the variables are dened as follows:
longitude longitude
latitude
M con;1 slatitude
; swest
R ,
north ; ssouth R seast


 Mcon,2 =    = Mcon,a = {},
 b = 0, c = 1, and d = 0

with the following meanings for the variables:

slatitude
north

max. deviation in northern direction

slatitude
south
slongitude
east
slongitude
west

max. deviation in southern direction

max. deviation in eastern direction, and
max. deviation in western direction

respectively for the intersection topologys center point.

These variables have to be dened for the nal test case.
Finally, the test case could be notated as a subset of the
mapping:

TCI # M con;1  Ind ! Ocon;1 ;

whereas, the set Mcon,1 contains the modications of the

input stream Icon,1 which ought to represent the input
stream of intersection topologies. Hence, the intersection
topology is broadcasted periodically, every send-event requires modications. In addition, it is assumed that the
RVW applications output stream Ocon,1 is from the same
type as in the rst example.
For the assessment, test runs for different combinations
of modied input streams are needed and methodologies
from the area of descriptive statistics (see Table 1) have to
be applied. Since the impacts from intersection topology
center point shifts on the points in time where the RVW
implementation detects a red light violation could not be
handled analytically, it was decided to shift the intersection topology center point inside its tolerance boundaries
stochastically with uniform distribution in longitude and
latitude direction.
Evaluation: In order to not exceed the scope of the
example, only denitions of the minimum T xmin , maximum T xmax , and quantile T xqquantile time stamp are given:

T xmin : minseqcon;1;x ;

10

T xmax : maxseqcon;1;x

11

according to [40]:

T xqquantile : sortseqcon;1;x :dq:jseqcon;1;x je;

12

with

seqcon;1;x : ftcon;1;x jNcon;1 2 ^ ucon;1;1

GONG ^ ucon;1;2 BRAKEg;

13

which contains all time stamps tcon,1,x where a GONG

(x = 1) or a BRAKE (x = 2) message appeared. Please notice
that in (12) the sort-function returns the ascending sorted
input sequence, the rst dot denotes the access of an element of the sequence (seq.n returns the n-th element of
seq), and the second dot denotes a multiplication.
5.3. Usage of real world traces
Since the previous sections deal with test case congurations, description of stream modications, and assessment strategies for evaluating output streams, it is still
open which streams could be used as input streams. So,
this section deals with the question:
How to use real-world traces exhaustively to test intersection related V2X applications?
Therefore, a two-layered approach with and without
modifying real-world traces is presented in the next two
subsections.
First, information ows of intersection related V2X
applications (Section 3.1) show two independent information ows from IRSs or ICSs, and ECUs to IVSs. Additionally,
the complete information ow shows no feedback loops
and, thus, the content of the ows is not affected by reactions or outputs of IVSs. Therefore, it is possible to record

3163

S. Rglinger / Computer Networks 55 (2011) 31543168

traces from the information ows. Second, the traces could

be modied as described in Section 5.2 before reapplying
them to IVSs. This limits the amount of required real-world
traces.
Vehicle driving simulators can also generate ECU traces
and, thus, can substitute real-world traces. This may provide more exibility by taking traces in required intersection topologies and trafc situations. Moreover, traces
taken this way are not even affected by disturbances like
real-world traces.
5.3.1. Reapply raw traces time shifted
The simplest way to use information ow traces from
IRSs or ICSs, and ECUs is to leave them unchanged and
reapply them time shifted. Herewith, test cases regarding
the timing between equipped intersections and equipped
vehicles could be stressed and reapplied. For example, it
is possible to stress the following situations by reusing
the same traces for IRS to IVS and ECU traces:
 the vehicle crosses the stop line at red light,
 the vehicle crosses the stop line at green light,
 the vehicle crosses the stop line at the point in time
when the trafc light switches from red/yellow to
green, and so on.
So, the requirements 1, 3, 5, 6, 7, and 9 which are postulated in Section 4.3 could be fullled. Moreover, test runs
could be executed automatized and, thus, the expensive
and time consuming test drives could be minimized. Also,
regression test strategies could be applied even in development phases.
To be able to use this approach, two independent sets of
traces are required:
 set of intersection IRS traces, and
 set of vehicle ECU traces.
The set of intersection IRS traces covers DSRC and cellular
mobile communication and has to include traces covering
all relevant intersection behaviors of all relevant intersection topologies. The intersections behavior targets the signal state timing as described in Section 3.1. Analogously,
the set of vehicle ECU traces has to include traces covering
all relevant approaching and leaving behaviors of vehicles
on all intersections which are covered by the set of intersection traces. Among others this includes modications
of: speed/acceleration, lane changing, stop & go, and driving direction. In addition, the GPS stream has to be part
of an ECU trace.
5.3.2. Reapply modied traces time shifted
In this subsection, a rened approach of the previous
subsections approach is described to target the remaining
requirements 2 and 4 of Section 4.3. The renement targets
modications of raw streams as foreseen in Section 5.2.
This approach uses the same traces as the approach in
the previous subsection. However, those traces combine
several streams and, thus, it is not possible to modify them
directly. To achieve a modication of the streams, the following steps have to be applied (see Fig. 6):

(1)
(2)
(3)
(4)

split up traces into streams

modify messages per stream
modify timing behavior per stream
combine streams into new traces.

5.4. Test system architecture

A test system for testing intersection related V2X applications needs, on the one hand, the possibility to stimulate
IVSs with test input data and, on the other hand, the possibility to record the systems outputs including automatic
verdict building. Fig. 7 depicts a test hull which encloses
IVSs as SUTs and handles test runs. To target the test needs
for intersection related V2X applications, the test hull contains subcomponents for emulating the behavior of IRSs,
ICSs, ECUs, and localization facilities and subcomponents
for emulating the inuences of DSRC, cellular mobile communication, vehicular internal communication busses, and
localization methodologies as postulated in Section 4.
In addition, a test system requires a test management
component to initiate and evaluate test runs as depicted
in Fig. 8. These points are realized by a test controller subcomponent and a test evaluation subcomponent. The test
controller subcomponent initiates test runs which are
specied in test suit documents, whereupon, the test hull
applies the test input data to the IVSs. Thereupon, the test
evaluation subcomponent takes the recorded IVSs outputs
and evaluates test verdicts. For test cases which require a
replay of the test runs until a test end criterion is fullled,
the test controller subcomponent handles the iterations
Icon,1
value

timing

Mcon,1

Tcon,1

Icon,2
original
trace

value

timing

Mcon,2

Tcon,2

split

modified
trace

merge

Icon,n

value

timing

Mcon,n

Tcon,n

Fig. 6. Trace modication.

TestHull
IRS
ICS
ECUs
Local.

<<flow>>

<<flow>>

<<flow>>

<<flow>>

DSRC
CeMo
ViBus
GPS

<<flow>>

<<flow>>

<<flow>>

IVS
<<flow>>

Rec.
<<flow>>

Fig. 7. Test hull.

3164

S. Rglinger / Computer Networks 55 (2011) 31543168

TestManagement

TestReport

TestEvaluation

<<flow>>

<<flow>>
CAN

V2XGate

<<flow>>
Ethernet

UMTS
modem

<<flow>>
USB

TestHull
<<flow>>

TestController

<<flow>>

<<flow>>

<<write>>

TestSuit

<<read>>

ECUs

CarGate
Ethernet

<<flow>>

CarPC

<<flow>>
DVI

HMI

IVS
Fig. 8. Test management.

and the test end criteria evaluation. Finally, all relevant

information is stored in test report les.

Fig. 9. Travolution system architecture.

6. Proof of concept
To proof the concept described in the previous section,
the two examples presented in Section 5.2.3 were used.
As IVS, a CarPC which executes the RVW implementation
from the Audi travolution predevelopment project was
used. First, a short description of the Audi travolution system architecture, the used test system architecture and the
used input streams is given. At second, the input stream
modications and the results of the test runs are presented
for both examples. Furthermore, it has to be mentioned
that only DSRC communication and no cellular mobile
communication is used in this proof of concept.
6.1. Environment
6.1.1. Travolution system architecture
As depicted in Fig. 9, the Audi travolution system architecture consists of the following components:
CarPC The CarPC acts as IVS and executes the
Audi travolution software. Additionally,
the CarPC generates the visualization
which could be seen in the drivers
instruments display and in the display
located in the center of the instrument
panel.
CarGate The CarGate connects the CarPC to the
relevant vehicles CAN-Busses and thus
to the other ECUs. Moreover, the CarGate
includes a GPS receiver which is used for
the positioning of the vehicle.
V2XGate The V2XGate receives the DSRC messages
from the trafc lights and forwards them
to the CarPC. Communication in the
opposite direction is possible but not
used.
UMTS modem The UMTS modem is used to establish
connections to servers which act as ICSs
and are available over internet. Those
servers are used by some applications as
alternatives to the DSRC communication.
As well as for the V2XGate, communication in the opposite direction is possible
but not used.
HMI The vehicles HMI displays the visualizations generated by the CarPC. For that
purpose, there is one display which is

part of the drivers instruments and an

additional display in the center of the
instrument panel.
For this proof of concept, a commercially available personal computer based on a 2.2 GHz Intel Pentium 4 CPU
with 1.25 GB RAM and Microsoft Windows XP was used
as CarPC.
6.1.2. Test system architecture
Since only information ows originated from ECUs and
IRSs are relevant for the two examples, it is sufcient to
emulate the behavior of the corresponding components.
Hence, traces for testing the Audi travolution system could
be recorded by snifng the ethernet communication between CarGate and CarPC, and V2XGate and CarPC. The test
hull used in this proof of concept has only to include a
component to stub a CarGate which includes the emulation of ECUs and localization facilities (see Fig. 7) and a
component to stub a V2XGate which includes the emulation of intersections IRSs. Beyond that, the test hull has
to include another component to record the CarPCs outputs. Hence, the Audi travolution CarPC provides an entirely rendered video stream to the vehicles HMI display
(see Fig. 9), the RVW applications output events are additionally fed out via UDP datagrams and, thus, can be
recorded by the test hulls recording component.

in kph

vehicle speed

50
45
40
35
30
25
10

20

30

40

time
in sec

40

time
in sec

steering wheel angle

in degree
40
20
-20
-40
-60
-80

10

20

30

Fig. 10. ECU input trace plot.

3165

S. Rglinger / Computer Networks 55 (2011) 31543168

time to switch, and current signal phase

time in sec
30

red

green
orange

20

green
yellow

25

15
10
5

min, max, likely

10

20

30

time
in sec

40

Fig. 11. IRSs signal phase timing input trace plot with graph showing the time to the next switch (black line) and the present trafc light signal state
(background).

intersections center point:

48 47' 02'' N
11 28' 28'' E

ce

PS

tra

23.9 sec.
S

30.0 sec.

app

roa

20.0 sec.

egr

ch

ess

10.0 sec.
0.0 sec.

200 m

Fig. 12. Intersection topology consisting of the center point (cross), approaches, egresses, and the GPS trace.

As already described, all inputs and outputs of the Audi

travolution system can be accessed via ethernet communication. So, all environmental components of the CarPC can
be stubbed by commercially available personal computers.
The benet is, that all test components are external and,
thus, the CarPC and the Audi travolution software has not
to be modied for test purposes. Moreover, no additional
CPU load is caused.
For the implementation, the programming language C
for the stubs, bash scripts for the test management, and
Java for a tool to visualize the recorded streams similar
to Figs. 1012 were used.

6.1.3. Input streams

For this proof of concept, a real drive at a real intersection in the outer conurbation area of Ingolstadt, Germany
was recorded. The streams vehicle speed and steering wheel
angle of the ECU trace are exemplarily depicted in Fig. 10,

whereas, the GPS stream is included as red curved arrow

(grayscale version: black dashed curved arrow) in the visualization of the intersections topology in Fig. 12. Furthermore, the IRS input trace consists of SPAT messages
whose ow is exemplarily depicted for the relevant lane
in Fig. 11 and intersection topology messages which do
not change over time. The intersections topology is depicted in Fig. 12, whereas, approaches are green1 arrows
(grayscale version: gray dashed arrows) pointing towards
the intersections center point and egresses are blue arrows
(grayscale version: gray dotted arrows) pointing in the
opposite direction. The approaches stop lines are located
at the ends of the last approach vectors. So, as it could be
seen in Fig. 12, the vehicle crosses the relevant lanes stop
line after 23.9 s.

1
For interpretation of color in Fig. 12, the reader is referred to the web
version of this article.

3166

S. Rglinger / Computer Networks 55 (2011) 31543168

6.2. Examples
This sub section picks up both examples introduced in
Section 5.2.3, whereas, Section 6.2.1 uses the rst and
example and Section 6.2.2 uses the second one.
6.2.1. Example 1
As foreseen in Example 1 in Section 5.2.3, the input
streams were applied in exactly the same fashion several
times. In order to not exceed the scope of this proof of concept, only the following two additional CPU load conditions
were used:
 no additional CPU load, and
 two threads calculating the 35th bonacci number in
endless loops in parallel.
For both additional CPU load conditions, the input
streams were applied N = 300 times. The resulting histogram plots for the points in time where the BRAKE signal
appeared are depicted in Fig. 13. In the context of this subsection, the difference in time between the 0.02-quantile
and the 0.98-quantile is used as denition of the term
tvariation.
It can be seen that the Audi travolution system reacts
onto the applied input streams in normal load conditions
within a variation of tvariation = 271 ms, whereas, the BRAKE
signal arises as foreseen approximately 2 s before the
vehicle crosses the stop line. In nontypical heavy load conditions, the Audi travolution system reacts reliably within a
variation of tvariation = 812 ms. Finally, it has to be mentioned
that in all test runs the BRAKE signal appeared and, thus, no
histograms for faulty test results can be generated.
However, this system behavior was not considered as
sufcient. So, after inspecting the involved software components, a non-optimal decision in the software design.
After a redesign and modication of this software component, the measurements were repeated. As it could easily
be seen in Fig. 14, the Audi travolution RVW implementation acts now more deterministic. If the Audi travolution
system runs now in normal load conditions, the variation
amounts now to tvariation = 174 ms, whereas, in heavy load
conditions the variation amounts now to tvariation = 642 ms.
6.2.2. Example 2
As foreseen in Example 2 in Section 5.2.3, modications
of the intersections center point shifted within its boundaries were used. Since no exact boundary specications
had been available, the following three ranges for
latitude longitude
slatitude
, and slongitude
were used:
west
north ; ssouth ; seast
 5 m,
 3 m, and
 1m
for shifting the intersections center point in longitude
and latitude direction. So, for each test run, values for the
uniform distributed random variables for shifting the
intersections center point ware determined.
The resulting not required in this example histograms
are depicted in Fig. 15 and the statistical valuations are pos-

Prob.
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
21.0

no additional load

21.5

22.0

22.5

23.0

23.5

time in
24.0 sec

Prob. two threads calc. fibonacci(35) in endless loops

0.7
0.6
0.5
0.4
0.3
0.2
0.1
time in
0.0
21.0
21.5
22.0
22.5
23.0
23.5
24.0 sec
Fig. 13. Histograms Hl=no_add_load,x = 2 and Hl=2_bonacci_35,x=2 for example 1
with not improved software; bar width w = 0.05 s, number of test runs
N = 300 in each case.

Prob.
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
21.0

no additional load

21.5

22.0

22.5

23.0

23.5

time in
24.0 sec

Prob. two threads calc. fibonacci(35) in endless loops

0.7
0.6
0.5
0.4
0.3
0.2
0.1
time in
0.0
21.0
21.5
22.0
22.5
23.0
23.5
24.0 sec
Fig. 14. histograms Hl=no_add_load,x=2 and Hl=2_bonacci_35,x=2 for example 1
with improved software; bar width w = 0.05 s, number of test runs
N = 300 in each case.

tulated in Table 3. The terms zero (and non-zero) in Fig. 15

indicates that the histogram bars in the referred areas are
really zero (not zero) and not just small. The results show,
that specications of the acceptable inaccuracy of the position of intersection center points may have considerable
inuence on the behavior of RVW applications if the application is not capable to deal with it. For example, T x2
min : valuates
to 14.95 s for a variation range of 3 m, whereas, it valuates
to 21.00 s for a varation range of 1 m. Thus, RVW implementations should indeed be tested regarding this issue.

7. Related work
Hence, testing of V2X communication systems is currently a topic of major research interest; many parties

S. Rglinger / Computer Networks 55 (2011) 31543168

Prob. shift

3167

8. Conclusions and future work

5 m in longitude and latitude direction

0.5

Range

T x2
min :

T x2
max :

x2
T 0:25quan:

T x2
0:50quan:

T x2
0:75quan:

The presented work constitutes a short overview of

Vehicle-2-X communication systems and introduces intersection related applications more comprehensively. Thereupon, test needs of intersection related V2X applications,
including requirements on a test system, are presented.
Furthermore, the paper introduces a stream based test approach and shows its feasibility by a proof of concept.
The approach uses real-world traces from all information sources as input streams of test cases. This is practicable because all environment components work
independently and, thus, the information ow does not
contain feedback loops. The test approach intends to combine those independent real-world traces in different ways
to create the test cases. Moreover, the real-world traces
may be modied before applying them to the system under test, to create further test cases without recoding
new traces. Depending on the systems output stream
types, several assessment strategies are introduced. By
means of those assessment strategies, test verdicts could
be determined.
Finally, the proof of concept uses the Audi travolution
predevelopment projects Vehicle-2-X implementation to
show the feasibility of the approach. Beyond this, the proof
of concept shows needs of statistical test strategies. Those
needs are caused by the non-determinism of the Vehicle-2X environment and the complexity of the Vehicle-2-X
implementations.
In the future, emphasize will be placed on further metrics and assessment strategies for the timing of streams
and for the messages of streams as already sketched in
Tables 1 and 2. Moreover, analysis on the transfer of the
papers approach to other types of V2X applications are
planned. The rst idea is the utilization of the approach
for V2X applications which deal with two cars like an intersection collision warning assistant.

5 m
3 m
1 m

14.84
14.95
21.00

24.67
23.10
21.96

21.53
21.69
21.74

21.75
21.75
21.80

21.91
21.85
21.86

Acknowledgements

0.4
0.3

non-zero

0.2
0.1
0.0
14

16

Prob. shift

18

20

22

time
in sec

24

3 m in longitude and latitude direction

0.5
0.4
0.3

non-zero

0.2
0.1
0.0
14

16

Prob. shift

18

20

22

time
in sec

24

1 m in longitude and latitude direction

0.5
0.4
0.3

zero

0.2
0.1
0.0
14

16

18

20

22

24

time
in sec

Fig. 15. Histograms for example 2 for points in time where BRAKE
massages appeared (x = 2); bar width w = 0.2 s, number of test runs
N = 300 in each case.

Table 3
Results of example 2, each value in seconds, N = 300.

work on it from many different points of view. E.g., the

German research project simTD pursues not only the research and development goals to increase trafc safety
and trafc efciency but also to develop test methodologies on any test level. Moreover, the project aims to connect industrial and scientic parties to push the V2X
system development.
For testing activities on acceptance test level, various
simulation techniques are currently used [4143]. On system and integration test levels, there are approaches using
TTCN-3 [44] as test architecture and test description language [45]. In addition, there are also more general approaches combining TTCN-3 with the concept of channels
and streams postulated in [46,47] which might also be
used for testing V2X applications.
Beyond this, test architecture descriptions targeted to
V2X applications were already publicized in [45,48]. A
more general description of test architectures for distributed systems can be found in [49].

The author acknowledges the nancial support for this

research which has been granted by the research program
of Bavaria for universities of applied sciences and is supported by the AUDI AG Ingolstadt, Germany. Especially
for providing access to the Audi travolution predevelopment project, the author would like to thank the Audi travolution team. Moreover, the author is particularly grateful
to Robert Mnz, Cornelius Menig, and Malte Mller for
supporting his work with their expertise in V2X communication. In addition, grateful acknowledgement is made to
Peter Trapp, Markus Meyer and Christian Facchi for proofreading and providing valuable comments.

References
[1] H. Hartenstein, K.P. Laberteaux, A tutorial survey on vehicular ad hoc
networks, IEEE Communications Magazine (2008) 164171.
[2] Car 2 Car Communication Consortium, Manifesto: Overview of the
C2C-CC System, 2007. <http://www.car-to-car.org/>.
[3] ETSI, ETSI TR 102 638, Basic set of applications, Denitions, V1.1.1,
2009.

3168

S. Rglinger / Computer Networks 55 (2011) 31543168

[4] COMeSafety, Deliverable D31 European ITS, Communication

architecture, Overall framework, Proof of concept implementation,
Version 2.0, 2009. <http://www.comesafety.org/>.
[5] IEEE draft standard for information technology, Telecommunications
and information exchange between systems, Local and metropolitan
area networks, Specic requirements, Part 11: Wireless LAN medium
access control (MAC) and physical layer (PHY) specications
amendment: wireless access in vehicular environments, 2010.
[6] ETSI, ETSI ES 202 663, European prole standard for the physical and
medium access control layer of Intelligent Transport Systems
operating in the 5 GHz frequency band, V1.1.0, 2010.
[7] M. Ergen, Critical Penetration for Vehicular Networks, IEEE
Communications Letters 14 (5) (2010) 414416.
[8] simTD, Deliverable D11.2, Ausgewhlte Funktionen, Version 1.0.
<http://www.simtd.de/>, 2009.
[9] COMeSafety, Deliverable D31 European ITS, Communication
Architecture, Overall Framework, Proof of Concept Implementation,
Version 3.0, 2010. <http://www.comesafety.org/>.
[10] ETSI, Final draft ETSI EN 302 665, Communications Architecture,
V1.1.0, 2010.
[11] simTD, Deliverable D21.2, Konsolidierter Systemarchitekturentwurf,
Version 3.0, 2009. <http://www.simtd.de/>.
[12] IEEE standard for information technology, Telecommunications and
information exchange between systems, Local and metropolitan
area networks, Specic requirements, Part 11: Wireless LAN
medium access control (MAC) and physical layer (PHY)
specications, 2007.
[13] D. Jiang, L. Delgrossi, IEEE 802.11p: towards an international
standard for wireless access in vehicular environments, in:
Vehicular Technology Conference, 2008, VTC Spring 2008, IEEE,
2008, pp. 20362040.
[14] Electronic communications committee, ECC decision of 14 March
2008 on the harmonised use of the 58755925 MHz frequency band
for intelligent transport systems (ITS), ECC/DEC/(08)01, 2008.
[15] Electronic communications committee, ECC recommendation, Use of
the band 58555875 MHz for intelligent transport systems (ITS),
ECC/REC/(08)01, 2008.
[16] S. Rglinger, C. Facchi, How can Car2X-communication improve road
safety A statistical based selection and discussion of feasible
scenarios, Arbeitsberichte/Working Papers 15, Ingolstadt University
of Applied Sciences, 2009.
[17] ETSI, ETSI TS 102 637-2, Basic set of applications, Part 2:
Specication of cooperative awareness basic service, V1.1.1, 2010.
[18] ETSI, ETSI TS 102 637-3, Basic set of applications, Part 3:
specication of decentralized environmental notication basic
service, V1.1.1, 2010.
[19] SAE, SAE J2735, Draft Rev. 31, Dedicated short range
communications (DSRC) message set dictionary, 2009. <http://
www.sae.org/>.
[20] simTD,
Deliverable
D21.4,
Spezikation
der
Kommunikationsprotokolle,
Version
1.0,
2009.
<http://
www.simtd.de/>.
[21] simTD, Deliverable D11.1, Beschreibung der C2X-Funktionen, Version
2.0, 2009. <http://www.simtd.de/>.
[22] ISTQB, Certied tester foundation level syllabus, 2010. <http://
www.istqb.org/>.
[23] G. Myers, The Art of Software Testing, Wiley, New York, 1979.
[24] D.W. Hoffmann, Software-Qualitt, Springer, 2008.
[25] E.H. Riedemann, H. Schippers, Testmethoden fr sequentielle und
nebenluge Software- Systeme, Vieweg+Teubner, 1997.
[26] IEEE Standard Glossary of Software Engineering Terminology, 1990.
[27] ISTQB, Standard glossary of terms used in Software Testing, 2010.
http://www.istqb.org/.
[28] B. Hetzel, The complete guide to software testing, second ed., QED
Information Sciences, Inc., 1988.
[29] E. Cota-Robles, J. Held, T. Barnes, Schedulability analysis for desktop
multimedia applications: simple ways to handle general-purpose
operating systems and open environments, in: Proceedings of the
IEEE International Conference on Multimedia Computing and
Systems97, 1997, pp. 475483.
[30] R. Motwani, P. Raghavan, Randomized Algorithms, Cambridge
University Press, 1995.
[31] W. Ming, Y. Lin-tao, L. Cheng-yi, J. Hao, Capacity, Collision and
interference of VANET with IEEE 802.11 MAC, in: First International
Conference on Intelligent Networks and Intelligent Systems, 2008,
ICINIS08, 2008, pp. 251254.
[32] W.
Wiesbeck,
N.
Geng,
Planungsmethoden
fr
die
Mobilkommunikation, Springer, Berlin, 1998.

[33] M. Torrent-Moreno, S. Corroy, F. Schmidt-Eisenlohr, H. Hartenstein,

IEEE
802.11-Based
One-Hop
Broadcast
Communications:
Understanding Transmission Success and Failure under Different
Radio Propagation Environments, in: MSWiM 06: Proceedings of the
9th ACM international symposium on Modeling analysis and
simulation of wireless and mobile systems, ACM, 2006, pp. 6877.
[34] M. Torrent-Moreno, J. Mittag, P. Santi, H. Hartenstein, Vehicle-tovehicle communication: fair transmit power control for safetycritical information, IEEE Transactions on Vehicular Technology 58
(7) (2009) 36843703.
[35] WordNet 3.0, Cognitive Science Laboratory, Princeton University, A
lexical
database
for
the
English
language.
<http://
wordnet.princeton.edu>, 2006 (accessed 09.10).
[36] M. Broy, Renement of time, in: M. Bertran, T. Rus (Eds.),
Transformation-based reactive system development, ARTS97, TCS,
1997, pp. 4463.
[37] E.W. Weisstein, Discrete Set. From MathWorld A Wolfram Web
Resource,
<http://mathworld.wolfram.com/DiscreteSet.html>
(accessed 09.10).
[38] E.W. Weisstein, Totally Ordered Set. From MathWorld A
Wolfram Web Resource, last access on Sep 2010 at: <http://
mathworld.wolfram.com/TotallyOrderedSet.html>.
[39] J. Zander-Nowicka, Model-based testing of real-time embedded
systems in the automotive domain, Ph.D. thesis, Technical University
of Berlin, 2009.
[40] T. Arens, F. Hettlich, C. Karpnger, U. Kockelkorn, K. Lichtenegger, H.
Stachel, Mathematik, SpringerVerlag, 2008.
[41] M. Glaab, A. Mauthofer, New test and evaluation methods for future
car2x communication based driver assistance, in: 21st International
Technical Conference on the Enhanced Safety of Vehicles (ESV),
2009.
[42] B. Schunemann, K. Massow, I. Radusch, A novel approach for realistic
emulation of Vehicle-2-X communication applications, in: IEEE
Vehicular Technology Conference, 2008, pp. 27092713.
[43] C. Sommer, Z. Yao, R. German, F. Dressler, Simulating the inuence of
IVC on road trafc using bidirectionally coupled simulators, in: IEEE
INFOCOM Workshops 2008, pp. 16.
[44] ETSI, TTCN-3 Standard: ETSI ES 201 873-1 V4.2.1, 2010. http://
www.etsi.org/.
[45] J. Gromann, C. Neumann, A. Hinnerichs, H. Rechner, T. Hecker, I.
Radusch, Test von verteilten C2X Applikationen, in: GI Jahrestagung,
vol. 2, 2010, pp. 491496.
[46] I. Schieferdecker, E. Bringmann, J. Gromann, Continuous TTCN-3:
testing of embedded control systems, in: SEAS06: Proceedings of the
2006 International Workshop on Software Engineering for
Automotive Systems, ACM, 2006, pp. 2936.
[47] I. Schieferdecker, J. Gromann, Testing embedded control systems
with TTCN-3: an overview on TTCN-3 continuous, in: SEUS07:
Proceedings of the 5th IFIP WG 10.2 International Conference on
Software Technologies for Embedded and Ubiquitous Systems,
Springer-Verlag, Berlin, Heidelberg, 2007, pp. 125136.
[48] A. Tomatis, M. Miche, F. Haeusler, M. Lenardi, T. Bohnert, I. Radusch,
A test architecture for V-2-X cooperative systems eld operational
tests, in: 2009 9th International Conference on Intelligent Transport
Systems Telecommunications (ITST), 2009, pp. 616621.
[49] T. Walter, I. Schieferdecker, J. Grabowski, Test architectures for
distributed systems State of the art and beyond, 1998.

Sebastian Rglinger is a research assistant at

the Institute of Applied Research of the
Ingolstadt University of Applied Sciences
since September 2008. His research interests
are in the area of communication systems and
automotive technologies and therefore for
Vehicle-2-X Communication. He was born in
1983 at Ksching, Germany. For his graduation in Electrical Engineering and Information
Technology at the Ingolstadt University of
Applied Sciences he was elected as the winner
of the E.ON Bayern culture award for 2008.
After his study he was employed at Elektrobit in Erlangen, Germany as a
Software Engineer in the Engineering Service Infotainment department.