Professional Documents
Culture Documents
Design-to-Test Approach For Programmable Controllers in Safety-Critical Automation Systems
Design-to-Test Approach For Programmable Controllers in Safety-Critical Automation Systems
fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2968480, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 1
Abstract—This paper presents a design-to-test approach for In order to enhance the testability of programmable con-
enhancing black-box complete conformance testing of pro- trollers in safety-critical systems where complete testing is
grammable controllers, where their specifications can be modeled required, a design-to-test (DTT) approach is presented in
as finite state machines. Given an automation system, the testing
objective is to check whether its implemented controllers conform this paper. In contrast to the traditional workflow, the DTT
to the expected behavior with regard to the specification models. approach considers testing much earlier in the development
The design-to-test approach analyzes the specification models and, process, i.e., during the specification design. After the func-
if necessary, automatically modifies them at the cost of limited tional specification being modeled as finite state machines,
design overhead, so that the testability of the final implemented the models are checked with regard to their testability and,
controllers can be ensured/improved. By design, this approach
also guarantees that the behavior of implemented controllers if necessary, modified by the DTT approach. As a result, the
remains unchanged during normal execution (i.e., when not DTT approach improves the functional testability with limited
connected to a test bench). design overhead, reduces the testing overhead, and keeps the
Index Terms—design-to-test, programmable controller, single- controller behavior unchanged during normal execution.
input-change, finite state machine, black-box testing A finalized version of the DTT approach is presented in
this paper, while early and intermediate results have been
published in two IEEE conferences in 2015 [4] and 2016 [5].
I. I NTRODUCTION
The main contributions of this paper are: modified mathemat-
1551-3203 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on May 11,2020 at 12:05:12 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2968480, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 2
methodology can be adapted and extended to other formalisms III. M ATHEMATICAL BACKGROUND
as well. To apply the DTT approach, system specification is modeled
as a set of communicating Moore machines extended with
B. Complete conformance testing Boolean signals1 . Actually, the approach can also be applied
As a type of model-based testing, conformance testing to other signal-based models (such as Grafcet) with stability
compares the observed behavior of an implementation, e.g., search semantics, where all possible paths of each transition
a programmable controller, to the expected behavior of the are considered independently of the evaluation/execution or-
specification models [10]. der [22].
With complete conformance testing (CCT), the test genera-
tion considers all possible combinations of input signals from A. Communicating Moore machine extended with Boolean
all states. It is highly advantageous for safety-critical systems signals
because the whole system behavior is covered by the tests. A communicating Moore machine extended with Boolean
However, this characteristic also restricts its applicability to signals is defined by an 8-tuple (L, linit , I, C, O, Gδ , δ, λ),
large-scale systems, because the test case number grows ex- where:
ponentially with the input number. To cope with this problem,
• L is a finite set of locations. A location represents a logic
one main direction is to generate a small set of test cases that
state of a single model for a subsystem/component2 .
has high code coverage [11] [12]. Nevertheless, as concluded
• linit is the initial location, linit ∈ L.
in [13], tests with coverage criteria satisfaction cannot always
• I is a finite set of Boolean input signals.
effectively find faults. [14] and [15] made efforts in another
• C is a finite set of internal Boolean communicating
direction: the former generated a small set of test case that
variables that are related to locations; a communicating
guarantees full coverage of nominal behavior of the controller;
variable is denoted as ‘X(location)’, e.g., ‘X(l1 )’.
the latter inserted a selected set of faults into set of test cases
• O is a finite set of Boolean output signals/output actions.
based on expertise, to test certain error handling routines.
• Gδ := expr(I, C) is a finite set of transition guards,
However, with both methods, no guarantee can be achieved
which are Boolean expressions3 built up by inputs and
that all critical faults are covered by their tests.
internal variables.
As a result, complete conformance testing is still essential to
• δ : L × Gδ → L is the transition function that maps the
safety-critical systems/parts, which is supported and enhanced
current location and transition guard to the next location;
by the DTT approach in this paper.
a transition is fired when its source location is active and
its guard is evaluated as True; ‘∆’ is used to denote the
C. Inspiration of DTT
set of ‘δ’.
The concept of DTT was inspired from a DFT (design-for- O
• λ : L → 2 is the output function that maps the locations
test) technique, which adds testability features in the design of to their corresponding output signals; ‘Λ’ is used to
integrated circuits (ICs) [16]. By considering testability early denote the set of ‘λ’.
in the design, users are able to lower power consumption, re- The models are also presented in graphical form. As exam-
duce test time, and improve fault coverage in testing [17] [18]. ples, two models are presented in Fig. 1.
A similar idea has also been found in the development of
automotive applications [19], where a fast functional safety Model 1 i1 ¬i1 ∧ ¬i2
verification method is applied during early design phase, so ∅ o1 ∅
X(l5 ) ∧ i1
that performance of software reliability and response time can l1 l2 l3
1551-3203 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on May 11,2020 at 12:05:12 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2968480, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 3
is assigned the value ‘1’ when l5 is activated; if i1 has the IV. T EST OBJECTIVE AND TEST GENERATION
value ‘1’, and at the same time if l3 is active, the transition A. Test objective
from l3 to l2 can be fired.
A programmable controller can be programmed with func-
tions and instructions. It executes in a cyclic mode which ful-
B. Stabilized composed automaton
fills the so-called hard real-time requirement. In each cycle, a
After the specification is modeled as individual Moore controller runs successively: input reading, program execution,
machines, the models are then composed in parallel to build and output updating.
a monolithic model. For more information about this parallel In the black-box conformance testing of a programmable
composition operation, readers can refer to [22] and [23] (page controller, the internal structure of the implementation is not
79). visible, while its expected behavior (from its specification
As introduced earlier in this section, the stability search models) is known to the testers.
semantics is used in the composition. Given a set of individual As a basic test case, a test unit for an evolution (e.g., the
models that can run in parallel, all the transitions from all the evolution from s2 to s4 in Fig. 2) consists of three phases [24]:
active locations are evaluated, and executed if they are enabled;
1) Before testing: to activate the expected source state (e.g.,
the firing of transitions continues until none of them can be
s2 ) from the current active state by inputting an auxiliary
fired anymore, without changing the values of inputs. Thus, a
sequence of signals (e.g., from s3 to s2 )
stable situation is reached. Thereby, the obtained monolithic
2) During testing: to feed the testing input signals to the
model is named a Stabilized Composed Automaton (SCA). A
controller (e.g., {101} in the order of (i1 , i2 , i3 ))
software program proposed in [22], Teloco, is used in this
3) After testing: to determine the destination state by com-
composing process.
paring the observed output signals to the expected out-
An SCA is defined by a 7-tuple (S, sinit , I, O, Ge , e, λs ),
puts; if the observed output is not unique, to apply another
where:
auxiliary sequence of signals (e.g. s4 and s6 have the
• S is a finite set of states. A state represents a combination
same outputs o1 ∧ o2 )
of locations from all the individual models.
During the three phases, three issues have been identified:
• sinit is the initial state, sinit ∈ S.
the observability issue, the controllability issue and the single-
• I is a finite set of Boolean input signals (same as used
input-change-testability (SIC-testability) issue. More details
in the individual models).
about them are presented in the next section.
• O is a finite set of Boolean output signals/output actions
(same as used in the individual models).
• Ge := expr(I) is a finite set of evolution guards, which B. Test generation
are Boolean expressions built up by inputs. With a model-based approach, test cases and sequences are
• e : S × Ge → S is the evolution function that maps generated from specification models.
the current state and evolution guard to the next state; Firstly, system specification is modeled (manually) as indi-
an evolution can be understood as a transition between vidual models, which are then composed into an SCA. Next,
states. from the SCA, an equivalent Mealy machine model is built by
O
• λs : S → 2 is the output function that maps the states explicitly representing all Boolean conditions of all evolutions
to their corresponding outputs. over the Boolean input set. Apart from evolutions between
As an example, the SCA of the two models in Fig. 1 is states, the Mealy machine contains also self-loop evolutions
presented in Fig. 2. For example, s4 is a state, which represents which are omitted in the graphical representation of the SCA.
a combination of l2 and l6 , and it has the output from the both Finally, a test sequence is constructed by traversing all the
locations, i.e., o1 and o2 . In the model, all the states are stable, evolutions from all the states in the Mealy machine. A test is
which means: without changing the input values, no evolution considered complete, when its test cases are generated from
will be fired, and an active state will remain active. the complete specification. This test generation method and
its implemented tool Teloco have been presented in details
¬i1 ∧ i2 ∧ ¬i3
o2 in [22].
∅
i1 ∧
¬i1 ∧¬i2 ∧¬i3 The DTT approach, as an enhancement of complete testing,
s1 (l1 , l4 ) s3 (l1 , l5 )
¬i2 ∧ i3 uses the tool Teloco for generating SCA and Mealy machine.
i1 ∧ (i2 ∨ ¬i3 ) i1
1551-3203 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on May 11,2020 at 12:05:12 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2968480, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 4
input signals simultaneously change their values, the actual MIC, i.e., ¬i1 ∧ i2 ∧ ¬i3 to i1 ∧ ¬i2 ∧ ¬i3 , can be regarded
signal values percepted by a controller may deviate from the as a SIC, and therefore s3 becomes fully SIC-testable.
expected values that the controller is supposed to read [26]. A generalized T-guard method is depicted in Alg. 1. In brief,
The physical causes are: in order to obtain full SIC-testability, a minimum set of T-
1) In natural world, two events, i.e., changes of two different guards are added to the individual specification models (not
physical signals, cannot occur at the same time exactly. only to the SCA).
2) Due to the cyclic execution mode, a programmable con-
troller reads the input values only in the first cycle phase. Algorithm 1: Pseudo-code of the T-guard method
If an input changes its value just after the phase, the new Input: L, ∆, Gδ , I
value can only be read in the next cycle. Result: GSIC
1 Initialization: IN SIC := ∅; Ttarget := ∅;
Experiments have proven that the occurrence of this issue 2 LT := ∅; GSIC := Gδ ;
cannot be neglected, especially for large-scale systems which 3 begin
4 GE−N SIC := T eloco(L, ∆, I);
contain multiple I/O cards [27]. To cope with this issue, some 5 foreach gE−N SIC ∈ GE−N SIC and i ∈ I do
researchers try to build test sequences that consist of as many 6 IN SIC += {i | i ∧ gE−N SIC = gE−N SIC };
SIC steps as possible [26] [28]. Therefore, the issue is named 7 IN SIC += {¬i | ¬i ∧ gE−N SIC = gE−N SIC };
/* check if i or ¬i is an element in gE−N SIC
as SIC-testability issue. */
However, it has been proven not always possible to build a 8 IN SIC := sort(IN SIC , GE−N SIC );
test sequence that contains only SIC test steps for a practical 9 foreach i ∈ IN SIC do
10 i.value := F alse;
system. As a part of the DTT approach, a T-guard method is /* set the initial values to False */
presented to provide a thorough solution to this issue. 11 foreach j ∈ IN SIC do
The two models presented in Fig. 1, and their SCA (Fig. 2) 12 j.value
_:= T rue;
are used in the following to illustrate the SIC-testability issue 13 if gE−N SIC 6= F alse then
and the T-guard method in details. GE−N SIC
14 Ttarget += {j};
Using the DTT approach, the SIC-testability result is calcu- 15 j.value := F alse;
lated by Teloco [22]. Due to its simplicity, the SIC-testability /* this means j is an essential element in
of the system in Fig. 1 can also be identified manually from Ttarget , a minimum set of IN SIC */
the SCA in Fig. 2. For instance, the non-SIC-testable part of 16 foreach gδ ∈ Gδ and l × gδ → l0 do
s3 is i1 ∧ ¬i2 ∧ ¬i3 , which is actually a guard of an evolution 17 foreach k ∈ Ttarget do
18 if k ∧ gδ = gδ then
from s3 to s6 in the Mealy machine. Obviously, it cannot 19 LT += {l};
be obtained with a SIC from the guard of the only incoming /* l is a location that has outgoing
evolution of s3 , i.e., ¬i1 ∧ i2 ∧ ¬i3 ; and all its SIC-“siblings” transitions that have inputs in
Ttarget */
will make either ¬i1 ∧¬i2 ∧¬i3 or i1 evaluated as True, which
enables an outgoing evolution to either s1 or s6 . Therefore, 20 foreach l ∈ LT and gδ ∈ Gδ do
s3 is not fully SIC-testable. 21 if ∃{l, gδ } | l × gδ → l0 then
22 GSIC −= {gδ };
To solve this issue, the T-guard method transforms all MICs 23 gδ := gδ ∧ T g;
into SICs with an extra non-functional input signal, which /* add a T-guard to the guards of outgoing
is named as T-guard, denoted as ‘T g’. As an example, a transitions of l */
24 GSIC += {gδ };
schematic diagram of signal and state changes before/after
adding a T-guard for the non-SIC-testable evolution of s3 is
presented in Fig. 3.
The inputs of Alg. 1, I, L, ∆ and Gδ , represent respectively
the full set of inputs, the unions of (with regard to all the
Controller I E O I E O I E O I E O I E O I E O I E O I individual models): the full sets of locations, the full sets of
Cycle
transitions, and the full sets of transition guards.
Tg 1
0 Firstly, GE−N SIC , a non-SIC-testable subset of the evolu-
i1 1
0 tion guards for all states, is obtained by running Teloco [22]
i2 10 (line 4 in Alg. 1). For the example in Fig. 2, GE−N SIC is
active state s3 s6 {i1 ∧ i2 ∧ ¬i3 , i1 ∧ ¬i2 ∧ ¬i3 }.
Then, all the inputs (considering both themselves and their
time
negations) that are involved in the evolution guards belonging
Fig. 3. Changes of signals and states after adding a T-guard to GE−N SIC are picked out and listed in IN SIC (lines 5 to
7). After that, according to the appearance frequency of its
Before a MIC is executed, the input signal T g is set to the elements in GE−N SIC , IN SIC is sorted in ascending order
value ‘0’, so all the outgoing and self-loop evolutions from (lines 8). The next step is to filter out the elements IN SIC
the current state, i.e., s3 , are frozen from being fired. After iteratively, in order to obtain Ttarget , a minimum set of IN SIC .
the MIC is stabilized, i.e., i1 and i2 have finished changing The criterion of the filtering is that: as long as all the inputs in
their values, T g is set back to the value ‘1’. Then, only the Ttarget are protected by T-guards, all the guards in GE−N SIC
expected evolution will be fired. In other words, a previous are guaranteed SIC-testable (lines 9 to 15).
1551-3203 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on May 11,2020 at 12:05:12 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2968480, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 5
Compared to the previous algorithm, the newly added Algorithm 2: Pseudo-code of the O-action method
sorting step helps to further improve the filtering effect, and Input: S, L, Λ, Λs , #model
therefore may obtain a further shrunk Ttarget . For the example Result: ΛObs
1 Initialization: LN Obs := ∅; ΛObs := Λ; #OA := 0 ;
in Fig. 2, IN SIC is {i2 , ¬i2 , i1 , ¬i3 }, Ttarget is {¬i3 }. 2 begin
Now the focus is moved from the SCA to the individual 3 foreach si ∈ S, sj ∈ S, si 6= sj do
models. All the transition guards that contain non-SIC-testable 4 if λs (si ) = λs (sj ) then
5 for n = 1 : #model do
inputs should be protected with T-guards. Besides, to pre- 6 if λn (li ) = λn (lj ),
vent an unexpected evolution/transition from being fired, if 7 li ∈ si , lj ∈ sj , li 6= lj then
a location has any outgoing transition containing a non-SIC- 8 LN Obs += {li , lj };
testable input, the guards of all the outgoing transitions from
this location should be protected by T-guards (lines 16 to 19). 9 for n = 1 : #model do
Finally, as output of Alg. 1, GSIC is obtained through 10 LN Obs,n := LN Obs ∩ Ln ;
/* gather the non-observable locations that
updating all the relevant transition guards by adding a T-guard are from Ln , the set of all the locations
to them (line 20 to 24). For the example in Fig. 1, guards for from the nth model */
11 #OA,n := dlog2 |LN Obs,n | e;
the transitions l4 to l5 and l5 to l4 are directly updated, and /* the minimum number of O-actions that are
the guard for the transition l4 to l6 is indirectly updated. The needed
h for a model i*/
updated models are presented in Fig. 4. 12 OAn := oa#OA +1 , oa#OA +2 , · · · , oa#OA +#OA,n ;
/* OAn is a list of O-actions (oa) for one
model */
Model 1 i1 ¬i1 ∧ ¬i2
∅ o1 ∅ 13 #OA += #OA,n ;
X(l5 ) ∧ i1 /* the number of O-actions that are needed for
l1 l2 l3 all models in the whole system */
¬i1 ∧¬i2 ∧¬i3 ∧T g i1 ∧ ¬i2 ∧ i3 ∧T g
14 foreach l ∈ LN Obs,n do
Model 2
o2 ∅ o2 15 ΛObs −= {λ(l)};
¬i1 ∧ i2 ∧ ¬i3 ∧T g i1 ∧ i2 ∧ i3 16 λ(l) := λ(l) ∧ minterm(OAn );
l5 l4 l6 /* update the output by adding a minterm,
which is a unique combination of
Fig. 4. The models in Fig. 1 updated with T-guards elements in OAn */
17 ΛObs += {λ(l)};
1551-3203 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on May 11,2020 at 12:05:12 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2968480, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 6
LN Obs,n , so one O-action is needed for each model. Therefore, Algorithm 3: Pseudo-code of the C-guard method
two O-actions are required to distinguish the total system. The Input: S, E, L, ∆, Gδ , #model , LimitCtr
updated models are presented in Fig. 57 . Result: ∆Ctr , GCtr
1 Initialization: ECtr := ∅; ∆Ctr := ∆; GCtr := Gδ ;
Compared to the previous algorithm, the new algorithm 2 begin
assigns exclusive O-actions to each relevant model instead of 3 foreach si ∈ S, sj ∈ S do
using shared O-actions for all models. As a result, the O- 4 if si = sj then
5 P athS (si , sj ) := 0;
action method improves the scalability of handling large-scale 6 else if ∃(si × ge → sj ) ∈ δL then
systems (especially distributed systems); when a subsystem 7 P athS (si , sj ) := 1;
or a part needs to be re-designed, the other subsystems/parts 8 else
9 P athS (si , sj ) := ∞;
are less affected. As a potential drawback, this might lead
to a larger total number of O-actions; nevertheless, users can 10 P athS := Floyd-Warshall(P athS );
11 while max(P athS ) > LimitCtr do
always select the result that fits best to their needs. 12 ∆Ctr , ∆new :=
EvoCalc(P athS , ∆Ctr , S, #model , L);
Model 1 i1 ¬i1 ∧ ¬i2 /* EvoCalc is presented in Alg. 4 */
∅ ∧oa1 o1 ∅ ∧¬oa1
X(l5 ) ∧ i1 13 #Ctr := |∆new |;
l1 l2 l3 /* #Ctr
n is the number of C-guards
o to be added */
Model 2 ¬i1 ∧ ¬i2 ∧ ¬i3 i1 ∧ ¬i2 ∧ i3 14 C := ctr1 , ctr2 , · · · , ctr#Ctr ;
o2 ∧oa2 ∅ o2 ∧¬oa2 /* C is a set of C-guards */
¬i1 ∧ i2 ∧ ¬i3 i1 ∧ i2 ∧ i3
l5 l4 l6 15 foreach δCtr ∈ ∆new do
16 gδ,Ctr := C(i);
/* gδ,Ctr is the guard for the δCtr created in
Fig. 5. The models in Fig. 1 updated with O-actions line 6 in Alg. 5; i is the index of δnew in
∆Ctr */
17 GCtr += {gδ,Ctr };
18 foreach gδ ∈ G∆ do
C. Controllability & C-guard method 19
0
if ∃ldes 0
| ldes × gδ → ldes then
As introduced in section IV-A, an expected state should be 20 GCtr −= {gδ };
21 gδ := gδ ∧ ¬C(i);
activated before testing. 22 GCtr += {gδ };
This process is actually a switch of the active status from
a random (current) state to another expected state, which
can be realized by applying a homing or a synchronizing
sequence [24]. For the example in Fig. 2, assuming the current
active state is s3 and the expected state is s2 , a homing states. It is initialized as follows: the path cost for a state to
sequence can be {000, 110}. itself is set to 0; the path cost for a pair of states is set to 1, if
For real systems, some states can be either not reachable there is a direct evolution between them; otherwise, the cost
from some other states, or reachable but over a long distance, is set to ∞ . Then, P athS is updated with the Floyd-Warshall
i.e., via very long sequences. The two questions, whether and algorithm, which calculates the indirect path costs for all pairs
how fast the switch of the active status can be performed, are of states (line 10).
the concerns of the so-called controllability issue. Tab. I presents the values of P athS for the example in
The C-guard method, as the last part of the DTT approach, Fig. 2. The values for the initial and updated path costs (for
aims at solving/easing the controllability issue by building the models before and after being updated with C-guards) are
connections among locations. To be more concrete, a mini- depicted in black and green, respectively.
mum set of C-guard transitions are added to the individual
models. C-guards are extra non-functional input signals. TABLE I
PATH COST MATRIX FOR THE SYSTEM IN F IG . 2
A generalized algorithm of the C-guard method is presented
in Alg. 3. Two functions of Alg. 3, EvoCalc and T ranCalc, From
s1 s2 s3 s4 s5 s6 s7 s8
are presented separately in Alg. 4 and Alg. 5. Again, the To
s1 0 ∞/2 1 ∞/3 ∞/1 ∞/2 ∞/2 ∞/2
example in Fig. 1 and Fig. 2 is used in the illustration. s2 1 0 2 1 ∞/2 ∞/3 ∞/3 ∞/3
As inputs of Alg. 3, S, E, L, ∆, Gδ , #model , LimitCtr rep- s3 1 ∞/3 0 ∞/4 ∞/2 ∞/3 ∞/3 ∞/3
resent respectively the full set of states, the full set of s4 1 1 2 0 ∞/2 ∞/3 ∞/3 ∞/3
evolutions, the unions of (with regard to all the individual s5 2 1 2 2 0 1 1 1
s6 2 1 1 2 2 0 3 1
models): the full sets of locations, the full sets of transitions, s7 2 2 3 1 1 2 0 2
and the full sets of transition guards. #model is again defined s8 3 2 2 3 1 1 2 0
as the number of individual models. The last input LimitCtr
is the expected controllability (i.e., path cost between states, The maximum of P athS is then compared to LimitCtr , the
a positive integer) to be specified by users. expected controllability: if it exceeds, then ∆new , a minimum
Firstly, the initial controllability is calculated (lines 3 to 9). set of C-guard transitions, is calculated for the individual
P athS is defined to be a path cost matrix for all pairs of models (lines 11 to 12). The guards of the newly created
7 For better display and clear illustration, not only active outputs but also de-
transitions are consequently assigned (lines 13 to 17). It is
activated O-actions are presented, e.g., in l3 , ∅ ∧ ¬oa1 is actually equivalent worth noting that, after the C-guard transitions have been
to ∅; besides, the T-guards are not presented in this figure. added, for stability reason, the negation of the C-guards should
1551-3203 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on May 11,2020 at 12:05:12 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2968480, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 7
be added to the direct consecutive transitions of the newly Algorithm 4: Pseudo-code of EvoCalc
created C-guard transitions (lines 18 to 22). In the example, the Data: P athS , ∆Ctr , S, #model , L
maximum of initial path cost is ∞, which means some states Result: ∆Ctr , ∆new
1 Initialization: maxsum−r := 0; maxsum−c := 0;
are not reachable from some other states. Actually, this can 2 ∆new := ∅, minsum−r := ∞; minsum−c := ∞;
even be easily observed from the SCA model. After assigning 3 begin
an expected value to LimitCtr (e.g., 10), a C-guard transition 4 foreach si ∈ S, sj P
∈ S do
5 Dsum−si := sj P athS (si , sj );
has been added from l3 to l1 , the negation of the C-guard has
/* Sum of path
P costs between states in row */
been added to the guard of an existing transition from l1 to l2 . 6 Dsum−sj := si P athS (si , sj );
As a result, all the states in the SCA become reachable from /* Sum of path costs between states in column
other states, as presented in Tab. I (the updated data is marked */
7 if Dsum−si > maxsum−r then
in green). The updated models are presented in Fig. 68 . 8 maxsum−r := Dsum−si ; smax−r := si ;
9 if Dsum−si < minsum−r then
ctr1 10 minsum−r := Dsum−si ; smin−r := si ;
1551-3203 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on May 11,2020 at 12:05:12 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2968480, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 8
Algorithm 5: Pseudo-code of T ranCalc (with their relative position remaining unchanged) but can pick
Data: ssrc , sdes , ∆, #model , L and place a workpiece independently. The robot uses its first
Result: ∆new arm to deliver the workpiece from the table to the press. After
1 Initialization:∆new := ∅;
2 begin
being forged by the press, the workpiece is delivered by the
3 for n = 1 : #model do robot’s second arm to the deposit belt. Then the workpiece
4 foreach lsrc ∈ Ln , ldes ∈ Ln do is delivered by the deposit belt to an end station, where it is
5 if lsrc ∈ ssrc and ldes ∈ sdes and lsrc 6= ldes and
@δCtr | δCtr : lsrc × gδ → ldes ∈ ∆ then
checked by a test unit whether the forging is successful. If
6 ∆new += {δCtr | δCtr : lsrc × gδ,Ctr → the workpiece passes the test, it will be output from the cell;
ldes }; otherwise, it will be picked up by the crane and delivered to
/* the expression of gδ,Ctr is assigned
in line 16 in Alg. 3 */
the feed belt for another forging process.
In the system specification, safety aspects also need to be
considered, such as collisions between the robot arms, the
table, and the press, collisions between workpieces on the feed
belt and the deposit belt, both of which has a capacity of two.
cannot affect the original transition behavior. The O-actions
are output signals and do not affect any system behavior at Feed belt X(ERTIdle )∧wp1 ∧T g
ctr1 X(PIdle )
rotary table (ERT), a robot (R), a press (P), a deposit belt (DB), R CW
loc3 ∧ ¬loc4 R A1P ick ∧
R A2P ick
¬wp3 ∧ ¬wp4 ∅ ∧oa1 ∧
oa2 ∧ oa3
and a crane (C) 9 . Apart from the synchronization needed for RM oveB1
X(DBBadOutN oIn ) ∨ X(DBOutput ) ∨
RArm12P icked
X(POutput ) ∧ X(ERTOutput )
cooperation and safety, each component runs independently. X(DBIdle ) ∧ X(PIdle ) ∧ ¬wp5 ∧T g
1551-3203 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on May 11,2020 at 12:05:12 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2968480, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 9
TABLE II
I NPUTS & OUTPUTS OF THE PRODUCTION CELL
Input Description
wp1 / wp2 / wp3 / wp4 / wp5 / wp6 activated when a workpiece is detected by a corresponding sensor
loc1 / loc2 / loc3 / loc4 activated when the crane or the robot reaches a corresponding horizontal position
p high / p middle / p low activated when the press reaches a corresponding vertical position
ert high / ert low activated when the elevating rotary table reaches a corresponding vertical position
test pass / test fail activated when the workpiece passes or fails the test executed by the test unit
Output Description
S Feed the stock feeds a workpiece to feed belt
FB Run / DB Run the feed belt / deposit belt runs (in a forward direction)
ERT Rise / ERT Fall the elevating rotary table rises or falls
R CW / R CCW the robot rotates in a clockwise or counterclockwise direction
R A1Pick / R A1Place / R A2Pick / R A2Place the robot picks or places a workpiece with its first or second arm
P Rise / P Fall the press rises or falls
C MoveFWD / C MoveBWD the crane moves in a forward or backward direction
C Pick / C Place the crane picks or places a workpiece
TABLE III
E VALUATION OF SYSTEM TESTABILITY BEFORE AND AFTER USING THE DTT APPROACH
precisely speaking, within 13 steps. Furthermore, after adding ST code. The result of the previous algorithms is also provided
more C-guards, the controllability can be further improved, as a reference.
e.g., the controllability of 11 steps can be achieved with 4 In summary, compared to the previous algorithms, the new
C-guards. algorithms require less T-guards, less C-guards, and less O-
actions in the individual models (though more O-actions in
total) in achieving full SIC-testability, full observability, and
C. Influences over the executable code
better controllability of the case study.
Further analysis has been done to investigate the influences
of the DTT approach over the executable code of the con-
VII. C ONCLUSIONS AND DISCUSSIONS
troller. PLC Structured Text (ST) code has been generated for
the individual models of the case study automatically via the This paper has presented a design-to-test approach for
tool Teloco [22]. enhancing complete testing of programmable controllers in
The code for the initial models contains 180 lines. The code safety-critical automation systems. The approach aims to im-
is increased by 1 line and modified in 26 lines after adding prove the testability and reduce the testing overhead with
the 26 T-guards, increased by 20 lines after adding the 10 limited design overhead.
O-actions, increased by 6 lines after adding the 3 C-guards. Firstly, system specification is modeled as Moore machines
As examples, following are some lines of the ST code extended with Boolean signals. The models are automatically
related to the T-guards, O-action and C-guards: checked and, if necessary, modified by the DTT approach
• tF B Id3 := X(F B Idle) AND wp1 AND T g; so that they are ensured to achieve full SIC-testability, full
• oa 4 := X(F B Idle) OR X(F B Output); observability and better controllability. The application and
• tR M ov1 := X(R M oveB1) AND ctr 1; benefits of the DTT approach have been illustrated on a
As a result, it can be concluded that the increased overhead benchmark case study.
for adding/modifying the code is linear to the number of the It is worth underlining that all the T-guards and C-guards
inserted T-guards, O-actions, and C-guards. added by the DTT approach can be inhibited in normal
execution, and all the O-actions are output signals that do not
influence the transition functions. Thus, the controller behavior
D. Comparison of system testability on the case study will remain strictly the same in its normal execution.
A quantitative evaluation of the system testability for the The DTT approach presented in this paper deals only with
case study, before and after using the DTT approach, is discrete systems, but its methodology should be more gener-
presented in Tab. III. The results include SIC-testability, ob- ally applicable. In the future, an extension to hybrid systems
servability and controllability, as well as the lines of executable with continuous dynamics would be of interest. Besides, the
1551-3203 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on May 11,2020 at 12:05:12 UTC from IEEE Xplore. Restrictions apply.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TII.2020.2968480, IEEE
Transactions on Industrial Informatics
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS 10
approach composes all individual specification models into an [21] Z. Gu, C. Wang, M. Zhang, and Z. Wu, “WCET-Aware Partial Control
monolithic SCA, which hinders its application to very large- Flow Checking for Soft Error Protection in Resource-Constrained Real-
Time Embedded Systems,” IEEE Transactions on Industrial Electronics,
scale systems, because the SCA size grows exponentially with vol. 61, no. 10, pp. 5652–5661, 2014.
the input number in a system. For future work, the monolithic [22] J. Provost, J.-M. Roussel, and J.-M. Faure, “Translating Grafcet speci-
composition is expected to be replaced by modular methods fications into Mealy machines for conformance test purposes,” Control
Engineering Practice, vol. 19, no. 9, pp. 947–957, 2011.
to enhance the scalability of the whole process. [23] C. G. Cassandras and S. Lafortune, Introduction to discrete event
systems, 2nd ed. Springer Science & Business Media, 2009.
[24] D. Lee and M. Yannakakis, “Principles and methods of testing finite
R EFERENCES state machines - a survey,” Proceedings of the IEEE, vol. 84, no. 8, pp.
1090–1123, 1996.
[1] I. Sommerville, Software Engineering, 8th ed. Pearson Education [25] I. K. Voyiatzis and D. J. Kavvadias, “On the generation of SIC pairs
Limited, 2007. in optimal time,” IEEE Transactions on Computers, vol. 64, no. 10, pp.
[2] V. Vyatkin, “Software engineering in industrial automation: State-of-the- 2891–2901, 2015.
art review,” IEEE Transactions on Industrial Informatics, vol. 9, no. 3, [26] J. Provost, J.-M. Roussel, and J.-M. Faure, “Generation of single input
pp. 1234–1249, aug 2013. change test sequences for conformance test of programmable logic
[3] G. Frey and L. Litz, “Formal methods in PLC programming,” in IEEE controllers,” IEEE Transactions on Industrial Informatics, vol. 10, no. 3,
International Conference on Systems, Man, and Cybernetics, vol. 4, pp. 1696–1704, 2014.
2000, pp. 2431–2436. [27] ——, “Technical report on Conformance Test of Programmable Logic
[4] C. Ma and J. Provost, “Design-to-test approach for black-box testing Controllers – Execution of Minimum-Length Test Sequences,” LURPA,
of programmable controllers,” in IEEE International Conference on ENS Cachan, France, Cachan, Tech. Rep., 2014.
Automation Science and Engineering (CASE), 2015, pp. 1018–1024. [28] A. Guignard and J.-M. Faure, “A conformance relation for model-based
[5] ——, “DTT-MAT: A software toolbox of design-to-test approach for testing of PLC,” in 12th Int. Workshop on Discrete Event Systems,
testing programmable controllers,” in IEEE International Conference on Cachan, may 2014, pp. 412–419.
Automation Science and Engineering (CASE), Fort Worth, Texas, USA, [29] L. Feng, K. Cai, and W. M. Wonham, “A structural approach to the non-
2016, pp. 878–884. blocking supervisory control of discrete-event systems,” International
[6] M. Utting, A. Pretschner, and B. Legeard, “A taxonomy of model- Journal of Advanced Manufacturing Technology, vol. 41, no. 11-12, pp.
based testing approaches,” Software Testing, Verification and Reliability, 1152–1168, 2009.
vol. 22, no. 5, pp. 297–312, aug 2012.
[7] T. Hussain and G. Frey, “UML-based development process for IEC
61499 with automatic test-case generation,” in IEEE International
Conference on Emerging Technologies and Factory Automation, ETFA,
2006, pp. 1277–1284.
[8] E. P. Enoiu, D. Sundmark, and P. Pettersson, “Model-based test suite
generation for function block diagrams using the UPPAAL model Canlong Ma received a B.Sc. degree in Mecha-
checker,” in IEEE 6th Int. Conf. on Software Testing, Verification and tronics from Tongji University, China in 2011,
Validation Workshops, 2013, pp. 158–167. a M.Sc. degree in Automation Engineering from
[9] M. Jamro, “SysML modeling of POU-oriented unit tests for IEC 61131- RWTH Aachen University, Germany in 2013, and
3 control software,” in 19th International Conference on Methods and a Ph.D. degree in the Assistant Professorship for
Models in Automation and Robotics (MMAR). IEEE, 2014, pp. 82–87. Safe Embedded Systems at Technical University
[10] A. Guignard, J.-M. Faure, and G. Faraut, “Model-based testing of PLC of Munich, Germany in 2019. The Ph.D. disserta-
programs with appropriate conformance relations,” IEEE Transactions tion title is “Advances in Model-Based Testing of
on Industrial Informatics, vol. 14, no. 1, pp. 350–359, 2018. Programmable Controllers: Automatic Test Genera-
[11] D. Bohlender, H. Simon, N. Friedrich, S. Kowalewski, and S. Hauck- tion using Design-to-Test and Plant Features”. He
Stattelmann, “Concolic test generation for PLC programs using coverage is currently a functional safety expert at Validas
metrics,” in Discrete Event Systems (WODES), 13th International Work- AG in Munich, Germany. His current research interests include automated
shop on. IEEE., 2016, pp. 432–437. verification and unit- and integration testing of automation and automotive
[12] S. Ulewicz and B. Vogel-Heuser, “Increasing system test coverage in systems.
production automation systems,” Control Engineering Practice, vol. 73,
pp. 171–185, 2018.
[13] G. Gay, M. Staats, M. Whalen, and M. P. Heimdahl, “The risks of
coverage-directed test case generation,” IEEE Transactions on Software
Engineering, vol. 41, no. 8, pp. 803–819, 2015.
[14] C. Ma and J. Provost, “Introducing plant features to model-based
testing of programmable controllers in automation systems,” Control
Engineering Practice, vol. 90, pp. 301–310, 2019.
[15] S. Rösch and B. Vogel-Heuser, “A light-weight fault injection approach Julien Provost received a Ph.D. degree from École
to test automated production system PLC software in industrial practice,” normale supérieure de Cachan, France, in 2011.
Control Engineering Practice, vol. 58, pp. 12–23, 2017. He then joined Chalmers University of Technology,
[16] C. Schotten and H. Meyr, “Test point insertion for an area efficient Sweden, as a Post-Doctoral Researcher for two
BIST,” in IEEE International Test Conference (TC), 1995, pp. 515–523. years. He was an Assistant Professor at Technical
[17] Y. J. Huang, J. F. Li, J. J. Chen, D. M. Kwai, Y. F. Chou, and C. W. University of Munich, Germany where he held the
Wu, “A built-in self-test scheme for the post-bond test of TSVs in 3D Assistant Professorship for Safe Embedded Sys-
ICs,” in 29th IEEE VLSI Test Symp., 2011, pp. 20–25. tems from 2013 to 2019. Currently he works as
[18] A. Koneru, S. Kannan, and K. Chakrabarty, “A Design-for-Test Solution a functional safety expert at Validas AG in Mu-
Based on Dedicated Test Layers and Test Scheduling for Monolithic 3D nich, Germany. His research interest focuses on
Integrated Circuits,” IEEE Transactions on Computer-Aided Design of formal methods for verification & validation (model-
Integrated Circuits and Systems, pp. 1942–1955, 2018. checking to black-box testing); formalization and control of discrete event
[19] G. Xie, G. Zeng, Y. Liu, J. Zhou, R. Li, and K. Li, “Fast Functional systems (requirements formalization, model-based design, and automatic code
Safety Verification for Distributed Automotive Applications during Early generation); and application to cyber-physical systems, automotive, industry
Design Phase,” IEEE Transactions on Industrial Electronics, vol. 65, 4.0, smart home for the aging society.
no. 5, pp. 4378–4391, 2018.
[20] S. Fischmeister and P. Lam, “Time-aware instrumentation of embedded
software,” IEEE Transactions on Industrial Informatics, vol. 6, no. 4,
pp. 652–663, 2010.
1551-3203 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
Authorized licensed use limited to: UNIVERSITY OF BIRMINGHAM. Downloaded on May 11,2020 at 12:05:12 UTC from IEEE Xplore. Restrictions apply.