Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Download
Standard view
Full view
of .
Look up keyword
Like this
1Activity
0 of .
Results for:
No results containing your search query
P. 1
Neural Networks Based Automated Test Oracle for Software Testing

Neural Networks Based Automated Test Oracle for Software Testing

Ratings: (0)|Views: 32|Likes:
Published by Chen Chun

More info:

Categories:Types, Research, Science
Published by: Chen Chun on Nov 27, 2010
Copyright:Attribution Non-commercial

Availability:

Read on Scribd mobile: iPhone, iPad and Android.
download as PDF, TXT or read online from Scribd
See more
See less

11/27/2010

pdf

text

original

 
I. King et al. (Eds.): ICONIP 2006, Part III, LNCS 4234, pp. 498
 
507, 2006.© Springer-Verlag Berlin Heidelberg 2006
Neural Networks Based Automated Test Oracle forSoftware Testing
Ye Mao
1
, Feng Boqin
1
, Zhu Li
2
, and Lin Yao
1
 
1
School of Electronic & Information Engineering, Xi’an Jiaotong University,Xi’an 710049, China
xjtuyemao@163.com
2
School of Software, Xi’an Jiaotong University,Xi’an 710049, China
Abstract.
A lot of test cases must be executed in statistical software testing tosimulate the usage of software. Therefore automated oracle is needed to auto-matically generate the expected outputs for these test cases and compare the ac-tual outputs with them. An attempt has been made in this paper to use neuralnetworks as automated test oracle. The oracle generates the approximate outputthat is close to expected output. The actual output from the application undertest is then compared with the approximate output to validate the correctness.By the method, oracle can be automated. It is of potential application in soft-ware testing.
1 Introduction
The software engineering community has turned attention to statistical software test-ing recently [1, 2, 3]. The main idea is that the reliability of software depends greatlyon the manner in which the software is used. The importance of a failure is largelydetermined by the likelihood of encountering it. Therefore software is tested accord-ing to the model which highlights the critical usage and a lot of test cases must beexecuted to simulate statistically the usage of the software. However it needs a lot of time and is often error-prone to manually generate the expected outputs for these testcases and compare the actual outputs of the application under test (AUT) with them.As a result automated test oracle is needed in statistical software testing to automati-cally generate the expected output and compare the actual output with it. Howeverthere are very few techniques developed to automate oracle. In most cases, a tester isassumed to provide the expected output of the software, which is specified by a tableof pairs [4], logical expressions to be satisfied by the software [5], or temporal con-straints that must not be violated during software execution [6]. Schroeder uses theinput-output (IO) relationships of the software to identify unique combinations of theinputs which influence outputs. By this information a lot of test cases which includingthe inputs and expected outputs can be automatically generated [7]. However deter-mining all IO relationships manually is rather difficult. When testing software withgraphical user interfaces (GUI) by capture/replay tool [8, 9], expected outputs aresaved for comparison while recording test scripts or inserted into the test scripts
 
Neural Networks Based Automated Test Oracle for Software Testing 499
manually. Memon presents planning method to generate expected outputs [10]. Itneeds to construct GUI model and set conditions for every operator manually. Chentests software by finite state machines (FSM), expected outputs are manually includedin the model [11]. Specification language Z uses predicate logic to specify an opera-tion as a relation between the input and output [12]. Test oracle can be generated fromZ specification [13]. However it needs that the user’s requirements of the software arerepresented by Z specification language. To implement more automatic oracle, Ag-garwal use neural networks (NN) based approach to generate expected output [14] fortriangle classification problem [15]. By the experiment they made the conclusion thatNN can be used as test oracle with reasonable degree of accuracy for classificationproblem in software testing.We propose in this paper that the relationship from inputs to outputs of AUT is innature a function. When the function is continuous, an automated oracle is proposed.NN is used to implement the oracle. To appeal of the NN approach lies in its ability toapproximate a function in any precision without the need to have knowledge of thatfunction [16-19]. Experiment has been conducted to validate the effectiveness of theproposed method.In the next section, we briefly describe the theory of multilayer NN. Automatedoracle based on NN is proposed in section 3. Section 4 presents the results of theexperiment. The conclusions and future directions are presented in section 5.
2 Multilayer Neural Networks for Function Approximation
Multilayer NN has been established to be effective method to approximate continuousor other kinds of functions defined on compact sets in
n
 R
[16, 17]. It can be used tolearning the relationship from inputs to outputs by training on the samples. Oncetraining process finishes, it can be given any input and produce an output by the rela-tionship learned. The output generated by NN can be arbitrarily close to the expectedoutput owing to the generalization capabilities of the networks. Back propagationbased on gradient descent in error is the most popular training algorithm for multi-layer NN [17, 18, 19]. In the algorithm the network is initialized with a random set of weights, and then trained from a set of input and output pairs, i.e. training samples.Training process stops when the training error is acceptable or a predefined number of epochs pass. Once trained, network weights are kept and used to approximate thefunction. When training NN, the weight update rule can be expressed as follows:
)()()1()(
n ynnwnw
i j ji ji
ηδ α 
+Δ=Δ
 (1)where
α 
is a positive number called the momentum constant,
)(
nw
 ji
Δ
is the cor-rection which is applied to the weight connecting the output of neuron
i
to the inputof neuron
 j
at the
n
th
iteration,
η 
is the learning rate,
)(
n
 j
δ 
is the local gradient atthe
n
th
iteration, and
)(
n y
i
is the function signal appearing at the output of neuron
i
at the
n
th
iteration. The training error can be the sum over output units of the
 
500 Y. Mao et al.
squared difference between the expected outputs
given by a teacher and the actualoutput
 z
:
=
==
c
z zw J 
122
21)(21)(
 (2)where
and
 z
are the target and the network output vectors of length
c
respectivelyand
w
are the weights in the network. Details about the back propagation algorithmcan be found in references [18, 19].
3 Automated Test Oracle Based on Neural Networks
3.1 Model of the Automated Oracle
Function testing involves executing an AUT and examining the output, which is im-plemented by oracle. General model of oracle is as Fig.1. In the model expectedoutputs are generated from inputs and then compared with actual outputs from AUT.If they are not same, it implies a failure. The process of generating expected outputsand comparing is traditionally done manually. Testers compute the expected outputsfrom program specifications or their knowledge of how a program should operate.Expected outputs are then compared with actual outputs by tester’s knowledge todetermine if a failure occurs.
Fig. 1.
General oracle model
The relationship from the input to output of the software is in nature a function. Let
),,,(
21
n
 x x x x
=
and
),,,(
21
m
 y y y y
=
be input and output vectors respec-tively. Therefore the relationship can be represented by
)(
 x f  y
=
, where
 f 
imple-ment the specification of the software. When
n
 R x
,
m
 R y
, and
 f 
is continuous,the function
 f 
can be approximated by NN after training. In this paper, we propose theautomated oracle in this situation. Each component in
 y
is also a function of 
x
, thatis
)(
 x f  y
 j j
=
. Let
)(
i
 x D
be the set of all possible values of 
i
 x
and
)(
 x D
be the setof all possible values of 
x
. Therefore
)(
 x D
includes every possible combination of the

You're Reading a Free Preview

Download
scribd
/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->