2013 IEEE Seventh International Symposium on Service-Oriented System Engineering
Automation of Test Case Generation and Execution
for Testing Web Service Orchestrations
Vera Stoyanova
Faculty of Mathematics and Informatics
Sofia University “St. Kliment Ohridski”
Sofia, Bulgaria
vera.stoyanova@yahoo.com
Abstract—Testing service orchestrations is a challenging research
domain due to necessity of additional testing efforts,
complementing traditional software testing. Although various
testing approaches and tools have been proposed, most of them
provide partial solutions covering single testing activities such as
test path analysis, test case generation, web service emulation,
fault injection and so on. Following the current research
direction, we have developed an integrated testing framework,
called TASSA, which aims to provide end-to-end testing of
Business Process Execution Language (BPEL) orchestrations.
This paper introduces its core capability for automation of test
cases generation, execution and management implemented in an
open source tool for WSDL-based testing of both single web
services and composite web services, described with BPEL. The
tool’s functionality includes identification of web service
operations as well BPEL variables in case of service composition
testing, generation of SOAP request templates, data driven
testing, definition of assertions at different levels (HTTP, SOAP
and BPEL variable), execution and management of test cases.
Keywords— BPEL, test case, web service testing, WSDL
I. INTRODUCTION
Service-Oriented Architecture (SOA) is a widely adopted
computing paradigm for development of distributed
applications. Its vision is preferably implemented using web
services, which are built and deployed on heterogeneous
platforms and orchestrated to achieve a particular business
goal. The web service orchestrations are usually described with
Business Process Language for Web Services (WS-BPEL) that
is a complex standard presenting additional challenges to
software development and testing.
Testing business processes brings various challenges at
infrastructure, service as well as orchestration level [6]. Since
the design information and the source code of both
infrastructure components and web services are missing, the
usage of white box testing techniques is not applicable. The
reusability of web services allows them to be consumed by
multiple clients with different quality requirements in variety
contexts, which are unknown at the development phase. This in
turn complicates the testing, since the coverage of all use cases
is practically impossible. Furthermore, the web services are
developed by different software teams and controlled by
different providers. That is why establishment of common
978-0-7695-4944-6/12 $26.00 © 2012 IEEE
DOI 10.1109/SOSE.2013.9
Authorized licensed use limited to: Instituto Nacional De Telecomunicações (INATEL). Downloaded on August 05,2022 at 22:12:23 UTC from IEEE Xplore. Restrictions apply.
Dessislava Petrova-Antonova, Sylvia Ilieva
Institute of Information and Communication Technologies
Bulgarian Academy of Science
Sofia, Bulgaria
d.petrova@fmi.uni-sofia.bg, sylvia@acad.bg
semantic, fault handling and transactional models is very hard
to achieve. This leads to difficulties when testing service
compositions. Additionally, some of the orchestrated web
services could be unavailable or under development at the time
of business process testing. The lack of control over partner
web services and their distributed nature cause performing of
emulations, generation of stubs, simulation of communication
failures, and so on.
Although various approaches and tools for WS-BPEL
testing have been proposed, most of them provide partial
solutions covering single testing activities such as test path
analysis, test case generation, web service emulation, fault
injection and so on. However, in order to facilitate the testing
process, it is important to integrate all testing activities in a
common automated platform. Therefore, the current research
efforts focus on the development of testing frameworks that
complements existing development environments and can be
used jointly to achieve end-to-end testing. Following this
research direction, we have developed our flexible TASSA
framework for testing of BPEL processes [9]. This paper
introduces its core capability for automation of test case
generation, execution and management implemented in Test
Case Generation and Execution Tool (TCGET).
The rest of the paper is organized as follows. Section II
analyses current tools for testing SOA implementations.
Section III presents the architecture of TASSA framework
focusing of the module structure of the TCGET. Section IV
describes the main features of the TCGET related to test case
generation and execution. A case study showing the usage of
the TCGET to testing a sample BPEL process is presented in
Section V. Section VI gives directions for future work.
II. RELATED WORK
This section presents a comparative analysis of the current
software tools for testing of SOA implementations. Note that
the goal of the analysis is to examine the solutions providing
similar functionality to that of TCGET. Here, the software
platforms that are competitive to TASSA framework as a
whole are not covered.
The criteria for comparison of the testing tools are chosen
according to their common features described in the available
documentation. They cover the generation of test cases based
274
 on WSDL or BPEL file (Feature 1); the definition of SOAP conditional test execution are available only in licensed
requests to particular operation from the WSDL file (Feature versions.
2); the support of data driven tests (Feature 3); the definition of
The results from the analysis are summarized in Table I.
assertions for the SOAP response (Feature 4); the definition of
assertions for the WS-BPEL process’s variables (Feature 5);
the possibility for integration with widely used Integrated TABLE I. TESTING TOOLS FOR SOA IMPLEMENTATIONS
Development Environments (Feature 6); the support of free
software license (Feature 7); the providing of standalone Testing Tools
Feature Oracle BPEL
Graphical User Interface (Feature 8). The core functionality of SoapUI SOAtest BPELUnit SoapSonar
Test Framework
the software tools for testing is defined by the testing artifacts.
Feature 2 + + + + +
Therefore, in order to provide objective comparison, all the
Feature 3 - - + + -
tools under analysis must cover the first criterion, namely their
Feature 4 + + + + +
main testing artifact should be WSDL or BPEL file. As a result
Feature 5 - + - + -
five testing tools were examined: SoapUI, BPELUnit, SOAtest,
Eclipse,
Oracle BPEL Test Framework and SoapSonar. Feature 6 IntelliJ Idea, JDeveloper Eclipse Eclipse -
NetBeans
SoapUI is an automated testing solution with rich
Feature 7 + + - + +
functionality [1]. It supports leading technologies and standards
Feature 8 + - + - +
such as SOAP and REST web services, Java Message Service
(JMS), Adobes ActionScript Messaging Format (AMF), Java
Database Connectivity (JDBC), etc. Its core capabilities Our solution covers all features mentioned so far. It
include functional testing, service mocking, security testing and supports testing of SOA implementations, which interface is
load testing. Unfortunately, some essential features such as described with WSDL. Therefore, it can be used for testing
data driven testing, multi environment support, coding free test both single web services and web service compositions,
assertion and tree-based input for easy manual testing are described with WS-BPEL. Furthermore, TCGET provides
provided only with the professional edition of the tool, which is well-defined project structure, in which each test case is
not free for use. In addition, the SoapUI does not allow described in separate XML file. In contrast, the BPELUnit
definition of assertions for WS-BPEL process’s variables when stores all test cases that belong to given testing project in a
the WSDL under test describe a business process interface. single file. This in turn hampers the testing activities, especially
Such feature is presented in Oracle BPEL Test Framework in case of large number of test cases. Additionally, TCGET is
provided by Oracle BPEL Process Manager [2]. It is fully implemented as open source solution, which is fully integrated
integrated with Oracle JDeveloper, providing functionality for with NetBeans IDE.
simulation of web service interactions, validation of process
actions with test data, calculation of activities’ coverage, etc.
III. ARCHITECTURE OF TASSA FRAMEWORK AND TCGET
Parasoft SOAtest is an integrated solution for end-to-end The TASSA framework provides a methodology and set of
testing of SOA implementations [3]. Since version 4.5, it tools for testing business processes described with WS-BPEL.
supports automated testing of business processes, described This section describes the general concept behind it as well as
with WS-BPEL. Along with this, the SOAtest performs the architecture of the TCGET.
checking of the reliability, security, and maintainability of web
services developed against WS-I, W3C, OASIS, and WS- A. TASSA framework
standards. However, since it is distributed with proprietary
software license, which is quite expensive, only the big The TASSA framework supports functional as well as non-
software vendors can take its advantages. In contrast, the functional testing of business processes. The functional testing
BPELUnit is a testing framework developed within an open includes path coverage according to functional requirements,
source project [4]. It provides white-box testing of business full branch coverage with shortest path length of the business
processes described with WS-BPEL, supporting service process, and path coverage according to new functionality. The
mocking and calculation of path coverage. Although the non-functional testing is achieved through robustness and
framework allows execution of data driven tests, it does not scalability testing. The robustness testing is conducted by
provide a GUI for data source selection and variable definition simulations of possible failures of unreliable service(s) and
for the request message. In addition, it supports only XPath tests compositions of both reliable and unreliable services. The
matching assertions. Assertions related to HTTP status code, scalability testing covers many configuration changes.
fault response message, valid response message and existence The testing capabilities of the TASSA framework are
of string token in response message, covered for example by provided by a set of tools each of which covers different testing
the SoapUI, are not provided. issues. The Isolation Tool (IT) provides the functionality
SoapSonar is a testing tool providing functional, needed to isolate the BPEL process under test from outside
performance, security and compliance testing of web services dependencies. It works by transforming the WS-BPEL file
[5]. It simulates web service interaction with clients by describing the business process so as to remove one or more
automated generation of client requests. The free version of the dependencies, and replace them with activities that are internal
tool supports only functional testing. Features such data driven to the BPEL and can mimic the output of the original activity.
tests, capture of dynamic response HTTP header value and For a given set of WS-BPEL activities, the Data Dependency

275

Authorized licensed use limited to: Instituto Nacional De Telecomunicações (INATEL). Downloaded on August 05,2022 at 22:12:23 UTC from IEEE Xplore. Restrictions apply.
 Analysis Tool (DDAT) finds a path that goes through all
activities in the set starting from one initial activity [7]. Also, it
finds all control activities on the discovered path and calculates
the condition that should be met in order for the process to
continue execution along the path. The Fault Injector Tool
(FIT) injects delays, errors, etc. in the message exchange for a
particular WS-BPEL activity. The main function of FIT is to
make modifications in a way that the BPEL file describing the
process can be tested against fault injection [8]. The Value
Generation Tool (VGT) provides appropriate values that
DDAT needs for the variables of the conditional activities. It
also communicates with FIT, when faults in message data need
to be simulated. The Test Case Generation and Execution Tool
(TCGET) automates the test case generation and execution and
along with other TASSA tools provide end-to-end testing of
web service compositions.
B. TCGET Architecture
TCGET is a solution for WSDL-based testing of both
single and composite web services. In other words, the main
testing artifact of the tool is a WSDL description of the service
under test. Its core functionality supports:
• Identification of web service operations as well BPEL
variables in case of business process testing;
• Generation of SOAP request templates;
• Execution of single test case, all test cases in a test
suite and all test cases in a test project;
• Management of test cases and file organization of
testing project including grouping of test cases in test
suits, updating of test cases and test suites, and so on;
• Data driven test cases;
• Assertions at HTTP, SOAP and BPEL variable level;
• Integration with NetBeans IDE.
The TCGET consists of nine modules depicted with a
component diagram in Fig. 1.
Figure 1. Test Case Generation and Execution Tool Architecture
The Test Model Definition module implements the main
interfaces and classes of the domain model, namely TestCase,
276
Authorized licensed use limited to: Instituto Nacional De Telecomunicações (INATEL). Downloaded on August 05,2022 at 22:12:23 UTC from IEEE Xplore. Restrictions apply.
TestSuite, BpelOperation, etc. The Test Definition Engine
module manages test cases and test suites. The Persistence
Engine module performs serialization and deserialization of test
cases. The SOAP Request Generator module handles the
generation of template SOAP requests based on a selected
operation from WSDL file. The Test Execution Engine module
performs execution of test cases. The Template Test Processor
module provides support for generation data driven tests. The
Settings module is responsible for the tool configuration related
to connection timeout, request charset, request content type,
proxy server address and port, address of OpenESB server and
port, user name and password. The Assertion Manager module
implements different types of assertions, described in the next
section, and provides GUI for definition and modification of an
assertion in a test case. The NetBeans plug-in packages the
functionality of the TCGET in a NetBeans plug-in.
IV. TEST CASE GENERATION AND EXECUTION
The core functionality of the TCGET is related to test case
generation, organization and execution. This section describes
the key aspects of these issues.
A. Test case generation
The BPEL Test Project, created by the TCGET, is related to
a single WSDL file that corresponds to the business process or
web service under test. During the creation of a new project the
wizard automatically extracts the WSDL file from a specified
location and creates an empty project with a node named
WSDL. The WSDL node provides a context menu, which is
populated with all available operations under test. After the
tester selects an operation for testing, he/she is prompted to
specify a name for a new test case and a test suite, where the
test case will be placed. Next, the Test Case Editor (TCE) of
the TCGET is opened. The newly created test case is
automatically populated with an empty SOAP request template
relevant to the selected operation. In order to facilitate test data
definition, the generated SOAP request is supplied with hints
for the data types of the SOAP body elements.
Each test case is stored in a separate XML file with
extension .BTC. The test case structure is displayed with UML
object diagram in Fig. 2.
Figure 2. Test case structure
As is shown in Fig. 2, the test case has two attributes –
“name”, defining the test case name, and “template”,
specifying whether the test case is data driven. The main XML
elements of the test case are “description”, “operation,
“testInput”, “datasource” and “assertions”. The element
“description” provides narrative description of the test case.
 The element “operation” specifies the WSDL operation under
test including service name, port name and operation name.
The element “testInput” stores the input data described with
elements “body”, “header” and “authorization”. The
“datasource” element defines the datasource when data driven
test case is created. Currently, two types of datasource are
supported, namely CSV and JDBC. The last element
“assertions” is responsible for definition of test case assertions.
The TCGET supports seven types of assertions. The “HTTP
Status Code” assertion verifies the status code of the HTTP
response. The “Response Time” assertion verifies that the
response is received in a given period of time. The “SOAP
Fault” assertion verifies that the HTTP response contains a
valid SOAP Fault message. The “SOAP response valid”
assertion verifies that the HTTP response contains a valid
SOAP message. The “XPath Equals” assertion verifies that
when a certain XPath expression applied to a SOAP body or a
BPEL variable, its result is equal to a particular value. It is
possible to make it case insensitive and/or match it with regular
expression. The “XPath Exists” assertion verifies if a certain
XPath expression applied to a SOAP body or a BPEL variable
is not empty. The “Contains” assertion verifies that a certain
string exists within a SOAP body or a BPEL variable. In
addition, the TCGET provides Negated assertion, which can be
applied to all assertions defined above.
B. Test case organisation
The test cases related to a certain WSDL file are organized
in a custom NetBeans project. The project’s files are visualized
in a tree structure. The project contains a WSDL file and an
optional XSD file that corresponds to the web service/business
process interface. The test cases are grouped in test suites. The
test suites are stored in separate directories, which are
subdirectories of one root directory, called “TestSuites”. As it
is already mentioned, each test case is stored in a separate
XML file. The metadata of the project is saved in a XML file,
named tassa-tcme-project.xml.
C. Test case execution
The execution of test case includes two steps: (1) sending
of SOAP request to the proper web service address defined in
the WSDL file of the business process, and (2) receiving and
assessment of obtained result according to the assertions
defined in the test case. On the first step the information stored
in the test case is used to retrieve the web service address,
where the SOAP request should be sent and the message parts
such as HTTP headers, user name and password if HTTP
authentication is needed, and data of SOAP request body. The
obtained result from test case execution consists of HTTP
headers of the received response, data of SOAP response body,
values of process variables and execution time.
The actual data of the SOAP request body and the expected
data in the SOAP response body are replaced with variables
when data driven tests are defined. In such case, first a
connection to the data source needs to be established. Second,
for each record in the data source a separate test case is created,
where the variables are replaced with concrete values. After,
the generated test cases are executed ordinarily.
277
Authorized licensed use limited to: Instituto Nacional De Telecomunicações (INATEL). Downloaded on August 05,2022 at 22:12:23 UTC from IEEE Xplore. Restrictions apply.
The most important task with respect of testing is analysis
of the testing result on which all assertions defined in the test
case should be applied. The obtained data from test case
execution are encapsulated in a single object that is processed
by the Test Execution Engine module of the tool. The
following test execution statuses are used by the module to
assess the test result:
• Success – The result satisfies the test assertions.
• Failure – The result does not satisfy the test assertions.
• Error – The assertion is not verified. For example, an
XPath assertion is defined and the XPath statement is
not valid according to the XPath specification.
• Unknown – The status before checking of the assertion.
The Test Execution Engine module provides functionality
for subscription to events related to the test case execution.
This functionality is used in the implementation of the GUI
panel, which is responsible for visualization of the test cases in
a tree structure with icons indicating their statuses.
V. CASE STUDY
This section presents the TCGET capabilities through
testing a manually generated business process providing
functionality for withdraw and transfer of amount in a bank
system.
A. Business process under test
The business process under test (BPUT) has eight partner
web services providing the functionality as follows: (1) Login,
logout and registration of users; (2) Risk assessment of the
performed transaction related to withdraw and sending amount;
(3) Deposit, withdraw and transfer of amount, and checking of
account balance; (4) Currency conversion; (5) Calculation of
commission. The commission is calculated based on FX Rate
of the currency and amount due for payment provider routing;
(6) Logging; (7) Automated notification via email, SMS, etc.;
(8) Finding payment provider. The web service returns a
unique identifier of the provider. The BPUT is deployed on a
GlassFish application server and communicates with its partner
web services through SOAP messages.
B. Test cases
In this case study, we have used the GUI of the TCGET to
define the following test cases, which cover all types of
assertions supported by the TCGET:
• TC1: The BPEL process should revoke any attempts to
transfer money without valid login credentials. The test
sends a valid SOAP request but with incorrect
password and checks that the response is a valid SOAP
response with the message that the login credentials are
not correct.
• TC2: The valid transaction types for the BPEL process
are “sendMoney” and “depositWithdraw”. Any
unknown transaction type should be revoked. The test
sends a valid SOAP request with correct login
credentials but with wrong transaction type and checks
 that the response is a valid SOAP response with the TABLE III. TEST CASE INPUT DATA
message that the transaction type is incorrect.
Test SOAP request element
• TC3: The BPEL process should deny using more case recipientUserId amount country paymentType currency
money than a person actually has. The test sends a
TC1 4 130.00 Bulgaria VISA BGN
valid bank operation request with an amount bigger
than the available for the user and checks that the TC2 4 130.00 Bulgaria MASTER BGN
money transfer is being revoked.
TC3 4 1300000.00 Poland VISA PLN
• TC4: The test sends a valid SOAP request with
“sendMoney” transaction type and verifies that the TC4 4 10.00 USA MASTER USD
transaction has been executed successfully. The test TC5 4 10.00 Poland MASTER PLN
also asserts the money conversion has passed
successfully by verifying the BPEL variable: TC6 2 10.00 USA VISA USD
SendMoneyIn and the amount in euro.
• TC5: The test sends a valid SOAP request with The test case outputs are presented in Table IV. The table
“depositWithdraw” transaction type and verifies that shows assertions defined for each test case as well as their
the transaction has been executed successfully. statuses after test case execution.
• TC6: The BPEL process should revoke any attempts to
send money to nonexistent recipients. The test sends a TABLE IV. TEST CASE RESULTS
valid request with invalid recipient identifier and
Test Assertion
checks that the transaction was not successful. case
Assertion description
Result
Status

Fig. 3 shows the format of the SOAP requests, which is HTTP Status Code (200) Value: “200” Passed
sent to the BPUT during test case executions.
Actual time: 572
Response Time = (650 ms) Passed
<soapenv:Envelope ...> ms
<soapenv:Body> Valid SOAP
<bank:bankOperation> SOAP response valid Passed
<request> TC1 response
<bank1:transactionType></bank1:transactionType> Contains (case sensitive: "true", is regex:
<bank1:login> Value found Passed
"false") text "Incorrect" in Response
<bank1:username>...</bank1:username>
<bank1:password>...</bank1:password> Response XPath Equals (xpath: "Envelope/ Value:
</bank1:login> Body/bankOperationResponse/response/test “Incorrect
<bank1:sendMoneyFraudCheck> Passed
", value:"Incorrect username or password", username or
<bank1:recipientUserId>...</bank1:recipientUserId>
<bank1:amount>...</bank1:amount> case sensitive: "true", is regex "false") password”
</bank1:sendMoneyFraudCheck>
<bank1:providerRouting>
HTTP Status Code (200) Value: “200” Passed
<bank1:country>...</bank1:country>
Actual time: 572
<bank1:paymentType>...</bank1:paymentType> Response Time (700 ms) Passed
<bank1:currency>...</bank1:currency> ms
</bank1:providerRouting> Valid SOAP
... SOAP response valid Passed
</soapenv:Envelope>
response
No SOAP Fault
Not SOAP Fault Passed
in response
Figure 3. SOAP request template TC2
XPath Exists in Response
XPath found Passed
("descendant::bankOperationResponse")
The actual values of SOAP body elements for each test case Response XPath Equals (xpath:
Value: “Invalid
are presented in Table II and Table III. "descendant::test", value:"Invalid
Transaction Passed
Transaction Type", case sensitive: "true", is
Type”
regex "false")
TABLE II. TEST CASE INPUT DATA Not Contains (case sensitive: "false", is
Value missing Passed
regex: "false") text "success" in Response
Test SOAP request element
case HTTP Status Code = 200 Value: “200” Passed
transactionType username password
WRONG Valid SOAP
TC1 sendMoney john.smith@yahoo.com TC3 SOAP response valid response
Passed
PASSWORD
INVALID Not Contains (case sensitive: "false", is
TC2 john.smith@yahoo.com johnspass Value found Failed
TRANSACTION TYPE regex: "false") text "success" in Response

TC3 sendMoney john.smith@yahoo.com johnspass HTTP Status Code = 200 Value: “200” Passed
Actual time: 517
TC4 sendMoney john.smith@yahoo.com johnspass Response Time = 600 ms Passed
ms
TC4
Valid SOAP
TC5 depositWithdraw john.smith@yahoo.com johnspass SOAP response valid Passed
response
TC6 sendMoney john.smith@yahoo.com johnspass No SOAP Fault
Not SOAP Fault Passed
in response

278

Authorized licensed use limited to: Instituto Nacional De Telecomunicações (INATEL). Downloaded on August 05,2022 at 22:12:23 UTC from IEEE Xplore. Restrictions apply.
 Test Assertion by various tools and frameworks, there is still lack of tool
Assertion description Status support for testing SOA implementations. The presented
case Result
Contains (case sensitive: "true", is regex: solution in this paper provides functionality for identification
Value found Passed
"false") text "success" in Response of web service operations as well as BPEL variables in case of
Response XPath Equals (xpath:
Value: business process testing, generation of SOAP request
"Envelope/Body/bankOperationResponse/r templates, data driven testing, definition of assertions at
“Transaction
esponse/test", value:"Transaction was Passed
successfully executed", case sensitive:
was successfully different levels (HTTP, SOAP and BPEL variable), and
executed” execution and management of test cases. As a part of TASSA
"true", is regex "false")
BPEL variable "SendMoneyIn" XPath framework, it contributes to achieve end-to-end testing of
Equals (xpath: Value: BPEL orchestrations, which is a challenging task, given the
"message/part/sendMoney/amount", ”5.08820512820 Passed distributed nature of the system under test. Since the tool has
value:"5.088205128205129", case 5129” its own GUI and is fully integrated with NetBeans IDE, it can
sensitive: "true", is regex "false") be used outside TASSA framework automating functional
HTTP Status Code = 200 Value: “200” Passed testing of web services as well as business processes, described
Actual time: 540 with WS-BPEL.
Response Time = 600 ms Passed
ms
The future work includes further improvement of the
Valid SOAP
SOAP response valid
response
Passed TCGET in two directions: support of more types of assertions
No SOAP Fault and implementation of GUI allowing usage of the tool as stand-
Not SOAP Fault Passed alone application outside of NetBeans IDE. In addition, we will
TC5 in response
Contains (case sensitive: "true", is regex: evaluate the performance of the proposed solution in
Value found Passed
"false") text "success" in Response comparison with other existing testing tools for web services.
Response XPath Equals (xpath:
Value:
"Envelope/Body/bankOperationResponse/r
esponse/test", value:"Transaction was
“Transaction
Passed
ACKNOWLEDGMENT
was successfully
successfully executed", case sensitive: This work has been supported by the National Scientific
executed”
"true", is regex "false") Fund, Bulgarian Ministry of Education and Science under grant
HTTP Status Code = 200 Value: “200” Passed agreement number D02-182.
Actual time: 442
TC6 Response Time = 600 ms ms
Passed
REFERENCES
Not Contains (case sensitive: "false", is
Value found Failed [1] soapUI v.2.5.1, Project Documentation, 2009, http://www.whiteboxqa.
regex: "false") text "success" in Response
com/StudentMaterial/Books/D%20-%20 soapui.pdf
[2] L. Dikmans, “Testing BPEL in the Real World,” 2007,
The experiment results show the ability of the TCGET to http://www.oracle.com/technetwork/issue-archive/2007/07-nov/o67bpel-
100400.html.
facilitate testing of SOA implementations by supporting variety
[3] Parasoft Corporation, “Parasoft SOAtest Best Practices,”
test case assertions. As is shown in Table III, TCGET provides http://www.jameslewiscoleman.info/jlc_stuff/project_research/SOAtest_
assertions related to HTTP status code, SOAP response, Best_Practices.pdf.
content of SOAP body and BPEL variable, and Response time. [4] Ph. Mayer and D. Lübke, “Towards a BPEL unit testing framework,” in
It is worth noting that the last one concerns the non-functional Proceedings of the 2006 workshop on Testing, analysis, and verification
testing of the BPUT and is not provided by Oracle BPEL test of web services and applications (TAV-WEB’06), July 17, 2006,
framework, BPELUnit and SoapSonar. Furthermore, three Portland, Maine, USA, pp. 33-42.
types of assertions for content verification of the SOAP body [5] D. Esposito, “SOAPSonar Measures the Distance Between Your Web
and BPEL variable are provided. Two of them are based on Services and the Real World,” 2006, http://www.codeproject.com/
Articles/15182/SOAPSonar-Measures-the-Distance-Between-Your-
XPath expressions. Additionally, all supported assertions can Web-S.
be negated. For comparison, Oracle BPEL test framework [6] Morris E., W. Anderson, S. Bala, D. Carney, J. Morley, P. Place, S.
supports only three assertions, namely Simple Value Assert, Simanta, “Testing in Service-Oriented Environments,” Technical Report
XML Assert and Activity Executed Assert. The BPELUnit CMU/SEI-2010-TR-011, Carnegie Mellon University, 2010.
provides only XPath assertions, which consist of an XPath [7] Spassov I., D. Petrova-Antonova, V. Pavlov, S. Ilieva, “DDAT: Data
query and an expected value. Rich functionality with respect of Dependency Analysis Tool for Web Service Business Processes,” Int.
assertions is presented in SoapUI. Since the main purpose of Workshop on “Software Quality” within ICCSA, June 20-23, 2011,
Santander, Spain, Part V, LNCS 6786, Springer, pp. 232-243.
the SoapUI is to test web services, the assertions over BPEL
[8] Manova, I., D. Manova, V. Pavlov, S. Ilieva, D. Petrova-Antonova,
variables are not supported. “Fault Injection Testing of Web Service Business Processes,”
International Journal on Information Technologies & Security, Year 3,
VI. CONCLUSION No. 2, ISSN 1313-8251, 2011, pp. 3-12.
[9] Pavlov V., B. Borisov, S. Ilieva, D. Petrova-Antonova, “Framework for
The present work addresses test case generation, execution Testing Service Compositions,” Proceeding of the 12th International
and management on single web services and BPEL Symposium on Symbolic and Numeric Algorithms for Scientific
orchestrations. While traditional software testing is supported Computing , Timisoara, Romania, September 23-26, 2010, pp. 557-560.

279

Authorized licensed use limited to: Instituto Nacional De Telecomunicações (INATEL). Downloaded on August 05,2022 at 22:12:23 UTC from IEEE Xplore. Restrictions apply.