International Journal of Computational Intelligence and Information Security, October 2012 Vol. 3, No.

8 ISSN: 1837-7823

Comparative Study of Automated Testing Tools: SAHI and SELENIUM

Manpreet Kaur DIAS, GGSIP University, Delhi, India Abstract
Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process Testing automation tools enables developers and testers to easily automate the entire process of testing in software development. The objective of the paper is to conduct a comparative study of automated tools such as the Sahi and Selenium based on criteria such as the efforts involved with generating test scripts, capability to playback the scripts, result reports, speed and cost. The primary objective of this research is to analyze the features supported by these two testing tools for script reuse. For the purpose of this project we took an existing VB based application that was inventory management and perform functional testing on it by these two automated testing tools. Keywords: Sahi, Selenium, testing process, software testing, automation tools.

1. Introduction
Software testing provides an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not limited to, the process of executing a program or application with the intent of finding software bugs [7]. Software testing can be stated as the process of validating and verifying that a software program/ application/ product. There are two ways of testing that is manual and automated. Manual testing carried out by the testers. Testers test the software manually for the defects. Here tester has to act as an end user. Automation testing automates the steps of manual testing using automation tools such as Sahi and Selenium. It increases the test execution speed.

2. Problem Statement
Testing plays a major role in software development process. There are various automated testing tools in the market. When we initiate research to find the right automated tool, it is vital to decide the requirements when choosing a tool for evaluation. This research evaluate two major tool vendors that are Sahi and Selenium on their test tool characteristics, test execution capability, test reporting capability, scripts reusability capability, and playback capability.

2.1 Background
In this section, I listed the papers that reviewed during this project. With the shrinking development cycles, pressure of reducing the time to go-to-market and repetitive processes, automation is no longer optional. Also, there is no relaxation on the quality of the final products; companies just cant afford to launch products that are not ready for the prime time. Therefore, it is not surprising that the demand for test automation tools has increased drastically. The paper surveys a set of tools that support the testing process in a variety of ways. Some tools simulate the final execution environment as a way of expediting test execution, others automate the development of test plans, and still others collect performance data during execution. In these tough economic times, software29

International Journal of Computational Intelligence and Information Security, October 2012 Vol. 3, No. 8 ISSN: 1837-7823 development managers are pushing to get more and better testing done faster. Most recognize the automated testing tools facilitate higher quality and more productive testing, but acquiring such tools is often complicated. The paper had given the evaluation criteria for selecting the testing tools

3. Evaluation Study
The two tools Sahi and Selenium has been opted for this study. Comparison between these tools have been made on the following parameters: 1. Recording efficiency 2. Programming language support 3. Ease of use 4. Stability of scripts and maintenance 5. Data Driven Testing 6. Reporting

3.1 Recording Efficiency

Both tools are recording and playback type tools. While recording commands are inserted to check the application works as intended. Sahi is a powerful recorder which works across browsers. It gives very easy access to controls. Sahi's core parts include the Sahi proxy server and the JavaScript engine.[6] Html responses which pass through the proxy are modified such that JavaScript is injected at the start and the end of the response. This allows the browser to record and playback scripts and talk back to the proxy when needed. Selenium does provide easy access to controls. When we hit the record button, application is started. It records all the actions as performed by user. But relabeling a button or moving it to another part of window may require the test to be re recorded.[5]

3.2 Programming language support

Thousands of different programming languages have been created, mainly in the computing field Programming languages differ from most other forms of human expression in that they require a greater degree of precision and completeness. Sahi supports Sahi Script , Java , Ruby Sahi Script has the syntax of java script but can interoperate with any Java code directly from script. Sahi Script can directly invoke anything exposed by javascript. Selenium supports Java, Ruby, Perl, Python, C#. Selenium needs language bridges for each new feature. For example, needs java bridge to invoke Flash via External API.[1]

3.3 Ease of Use

Sahi is difficult to start because it needs installation of Sahi. Estimated start time 10-30 minutes, depending on Java installation etc. There is only one mode of operation for Sahi. Extremely simple to learn and use for testers.[3]It Can achieve most automation with just functions and variables. Sahi has inbuilt APIs for most other complex tasks like data driven testing. It Has nearness APIs like _in and _near which can help show nearness of elements. eg. _image (delete.gif, _near(username 4). Selenium is easy to start with because of Selenium IDE which is a fire fox plug in. Estimated start time less than 5 minutes. It is little difficult for testers to use, knowledge of programming language required. It uses XPath based identification for elements in complex html structures or those with dynamic ids. css selectors and java script may also be used.

3.4 Stability of Scripts and maintenance

Stability of scripts plays a major role in deciding on the automation tool to be selected as it decides on the speed, time required by an application to complete. In Sahi, there is no need to explicitly wait for page load and AJAX which thus automatically saves time, keeps script simple and also reduces random failures. Sahi is easy to understand and thus maintained, it is easily adopted by a team of testers.[6] 30

International Journal of Computational Intelligence and Information Security, October 2012 Vol. 3, No. 8 ISSN: 1837-7823 In Selenium, on the other hand, we have to explicitly wait for page load thus wasting lot of time on it and probably the random failures. Testers need to know about Test NG/Junit,Xpaths,HTML and frames knowledge before using it.

3.5 Data Driven Testing

Data-driven testing is the creation of test scripts to run together with their related data sets in a framework. The framework provides re-usable test logic to reduce maintenance and improve test coverage. Input and result (test criteria) data values can be stored in one or more central data sources or databases, the actual format and organization can be implementation specific.[2]Nowadays data-driven testing becomes very important part of testing. Instead of recording multiple tests to test multiple sets of input data, it is possible to make the scripts access the different sets of input data from external source line data tables. Sahi has some nice inbuilt features for data-driven testing which make the testing simple. The core concept is that data is present as a grid of rows and columns, and tests need to run with each row of the grid as input. Selenium does not provide any out-of-the box solution for data driven testing but leaves it up to the user to implement this on his own input.

4. Result
For the purpose of rating the comparison parameters, we have used 5-point scale i.e. 5, 4, 3, 2, 1 as very Good, Good, Average, Bad, and very Bad respectively. Each parameter has been divided into sub criteria. The total value of parameter is calculated by taking average of sub criteria values.

4.1 Recording Efficiency

Sub-criteria Insert commands while recording pause Auto documentation Sahi 2 5 5 Selenium 5 5 5 Remarks In sahi we cannot insert command Both tools provide facility to pause while recording. Both tool provides this facility

For sahi the value is (2+5+5)/3 =4 For selenium the value is (5+5+5)/3=5 Thus selenium is very good in recording efficiency.

4.2 Programming Language Support

Sub-criteria Multiple language support Language bridge 2 5 Sahi Selenium 5 1 Remarks Sahi works majorly with javascript Sahi script can directly invoke anything

For sahi the value is (2+5)/2=3.5 For selenium the value is (5+1)/2=3 Therefore sahi is more than average as compared to selenium

31

International Journal of Computational Intelligence and Information Security, October 2012 Vol. 3, No. 8 ISSN: 1837-7823

4.3 Ease of use

Sub-criteria Easy to start Knowledge of programming language Identification of elements Sahi 1 5 5
Selenium

Remarks Easy to start selenium because of IDE Sahi doesnot requires core developers. . Selenium uses x path based identification

5 1 2

For sahi the value is (1+5+5)/3=3.6 For selenium the value is (5+1+2)/3=2.6 Thus, sahi is good than selenium. .

4.4 Stability of Scriipts and maintenance

Sub-criteria Implicit wait for page load Ease of adoption 5 5 Sahi Selenium 1 2 Remarks In selenium explicit wait is needed Needs tester to know Test NG/ junit

For sahi the value is (5+5)/2=5 For selenium the value is (1+2)/2=1.5 Therefore, sahi is best in stability of scripts as well as in the adoption by tester.

4.5 Data Driven Testing

Sub-criteria Access data from external sources Way of testing 5 For sahi the value is (5+5)/2=5 and For selenium the value is (4+5)/2= 4.5 4 Sahi 5 Selenium 5 Remarks Both provide access external sources to different

Needs tester to know Test NG/ junit

5. Conclusion
Automated software testing is the best way to increase the effectiveness, efficiency and coverage of your software testing. Both of the tools compared have been proved to be beneficial. The choice, on which tool is to be selected depends on various parameters. In case of data driven testing both are best. And if you are a non programmer then sahi is a better option.

32

International Journal of Computational Intelligence and Information Security, October 2012 Vol. 3, No. 8 ISSN: 1837-7823

References
[1]. Selenium http:// seleniumhq.org/ [2]. Thom Garett Bernie Gauf Implementing automated software Testing [3]. William E Perry, Effective methods for Software Testing [4]. Michael Jagrre, Testing applications on the web [5]. David Burns, Selenium 1.0 Testing tools [6]. Sahi Automation Testing, http:// Sahi.co.in [7]. Software Testing, en.wikipedia.org/wiki/Softwaretesting

33