Automation Testing

Introduction to Automation Testing

Another software testing method is automation testing, which is used some
specific tools to execute the test scripts without any human interference. It is the
most acceptable way to enhance the efficiency, productivity, and test coverage of
Software testing. With the help of an automation testing tool, we can easily
approach the test data, handle the test implementation, and compares the actual
output against the expected outcome.

In automation testing, the test automation engineer will write the test script or use
the automation testing tools to execute the application. On the other hand, in
manual testing, the test engineer will write the test cases and implement the
software on the basis of written test cases.
In test automation, the test engineer can execute repetitive tasks and other related
tasks. In manual testing, it is a tedious process to implement the repetitive take
again and again.
In other words, we can say that the main concentration of test Automation is to
change the manual human activity with systems or devices.
The automation testing process is a time-saving process as it spends less time in
exploratory testing and more time in keeping the test scripts whereas enhancing
the complete test coverage.
Note: We perform exploratory testing whenever the requirement does not exist.

Why do we need to perform automation testing?

 In software testing, automation testing is required to test the application
because it offers us a better application with less effort and time.
 Some organizations still perform only manual testing to test the application
as those companies are not fully aware of the automation testing process.
 But now, they are aware of automated testing and executing the test
automation procedure in their application development process.
 To implement the automation testing, we required pretty a considerable
investment of resources and money.
The execution of automation testing provides us various advantages, which are as
discussed below:

1. Reusability
2. Consistency
3. Running tests anytime (24/7)
4. Early Bug detection
5. Less Human Resources
1. Reusability
We can re-use the test scripts in automation testing, and we don't need to write
the new test scripts again and again. And, we can also re-create the steps which are
detailed as the earlier ones.
2. Consistency
As compared to manual testing, automation testing is more consistent and way
faster than executing the regular monotonous tests that cannot be missed but may
cause faults when tested manually.
3. Running Tests 24/7
In automation testing, we can start the testing process from anywhere in the
world and anytime we want to. And even we can do that remotely if we don't have
many approaches or the option to purchase them.
4. Early Bug Detection
We can easily detect the critical bugs in the software development process's
initial phases by executing automation testing. It also helps us spend fewer working
hours to fix these problems and reduce costs
5. Less Human Resources
To implement the automation test script, we need a test automation engineer
who can write the test scripts to automate our tests, rather than having several
people who are repeatedly performing the tedious manual tests.

Automation Testing Methodologies

Automation testing contains the following three different methodologies and
approaches, which will help the test engineer to enhance the software product's
quality.
 GUI Testing
 Code-Driven
 Test Automation Framework
Now, let's understand the different approaches of automation testing one by one:
1. GUI (Graphical user interface) Testing
In this approach, we can implement that software or an application, which
contains GUIs. So, that the automation test engineers can record user actions and
evaluate them many times.
We know that the Test cases can be written in several programming languages
like JAVA, C#, Python, Perl, etc.
2. Code-Driven
The code-driven technique is the subsequent methodology used in automation
testing. In this method, the test engineer will mainly concentrate on test case
execution in order to identify whether the several parts of code are performing
according to the given requirement or not.
Hence, it is very a commonly used method in agile software development.
3. Test Automation Framework
Another approach in automation testing is test automation framework. The
test automation framework is a set of rules used to generate valuable results of the
automated testing activity.
Similarly, it brings together test data sources, function libraries, object details, and
other reusable modules.
Automation Testing Process
The automation testing process is a systematic approach to organize and
execute testing activities in a manner that provides maximum test coverage with
limited resources. The structure of the test involves a multi-step process that
supports the required, detailed and inter-related activities to perform the task.

The automation testing process completed in the following steps:

Step1: Decision to Automation Testing
It is the first phase of Automation Test Life-cycle Methodology (ATLM). At
this phase, the main focus of the testing team is to manage expectations from the
test and find out the potential benefits if applying the automated testing correctly.
On adopting an automated testing suit, organizations have to face many issues,
some are listed below:
 Testing tool experts are required for automation testing, so the first issue, to
appoint a testing equipment specialist.
 The second issue is, choose the exact tool for the testing of a particular
function.
 The issue of design and development standards in the implementation of an
automated testing process.
 Analysis of various automated testing tools to choose the best tool for
automation testing.
 The issue of money and time occurs as the consumption of money and time
is high in the beginning of the testing.
Step2: Test Tool Selection
Test Tool Selection represents the second phase of the Automation Test
Life-cycle Methodology (ATLM). This phase guides the tester in the evaluation and
selection of the testing tool.
Since the testing tool supports almost all testing requirements, the tester still needs
to review the system engineering environment and other organizational needs and
then make a list of evaluation parameters of the tools. Test engineers evaluate the
equipment based on the provided sample criteria.
Step3: Scope Introduction
This phase represents the third phase of Automation Test Life-cycle
Methodology (ATLM). The scope of automation includes the testing area of the
application. The determination of scope is based on the following points:
 Common functionalities of the software application that are held by every
software application.
 Automation test sets the reusable range of business components.
 Automation Testing decides the extent of reusability of the business
components.
 An application should have business-specific features and must be
technically feasible.
 Automation testing provides the repetition of test cases in the case of cross-
browser testing.
This phase ensures the overall testing strategy that should be well managed and
modified if required. In order to ensure the availability of skills, testing skills of a
particular member and whole team are analyzed against the required specific skills
for a particular software application.
Step4: Test Planning and Development
Test planning and development is the fourth and most important phase of
Automation Test Life -cycle Methodology (ATLM) because all the testing strategies
are defined here. Planning of long -lead test activities, the creation of standards
and guidelines, an arrangement of the required combination of hardware, software
and network to create a test environment, defect tracking procedure, guidelines to
control test configuration and environment all are identified in this phase. Tester
determines estimated effort and cost for the entire project. Test strategy and effort
estimation documents are the deliverables provided by this phase. Test case
execution can be started after the successful completion of test planning.
Step5: Test Case Execution
Test case Execution is the sixth phase of Automation Test Life -cycle
Methodology (ATLM). It takes place after the successful completion of test
planning. At this stage, the testing team defines test design and development. Now,
test cases can be executed under product testing. In this phase, the testing team
starts case development and execution activity by using automated tools. The
prepared test cases are reviewed by peer members of the testing team or quality
assurance leaders.
During the execution of test procedures, the testing team directed to comply with
the execution schedule. Execution phase implements the strategies such as
integration, acceptance and unit testing that have defined in the test plan
previously.
Step6: Review and Assessment
Review and assessment is the sixth and final stage of the automated testing
life cycle but the activities of this phase are conducted throughout the whole life
cycle to maintain continuous quality improvement. The improvement process is
done via the evaluation of matrices, review and assessment of the activities.
During the review, the examiner concentrates whether the particular metric
satisfies the acceptance criteria or not, if yes, then it is ready to use in software
production. It is comprehensive as test cases cover each feature of the application.
The test team performs its own survey to inquire about the potential value of the
process; if the potential benefit is less than sufficient, the testing team can change
the testing tool. The team also provides a sample survey form to ask for feedback
from the end user about the attributes and management of the software product.
What are different challenges faced during the Automation testing
process, and how to overcome them?
The automation testing process carries about large numbers of advantages
for an organization. Usually, with automated testing, the testing team of the
specified method of software validation can accomplish the amplified test
coverage.
Though we may face various challenges during the automation testing process;
therefore, we need a thorough follow-up process to reach successful test
automation execution.
Some of the common challenges are as discuss below, which we might encounter
throughout the test automation process of an application:

1. Lack of Highly Skilled Test Automation Experts

 The most fundamental reason behind lacking an automation testing process
is the absence of highly skilled automation test engineers.
 To design and maintain the test automation framework and test scripts, the
test automation engineer needs to be skilled and have details knowledge
about automation technical skills.
 To create a test script correctly and sustain them, the team should
implement the test scripts successfully to check the application performance
and fix the technical issues.
  To achieve the operative automation aspects, the automation test engineers
should have strong knowledge about test automation frameworks because
it would be easy for them to design and execute the test scripts.
2. Appropriate Selection of Right Test Automation Tool
 The next challenge, we mostly face while performing the automation testing
is selecting thecorrect automation testing tool.
 A suitable detailed assessment of the application under test (AUT) is
mandatory.
 With the accessibility of various automation testing tools for both open-
source and paid tools. And it is essential to check appropriately before
accepting a tool.
3. Challenges with Scaling Test environments
 It is a crucial challenge in present day as testing teams that don't providing
test environments in the cloud.
 It is essential to quickly deliver the changed test environments that
automated testing requires, measure them up, execute the tests, to make
sure that the success while testing in the cloud.
 Generally, the internal teams have a partial number of test environments
which they can use to execute the fewer tests and deployed them at any
given time.
 As a result, testing takes a much longer time. To overcome this, it is essential
to move test environments to the cloud to scale the test automation, and
it will also reduce testing teams operating costs.
4. Including High Initial Investment Costs
 Generally, the execution of the automation testing process in the initial
phase is a bit expensive because it is necessary to evaluate, design, and build
an automation testing framework, libraries, or reusable functions over a
detailed AUT analysis.
 Additionally, if an automation test engineer picks the licensed test
automation tool, the operating cost should be appraised.
 If the automation test engineer selects the open-source tools, then it should
also be significant efforts spent on learning, training, and maintaining them.
 Later, the selection of automation tools, either open-source or licensed,
depends on the AUT priority and availability of cost facility.
5. Miscommunication and lack of collaboration between Development and
testing teams
 Miscommunication or lack of proper collaboration between the developer
and the test engineers' team and operations teams does not lead us to
successful test automation.
 Therefore, it is significant to place all the team (developer, testing,
operation) members to categorize the test automation objectives and set
targets.
 To ensure the test automation success, these teams need to spend some
time to get an effective communication and to get a clear understanding of
the business needs and project specifications.

Automation Testing Tools

Automation testing tools can describe in two categories, which are as follows:
1. Functional Testing Tools
2. Non-Functional Testing Tools
Let's understand them one by one in-details:
1. Functional Automation Testing Tools
The automation test engineer uses the functional automation testing tool to
implement the functional test cases. For example, the Repetitive Regression tests
are automated under the function automation testing tools.
This type of tools can be further divided into two different parts, which are as
below:
 Commercial Tool
 Open-source Tool
Commercial Functional Testing Tools
Commercial functional testing tools are those testing tools, which are not
available freely in the market. These tools are also known as licensed tools. The
licensed tools include various features and approaches as compared to the open-
source tools.
Some of the most important commercial tools are as follows:
 QTP[Quick Test Professional]
 Rational Functional Tester [RFT]
 TestComplete
 SoapUI

Let's take a look on the above listed commercial tools:

QTP
 QTP signify as Quick Test Professional, but now it is known as Micro
Focus UFT (Unified Functional Testing).
 Mainly, it is used to automate the functional regression test cases of the web-
based application or the software.
 In order to test the application, deploy the objects, and for analysis purposes,
it is designed on the scripting languages such as VBScript.
 It follows the concept of keyword-driven testing for defining the test
creation and maintenance.
 The new test engineers can easily use the QTP tool because it allows them to
develop the test cases right from the application.
Characteristics of QTP
Some of the most standard attributes of QTP are as follows:
 The record and playback feature enhance the credibility of the QTP tool.
 Both technical and non-technical test engineers can operate the QTP tool.
 It supports several software development environments such as SAP, JAVA,
Oracle, and so on.
  We can check both web-based and desktop-based applications by using the
QTP.
RFT
 RFT stands for Rational Functional Tester, which is used to execute the
functional regression test cases.
 It follows the concept of Data-driven testing and GUI testing.
 It builds the automated functional tests by recording an end-user's activities
on the system under test and repeating the actions on request to perform a
test.
Features of RFT
The RFT tool contains the following characteristics:
 RFT supports an extensive range of applications and protocols, like HTML,
Java, .NET, Windows, Eclipse, SAP, Visual Basic, PowerBuilder
 It sustains custom controls by proxy SDK like Java or.Net.
 It allows developers to develop the keyword-associated script; hence, it can
be re-used again, which enhances the productivity of the project.
TestComplete
 Another functional automated testing tool is TestComplete, which acquires
by SmartBear Software.
 TestComplete has some inbuild capability, which helps the test engineers to
develop the automated tests for Microsoft Windows, Web, Android, and iOS
applications.
 In this tool, the tests can be recorded, scripted, or manually created with
keyword-driven processes and used for automatic playback and error
logging.
 It includes the three different modules, like Web, Desktop, and Mobile,
where all the modules involve the functionality to develop the automated
tests on the particular platform.
Characteristics of TestComplete Tool
Below is the essential characteristic of the TestComplete Tool:
  It is used to test several types of specific applications such as Web, Windows,
Android, iOS, .NET, VCL, and Java.
 It involves the issue-tracking templates, which can be used to develop or
change items stored in issue-tracking systems.
 It records the key actions essentially to repeat the test and rejects all
unnecessary efforts.
SoapUI
 SoapUI is the most extensively used automation tool. It is mainly used to test
the web servicesand web APIs of SOAP and REST interfaces.
 It allows the test engineers to test functional, regression testing, and other
testing types on various Web services and APIs.
 SoapUI tool develops the mocks where test engineers can test real
applications.
 It is entirely written on JAVA and Groovy programming languages.
Features of SoapUI
Some of the most common characteristics of the SoapUI tool are as follows:
 The core features of SoapUI contains various web services, such
as development, mocking, Inspection, simulation, and invoking.
 It supports all standard protocols and technologies like HTTP, HTTPS, AMF,
JDBC, SOAP, WSDL, etc.
 SoapUI tool offers a fast and efficient framework which creates many web
service tests.
 It provides the services to obtain data from several sources of web service
without developing any code.

Open-source Functional Testing Tools

Open-source functional testing tools are those tools, which are available
freely in the market. These tools have less functionality and features than
the commercial/licensed tools, but sometimes working on the commercial tool
became an expensive process.
That's why some well-known organizations preferred to use open-source tools.
Following are the most commonly used open-source functional automation testing
tools:
 Selenium
 Sikuli
 Autoit
Let's understand the open-source functional automation testing tools one by one:

Selenium
Whenever we are talking about the open-source tools of automation testing,
one name came into every automation test engineer's mind:
 Selenium is the highly recommended and widely used functional testing
tool, which is well-suited for non-functional testing tools.
 Selenium is an open-source tool which means, it does not require any
licensing.
 We can only test the web application using the selenium tool, and the Stand-
alone application cannot automate in Selenium.
 It is most commonly used to implement Functional test scripts.
 It can be combined with several automation testing tools to achieve
continuous testing, for example, Maven and Jenkins.
 It can be associated with many devices such as TestNG and JUnit to manage
the test cases and generate the test reports.
Characteristics of Selenium
Following are the basic features of selenium:
 Selenium can be easily deployed on several platforms like Windows, Linux,
Solaris, and MacOS.
  Furthermore, it also supports operating systems for mobile applications, for
example, iOS, Windows mobile, and android.
 Selenium supports several programming languages such as C#, Java, Perl,
PHP, Python, and Ruby.
 Selenium become extremely resourceful while we were implementing the
test script and analyzing it across many browsers concurrently.
 It provides a user-friendly interface that helps the test engineer develop and
perform the test scripts resourcefully and efficiently.
Sikuli
 Another open-source functional automation testing tool is Sikuli.
 It is a GUI based test automation tool, which can easily automate the Flash
objects as most of the automation testing tool like selenium does not
support the flash-object automation.
 Most commonly Sikuli is used to interact with fundamentals of web pages
and control the windows-based popups.
 With the help of Sikuli tool, we can easily test the windows application.
Features of Sikuli
The most standard features of the Sikuli tool are as discussed below:
 The Sikuli tool can be easily combined with Selenium WebDriver and all other
automation testing tools by using the Sikuli Jar files.
 We can also automate desktop applications or stand-alone applications with
the help of the Sikuli tool.
 It provides a simple API, which implies that all methods can be retrieved
using screen class objects.
Note: As we know that Selenium WebDriver can only test the web application, but
on the other hand, the Sikuli tool can test the web-based as well as window-based
applications.
AutoIt
 AutoIt is another open-source tool used for functional automation testing.
 It is a freeware scripting language designed to test the Windows GUI and
general scripting.
  The AutoIt script is written in a primary language.
 It can replicate any grouping of keystrokes, mouse movement, and window
or control operation.
Features of AutoIT
AutoIT tools include the following features:
 It uses a straightforward syntax for its scripting language that can be easily
understood and executed to automate any process.
 It has an inbuilt RunAs function that helps in executing any external program
using some outside user.
 It can even record and create a script on its own for detailed processes, which
need to be automated.
 All types of standard Windows controls and other GUIs without any object
identification issue can easily relate to the AutoIt tool.
2. Non-functional Automation Testing Tools
The automation test engineer uses the non-functional automation testing
tool to execute the non-functional performance test cases.
For example, testing the application's response time under significant load, let say,
100 users.
Just like functional automation testing tools, the non-functional automation testing
tools divided into two different categories, which are as below:
1. Commercial Tools
2. Open-source Tools
Commercial Non-Functional Automation Testing Tools
These are those tools, which cannot be used freely, as they required a proper
license. The commercial tools have additional features and functionality as
compared to the other open-source testing tools.
The usage of these types of tools helps us to enhance the efficiency of the software
product.
Let's see some of the most commonly used commercial non-functional automation
testing tools.
  LoadRunner
 Silk Performer
LoadRunner [HP Performance Tester]
LoadRunner is one of the most popular non-functional tools. It is mainly
used to support performance testing for a wide range of protocols, several
technologies, and application environments. LoadRunner is the licensed tool.
It rapidly classifies the maximum number of performance issues. And precisely
predict the application scalability and size.
Feature of LoadRunner
Some of the most significant features of the LoadRunner tool are as follows:
 LoadRunner tool will help us to reduce the cost of distributed load testing
and hardware and software costs.
 We can easily view and handle the XML data within the test scripts as it
supports the XML scripting language.
 We can get detailed performance test reports by using the LoadRunner tool.
 It offers the operational tool for deployment tracking.
Silk Performer
Another non-functional automation testing tool is Silk Performer. It can test
the various application environments with thousands of simultaneous users.
It makes sure that applications and server up times are sustained when faced with
the highest customer usage.
It is one of the most commonly used enterprise-class load and stress testing tools,
supporting an extensive range of protocols.
Characteristic of Silk Performer
Following are the standard features of the Silk performer tool:
 It is used to pretends changeable virtual users.
 In this tool, the correlation and parameterization are user-friendly.
 It supports integrated server monitoring.
 It provides Version Control Management.
  There is no license condition required for controllers or Individual Protocols.
 It can quickly create reports with tables and graphs and allows
customization.
Open-Source Non-functional Automation Testing Tools
The open-source non-functional automation testing tools can be used easily
as they are freely available in the market. These tools have less functionality as
compared to the commercial testing tools.
But because they are open-source and used quickly, some organizations prefer to
use these tools as they do not require any cost.
Some of the most commonly used open-source non-functional automation testing
tools are as follows:
 JMeter
 NeoLoad

JMeter
JMeter is one of the most significantly used open-source non-functional
automation testing tools. JMeter is entirely designed on the JAVA application to
load the efficient test performance and measure an application's productivity.
It is mainly simplifying users or developers to use the source code for the
enhancement of other applications. It is used to test the implementation of both
static and dynamic resources and dynamic web applications.
It is used to reproduce the massive load on a server, object, or network, group of
servers to test its power or explore the complete performance in multiple load
types.
It is highly extensible to load the performance test in several types of servers like:
 Database Server: LDAP, JDBC
 Web Server: SOAP, HTTPS, HTTP
 Mail Server: POP3
Characteristics of JMeter
Some of the significant attributes of the JMeter tool are as follows:
 It sustains several testing approaches, for example, functional, distributed,
and load testing.
 It supports a user-friendly GUI that is interactive and easy.
 It is a platform-independent tool as it is developed with the help of JAVA;
therefore, it can execute on any platform that accepts a JVM like Windows,
Mac, and Linux, etc.
NeoLoad
Another most commonly used open-source tool in automation testing is
NeoLoad and Neotys develop it.
It is used to test the performance test scenarios and also helps us to identify the
bottleneck areas in the web and the mobile application development process.
It is faster as compared to other traditional tools. NeoLoad will support a wide
range of web, mobile, and packaged applications, such as SAP, Oracle, Salesforce,
etc., that cover all our testing needs.
Features of NeoLoad
Below are some of the vital features of NeoLoad:
 It is also used to share and manage the test resources.
 It will support various frameworks and protocols such as HTTP/2, HTML5,
API, AngularJS, Web Socket, SOAP, etc.
 It provides a robust code-less design.
  NeoLoad can modify the functional test script into the performance test
scripts.
 NeoLoad can automatically update the test scripts and developed the real-
time test results.

Advantages of Automation Testing

Automation testing takes less time than manual testing.
A tester can test the response of the software if the execution of the same
operation is repeated several times.
Automation Testing provides re-usability of test cases on testing of different
versions of the same software.
Automation testing is reliable as it eliminates hidden errors by executing test
cases again in the same way.
Automation Testing is comprehensive as test cases cover each and every
feature of the application.
It does not require many human resources, instead of writing test cases and
testing them manually, they need an automation testing engineer to run
them.
The cost of automation testing is less than manual testing because it requires
a few human resources.

Disadvantages of Automation Testing

Automation Testing requires high-level skilled testers.
It requires high-quality testing tools.
When it encounters an unsuccessful test case, the analysis of the whole
event is complicated.
Test maintenance is expensive because high fee license testing equipment is
necessary.
Debugging is mandatory if a less effective error has not been solved, it can
lead to fatal results.