Step by Step Guide to Implement Proof of Concept
(POC) in Automation Testing
Automation Testing
How to Implement Proof of Concept (POC) in Automation Testing?
Every organization has different testing processes and procedures. Manual Testing is important and
irreplaceable- however, automation is picking speed.
Introducing automation testing to an organization is a challenge and following points will determine
if it is required at all:
Biss yeois ceu)ib sate)
IMPLEMENT PROOF OF CONCEPT (POC)
IN AUTOMATION TESTING
#1. Duration of the project: Short-term or long-term ~ long-term projects are good candidates
for automation
#2. How much regression is done in each testing cycle? — projects that have repetitive and
lengthy regression tests as automation reduces the overall testing time and ensures complete
coverage.
#3. Stability of the application: Application that is not susceptible to frequent changes should be
considered for automation. The produet which is not stable, where GUI/Functionality keeps
changing, elements or its XPath on page keep changing should not be automated until stable,
#4. Is the project data secure and does testing it requires some complicated procedures?
~ In this case, it is best to go for manual testing,
#5. Does the organization have a budget for automation? — Automation will add to additional
expenditure for the organization like automation tool cost, resource cost, a time required for
framework development and writing/maintaining automation test scripts.
With automation, missing tests or taking some test results for granted will never happen. It ensures
100% coverage of given module each and every time the same is tested. Automation will also help
perform the same test multiple times on multiple browsers and platforms.
The following figure will help understand the process of automation testing
accel co
Develop test
Sra ea scar
Eee En eT on)
aos
Liretets
ee teeenict
Gus
Figure 1: Automation Testing Process
From the technical testing paint of view, the QA team needs to understand the following
aspects about their automation tool:
Platform and OS te:
Data-driven capability
Reporting capability and report portability
Easy debugging and logging
Version control supported
Extensible & Customizable (able to integrate with other tools like Ant, TestNG)
Continuous Integration
8, Email Notifications (Custom email message received if tests are passed successfully/ failed/or
any network failure)
9. If cross-browser testing and multiple platforms testing are required then distributed testing
environment is supported or not.
g matrix
Mounbene
What You Will Learn: [show]
Selecting correct automation tool:
Tool 2
Tool 3
Figure 2: Selecting correct tool
#1. An application under test is a web application or a desktop application.
#2. Choosing an open source tool Vs paid one.
#3. Tool should fulfill the application's testing requirement
#4. Using the tool - the team's expertise and comfort level in terms of using and learning the tools
#5. Does it support reporting - If No what other options of reporting are available (open source or
paid). If yes then how good itis in terms of conveying correct data from presentations as well as
content point of
w.
Also read => The A to Z Guide on Selecting the Best Automation Tool
In addition, tool evaluation includes:
While selecting an automation tool it is very important to consider if
applications GUI implementation
supported on the
1. GUI is implemented using traditional HTML or AJAX or other web development toolkit
2. Does GUI include videos, images or lot of written content?
3. It is interactive or only informative
4. Browsers required to be tested
It is important to assess the tool on the above points to understand if the tool really meets the
project's testing requirements.
Developing proof of concept on automation:
Implementing an automation testing POC is a crucial and most often used method of introducing
a tool to an organization. Once it is decided that automation is to be done and a tool has been
chosen, it is time to create a prototype as a POC and present it to the management to showcase the
real-time usage and benefits.
To do so:
1) Decide the test cases that we will use in the POC.
2) It helps to pick the areas the clients will be most
3) Plan to show manual vs automation in a way that proves that there is no degradation in the
quality by choosing automation
4) Include a test case that fails and results in finding a defect- this helps reinforce that the tool
indeed can find defects
5) Use assertions and validation points wherever necessary.
6) Show clearly areas that can and cannot be automated. Usually, the following aspects cannot be
automated:
iterested in
+ Video steams
+ Flash content (non-static contents)
+ Non-static images
7) Highlight if the tool satisfies the following requirements?
+ can
+ Is automation possible on the same browser that is required by the project
+ Will automation call for change in application implementation? (like for automation it is
important that element identifiers are unique and does not change every time the page is
invoked)
automate all the key features of desired application
The result of POC- it is usually one of the following:
1. Tools meet the project requirements- Work out further details. Such as cost of
implementation - negotiate prices is necessary, finalize license fees, training & support costs,
consultation, and implementation expenditures etc. In case of open source, tools determine
the maturity of the tool, learning resources available, learning curve, support available etc
For both licensed and open source tools, maintenance costs have to be considered too. It has
to be kept in mind that the benefits are substantial only over a long period of time.
ns ~ the tool is no longer
2. The tool does not meet the requirements and has limitat
considered.
3. Tool partially meets the requirements ~ revisit and check if another satisfies the
automation is totally out of the picture OR if there is any other
workaround with the same tool.
Once we present our proof of concept to the management and we get a go-ahead from them, the
next step is implementing a pilot project using that tool
POC Template:
There is no one perfect POC template. It generally includes:
Requirement for POC
Candidates of POC (All automation tools)
Project requirements
Pros and cons of every tool based on the project requirements
POC result
ahwne
Here are a couple of Automation POC templates for reference:
=> POC template 1
=> POC template 2
Implementing a Pilot project:
We should define our pilot project by:
tool or not.
* Quantifying business cases which
* Define naming convention and various guidelines for apy
+ Benefits of a tool like financial and others, what can be done and what cannot be done and
also its possible workarounds.
determine whether we should be using t
tion tool.
Step #1. Choosing test cases for pilot
+ Modules / Features important from client perspective
+ Functionality easy to demonstrate (happy path end to end)
+ Test cases difficult to test manually and once automated will simplify testing them
Broken functionality to demonstrate how automation can help identify failed test case
Step #2. Automation framework development
A test automation framework is the set of concepts, process, procedures, practices, and
environment. It is nothing but an integrated system that consists of rules to automate any given
product. This system includes set of functional libraries, APIs, test data, object repository and
various other modules. The framework and approach of scripting used for test automation have
effect on its costs.
Following scripting techniques can be used:
+ Linear
+ Hybrid
* Data-driven
+ Keyword driven and
* Structured
Using any of the above technique, a testing framework can be designed that will help in achieve a
specific format to drive the test, simplify test execution and reporting
Determine templates, naming conventions for objects, test cases, test suites, data repository etc.
‘Step #3. Script development and execution
‘Step #4. Reporting: Does the tool have in-built reporting capabilities? Are the inbuilt reports
capable of conveying all the required information precisely? Are we going to need another tool for
reporting purpose like crystal reports, reportNG, etc.?
ing automation scripts
Presenting to the stakeholders:
‘As much as proof of concept and implementing a pilot is important so is presenting it in the correct
manner, Following points will help to present it in a positive way.
1. Begin with how much manual testing effort is put in every testing cycle, challenges faced
during manual testing and how can we use automation to overcome them.
2. Explain how you selected the tool based on the proof of concept
3. Highlight features of automation tool and how it complements the testing requirements
4. While running through the automation, explain how automation tool will net only help in faster
tion.
test execution but also its capability of performing verification and bug identi
5. Demonstrate how report will show test case execution status
6, Highlight reporting features like colorful legends for different test case status, snapshots of
failed test cases, and report portability
7. And finally show how much testing time will be reduced for every testing cycle.
8, Also explain how you are able to achieve the entire automation framework that you have
developed and its benefits in terms of usage and maintenance.
Be prepared to answer questions related to how much time will it take to automate a single simple
or critical functionality. Also, if a minor change happens on application front how much script
changes will be required as how much time will be taken to modify.
We hope this guide is useful for you to start writing an automation testing POC document.
Let us know if you have any questions.