Software Testing and Automation

UNIT-5

Cross Platform Testing

What is Cross Platform Testing?

Cross platform testing is used to check the applicability and working

of an application across different platforms such as different
operating systems, browsers or even devices. Therefore, it
encapsulates both cross browser (applicability of working on
different browsers such as Opera, Mozilla Firefox, etc) and cross OS
(applicability to work across different operating systems such as
android, iOS or Windows) testing for software products.
Cross-platform testing is performed to determine the behavior of your
application and website in different environments. Cross-platform
testing helps in identifying issues that may vary with platforms or
configurations such as consistency, user interface, usability, and
performance issues.
For example, the user interface of your website may appear correctly in
the Chrome browser, but the layout might be disturbed in the Internet
Explorer browser. Similarly, an Android application might run correctly
on Nougat, but does not support Kitkat. These types of issues can be
identified only when cross-platform testing is done during the testing
phase.

Approach to Cross platform Testing

 Identifying the Target Audience:

 With the help of statistical data, the sections of population
which are going to form the core of your application’s user
base are identified.

 Select Platforms to Support

As soon as you are familiar with the user-base, the next step is to decide
which platform to support or how the solution will be implemented,
taking in view your user-base. If your user-base indicates that the end
users will be from western countries, then the dedicated platform can be
the iOS platform.

 Limit your test: From the statistical data highlighting the

use of a specific platform used by a majority of the users,
available at our disposal, a well informed and thought out
decision is taken to select only those platforms which cover the
maximum of user base. Based on this decision, priorities are
set for testing the app on those devices or platforms first,
which are favoured for use by majority of the targeted user
base.

 Use of Automation Tools:Testing the product for

innumerable platforms can be a cumbersome task and lead to
fatigue in the tester at a time when he is most likely to detect a
flaw in it. In order to prevent the likelihood of missing out on
bugs and expedite the task of testing across countless
platforms, the use of automation tools makes sense. However,
human perception is not to be neglected for testing issues
related to usability and appearance.

 Invest in Emulator Software

Purchasing multiple devices and gadgets can be very costly. The
best option to perform cross-platform testing on different
 environments is to use emulators and virtualization. You can
create your own virtual machines with various platforms and
configuration combinations to test applications. Also, you can use
emulators to see the behavior of your application on mobile
devices with different screen sizes.

Steps for Cross Platform Testing:

1. Creation of a Test Matrix:Before starting it is sensible to

create a test matrix listing out all the possible browsers,
operating systems and varied devices against which the
application has to be tested.

2. Risk Prioritisation:The platforms which form the elements

of the matrix are prioritised as low risk, medium risk and high
risked. This segregation is very handy while working in an
environment where working with deadlines is a regular
feature.

3. Selecting the Most apt tool for Testing:The market is

searched in the lookout for tools which have shown to create
maximum impact with their efficacy while being used for
testing similar software products. Comparative analysis is
done between shortlisted tools. In the end a proof of concept is
done to come down on a tool which fits all the bills.
Information can also be gathered via interacting with testers
on forums of social media. Examples of well known tools are
Browserstack and Testplant.

4. Test Execution:The product is tested by running test cases

either by simulation, automation, and virtual machines by the
parent or the outsourced company as the case maybe.
 Code Coverage metrics
Code coverage is software testing metric that determines the number of lines of code
that is successfully validated under a test procedure, which in turn, helps in analyzing
how comprehensively software is verified.
Code coverage measures how much code of the application is being exercised when
the tests are run. This makes sure that the tests being run are actually testing the code
of application.
Measurement of Coverage can be determined by the following formula.
Coverage= Number of coverage items exercised / Total number of coverage items
*100%.
It should be kept in mind that 100% code coverage does not mean that the application
is 100% tested. An application with high code coverage means it has been more
thoroughly tested and would contain less software bugs than an application with low
code coverage. There are numerous ways to calculate code coverage like program
subroutines and program statements called during the execution of test suite.

What are test code coverage criteria?

Coverage criteria are method to measure how much code has been exercised by the
test suite. The test suite must satisfy the rule of the coverage criteria. The various
Coverage criteria are as follows:

Statement Coverage

Decision Coverage

Condition Coverage

Path Coverage

Let us understand each of these in details

1. Statement Coverage: The statement coverage tells us whether each executable

statement is covered. This technique is better than other techniques as the defects are
evenly distributed over the entire code. Hence the percentage of executable covered
can be found out using the percentage of defects found. For Control statements like
if else, for and switch are covered only if the expression of the control statement is
covered in the executable statements .The statement coverage does not work with
some control structures. For If-Statement Structures, if ever the if condition is false,
that statement coverage method would fail. If there is no test case that causes the
condition to be false, the statement coverage would state the code is fully covered.
 The loops that contain break statement are not covered by statement coverage. It does
not tell whether the code reached its termination stage. For do-while loops the code is
taken as non branching statement code by the statement coverage. There are many
other shortcomings of the statement coverage. It does not work for logical operators
like And and OR. It is also insensitive of the switch statements.

2. Decision Coverage: Decision coverage tells us whether the expressions in control

structures came out to be true or false. The whole expression is taken as one true and
one false regardless of whether it contains Logical AND and Logical OR operators. It
also includes coverage of switch statement cases, exception handling and all entry and
exit points. The contact expressions which control the flow are ignored by decision
coverage. In decision coverage metrics all the Boolean expressions should evaluate to
true and false even if it is not included in the control flow. But the exact definition of
Boolean Expressions is not defined here. Some programming language like C
language can take both integer and Boolean together without requiring it to be
declared Boolean. Generally, a Boolean operator is one with Boolean operand and
Boolean output is taken as Boolean expression. The decision coverage is also known
as branch coverage, all-edge coverage. The decision coverage has many
disadvantages like it ignores some Boolean expressions branches which occur because
of short circuit operators.
3. Condition Coverage: It is quite similar to decision coverage but it is better than
the decision coverage. It is more sensitive to control flow. In condition coverage the
conditions are taken independent of each other. It tells us about the true or false
result of each condition. The condition is taken as the operand of the logical operator
that does not contain logical operator.
4. Multiple Condition Coverage: It tells us whether every possible combination of
condition occurs. The test cases for multiple condition coverage can be found out by
logical operator truth table For languages like C,Java multiple condition c overage
requires very thorough testing. This technique has some disadvantages as well. For
any complex Boolean expressions it is very cumbersome to find out the number of
test cases required. Another disadvantage of this coverage is that the number of test
cases varies among conditions that have same complexity.

5. Path Coverage: Path Coverage tells us whether all the paths in each function have
been followed. The path coverage determines the paths are possible combinations of
logical operators. A path is a sequence of branches from entry point to exit point. It is
useful as it requires thorough testing. Path Coverage has disadvantages as the number
of paths is exponential to number of branches. Also due to relationship of data many
paths are not possible.
 What Is Code Walkthrough
Code Walkthrough is a form of peer review in which a programmer leads the review
process. The other team members ask questions. At the same time, spot possible
errors against development standards and other issues.

Code Walkthrough is a form of peer review in which a programmer leads the review
process and the other team members ask questions and spot possible errors against
development standards and other issues.

The Walkthrough Meeting

 The meeting is usually led by the author of the document under review. It is attended
by other members of the team.
 Review sessions may be formal or informal.
 Before the walkthrough meeting, the preparation by reviewers and then a review
report with a list of findings.
 The scribe, who is not the author, marks the minutes of meeting. Then notes down all
the defects/issues so that it can be tracked to closure.
 The main purpose of walkthrough is to enable learning about the content of the
document under review. This is to help team members gain an understanding of the content
of the document. Likewise, to find defects.

Advantages of Code Walkthrough

 This type of meeting is normally managed by the document creator under analysis
and along with the team-mates.
 The sessions which comes under the review may be formal or informal.
 The primary aim of the walkthrough is to empower knowledge around the content of
the documents under review to support team-mates and increase the awareness of the
content of documents and also discover the bugs.

Disadvantages of Walkthrough
 Not an enforceable process, no record of the review.
 Easy for the author to unintentionally miss a change.
 Reviewers rarely verify that defects were fixed.

Goals Of Code Walkthrough

 Gather information regarding the topic in the document by involving stakeholders,
both within and outside the software discipline. In other words, to present the documents
both within and outside the software discipline. Hence to gather the information regarding
the topic under documentation.
 Describe and justify the contents of the document. Thus explain or do the knowledge
transfer. Also evaluate the contents of the document.
 Reach a common consensus on the document. In other words, to achieve a common
understanding and to gather feedback.
 Check and discuss the different solutions to a problem and different suggested
alternatives. In short, to examine and discuss the validity of the proposed solutions

Characteristics of Walkthroughs
 It is not a formal process/review
 It is led by the authors
 Author guide the participants through the document according to his or her thought
process to achieve a common understanding and to gather feedback.
 Useful for the people if they are not from the software discipline, who are not used to
or cannot easily understand software development process.
 Is especially useful for higher level documents like requirement specification, and a
lot more.

Inspections
Inspection is a more formally defined process than a walkthrough. This term is usually
associated with code. Several organizations consider formal code inspections as a tool to
improve code quality at a lower cost than incurred when dynamic testing is used.
Organizations have reported significant increases in productivity and software quality due to
the use of code inspections.

Code inspection is carried out by a team. The team works according to an inspection plan
that consists of the following elements: (a) statement of purpose; (b) work product to be
inspected, this includes code and associated documents needed for inspection; (c) team
formation, roles, and tasks to be performed; (d) rate at which the inspection task is to be
completed; and (e) data collection forms where the team will record its findings such as
defects discovered, coding standard violations, and time spent in each task.

Members of the inspection team are assigned roles of moderator, reader, recorder, and
author. The moderator is in charge of the process and leads the review. Actual code is read
by the reader, perhaps with the help of a code browser and with large monitors for all in the
team to view the code. The recorder records any errors discovered or issues to be looked
into. The author is the actual developer whose primary task is to help others understand
code. It is important that the inspection process be friendly and non-confrontational.

Test automation frameworks are a set of rules and corresponding tools that are used
for building test cases. It is designed to help engineering functions work more
efficiently.
The general rules for automation frameworks include coding standards that you can
avoid manually entering, test data handling techniques and benefits, accessible
storage for the derived test data results, object repositories, and additional
information that might be utilized to run the tests in a suitable manner.
What is Verification?
Definition : The process of evaluating software to determine whether the products of
a given development phase satisfy the conditions imposed at the start of that phase.

Verification is a static practice of verifying documents, design, code and program. It

includes all the activities associated with producing high quality software: inspection,
design analysis and specification analysis. It is a relatively objective process.

Verification will help to determine whether the software is of high quality, but it will
not ensure that the system is useful. Verification is concerned with whether the
system is well-engineered and error-free.

What is Validation?
Definition: The process of evaluating software during or at the end of the
development process to determine whether it satisfies specified requirements.

Validation is the process of evaluating the final product to check whether the
software meets the customer expectations and requirements. It is a dynamic
mechanism of validating and testing the actual product.

Validation is the process of checking whether the specification captures the

customer's needs. “Did I build what I said I would?”

Verification is the process of checking that the software meets the specification.
“Did I build what I need?”

Verification Validation
1. Verification is a static practice of 1. Validation is a dynamic mechanism
verifying documents, design, code and of validating and testing the actual
program. product.
2. It always involves executing the
2. It does not involve executing the code.
code.
3. It is human based checking of documents 3. It is computer based execution of
and files. program.
4. Validation uses methods like black
4. Verification uses methods like
box (functional) testing, gray box
inspections, reviews, walkthroughs, and
testing, and white box (structural)
Desk-checking etc.
testing etc.
5. Validation is to check whether
5. Verification is to check whether the
software meets the customer
software conforms to specifications.
expectations and requirements.
6. It can catch errors that validation cannot 6. It can catch errors that verification
catch. It is low level exercise. cannot catch. It is High Level Exercise.
7. Target is requirements specification, 7. Target is actual product-a unit, a
application and software architecture, high module, a bent of integrated modules,
 Verification Validation
level, complete design, and database and effective final product.
design etc.
8. Verification is done by QA team to
8. Validation is carried out with the
ensure that the software is as per the
involvement of testing team.
specifications in the SRS document.
9. It generally comes first-done before
9. It generally follows after verification.
validation.

What is Quality Assurance?

Quality Assurance is popularly known as QA Testing, is defined as an activity to ensure that
an organization is providing the best possible product or service to customers.

Quality Control in Software Testing

Quality Control in Software Testing is a systematic set of processes used to ensure the
quality of software products or services. The main purpose of the quality control process is
ensuring that the software product meets the actual requirements by testing and reviewing its
functional and non-functional requirements. Quality control is popularly abbreviated as QC.

Difference between Quality Assurance (QA) and

Quality Control (QC)
Quality Assurance (QA) Quality Control (QC)

 It is a procedure that
focuses on providing  It is a procedure that focuses on fulfilling
assurance that quality the quality requested.
requested will be achieved

 QA aims to prevent the

 QC aims to identify and fix defects
defect

 It is a method to manage
 It is a method to verify the quality-Validation
the quality- Verification

 It does not involve

 It always involves executing a program
executing the program

 It’s a Preventive technique  It’s a Corrective technique

 It’s a Proactive measure  It’s a Reactive measure

 It is the procedure to create
 It is the procedure to verify that deliverables
the deliverables

 QA involves in full software

 QC involves in full software testing life cycle
development life cycle

 In order to meet the  QC confirms that the

customer requirements, QA standards are
defines standards and followed while working
methodologies on the product

 It is performed before
 It is performed only after QA activity is done
Quality Control

 It is a Low-Level Activity, it
 It is a High-Level Activity, it can identify an
can identify an error and
error that QA cannot
mistakes which QC cannot

 Its main motive is to prevent  Its main motive is to identify defects or

defects in the system. It is a bugs in the system. It is a more time-
less time-consuming activity consuming activity

 QA ensures that everything

 QC ensures that whatever we have done is
is executed in the right way,
as per the requirement, and that is why it
and that is why it falls under
falls under validation activity
verification activity

 It requires the involvement

 It requires the involvement of the Testing team
of the whole team

 The statistical technique

applied on QA is known as  The statistical technique applied to QC is
SPC or Statistical Process known as SQC or Statistical Quality Control
Control (SPC)
 A “Test Automation Framework” is scaffolding that is laid to provide an execution environment for the
automation test scripts. The framework provides the user with various benefits that help them to develop,
execute and report the automation test scripts efficiently. It is more like a system that has created
specifically to automate our tests.

A testing framework is always application-independent that is it can be used with any application
irrespective of the complications (like Technology stack, architecture etc.) of the application
under test. The framework should be scalable and maintainable.

Advantage of Test Automation framework

1. Reusability of code
2. Maximum coverage
3. Recovery scenario
4. Low-cost maintenance
5. Minimal manual intervention
6. Easy Reporting

Advantages of using Test Automation Frameworks

The advantages of having test automation frameworks in check are as follows –

 Reusability of Code

Since these frameworks come with the required coding data beneficial for the
automation test to become a success, the valuable data is saved for future use and
can be reused at any given point of time. There is no need to insert codes manually
or rearrange them. The code can also be utilized to run other automation framework
tests. The developed scripts can also be retained.

 Low Cost

Test cases can be developed at a fairly low cost because the frameworks already
have established rules. Also, given these codes can be reused multiple times, the
cost and time of building test cases for new features are low.

 Minimal Manual Interference

The automation frameworks run according to the guidelines that surround them.
Since maximum coverage is already in-built and achieved at the initial stage, there is
very little or no intervention required by individuals to run the automation tests. In
case the process fails, the automation frameworks can be run again with a little
change, but the data remains constant and devoid of additional effort from an
individual or team.

 Improved Efficiency

Test automation frameworks increase productivity due to standardization. It

guarantees maximum test coverage as the set of codes in a framework are executed
in a standard manner from the beginning.
Types of Test Automation Framework

Data Driven Testing Framework - While automating or testing any application, at times it may be
required to test the same functionality multiple times with the different set of input data. Thus, in
such cases, we can’t let the test data embedded in the test script. Hence it is advised to retain test
data into some external database outside the test scripts.

Data Driven Testing Framework helps the user segregate the test script logic and the test data
from each other. It lets the user store the test data into an external database. The external
databases can be property files, xml files, excel files, text files, CSV files, ODBC repositories etc.
The data is conventionally stored in “Key-Value” pairs. Thus, the key can be used to access and
populate the data within the test scripts.

Keyword Driven Testing Framework

The Keyword driven testing framework is an extension to Data driven Testing Framework in a
sense that it not only segregates the test data from the scripts, it also keeps the certain set of code
belonging to the test script into an external data file.

These set of code are known as Keywords and hence the framework is so named. Keywords are
self-guiding as to what actions need to be performed on the application.

The keywords and the test data are stored in a tabular like structure and thus it is also popularly
regarded as Table driven Framework. Take a notice that keywords and test data are entities
independent of the automation tool being used.
TestNG
TestNG is a testing framework for the Java programming language created by Cédric Beust
and inspired by JUnit and NUnit. The design goal of TestNG is to cover a wider range of test
categories: unit, functional, end-to-end, integration, etc., with more powerful and easy-to-use
functionalities.

TestNG is an open source automated testing framework; where NG means NextGeneration.

TestNG gives the developer the ability to write more flexible and powerful tests.

TestNG Features
 Supports annotations.
 TestNG uses more Java and OO features.
 Supports testing integrated classes (e.g., by default, no need to create a new test class
instance for every test method).
 Separates compile-time test code from run-time configuration/data info.
 Flexible runtime configuration.
 Introduces ‘test groups’. Once you have compiled your tests, you can just ask TestNG to
run all the "front-end" tests, or "fast", "slow", "database" tests, etc.
 Supports Dependent test methods, parallel testing, load testing, and partial failure.
 Flexible plug-in API.
 Support for multi-threaded testing.