Software Testing

Lecture #1
The Basics of Software Testing

What is Software Testing?

 Software Testing is a method to check whether the actual software product matches expected
requirements and to ensure that software product is Defect free.
 It involves execution of software/system components using manual or automated tools to
evaluate one or more properties of interest.
 The purpose of software testing is to identify errors, gaps or missing requirements in contrast
to actual requirements.

Why Software Testing is Important?

 Software Testing is Important because if there are any bugs or errors in the software, it can
be identified early and can be solved before delivery of the software product.
 Properly tested software product ensures reliability, security and high performance which
further results in time saving, cost effectiveness and customer satisfaction.

What is the need of Testing?

 Testing is important because software bugs could be expensive or even dangerous.
 Software bugs can potentially cause monetary and human loss, and history is full of such
examples.
 In April 2015, Bloomberg terminal in London crashed due to software glitch affected more
than 300,000 traders on financial markets. It forced the government to postpone a 3bn pound
debt sale.
 Nissan cars recalled over 1 million cars from the market due to software failure in the airbag
sensory detectors. There has been reported two accidents due to this software failure.
 Starbucks was forced to close about 60 percent of stores in the U.S and Canada due to
software failure in its POS system. At one point, the store served coffee for free as they were
unable to process the transaction.
 Some of Amazon’s third-party retailers saw their product price is reduced to 1p due to a
software glitch. They were left with heavy losses.
 Vulnerability in Windows 10. This bug enables users to escape from security sandboxes
through a flaw in the win32k system.
 In 2015 fighter plane F-35 fell victim to a software bug, making it unable to detect targets
correctly.
 China Airlines Airbus A300 crashed due to a software bug on April 26, 1994, killing 264
innocents live
 In 1985, Canada’s Therac-25 radiation therapy machine malfunctioned due to software bug
and delivered lethal radiation doses to patients, leaving 3 people dead and critically injuring 3
others.
 In April of 1999, a software bug caused the failure of a $1.2 billion military satellite launch,
the costliest accident in history
 In May of 1996, a software bug caused the bank accounts of 823 customers of a major U.S.
bank to be credited with 920 million US dollars.

What are the benefits/Objectives of Software Testing?

 Finding Defects:  which may get created by the programmer while developing the software.
 Cost-Effective: It is one of the important objective of software testing. Testing any IT project
on time helps you to save your money for the long term. In case if the bugs caught in the
earlier stage of software testing, it costs less to fix.
 Security: It is the most vulnerable and sensitive benefit of software testing. People are
looking for trusted products. It helps in removing risks and problems earlier.
 Product quality: It is an essential requirement of any software product. Testing ensures a
quality product is delivered to customers.
 Customer Satisfaction: The main aim of any product is to give satisfaction to their
customers. UI/UX Testing ensures the best user experience.

7 principles of software testing

There are seven foundational principles for software testing that reinforce its importance in the
development process. 
 Testing shows the presence of defects, not their absence — It’s important to note
that not finding defects does not make your software bug-free. Because no software is
without its defects, testers iterate on their tests to find as many defects as possible.
 Exhaustive testing is impossible — Testing all possible combinations of inputs and
preconditions is not typically possible. Rather, teams and managers should prioritize testing
based on risk analysis to the product and business.
 Early testing saves time and money — Testers don’t need to wait until the software is
deployed to test it. Bugs discovered earlier in a product’s lifecycle are significantly cheaper
and easier to address than if it were a customer-discovered bug. 
 Defects cluster together — Most software issues follow the Pareto Principle–80% of the
issues stem from the same 20% of its modules. While there may be outliers to this, it’s a
helpful rule for focusing testing. 
 Beware the Pesticide Paradox — This borrows from the idea in agriculture that using the
same pesticide over and over again will lead to a decline in its efficacy. In the software
world, this means that the usual test cases will eventually stop finding new defects. Review
and revise tests regularly.
 Testing is context-dependent — A rinse-and-repeat testing model won’t work for all
scenarios. For instance, a high-traffic ecommerce website must have different test cases than
an inventory app used by warehouse staff.
 Absence of errors is a fallacy — Software without known issues doesn’t equal error-free
software. Finding and fixing some defects won’t guarantee the software’s overall success. 

What Is Software Testing Life Cycle (STLC)

 Software Testing Life Cycle refers to a testing process which has specific steps to be
executed in a definite sequence to ensure that the quality goals have been met.
 In the STLC process, each activity is carried out in a planned and systematic way.
 Each phase has different goals and deliverables. Different organizations have different phases
in STLC; however, the basis remains the same.

Phases of software testing process

1. Requirements phase
2. Planning Phase
3. Analysis phase
4. Design Phase
5. Implementation Phase
6. Execution Phase
7. Conclusion Phase
8. Closure Phase

#1. Requirement Phase:

 During this phase of STLC, analyze and study the requirements.
 Have brainstorming sessions with other teams and try to find out whether the requirements
are testable or not. This phase helps to identify the scope of the testing.
 If any feature is not testable, communicate it during this phase so that the mitigation strategy
can be planned.
#2. Planning Phase:
 In practical scenarios, Test planning is the first step of the testing process. In this phase, we
identify the activities and resources which would help to meet the testing objectives. During
 planning we also try to identify the metrics, the method of gathering and tracking those
metrics.
 On what basis the planning is done? Only requirements?
 The answer is NO. Requirements do form one of the bases but there are 2 other very
important factors which influence test planning. These are:
– Test strategy of the organization.
– Risk analysis / Risk Management and mitigation.

#3. Analysis Phase:

 This STLC phase defines “WHAT” to be tested.
 We basically identify the test conditions through the requirements document, product risks,
and other test bases.
 The test condition should be traceable back to the requirement.

There are various factors which affect the identification of test conditions:
Levels and depth of testing
– The complexity of the product
– Product and project risks
– Software development life cycle involved.
– Test management
– Skills and knowledge of the team.
– Availability of the stakeholders.

#4. Design Phase:

 This phase defines “HOW” to test. This phase involves the following tasks:
–Detail the test condition. Break down the test conditions into multiple sub-conditions to
increase coverage.
– Identify and get the test data
– Identify and set up the test environment.
– Create the requirement traceability metrics
– Create test coverage metrics.

#5. Implementation Phase:

 The major task in this STLC phase is of creation of the detailed test cases. Prioritize the test
cases also identify which test case will become part of the regression suite. Before finalizing
the test case, It is important to carry out the review to ensure the correctness of the test cases.
Also, don’t forget to take the sign off of the test cases before actual execution starts.
 If your project involves automation, identify the candidate test cases for automation and
proceed for scripting the test cases. Don’t forget to review them!

#6. Execution Phase:

 As the name suggests, this is the Software Testing Life Cycle phase where the actual
execution takes place.
 But before you start your execution, make sure that your entry criterion is met.
 Execute the test cases, log defects in case of any discrepancy.
 Simultaneously fill your traceability metrics to track your progress.
#7.Conclusion Phase:
 This STLC phase concentrates on the exit criteria and reporting. Depending on your project
and stakeholders choice, you can decide on reporting whether you want to send out a daily
report of the weekly report, etc.
 There are different types of reports ( DSR – Daily status report, WSR – Weekly status
reports) which you can send, but the important point is, the content of the report changes and
depends upon whom you are sending your reports.
 If Project managers belong to testing background then they are more interested in the
technical aspect of the project, so include the technical things in your report ( number of test
cases passed, failed, defects raised, severity 1 defects, etc.).
 But if you are reporting to upper stakeholders, they might not be interested in the technical
things so report them about the risks that have been mitigated through the testing.

#8. Closure Phase:

 Tasks for the closure activities include the following:
– Check for the completion of the test. Whether all the test cases are executed or
mitigated deliberately. Check there is no severity 1 defects opened.
– Do lessons learned meeting and create lessons learned document. (Include what
went well, where are the scope of improvements and what can be improved)

Lecture # 2
The Basics of Software Testing

V-Model
 V-Model also referred to as the Verification and Validation Model.
 In this, each phase of SDLC must complete before the next phase starts.
 It follows a sequential design process same as the waterfall model.
 Testing of the device is planned in parallel with a corresponding stage of development.
Verification: It involves a static analysis method (review) done without executing code. It is the
process of evaluation of the product development process to find whether specified requirements
meet.
Validation: It involves dynamic analysis method (functional, non-functional), testing is done by
executing code. Validation is the process to classify the software after the completion of the
development process to determine whether the software meets the customer expectations and
requirements.

 So V-Model contains Verification phases on one side of the Validation

 phases on the other side.
 Verification and Validation process is joined by coding phase in V-shape.
 Thus it is known as V-Model.

Phases of V-Model
1. Business requirement analysis: This is the first step where product requirements understood
from the customer's side. This phase contains detailed communication to understand
customer's expectations and exact requirements.
2. System Design: In this stage system engineers analyze and interpret the business of the
proposed system by studying the user requirements document.
3. Architecture Design: The baseline in selecting the architecture is that it should understand
all which typically consists of the list of modules, brief functionality of each module, their
interface relationships, dependencies, database tables, architecture diagrams, technology
detail, etc. The integration testing model is carried out in a particular phase.
4. Module Design: In the module design phase, the system breaks down into small modules.
The detailed design of the modules is specified, which is known as Low-Level Design
5. Coding Phase: After designing, the coding phase is started. Based on the requirements, a
suitable programming language is decided. There are some guidelines and standards for
coding. Before checking in the repository, the final build is optimized for better performance,
and the code goes through many code reviews to check the performance.
 Validation Phase of V-model:
1. Unit Testing: In the V-Model, Unit Test Plans (UTPs) are developed during the module
design phase. These UTPs are executed to eliminate errors at code level or unit level. A unit
is the smallest entity which can independently exist, e.g., a program module. Unit testing
verifies that the smallest entity can function correctly when isolated from the rest of the
codes/ units.
2. Integration Testing: Integration Test Plans are developed during the Architectural Design
Phase. These tests verify that groups created and tested independently can coexist and
communicate among themselves.
3. System Testing: System Tests Plans are developed during System Design Phase. Unlike Unit
and Integration Test Plans, System Tests Plans are composed by the client?s business team.
System Test ensures that expectations from an application developer are met.
4. Acceptance Testing: Acceptance testing is related to the business requirement analysis part.
It includes testing the software product in user atmosphere. Acceptance tests reveal the
compatibility problems with the different systems, which is available within the user
atmosphere. It conjointly discovers the non-functional problems like load and performance
defects within the real user atmosphere.

When to use V-Model?

 When the requirement is well defined and not ambiguous.
 The V-shaped model should be used for small to medium-sized projects where requirements
are clearly defined and fixed.
 The V-shaped model should be chosen when sample technical resources are available with
essential technical expertise.

Advantage (Pros) of V-Model:

 Easy to Understand.
 Testing Methods like planning, test designing happens well before coding.
 This saves a lot of time. Hence a higher chance of success over the waterfall model.
 Avoids the downward flow of the defects.
 Works well for small plans where requirements are easily understood.

Disadvantage (Cons) of V-Model:

 Very rigid and least flexible.
 Not a good for a complex project.
 Software is developed during the implementation stage, so no early prototypes of the
software are produced.
 If any changes happen in the midway, then the test documents along with the required
documents, has to be updated.

Lecture #3
Psychology of Software Testing
 Software development, including software testing, involves human beings.
 Therefore, human psychology has important effect on software testing.
 In software testing, psychology plays an extremely important role.
 It is one of those factors that stay behind the scene, but has a great impact on the end result.
 Categorized into three sections, the psychology of testing enables smooth testing.
 It is mainly dependent on the mindset of the developers and testers, as well as the quality of
communication between them.
 Moreover, the psychology of testing improves mutual understanding among team members
and helps them work towards a common goal.

Three sections of the psychology of testing

1. The mindset of Developers and Testers.
2. Communication in a Constructive Manner.
3. Test Independence.

1. The mindset of Developers and Testers

 Developers synthesize code. They build up software by putting units together and figuring
ways of combining those distinct units to do wonderful and amazing things for software
development.
 But Testers are all about analysis. Once it has all been put together, the tester likes to take it
apart again, piece by piece, this time looking for those little corners, edges, and absurdities
that hide in those weird and strange interactions that come from those new and amazing ways
of putting pieces together.
 As an example,
Identifying defects during static testing such as requirements review or user story refinement
session or identifying failures during dynamic test execution, may be perceived
as Criticism of the product and of its author.
 So the developer or the analyst may have problems with you as a tester because he thinks that
you are criticizing them.
 There’s an element of human psychology called Confirmation bias, which means that most
of people find it difficult to accept information that disagrees with currently held believes.
 Since developers expect their code to be correct, they have a confirmation bias that makes it
difficult to accept that the code is incorrect.
2. Communication in a Constructive Manner
 Testers and test managers need to have good interpersonal skills to be able to communicate
effectively about the defects, failures, test results, test progress and risks and to build positive
relationships among colleagues.
 Communication, if done in a polite and respectful manner can help build a strong and reliable
relationship between the team members and help them avoid any misunderstanding.
3. Test Independence
 Giving applications to the testing specialists or professional testers allows an independent
test of the system.
 As this type of testing is mainly performed by individuals, who are not related to the project
directly or are from a different organization, they are hired mainly to test the quality as well
as the effectiveness of the developed product.
 While testing their own code of the programmer, they find many problems so the
programmers, architect, and developers always test their own code before giving it to anyone.
 However, we all know that it is difficult to find our own mistakes. It might be due to some
reasons such as,
o “Parental feelings” towards their code
o Focus on the “Positive Paths”
o Work-based on simplifying complex scenarios
o Inability to catch small things in big pictures
o Lack of end-to-end & real-user perspective
o Less experience with common bugs

Levels of Test Independence

There are several levels of independence in software testing which is listed here from the lowest
level of independence to the highest:
i. Tests by the person who wrote the item.
ii. Tests by another person within the same team, like another programmer.
iii. Tests by the person from some different groups such as an independent test team.
iv. Tests by a person from a different organization or company, such as outsourced testing or
certification by an external body.

Lecture # 4
Code of Ethics of Software Testing

 In software testing there are numerous rules and regulations that ought to be followed by
software engineers to maintain the credibility of the software as well as to make it suitable
according the universal standards.
 Moreover, there are various code of ethics that are required to be followed by the testers to
ensure that the information accessed by them is not put to inappropriate use and is managed
with confidentiality and secrecy.
 In short, following these code of ethics is vital for the testers to avoid future conflicts and
information breaches. So, to help you understand the various code of ethics followed by
software testers during the process of testing, here is a detailed discussion about these code of
ethics.
 As defined by Merriam Webster dictionary, Code of ethics is “a set of rules about good and
bad behavior” or “a systematic statement of law” that ought to be followed and considered by
individuals performing important tasks or by those dealing with confidential information.
 Likewise, in software testing also, code of ethics specifies various rules and regulations of
conduct that the members of the team must adhere to.
 It sets forth the values, principles, and standards that guide the testers to perform their tasks
appropriately and helps them use the information they have in an ethical and appropriate
manner.
 Code of ethics, in short defines the acceptance of responsibility by the software testers, while
keeping the best interest of their clients as priority.
Types of Code of Ethics in Testing:
 Public: During the process of software development and testing, the public interest and
benefit should be considered before corporate and personal gain. They should consistently act
in the interest of the public.
 Client & Employer: Apart from public interest, the testers should consider the interest of
their client and employer while conducting tests on the software. They should act according
to the requirements of their clients and should fulfill all their needs.
 Product: The most important factor that needs consideration during the process of software
testing is the product itself. The testers should perform their tasks while ensuring that the
deliverable is of best quality and has exceptional effectiveness. Also, make sure that all
deliverables on the product are in compliance with the stated standards.
 Judgement: The testes should maintain integrity and independence while making judgments
regarding the process of testing or any other aspects related to it.
 Management: Here, the team managers and leads should take the responsibility and ethical
steps to manage the process of software testing, development, and maintenance. This will
help them avoid any confusion as well as allow them to test each component of the software
accurately.
 Profession: The team of testers should follow the set f values, principles, and standards and
advance the integrity of their profession.

Why is Code of Ethics Important?

1. It protects the interests of consumers and offers them assurance that they are not being
misguided or misled by false promises and advertisements.
2. Helps firms and organizations in obeying the law and treating people honestly and fairly.
3. Clarifies organization’s missions, values, and principles.
4. Promotes employment ethics, such as security, promotions, health and safe working
condition, etc.
5. Allows one to create a professional environment wherein ethical behavior is a norm.
6. Serves as a guide or reminder of how to perform tasks as well as the way one should act in a
particular situation.
7. Can indicate that individuals are seriously concerned about responsible and professional
conduct.
8. It is a central guide and reference for employers.
9. Prevents unjust treatment.
10. Brings out best in people as well as high standards in organizations.

Lecture # 5
Types of Software Testing
What is Software Testing?
Software testing is a process, to evaluate the functionality of a software application with an
intent to find whether the developed software met the specified requirements or not and to
identify the defects to ensure that the product is defect-free in order to produce the quality
product.

Verification vs. Validation

Verification:
"Are we building the product right"
The software should conform to its specification

Validation:
"Are we building the right product"
The software should do what the user requires correctly.

Types of Testing

1. Unit Testing
 It focuses on smallest unit of software design. In this we test an individual unit or group of
inter related units. It is often done by programmer by using sample input and observing its
corresponding outputs.
Example:
a) In a program we are checking if loop, method or function is working fine.
b) Misunderstood or incorrect, arithmetic precedence.
c) Incorrect initialization

2. Integration Testing
 The objective is to take unit tested components and build a program structure that has been
dictated by design. Integration testing is testing in which a group of components are
combined to produce output.
Example: (i) Top down (ii) Bottom up (iii) Sandwich (iv) Big-Bang

3. Regression Testing
 Every time new module is added leads to changes in program. This type of testing make sure
that whole component works properly even after adding components to the complete
program.
 Example
 In school record suppose we have module staff, students and finance combining these
modules and checking if on integration these module works fine is regression testing

4. Smoke Testing
 This test is done to make sure that software under testing is ready or stable for further testing
It is called smoke test as testing initial pass is done to check if it did not catch the fire or
smoked in the initial switch on.
 Example:
If project has 2 modules so before going to module make sure that module 1 works properly
5. Alpha Testing
 This is a type of validation testing. It is a type of acceptance testing which is done before the
product is released to customers. It is typically done by QA people.
 Example:
When software testing is performed internally within the organization.

6. Beta Testing
 The beta test is conducted at one or more customer sites by the end-user of the software. This
version is released for the limited number of users for testing in real time environment
Example:
When software testing is performed for the limited number of people

7. System Testing
 In this software is tested such that it works fine for different operating system.It is covered
under the black box testing technique. In this we just focus on required input and output
without focusing on internal working.
In this we have security testing, recovery testing , stress testing and performance testing.
 Example: This include functional as well as non-functional testing

8. Stress Testing
 In this we gives unfavorable conditions to the system and check how they perform in those
condition.
Example:
(a) Test cases that require maximum memory or other resources are executed (b) Test cases that
may cause thrashing in a virtual operating system (c) Test cases that may cause excessive disk
requirement

9. Performance Testing
 It is designed to test the run-time performance of software within the context of an integrated
system. It is used to test speed and effectiveness of program.
Example:
Checking number of processor cycles.

Testing priorities
 Only exhaustive testing can show a program is
free from defects. However, exhaustive testing
is impossible
 Tests should exercise a system's capabilities
rather than its components
 Testing old capabilities is more important than
testing new capabilities
 Testing typical situations is more important

Test data and test cases

 Test data Inputs which have been devised to
test the system
 Test cases Inputs to test the system and the
predicted outputs from these inputs if the
system operates according to its specification

Design of test cases

 Exhaustive testing of almost any non-trivial system is impractical due to the fact that the
domain of input data values to most practical software systems is either extremely large or
infinite.
 Therefore, we must design an optional test suite that is of reasonable size and can uncover as
many errors existing in the system as possible.

 Consider the following example code segment which finds the greater of two integer values x
and y. This code segment has a simple programming error.

If (x>y) max = x;
else max = x;

 For the above code segment, the test suite, {(x=3,y=2);(x=2,y=3)} can detect the error.

 whereas a larger test suite {(x=3,y=2);(x=4,y=3);(x=5,y=1)} does not detect the error.

 This implies that the test suite should be carefully designed than picked randomly. Therefore,
systematic approaches should be followed to design an optimal test suite. In an optimal test
suite, each test case is designed to detect different errors.

Lecture # 6
Plan of Software Testing
What is Software Test Plan?
 A test plan is a document that sets out the scope, approach, and schedule of intended testing
activities.
 The test plan may also list the resources the software tester needs to function effectively. 
3 Types of Test Plan
 1. Master Test Plan
2. Phase Test Plan
3. Testing Type Specific Test Plans

Master Test Plan

Master Test Plan is a type of test plan that has multiple levels of testing. It includes a complete
test strategy.

Phase Test Plan

 Phase test plan is a type of test plan that addresses any one phase of the testing strategy. For
example, a list of tools, a list of test cases, etc.

Specific Test Plans

Specific test plan designed for major types of testing like security testing, load testing,
performance testing, etc.
In other words, a specific test plan designed for non-functional testing.

How to write a Test Plan

According to IEEE 829, follow the following seven steps to prepare a test plan.
1. First, analyze product structure and architecture.
2. Now design the test strategy.
3. Define all the test objectives.
4. Define the testing area.
5. Define all the useable resources.
6. Schedule all activities in an appropriate manner.
7. Determine all the Test Deliverables.

Test plan components or attributes

Objectives: It consists of information about modules, features, test data etc., which indicate the
aim of the application means the application behavior, goal, etc.
Scope: It contains information that needs to be tested with respective of an application. The
Scope can be further divided into two parts:
  In scope
 Out scope

In scope: These are the modules that need to be tested rigorously (in-detail).

Out scope: These are the modules, which need not be tested rigorously.
For example, Suppose we have a Gmail application to test, where features to be tested such
as Compose mail, Sent Items, Inbox, Drafts and the features which not be tested such as Help,
and so on which means that in the planning stage, we will decide that which functionality has to
be checked or not based on the time limit given in the product.

Test methodology:
 It contains information about performing a different kind of testing like Functional testing,
Integration testing, and System testing, etc. on the application. In this, we will decide what
type of testing; we will perform on the various features based on the application requirement.
And here, we should also define that what kind of testing we will use in the testing
methodologies so that everyone, like the management, the development team, and the testing
team can understand easily because the testing terms are not standard.
 For example, for standalone application such as Adobe Photoshop, we will perform the
following types of testing:

Different Test Methodologies

moke testing Functional testing Integration testing

System testing Adhoc testing Compatibility testing

Regression testing Globalization testing Accessibility testing

Usability testing Performance testing

Approach
 This attribute is used to describe the flow of the application while performing testing and for
the future reference.
 We can understand the flow of the application with the help of below aspects:
o By writing the high-level scenarios
o By writing the flow graph

By writing the high-level scenarios

 For example, suppose we are testing the Gmail application:
 Login to Gmail- sends an email and check whether it is in the Sent Items page
 Login to …….
 ……
 …....

By writing the flow graph

The flow graph is written because writing the high-level scenarios are bit time taking process, as
we can see in the below image:

Assumption
It contains information about a problem or issue which maybe occurred during the testing
process and when we are writing the test plans, the assured assumptions would be made like
resources and technologies, etc.

Risk
These are the challenges which we need to face to test the application in the current release and if
the assumptions will fail then the risks are involved.
For example, the effect for an application, release date becomes postponed.

Mitigation Plan or Contingency Plan

It is a back-up plan which is prepared to overcome the risks or issues.
Let us see one example for assumption, risk, and the contingency plan together because they are
co-related to each other.

Role & Responsibility

It defines the complete task which needs to be performed by the entire testing team.
When a large project comes, then the Test Manager is a person who writes the test plan. If there
are 3-4 small projects, then the test manager will assign each project to each Test Lead. And then,
the test lead writes the test plan for the project, which he/she is assigned.
Schedule
It is used to explain the timing to work, which needs to be done or this attribute covers when
exactly each testing activity should start and end? And the exact data is also mentioned for every
testing activity for the particular date.

Defect tracking
It is generally done with the help of tools because we cannot track the status of each bug
manually. And we also comment about how we communicate the bugs which are identified
during the testing process and send it back to the development team and how the development
team will reply. Here we also mention the priority of the bugs such as high, medium, and low.

Test Environments
These are the environments where we will test the application, and here we have two types of
environments, which are of software and hardware configuration.
The software configuration means the details about different Operating Systems such
as Windows, Linux, UNIX, and Mac and various Browsers like Google Chrome, Firefox, Opera,
Internet Explorer, and so on.
And the hardware configuration means the information about different sizes of RAM, ROM, and
the Processors.

Entry and Exit criteria

It is a necessary condition, which needs to be satisfied before starting and stopping the testing
process.

 Entry Criteria
o The entry criteria contain the following conditions:
o White box testing should be finished.
o Understand and analyze the requirement and prepare the test documents or when the test
documents are ready.
o Test data should be ready.
o Build or the application must be prepared
o Modules or features need to be assigned to the different test engineers.
o The necessary resource must be ready.
 Exit Criteria
o The exit criteria contain the following conditions:
o When all the test cases are executed.
o Most of the test cases must be passed.
o Depends on severity of the bugs which means that there must not be any blocker or major
bug, whereas some minor bugs exist.

Test Automation
 In this, we will decide the following:
 Which feature has to be automated and not to be automated?
 Which test automation tool we are going to use on which automation framework?
 We automate the test case only after the first release.
 Here the question arises that on what basis we will decide which features have to be
tested?
Effort estimation
In this, we will plan the effort need to be applied by every team member.

Test Deliverable
These are the documents which are the output from the testing team, which we handed over to
the customer along with the product. It includes the following:
 Test plan
 Test Cases
 Test Scripts
 RTM(Requirement Traceability Matrix)
 Defect Report
 Test Execution Report
 Graphs and metrics
 Release Notes

Template
This part contains all the templates for the documents that will be used in the product, and all the
test engineers will use only these templates in the project to maintain the consistency of the
product. Here, we have different types of the template which are used during the entire testing
process such as:
 Test case template
 Test case review template
 RTM (Requirements Traceability Matrix) Template
 Bug Report Template
 Test execution Report
 Lecture #7
Myths of Software Testing

What is Software Test Myths?

 There are many myths and false assumptions that continue to loom around System Testing.
There are also lots of myths out there that prevent fully testing the software application that
which can lead to producing an insecure and less productive software application. Software
testing not only provides security to the system or software application but also concentrates
on fulfilling the requirements of the end-users. 
 Following are the myths that need to be challenged for the purpose of good software quality: 

Myth 1: Testing is Too Expensive

Reality − There is a saying, pay less for testing during software development or pay more for
maintenance or correction later. Early testing saves both time and cost in many aspects, however
reducing the cost without testing may result in improper design of a software application
rendering the product useless.
Myth 2: Testing is Time-Consuming
Reality − During the SDLC phases, testing is never a time-consuming process. However
diagnosing and fixing the errors identified during proper testing is a time-consuming but
productive activity.

Myth 3: Only Fully Developed Products Are Tested

Reality − No doubt, testing depends on the source code but reviewing requirements and
developing test cases is independent from the developed code. However iterative or incremental
approach as a development life cycle model may reduce the dependency of testing on the fully
developed software.
Myth 4: Complete Testing is Possible
Reality − It becomes an issue when a client or tester thinks that complete testing is possible. It is
possible that all paths have been tested by the team but occurrence of complete testing is never
possible. There might be some scenarios that are never executed by the test team or the client
during the software development life cycle and may be executed once the project has been
deployed.

Myth 5: A Tested Software is Bug-Free

Reality − This is a very common myth that the clients, project managers, and the management
team believes in. No one can claim with absolute certainty that a software application is 100%
bug-free even if a tester with superb testing skills has tested the application.

Myth 6: Missed Defects are due to Testers

Reality − It is not a correct approach to blame testers for bugs that remain in the application even
after testing has been performed. This myth relates to Time, Cost, and Requirements changing
Constraints. However the test strategy may also result in bugs being missed by the testing team.

Myth 7: Testers are Responsible for Quality of Product

Reality − It is a very common misinterpretation that only testers or the testing team should be
responsible for product quality. Testers’ responsibilities include the identification of bugs to the
stakeholders and then it is their decision whether they will fix the bug or release the software.
Releasing the software at the time puts more pressure on the testers, as they will be blamed for
any error.
Myth 8: Test Automation should be used wherever possible to Reduce Time
Reality − Yes, it is true that Test Automation reduces the testing time, but it is not possible to
start test automation at any time during software development. Test automaton should be started
when the software has been manually tested and is stable to some extent. Moreover, test
automation can never be used if requirements keep changing.

Myth 9: Anyone can Test a Software Application

Reality − People outside the IT industry think and even believe that anyone can test a software
and testing is not a creative job. However testers know very well that this is a myth. Thinking
alternative scenarios, try to crash a software with the intent to explore potential bugs is not
possible for the person who developed it.

Myth 10: A Tester's only Task is to Find Bugs

Reality − Finding bugs in a software is the task of the testers, but at the same time, they are
domain experts of the particular software. Developers are only responsible for the specific
component or area that is assigned to them but testers understand the overall workings of the
software, what the dependencies are, and the impacts of one module on another module.

Lecture # 8
Monitoring and Controlling of Software Testing
What is Test Monitoring?
Test Monitoring in test execution is a process in which the testing activities and testing efforts
are evaluated in order to track current progress of testing activity, finding and tracking test
metrics, estimating the future actions based on the test metrics and providing feedback to the
concerned team as well as stakeholders about current testing process.

What is Test Control?

Test Control in test execution is a process of taking actions based on results of the test
monitoring process.
In the test control phase, test activities are prioritized, test schedule is revised, test environment is
reorganized and other changes related to testing activities are made in order to improve the
quality and efficiency of future testing process.

Why do we monitor?
 This small example shows you why we need to monitor and control test activity. After
finishing the Test Estimation and test planning, the management board agreed with your plan
and the milestones are set as per the following figure.

 But after 4 weeks, things are not going as per plan. The task of “Making Test specification”
is delayed by 4 working days. It has a cascading effect and all succeeding tasks are delayed.
 You missed the milestone as well the overall project deadline.
 As a consequence, your project fails and your company loses the customer trust. You must
take full responsibility for the project’s failure.

Why did you miss the deadline?

I forgot to monitor and control the project progress
My team member did not work well
I don’t know why.

What do we monitor?
Monitoring will allow you to make comparisons between your original plan and your progress so
far. You will be able to implement changes, where necessary, to complete the project
successfully.
In your project, as the Test Manager, you should monitor the key parameters as below.

1. Cost
 Costs are an important aspect of project monitoring and control. Having accurate project
estimates and a robust project budget is necessary to deliver project within the decided
budget.
 Suppose, your boss has agreed to fund the project with $100,000. You must keep an eye
on the actual costs while the project is being implemented.
2. Schedules
 How can you work without a schedule? It can be compared to driving your car but
without any idea of how long it takes you get to the destination. The schedule tells you
 When should each activity be done?
 What has already been completed?
 The sequence in which things need to be finished.

3. Resources
 Resources are all things required to carry out the project tasks. They can be people or
equipment required to complete the project activity. Lack of resources can affect the
project progress.
 The truth is, everything may not happen as planned, employees will leave, the project
budget may be cut, or the schedule will get pushed. Monitoring resources will help you to
early detect any resource crunch and find a solution to deal with it.
4.Quality
 Quality monitoring involves monitoring the results of specific work products (like test
case suite, test execution log), to evaluate whether its meets the defined quality standards.
In case results do not meet quality standards, you need to identify potential resolution.
 Example: Suppose that you monitored and controlled the project progress very well.
Finally, you delivered the product at the deadline. The project seems to be successful.
 But after delivering 2 weeks, you got this feedback from customer

What did I do wrong?

I did nothing wrong. Maybe the customer made mistake.
I forgot monitoring the quality of the project output.
I don’t know why.

How to monitor?
As your project comes to life, keep these questions in mind:
Are you on schedule? If not, how far behind are you, and how can you catch up?
Are you over budget?
Are you still working toward the same project goal?
Are you running low on resources?
Are there warning signs of impending problems?
Is there pressure from management to complete the project sooner?

Project Monitoring Process

Step 1) Create Monitoring Plan

In the Monitoring Plan, you must plan carefully about.

 What metrics need to collect and measure?

o The cost (time, money) spent for the project so far
 o Resource (employees, equipment) are used for the project
o The status of the task (on schedule, behind or before the schedule)
o The quality of the work product (Run rate/pass rate, defect metrics)

 When to collect the data?

o weekly or daily basis

 How to evaluate the project’s progress via metrics?

o Compare the progress in plan with the actual progress that the team has made
o Define the criteria which are used to evaluate the project’s progress. For example, if the
effort to complete a task took more than 30% effort than planed a project delay.

Step 2) Update progress record

One of the best methods to track the member progress is holding regular meetings.

Step 3) Analyze record and make the adjustment

Step 3.1) Analyze
 In this step, you compare the progress you defined in plan with the actual progress that
the team has made. By analyzing the record, you can also see how much time has been
spent on individual task and the total time spent on the project overall. In the analysis, we
check:
o Nothing wrong, it’s still good
o The task progress seems to be delayed
o I could not find any issue in that report

Step 3.2) Adjustment

 Make the necessary adjustments keep your project on track. Reassign tasks, modify
schedules, or reassess your goals. This will help you keep moving toward the finish line.
In the adjustment, we will :
o Modify the schedules
o Do nothing
o Modify project goals
o Ask support from other team member to speed up the task
Step 4) Produce the report
Using the report is a good option to share the overall project progress with team members or the
Management Board. It is also a useful way to show your boss whether the project is on track.