Software Quality and Testing

(CSC 4133)

Common Interview Questions

Why are there so many bugs in software?
• Faulty requirements definition
• Client-Developer communication failure
• Software complexity
• Logical design errors
• Programming errors
• Non-compliance with documentation & coding instructions
• Time pressure
• Documentation errors
• Software development tools
• Lack of skilled testers
What is the difference between QA and Testing?

 Software Testing:
• Testing is the process of executing a system or
component under specified conditions with the
intent of finding defects and to verify that it satisfies
specified requirements.
• Testing is one of the most important activities of
quality assurance (QA).
• Testing is a product-oriented activity.
• Testing is oriented to bug-detection.
What is the difference between QA and Testing?
 Software Quality Assurance(SQA):
• Defined as a planned and systematic approach to the
evaluation of the quality of and adherence to software
product standards, processes, and procedures.
• An umbrella activity that is applied throughout the
software process.
• QA is a process-oriented activity.
• QA is oriented to bug-prevention.
 Difference between SQA & Software Testing

SQA Software Testing

• Process-oriented activity. • Product-oriented activity.

• Oriented to bug-prevention. • Oriented to bug-detection.
 What is Quality Control?
• QC is the set of activities designed to evaluate the quality of
developed product.
• QC is a responsibility internal to the team.
• QC is a product-oriented activity.
• The term QC is typically used in a production/hardware
manufacturing environment, where a large number of physical
items are produced and shipped. Each of the items has to go
through a testing process to ensure that the quality of the product
is good enough for shipment; otherwise, the item is rejected. The
quality check is conducted by the quality control group within the
manufacturing organization, and the person who conducts the
testing is called a quality controller.
 What is difference between
Verification and Validation (V&V)?
 Verification vs. Validation
 Verification:
– Verification is a process that ensures the software product
works properly.
– Software systems must perform the specific tasks
correctly; they must do the things right.
– Building the product correctly
– Verification typically involves reviews and meetings to
evaluate documents, plans, code, requirements, and
specifications. This can be done with checklists, issues lists,
walkthroughs, and inspection meetings.
– It is a low-level activity.
 Verification vs. Validation
 Validation:
– Validation is a process that ensures the software
product meets the customer requirements.
– Software systems must do what they are supposed
to do; they must do the right things.
– Building the correct product
– Validation typically involves actual testing and
takes place after verifications are completed.
– It is a high-level activity.
 Testing Levels/Phases/Stages

1. Unit Testing ==>

2. Integration Testing ==>

3. System Testing ==>

4. Acceptance Testing ==>

 Testing Levels
 Unit Testing:
• Testing of individual software unit/component
• First level of dynamic testing
• Typically white-box testing
• Usually done by programmers
• AKA: Component testing, module testing
• METHOD/TECHNIQUE
– Unit Testing is usually performed by using the White Box Testing method
• When is it performed?
– Unit Testing is the first level of testing and is performed prior to Integration Testing
• Who performs it?
– Unit Testing is normally performed by software developers themselves or their peers. In
rare cases it may also be performed by independent software testers
 Testing Levels
 Integration Testing:
• Testing of two or more units/modules together to identify interface
defects
• Objective is to detect Interface defects between units/modules
• Gray-box testing
 When is Integration Testing performed?
– Integration Testing is performed after Unit Testing and before System
Testing
 Who performs Integration Testing?
– Either Developers themselves or independent Testers perform
Integration Testing
 Testing Levels
 System Testing:
• Formal testing which is conducted on complete, integrated
system to verify that it meets specified requirements
• The purpose of this test is to evaluate the system’s
compliance with the specified requirements
• Black-box testing
* Done before Performance testing
• When is it performed?
– System Testing is performed after Integration Testing and before Acceptance Testing
• Who performs it?
– Normally, independent Testers perform System Testing
 Testing Levels
 Acceptance Testing:
• Formal testing conducted to determine whether a system
satisfies its acceptance criteria
• The purpose of this test is to evaluate the system’s
compliance with the business requirements and assess
whether it is acceptable for delivery
• Verifies functionality and usability of the software
• Black-box testing
• Performed by customers/end users (preferably)
• Prior to software being released to live operation
 Testing Techniques
• Two basic types of testing techniques:
1) Black-box testing
2) White-box testing

Q. What is the difference between BBT & WBT?

 BBT vs. WBT
Black box testing:
• View components as opaque
• Based on requirements and functionality
• Without any knowledge of internal design, code or
language.
• Can be either functional or non-functional, without
reference to the internal structure of the component
or system.
• AKA : Functional testing, behavioral testing
 BBT vs. WBT
White box testing:
• View components as transparent
• Based on knowledge of the internal logic
• Testing based on an analysis of the internal structure
of the component or system
• Done by programmers (usually)
• AKA: Structural testing, Glass-box testing, clear-box
testing
 Manual Testing vs. Automated Testing

Manual Testing:
• Oldest and most rigorous type of software testing
• Requires a tester to perform manual test operations
• Drawbacks –
– Hard to repeat
– Not always reliable
– Costly
• time consuming
• labor intensive
 Manual Testing vs. Automated Testing

Automated Testing:
• Testing employing software tools
• Execute tests without manual intervention
• Benefits –
– Fast
– Repeatable
– Reliable
– Reusable
– Programmable
– Saves time
 Alpha Testing vs. Beta Testing
Alpha testing:
• Alpha testing is performed by potential users/customers or an
independent test team at the developer’s site.
• Conducted when code is mostly complete or contains most of
the functionality and prior to users being involved.
• Minor design changes may still be made as a result of such
testing. Sometimes a select group of users are involved.
 Alpha Testing vs. Beta Testing
Beta testing:
• Beta testing is done at the customer’s site by the customer in
the open environment.
• Testing when development and testing are essentially completed and final
bugs and problems need to be found before final release.
• Typically done by end-users or others, not by programmers. Betas are
often widely distributed or even distributed to the public at large in hopes
that they will buy the final product when it is released.
• Selected users receive the system first and report problems back to the
developer.
• Beta testing is a type of acceptance testing involving a software product to
be marketed for use by many users.
 What is Regression Testing?
• Regression testing is the re-testing of a system or component
of a system to verify that modifications have not introduced
any new defects and that the system/component still
complies with its specified requirements.
• Regression Testing is done to find out the defects that arise
due to code changes made in existing code like functional
enhancements or configuration changes.
• The main intention behind regression testing is to ensure that
any code changes made for software enhancements or
configuration changes has not introduced any new defects in
the software.
 What is Regression Testing?
• Anytime the changes are made to the existing working
code, a suite of test cases is executed to ensure that the
new changes have not introduced any bugs in the
software.
• It is necessary to have a regression test suite and execute
that suite after every new version of software is
available.
• Regression test suite is the ideal candidate for
automation because it needs to be executed after every
new version.
 What is smoke testing?
• Smoke testing is done for the software in order to verify that the software
is stable enough for further testing. Smoke testing “touches” all areas of
the application without getting too deep; tester looks for answers to basic
questions like, “Does the application window opens?”, “Can tester launch
the software?” etc.
• Smoke Testing is performed after software build to ascertain that the
critical functionalities of the program is working fine. It is executed
"before" any detailed functional or regression tests are executed on the
software build.
• The purpose
– is to determine whether the application is stable enough so that a
more detailed testing can be performed.
– is to reject a badly broken application, so that the QA team does not
waste time installing and testing the software application.
 What is non-functional testing?
• Non-functional testing tests the characteristics of the software
like how fast the response is, or what time does the software
takes to perform any operation.
• Non-functional testing focuses on the software’s performance
i.e. How well it works.
• Some examples of Non-Functional Testing are:
– Performance testing
– Load testing
– Stress testing
– Volume testing
– Usability testing
– Security testing
 What is a Bug?

• A bug is a defect/fault in a system which causes the

system to perform in an unintended or unanticipated
manner.
 Distinguish between s/w error, s/w fault
and s/w failure
• Software errors are sections of the code that are partially or
totally incorrect as a result of a grammatical, logical or other
mistake made by a systems analyst, a programmer, or another
member of the software development team.
• Software faults are software errors that cause the incorrect
functioning of the software during a specific application.
• Software faults become software failures when they are
“activated”, that is, when a user tries to apply the specific
software section that is faulty. Thus, the root of any software
failure is a software error.
 Distinguish between error, fault and failure
• Error: incorrect/missing human action
• Fault: internal characteristics  cause for failures
• Failure: external behavior  deviation from expected behavior
• Defect: error, fault, failure collectively known as defect/bug.
• Bug: Software problems/defects, never precisely defined
• Relations: Errors ==> Faults ==> Failures (not necessarily 1-1)
 Bug Life Cycle
• What is a Bug’s Life Cycle? Explain.
• You have to discuss the Fix-Verification cycle
(between developer and tester) with variations of
the status of the bug.
• In general, a bug is first detected by the tester and
ultimately closed by the tester; in between the
developer debugs and fixes the bug.
 Software Life Cycle
 What is the 'software life cycle'?
• A ‘software life cycle’ begins when a software is first
conceived and ends when it is no longer in use.
– It includes aspects such as initial concept,
requirements analysis, functional design, internal
design, documentation planning, test planning,
coding, document preparation, integration,
testing, maintenance, updates, retesting, phase-
out.
 Bug Status: Severity vs. Priority
• What is the difference between Severity of a bug & Priority of a
bug?
• Priority of a bug is how soon/important it is for a bug to be fixed.
– A priority level is a measure of how soon the defect needs to be
fixed, i.e., urgency.
– Priority is the order in which developer has to fix the bug.
– Defect priority levels: Critical (1), High (2), Medium (3), Low(4)
• Severity of a bug is a measure of how bad(impact).
– A severity level is a measure of the extent of the detrimental
effect of the defect on the operation of the product.
– Severity is how severly the bug is impacting the application.
– Defect severity levels: Critical (1), High (2), Medium (3), Low (4)
 Severity vs. Priority
• “A bug with high severity must receive a high priority
in the resolution phase.”
–Why or why not? Explain.
 What is a Test Plan?
• A test plan is a document that describes the objectives,
scope, approach, resources, schedule and focus of software
testing activities.
• A test plan gives detailed testing information regarding an
upcoming testing effort. In other words, a test plan is a
systematic approach to testing a system and typically contains
a detailed understanding of what the eventual workflow will
be.
• Organizations may follow standard test plan guidelines (e.g.
IEEE 829) or they can have their own customized test plan
outlines.
 Test Plan Format
• Describe a test plan format/template, or
• What are items of a test plan? , or
• Tell me what are contained in a test plan..

 See Project
 What is a Test Case?
• A test case is a document that describes an input,
action, or event, and its expected results, in order to
determine if a feature of an application is working
correctly.
• In other words, a test case is document specifying
inputs, predicted results and a set of execution
conditions for a test item.
• Different organizations may use different test case
format.
 Test Case Format
• Explain a typical test case format. // or
• What are the contents of a test case?

See Assignment
 Test Suite
• The collection of individual test cases that will be
run in a test sequence until some stopping criteria
are satisfied is called a test suite.
– Group of test cases that can be executed as a package in a
particular sequence.
– Test suites are usually related by the area of the system
that they exercise, by their priority, or by content.
 Test Log

• A chronological record of all relevant details about

the execution of a test.
 Test Bed
• An execution environment configured for testing.
• AKA : Test environment.
• A test bed may consists of specific hardware, OS,
network topology, configuration of AUT, other
application or system software.
 Test Script
• A test script in software testing is a set of instructions that will be
performed on the system under test to test that the system functions as
expected.
• There are various means for executing test scripts.
• Manual testing: These are more commonly called test cases.
• Automated testing:
– Short program written in a programming language used to test part of
the functionality of a software system. Test scripts written as a short
program can either be written using a special automated functional
GUI test tool (such as HP QuickTest Professional, Rational Robot) or in
a well-known programming language (such as C++, C#, Java, PHP, Perl,
Powershell, Python, or Ruby).
 Test Deliverables
 What are the typical Test Deliverables?
– Test plan
– Test case
– Test scripts
– Test matrix
– Execution Log
– Defect log
– Test Summary Report
 What is a successful Test Plan?
• A successful test plan is one that verifies that all
functions specified in the requirements are included
in the final system.
• Such a test plan provides 100% test coverage and is
accomplished with minimum cost to resources and
schedule.
 How did you develop/write Test Plan?
• I developed/wrote the test plan and test cases based
on the requirements and functionalities of the
system.
• Test plan is developed based on the requirements
and functionality of the system.
– FRD
– SRS
– Technical specifications
– User manuals (if there is any)
– Use cases
 Inspection vs. Walkthrough
 Walkthrough: A 'walkthrough' is an informal meeting for evaluation or
informational purposes. Little or no preparation is usually required.
– Optional pre-meeting
– Lead by Author
– no Moderator or Scribe required
– Initiated on short time notice
 Inspection : An inspection is more formalized (meeting) than a
'walkthrough', typically with 3-8 people including a moderator, reader, and
a recorder to take notes.
– Formal process based on rules
– Moderator leads/runs the meeting, also need scribe/recorder
– Main Purpose: Finding defects using a structured method
 What is a good quality software?
• A good quality software is one that meets
customers’ requirements, contains minimum
number of defects, and is developed within
time and budget.
 What are the Software Quality Characteristics?

• The ISO 9126 software quality model identifies six

main quality characteristics, namely:
1) Functionality
2) Reliability
3) Usability
4) Efficiency
5) Maintainability
6) Portability